Skip to content

Commit 9ea865d

Browse files
committed
corrected some pages
Several small mistakes/errors were corrected in several proofs/definitions.
1 parent 11db948 commit 9ea865d

10 files changed

Lines changed: 96 additions & 94 deletions

D/cov-samp.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -36,7 +36,7 @@ $$
3636
and the unbiased sample covariance of $x$ and $y$ is given by
3737

3838
$$ \label{eq:cov-samp-unb}
39-
s^2_{xy} = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y})
39+
s_{xy} = \frac{1}{n-1} \sum_{i=1}^{n} (x_i - \bar{x}) (y_i - \bar{y})
4040
$$
4141

4242
where $\bar{x}$ and $\bar{y}$ are the [sample means](/D/mean-samp).

D/regline.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ $$
4242
is called a "regression line" and the set
4343

4444
$$ \label{eq:regline-ols}
45-
L(\hat{\beta}_0, \hat{\beta}_1) = \left\lbrace (x,y) \in \mathbb{R}^2 \mid y = \hat{\beta}_0 + \hat{\beta}_1 x \right\rbrace
45+
L(\hat{\beta}_0, \hat{\beta}_1)
4646
$$
4747

4848
is called the "fitted regression line", with estimated regression coefficients $\hat{\beta}_0, \hat{\beta}_1$, e.g. obtained via [ordinary least squares](/P/slr-ols).

I/Table_of_Contents.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -490,7 +490,7 @@ title: "Table of Contents"
490490
&emsp;&ensp; 1.3.7. **[Regression line includes center of mass](/P/slr-comp)** <br>
491491
&emsp;&ensp; 1.3.8. **[Sum of residuals is zero](/P/slr-ressum)** <br>
492492
&emsp;&ensp; 1.3.9. **[Correlation with covariate is zero](/P/slr-rescorr)** <br>
493-
&emsp;&ensp; 1.3.10. **[Residual variance in terms of sample variance](/P/slr-vars)** <br>
493+
&emsp;&ensp; 1.3.10. **[Residual variance in terms of sample variance](/P/slr-resvar)** <br>
494494
&emsp;&ensp; 1.3.11. **[Correlation coefficient in terms of slope estimate](/P/slr-corr)** <br>
495495
&emsp;&ensp; 1.3.12. **[Coefficient of determination in terms of correlation coefficient](/P/slr-rsq)** <br>
496496

P/mlr-ols2.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ $$
4949
**Proof:** The [residual sum of squares](/D/rss) is defined as
5050

5151
$$ \label{eq:RSS}
52-
\mathrm{RSS}(\beta) = \sum_{i=1}^n \varepsilon_i = \varepsilon^\mathrm{T} \varepsilon = (y-X\beta)^\mathrm{T} (y-X\beta)
52+
\mathrm{RSS}(\beta) = \sum_{i=1}^n \varepsilon_i^2 = \varepsilon^\mathrm{T} \varepsilon = (y-X\beta)^\mathrm{T} (y-X\beta)
5353
$$
5454

5555
which can be developed into

P/slr-corr.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,7 @@ $$ \label{eq:slr-ols-sl}
5252
\hat{\beta}_1 = \frac{s_{xy}}{s_x^2} \; .
5353
$$
5454

55-
Using the [relationship between covariance and correlation](/D/cov-corr)
55+
Using the [relationship between covariance and correlation](/P/cov-corr)
5656

5757
$$ \label{eq:cov-corr}
5858
\mathrm{Cov}(X,Y) = \sigma_X \, \mathrm{Corr}(X,Y) \, \sigma_Y

P/slr-olsmean.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,8 @@ $$ \label{eq:sum-ci}
8181
\end{split}
8282
$$
8383

84+
and
85+
8486
$$ \label{eq:sum-ci-xi}
8587
\begin{split}
8688
\sum_{i=1}^n c_i x_i &= \frac{\sum_{i=1}^n (x_i - \bar{x}) x_i}{\sum_{i=1}^n (x_i - \bar{x})^2} = \frac{\sum_{i=1}^n \left( x_i^2 - \bar{x} x_i \right)}{\sum_{i=1}^n (x_i - \bar{x})^2} \\

P/slr-olsvar.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ $$ \label{eq:slr-ols-sl}
9292
\end{split}
9393
$$
9494

95-
and with \eqref{eq:Var-yi} and \eqref{eq:sum-ci2} as well as [invariance](/P/var-inv), [scaling](/P/var-scal) and [additivity](/P/var-add) of the variance, the variance of $\hat{\beta}_1$ becomes:
95+
and with \eqref{eq:Var-yi} and \eqref{eq:sum-ci2} as well as [invariance](/P/var-inv), [scaling](/P/var-scal) and [additivity](/P/var-add) of the variance, the variance of $\hat{\beta}_1$ is:
9696

9797
$$ \label{eq:Var-b1}
9898
\begin{split}
Lines changed: 86 additions & 86 deletions
Original file line numberDiff line numberDiff line change
@@ -1,87 +1,87 @@
1-
---
2-
layout: proof
3-
mathjax: true
4-
5-
author: "Joram Soch"
6-
affiliation: "BCCN Berlin"
7-
e_mail: "joram.soch@bccn-berlin.de"
8-
date: 2021-10-27 14:37:00
9-
10-
title: "Relationship between residual variance and sample variance in simple linear regression"
11-
chapter: "Statistical Models"
12-
section: "Univariate normal data"
13-
topic: "Simple linear regression"
14-
theorem: "Residual variance in terms of sample variance"
15-
16-
sources:
17-
- authors: "Penny, William"
18-
year: 2006
19-
title: "Relation to correlation"
20-
in: "Mathematics for Brain Imaging"
21-
pages: "ch. 1.2.3, p. 18, eq. 1.28"
22-
url: "https://ueapsylabs.co.uk/sites/wpenny/mbi/mbi_course.pdf"
23-
- authors: "Wikipedia"
24-
year: 2021
25-
title: "Simple linear regression"
26-
in: "Wikipedia, the free encyclopedia"
27-
pages: "retrieved on 2021-10-27"
28-
url: "https://en.wikipedia.org/wiki/Simple_linear_regression#Numerical_properties"
29-
30-
proof_id: "P278"
31-
shortcut: "slr-vars"
32-
username: "JoramSoch"
33-
---
34-
35-
36-
**Theorem:** Assume a [simple linear regression model](/D/slr) with independent observations
37-
38-
$$ \label{eq:slr}
39-
y = \beta_0 + \beta_1 x + \varepsilon, \; \varepsilon_i \sim \mathcal{N}(0, \sigma^2), \; i = 1,\ldots,n
40-
$$
41-
42-
and consider estimation using [ordinary least squares](/P/slr-ols). Then, [residual variance](/D/resvar) and [sample variance](/D/var-samp) are related to each other via the [correlation coefficient](/D/corr):
43-
44-
$$ \label{eq:slr-vars}
45-
\hat{\sigma}^2 = \left( 1 - r_{xy}^2 \right) s_y^2 \; .
46-
$$
47-
48-
49-
**Proof:** The [residual variance](/D/resvar) can be expressed in terms of the [residual sum of squares](/D/rss):
50-
51-
$$ \label{eq:slr-res}
52-
\hat{\sigma}^2 = \frac{1}{n-1} \, \mathrm{RSS}(\hat{\beta}_0,\hat{\beta}_1)
53-
$$
54-
55-
and [the residual sum of squares for simple linear regression](/P/slr-sss) is
56-
57-
$$ \label{eq:slr-rss}
58-
\mathrm{RSS}(\hat{\beta}_0,\hat{\beta}_1) = (n-1) \left( s_y^2 - \frac{s_{xy}^2}{s_x^2} \right) \; .
59-
$$
60-
61-
Combining \eqref{eq:slr-res} and \eqref{eq:slr-rss}, we obtain:
62-
63-
$$ \label{eq:slr-vars-s1}
64-
\begin{split}
65-
\hat{\sigma}^2 &= \left( s_y^2 - \frac{s_{xy}^2}{s_x^2} \right) \\
66-
&= \left( 1 - \frac{s_{xy}^2}{s_x^2 s_y^2} \right) s_y^2 \\
67-
&= \left( 1 - \left( \frac{s_{xy}}{s_x \, s_y} \right)^2 \right) s_y^2 \; .
68-
\end{split}
69-
$$
70-
71-
Using the [relationship between correlation, covariance and standard deviation](/D/corr)
72-
73-
$$ \label{eq:corr-cov-std}
74-
\mathrm{Corr}(X,Y) = \frac{\mathrm{Cov}(X,Y)}{\sqrt{\mathrm{Var}(X)} \sqrt{\mathrm{Var}(Y)}}
75-
$$
76-
77-
which also holds for sample correlation, [sample covariance](/D/cov-samp) and sample [standard deviation](/D/std)
78-
79-
$$ \label{eq:corr-cov-std-samp}
80-
r_{xy} = \frac{s_{xy}}{s_x \, s_y} \; ,
81-
$$
82-
83-
we get the final result:
84-
85-
$$ \label{eq:slr-vars-s2}
86-
\hat{\sigma}^2 = \left( 1 - r_{xy}^2 \right) s_y^2 \; .
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2021-10-27 14:37:00
9+
10+
title: "Relationship between residual variance and sample variance in simple linear regression"
11+
chapter: "Statistical Models"
12+
section: "Univariate normal data"
13+
topic: "Simple linear regression"
14+
theorem: "Residual variance in terms of sample variance"
15+
16+
sources:
17+
- authors: "Penny, William"
18+
year: 2006
19+
title: "Relation to correlation"
20+
in: "Mathematics for Brain Imaging"
21+
pages: "ch. 1.2.3, p. 18, eq. 1.28"
22+
url: "https://ueapsylabs.co.uk/sites/wpenny/mbi/mbi_course.pdf"
23+
- authors: "Wikipedia"
24+
year: 2021
25+
title: "Simple linear regression"
26+
in: "Wikipedia, the free encyclopedia"
27+
pages: "retrieved on 2021-10-27"
28+
url: "https://en.wikipedia.org/wiki/Simple_linear_regression#Numerical_properties"
29+
30+
proof_id: "P278"
31+
shortcut: "slr-resvar"
32+
username: "JoramSoch"
33+
---
34+
35+
36+
**Theorem:** Assume a [simple linear regression model](/D/slr) with independent observations
37+
38+
$$ \label{eq:slr}
39+
y = \beta_0 + \beta_1 x + \varepsilon, \; \varepsilon_i \sim \mathcal{N}(0, \sigma^2), \; i = 1,\ldots,n
40+
$$
41+
42+
and consider estimation using [ordinary least squares](/P/slr-ols). Then, [residual variance](/D/resvar) and [sample variance](/D/var-samp) are related to each other via the [correlation coefficient](/D/corr):
43+
44+
$$ \label{eq:slr-vars}
45+
\hat{\sigma}^2 = \left( 1 - r_{xy}^2 \right) s_y^2 \; .
46+
$$
47+
48+
49+
**Proof:** The [residual variance](/D/resvar) can be expressed in terms of the [residual sum of squares](/D/rss):
50+
51+
$$ \label{eq:slr-res}
52+
\hat{\sigma}^2 = \frac{1}{n-1} \, \mathrm{RSS}(\hat{\beta}_0,\hat{\beta}_1)
53+
$$
54+
55+
and [the residual sum of squares for simple linear regression](/P/slr-sss) is
56+
57+
$$ \label{eq:slr-rss}
58+
\mathrm{RSS}(\hat{\beta}_0,\hat{\beta}_1) = (n-1) \left( s_y^2 - \frac{s_{xy}^2}{s_x^2} \right) \; .
59+
$$
60+
61+
Combining \eqref{eq:slr-res} and \eqref{eq:slr-rss}, we obtain:
62+
63+
$$ \label{eq:slr-vars-s1}
64+
\begin{split}
65+
\hat{\sigma}^2 &= \left( s_y^2 - \frac{s_{xy}^2}{s_x^2} \right) \\
66+
&= \left( 1 - \frac{s_{xy}^2}{s_x^2 s_y^2} \right) s_y^2 \\
67+
&= \left( 1 - \left( \frac{s_{xy}}{s_x \, s_y} \right)^2 \right) s_y^2 \; .
68+
\end{split}
69+
$$
70+
71+
Using the [relationship between correlation, covariance and standard deviation](/D/corr)
72+
73+
$$ \label{eq:corr-cov-std}
74+
\mathrm{Corr}(X,Y) = \frac{\mathrm{Cov}(X,Y)}{\sqrt{\mathrm{Var}(X)} \sqrt{\mathrm{Var}(Y)}}
75+
$$
76+
77+
which also holds for sample correlation, [sample covariance](/D/cov-samp) and sample [standard deviation](/D/std)
78+
79+
$$ \label{eq:corr-cov-std-samp}
80+
r_{xy} = \frac{s_{xy}}{s_x \, s_y} \; ,
81+
$$
82+
83+
we get the final result:
84+
85+
$$ \label{eq:slr-vars-s2}
86+
\hat{\sigma}^2 = \left( 1 - r_{xy}^2 \right) s_y^2 \; .
8787
$$

P/slr-rsq.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -88,7 +88,7 @@ R^2 &= \frac{\sum_{i=1}^{n} (\bar{y} - \hat{\beta}_1 \bar{x} + \hat{\beta}_1 x_i
8888
\end{split}
8989
$$
9090

91-
Using the [relationship between correlation coefficient and slope estimate](/D/slr-corr), we conclude:
91+
Using the [relationship between correlation coefficient and slope estimate](/P/slr-corr), we conclude:
9292

9393
$$ \label{eq:slr-R2-qed}
9494
R^2 = \left( \frac{s_x}{s_y} \, \hat{\beta}_1 \right)^2 = r_{xy}^2 \; .

P/tglm-para.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -70,7 +70,7 @@ $$
7070
The [covariance across rows for the transformed general linear model](/P/tglm-dist) is equal to
7171

7272
$$ \label{eq:U}
73-
U = ( X_t^\mathrm{T} V^{-1} X_t )^{-1}
73+
U = ( X_t^\mathrm{T} V^{-1} X_t )^{-1} \; .
7474
$$
7575

7676
Applying \eqref{eq:U}, \eqref{eq:X-Xt-T} and \eqref{eq:glm2-wls}, the estimates in \eqref{eq:tglm-wls} can be developed into

0 commit comments

Comments
 (0)