Skip to content

Commit 8de445e

Browse files
JoramSochJoramSoch
authored
Merge pull request #185 from JoramSoch/master
added 2 definitions and 3 proofs
2 parents ab7265d + 82acf48 commit 8de445e

6 files changed

Lines changed: 620 additions & 56 deletions

File tree

‎D/anova1.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
---
2+
layout: definition
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2022-11-06 10:23:00
9+
10+
title: "One-way analysis of variance"
11+
chapter: "Statistical Models"
12+
section: "Univariate normal data"
13+
topic: "Analysis of variance"
14+
definition: "One-way ANOVA"
15+
16+
sources:
17+
- authors: "Bortz, Jürgen"
18+
year: 1977
19+
title: "Einfaktorielle Varianzanalyse"
20+
in: "Lehrbuch der Statistik. Für Sozialwissenschaftler"
21+
pages: "ch. 12.1, pp. 528ff."
22+
url: "https://books.google.de/books?id=lNCyBgAAQBAJ"
23+
- authors: "Denziloe"
24+
year: 2018
25+
title: "Derive the distribution of the ANOVA F-statistic under the alternative hypothesis"
26+
in: "StackExchange CrossValidated"
27+
pages: "retrieved on 2022-11-06"
28+
url: "https://stats.stackexchange.com/questions/355594/derive-the-distribution-of-the-anova-f-statistic-under-the-alternative-hypothesi"
29+
30+
def_id: "D181"
31+
shortcut: "anova1"
32+
username: "JoramSoch"
33+
---
34+
35+
36+
**Definition:** Consider measurements $y_{ij} \in \mathbb{R}$ from disctinct objects $j = 1, \ldots, n_i$ in separate groups $i = 1, \ldots, k$.
37+
38+
Then, in one-way analysis of variance (ANOVA), these measurements are assumed to come from [normal distributions](/D/norm)
39+
40+
$$ \label{eq:anova1}
41+
y_{ij} \sim \mathcal{N}(\mu_i, \sigma^2) \quad \text{for all} \quad i = 1, \ldots, k \quad \text{and} \quad j = 1, \dots, n_i
42+
$$
43+
44+
where
45+
46+
* $\mu_i$ is the [expected value](/D/mean) in group $i$ and
47+
48+
* $\sigma^2$ is the common [variance](/D/var) across groups.
49+
50+
Alternatively, the model may be written as
51+
52+
$$ \label{eq:anova1-alt}
53+
\begin{split}
54+
y_{ij} &= \mu_i + \varepsilon_{ij} \\
55+
\varepsilon_{ij} &\overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, \sigma^2)
56+
\end{split}
57+
$$
58+
59+
where $\varepsilon_{ij}$ is the [error term](/D/slr) belonging to observation $j$ in category $i$ and $\varepsilon_{ij}$ are the [independent and identically distributed](/D/iid).

‎D/anova2.md‎

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,86 @@
1+
---
2+
layout: definition
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2022-11-06 13:41:00
9+
10+
title: "Two-way analysis of variance"
11+
chapter: "Statistical Models"
12+
section: "Univariate normal data"
13+
topic: "Analysis of variance"
14+
definition: "Two-way ANOVA"
15+
16+
sources:
17+
- authors: "Bortz, Jürgen"
18+
year: 1977
19+
title: "Zwei- und mehrfaktorielle Varianzanalyse"
20+
in: "Lehrbuch der Statistik. Für Sozialwissenschaftler"
21+
pages: "ch. 12.2, pp. 538ff."
22+
url: "https://books.google.de/books?id=lNCyBgAAQBAJ"
23+
- authors: "ttd"
24+
year: 2021
25+
title: "Proof on SSAB/s2~chi2(I-1)(J-1) under the null hypothesis HAB: dij=0 for i=1,...,I and j=1,...,J"
26+
in: "StackExchange CrossValidated"
27+
pages: "retrieved on 2022-11-06"
28+
url: "https://stats.stackexchange.com/questions/545807/proof-on-ss-ab-sigma2-sim-chi2-i-1j-1-under-the-null-hypothesis"
29+
30+
def_id: "D182"
31+
shortcut: "anova2"
32+
username: "JoramSoch"
33+
---
34+
35+
36+
**Definition:** Let there be two factors $A$ and $B$ with levels $i = 1, \ldots, a$ and $j = 1, \ldots, b$ that are used to group measurements $y_{ijk} \in \mathbb{R}$ from distinct objects $k = 1, \ldots, n_{ij}$ into $a \cdot b$ categories $(i,j) \in \left\lbrace 1, \ldots, a \right\rbrace \times \left\lbrace 1, \ldots, b \right\rbrace$.
37+
38+
Then, in two-way analysis of variance (ANOVA), these measurements are assumed to come from [normal distributions](/D/norm)
39+
40+
$$ \label{eq:anova2-p1}
41+
y_{ij} \sim \mathcal{N}(\mu_{ij}, \sigma^2) \quad \text{for all} \quad i = 1, \ldots, a, \quad j = 1, \ldots, b, \quad \text{and} \quad k = 1, \dots, n_{ij}
42+
$$
43+
44+
with
45+
46+
$$ \label{eq:anova2-p2}
47+
\mu_{ij} = \mu + \alpha_i + \beta_j + \gamma_{ij}
48+
$$
49+
50+
where
51+
52+
* $\mu$ is called the "grand mean";
53+
54+
* $\alpha_i$ is the additive "main effect" of the $i$-th level of factor $A$;
55+
56+
* $\beta_j$ is the additive "main effect" of the $j$-th level of factor $B$;
57+
58+
* $\gamma_{ij}$ is the non-additive "interaction effect" of category $(i,j)$;
59+
60+
* $\mu_{ij}$ is the [expected value](/D/mean) in category $(i,j)$; and
61+
62+
* $\sigma^2$ is common [variance](/D/var) across all categories.
63+
64+
Alternatively, the model may be written as
65+
66+
$$ \label{eq:anova2-alt}
67+
\begin{split}
68+
y_{ijk} &= \mu + \alpha_i + \beta_j + \gamma_{ij} + \varepsilon_{ijk} \\
69+
\varepsilon_{ijk} &\sim \mathcal{N}(0, \sigma^2)
70+
\end{split}
71+
$$
72+
73+
where $\varepsilon_{ijk}$ is the [error term](/D/slr) corresponding to observation $k$ belonging to the $i$-th level of $A$ and the $j$-th level of $B$.
74+
75+
As the two-way ANOVA model is underdetermined, the parameters of the model are additionally subject to the constraints
76+
77+
$$ \label{eq:anova2-cons}
78+
\begin{split}
79+
\sum_{i=1}^{a} w_{ij} \alpha_i &= 0 \quad \text{for all} \quad j = 1, \ldots, b \\
80+
\sum_{j=1}^{b} w_{ij} \beta_j &= 0 \quad \text{for all} \quad i = 1, \ldots, a \\
81+
\sum_{i=1}^{a} w_{ij} \gamma_{ij} &= 0 \quad \text{for all} \quad j = 1, \ldots, b \\
82+
\sum_{j=1}^{b} w_{ij} \gamma_{ij} &= 0 \quad \text{for all} \quad i = 1, \ldots, a
83+
\end{split}
84+
$$
85+
86+
where the weights are $w_{ij} = n_{ij}/n$ and the total sample size is $n = \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij}$.

‎I/ToC.md‎

Lines changed: 63 additions & 56 deletions
Original file line numberDiff line numberDiff line change
@@ -563,61 +563,68 @@ title: "Table of Contents"
563563
&emsp;&ensp; 1.2.13. **[Cross-validated log Bayes factor](/P/ugkv-cvlbf)** <br>
564564
&emsp;&ensp; 1.2.14. **[Expectation of cross-validated log Bayes factor](/P/ugkv-cvlbfmean)** <br>
565565

566-
1.3. Simple linear regression <br>
567-
&emsp;&ensp; 1.3.1. *[Definition](/D/slr)* <br>
568-
&emsp;&ensp; 1.3.2. **[Special case of multiple linear regression](/P/slr-mlr)** <br>
569-
&emsp;&ensp; 1.3.3. **[Ordinary least squares](/P/slr-ols)** (1) <br>
570-
&emsp;&ensp; 1.3.4. **[Ordinary least squares](/P/slr-ols2)** (2) <br>
571-
&emsp;&ensp; 1.3.5. **[Expectation of estimates](/P/slr-olsmean)** <br>
572-
&emsp;&ensp; 1.3.6. **[Variance of estimates](/P/slr-olsvar)** <br>
573-
&emsp;&ensp; 1.3.7. **[Distribution of estimates](/P/slr-olsdist)** <br>
574-
&emsp;&ensp; 1.3.8. **[Correlation of estimates](/P/slr-olscorr)** <br>
575-
&emsp;&ensp; 1.3.9. **[Effects of mean-centering](/P/slr-meancent)** <br>
576-
&emsp;&ensp; 1.3.10. *[Regression line](/D/regline)* <br>
577-
&emsp;&ensp; 1.3.11. **[Regression line includes center of mass](/P/slr-comp)** <br>
578-
&emsp;&ensp; 1.3.12. **[Projection of data point to regression line](/P/slr-proj)** <br>
579-
&emsp;&ensp; 1.3.13. **[Sums of squares](/P/slr-sss)** <br>
580-
&emsp;&ensp; 1.3.14. **[Transformation matrices](/P/slr-mat)** <br>
581-
&emsp;&ensp; 1.3.15. **[Weighted least squares](/P/slr-wls)** (1) <br>
582-
&emsp;&ensp; 1.3.16. **[Weighted least squares](/P/slr-wls2)** (2) <br>
583-
&emsp;&ensp; 1.3.17. **[Maximum likelihood estimation](/P/slr-mle)** (1) <br>
584-
&emsp;&ensp; 1.3.18. **[Maximum likelihood estimation](/P/slr-mle2)** (2) <br>
585-
&emsp;&ensp; 1.3.19. **[Sum of residuals is zero](/P/slr-ressum)** <br>
586-
&emsp;&ensp; 1.3.20. **[Correlation with covariate is zero](/P/slr-rescorr)** <br>
587-
&emsp;&ensp; 1.3.21. **[Residual variance in terms of sample variance](/P/slr-resvar)** <br>
588-
&emsp;&ensp; 1.3.22. **[Correlation coefficient in terms of slope estimate](/P/slr-corr)** <br>
589-
&emsp;&ensp; 1.3.23. **[Coefficient of determination in terms of correlation coefficient](/P/slr-rsq)** <br>
590-
591-
1.4. Multiple linear regression <br>
592-
&emsp;&ensp; 1.4.1. *[Definition](/D/mlr)* <br>
593-
&emsp;&ensp; 1.4.2. **[Special case of general linear model](/P/mlr-glm)** <br>
594-
&emsp;&ensp; 1.4.3. **[Ordinary least squares](/P/mlr-ols)** (1) <br>
595-
&emsp;&ensp; 1.4.4. **[Ordinary least squares](/P/mlr-ols2)** (2) <br>
596-
&emsp;&ensp; 1.4.5. *[Total sum of squares](/D/tss)* <br>
597-
&emsp;&ensp; 1.4.6. *[Explained sum of squares](/D/ess)* <br>
598-
&emsp;&ensp; 1.4.7. *[Residual sum of squares](/D/rss)* <br>
599-
&emsp;&ensp; 1.4.8. **[Total, explained and residual sum of squares](/P/mlr-pss)** <br>
600-
&emsp;&ensp; 1.4.9. *[Estimation matrix](/D/emat)* <br>
601-
&emsp;&ensp; 1.4.10. *[Projection matrix](/D/pmat)* <br>
602-
&emsp;&ensp; 1.4.11. *[Residual-forming matrix](/D/rfmat)* <br>
603-
&emsp;&ensp; 1.4.12. **[Estimation, projection and residual-forming matrix](/P/mlr-mat)** <br>
604-
&emsp;&ensp; 1.4.13. **[Idempotence of projection and residual-forming matrix](/P/mlr-idem)** <br>
605-
&emsp;&ensp; 1.4.14. **[Weighted least squares](/P/mlr-wls)** (1) <br>
606-
&emsp;&ensp; 1.4.15. **[Weighted least squares](/P/mlr-wls2)** (2) <br>
607-
&emsp;&ensp; 1.4.16. **[Maximum likelihood estimation](/P/mlr-mle)** <br>
608-
&emsp;&ensp; 1.4.17. **[Maximum log-likelihood](/P/mlr-mll)** <br>
609-
&emsp;&ensp; 1.4.18. **[Deviance function](/P/mlr-dev)** <br>
610-
&emsp;&ensp; 1.4.19. **[Akaike information criterion](/P/mlr-aic)** <br>
611-
&emsp;&ensp; 1.4.20. **[Bayesian information criterion](/P/mlr-bic)** <br>
612-
&emsp;&ensp; 1.4.21. **[Corrected Akaike information criterion](/P/mlr-aicc)** <br>
613-
614-
1.5. Bayesian linear regression <br>
615-
&emsp;&ensp; 1.5.1. **[Conjugate prior distribution](/P/blr-prior)** <br>
616-
&emsp;&ensp; 1.5.2. **[Posterior distribution](/P/blr-post)** <br>
617-
&emsp;&ensp; 1.5.3. **[Log model evidence](/P/blr-lme)** <br>
618-
&emsp;&ensp; 1.5.4. **[Deviance information criterion](/P/blr-dic)** <br>
619-
&emsp;&ensp; 1.5.5. **[Posterior probability of alternative hypothesis](/P/blr-pp)** <br>
620-
&emsp;&ensp; 1.5.6. **[Posterior credibility region excluding null hypothesis](/P/blr-pcr)** <br>
566+
1.3. Analysis of variance <br>
567+
&emsp;&ensp; 1.3.1. *[One-way ANOVA](/D/anova1)* <br>
568+
&emsp;&ensp; 1.3.2. **[Ordinary least squares for one-way ANOVA](/P/anova1-ols)** <br>
569+
&emsp;&ensp; 1.3.3. **[F-test for main effect in one-way ANOVA](/P/anova1-f)** <br>
570+
&emsp;&ensp; 1.3.4. *[Two-way ANOVA](/D/anova2)* <br>
571+
&emsp;&ensp; 1.3.5. **[Ordinary least squares for two-way ANOVA](/P/anova2-ols)** <br>
572+
573+
1.4. Simple linear regression <br>
574+
&emsp;&ensp; 1.4.1. *[Definition](/D/slr)* <br>
575+
&emsp;&ensp; 1.4.2. **[Special case of multiple linear regression](/P/slr-mlr)** <br>
576+
&emsp;&ensp; 1.4.3. **[Ordinary least squares](/P/slr-ols)** (1) <br>
577+
&emsp;&ensp; 1.4.4. **[Ordinary least squares](/P/slr-ols2)** (2) <br>
578+
&emsp;&ensp; 1.4.5. **[Expectation of estimates](/P/slr-olsmean)** <br>
579+
&emsp;&ensp; 1.4.6. **[Variance of estimates](/P/slr-olsvar)** <br>
580+
&emsp;&ensp; 1.4.7. **[Distribution of estimates](/P/slr-olsdist)** <br>
581+
&emsp;&ensp; 1.4.8. **[Correlation of estimates](/P/slr-olscorr)** <br>
582+
&emsp;&ensp; 1.4.9. **[Effects of mean-centering](/P/slr-meancent)** <br>
583+
&emsp;&ensp; 1.4.10. *[Regression line](/D/regline)* <br>
584+
&emsp;&ensp; 1.4.11. **[Regression line includes center of mass](/P/slr-comp)** <br>
585+
&emsp;&ensp; 1.4.12. **[Projection of data point to regression line](/P/slr-proj)** <br>
586+
&emsp;&ensp; 1.4.13. **[Sums of squares](/P/slr-sss)** <br>
587+
&emsp;&ensp; 1.4.14. **[Transformation matrices](/P/slr-mat)** <br>
588+
&emsp;&ensp; 1.4.15. **[Weighted least squares](/P/slr-wls)** (1) <br>
589+
&emsp;&ensp; 1.4.16. **[Weighted least squares](/P/slr-wls2)** (2) <br>
590+
&emsp;&ensp; 1.4.17. **[Maximum likelihood estimation](/P/slr-mle)** (1) <br>
591+
&emsp;&ensp; 1.4.18. **[Maximum likelihood estimation](/P/slr-mle2)** (2) <br>
592+
&emsp;&ensp; 1.4.19. **[Sum of residuals is zero](/P/slr-ressum)** <br>
593+
&emsp;&ensp; 1.4.20. **[Correlation with covariate is zero](/P/slr-rescorr)** <br>
594+
&emsp;&ensp; 1.4.21. **[Residual variance in terms of sample variance](/P/slr-resvar)** <br>
595+
&emsp;&ensp; 1.4.22. **[Correlation coefficient in terms of slope estimate](/P/slr-corr)** <br>
596+
&emsp;&ensp; 1.4.23. **[Coefficient of determination in terms of correlation coefficient](/P/slr-rsq)** <br>
597+
598+
1.5. Multiple linear regression <br>
599+
&emsp;&ensp; 1.5.1. *[Definition](/D/mlr)* <br>
600+
&emsp;&ensp; 1.5.2. **[Special case of general linear model](/P/mlr-glm)** <br>
601+
&emsp;&ensp; 1.5.3. **[Ordinary least squares](/P/mlr-ols)** (1) <br>
602+
&emsp;&ensp; 1.5.4. **[Ordinary least squares](/P/mlr-ols2)** (2) <br>
603+
&emsp;&ensp; 1.5.5. *[Total sum of squares](/D/tss)* <br>
604+
&emsp;&ensp; 1.5.6. *[Explained sum of squares](/D/ess)* <br>
605+
&emsp;&ensp; 1.5.7. *[Residual sum of squares](/D/rss)* <br>
606+
&emsp;&ensp; 1.5.8. **[Total, explained and residual sum of squares](/P/mlr-pss)** <br>
607+
&emsp;&ensp; 1.5.9. *[Estimation matrix](/D/emat)* <br>
608+
&emsp;&ensp; 1.5.10. *[Projection matrix](/D/pmat)* <br>
609+
&emsp;&ensp; 1.5.11. *[Residual-forming matrix](/D/rfmat)* <br>
610+
&emsp;&ensp; 1.5.12. **[Estimation, projection and residual-forming matrix](/P/mlr-mat)** <br>
611+
&emsp;&ensp; 1.5.13. **[Idempotence of projection and residual-forming matrix](/P/mlr-idem)** <br>
612+
&emsp;&ensp; 1.5.14. **[Weighted least squares](/P/mlr-wls)** (1) <br>
613+
&emsp;&ensp; 1.5.15. **[Weighted least squares](/P/mlr-wls2)** (2) <br>
614+
&emsp;&ensp; 1.5.16. **[Maximum likelihood estimation](/P/mlr-mle)** <br>
615+
&emsp;&ensp; 1.5.17. **[Maximum log-likelihood](/P/mlr-mll)** <br>
616+
&emsp;&ensp; 1.5.18. **[Deviance function](/P/mlr-dev)** <br>
617+
&emsp;&ensp; 1.5.19. **[Akaike information criterion](/P/mlr-aic)** <br>
618+
&emsp;&ensp; 1.5.20. **[Bayesian information criterion](/P/mlr-bic)** <br>
619+
&emsp;&ensp; 1.5.21. **[Corrected Akaike information criterion](/P/mlr-aicc)** <br>
620+
621+
1.6. Bayesian linear regression <br>
622+
&emsp;&ensp; 1.6.1. **[Conjugate prior distribution](/P/blr-prior)** <br>
623+
&emsp;&ensp; 1.6.2. **[Posterior distribution](/P/blr-post)** <br>
624+
&emsp;&ensp; 1.6.3. **[Log model evidence](/P/blr-lme)** <br>
625+
&emsp;&ensp; 1.6.4. **[Deviance information criterion](/P/blr-dic)** <br>
626+
&emsp;&ensp; 1.6.5. **[Posterior probability of alternative hypothesis](/P/blr-pp)** <br>
627+
&emsp;&ensp; 1.6.6. **[Posterior credibility region excluding null hypothesis](/P/blr-pcr)** <br>
621628

622629
2. Multivariate normal data
623630

@@ -773,4 +780,4 @@ title: "Table of Contents"
773780
3.5. Bayesian model averaging <br>
774781
&emsp;&ensp; 3.5.1. *[Definition](/D/bma)* <br>
775782
&emsp;&ensp; 3.5.2. **[Derivation](/P/bma-der)** <br>
776-
&emsp;&ensp; 3.5.3. **[Calculation from log model evidences](/P/bma-lme)** <br>
783+
&emsp;&ensp; 3.5.3. **[Calculation from log model evidences](/P/bma-lme)** <br>

0 commit comments

Comments
 (0)