Skip to content

Commit c63c78c

Browse files
JoramSochJoramSoch
authored
added 3 proofs
1 parent eaa1d40 commit c63c78c

3 files changed

Lines changed: 770 additions & 0 deletions

File tree

‎P/anova2-fgm.md‎

Lines changed: 249 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,249 @@
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2022-11-11 16:54:00
9+
10+
title: "F-test for grand mean in two-way analysis of variance"
11+
chapter: "Statistical Models"
12+
section: "Univariate normal data"
13+
topic: "Analysis of variance"
14+
theorem: "F-test for interaction in two-way ANOVA"
15+
16+
sources:
17+
- authors: "Nandy, Siddhartha"
18+
year: 2018
19+
title: "Two-Way Analysis of Variance"
20+
in: "Stat 512: Applied Regression Analysis"
21+
pages: "Purdue University, Summer 2018, Ch. 19"
22+
url: "https://www.stat.purdue.edu/~snandy/stat512/topic7.pdf"
23+
- authors: "Olbricht, Gayla R."
24+
year: 2011
25+
title: "Two-Way ANOVA: Interaction"
26+
in: "Stat 512: Applied Regression Analysis"
27+
pages: "Purdue University, Spring 2011, Lect. 27"
28+
url: "https://www.stat.purdue.edu/~ghobbs/STAT_512/Lecture_Notes/ANOVA/Topic_27.pdf"
29+
30+
proof_id: "P373"
31+
shortcut: "anova2-fgm"
32+
username: "JoramSoch"
33+
---
34+
35+
36+
**Theorem:** Assume the [two-way analysis of variance](/D/anova2) model
37+
38+
$$ \label{eq:anova2}
39+
\begin{split}
40+
y_{ijk} &= \mu + \alpha_i + \beta_j + \gamma_{ij} + \varepsilon_{ijk} \\
41+
\varepsilon_{ijk} &\overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, \sigma^2), \; i = 1, \ldots, a, \; j = 1, \ldots, b, \; k = 1, \dots, n_{ij} \; .
42+
\end{split}
43+
$$
44+
45+
Then, the [test statistic](/D/tstat)
46+
47+
$$ \label{eq:anova2-fgm}
48+
F_M = \frac{n (\bar{y}_{\bullet \bullet \bullet})^2}{\frac{1}{n-ab} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (y_{ijk} - \bar{y}_{i j \bullet})^2}
49+
$$
50+
51+
follows an [F-distribution](/D/f)
52+
53+
$$ \label{eq:anova2-fgm-h0}
54+
F_M \sim \mathrm{F}\left( 1, n-ab \right)
55+
$$
56+
57+
under the [null hypothesis](/D/h0) for the [grand mean](/D/anova2)
58+
59+
$$ \label{eq:anova2-h0}
60+
\begin{split}
61+
H_0: &\; \mu = 0 \\
62+
H_1: &\; \mu \neq 0 \; .
63+
\end{split}
64+
$$
65+
66+
67+
**Proof:** Denote sample sizes as
68+
69+
$$ \label{eq:samp-size}
70+
\begin{split}
71+
n_{ij} &- \text{number of samples in category} \; (i,j) \\
72+
n_{i \bullet} &= \sum_{j=1}^{b} n_{ij} \\
73+
n_{\bullet j} &= \sum_{i=1}^{a} n_{ij} \\
74+
n &= \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij}
75+
\end{split}
76+
$$
77+
78+
and denote sample means as
79+
80+
$$ \label{eq:mean-samp}
81+
\begin{split}
82+
\bar{y}_{\bullet \bullet \bullet} &= \frac{1}{n} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} y_{ijk} \\
83+
\bar{y}_{i \bullet \bullet} &= \frac{1}{n_{i \bullet}} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} y_{ijk} \\
84+
\bar{y}_{\bullet j \bullet} &= \frac{1}{n_{\bullet j}} \sum_{i=1}^{a} \sum_{k=1}^{n_{ij}} y_{ijk} \\
85+
\bar{y}_{i j \bullet} &= \frac{1}{n_{ij}} \sum_{k=1}^{n_{ij}} y_{ijk} \; .
86+
\end{split}
87+
$$
88+
89+
Assume that $\mu$ zero, according to $H_0$ given by \eqref{eq:anova2-h0}. Under this null hypothesis, we have:
90+
91+
$$ \label{eq:yijk-h0}
92+
y_{ijk} \sim \mathcal{N}(\alpha_i + \beta_j + \gamma_{ij}, \sigma^2) \quad \text{for all} \quad i, j, k \; .
93+
$$
94+
95+
Thus, the [random variable](/D/rvar) $U_{ijk} = (y_{ijk} - \alpha_i - \beta_j - \gamma_{ij})/\sigma$ [follows a standard normal distribution](/P/norm-snorm)
96+
97+
$$ \label{eq:Uijk-h0}
98+
U_{ijk} = \frac{y_{ijk} - \alpha_i - \beta_j - \gamma_{ij}}{\sigma} \sim \mathcal{N}(0, 1) \; .
99+
$$
100+
101+
Now consider the following sum
102+
103+
$$ \label{eq:sum-Uijk-s1}
104+
\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} U_{ijk}^2 = \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} \left( \frac{y_{ijk} - \alpha_i - \beta_j - \gamma_{ij}}{\sigma} \right)^2 \\
105+
$$
106+
107+
which can be rewritten as follows:
108+
109+
$$ \label{eq:sum-Uijk-s2}
110+
\begin{split}
111+
\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} U_{ijk}^2 = \frac{1}{\sigma^2} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} & \left[ (y_{ijk} - \alpha_i - \beta_j - \gamma_{ij}) - \right. \\
112+
&\left. [\bar{y}_{\bullet \bullet \bullet} + (\bar{y}_{i \bullet \bullet} - \bar{y}_{\bullet \bullet \bullet}) + (\bar{y}_{\bullet j \bullet} - \bar{y}_{\bullet \bullet \bullet}) + (\bar{y}_{i j \bullet} - \bar{y}_{i \bullet \bullet} - \bar{y}_{\bullet j \bullet} + \bar{y}_{\bullet \bullet \bullet})] \right. + \\
113+
&\left. [\bar{y}_{\bullet \bullet \bullet} + (\bar{y}_{i \bullet \bullet} - \bar{y}_{\bullet \bullet \bullet}) + (\bar{y}_{\bullet j \bullet} - \bar{y}_{\bullet \bullet \bullet}) + (\bar{y}_{i j \bullet} - \bar{y}_{i \bullet \bullet} - \bar{y}_{\bullet j \bullet} + \bar{y}_{\bullet \bullet \bullet})] \right]^2 \\
114+
= \frac{1}{\sigma^2}\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} & \left[ (y_{ijk} - [\bar{y}_{\bullet \bullet \bullet} + (\bar{y}_{i \bullet \bullet} - \bar{y}_{\bullet \bullet \bullet}) + (\bar{y}_{\bullet j \bullet} - \bar{y}_{\bullet \bullet \bullet}) + (\bar{y}_{i j \bullet} - \bar{y}_{i \bullet \bullet} - \bar{y}_{\bullet j \bullet} + \bar{y}_{\bullet \bullet \bullet})]) + \right. \\
115+
&\left. (\bar{y}_{\bullet \bullet \bullet}) + ([\bar{y}_{i \bullet \bullet} - \bar{y}_{\bullet \bullet \bullet}] - \alpha_i) + ([\bar{y}_{\bullet j \bullet} - \bar{y}_{\bullet \bullet \bullet}] - \beta_j) \right. + \\
116+
&\left. ([\bar{y}_{i j \bullet} - \bar{y}_{i \bullet \bullet} - \bar{y}_{\bullet j \bullet} + \bar{y}_{\bullet \bullet \bullet}] - \gamma_{ij}) \right]^2 \\
117+
= \frac{1}{\sigma^2}\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} & \left[ (y_{ijk} - \bar{y}_{i j \bullet}) + (\bar{y}_{\bullet \bullet \bullet}) + (\bar{y}_{i j \bullet} - \bar{y}_{\bullet \bullet \bullet} - \alpha_i - \beta_j - \gamma_{ij}) \right]^2
118+
\end{split}
119+
$$
120+
121+
Because the following sum over $k$ is zero for all $(i,j)$
122+
123+
$$ \label{eq:sum-yijk}
124+
\begin{split}
125+
\sum_{k=1}^{n_{ij}} (y_{ijk} - \bar{y}_{i j \bullet}) &= \sum_{k=1}^{n_{ij}} y_{ijk} - n_{ij} \bar{y}_{ij \bullet} \\
126+
&= \sum_{k=1}^{n_{ij}} y_{ijk} - n_{ij} \cdot \frac{1}{n_{ij}} \sum_{k=1}^{n_{ij}} y_{ijk} \\
127+
&= 0, \; (i,j) \in \left\lbrace 1, \ldots, a \right\rbrace \times \left\lbrace 1, \ldots, b \right\rbrace \; ,
128+
\end{split}
129+
$$
130+
131+
the following sum over $(i,j,k)$ and is also zero
132+
133+
$$ \label{eq:sum-yib}
134+
\begin{split}
135+
\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (\bar{y}_{i j \bullet} - \bar{y}_{\bullet \bullet \bullet}) &= \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} \bar{y}_{i j \bullet} - \bar{y}_{\bullet \bullet \bullet} \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} \\
136+
&= \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} \cdot \frac{1}{n_{ij}} \sum_{k=1}^{n_{ij}} y_{ijk} - n \cdot \frac{1}{n} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} y_{ijk} \\
137+
&= 0
138+
\end{split}
139+
$$
140+
141+
and the term $\bar{y}_{\bullet \bullet \bullet}$ does not depend on $i$, $j$ and $k$
142+
143+
$$ \label{eq:yb-const}
144+
\begin{split}
145+
\bar{y}_{\bullet \bullet \bullet} = \text{const.} \; ,
146+
\end{split}
147+
$$
148+
149+
non-square products in \eqref{eq:sum-Uijk-s2} disappear and the sum reduces to
150+
151+
$$ \label{eq:sum-Uijk-s3}
152+
\begin{split}
153+
\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} U_{ijk}^2 = \frac{1}{\sigma^2} & \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} \left[ (y_{ijk} - \bar{y}_{i j \bullet})^2 + (\bar{y}_{\bullet \bullet \bullet})^2 + (\bar{y}_{i j \bullet} - \bar{y}_{\bullet \bullet \bullet} - \alpha_i - \beta_j - \gamma_{ij})^2 + \right] \\
154+
= \frac{1}{\sigma^2} & \left[ \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} \right. (y_{ijk} - \bar{y}_{i j \bullet})^2 + \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (\bar{y}_{\bullet \bullet \bullet})^2 + \\
155+
& \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} \left. (\bar{y}_{i j \bullet} - \bar{y}_{\bullet \bullet \bullet} - \alpha_i - \beta_j - \gamma_{ij})^2 \right]
156+
\end{split}
157+
$$
158+
159+
[Cochran's theorem](/P/snorm-cochran) states that, if a sum of squared [standard normal](/D/snorm) [random variables](/D/rvar) can be written as a sum of squared forms
160+
161+
$$ \label{eq:cochran-p1}
162+
\begin{split}
163+
\sum_{i=1}^{n} U_i^2 = \sum_{j=1}^{m} Q_j \quad &\text{where} \quad Q_j = U^\mathrm{T} B^{(j)} U \\
164+
&\text{with} \quad \sum_{j=1}^{m} B^{(j)} = I_n \\
165+
&\text{and} \quad r_j = \mathrm{rank}(B^{(j)}) \; ,
166+
\end{split}
167+
$$
168+
169+
then the terms $Q_j$ are [independent](/D/ind) and each term $Q_j$ follows a [chi-squared distribution](/D/chi2) with $r_j$ degrees of freedom:
170+
171+
$$ \label{eq:cochran-p2}
172+
Q_j \sim \chi^2(r_j), \; j = 1, \ldots, m \; .
173+
$$
174+
175+
First, we define the $n \times 1$ vector $U$:
176+
177+
$$ \label{eq:U}
178+
U = \left[ \begin{matrix} u_{1 \bullet} \\ \vdots \\ u_{a \bullet} \end{matrix} \right] \quad \text{where} \quad u_{i \bullet} = \left[ \begin{matrix} u_{i1} \\ \vdots \\ u_{ib} \end{matrix} \right] \quad \text{where} \quad u_{ij} = \left[ \begin{matrix} (y_{i,j,1} - \alpha_i - \beta_j - \gamma_{ij})/\sigma \\ \vdots \\ (y_{i,j,n_{ij}} - \mu - \alpha_i - \beta_j)/\sigma \end{matrix} \right] \; .
179+
$$
180+
181+
Next, we specify the $n \times n$ matrices $B$
182+
183+
$$ \label{eq:B}
184+
\begin{split}
185+
B^{(1)} &= I_n - \mathrm{diag}\left[ \mathrm{diag}\left( \frac{1}{n_{11}} J_{n_{11}}, \; \ldots, \; \frac{1}{n_{1b}} J_{n_{1b}} \right), \; \ldots, \; \mathrm{diag}\left( \frac{1}{n_{a1}} J_{n_{a1}}, \; \ldots, \; \frac{1}{n_{ab}} J_{n_{ab}} \right) \right] \\
186+
B^{(2)} &= \frac{1}{n} J_n \\
187+
B^{(3)} &= \mathrm{diag}\left[ \mathrm{diag}\left( \frac{1}{n_{11}} J_{n_{11}}, \; \ldots, \; \frac{1}{n_{1b}} J_{n_{1b}} \right), \; \ldots, \; \mathrm{diag}\left( \frac{1}{n_{a1}} J_{n_{a1}}, \; \ldots, \; \frac{1}{n_{ab}} J_{n_{ab}} \right) \right] - \frac{1}{n} J_n
188+
\end{split}
189+
$$
190+
191+
where $J_n$ is an $n \times n$ matrix of ones, $J_{n,m}$ is an $n \times m$ matrix of ones and $\mathrm{diag}\left( A_1, \ldots, A_n \right)$ denotes a block-diagonal matrix composed of $A_1, \ldots, A_n$. We observe that those matrices satisfy
192+
193+
$$ \label{eq:U-Q-B}
194+
\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} U_i^2 = Q_1 + Q_2 + Q_3 = U^\mathrm{T} B^{(1)} U + U^\mathrm{T} B^{(2)} U + U^\mathrm{T} B^{(3)} U
195+
$$
196+
197+
as well as
198+
199+
$$ \label{eq:B-In}
200+
B^{(1)} + B^{(2)} + B^{(3)} = I_n
201+
$$
202+
203+
and their ranks are
204+
205+
$$ \label{eq:B-rk}
206+
\begin{split}
207+
\mathrm{rank}\left( B^{(1)} \right) &= n - a \cdot b \\
208+
\mathrm{rank}\left( B^{(2)} \right) &= 1 \\
209+
\mathrm{rank}\left( B^{(3)} \right) &= n - (n-ab) - 1 = a \cdot b - 1 \; .
210+
\end{split}
211+
$$
212+
213+
Let's write down the [explained sum of squares](/D/ess) and the [residual sum of squares](/D/rss) for [two-way analysis of variance](/D/anova2) as
214+
215+
$$ \label{eq:ess-rss}
216+
\begin{split}
217+
\mathrm{ESS} &= \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (\bar{y}_{\bullet \bullet \bullet})^2 \\
218+
\mathrm{RSS} &= \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (y_{ijk} - \bar{y}_{i j \bullet})^2 \; .
219+
\end{split}
220+
$$
221+
222+
Then, using \eqref{eq:sum-Uijk-s3}, \eqref{eq:cochran-p1}, \eqref{eq:cochran-p2}, \eqref{eq:B} and \eqref{eq:B-rk}, we find that
223+
224+
$$ \label{eq:ess-rss-dist}
225+
\begin{split}
226+
\frac{\mathrm{ESS}}{\sigma^2} = \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} \left( \frac{\bar{y}_{\bullet \bullet \bullet}}{\sigma} \right)^2 &= Q_2 = U^\mathrm{T} B^{(2)} U \sim \chi^2(1) \\
227+
\frac{\mathrm{RSS}}{\sigma^2} = \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} \left( \frac{y_{ijk} - \bar{y}_{i j \bullet}}{\sigma} \right)^2 &= Q_1 = U^\mathrm{T} B^{(1)} U \sim \chi^2(n-ab) \; .
228+
\end{split}
229+
$$
230+
231+
Because $\mathrm{ESS}/\sigma^2$ and $\mathrm{RSS}/\sigma^2$ are also independent by \eqref{eq:cochran-p2}, the F-statistic from \eqref{eq:anova2-fgm} is equal to the ratio of two independent [chi-squared distributed](/D/chi2) [random variables](/D/rvar) divided by their degrees of freedom
232+
233+
$$ \label{eq:anova2-fgm-ess-tss}
234+
\begin{split}
235+
F_M &= \frac{(\mathrm{ESS}/\sigma^2)/(1)}{(\mathrm{RSS}/\sigma^2)/(n-ab)} \\
236+
&= \frac{\mathrm{ESS}/(1)}{\mathrm{RSS}/(n-ab)} \\
237+
&= \frac{\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (\bar{y}_{\bullet \bullet \bullet})^2}{\frac{1}{n-ab} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (y_{ijk} - \bar{y}_{i j \bullet})^2} \\
238+
&= \frac{(\bar{y}_{\bullet \bullet \bullet})^2 \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij}}{\frac{1}{n-ab} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (y_{ijk} - \bar{y}_{i j \bullet})^2} \\
239+
&= \frac{n (\bar{y}_{\bullet \bullet \bullet})^2}{\frac{1}{n-ab} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (y_{ijk} - \bar{y}_{i j \bullet})^2}
240+
\end{split}
241+
$$
242+
243+
which, [by definition of the F-distribution](/D/f), is distributed as
244+
245+
$$ \label{eq:anova2-fia-qed}
246+
F_M \sim \mathrm{F}(1, n-ab)
247+
$$
248+
249+
under the [null hypothesis](/D/h0) for the grand mean.

0 commit comments

Comments
 (0)