Merge pull request #100 from StatProofBook/master

update to master

Author: JoramSoch

I/PbA.md

@@ -8,7 +8,7 @@ title: "Proof by Author"
 
 - [Covariance matrix of the multinomial distribution](/P/mult-cov)
 
-### JoramSoch (355 proofs)
+### JoramSoch (361 proofs)
 
 - [Accuracy and complexity for the univariate Gaussian](/P/ug-anc)
 - [Accuracy and complexity for the univariate Gaussian with known variance](/P/ugkv-anc)
@@ -18,6 +18,7 @@ title: "Proof by Author"
 - [Additivity of the Kullback-Leibler divergence for independent distributions](/P/kl-add)
 - [Additivity of the variance for independent random variables](/P/var-add)
 - [Akaike information criterion for multiple linear regression](/P/mlr-aic)
+- [Application of Cochran's theorem to two-way analysis of variance](/P/anova2-cochran)
 - [Bayes' rule](/P/bayes-rule)
 - [Bayes' theorem](/P/bayes-th)
 - [Bayesian information criterion for multiple linear regression](/P/mlr-bic)
@@ -105,6 +106,8 @@ title: "Proof by Author"
 - [Expression of the cumulative distribution function of the normal distribution without the error function](/P/norm-cdfwerf)
 - [Expression of the probability mass function of the beta-binomial distribution using only the gamma function](/P/betabin-pmfitogf)
 - [Extreme points of the probability density function of the normal distribution](/P/norm-extr)
+- [F-statistic in terms of ordinary least squares estimates in one-way analysis of variance](/P/anova1-fols)
+- [F-statistics in terms of ordinary least squares estimates in two-way analysis of variance](/P/anova2-fols)
 - [F-test for grand mean in two-way analysis of variance](/P/anova2-fgm)
 - [F-test for interaction in two-way analysis of variance](/P/anova2-fia)
 - [F-test for main effect in one-way analysis of variance](/P/anova1-f)
@@ -228,7 +231,9 @@ title: "Proof by Author"
 - [Parameters of the corresponding forward model](/P/cfm-para)
 - [Partition of a covariance matrix into expected values](/P/covmat-mean)
 - [Partition of covariance into expected values](/P/cov-mean)
+- [Partition of sums of squares in one-way analysis of variance](/P/anova1-pss)
 - [Partition of sums of squares in ordinary least squares](/P/mlr-pss)
+- [Partition of sums of squares in two-way analysis of variance](/P/anova2-pss)
 - [Partition of the log model evidence into accuracy and complexity](/P/lme-anc)
 - [Partition of the mean squared error into bias and variance](/P/mse-bnv)
 - [Partition of variance into expected values](/P/var-mean)
@@ -327,6 +332,7 @@ title: "Proof by Author"
 - [Relationship between residual variance and sample variance in simple linear regression](/P/slr-resvar)
 - [Relationship between second raw moment, variance and mean](/P/momraw-2nd)
 - [Relationship between signal-to-noise ratio and R²](/P/snr-rsq)
+- [Reparametrization for one-way analysis of variance](/P/anova1-repara)
 - [Sampling from the matrix-normal distribution](/P/matn-samp)
 - [Sampling from the normal-gamma distribution](/P/ng-samp)
 - [Scaling of the covariance matrix upon multiplication with constant matrix](/P/covmat-scal)

I/PbN.md

@@ -380,3 +380,9 @@ title: "Proof by Number"
 | P372 | anova2-fme | [F-test for main effect in two-way analysis of variance](/P/anova2-fme) | JoramSoch | 2022-11-10 |
 | P373 | anova2-fia | [F-test for interaction in two-way analysis of variance](/P/anova2-fia) | JoramSoch | 2022-11-11 |
 | P374 | anova2-fgm | [F-test for grand mean in two-way analysis of variance](/P/anova2-fgm) | JoramSoch | 2022-11-11 |
+| P375 | anova1-repara | [Reparametrization for one-way analysis of variance](/P/anova1-repara) | JoramSoch | 2022-11-15 |
+| P376 | anova1-pss | [Partition of sums of squares in one-way analysis of variance](/P/anova1-pss) | JoramSoch | 2022-11-15 |
+| P377 | anova1-fols | [F-statistic in terms of ordinary least squares estimates in one-way analysis of variance](/P/anova1-fols) | JoramSoch | 2022-11-15 |
+| P378 | anova2-cochran | [Application of Cochran's theorem to two-way analysis of variance](/P/anova2-cochran) | JoramSoch | 2022-11-16 |
+| P379 | anova2-pss | [Partition of sums of squares in two-way analysis of variance](/P/anova2-pss) | JoramSoch | 2022-11-16 |
+| P380 | anova2-fols | [F-statistics in terms of ordinary least squares estimates in two-way analysis of variance](/P/anova2-fols) | JoramSoch | 2022-11-16 |

I/PbT.md

@@ -14,6 +14,7 @@ title: "Proof by Topic"
 - [Additivity of the Kullback-Leibler divergence for independent distributions](/P/kl-add)
 - [Additivity of the variance for independent random variables](/P/var-add)
 - [Akaike information criterion for multiple linear regression](/P/mlr-aic)
+- [Application of Cochran's theorem to two-way analysis of variance](/P/anova2-cochran)
 
 ### B
 
@@ -120,6 +121,8 @@ title: "Proof by Topic"
 
 ### F
 
+- [F-statistic in terms of ordinary least squares estimates in one-way analysis of variance](/P/anova1-fols)
+- [F-statistics in terms of ordinary least squares estimates in two-way analysis of variance](/P/anova2-fols)
 - [F-test for grand mean in two-way analysis of variance](/P/anova2-fgm)
 - [F-test for interaction in two-way analysis of variance](/P/anova2-fia)
 - [F-test for main effect in one-way analysis of variance](/P/anova1-f)
@@ -277,7 +280,9 @@ title: "Proof by Topic"
 - [Parameters of the corresponding forward model](/P/cfm-para)
 - [Partition of a covariance matrix into expected values](/P/covmat-mean)
 - [Partition of covariance into expected values](/P/cov-mean)
+- [Partition of sums of squares in one-way analysis of variance](/P/anova1-pss)
 - [Partition of sums of squares in ordinary least squares](/P/mlr-pss)
+- [Partition of sums of squares in two-way analysis of variance](/P/anova2-pss)
 - [Partition of the log model evidence into accuracy and complexity](/P/lme-anc)
 - [Partition of the mean squared error into bias and variance](/P/mse-bnv)
 - [Partition of variance into expected values](/P/var-mean)
@@ -386,6 +391,7 @@ title: "Proof by Topic"
 - [Relationship between residual variance and sample variance in simple linear regression](/P/slr-resvar)
 - [Relationship between second raw moment, variance and mean](/P/momraw-2nd)
 - [Relationship between signal-to-noise ratio and R²](/P/snr-rsq)
+- [Reparametrization for one-way analysis of variance](/P/anova1-repara)
 
 ### S
 

I/PwS.md

@@ -37,6 +37,8 @@ title: "Proofs without Source"
 - [Expectation of the cross-validated log Bayes factor for the univariate Gaussian with known variance](/P/ugkv-cvlbfmean)
 - [Expectation of the log Bayes factor for the univariate Gaussian with known variance](/P/ugkv-lbfmean)
 - [Exponential distribution is a special case of gamma distribution](/P/exp-gam)
+- [F-statistic in terms of ordinary least squares estimates in one-way analysis of variance](/P/anova1-fols)
+- [F-statistics in terms of ordinary least squares estimates in two-way analysis of variance](/P/anova2-fols)
 - [First raw moment is mean](/P/momraw-1st)
 - [Gamma distribution is a special case of Wishart distribution](/P/gam-wish)
 - [Joint likelihood is the product of likelihood function and prior density](/P/jl-lfnprior)

P/anova1-f.md

@@ -30,32 +30,50 @@ username: "JoramSoch"
 **Theorem:** Assume the [one-way analysis of variance](/D/anova1) model
 
 $$ \label{eq:anova1}
-y_{ij} = \mu_i + \varepsilon_{ij}, \; \varepsilon_{ij} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, \sigma^2), \; i = 1, \ldots, k, \; j = 1, \dots, n_i \; ,
+y_{ij} = \mu_i + \varepsilon_{ij}, \; \varepsilon_{ij} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, \sigma^2), \; i = 1, \ldots, k, \; j = 1, \dots, n_i \; .
 $$
 
-and consider the [null](/D/h0) and [alternative](/D/h1) hypothesis
+Then, the [test statistic](/D/tstat)
+
+$$ \label{eq:anova1-f}
+F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\bar{y}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2}
+$$
+
+follows an [F-distribution](/D/f)
+
+$$ \label{eq:anova1-f-h0}
+F \sim \mathrm{F}(k-1, n-k)
+$$
+
+under the [null hypothesis](/D/h0)
 
 $$ \label{eq:anova1-h0}
 \begin{split}
-H_0: &\; \mu_1 = \ldots = \mu_K \\
+H_0: &\; \mu_1 = \ldots = \mu_k \\
 H_1: &\; \mu_i \neq \mu_j \quad \text{for at least one} \quad i,j \in \left\lbrace 1, \ldots, k \right\rbrace, \; i \neq j \; .
 \end{split}
 $$
 
-Then, the [test statistic](/D/tstat)
 
-$$ \label{eq:anova1-f}
-F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\bar{y}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2}
+**Proof:** Denote sample sizes as
+
+$$ \label{eq:samp-size}
+\begin{split}
+n_i &- \text{number of samples in category} \; i \\
+n &= \sum_{i=1}^{k} n_ij
+\end{split}
 $$
 
-follows an [F-distribution](/D/f) under the null hypothesis:
+and denote sample means as
 
-$$ \label{eq:anova1-f-h0}
-F \sim \mathrm{F}(k-1, n-k), \; \text{if} \; H_0 \; .
+$$ \label{eq:mean-samp}
+\begin{split}
+\bar{y}_i &= \frac{1}{n_i} \sum_{j=1}^{n_i} y_{ij} \\
+\bar{y} &= \frac{1}{n} \sum_{i=1}^{k} \sum_{j=1}^{n_i} y_{ij} \; .
+\end{split}
 $$
 
-
-**Proof:** Let $\mu$ be the common [mean](/D/mean) under the [null hypothesis](/D/h0) $\mu_1 = \ldots = \mu_K = \mu$. Under $H_0$, we have:
+Let $\mu$ be the common [mean](/D/mean) according to $H_0$ given by \eqref{eq:anova1-h0}, i.e. $\mu_1 = \ldots = \mu_k = \mu$. Under this null hypothesis, we have:
 
 $$ \label{eq:yij-h0}
 y_{ij} \sim \mathcal{N}(\mu, \sigma^2) \quad \text{for all} \quad i = 1, \ldots, k, \; j = 1, \ldots, n_i \; .
@@ -98,7 +116,7 @@ $$ \label{eq:sum-yib}
 \end{split}
 $$
 
-where $n = \sum_{i=1}^{k} n_i$, the sum in \eqref{eq:sum-Uij-s1} reduces to
+non-square products in \eqref{eq:sum-Uij-s1} disappear and the sum reduces to
 
 $$ \label{eq:sum-Uij-s2}
 \begin{split}
@@ -137,17 +155,29 @@ $$
 
 Then, we observe that the sum in \eqref{eq:sum-Uij-s2} can be represented in the form of \eqref{eq:cochran-p1} using the matrices
 
-$$ \label{eq:sum-Uij-s3-Bj}
+$$ \label{eq:B}
 \begin{split}
-B^{(1)} &= I_n - \mathrm{diag}\left( \frac{1}{n_1} J_{n_1}, \; \ldots, \; \frac{1}{n_K} J_{n_K} \right) \\
-B^{(2)} &= \mathrm{diag}\left( \frac{1}{n_1} J_{n_1}, \; \ldots, \; \frac{1}{n_K} J_{n_K} \right) - \frac{1}{n} J_n \\
+B^{(1)} &= I_n - \mathrm{diag}\left( \frac{1}{n_1} J_{n_1}, \; \ldots, \; \frac{1}{n_k} J_{n_k} \right) \\
+B^{(2)} &= \mathrm{diag}\left( \frac{1}{n_1} J_{n_1}, \; \ldots, \; \frac{1}{n_k} J_{n_k} \right) - \frac{1}{n} J_n \\
 B^{(3)} &= \frac{1}{n} J_n
 \end{split}
 $$
 
-where $J_n$ is an $n \times n$ matrix of ones and $\mathrm{diag}\left( A_1, \ldots, A_n \right)$ denotes a block-diagonal matrix composed of $A_1, \ldots, A_n$. The matrices in \eqref{eq:sum-Uij-s3-Bj} fulfill $B^{(1)} + B^{(2)} + B^{(3)} = I_n$ and their ranks are given by:
+where $J_n$ is an $n \times n$ matrix of ones and $\mathrm{diag}\left( A_1, \ldots, A_n \right)$ denotes a block-diagonal matrix composed of $A_1, \ldots, A_n$. We observe that those matrices satisfy
 
-$$ \label{eq:sum-Uij-s3-Bj-rk}
+$$ \label{eq:U-Q-B}
+\sum_{i=1}^{k} \sum_{j=1}^{n_i} U_{ij}^2 = Q_1 + Q_2 + Q_3 = U^\mathrm{T} B^{(1)} U + U^\mathrm{T} B^{(2)} U + U^\mathrm{T} B^{(3)} U
+$$
+
+as well as
+
+$$ \label{eq:B-In}
+B^{(1)} + B^{(2)} + B^{(3)} = I_n
+$$
+
+and their ranks are:
+
+$$ \label{eq:B-rk}
 \begin{split}
 \mathrm{rank}\left( B^{(1)} \right) &= n-k \\
 \mathrm{rank}\left( B^{(2)} \right) &= k-1 \\
@@ -164,7 +194,7 @@ $$ \label{eq:ess-rss}
 \end{split}
 $$
 
-Then, using \eqref{eq:sum-Uij-s2}, \eqref{eq:cochran-p1}, \eqref{eq:cochran-p2}, \eqref{eq:sum-Uij-s3-Bj} and \eqref{eq:sum-Uij-s3-Bj-rk}, we find that
+Then, using \eqref{eq:sum-Uij-s2}, \eqref{eq:cochran-p1}, \eqref{eq:cochran-p2}, \eqref{eq:B} and \eqref{eq:B-rk}, we find that
 
 $$ \label{eq:ess-rss-dist}
 \begin{split}
@@ -184,8 +214,10 @@ F &= \frac{(\mathrm{ESS}/\sigma^2)/(k-1)}{(\mathrm{RSS}/\sigma^2)/(n-k)} \\
 \end{split}
 $$
 
-which, [by definition of the F-distribution](/D/f), is distributed as:
+which, [by definition of the F-distribution](/D/f), is distributed as
 
 $$ \label{eq:anova1-f-qed}
-F \sim \mathrm{F}(k-1, n-k), \; \text{if} \; H_0 \; .
+F \sim \mathrm{F}(k-1, n-k)
 $$
+
+under the [null hypothesis](/D/h0) for the main effect.

P/anova1-fols.md

@@ -7,7 +7,7 @@ affiliation: "BCCN Berlin"
 e_mail: "joram.soch@bccn-berlin.de"
 date: 2022-11-15 17:35:00
 
-title: "F-statistic for main effect in terms of ordinary least squares estimates in one-way analysis of variance"
+title: "F-statistic in terms of ordinary least squares estimates in one-way analysis of variance"
 chapter: "Statistical Models"
 section: "Univariate normal data"
 topic: "Analysis of variance"
@@ -46,7 +46,7 @@ $$ \label{eq:anova1-f}
 F = \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\bar{y}_i - \bar{y})^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2}
 $$
 
-where $\bar{y}_i$ is the average of all values $y_{ij}$ from category $i$ and $\bar{y}$ is the grand mean of all values $y_{ij}$ from all categories $i = 1, \ldots, k$.
+where $\bar{y} _i$ is the average of all values $y_{ij}$ from category $i$ and $\bar{y}$ is the grand mean of all values $y_{ij}$ from all categories $i = 1, \ldots, k$.
 
 1) The [ordinary least squares estimates for one-way ANOVA](/P/anova1-ols) are
 

P/anova1-pss.md

@@ -39,7 +39,7 @@ $$ \label{eq:anova1-pss}
 \mathrm{SS}_\mathrm{tot} = \mathrm{SS}_\mathrm{treat} + \mathrm{SS}_\mathrm{res}
 $$
 
-where $\mathrm{SS}_\mathrm{tot}$ is the [total sum of squares](/D/tss), $\mathrm{SS}_\mathrm{treat}$ is the [treatment sum of squares](/D/trss) (equivalent to [explained sum of squares](/D/ess)) and $\mathrm{SS}_\mathrm{res}$ is the [residual sum of squares](/D/rss).
+where $\mathrm{SS} _\mathrm{tot}$ is the [total sum of squares](/D/tss), $\mathrm{SS} _\mathrm{treat}$ is the [treatment sum of squares](/D/trss) (equivalent to [explained sum of squares](/D/ess)) and $\mathrm{SS} _\mathrm{res}$ is the [residual sum of squares](/D/rss).
 
 
 **Proof:** The [total sum of squares](/D/tss) for [one-way ANOVA](/D/anova1) is given by

P/anova1-repara.md

@@ -11,7 +11,7 @@ title: "Reparametrization for one-way analysis of variance"
 chapter: "Statistical Models"
 section: "Univariate normal data"
 topic: "Analysis of variance"
-theorem: "Reparametrization for one-way ANOVA"
+theorem: "Reparametrization of one-way ANOVA"
 
 sources:
   - authors: "Wikipedia"
@@ -94,7 +94,7 @@ $$
 
 $$ \label{eq:anova1-repara-c2-qed}
 \begin{split}
-\hat{\mu} &= \bar{y}_{\bullet \bullet} \hphantom{\bar{y}_{i \bullet} - } = \frac{1}{n} \sum_{i=1}^{k} \sum_{j=1}^{n_i} y_{ij} \\
+\hat{\mu} &= \bar{y}_{\bullet \bullet} = \frac{1}{n} \sum_{i=1}^{k} \sum_{j=1}^{n_i} y_{ij} \\
 \hat{\delta}_i &= \bar{y}_{i \bullet} - \bar{y}_{\bullet \bullet} = \frac{1}{n_i} \sum_{j=1}^{n_i} y_{ij} - \frac{1}{n} \sum_{i=1}^{k} \sum_{j=1}^{n_i} y_{ij} \; .
 \end{split}
 $$
@@ -133,7 +133,7 @@ $$ \label{eq:anova1-repara-c4-s1}
 F &= \frac{\left( \frac{1}{\sigma^2} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (\hat{\delta}_i - \delta_i)^2 \right)/(k-1)}{\left( \frac{1}{\sigma^2} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2 \right)/(n-k)} \\
 &= \frac{\frac{1}{k-1} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (\hat{\delta}_i - \delta_i)^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2} \\
 &= \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i (\hat{\delta}_i - \delta_i)^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2} \\
-&= \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i \hat{\delta}_i^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2}
+&\overset{\eqref{eq:anova1-repara-c4-h0}}{=} \frac{\frac{1}{k-1} \sum_{i=1}^{k} n_i \hat{\delta}_i^2}{\frac{1}{n-k} \sum_{i=1}^{k} \sum_{j=1}^{n_i} (y_{ij} - \bar{y}_i)^2}
 \end{split}
 $$
 
@@ -143,4 +143,4 @@ $$ \label{eq:anova1-repara-c4-qed}
 F \sim \mathrm{F}(k-1, n-k)
 $$
 
-under the null hypothesis \eqref{eq:anova1-repara-c4-h0}.
+under the null hypothesis.

P/anova2-cochran.md

@@ -152,7 +152,7 @@ $$ \label{eq:sum-Uijk-s3d}
 &\overset{\eqref{eq:sum-Uijk-s3b}}{=} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} (\bar{y}_{i j \bullet} - \bar{y}_{\bullet \bullet \bullet} - \gamma_{ij}) \\
 &= \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} \bar{y}_{i j \bullet} - \bar{y}_{\bullet \bullet \bullet} \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} - \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} \gamma_{ij} \\
 &\overset{\eqref{eq:mean-samp}}{=} \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} \cdot \frac{1}{n_{ij}} \sum_{k=1}^{n_{ij}} y_{ijk} - n \cdot \frac{1}{n} \sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} y_{ijk} - \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} \gamma_{ij} \\
-&= - \frac{1}{n} \sum_{i=1}^{a} \sum_{j=1}^{b} \frac{n_{ij}}{n} \gamma_{ij} \overset{\eqref{eq:anova2-constr}}{=} 0 \; .
+&= - \sum_{i=1}^{a} \sum_{j=1}^{b} n_{ij} \gamma_{ij} = - \frac{1}{n} \sum_{i=1}^{a} \sum_{j=1}^{b} \frac{n_{ij}}{n} \gamma_{ij} \overset{\eqref{eq:anova2-constr}}{=} 0 \; .
 \end{split}
 $$
 

P/anova2-fgm.md

@@ -11,7 +11,7 @@ title: "F-test for grand mean in two-way analysis of variance"
 chapter: "Statistical Models"
 section: "Univariate normal data"
 topic: "Analysis of variance"
-theorem: "F-test for interaction in two-way ANOVA"
+theorem: "F-test for grand mean in two-way ANOVA"
 
 sources:
   - authors: "Nandy, Siddhartha"
@@ -141,9 +141,7 @@ $$
 and the term $\bar{y}_{\bullet \bullet \bullet}$ does not depend on $i$, $j$ and $k$
 
 $$ \label{eq:yb-const}
-\begin{split}
 \bar{y}_{\bullet \bullet \bullet} = \text{const.} \; ,
-\end{split}
 $$
 
 non-square products in \eqref{eq:sum-Uijk-s2} disappear and the sum reduces to
@@ -191,7 +189,7 @@ $$
 where $J_n$ is an $n \times n$ matrix of ones, $J_{n,m}$ is an $n \times m$ matrix of ones and $\mathrm{diag}\left( A_1, \ldots, A_n \right)$ denotes a block-diagonal matrix composed of $A_1, \ldots, A_n$. We observe that those matrices satisfy
 
 $$ \label{eq:U-Q-B}
-\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} U_i^2 = Q_1 + Q_2 + Q_3 = U^\mathrm{T} B^{(1)} U + U^\mathrm{T} B^{(2)} U + U^\mathrm{T} B^{(3)} U
+\sum_{i=1}^{a} \sum_{j=1}^{b} \sum_{k=1}^{n_{ij}} U_{ijk]^2 = Q_1 + Q_2 + Q_3 = U^\mathrm{T} B^{(1)} U + U^\mathrm{T} B^{(2)} U + U^\mathrm{T} B^{(3)} U
 $$
 
 as well as