Skip to content

Commit ed0b7bc

Browse files
JoramSochJoramSoch
authored
Merge pull request #135 from JoramSoch/master
added 3 definitions and 6 proofs
2 parents 0dca886 + c246248 commit ed0b7bc

10 files changed

Lines changed: 659 additions & 4 deletions

File tree

‎D/cfm.md‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
---
2+
layout: definition
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2021-10-21 17:01
9+
10+
title: "General linear model"
11+
chapter: "Statistical Models"
12+
section: "Multivariate normal data"
13+
topic: "Inverse general linear model"
14+
definition: "Corresponding forward model"
15+
16+
sources:
17+
- authors: "Haufe S, Meinecke F, Görgen K, Dähne S, Haynes JD, Blankertz B, Bießmann F"
18+
year: 2014
19+
title: "On the interpretation of weight vectors of linear models in multivariate neuroimaging"
20+
in: "NeuroImage"
21+
pages: "vol. 87, pp. 96–110, eq. 3"
22+
url: "https://www.sciencedirect.com/science/article/pii/S1053811913010914"
23+
doi: "10.1016/j.neuroimage.2013.10.067"
24+
25+
def_id: "D162"
26+
shortcut: "cfm"
27+
username: "JoramSoch"
28+
---
29+
30+
31+
**Definition:** Let there be observations $Y \in \mathbb{R}^{n \times v}$ and $X \in \mathbb{R}^{n \times p}$ and consider a weight matrix $W = f(Y,X) \in \mathbb{R}^{v \times p}$ estimated from $Y$ and $X$, such that right-multiplying $Y$ with the weight matrix gives an estimate or prediction of $X$:
32+
33+
$$ \label{eq:bda}
34+
\hat{X} = Y W \; .
35+
$$
36+
37+
Given that the columns of $\hat{X}$ are linearly independent, then
38+
39+
$$ \label{eq:cfm}
40+
Y = \hat{X} A^\mathrm{T} + E \quad \text{with} \quad \hat{X}^\mathrm{T} E = 0
41+
$$
42+
43+
is called the corresponding forward model relative to the weight matrix $W$.

‎D/iglm.md‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
layout: definition
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2021-10-21 15:31:00
9+
10+
title: "General linear model"
11+
chapter: "Statistical Models"
12+
section: "Multivariate normal data"
13+
topic: "Inverse general linear model"
14+
definition: "Definition"
15+
16+
sources:
17+
- authors: "Soch J, Allefeld C, Haynes JD"
18+
year: 2020
19+
title: "Inverse transformed encoding models – a solution to the problem of correlated trial-by-trial parameter estimates in fMRI decoding"
20+
in: "NeuroImage"
21+
pages: "vol. 209, art. 116449, Appendix C"
22+
url: "https://www.sciencedirect.com/science/article/pii/S1053811919310407"
23+
doi: "10.1016/j.neuroimage.2019.116449"
24+
25+
def_id: "D161"
26+
shortcut: "iglm"
27+
username: "JoramSoch"
28+
---
29+
30+
31+
**Definition:** Let there be a [general linear models](/D/glm) of measured data $Y \in \mathbb{R}^{n \times v}$ in terms of the [design matrix](/D/glm) $X \in \mathbb{R}^{n \times p}$:
32+
33+
$$ \label{eq:glm}
34+
Y = X B + E, \; E \sim \mathcal{MN}(0, V, \Sigma) \; .
35+
$$
36+
37+
Then, a [linear model](/D/glm) of $X$ in terms of $Y$, under the assumption of \eqref{eq:glm}, is called an inverse general linear model.

‎D/tglm.md‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
---
2+
layout: definition
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2021-10-21 14:43:00
9+
10+
title: "General linear model"
11+
chapter: "Statistical Models"
12+
section: "Multivariate normal data"
13+
topic: "Transformed general linear model"
14+
definition: "Definition"
15+
16+
sources:
17+
- authors: "Soch J, Allefeld C, Haynes JD"
18+
year: 2020
19+
title: "Inverse transformed encoding models – a solution to the problem of correlated trial-by-trial parameter estimates in fMRI decoding"
20+
in: "NeuroImage"
21+
pages: "vol. 209, art. 116449, Appendix A"
22+
url: "https://www.sciencedirect.com/science/article/pii/S1053811919310407"
23+
doi: "10.1016/j.neuroimage.2019.116449"
24+
25+
def_id: "D160"
26+
shortcut: "tglm"
27+
username: "JoramSoch"
28+
---
29+
30+
31+
**Definition:** Let there be two [general linear models](/D/glm) of measured data $Y \in \mathbb{R}^{n \times v}$ using [design matrices](/D/glm) $X \in \mathbb{R}^{n \times p}$ and $X_t \in \mathbb{R}^{n \times t}$
32+
33+
$$ \label{eq:glm1}
34+
Y = X B + E, \; E \sim \mathcal{MN}(0, V, \Sigma)
35+
$$
36+
37+
$$ \label{eq:glm2}
38+
Y = X_t \Gamma + E_t, \; E_t \sim \mathcal{MN}(0, V, \Sigma_t)
39+
$$
40+
41+
and assume that $X_t$ can be transformed into $X$ using a transformation matrix $T \in \mathbb{R}^{t \times p}$
42+
43+
$$ \label{eq:X-Xt-T}
44+
X = X_t \, T
45+
$$
46+
47+
where $p < t$ and $X$, $X_t$ and $T$ have full ranks $\mathrm{rk}(X) = p$, $\mathrm{rk}(X_t) = t$ and $\mathrm{rk}(T) = p$.
48+
49+
Then, a [linear model](/D/glm) of the parameter estimates from \eqref{eq:glm2}, under the assumption of \eqref{eq:glm1}, is called a transformed general linear model.

‎I/Table_of_Contents.md‎

Lines changed: 17 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -512,10 +512,23 @@ title: "Table of Contents"
512512
&emsp;&ensp; 2.1.3. **[Weighted least squares](/P/glm-wls)** <br>
513513
&emsp;&ensp; 2.1.4. **[Maximum likelihood estimation](/P/glm-mle)** <br>
514514

515-
2.2. Multivariate Bayesian linear regression <br>
516-
&emsp;&ensp; 2.2.1. **[Conjugate prior distribution](/P/mblr-prior)** <br>
517-
&emsp;&ensp; 2.2.2. **[Posterior distribution](/P/mblr-post)** <br>
518-
&emsp;&ensp; 2.2.3. **[Log model evidence](/P/mblr-lme)** <br>
515+
2.2. Transformed general linear model <br>
516+
&emsp;&ensp; 2.2.1. *[Definition](/D/tglm)* <br>
517+
&emsp;&ensp; 2.2.2. **[Derivation of the distribution](/P/tglm-dist)** <br>
518+
&emsp;&ensp; 2.2.3. **[Equivalence of parameter estimates](/P/tglm-para)** <br>
519+
520+
2.3. Inverse general linear model <br>
521+
&emsp;&ensp; 2.3.1. *[Definition](/D/iglm)* <br>
522+
&emsp;&ensp; 2.3.2. **[Derivation of the distribution](/P/iglm-dist)** <br>
523+
&emsp;&ensp; 2.3.3. **[Best linear unbiased estimator](/P/iglm-blue)** <br>
524+
&emsp;&ensp; 2.3.4. *[Corresponding forward model](/D/cfm)* <br>
525+
&emsp;&ensp; 2.3.5. **[Derivation of parameters](/P/cfm-para)** <br>
526+
&emsp;&ensp; 2.3.6. **[Proof of existence](/P/cfm-exist)** <br>
527+
528+
2.4. Multivariate Bayesian linear regression <br>
529+
&emsp;&ensp; 2.4.1. **[Conjugate prior distribution](/P/mblr-prior)** <br>
530+
&emsp;&ensp; 2.4.2. **[Posterior distribution](/P/mblr-post)** <br>
531+
&emsp;&ensp; 2.4.3. **[Log model evidence](/P/mblr-lme)** <br>
519532

520533
3. Poisson data
521534

‎P/cfm-exist.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2021-10-21 17:43:00
9+
10+
title: "Existence of the corresponding forward model"
11+
chapter: "Statistical Models"
12+
section: "Multivariate normal data"
13+
topic: "Inverse general linear model"
14+
theorem: "Proof of existence"
15+
16+
sources:
17+
- authors: "Haufe S, Meinecke F, Görgen K, Dähne S, Haynes JD, Blankertz B, Bießmann F"
18+
year: 2014
19+
title: "On the interpretation of weight vectors of linear models in multivariate neuroimaging"
20+
in: "NeuroImage"
21+
pages: "vol. 87, pp. 96–110, Appendix B"
22+
url: "https://www.sciencedirect.com/science/article/pii/S1053811913010914"
23+
doi: "10.1016/j.neuroimage.2013.10.067"
24+
25+
proof_id: "P270"
26+
shortcut: "cfm-exist"
27+
username: "JoramSoch"
28+
---
29+
30+
31+
**Theorem:** Let there be observations $Y \in \mathbb{R}^{n \times v}$ and $X \in \mathbb{R}^{n \times p}$ and consider a weight matrix $W \in \mathbb{R}^{v \times p}$ predicting $X$ from $Y$:
32+
33+
$$ \label{eq:bda}
34+
\hat{X} = Y W \; .
35+
$$
36+
37+
Then, there exists a [corresponding forward model](/D/cfm).
38+
39+
40+
**Proof:** The [corresponding forward model](/D/cfm) is defined as
41+
42+
$$ \label{eq:cfm}
43+
Y = \hat{X} A^\mathrm{T} + E \quad \text{with} \quad \hat{X}^\mathrm{T} E = 0
44+
$$
45+
46+
and the [parameters of the corresponding forward model](/P/cfm-para) are equal to
47+
48+
$$ \label{eq:cfm-para}
49+
A = \Sigma_y W \Sigma_x^{-1} \quad \text{where} \quad \Sigma_x = \hat{X}^\mathrm{T} \hat{X} \quad \text{and} \quad \Sigma_y = Y^\mathrm{T} Y \; .
50+
$$
51+
52+
<br>
53+
1) Because the columns of $\hat{X}$ are assumed to be linearly independent [by definition of the corresponding forward model](/D/cfm), the matrix $\Sigma_x = \hat{X}^\mathrm{T} \hat{X}$ is invertible, such that $A$ in \eqref{eq:cfm-para} is well-defined.
54+
55+
<br>
56+
2) Moreover, the solution for the matrix $A$ satisfies the [constraint of the corresponding forward model](/D/cfm) for predicted $X$ and errors $E$ to be uncorrelated which can be shown as follows:
57+
58+
$$ \label{eq:X-E-0}
59+
\begin{split}
60+
\hat{X}^\mathrm{T} E &\overset{\eqref{eq:cfm}}{=} \hat{X}^\mathrm{T} \left( Y - \hat{X} A^\mathrm{T} \right) \\
61+
&\overset{\eqref{eq:cfm-para}}{=} \hat{X}^\mathrm{T} \left( Y - \hat{X} \, \Sigma_x^{-1} W^\mathrm{T} \Sigma_y \right) \\
62+
&= \hat{X}^\mathrm{T} Y - \hat{X}^\mathrm{T} \hat{X} \, \Sigma_x^{-1} W^\mathrm{T} \Sigma_y \\
63+
&\overset{\eqref{eq:cfm-para}}{=} \hat{X}^\mathrm{T} Y - \hat{X}^\mathrm{T} \hat{X} \left( \hat{X}^\mathrm{T} \hat{X} \right)^{-1} W^\mathrm{T} \left( Y^\mathrm{T} Y \right) \\
64+
% &= \hat{X}^\mathrm{T} Y - W^\mathrm{T} \left( Y^\mathrm{T} Y \right) \\
65+
&\overset{\eqref{eq:bda}}{=} (Y W)^\mathrm{T} Y - W^\mathrm{T} \left( Y^\mathrm{T} Y \right) \\
66+
&= W^\mathrm{T} Y^\mathrm{T} Y - W^\mathrm{T} Y^\mathrm{T} Y \\
67+
&= 0 \; .
68+
\end{split}
69+
$$
70+
71+
This completes the proof.

‎P/cfm-para.md‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2021-10-21 17:20:00
9+
10+
title: "Parameters of the corresponding forward model"
11+
chapter: "Statistical Models"
12+
section: "Multivariate normal data"
13+
topic: "Inverse general linear model"
14+
theorem: "Derivation of parameters"
15+
16+
sources:
17+
- authors: "Haufe S, Meinecke F, Görgen K, Dähne S, Haynes JD, Blankertz B, Bießmann F"
18+
year: 2014
19+
title: "On the interpretation of weight vectors of linear models in multivariate neuroimaging"
20+
in: "NeuroImage"
21+
pages: "vol. 87, pp. 96–110, Theorem 1"
22+
url: "https://www.sciencedirect.com/science/article/pii/S1053811913010914"
23+
doi: "10.1016/j.neuroimage.2013.10.067"
24+
25+
proof_id: "P269"
26+
shortcut: "cfm-para"
27+
username: "JoramSoch"
28+
---
29+
30+
31+
**Theorem:** Let there be observations $Y \in \mathbb{R}^{n \times v}$ and $X \in \mathbb{R}^{n \times p}$ and consider a weight matrix $W \in \mathbb{R}^{v \times p}$ predicting $X$ from $Y$:
32+
33+
$$ \label{eq:bda}
34+
\hat{X} = Y W \; .
35+
$$
36+
37+
Then, the parameter matrix of the [corresponding forward model](/D/cfm) is equal to
38+
39+
$$ \label{eq:cfm-para}
40+
A = \Sigma_y W \Sigma_x^{-1}
41+
$$
42+
43+
with the [sample covariance](/D/cov-samp)
44+
45+
$$ \label{eq:Sx-Sy}
46+
\begin{split}
47+
\Sigma_x &= \hat{X}^\mathrm{T} \hat{X} \\
48+
\Sigma_y &= Y^\mathrm{T} Y \; .
49+
\end{split}
50+
$$
51+
52+
53+
**Proof:** The [corresponding forward model](/D/cfm) is given by
54+
55+
$$ \label{eq:cfm}
56+
Y = \hat{X} A^\mathrm{T} + E \; ,
57+
$$
58+
59+
subject to the constraint that predicted $X$ and errors $E$ are uncorrelated:
60+
61+
$$ \label{eq:cfm-con}
62+
\hat{X}^\mathrm{T} E = 0 \; .
63+
$$
64+
65+
With that, we can directly derive the parameter matrix $A$:
66+
67+
$$ \label{eq:cfm-para-qed}
68+
\begin{split}
69+
Y &\overset{\eqref{eq:cfm}}{=} \hat{X} A^\mathrm{T} + E \\
70+
\hat{X} A^\mathrm{T} &= Y - E \\
71+
\hat{X}^\mathrm{T} \hat{X} A^\mathrm{T} &= \hat{X}^\mathrm{T} (Y - E) \\
72+
\hat{X}^\mathrm{T} \hat{X} A^\mathrm{T} &= \hat{X}^\mathrm{T} Y - \hat{X}^\mathrm{T} E \\
73+
\hat{X}^\mathrm{T} \hat{X} A^\mathrm{T} &\overset{\eqref{eq:cfm-con}}{=} \hat{X}^\mathrm{T} Y \\
74+
\hat{X}^\mathrm{T} \hat{X} A^\mathrm{T} &\overset{\eqref{eq:bda}}{=} W^\mathrm{T} Y^\mathrm{T} Y \\
75+
\Sigma_x A^\mathrm{T} &\overset{\eqref{eq:Sx-Sy}}{=} W^\mathrm{T} \Sigma_y \\
76+
A^\mathrm{T} &= \Sigma_x^{-1} W^\mathrm{T} \Sigma_y \\
77+
A &= \Sigma_y W \Sigma_x^{-1} \; .
78+
\end{split}
79+
$$

0 commit comments

Comments
 (0)