Skip to content

Commit b1701fd

Browse files
JoramSochJoramSoch
authored
added 5 proofs
1 parent 82c6723 commit b1701fd

5 files changed

Lines changed: 505 additions & 0 deletions

File tree

‎P/matn-cov.md‎

Lines changed: 75 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2022-09-15 12:23:00
9+
10+
title: "Covariance matrices of the matrix-normal distribution"
11+
chapter: "Probability Distributions"
12+
section: "Matrix-variate continuous distributions"
13+
topic: "Matrix-normal distribution"
14+
theorem: "Covariance"
15+
16+
sources:
17+
- authors: "Wikipedia"
18+
year: 2022
19+
title: "Matrix normal distribution"
20+
in: "Wikipedia, the free encyclopedia"
21+
pages: "retrieved on 2022-09-15"
22+
url: "https://en.wikipedia.org/wiki/Matrix_normal_distribution#Expected_values"
23+
24+
proof_id: "P342"
25+
shortcut: "matn-cov"
26+
username: "JoramSoch"
27+
---
28+
29+
30+
**Theorem:** Let $X$ be an $n \times p$ [random matrix](/D/rmat) following a [matrix-normal distribution](/D/matn):
31+
32+
$$ \label{eq:matn}
33+
X \sim \mathcal{MN}(M, U, V) \; .
34+
$$
35+
36+
Then,
37+
38+
1) the [covariance matrix](/D/covmat) of each row of $X$ is a scalar multiple of $V$
39+
40+
$$ \label{eq:matn-cov-row}
41+
\mathrm{Cov}(x_{i,\bullet}^\mathrm{T}) \propto V \quad \text{for all} \quad i = 1,\ldots,n \; ;
42+
$$
43+
44+
2) the [covariance matrix](/D/covmat) of each column of $X$ is a scalar multiple of $U$
45+
46+
$$ \label{eq:matn-cov-col}
47+
\mathrm{Cov}(x_{\bullet,j}) \propto U \quad \text{for all} \quad i = 1,\ldots,p \; .
48+
$$
49+
50+
51+
**Proof:**
52+
53+
1) The [marginal distribution](/D/dist-marg) of a given row of $X$ [is a multivariate normal distribution](/P/matn-marg)
54+
55+
$$ \label{eq:matn-marg-row}
56+
x_{i,\bullet}^\mathrm{T} \sim \mathcal{N}(m_{i,\bullet}^\mathrm{T}, u_{ii} V) \; ,
57+
$$
58+
59+
and [the covariance of this multivariate normal distribution](/P/mvn-cov) is
60+
61+
$$ \label{eq:matn-cov-row-qed}
62+
\mathrm{Cov}(x_{i,\bullet}^\mathrm{T}) = u_{ii} V \propto V \; .
63+
$$
64+
65+
2) The [marginal distribution](/D/dist-marg) of a given column of $X$ [is a multivariate normal distribution](/P/matn-marg)
66+
67+
$$ \label{eq:matn-marg-col}
68+
x_{\bullet,j} \sim \mathcal{N}(m_{\bullet,j}, v_{jj} U) \; ,
69+
$$
70+
71+
and [the covariance of this multivariate normal distribution](/P/mvn-cov) is
72+
73+
$$ \label{eq:matn-cov-col-qed}
74+
\mathrm{Cov}(x_{\bullet,j}) = v_{jj} U \propto U \; .
75+
$$

‎P/matn-marg.md‎

Lines changed: 170 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,170 @@
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2022-09-15 11:41:00
9+
10+
title: "Marginal distributions for the matrix-normal distribution"
11+
chapter: "Probability Distributions"
12+
section: "Matrix-variate continuous distributions"
13+
topic: "Matrix-normal distribution"
14+
theorem: "Marginal distributions"
15+
16+
sources:
17+
18+
proof_id: "P343"
19+
shortcut: "matn-marg"
20+
username: "JoramSoch"
21+
---
22+
23+
24+
**Theorem:** Let $X$ be an $n \times p$ [random matrix](/D/rmat) following a [matrix-normal distribution](/D/matn):
25+
26+
$$ \label{eq:matn}
27+
X \sim \mathcal{MN}(M, U, V) \; .
28+
$$
29+
30+
Then,
31+
32+
1) the [marginal distribution](/D/dist-marg) of any subset matrix $X_{I,J}$, obtained by dropping some rows and/or columns from $X$, is also a [matrix-normal distribution](/D/matn)
33+
34+
$$ \label{eq:matn-marg-subs}
35+
X_{I,J} \sim \mathcal{MN}(M_{I,J}, U_{I,I}, V_{J,J})
36+
$$
37+
38+
where $I \subseteq \left\lbrace 1, \ldots, n \right\rbrace$ is an (ordered) subset of all row indices and $J \subseteq \left\lbrace 1, \ldots, p \right\rbrace$ is an (ordered) subset of all column indices, such that $M_{I,J}$ is the matrix dropping the irrelevant rows and columns (the ones not in the subset, i.e. marginalized out) from the mean matrix $M$; $U_{I,I}$ is the matrix dropping rows not in $I$ from $U$; and $V_{J,J}$ is the matrix dropping columns not in $J$ from $V$;
39+
40+
2) the [marginal distribution](/D/dist-marg) of each row vector is a [multivariate normal distribution](/D/mvn)
41+
42+
$$ \label{eq:matn-marg-row}
43+
x_{i,\bullet}^\mathrm{T} \sim \mathcal{N}(m_{i,\bullet}^\mathrm{T}, u_{ii} V)
44+
$$
45+
46+
where $m_{i,\bullet}$ is the $i$-th row of $M$ and $u_{ii}$ is the $i$-th diagonal entry of $U$;
47+
48+
3) the [marginal distribution](/D/dist-marg) of each column vector is a [multivariate normal distribution](/D/mvn)
49+
50+
$$ \label{eq:matn-marg-col}
51+
x_{\bullet,j} \sim \mathcal{N}(m_{\bullet,j}, v_{jj} U)
52+
$$
53+
54+
where $m_{\bullet,j}$ is the $j$-th column of $M$ and $v_{jj}$ is the $j$-th diagonal entry of $V$; and
55+
56+
4) the [marginal distribution](/D/dist-marg) of one element of $X$ is a [univariate normal distribution](/D/norm)
57+
58+
$$ \label{eq:matn-marg-elem}
59+
x_{ij} \sim \mathcal{N}(m_{ij}, u_{ii} v_{jj})
60+
$$
61+
62+
where $m_{ij}$ is the $(i,j)$-th entry of $M$.
63+
64+
65+
**Proof:**
66+
67+
1) Define a selector matrix $A$, such that $a_{ij} = 1$, if the $i$-th row in the subset matrix should be the $j$-th row from the original matrix (and $a_{ij} = 0$ otherwise)
68+
69+
$$ \label{eq:A}
70+
A \in \mathbb{R}^{\lvert I \rvert \times n}, \quad \text{s.t.} \quad a_{ij} = \left\{
71+
\begin{array}{rl}
72+
0 \; , & \text{if} \; I_i = j \\
73+
1 \; , & \text{otherwise}
74+
\end{array}
75+
\right.
76+
$$
77+
78+
and define a selector matrix $B$, such that $b_{ij} = 1$, if the $j$-th column in the subset matrix should be the $i$-th column from the original matrix (and $b_{ij} = 0$ otherwise)
79+
80+
$$ \label{eq:B}
81+
B \in \mathbb{R}^{p \times \lvert J \rvert}, \quad \text{s.t.} \quad b_{ij} = \left\{
82+
\begin{array}{rl}
83+
0 \; , & \text{if} \; J_j = i \\
84+
1 \; , & \text{otherwise} \; .
85+
\end{array}
86+
\right.
87+
$$
88+
89+
Then, $X_{I,J}$ can be expressed as
90+
91+
$$ \label{eq:XIJ}
92+
X_{I,J} = A X B
93+
$$
94+
95+
and we can apply the [linear transformation theorem](/P/matn-ltt) to give
96+
97+
$$ \label{eq:XIJ-marg}
98+
X_{I,J} \sim \mathcal{MN}(A M B, A U A^\mathrm{T}, B^\mathrm{T} V B) \; .
99+
$$
100+
101+
Finally, we see that $A M B = M_{I,J}$, $A U A^\mathrm{T} = U_{I,I}$ and $B^\mathrm{T} V B = V_{J,J}$.
102+
103+
2) This is a special case of 1). Setting $A$ to the $i$-th elementary row vector in $n$ dimensions and $B$ to the $p \times p$ identity matrix
104+
105+
$$ \label{eq:AB-row}
106+
A = e_i, \; B = I_p \; ,
107+
$$
108+
109+
the $i$-th row of $X$ can be expressed as
110+
111+
$$ \label{eq:xi-marg}
112+
\begin{split}
113+
x_{i,\bullet} &= AXB = e_i X I_p = e_i X \\
114+
&\overset{\eqref{eq:XIJ-marg}}{\sim} \mathcal{MN}(m_{i,\bullet}, u_{ii}, V) \; .
115+
\end{split}
116+
$$
117+
118+
Thus, the [transpose of the row vector is distributed as](/P/matn-trans)
119+
120+
$$ \label{eq:xi-marg-trans}
121+
x_{i,\bullet}^\mathrm{T} \sim \mathcal{MN}(m_{i,\bullet}^\mathrm{T}, V, u_{ii})
122+
$$
123+
124+
which [is equivalent to a multivariate normal distribution](/P/matn-mvn):
125+
126+
$$ \label{eq:xi-marg-trans-mvn}
127+
x_{i,\bullet}^\mathrm{T} \sim \mathcal{N}(m_{i,\bullet}^\mathrm{T}, u_{ii} V) \; .
128+
$$
129+
130+
3) This is a special case of 1). Setting $A$ to the $n \times n$ identity matrix and $B$ to the $j$-th elementary row vector in $p$ dimensions
131+
132+
$$ \label{eq:AB-col}
133+
A = I_n, \; B = e_j^\mathrm{T} \; ,
134+
$$
135+
136+
the $j$-th column of $X$ can be expressed as
137+
138+
$$ \label{eq:xj-marg}
139+
\begin{split}
140+
x_{\bullet,j} &= AXB = I_n X e_j^\mathrm{T} = X e_j^\mathrm{T} \\
141+
&\overset{\eqref{eq:XIJ-marg}}{\sim} \mathcal{MN}(m_{\bullet,j}, U, v_{jj})
142+
\end{split}
143+
$$
144+
145+
which [is equivalent to a multivariate normal distribution](/P/matn-mvn):
146+
147+
$$ \label{eq:xj-marg-mvn}
148+
x_{\bullet,j} \sim \mathcal{N}(m_{\bullet,j}, v_{jj} U) \; .
149+
$$
150+
151+
4) This is a special case of 1). Setting $A$ to the $i$-th elementary row vector in $n$ dimensions and $B$ to the $j$-th elementary row vector in $p$ dimensions
152+
153+
$$ \label{eq:AB-elem}
154+
A = e_i, \; B = e_j^\mathrm{T} \; ,
155+
$$
156+
157+
the $(i,j)$-th entry of $X$ can be expressed as
158+
159+
$$ \label{eq:xij-marg}
160+
\begin{split}
161+
x_{ij} &= AXB = e_i X e_j^\mathrm{T} \\
162+
&\overset{\eqref{eq:XIJ-marg}}{\sim} \mathcal{MN}(m_{ij}, u_{ii}, v_{jj}) \; .
163+
\end{split}
164+
$$
165+
166+
As $x_{ij}$ is a scalar, this is equivalent to [a univariate normal distribution](/D/norm) as [a special case of](/P/norm-mvn) of [the matrix-normal distribution](/P/mvn-matn):
167+
168+
$$ \label{eq:xij-marg-norm}
169+
x_{ij} \sim \mathcal{N}(m_{ij}, u_{ii} v_{jj}) \; .
170+
$$

‎P/matn-mean.md‎

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2022-09-15 12:05:00
9+
10+
title: "Mean of the matrix-normal distribution"
11+
chapter: "Probability Distributions"
12+
section: "Matrix-variate continuous distributions"
13+
topic: "Matrix-normal distribution"
14+
theorem: "Mean"
15+
16+
sources:
17+
- authors: "Wikipedia"
18+
year: 2022
19+
title: "Matrix normal distribution"
20+
in: "Wikipedia, the free encyclopedia"
21+
pages: "retrieved on 2022-09-15"
22+
url: "https://en.wikipedia.org/wiki/Matrix_normal_distribution#Expected_values"
23+
24+
proof_id: "P341"
25+
shortcut: "matn-mean"
26+
username: "JoramSoch"
27+
---
28+
29+
30+
**Theorem:** Let $X$ be a [random matrix](/D/rmat) following a [matrix-normal distribution](/D/matn):
31+
32+
$$ \label{eq:matn}
33+
X \sim \mathcal{MN}(M, U, V) \; .
34+
$$
35+
36+
Then, the [mean or expected value](/D/mean) of $X$ is
37+
38+
$$ \label{eq:matn-mean}
39+
\mathrm{E}(X) = M \; .
40+
$$
41+
42+
43+
**Proof:** When $X$ follows a [matrix-normal distribution](/D/matn), [its vectorized version follows a multivariate normal distribution](/P/matn-mvn)
44+
45+
$$ \label{eq:matn-mvn}
46+
\mathrm{vec}(X) \sim \mathcal{N}(\mathrm{vec}(M), V \otimes U)
47+
$$
48+
49+
and [the expected value of this multivariate normal distribution is](/P/mvn-mean)
50+
51+
$$ \label{eq:mvn-mean}
52+
\mathrm{E}[\mathrm{vec}(X)] = \mathrm{vec}(M) \; .
53+
$$
54+
55+
Since [the expected value of a random matrix is calculated element-wise](/D/mean-rmat), we can invert the vectorization operator to get:
56+
57+
$$ \label{eq:matn-mean-qed}
58+
\mathrm{E}[X] = M \; .
59+
$$

0 commit comments

Comments
 (0)