You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Theorem:** The [covariance matrix](/D/covmat) $\Sigma_{XX}$ of a [random vector](/D/rvec) $X$ is invariant under addition of a [constant vector](/D/const) $a$:
31
+
32
+
$$ \label{eq:covmat-inv}
33
+
\Sigma(X+a) = \Sigma(X) \; .
34
+
$$
35
+
36
+
37
+
**Proof:** The [covariance matrix](/D/covmat) of $X$ [can be expressed](/P/covmat-mean) in terms of [expected values](/D/mean) as follows:
**Theorem:** The [covariance matrix](/D/covmat) $\Sigma_{XX}$ of a [random vector](/D/rvec) $X$ scales upon multiplication with a [constant matrix](/D/const) $A$:
31
+
32
+
$$ \label{eq:covmat-scal}
33
+
\Sigma(AX) = A \, \Sigma(X) A^\mathrm{T} \; .
34
+
$$
35
+
36
+
37
+
**Proof:** The [covariance matrix](/D/covmat) of $X$ [can be expressed](/P/covmat-mean) in terms of [expected values](/D/mean) as follows:
**Proof:** The [matrix-normal distribution is equivalent to the multivariate normal distribution](/P/matn-mvn),
38
+
39
+
$$ \label{eq:matn-mvn}
40
+
X \sim \mathcal{MN}(M, U, V) \quad \Leftrightarrow \quad \mathrm{vec}(X) \sim \mathcal{N}(\mathrm{vec}(M), V \otimes U) \; ,
41
+
$$
42
+
43
+
and the [differential entropy for the multivariate normal distribution](/P/mvn-dent) in nats is
44
+
45
+
$$ \label{eq:mvn-dent}
46
+
X \sim \mathcal{N}(\mu, \Sigma) \quad \Rightarrow \quad \mathrm{h}(X) = \frac{n}{2} \ln(2\pi) + \frac{1}{2} \ln|\Sigma| + \frac{1}{2} n
47
+
$$
48
+
49
+
where $X$ is an $n \times 1$ [random vector](/D/rvec).
50
+
51
+
Thus, we can plug the distribution parameters from \eqref{eq:matn} into the differential entropy in \eqref{eq:mvn-dent} using the relationship given by \eqref{eq:matn-mvn}
title: "Covariance and variance of the normal-gamma distribution"
11
+
chapter: "Probability Distributions"
12
+
section: "Multivariate continuous distributions"
13
+
topic: "Normal-gamma distribution"
14
+
theorem: "Covariance"
15
+
16
+
sources:
17
+
18
+
proof_id: "P345"
19
+
shortcut: "ng-cov"
20
+
username: "JoramSoch"
21
+
---
22
+
23
+
24
+
**Theorem:** Let $x \in \mathbb{R}^n$ and $y > 0$ follow a [normal-gamma distribution](/D/ng):
25
+
26
+
$$ \label{eq:ng}
27
+
x,y \sim \mathrm{NG}(\mu, \Lambda, a, b) \; .
28
+
$$
29
+
30
+
Then,
31
+
32
+
1) the [covariance](/D/cov) of $x$, [conditional](/D/dist-cond) on $y$ is
33
+
34
+
$$ \label{eq:ng-cov-cond}
35
+
\mathrm{Cov}[x|y] = \frac{1}{y} \Lambda^{-1} \; ;
36
+
$$
37
+
38
+
2) the [covariance](/D/cov) of $x$, [unconditional](/D/dist-marg) on $y$ is
39
+
40
+
$$ \label{eq:ng-cov-x}
41
+
\mathrm{Cov}[x] = \frac{b}{a-1} \Lambda^{-1} \; ;
42
+
$$
43
+
44
+
3) the [variance](/D/var) of $y$ is
45
+
46
+
$$ \label{eq:ng-var-y}
47
+
\mathrm{Var}[y] = \frac{a}{b^2} \; .
48
+
$$
49
+
50
+
51
+
**Proof:**
52
+
53
+
1) According to the [definition of the normal-gamma distribution](/D/ng), the distribution of $x$ given $y$ is a [multivariate normal distribution](/D/mvn):
54
+
55
+
$$ \label{eq:ng-mvn}
56
+
x \vert y \sim \mathcal{N}(\mu, (y \Lambda)^{-1}) \; .
57
+
$$
58
+
59
+
The [covariance of the multivariate normal distribution](/P/mvn-cov) is
0 commit comments