Merge pull request #94 from StatProofBook/master

JoramSoch · web-flow · commit 7d9e1d73e9f9 · 2022-09-29T15:58:24.000+02:00
update to master
diff --git a/I/PbA.md b/I/PbA.md
@@ -8,7 +8,7 @@ title: "Proof by Author"
 
 - [Covariance matrix of the multinomial distribution](/P/mult-cov)
 
-### JoramSoch (326 proofs)
+### JoramSoch (331 proofs)
 
 - [Accuracy and complexity for the univariate Gaussian](/P/ug-anc)
 - [Accuracy and complexity for the univariate Gaussian with known variance](/P/ugkv-anc)
@@ -44,6 +44,7 @@ title: "Proof by Author"
 - [Corrected Akaike information criterion in terms of maximum log-likelihood](/P/aicc-mll)
 - [Correlation always falls between -1 and +1](/P/corr-range)
 - [Correlation coefficient in terms of standard scores](/P/corr-z)
+- [Covariance and variance of the normal-gamma distribution](/P/ng-cov)
 - [Covariance matrices of the matrix-normal distribution](/P/matn-cov)
 - [Covariance matrix of the categorical distribution](/P/cat-cov)
 - [Covariance matrix of the multivariate normal distribution](/P/mvn-cov)
@@ -71,6 +72,7 @@ title: "Proof by Author"
 - [Deviance for multiple linear regression](/P/mlr-dev)
 - [Deviance information criterion for multiple linear regression](/P/blr-dic)
 - [Differential entropy can be negative](/P/dent-neg)
+- [Differential entropy for the matrix-normal distribution](/P/matn-dent)
 - [Differential entropy of the gamma distribution](/P/gam-dent)
 - [Differential entropy of the multivariate normal distribution](/P/mvn-dent)
 - [Differential entropy of the normal distribution](/P/norm-dent)
@@ -105,6 +107,7 @@ title: "Proof by Author"
 - [Gaussian integral](/P/norm-gi)
 - [Gibbs' inequality](/P/gibbs-ineq)
 - [Inflection points of the probability density function of the normal distribution](/P/norm-infl)
+- [Invariance of the covariance matrix under addition of constant vector](/P/covmat-inv)
 - [Invariance of the differential entropy under addition of a constant](/P/dent-inv)
 - [Invariance of the Kullback-Leibler divergence under parameter transformation](/P/kl-inv)
 - [Invariance of the variance under addition of a constant](/P/var-inv)
@@ -307,6 +310,8 @@ title: "Proof by Author"
 - [Relationship between second raw moment, variance and mean](/P/momraw-2nd)
 - [Relationship between signal-to-noise ratio and R²](/P/snr-rsq)
 - [Sampling from the matrix-normal distribution](/P/matn-samp)
+- [Sampling from the normal-gamma distribution](/P/ng-samp)
+- [Scaling of the covariance matrix upon multiplication with constant matrix](/P/covmat-scal)
 - [Scaling of the variance upon multiplication with a constant](/P/var-scal)
 - [Second central moment is variance](/P/momcent-2nd)
 - [Simple linear regression is a special case of multiple linear regression](/P/slr-mlr)
diff --git a/I/PbN.md b/I/PbN.md
@@ -349,3 +349,8 @@ title: "Proof by Number"
 | P341 | matn-mean | [Mean of the matrix-normal distribution](/P/matn-mean) | JoramSoch | 2022-09-15 |
 | P342 | matn-cov | [Covariance matrices of the matrix-normal distribution](/P/matn-cov) | JoramSoch | 2022-09-15 |
 | P343 | matn-marg | [Marginal distributions for the matrix-normal distribution](/P/matn-marg) | JoramSoch | 2022-09-15 |
+| P344 | matn-dent | [Differential entropy for the matrix-normal distribution](/P/matn-dent) | JoramSoch | 2022-09-22 |
+| P345 | ng-cov | [Covariance and variance of the normal-gamma distribution](/P/ng-cov) | JoramSoch | 2022-09-22 |
+| P346 | ng-samp | [Sampling from the normal-gamma distribution](/P/ng-samp) | JoramSoch | 2022-09-22 |
+| P347 | covmat-inv | [Invariance of the covariance matrix under addition of constant vector](/P/covmat-inv) | JoramSoch | 2022-09-22 |
+| P348 | covmat-scal | [Scaling of the covariance matrix upon multiplication with constant matrix](/P/covmat-scal) | JoramSoch | 2022-09-22 |
diff --git a/I/PbT.md b/I/PbT.md
@@ -47,6 +47,7 @@ title: "Proof by Topic"
 - [Corrected Akaike information criterion in terms of maximum log-likelihood](/P/aicc-mll)
 - [Correlation always falls between -1 and +1](/P/corr-range)
 - [Correlation coefficient in terms of standard scores](/P/corr-z)
+- [Covariance and variance of the normal-gamma distribution](/P/ng-cov)
 - [Covariance matrices of the matrix-normal distribution](/P/matn-cov)
 - [Covariance matrix of the categorical distribution](/P/cat-cov)
 - [Covariance matrix of the multinomial distribution](/P/mult-cov)
@@ -79,6 +80,7 @@ title: "Proof by Topic"
 - [Deviance for multiple linear regression](/P/mlr-dev)
 - [Deviance information criterion for multiple linear regression](/P/blr-dic)
 - [Differential entropy can be negative](/P/dent-neg)
+- [Differential entropy for the matrix-normal distribution](/P/matn-dent)
 - [Differential entropy of the gamma distribution](/P/gam-dent)
 - [Differential entropy of the multivariate normal distribution](/P/mvn-dent)
 - [Differential entropy of the normal distribution](/P/norm-dent)
@@ -126,6 +128,7 @@ title: "Proof by Topic"
 ### I
 
 - [Inflection points of the probability density function of the normal distribution](/P/norm-infl)
+- [Invariance of the covariance matrix under addition of constant vector](/P/covmat-inv)
 - [Invariance of the differential entropy under addition of a constant](/P/dent-inv)
 - [Invariance of the Kullback-Leibler divergence under parameter transformation](/P/kl-inv)
 - [Invariance of the variance under addition of a constant](/P/var-inv)
@@ -368,7 +371,9 @@ title: "Proof by Topic"
 ### S
 
 - [Sampling from the matrix-normal distribution](/P/matn-samp)
+- [Sampling from the normal-gamma distribution](/P/ng-samp)
 - [Savage-Dickey Density Ratio for computing Bayes Factors](/P/bf-sddr)
+- [Scaling of the covariance matrix upon multiplication with constant matrix](/P/covmat-scal)
 - [Scaling of the variance upon multiplication with a constant](/P/var-scal)
 - [Second central moment is variance](/P/momcent-2nd)
 - [Simple linear regression is a special case of multiple linear regression](/P/slr-mlr)
diff --git a/I/PwS.md b/I/PwS.md
@@ -9,6 +9,7 @@ title: "Proofs without Source"
 - [Bayesian information criterion for multiple linear regression](/P/mlr-bic)
 - [Chi-squared distribution is a special case of gamma distribution](/P/chi2-gam)
 - [Conditional distributions of the normal-gamma distribution](/P/ng-cond)
+- [Covariance and variance of the normal-gamma distribution](/P/ng-cov)
 - [Covariance matrix of the categorical distribution](/P/cat-cov)
 - [Cross-validated log Bayes factor for the univariate Gaussian with known variance](/P/ugkv-cvlbf)
 - [Cross-validated log model evidence for the univariate Gaussian with known variance](/P/ugkv-cvlme)
@@ -24,6 +25,7 @@ title: "Proofs without Source"
 - [Derivation of the posterior model probability](/P/pmp-der)
 - [Deviance for multiple linear regression](/P/mlr-dev)
 - [Deviance information criterion for multiple linear regression](/P/blr-dic)
+- [Differential entropy for the matrix-normal distribution](/P/matn-dent)
 - [Differential entropy of the normal-gamma distribution](/P/ng-dent)
 - [Effects of mean-centering on parameter estimates for simple linear regression](/P/slr-meancent)
 - [Entropy of the binomial distribution](/P/bin-ent)
diff --git a/P/matn-marg.md b/P/matn-marg.md
@@ -148,7 +148,7 @@ $$ \label{eq:xj-marg-mvn}
 x_{\bullet,j} \sim \mathcal{N}(m_{\bullet,j}, v_{jj} U) \; .
 $$
 
-4) This is a special case of 1). Setting $A$ to the $i$-th elementary row vector in $n$ dimensions and $B$ to the $j$-th elementary row vector in $p$ dimensions
+4) This is a special case of 2) and 3). Setting $A$ to the $i$-th elementary row vector in $n$ dimensions and $B$ to the $j$-th elementary row vector in $p$ dimensions
 
 $$ \label{eq:AB-elem}
 A = e_i, \; B = e_j^\mathrm{T} \; ,
diff --git a/P/matn-samp.md b/P/matn-samp.md
@@ -27,7 +27,9 @@ username: "JoramSoch"
 ---
 
 
-**Theorem:** Let $X \in \mathbb{R}^{n \times p}$ be a [random matrix](/D/rmat) with all entries independently following a [standard normal distribution](/D/snorm). Moreover, let $A \in \mathbb{R}^{n \times n}$ and $B \in \mathbb{R}^{p \times p}$, such that $A A^\mathrm{T} = U$ and $B^\mathrm{T} B = V$. Then, $Y = M + A X B$ follows a [matrix-normal distribution](/D/matn) with [mean](/D/mean-rmat) $M$, [covariance](/D/covmat) across rows $U$ and [covariance](/D/covmat) across rows $U$:
+**Theorem:** Let $X \in \mathbb{R}^{n \times p}$ be a [random matrix](/D/rmat) with all entries independently following a [standard normal distribution](/D/snorm). Moreover, let $A \in \mathbb{R}^{n \times n}$ and $B \in \mathbb{R}^{p \times p}$, such that $A A^\mathrm{T} = U$ and $B^\mathrm{T} B = V$.
+
+Then, $Y = M + A X B$ follows a [matrix-normal distribution](/D/matn) with [mean](/D/mean-rmat) $M$, [covariance](/D/covmat) across rows $U$ and [covariance](/D/covmat) across rows $U$:
 
 $$ \label{eq:matn-samp}
 Y = M + A X B \sim \mathcal{MN}(M, U, V) \; .
@@ -45,7 +47,7 @@ this [implies a multivariate normal distribution with diagonal covariance matrix
 $$ \label{eq:vecX-dist}
 \begin{split}
 \mathrm{vec}(X) &\sim \mathcal{N}\left(\mathrm{vec}(0_{np}), I_{np} \right) \\
-&\sim \mathcal{N}\left(\mathrm{vec}(0_{np}), I_p \otimes I_n \right) \; .
+&\sim \mathcal{N}\left(\mathrm{vec}(0_{np}), I_p \otimes I_n \right)
 \end{split}
 $$
 
@@ -67,4 +69,4 @@ Y = M + AXB &\sim \mathcal{MN}\left(M + A 0_{np} B, A I_n A^\mathrm{T}, B^\mathr
 \end{split}
 $$
 
-Thus, given $X$ defined by \eqref{eq:xij-dist}, $Y$ defined by \eqref{eq:matn-samp} is a [sample](/D/samp) from $\mathcal{N}\left(M, U, V \right)$.
+Thus, given $X$ defined by \eqref{eq:xij-dist}, $Y$ defined by \eqref{eq:matn-samp} is a [sample](/D/samp) from $\mathcal{MN}\left(M, U, V \right)$.
diff --git a/P/mvn-cov.md b/P/mvn-cov.md
@@ -84,13 +84,13 @@ $$ \label{eq:x-mean}
 \mathrm{Cov}(x) = \mathrm{Cov}( Az + \mu ) \; .
 $$
 
-With the [invariance of the covariance matrix under addition](/P/cov-inv)
+With the [invariance of the covariance matrix under addition](/P/covmat-inv)
 
 $$ \label{eq:cov-inv}
 \mathrm{Cov}(x + a) = \mathrm{Cov}(x)
 $$
 
-and the [scaling of the covariance matrix upon multiplication](/P/cov-scal)
+and the [scaling of the covariance matrix upon multiplication](/P/covmat-scal)
 
 $$ \label{eq:cov-scal}
 \mathrm{Cov}(Ax) = A \mathrm{Cov}(x) A^\mathrm{T} \; ,
diff --git a/P/ng-cov.md b/P/ng-cov.md
@@ -32,19 +32,19 @@ Then,
 1) the [covariance](/D/cov) of $x$, [conditional](/D/dist-cond) on $y$ is
 
 $$ \label{eq:ng-cov-cond}
-\mathrm{Cov}[x|y] = \frac{1}{y} \Lambda^{-1} \; ;
+\mathrm{Cov}(x \vert y) = \frac{1}{y} \Lambda^{-1} \; ;
 $$
 
 2) the [covariance](/D/cov) of $x$, [unconditional](/D/dist-marg) on $y$ is
 
 $$ \label{eq:ng-cov-x}
-\mathrm{Cov}[x] = \frac{b}{a-1} \Lambda^{-1} \; ;
+\mathrm{Cov}(x) = \frac{b}{a-1} \Lambda^{-1} \; ;
 $$
 
 3) the [variance](/D/var) of $y$ is
 
 $$ \label{eq:ng-var-y}
-\mathrm{Var}[y] = \frac{a}{b^2} \; .
+\mathrm{Var}(y) = \frac{a}{b^2} \; .
 $$
 
 
@@ -65,7 +65,7 @@ $$
 such that we have:
 
 $$ \label{eq:ng-cov-cond-qed}
-\mathrm{Cov}[x|y] = (y \Lambda)^{-1} = \frac{1}{y} \Lambda^{-1} \; .
+\mathrm{Cov}(x \vert y) = (y \Lambda)^{-1} = \frac{1}{y} \Lambda^{-1} \; .
 $$
 
 2) The [marginal distribution of the normal-gamma distribution](/P/ng-marg) with respect to $x$ is a [multivariate t-distribution](/D/mvt):
@@ -83,7 +83,7 @@ $$
 such that we have:
 
 $$ \label{eq:ng-cov-x-qed}
-\mathrm{Cov}[x] = \frac{2a}{2a-2} \left(\frac{a}{b} \Lambda \right)^{-1} = \frac{a}{a-1} \, \frac{b}{a} \, \Lambda^{-1} = \frac{b}{a-1} \Lambda^{-1} \; .
+\mathrm{Cov}(x) = \frac{2a}{2a-2} \left(\frac{a}{b} \Lambda \right)^{-1} = \frac{a}{a-1} \, \frac{b}{a} \, \Lambda^{-1} = \frac{b}{a-1} \Lambda^{-1} \; .
 $$
 
 3) The [marginal distribution of the normal-gamma distribution](/P/ng-marg) with respect to $y$ is a [univariate gamma distribution](/D/gam):
@@ -101,5 +101,5 @@ $$
 such that we have:
 
 $$ \label{eq:ng-var-y-qed}
-\mathrm{Var}[y] = \frac{a}{b^2} \; .
+\mathrm{Var}(y) = \frac{a}{b^2} \; .
 $$
diff --git a/P/ng-samp.md b/P/ng-samp.md
@@ -27,7 +27,7 @@ username: "JoramSoch"
 ---
 
 
-**Theorem:** Let $Z_1 \in \mathbb{R}^{n \times 1}$ be a [random vector](/D/rvec) with all entries independently following a [standard normal distribution](/D/snorm) and let $Z_2 \in \mathbb{R}$ be a [random variable](/D/rvar) following a [standard gamma distribution](/D/sgam) with shape $a$. Moreover, let $A \in \mathbb{R}^{n \times n}$ be a matrix such that, such that $A A^\mathrm{T} = \Lambda^{-1}$.
+**Theorem:** Let $Z_1 \in \mathbb{R}^n$ be a [random vector](/D/rvec) with all entries independently following a [standard normal distribution](/D/snorm) and let $Z_2 \in \mathbb{R}$ be a [random variable](/D/rvar) following a [standard gamma distribution](/D/sgam) with shape $a$. Moreover, let $A \in \mathbb{R}^{n \times n}$ be a matrix, such that $A A^\mathrm{T} = \Lambda^{-1}$.
 
 Then, $X = \mu + A Z_1 / \sqrt{Z_2/b}$ and $Y = Z_2/b$ jointly follow a [normal-gamma distribution](/D/ng) with [mean vector](/D/mean-rvec) $\mu$, [precision matrix](/D/precmat) $\Lambda$, shape parameter $a$ and rate parameter $b$:
 
