corrected some pages

Several small mistakes/errors were corrected in several proofs/definitions.

Author: JoramSoch

D/mult.md

@@ -33,4 +33,4 @@ $$ \label{eq:mult}
 X \sim \mathrm{Mult}(n, \left[p_1, \ldots, p_k \right]) \; ,
 $$
 
-if $X$ are the numbers of observations belonging to $k$ distinct categories in $n$ [independent](/D/ind) trials, where each trial has [$k$ possible outcomes](/D/cat) and the category probabilities are identical across trials.
+if $X$ are the numbers of observations belonging to $k$ distinct categories in $n$ [independent](/D/ind) trials, where each trial has $k$ [possible outcomes](/D/cat) and the category probabilities are identical across trials.

P/kl-ent.md

@@ -7,7 +7,7 @@ affiliation: "BCCN Berlin"
 e_mail: "joram.soch@bccn-berlin.de"
 date: 2020-05-27 23:20:00
 
-title: "Relation of Kullback-Leibler divergence to entropy"
+title: "Relation of discrete Kullback-Leibler divergence to Shannon entropy"
 chapter: "General Theorems"
 section: "Information theory"
 topic: "Kullback-Leibler divergence"

P/matn-marg.md

@@ -69,8 +69,8 @@ where $m_{ij}$ is the $(i,j)$-th entry of $M$.
 $$ \label{eq:A}
 A \in \mathbb{R}^{\lvert I \rvert \times n}, \quad \text{s.t.} \quad a_{ij} = \left\{
 \begin{array}{rl}
-0 \; , & \text{if} \; I_i = j \\
-1 \; , & \text{otherwise}
+1 \; , & \text{if} \; I_i = j \\
+0 \; , & \text{otherwise}
 \end{array}
 \right.
 $$
@@ -80,8 +80,8 @@ and define a selector matrix $B$, such that $b_{ij} = 1$, if the $j$-th column i
 $$ \label{eq:B}
 B \in \mathbb{R}^{p \times \lvert J \rvert}, \quad \text{s.t.} \quad b_{ij} = \left\{
 \begin{array}{rl}
-0 \; , & \text{if} \; J_j = i \\
-1 \; , & \text{otherwise} \; .
+1 \; , & \text{if} \; J_j = i \\
+0 \; , & \text{otherwise} \; .
 \end{array}
 \right.
 $$

P/matn-samp.md

@@ -37,7 +37,7 @@ $$
 **Proof:** If all entries of $X$ are independent and [standard normally distributed](/D/snorm)
 
 $$ \label{eq:xij-dist}
-x_{ij} \sim \mathcal{N}(0, 1) \quad \text{ind. for all} \quad i = 1,\ldots,n \quad \text{and} \quad j = 1,\ldots,p \; ,
+x_{ij} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0, 1) \quad \text{for all} \quad i = 1,\ldots,n \quad \text{and} \quad j = 1,\ldots,p \; ,
 $$
 
 this [implies a multivariate normal distribution with diagonal covariance matrix](/P/mvn-ind):

P/mult-cov.md

@@ -46,13 +46,13 @@ $$
 which [has the variance](/P/bin-var) $\mathrm{Var}(X_i) = n p_i(1-p_i) = n (p_i - p_i^2)$, constituting the elements of the main diagonal in $\mathrm{Cov}(X)$ in \eqref{eq:mult-cov}. To prove $\mathrm{Cov}(X_i, X_j) = -n p_i p_j$ for $i \ne j$ (which constitutes the off-diagonal elements of the covariance matrix), we first recognize that
 
 $$ \label{eq:bin-sum}
-X_i = \sum_{k=1}^n \mathbb{I}_i(k), \quad \text{with} \quad \mathbb{I}_i(k) := \begin{cases}
+X_i = \sum_{k=1}^n \mathbb{I}_i(k), \quad \text{with} \quad \mathbb{I}_i(k) = \begin{cases}
     1 & \text{if $k$-th draw was of category $i$}, \\
-    0 & \text{otherwise},
+    0 & \text{otherwise} \; ,
 \end{cases}
 $$
 
-the indicator function $\mathbb{I}_i$ being a [Bernoulli-distributed](/D/bern) random variable with the [expected value](/P/bern-mean) $p_i$. Then, we have 
+where the indicator function $\mathbb{I}_i$ is a [Bernoulli-distributed](/D/bern) random variable with the [expected value](/P/bern-mean) $p_i$. Then, we have 
 
 $$ \label{eq:mult-cov-qed}
 \begin{split}

P/mvn-cov.md

@@ -102,7 +102,7 @@ $$ \label{eq:mvn-cov-qed}
 \begin{split}
 \mathrm{Cov}(x) &= \mathrm{Cov}( Az + \mu ) \\
 &\overset{\eqref{eq:cov-inv}}{=} \mathrm{Cov}(Az) \\
-&\overset{\eqref{eq:cov-scal}}{=} A \mathrm{Cov}(z) A^\mathrm{T} \\
+&\overset{\eqref{eq:cov-scal}}{=} A \, \mathrm{Cov}(z) A^\mathrm{T} \\
 &\overset{\eqref{eq:z-cov}}{=} A I_n A^\mathrm{T} \\
 &= A A^\mathrm{T} \\
 &= \Sigma \; .

P/mvn-mean.md

@@ -84,8 +84,8 @@ $$ \label{eq:mvn-mean-qed}
 \begin{split}
 \mathrm{E}(x) &= \mathrm{E}( Az + \mu ) \\
 &= \mathrm{E}(Az) + \mathrm{E}(\mu) \\
-&= A \mathrm{E}(z) + \mu \\
-&\overset{\eqref{eq:z-mean}}{=} A 0_n + \mu \\
+&= A \, \mathrm{E}(z) + \mu \\
+&\overset{\eqref{eq:z-mean}}{=} A \, 0_n + \mu \\
 &= \mu \; .
 \end{split}
 $$

P/ug-ttest1.md

@@ -64,13 +64,13 @@ $$ \label{eq:var-samp}
 s^2 = \frac{1}{n-1} \sum_{i=1}^{n} (y_i - \bar{y})^2 \; .
 $$
 
-Using the [linearity of the expected value](/P/mean-lin), the [additivity of the variance under independence](/P/var-add) and [scaling of the variance upon multiplication](/P/var-scal), the sample mean follows a [normal distribution](/D/norm)
+Using the [linear combination formula for normal random variables](/P/norm-lincomb), the sample mean follows a [normal distribution](/D/norm) with the following parameters:
 
 $$ \label{eq:mean-samp-dist}
-\bar{y} = \frac{1}{n} \sum_{i=1}^{n} y_i \sim \mathcal{N}\left( \frac{1}{n} n \mu, \left(\frac{1}{n}\right)^2 n \sigma^2 \right) = \mathcal{N}\left( \mu, \sigma^2/n \right)
+\bar{y} = \frac{1}{n} \sum_{i=1}^{n} y_i \sim \mathcal{N}\left( \frac{1}{n} n \mu, \left(\frac{1}{n}\right)^2 n \sigma^2 \right) = \mathcal{N}\left( \mu, \sigma^2/n \right) \; .
 $$
 
-and additionally using the [invariance of the variance under addition](/P/var-inv) and applying the null hypothesis from \eqref{eq:ttest1-h0}, the distribution of $Z = \sqrt{n}(\bar{y}-\mu_0)/\sigma$ becomes [standard normal](/D/snorm)
+Again employing the linear combination theorem and applying the null hypothesis from \eqref{eq:ttest1-h0}, the distribution of $Z = \sqrt{n}(\bar{y}-\mu_0)/\sigma$ becomes [standard normal](/D/snorm)
 
 $$ \label{eq:Z-dist}
 Z = \frac{\sqrt{n}(\bar{y}-\mu_0)}{\sigma} \sim \mathcal{N}\left( \frac{\sqrt{n}}{\sigma} (\mu - \mu_0), \left(\frac{\sqrt{n}}{\sigma}\right)^2 \frac{\sigma^2}{n} \right) \overset{H_0}{=} \mathcal{N}\left( 0, 1 \right) \; .

P/ug-ttest2.md

@@ -42,7 +42,7 @@ y_{2i} &\sim \mathcal{N}(\mu_2, \sigma^2), \quad i = 1, \ldots, n_2
 \end{split}
 $$
 
-be a [univariate Gaussian data set](/D/ug) representing two groups of unequal size $n_1$ and $n_2$ with unknown means $\mu_1$ and $\mu_2$ and equal unknown variance $\sigma^2$. Then, the [test statistic](/D/tstat)
+be two [univariate Gaussian data sets](/D/ug) representing two groups of unequal size $n_1$ and $n_2$ with unknown means $\mu_1$ and $\mu_2$ and equal unknown variance $\sigma^2$. Then, the [test statistic](/D/tstat)
 
 $$ \label{eq:t}
 t = \frac{(\bar{y}_1-\bar{y}_2)-\mu_\Delta}{s_p \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}
@@ -85,16 +85,16 @@ s^2_2 &= \frac{1}{n_2-1} \sum_{i=1}^{n_2} (y_{2i} - \bar{y}_2)^2 \; .
 \end{split}
 $$
 
-Using the [linearity of the expected value](/P/mean-lin), the [additivity of the variance under independence](/P/var-add) and [scaling of the variance upon multiplication](/P/var-scal), the sample means follow a [normal distribution](/D/norm)
+Using the [linear combination formula for normal random variables](/P/norm-lincomb), the sample means follows [normal distributions](/D/norm) with the following parameters:
 
 $$ \label{eq:mean-samp-dist}
 \begin{split}
 \bar{y}_1 &= \frac{1}{n_1} \sum_{i=1}^{n_1} y_{1i} \sim \mathcal{N}\left( \frac{1}{n_1} n_1 \mu_1, \left(\frac{1}{n_1}\right)^2 n_1 \sigma^2 \right) = \mathcal{N}\left( \mu_1, \sigma^2/n_1 \right) \\
-\bar{y}_2 &= \frac{1}{n_2} \sum_{i=1}^{n_2} y_{2i} \sim \mathcal{N}\left( \frac{1}{n_2} n_2 \mu_2, \left(\frac{1}{n_2}\right)^2 n_2 \sigma^2 \right) = \mathcal{N}\left( \mu_2, \sigma^2/n_2 \right)
+\bar{y}_2 &= \frac{1}{n_2} \sum_{i=1}^{n_2} y_{2i} \sim \mathcal{N}\left( \frac{1}{n_2} n_2 \mu_2, \left(\frac{1}{n_2}\right)^2 n_2 \sigma^2 \right) = \mathcal{N}\left( \mu_2, \sigma^2/n_2 \right) \; .
 \end{split}
 $$
 
-and additionally using the [invariance of the variance under addition](/P/var-inv) and applying the null hypothesis from \eqref{eq:ttest2-h0}, the distribution of $Z = ( ( \bar{y}_1 - \bar{y}_2 ) - \mu_{\Delta} ) / ( \sigma \sqrt{1/n_1+1/n_2} )$ becomes [standard normal](/D/snorm)
+Again employing the linear combination theorem and applying the null hypothesis from \eqref{eq:ttest2-h0}, the distribution of $Z = ( ( \bar{y}_1 - \bar{y}_2 ) - \mu_{\Delta} ) / ( \sigma \sqrt{1/n_1+1/n_2} )$ becomes [standard normal](/D/snorm)
 
 $$ \label{eq:Z-dist}
 Z = \frac{(\bar{y}_1-\bar{y}_2)-\mu_\Delta}{\sigma \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}} \sim \mathcal{N}\left( \frac{(\mu_1-\mu_2)-\mu_\Delta}{\sigma \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}, \left(\frac{1}{\sigma \cdot \sqrt{\frac{1}{n_1}+\frac{1}{n_2}}}\right)^2 \left( \frac{\sigma^2}{n_1} + \frac{\sigma^2}{n_2} \right) \right) \overset{H_0}{=} \mathcal{N}\left( 0, 1 \right) \; .

P/ugkv-ztest1.md

@@ -64,13 +64,13 @@ $$ \label{eq:mean-samp}
 \bar{y} = \frac{1}{n} \sum_{i=1}^{n} y_i \; .
 $$
 
-Using the [linearity of the expected value](/P/mean-lin), the [additivity of the variance under independence](/P/var-add) and [scaling of the variance upon multiplication](/P/var-scal), the sample mean follows a [normal distribution](/D/norm)
+Using the [linear combination formula for normal random variables](/P/norm-lincomb), the sample mean follows a [normal distribution](/D/norm) with the following parameters:
 
 $$ \label{eq:mean-samp-dist}
-\bar{y} = \frac{1}{n} \sum_{i=1}^{n} y_i \sim \mathcal{N}\left( \frac{1}{n} n \mu, \left(\frac{1}{n}\right)^2 n \sigma^2 \right) = \mathcal{N}\left( \mu, \sigma^2/n \right)
+\bar{y} = \frac{1}{n} \sum_{i=1}^{n} y_i \sim \mathcal{N}\left( \frac{1}{n} n \mu, \left(\frac{1}{n}\right)^2 n \sigma^2 \right) = \mathcal{N}\left( \mu, \sigma^2/n \right) \; .
 $$
 
-and additionally using the [invariance of the variance under addition](/P/var-inv), the distribution of $z = \sqrt{n/\sigma^2} (\bar{y}-\mu_0)$ becomes
+Again employing the linear combination theorem, the distribution of $z = \sqrt{n/\sigma^2} (\bar{y}-\mu_0)$ becomes
 
 $$ \label{eq:z-dist-s1}
 z = \sqrt{\frac{n}{\sigma^2}} (\bar{y} - \mu_0) \sim \mathcal{N}\left( \sqrt{\frac{n}{\sigma^2}} (\mu - \mu_0), \left(\sqrt{\frac{n}{\sigma^2}}\right)^2 \frac{\sigma^2}{n} \right) = \mathcal{N}\left( \sqrt{n} \, \frac{\mu-\mu_0}{\sigma}, 1 \right) \; ,