Merge pull request #37 from StatProofBook/master

JoramSoch · web-flow · commit bc0dfbd77edf · 2020-10-29T07:34:22.000+01:00
update to master
diff --git a/D/logreg.md b/D/logreg.md
@@ -9,7 +9,7 @@ date: 2020-06-28 20:51:00
 
 title: "Logistic regression"
 chapter: "Statistical Models"
-section: "Probability data"
+section: "Categorical data"
 topic: "Logistic regression"
 definition: "Definition"
 
diff --git a/I/Definition_by_Author.md b/I/Definition_by_Author.md
@@ -4,7 +4,7 @@ title: "Definition by Author"
 ---
 
 
-### JoramSoch (98 definitions)
+### JoramSoch (100 definitions)
 
 - [Akaike information criterion](/D/aic)
 - [Bayesian information criterion](/D/bic)
@@ -33,8 +33,10 @@ title: "Definition by Author"
 - [Differential cross-entropy](/D/dent-cross)
 - [Differential entropy](/D/dent)
 - [Dirichlet distribution](/D/dir)
+- [Dirichlet-distributed data](/D/dir-data)
 - [Discrete uniform distribution](/D/duni)
 - [Estimation matrix](/D/emat)
+- [Exceedance probability](/D/prob-exc)
 - [Expected value](/D/mean)
 - [Explained sum of squares](/D/ess)
 - [Exponential distribution](/D/exp)
diff --git a/I/Definition_by_Number.md b/I/Definition_by_Number.md
@@ -109,3 +109,5 @@ title: "Definition by Number"
 | D100 | chi2 | [Chi-square distribution](/D/chi2) | kjpetrykowski | 2020-10-13 |
 | D101 | med | [Median](/D/med) | JoramSoch | 2020-10-15 |
 | D102 | mode | [Mode](/D/mode) | JoramSoch | 2020-10-15 |
+| D103 | prob-exc | [Exceedance probability](/D/prob-exc) | JoramSoch | 2020-10-22 |
+| D104 | dir-data | [Dirichlet-distributed data](/D/dir-data) | JoramSoch | 2020-10-22 |
diff --git a/I/Definition_by_Topic.md b/I/Definition_by_Topic.md
@@ -45,12 +45,14 @@ title: "Definition by Topic"
 - [Differential cross-entropy](/D/dent-cross)
 - [Differential entropy](/D/dent)
 - [Dirichlet distribution](/D/dir)
+- [Dirichlet-distributed data](/D/dir-data)
 - [Discrete uniform distribution](/D/duni)
 
 ### E
 
 - [Encompassing model](/D/encm)
 - [Estimation matrix](/D/emat)
+- [Exceedance probability](/D/prob-exc)
 - [Expected value](/D/mean)
 - [Explained sum of squares](/D/ess)
 - [Exponential distribution](/D/exp)
diff --git a/I/Definitions_without_Source.md b/I/Definitions_without_Source.md
@@ -7,6 +7,7 @@ title: "Definitions without Source"
 - [Beta-distributed data](/D/beta-data)
 - [Binomial observations](/D/bin-data)
 - [Conditional differential entropy](/D/dent-cond)
+- [Dirichlet-distributed data](/D/dir-data)
 - [Estimation matrix](/D/emat)
 - [Full probability model](/D/fpm)
 - [Generative model](/D/gm)
diff --git a/I/Proof_by_Author.md b/I/Proof_by_Author.md
@@ -4,7 +4,7 @@ title: "Proof by Author"
 ---
 
 
-### JoramSoch (170 proofs)
+### JoramSoch (173 proofs)
 
 - [(Non-)Multiplicativity of the expected value](/P/mean-mult)
 - [Additivity of the Kullback-Leibler divergence for independent distributions](/P/kl-add)
@@ -41,6 +41,7 @@ title: "Proof by Author"
 - [Differential entropy of the multivariate normal distribution](/P/mvn-dent)
 - [Differential entropy of the normal distribution](/P/norm-dent)
 - [Equivalence of matrix-normal distribution and multivariate normal distribution](/P/matn-mvn)
+- [Exceedance probabilities for the the Dirichlet distribution](/P/dir-ep)
 - [Expectation of a quadratic form](/P/mean-qf)
 - [Expected value of a non-negative random variable](/P/mean-nnrvar)
 - [Expected value of x times ln(x) for a gamma distribution](/P/gam-xlogx)
@@ -74,6 +75,7 @@ title: "Proof by Author"
 - [Marginal distributions of the multivariate normal distribution](/P/mvn-marg)
 - [Marginal distributions of the normal-gamma distribution](/P/ng-marg)
 - [Marginal likelihood is a definite integral of joint likelihood](/P/ml-jl)
+- [Maximum likelihood estimation for Dirichlet-distributed data](/P/dir-mle)
 - [Maximum likelihood estimation for Poisson-distributed data](/P/poiss-mle)
 - [Maximum likelihood estimation for multiple linear regression](/P/mlr-mle)
 - [Maximum likelihood estimation for the general linear model](/P/glm-mle)
@@ -149,6 +151,7 @@ title: "Proof by Author"
 - [Relation between gamma distribution and standard gamma distribution](/P/gam-sgam2)
 - [Relation between normal distribution and standard normal distribution](/P/norm-snorm)
 - [Relation between normal distribution and standard normal distribution](/P/norm-snorm2)
+- [Relation between normal distribution and standard normal distribution](/P/norm-snorm3)
 - [Relation of Kullback-Leibler divergence to entropy](/P/kl-ent)
 - [Relation of continuous Kullback-Leibler divergence to differential entropy](/P/kl-dent)
 - [Relation of continuous mutual information to joint and conditional differential entropy](/P/cmi-jcde)
diff --git a/I/Proof_by_Number.md b/I/Proof_by_Number.md
@@ -186,3 +186,6 @@ title: "Proof by Number"
 | P177 | gam-sgam2 | [Relation between gamma distribution and standard gamma distribution](/P/gam-sgam2) | JoramSoch | 2020-10-15 |
 | P178 | gam-cdf | [Cumulative distribution function of the gamma distribution](/P/gam-cdf) | JoramSoch | 2020-10-15 |
 | P179 | gam-xlogx | [Expected value of x times ln(x) for a gamma distribution](/P/gam-xlogx) | JoramSoch | 2020-10-15 |
+| P180 | norm-snorm3 | [Relation between normal distribution and standard normal distribution](/P/norm-snorm3) | JoramSoch | 2020-10-22 |
+| P181 | dir-ep | [Exceedance probabilities for the the Dirichlet distribution](/P/dir-ep) | JoramSoch | 2020-10-22 |
+| P182 | dir-mle | [Maximum likelihood estimation for Dirichlet-distributed data](/P/dir-mle) | JoramSoch | 2020-10-22 |
diff --git a/I/Proof_by_Topic.md b/I/Proof_by_Topic.md
@@ -58,6 +58,7 @@ title: "Proof by Topic"
 
 - [Encompassing Prior Method for computing Bayes Factors](/P/bf-ep)
 - [Equivalence of matrix-normal distribution and multivariate normal distribution](/P/matn-mvn)
+- [Exceedance probabilities for the the Dirichlet distribution](/P/dir-ep)
 - [Expectation of a quadratic form](/P/mean-qf)
 - [Expected value of a non-negative random variable](/P/mean-nnrvar)
 - [Expected value of x times ln(x) for a gamma distribution](/P/gam-xlogx)
@@ -112,6 +113,7 @@ title: "Proof by Topic"
 - [Marginal distributions of the multivariate normal distribution](/P/mvn-marg)
 - [Marginal distributions of the normal-gamma distribution](/P/ng-marg)
 - [Marginal likelihood is a definite integral of joint likelihood](/P/ml-jl)
+- [Maximum likelihood estimation for Dirichlet-distributed data](/P/dir-mle)
 - [Maximum likelihood estimation for Poisson-distributed data](/P/poiss-mle)
 - [Maximum likelihood estimation for multiple linear regression](/P/mlr-mle)
 - [Maximum likelihood estimation for the general linear model](/P/glm-mle)
@@ -207,6 +209,7 @@ title: "Proof by Topic"
 - [Relation between gamma distribution and standard gamma distribution](/P/gam-sgam2)
 - [Relation between normal distribution and standard normal distribution](/P/norm-snorm)
 - [Relation between normal distribution and standard normal distribution](/P/norm-snorm2)
+- [Relation between normal distribution and standard normal distribution](/P/norm-snorm3)
 - [Relation of Kullback-Leibler divergence to entropy](/P/kl-ent)
 - [Relation of continuous Kullback-Leibler divergence to differential entropy](/P/kl-dent)
 - [Relation of continuous mutual information to joint and conditional differential entropy](/P/cmi-jcde)
diff --git a/I/Proofs_without_Source.md b/I/Proofs_without_Source.md
@@ -62,6 +62,7 @@ title: "Proofs without Source"
 - [Relation between gamma distribution and standard gamma distribution](/P/gam-sgam2)
 - [Relation between normal distribution and standard normal distribution](/P/norm-snorm)
 - [Relation between normal distribution and standard normal distribution](/P/norm-snorm2)
+- [Relation between normal distribution and standard normal distribution](/P/norm-snorm3)
 - [Relationship between R² and maximum log-likelihood](/P/rsq-mll)
 - [Relationship between covariance and correlation](/P/cov-corr)
 - [Relationship between precision matrix and correlation matrix](/P/precmat-corrmat)
diff --git a/I/Table_of_Contents.md b/I/Table_of_Contents.md
@@ -391,11 +391,6 @@ title: "Table of Contents"
    &emsp;&ensp; 4.2.1. *[Definition](/D/dir-data)* <br>
    &emsp;&ensp; 4.2.2. **[Maximum likelihood estimation](/P/dir-mle)** <br>
    
-   4.3. Logistic regression <br>
-   &emsp;&ensp; 4.3.1. *[Definition](/D/logreg)* <br>
-   &emsp;&ensp; 4.3.2. **[Probability and log-odds](/P/logreg-pnlo)** <br>
-   &emsp;&ensp; 4.3.3. **[Log-odds and probability](/P/logreg-lonp)** <br>
-
 5. Categorical data
    
    5.1. Binomial observations <br>
@@ -410,6 +405,11 @@ title: "Table of Contents"
    &emsp;&ensp; 5.2.3. **[Posterior distribution](/P/mult-post)** <br>
    &emsp;&ensp; 5.2.4. **[Log model evidence](/P/mult-lme)** <br>
 
+   5.3. Logistic regression <br>
+   &emsp;&ensp; 5.3.1. *[Definition](/D/logreg)* <br>
+   &emsp;&ensp; 5.3.2. **[Probability and log-odds](/P/logreg-pnlo)** <br>
+   &emsp;&ensp; 5.3.3. **[Log-odds and probability](/P/logreg-lonp)** <br>
+
 
 <br>
 <section class="chapter" id="Model Selection">
diff --git a/P/dir-ep.md b/P/dir-ep.md
@@ -131,7 +131,7 @@ $$
 The [probability density function of the gamma distribution](/P/gam-pdf) is given by
 
 $$ \label{eq:Gam-pdf}
-\mathrm{Gam}(x; a, b) = \frac{{b}^{a}}{\Gamma(a)} \, x^{a-1} \, \exp[-b x] \quad \text{for} \quad x > 0 \; .
+\mathrm{Gam}(x; a, b) = \frac{ {b}^{a} }{\Gamma(a)} \, x^{a-1} \, \exp[-b x] \quad \text{for} \quad x > 0 \; .
 $$
 
 Consider the [gamma random variables](/D/gam)
@@ -194,4 +194,4 @@ $$ \label{eq:Dir-EP-qed}
 \varphi_i = \int_0^\infty \prod_{j \neq i} \left( \frac{\gamma(\alpha_j,q_i)}{\Gamma(\alpha_j)} \right) \cdot \frac{q_i^{\alpha_i-1} \exp[-q_i]}{\Gamma(\alpha_i)} \, \mathrm{d}q_i \; .
 $$
 
-In other words, the [exceedance probability](/D/prob-exc) for one element from a [Dirichlet-distributed](/D/dir) [random vector](/D/rvec) is an integral from zero to infinity where the first term in the integrand conforms to a product of [gamma](/D/gam) [cumulative distribution functions](/D/cdf) and the second term is a [gamma](/D/gam) [probability density function](/D/pdf).
+In other words, the [exceedance probability](/D/prob-exc) for one element from a [Dirichlet-distributed](/D/dir) [random vector](/D/rvec) is an integral from zero to infinity where the first term in the integrand conforms to a product of [gamma](/D/gam) [cumulative distribution functions](/D/cdf) and the second term is a [gamma](/D/gam) [probability density function](/D/pdf).
diff --git a/P/logreg-lonp.md b/P/logreg-lonp.md
@@ -9,7 +9,7 @@ date: 2020-03-03 12:01:00
 
 title: "Log-odds and probability in logistic regression"
 chapter: "Statistical Models"
-section: "Probability data"
+section: "Categorical data"
 topic: "Logistic regression"
 theorem: "Log-odds and probability"
 
diff --git a/P/logreg-pnlo.md b/P/logreg-pnlo.md
@@ -9,7 +9,7 @@ date: 2020-05-19 05:08:00
 
 title: "Probability and log-odds in logistic regression"
 chapter: "Statistical Models"
-section: "Probability data"
+section: "Categorical data"
 topic: "Logistic regression"
 theorem: "Probability and log-odds"
 
diff --git a/P/mean-lotus.md b/P/mean-lotus.md
@@ -53,18 +53,18 @@ $$
 Writing the probability mass function $f_Y(y)$ in terms of $y = g(x)$, we have:
 
 $$ \label{eq:mean-lotus-disc-s2}
-\mathrm{E}[g(X)] = \sum_{y \in \mathcal{Y}} y \, \mathrm{Pr}(g(x) = y) \; .
+\begin{split}
+\mathrm{E}[g(X)] &= \sum_{y \in \mathcal{Y}} y \, \mathrm{Pr}(g(x) = y) \\
+&= \sum_{y \in \mathcal{Y}} y \, \mathrm{Pr}(x = g^{-1}(y)) \\
+&= \sum_{y \in \mathcal{Y}} y \sum_{x = g^{-1}(y)} f_X(x) \\
+&= \sum_{y \in \mathcal{Y}} \sum_{x = g^{-1}(y)} y f_X(x) \\
+&= \sum_{y \in \mathcal{Y}} \sum_{x = g^{-1}(y)} g(x) f_X(x) \; .
+\end{split}
 $$
 
-Replacing the probability that $g(x) = y$ in terms of $f_X(x)$, this becomes:
+Finally, noting that "for all $y$, then for all $x = g^{-1}(y)$" is equivalent to "for all $x$" if $g^{-1}$ is a monotonic function, we can conclude that
 
 $$ \label{eq:mean-lotus-disc-s3}
-\mathrm{E}[g(X)] = \sum_{y \in \mathcal{Y}} y \sum_{x: \; g(x) = y} f_X(x) \; .
-$$
-
-Observe that $y = g(x)$ can be moved into the inner sum. Finally, noting that "for all $y$, then for all $x$, such that $g(x) = y$" is equivalent to "for all $x$" if $g^{-1}$ is a monotonic function, we can conclude that
-
-$$ \label{eq:mean-lotus-disc-s4}
 \mathrm{E}[g(X)] = \sum_{x \in \mathcal{X}} g(x) f_X(x) \; .
 $$
 
diff --git a/P/ng-kl.md b/P/ng-kl.md
@@ -14,7 +14,7 @@ topic: "Normal-gamma distribution"
 theorem: "Kullback-Leibler divergence"
 
 sources:
-  - authors: "Soch & Allefeld"
+  - authors: "Soch J, Allefeld A"
     year: 2016
     title: "Kullback-Leibler Divergence for the Normal-Gamma Distribution"
     in: "arXiv math.ST"
diff --git a/P/norm-cdfwerf.md b/P/norm-cdfwerf.md
@@ -33,7 +33,7 @@ username: "JoramSoch"
 ---
 
 
-**Theorem:** Let $X$ be a [random variable](/D/rvar) following a [normal distributions](/D/norm):
+**Theorem:** Let $X$ be a [random variable](/D/rvar) following a [normal distribution](/D/norm):
 
 $$ \label{eq:norm}
 X \sim \mathcal{N}(\mu, \sigma^2) \; .
@@ -104,7 +104,7 @@ T(0) = \varphi(0) \cdot \sum_{i=1}^{\infty} \frac{0^{2i-1}}{(2i-1)!!} + c &= \fr
 \end{split}
 $$
 
-3) Finally, the cumulative distribution functions of the standard normal distribution and the general normal distribution [are related to each other](/P/norm-snorm) as
+3) Finally, the [cumulative distribution functions](/D/cdf) of the [standard normal distribution](/D/snorm) and the general [normal distribution](/D/norm) [are related to each other](/P/norm-snorm) as
 
 $$ \label{eq:norm-snorm-cdf}
 \Phi_{\mu,\sigma}(x) = \Phi\left( \frac{x-\mu}{\sigma} \right) \; .
diff --git a/P/norm-snorm2.md b/P/norm-snorm2.md
@@ -62,7 +62,7 @@ where $\mathcal{Y} = \left\lbrace y = g(x): x \in \mathcal{X} \right\rbrace$. Wi
 $$ \label{eq:pdf-Z}
 \begin{split}
 f_Z(z) &= \frac{1}{\sqrt{2 \pi} \sigma} \cdot \exp \left[ -\frac{1}{2} \left( \frac{g^{-1}(z)-\mu}{\sigma} \right)^2 \right] \cdot \frac{\mathrm{d}g^{-1}(z)}{\mathrm{d}z} \\
-&= \frac{1}{\sqrt{2 \pi} \sigma} \cdot \exp \left[ -\frac{1}{2} \left( \frac{(\sigma z + \mu)-\mu}{\sigma} \right)^2 \right] \cdot \frac{\mathrm{d}(\sigma Z + \mu)}{\mathrm{d}z} \\
+&= \frac{1}{\sqrt{2 \pi} \sigma} \cdot \exp \left[ -\frac{1}{2} \left( \frac{(\sigma z + \mu)-\mu}{\sigma} \right)^2 \right] \cdot \frac{\mathrm{d}(\sigma z + \mu)}{\mathrm{d}z} \\
 &= \frac{1}{\sqrt{2 \pi} \sigma} \cdot \exp \left[ -\frac{1}{2} z^2 \right] \cdot \sigma \\
 &= \frac{1}{\sqrt{2 \pi}} \cdot \exp \left[ -\frac{1}{2} z^2 \right]
 \end{split}
