Merge pull request #102 from StatProofBook/master

JoramSoch · web-flow · commit 83d84733e9db · 2022-12-15T23:10:31.000+01:00
update to master
diff --git a/I/PbA.md b/I/PbA.md
@@ -8,7 +8,7 @@ title: "Proof by Author"
 
 - [Covariance matrix of the multinomial distribution](/P/mult-cov)
 
-### JoramSoch (361 proofs)
+### JoramSoch (369 proofs)
 
 - [Accuracy and complexity for the univariate Gaussian](/P/ug-anc)
 - [Accuracy and complexity for the univariate Gaussian with known variance](/P/ugkv-anc)
@@ -142,6 +142,8 @@ title: "Proof by Author"
 - [Linear transformation theorem for the moment-generating function](/P/mgf-ltt)
 - [Linear transformation theorem for the multivariate normal distribution](/P/mvn-ltt)
 - [Linearity of the expected value](/P/mean-lin)
+- [Log Bayes factor for binomial observations](/P/bin-lbf)
+- [Log Bayes factor for multinomial observations](/P/mult-lbf)
 - [Log Bayes factor for the univariate Gaussian with known variance](/P/ugkv-lbf)
 - [Log Bayes factor in terms of log model evidences](/P/lbf-lme)
 - [Log family evidences in terms of log model evidences](/P/lfe-lme)
@@ -163,7 +165,9 @@ title: "Proof by Author"
 - [Marginal distributions of the normal-gamma distribution](/P/ng-marg)
 - [Marginal likelihood is a definite integral of joint likelihood](/P/ml-jl)
 - [Maximum likelihood estimation can result in biased estimates](/P/mle-bias)
+- [Maximum likelihood estimation for binomial observations](/P/bin-mle)
 - [Maximum likelihood estimation for Dirichlet-distributed data](/P/dir-mle)
+- [Maximum likelihood estimation for multinomial observations](/P/mult-mle)
 - [Maximum likelihood estimation for multiple linear regression](/P/mlr-mle)
 - [Maximum likelihood estimation for Poisson-distributed data](/P/poiss-mle)
 - [Maximum likelihood estimation for simple linear regression](/P/slr-mle)
@@ -173,6 +177,8 @@ title: "Proof by Author"
 - [Maximum likelihood estimation for the univariate Gaussian](/P/ug-mle)
 - [Maximum likelihood estimation for the univariate Gaussian with known variance](/P/ugkv-mle)
 - [Maximum likelihood estimator of variance is biased](/P/resvar-bias)
+- [Maximum log-likelihood for binomial observations](/P/bin-mll)
+- [Maximum log-likelihood for multinomial observations](/P/mult-mll)
 - [Maximum log-likelihood for multiple linear regression](/P/mlr-mll)
 - [Mean of the Bernoulli distribution](/P/bern-mean)
 - [Mean of the beta distribution](/P/beta-mean)
@@ -252,6 +258,8 @@ title: "Proof by Author"
 - [Posterior model probabilities in terms of log model evidences](/P/pmp-lme)
 - [Posterior model probability in terms of log Bayes factor](/P/pmp-lbf)
 - [Posterior probability of the alternative hypothesis for Bayesian linear regression](/P/blr-pp)
+- [Posterior probability of the alternative model for binomial observations](/P/bin-pp)
+- [Posterior probability of the alternative model for multinomial observations](/P/mult-pp)
 - [Probability and log-odds in logistic regression](/P/logreg-pnlo)
 - [Probability density function is first derivative of cumulative distribution function](/P/pdf-cdf)
 - [Probability density function of a linear function of a continuous random vector](/P/pdf-linfct)
diff --git a/I/PbN.md b/I/PbN.md
@@ -386,3 +386,11 @@ title: "Proof by Number"
 | P378 | anova2-cochran | [Application of Cochran's theorem to two-way analysis of variance](/P/anova2-cochran) | JoramSoch | 2022-11-16 |
 | P379 | anova2-pss | [Partition of sums of squares in two-way analysis of variance](/P/anova2-pss) | JoramSoch | 2022-11-16 |
 | P380 | anova2-fols | [F-statistics in terms of ordinary least squares estimates in two-way analysis of variance](/P/anova2-fols) | JoramSoch | 2022-11-16 |
+| P381 | bin-mle | [Maximum likelihood estimation for binomial observations](/P/bin-mle) | JoramSoch | 2022-11-23 |
+| P382 | bin-mll | [Maximum log-likelihood for binomial observations](/P/bin-mll) | JoramSoch | 2022-11-24 |
+| P383 | bin-lbf | [Log Bayes factor for binomial observations](/P/bin-lbf) | JoramSoch | 2022-11-25 |
+| P384 | bin-pp | [Posterior probability of the alternative model for binomial observations](/P/bin-pp) | JoramSoch | 2022-11-26 |
+| P385 | mult-mle | [Maximum likelihood estimation for multinomial observations](/P/mult-mle) | JoramSoch | 2022-12-02 |
+| P386 | mult-mll | [Maximum log-likelihood for multinomial observations](/P/mult-mll) | JoramSoch | 2022-12-02 |
+| P387 | mult-lbf | [Log Bayes factor for multinomial observations](/P/mult-lbf) | JoramSoch | 2022-12-02 |
+| P388 | mult-pp | [Posterior probability of the alternative model for multinomial observations](/P/mult-pp) | JoramSoch | 2022-12-02 |
diff --git a/I/PbT.md b/I/PbT.md
@@ -172,6 +172,8 @@ title: "Proof by Topic"
 - [Linear transformation theorem for the moment-generating function](/P/mgf-ltt)
 - [Linear transformation theorem for the multivariate normal distribution](/P/mvn-ltt)
 - [Linearity of the expected value](/P/mean-lin)
+- [Log Bayes factor for binomial observations](/P/bin-lbf)
+- [Log Bayes factor for multinomial observations](/P/mult-lbf)
 - [Log Bayes factor for the univariate Gaussian with known variance](/P/ugkv-lbf)
 - [Log Bayes factor in terms of log model evidences](/P/lbf-lme)
 - [Log family evidences in terms of log model evidences](/P/lfe-lme)
@@ -196,7 +198,9 @@ title: "Proof by Topic"
 - [Marginal distributions of the normal-gamma distribution](/P/ng-marg)
 - [Marginal likelihood is a definite integral of joint likelihood](/P/ml-jl)
 - [Maximum likelihood estimation can result in biased estimates](/P/mle-bias)
+- [Maximum likelihood estimation for binomial observations](/P/bin-mle)
 - [Maximum likelihood estimation for Dirichlet-distributed data](/P/dir-mle)
+- [Maximum likelihood estimation for multinomial observations](/P/mult-mle)
 - [Maximum likelihood estimation for multiple linear regression](/P/mlr-mle)
 - [Maximum likelihood estimation for Poisson-distributed data](/P/poiss-mle)
 - [Maximum likelihood estimation for simple linear regression](/P/slr-mle)
@@ -206,6 +210,8 @@ title: "Proof by Topic"
 - [Maximum likelihood estimation for the univariate Gaussian](/P/ug-mle)
 - [Maximum likelihood estimation for the univariate Gaussian with known variance](/P/ugkv-mle)
 - [Maximum likelihood estimator of variance is biased](/P/resvar-bias)
+- [Maximum log-likelihood for binomial observations](/P/bin-mll)
+- [Maximum log-likelihood for multinomial observations](/P/mult-mll)
 - [Maximum log-likelihood for multiple linear regression](/P/mlr-mll)
 - [Mean of the Bernoulli distribution](/P/bern-mean)
 - [Mean of the beta distribution](/P/beta-mean)
@@ -301,6 +307,8 @@ title: "Proof by Topic"
 - [Posterior model probabilities in terms of log model evidences](/P/pmp-lme)
 - [Posterior model probability in terms of log Bayes factor](/P/pmp-lbf)
 - [Posterior probability of the alternative hypothesis for Bayesian linear regression](/P/blr-pp)
+- [Posterior probability of the alternative model for binomial observations](/P/bin-pp)
+- [Posterior probability of the alternative model for multinomial observations](/P/mult-pp)
 - [Probability and log-odds in logistic regression](/P/logreg-pnlo)
 - [Probability density function is first derivative of cumulative distribution function](/P/pdf-cdf)
 - [Probability density function of a linear function of a continuous random vector](/P/pdf-linfct)
diff --git a/I/PwS.md b/I/PwS.md
@@ -46,6 +46,8 @@ title: "Proofs without Source"
 - [Kullback-Leibler divergence for the normal distribution](/P/norm-kl)
 - [Linear combination of independent normal random variables](/P/norm-lincomb)
 - [Linear transformation theorem for the matrix-normal distribution](/P/matn-ltt)
+- [Log Bayes factor for binomial observations](/P/bin-lbf)
+- [Log Bayes factor for multinomial observations](/P/mult-lbf)
 - [Log Bayes factor for the univariate Gaussian with known variance](/P/ugkv-lbf)
 - [Log model evidence for multinomial observations](/P/mult-lme)
 - [Log model evidence for multivariate Bayesian linear regression](/P/mblr-lme)
@@ -58,12 +60,15 @@ title: "Proofs without Source"
 - [Marginal distributions of the normal-gamma distribution](/P/ng-marg)
 - [Marginal likelihood is a definite integral of joint likelihood](/P/ml-jl)
 - [Maximum likelihood estimation can result in biased estimates](/P/mle-bias)
+- [Maximum likelihood estimation for multinomial observations](/P/mult-mle)
 - [Maximum likelihood estimation for multiple linear regression](/P/mlr-mle)
 - [Maximum likelihood estimation for Poisson-distributed data](/P/poiss-mle)
 - [Maximum likelihood estimation for simple linear regression](/P/slr-mle)
 - [Maximum likelihood estimation for simple linear regression](/P/slr-mle2)
 - [Maximum likelihood estimation for the general linear model](/P/glm-mle)
 - [Maximum likelihood estimation for the Poisson distribution with exposure values](/P/poissexp-mle)
+- [Maximum log-likelihood for binomial observations](/P/bin-mll)
+- [Maximum log-likelihood for multinomial observations](/P/mult-mll)
 - [Mean of the categorical distribution](/P/cat-mean)
 - [Mean of the continuous uniform distribution](/P/cuni-mean)
 - [Mean of the multinomial distribution](/P/mult-mean)
@@ -84,6 +89,8 @@ title: "Proofs without Source"
 - [Ordinary least squares for the general linear model](/P/glm-ols)
 - [Parameter estimates for simple linear regression are uncorrelated after mean-centering](/P/slr-olscorr)
 - [Posterior density is proportional to joint likelihood](/P/post-jl)
+- [Posterior probability of the alternative model for binomial observations](/P/bin-pp)
+- [Posterior probability of the alternative model for multinomial observations](/P/mult-pp)
 - [Probability density function of the beta distribution](/P/beta-pdf)
 - [Probability density function of the continuous uniform distribution](/P/cuni-pdf)
 - [Probability density function of the Dirichlet distribution](/P/dir-pdf)
diff --git a/P/bin-lbf.md b/P/bin-lbf.md
@@ -7,7 +7,7 @@ affiliation: "BCCN Berlin"
 e_mail: "joram.soch@bccn-berlin.de"
 date: 2022-11-25 14:40:00
 
-title: "Log model evidence for binomial observations"
+title: "Log Bayes factor for binomial observations"
 chapter: "Statistical Models"
 section: "Count data"
 topic: "Binomial observations"
diff --git a/P/bin-lme.md b/P/bin-lme.md
@@ -118,7 +118,7 @@ the LME finally becomes
 
 $$ \label{eq:Bin-LME-s2}
 \begin{split}
-\log \mathrm{p}(y|m) = \log \Gamma(n+1) &- \log \Gamma(k+1) - \log \Gamma(n-k+1) \\
+\log \mathrm{p}(y|m) = \log \Gamma(n+1) &- \log \Gamma(y+1) - \log \Gamma(n-y+1) \\
 &+ \log B(\alpha_n,\beta_n) - \log B(\alpha_0,\beta_0) \; .
 \end{split}
 $$
diff --git a/P/bin-mll.md b/P/bin-mll.md
@@ -39,8 +39,6 @@ $$
 
 **Proof:** The [log-likelihood function for binomial data](/P/bin-mle) is given by
 
-With the [probability mass function of the binomial distribution](/P/bin-pmf), equation \eqref{eq:Bin} implies the following [likelihood function](/D/lf):
-
 $$ \label{eq:Bin-LL}
 \mathrm{LL}(p) = \log {n \choose y} + y \log p + (n-y) \log (1-p)
 $$
diff --git a/P/mult-lbf.md b/P/mult-lbf.md
@@ -64,7 +64,7 @@ $$ \label{eq:Mult-LME-m1}
 \end{split}
 $$
 
-Because the null model $m_0$ has no free parameter, its [log model evidence](/D/lme) (logarithmized [marginal likelihood](/D/ml)) is equal to the [log-likelihood function for multinomial observations](/P/mult-mle) at the value $p = [1/k, \ldots, 1/k]$:
+Because the null model $m_0$ has no free parameter, its [log model evidence](/D/lme) (logarithmized [marginal likelihood](/D/ml)) is equal to the [log-likelihood function for multinomial observations](/P/mult-mle) at the value $p_0 = [1/k, \ldots, 1/k]$:
 
 $$ \label{eq:Mult-LME-m0}
 \begin{split}
diff --git a/P/mult-mle.md b/P/mult-mle.md
@@ -40,7 +40,7 @@ $$ \label{eq:Mult-marg}
 X \sim \mathrm{Mult}(n,p) \quad \Rightarrow \quad X_j \sim \mathrm{Bin}(n, p_j) \quad \text{for all} \quad j = 1, \ldots, k \; .
 $$
 
-Thus, combining \eqref{eq:Mult} with \eqref{eq:Mult}, we have
+Thus, combining \eqref{eq:Mult} with \eqref{eq:Mult-marg}, we have
 
 $$ \label{eq:Mult-Bin}
 y_j \sim \mathrm{Bin}(n,p_j)
diff --git a/P/mult-pp.md b/P/mult-pp.md
@@ -39,7 +39,7 @@ $$
 Then, the [posterior probability](/D/pmp) of the [alternative model](/D/h1) is given by
 
 $$ \label{eq:Mult-PP1}
-p(m_1|y) = 
+p(m_1|y) = \frac{1}{1 + k^{-n} \cdot \frac{\Gamma \left( \sum_{j=1}^{k} \alpha_{nj} \right)}{\Gamma \left( \sum_{j=1}^{k} \alpha_{0j} \right)} \cdot \frac{\prod_{j=1}^k \Gamma(\alpha_{0j})}{\prod_{j=1}^k \Gamma(\alpha_{nj})}}
 $$
 
 where $\Gamma(x)$ is the gamma function and $\alpha_n$ are the [posterior hyperparameters for multinomial observations](/P/mult-post) which are functions of the [numbers of observations](/D/mult) $y_1, \ldots, y_k$.
diff --git a/P/pval-h0.md b/P/pval-h0.md
@@ -52,7 +52,7 @@ $$ \label{eq:pval-cdf-s1}
 F_P(p) &= \mathrm{Pr}(P < p) \\
 &= \mathrm{Pr}(F_T(T) < p) \\
 &= \mathrm{Pr}(T < F_T^{-1}(p)) \\
-&= F_T^{-1}(F_T^{-1}(p)) \\
+&= F_T(F_T^{-1}(p)) \\
 &= p
 \end{split}
 $$
@@ -61,4 +61,4 @@ which is the [cumulative distribution function of a continuous uniform distribut
 
 $$ \label{eq:cuni-cdf}
 F_X(x) = \int_{-\infty}^{x} \mathcal{U}(z; 0, 1) \, \mathrm{d}z = x \quad \text{where} \quad 0 \leq x \leq 1 \; .
-$$
+$$
diff --git a/P/ugkv-lbf.md b/P/ugkv-lbf.md
@@ -54,7 +54,7 @@ $$
 The LME of the alternative $m_1$ is equal to the [log model evidence for the univariate Gaussian with known variance](/P/ugkv-lme):
 
 $$ \label{eq:UGkv-LME-m1}
-\mathrm{LME}(m_1) = \frac{n}{2} \log\left( \frac{\tau}{2 \pi} \right) + \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) - \frac{1}{2} \left( \tau y^\mathrm{T} y + \lambda_0 \mu_0^2 - \lambda_n \mu_n^2 \right) \; .
+\mathrm{LME}(m_1) = \log p(y|m_1) = \frac{n}{2} \log\left( \frac{\tau}{2 \pi} \right) + \frac{1}{2} \log\left( \frac{\lambda_0}{\lambda_n} \right) - \frac{1}{2} \left( \tau y^\mathrm{T} y + \lambda_0 \mu_0^2 - \lambda_n \mu_n^2 \right) \; .
 $$
 
 Because the null model $m_0$ has no free parameter, its [log model evidence](/D/lme) (logarithmized [marginal likelihood](/D/ml)) is equal to the [log-likelihood function for the univariate Gaussian with known variance](/P/ugkv-mle) at the value $\mu = 0$:
