Merge pull request #219 from JoramSoch/master

added 1 proof and 1 definition

Author: JoramSoch

D/map.md

@@ -0,0 +1,40 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2023-12-01 14:32:38
+
+title: "Maximum-a-posteriori estimation"
+chapter: "General Theorems"
+section: "Bayesian statistics"
+topic: "Probabilistic modeling"
+definition: "Maximum-a-posteriori estimation"
+
+sources:
+  - authors: "Wikipedia"
+    year: 2023
+    title: "Maximum a posteriori estimation"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2023-12-01"
+    url: "https://en.wikipedia.org/wiki/Maximum_a_posteriori_estimation#Description"
+
+def_id: "D191"
+shortcut: "map"
+username: "JoramSoch"
+---
+
+
+**Definition:** Consider a [posterior distribution](/D/post) of an unknown parameter $\theta$, given measured data $y$, parametrized by posterior hyperparameters $\phi$:
+
+$$ \label{eq:post}
+\theta|y \sim \mathcal{D}(\phi) \; .
+$$
+
+Then, the value of $\theta$ at which the [posterior density](/D/post) attains its maximum is called the "maximum-a-posteriori estimate" or "MAP estimate" of $\theta$:
+
+$$ \label{eq:prior-pdf}
+\hat{\theta}_\mathrm{MAP} = \operatorname*{arg\,max}_\theta \mathcal{D}(\theta; \phi) \; .
+$$

I/ToC.md

@@ -288,10 +288,11 @@ title: "Table of Contents"
    &emsp;&ensp; 5.1.5. *[Joint likelihood](/D/jl)* <br>
    &emsp;&ensp; 5.1.6. **[Joint likelihood is product of likelihood and prior](/P/jl-lfnprior)** <br>
    &emsp;&ensp; 5.1.7. *[Posterior distribution](/D/post)* <br>
-   &emsp;&ensp; 5.1.8. **[Posterior density is proportional to joint likelihood](/P/post-jl)** <br>
-   &emsp;&ensp; 5.1.9. **[Combined posterior distribution from independent data](/P/post-ind)** <br>
-   &emsp;&ensp; 5.1.10. *[Marginal likelihood](/D/ml)* <br>
-   &emsp;&ensp; 5.1.11. **[Marginal likelihood is integral of joint likelihood](/P/ml-jl)** <br>
+   &emsp;&ensp; 5.1.8. *[Maximum-a-posteriori estimation](/D/map)* <br>
+   &emsp;&ensp; 5.1.9. **[Posterior density is proportional to joint likelihood](/P/post-jl)** <br>
+   &emsp;&ensp; 5.1.10. **[Combined posterior distribution from independent data](/P/post-ind)** <br>
+   &emsp;&ensp; 5.1.11. *[Marginal likelihood](/D/ml)* <br>
+   &emsp;&ensp; 5.1.12. **[Marginal likelihood is integral of joint likelihood](/P/ml-jl)** <br>
 
    5.2. Prior distributions <br>
    &emsp;&ensp; 5.2.1. *[Flat vs. hard vs. soft](/D/prior-flat)* <br>
@@ -719,11 +720,12 @@ title: "Table of Contents"
    &emsp;&ensp; 3.1.1. *[Definition](/D/bin-data)* <br>
    &emsp;&ensp; 3.1.2. **[Maximum likelihood estimation](/P/bin-mle)** <br>
    &emsp;&ensp; 3.1.3. **[Maximum log-likelihood](/P/bin-mll)** <br>
-   &emsp;&ensp; 3.1.4. **[Conjugate prior distribution](/P/bin-prior)** <br>
-   &emsp;&ensp; 3.1.5. **[Posterior distribution](/P/bin-post)** <br>
-   &emsp;&ensp; 3.1.6. **[Log model evidence](/P/bin-lme)** <br>
-   &emsp;&ensp; 3.1.7. **[Log Bayes factor](/P/bin-lbf)** <br>
-   &emsp;&ensp; 3.1.8. **[Posterior probability](/P/bin-pp)** <br>
+   &emsp;&ensp; 3.1.4. **[Maximum-a-posteriori estimation](/P/bin-map)** <br>
+   &emsp;&ensp; 3.1.5. **[Conjugate prior distribution](/P/bin-prior)** <br>
+   &emsp;&ensp; 3.1.6. **[Posterior distribution](/P/bin-post)** <br>
+   &emsp;&ensp; 3.1.7. **[Log model evidence](/P/bin-lme)** <br>
+   &emsp;&ensp; 3.1.8. **[Log Bayes factor](/P/bin-lbf)** <br>
+   &emsp;&ensp; 3.1.9. **[Posterior probability](/P/bin-pp)** <br>
 
    3.2. Multinomial observations <br>
    &emsp;&ensp; 3.2.1. *[Definition](/D/mult-data)* <br>

P/bin-map.md

@@ -0,0 +1,72 @@
+---
+layout: proof
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2023-12-01 14:36:54
+
+title: "Maximum-a-posteriori estimation for binomial observations"
+chapter: "Statistical Models"
+section: "Count data"
+topic: "Binomial observations"
+theorem: "Maximum-a-posteriori estimation"
+
+sources:
+
+proof_id: "P427"
+shortcut: "bin-map"
+username: "JoramSoch"
+---
+
+
+**Theorem:** Let $y$ be the number of successes resulting from $n$ independent trials with unknown success probability $p$, such that $y$ follows a [binomial distribution](/D/bin):
+
+$$ \label{eq:Bin}
+y \sim \mathrm{Bin}(n,p) \; .
+$$
+
+Moreover, assume a [beta prior distribution](/P/bin-prior) over the model parameter $p$:
+
+$$ \label{eq:Bin-prior}
+\mathrm{p}(p) = \mathrm{Bet}(p; \alpha_0, \beta_0) \; .
+$$
+
+Then, the [maximum-a-posteriori estimate](/D/map) of $p$ is
+
+$$ \label{eq:Bin-MLE}
+\hat{p}_\mathrm{MAP} = \frac{\alpha_0+y-1}{\alpha_0+\beta_0+n-2} \; .
+$$
+
+
+**Proof:** Given the [prior distribution](/D/prior) in \eqref{eq:Bin-prior}, the [posterior distribution](/D/post) for [binomial observations](/D/bin-data) [is also a beta distribution](/P/bin-post)
+
+$$ \label{eq:Bin-post}
+\mathrm{p}(p|y) = \mathrm{Bet}(p; \alpha_n, \beta_n)
+$$
+
+where the [posterior hyperparameters](/D/post) are equal to
+
+$$ \label{eq:Bin-post-par}
+\begin{split}
+\alpha_n &= \alpha_0 + y \\
+\beta_n &= \beta_0 + (n-y) \; .
+\end{split}
+$$
+
+The [mode of the beta distribution](/P/beta-mode) is given by:
+
+$$ \label{eq:beta-mode}
+X \sim \mathrm{Bet}(\alpha, \beta) \quad \Rightarrow \quad \mathrm{mode}(X) = \frac{\alpha-1}{\alpha+\beta-2} \; .
+$$
+
+Applying \eqref{eq:beta-mode} to \eqref{eq:Bin-post} with \eqref{eq:Bin-post-par}, the [maximum-a-posteriori estimate](/D/map) of $p$ follows as:
+
+$$ \label{eq:Bin-MAP}
+\begin{split}
+\hat{p}_\mathrm{MAP} &= \frac{\alpha_n-1}{\alpha_n+\beta_n-2} \\
+&\overset{\eqref{eq:Bin-post-par}}{=} \frac{\alpha_0+y-1}{\alpha_0+y+\beta_0+(n-y)-2} \\
+&= \frac{\alpha_0+y-1}{\alpha_0+\beta_0+n-2} \; .
+\end{split}
+$$