Merge pull request #184 from JoramSoch/master

added 4 definitions and 5 proofs

Author: JoramSoch

D/betabin-data.md

@@ -0,0 +1,28 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2022-10-20 08:20:00
+
+title: "Beta-binomial data"
+chapter: "Statistical Models"
+section: "Frequency data"
+topic: "Beta-binomial data"
+definition: "Definition"
+
+sources:
+
+def_id: "D178"
+shortcut: "betabin-data"
+username: "JoramSoch"
+---
+
+
+**Definition:** Beta-binomial data are defined as a set of counts $y = \left\lbrace y_1, \ldots, y_N \right\rbrace$ with $y_i \in \mathbb{N}, \; i = 1,\ldots,N$, independent and identically distributed according to a [beta-binomial distribution](/D/betabin) with number of trials $n$ as well as shapes $\alpha$ and $\beta$:
+
+$$ \label{eq:betabin-data}
+y_i \sim \mathrm{BetBin}(n,\alpha,\beta), \quad i = 1, \ldots, N \; .
+$$

D/betabin.md

@@ -0,0 +1,48 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2022-10-20 08:09:00
+
+title: "Beta-binomial distribution"
+chapter: "Probability Distributions"
+section: "Univariate discrete distributions"
+topic: "Beta-binomial distribution"
+definition: "Definition"
+
+sources:
+  - authors: "Wikipedia"
+    year: 2022
+    title: "Beta-binomial distribution"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2022-10-20"
+    url: "https://en.wikipedia.org/wiki/Beta-binomial_distribution#Motivation_and_derivation"
+
+def_id: "D177"
+shortcut: "betabin"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let $p$ be a [random variable](/D/rvar) following a [beta distribution](/D/beta)
+
+$$ \label{eq:beta}
+p \sim \mathrm{Bet}(\alpha, \beta)
+$$
+
+and let $X$ be a [random variable](/D/rvar) following a [binomial distribution](/D/bin) conditional on $p$
+
+$$ \label{eq:bin}
+X \mid p \sim \mathrm{Bin}(n, p) \; .
+$$
+
+Then, the [marginal distribution](/D/dist-marg) of $X$ is called a beta-binomial distribution
+
+$$ \label{eq:betabin}
+X \sim \mathrm{BetBin}(n, \alpha, \beta)
+$$
+
+with [number of trials](/D/bin) $n$ and [shape parameters](/D/beta) $\alpha$ and $\beta$.

D/fe.md

@@ -0,0 +1,41 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2022-10-20 09:57:00
+
+title: "Family evidence"
+chapter: "Model Selection"
+section: "Bayesian model selection"
+topic: "Family evidence"
+definition: "Definition"
+
+sources:
+  - authors: "Soch J, Allefeld C"
+    year: 2018
+    title: "MACS – a new SPM toolbox for model assessment, comparison and selection"
+    in: "Journal of Neuroscience Methods"
+    pages: "vol. 306, pp. 19-31, eq. 16"
+    url: "https://www.sciencedirect.com/science/article/pii/S0165027018301468"
+    doi: "10.1016/j.jneumeth.2018.05.017"
+
+def_id: "D180"
+shortcut: "fe"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let $f$ be a family of $M$ [generative models](/D/gm) $m_1, \ldots, m_M$, such that the following statement holds true:
+
+$$ \label{eq:fam}
+f \Leftrightarrow m_1 \vee \ldots \vee m_M \; .
+$$
+
+Then, the family evidence (FE) of $f$ is defined as the [marginal probability](/D/prob-marg) relative to the [model evidences](/D/me) $p(y \vert m_i), conditional only on $f$:
+
+$$ \label{eq:fe}
+\mathrm{FE}(f) = p(y|f) \; .
+$$

D/me.md

@@ -0,0 +1,35 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2022-10-20 09:43:00
+
+title: "Model evidence"
+chapter: "Model Selection"
+section: "Bayesian model selection"
+topic: "Model evidence"
+definition: "Definition"
+
+sources:
+  - authors: "Penny WD"
+    year: 2012
+    title: "Comparing Dynamic Causal Models using AIC, BIC and Free Energy"
+    in: "NeuroImage"
+    pages: "vol. 59, iss. 2, pp. 319-330, eq. 15"
+    url: "https://www.sciencedirect.com/science/article/pii/S1053811911008160"
+    doi: "10.1016/j.neuroimage.2011.07.039"
+
+def_id: "D179"
+shortcut: "me"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let $m$ be a [generative model](/D/gm) with [likelihood function](/D/lf) $p(y \vert \theta, m)$ and [prior distribution](/D/prior) $p(\theta \vert m)$. Then, the model evidence (ME) of $m$ is defined as the [marginal likelihood](/D/ml) of this model:
+
+$$ \label{eq:ME}
+\mathrm{ME}(m) = p(y|m) \; .
+$$

I/ToC.md

@@ -329,11 +329,17 @@ title: "Table of Contents"
    &emsp;&ensp; 1.3.7. **[Shannon entropy](/P/bin-ent)** <br>
    &emsp;&ensp; 1.3.8. **[Conditional binomial](/P/bin-margcond)** <br>
 
-   1.4. Poisson distribution <br>
-   &emsp;&ensp; 1.4.1. *[Definition](/D/poiss)* <br>
-   &emsp;&ensp; 1.4.2. **[Probability mass function](/P/poiss-pmf)** <br>
-   &emsp;&ensp; 1.4.3. **[Mean](/P/poiss-mean)** <br>
-   &emsp;&ensp; 1.4.4. **[Variance](/P/poiss-var)** <br>
+   1.4. Beta-binomial distribution <br>
+   &emsp;&ensp; 1.4.1. *[Definition](/D/betabin)* <br>
+   &emsp;&ensp; 1.4.2. **[Probability mass function](/P/betabin-pmf)** <br>
+   &emsp;&ensp; 1.4.3. **[Probability mass function in terms of gamma function](/P/betabin-pmfitogf)** <br>
+   &emsp;&ensp; 1.4.4. **[Cumulative distribution function](/P/betabin-cdf)** <br>
+   
+   1.5. Poisson distribution <br>
+   &emsp;&ensp; 1.5.1. *[Definition](/D/poiss)* <br>
+   &emsp;&ensp; 1.5.2. **[Probability mass function](/P/poiss-pmf)** <br>
+   &emsp;&ensp; 1.5.3. **[Mean](/P/poiss-mean)** <br>
+   &emsp;&ensp; 1.5.4. **[Variance](/P/poiss-var)** <br>
 
 2. Multivariate discrete distributions
 
@@ -678,7 +684,8 @@ title: "Table of Contents"
    &emsp;&ensp; 4.2.2. **[Maximum likelihood estimation](/P/dir-mle)** <br>
 
    4.3. Beta-binomial data <br>
-   &emsp;&ensp; 4.3.1. **[Method of moments](/P/betabin-mome)** <br>
+   &emsp;&ensp; 4.3.1. *[Definition](/D/betabin-data)* <br>
+   &emsp;&ensp; 4.3.2. **[Method of moments](/P/betabin-mome)** <br>
 
 5. Categorical data
 
@@ -727,41 +734,49 @@ title: "Table of Contents"
 
 3. Bayesian model selection
 
-   3.1. Log model evidence <br>
-   &emsp;&ensp; 3.1.1. *[Definition](/D/lme)* <br>
-   &emsp;&ensp; 3.1.2. **[Derivation](/P/lme-der)** <br>
-   &emsp;&ensp; 3.1.3. **[Expression using prior and posterior](/P/lme-pnp)** <br>
-   &emsp;&ensp; 3.1.4. **[Partition into accuracy and complexity](/P/lme-anc)** <br>
-   &emsp;&ensp; 3.1.5. *[Uniform-prior log model evidence](/D/uplme)* <br>
-   &emsp;&ensp; 3.1.6. *[Cross-validated log model evidence](/D/cvlme)* <br>
-   &emsp;&ensp; 3.1.7. *[Empirical Bayesian log model evidence](/D/eblme)* <br>
-   &emsp;&ensp; 3.1.8. *[Variational Bayesian log model evidence](/D/vblme)* <br>
-   
-   3.2. Log family evidence <br>
-   &emsp;&ensp; 3.2.1. *[Definition](/D/lfe)* <br>
-   &emsp;&ensp; 3.2.2. **[Derivation](/P/lfe-der)** <br>
-   &emsp;&ensp; 3.2.3. **[Calculation from log model evidences](/P/lfe-lme)** <br>
-   
-   3.3. Log Bayes factor <br>
-   &emsp;&ensp; 3.3.1. *[Definition](/D/lbf)* <br>
-   &emsp;&ensp; 3.3.2. **[Derivation](/P/lbf-der)** <br>
-   &emsp;&ensp; 3.3.3. **[Calculation from log model evidences](/P/lbf-lme)** <br>
-   
-   3.4. Bayes factor <br>
-   &emsp;&ensp; 3.4.1. *[Definition](/D/bf)* <br>
-   &emsp;&ensp; 3.4.2. **[Transitivity](/P/bf-trans)** <br>
-   &emsp;&ensp; 3.4.3. **[Computation using Savage-Dickey Density Ratio](/P/bf-sddr)** <br>
-   &emsp;&ensp; 3.4.4. **[Computation using Encompassing Prior Method](/P/bf-ep)** <br>
-   &emsp;&ensp; 3.4.5. *[Encompassing model](/D/encm)* <br>
-   
-   3.5. Posterior model probability <br>
-   &emsp;&ensp; 3.5.1. *[Definition](/D/pmp)* <br>
-   &emsp;&ensp; 3.5.2. **[Derivation](/P/pmp-der)** <br>
-   &emsp;&ensp; 3.5.3. **[Calculation from Bayes factors](/P/pmp-bf)** <br>
-   &emsp;&ensp; 3.5.4. **[Calculation from log Bayes factor](/P/pmp-lbf)** <br>
-   &emsp;&ensp; 3.5.5. **[Calculation from log model evidences](/P/pmp-lme)** <br>
-   
-   3.6. Bayesian model averaging <br>
-   &emsp;&ensp; 3.6.1. *[Definition](/D/bma)* <br>
-   &emsp;&ensp; 3.6.2. **[Derivation](/P/bma-der)** <br>
-   &emsp;&ensp; 3.6.3. **[Calculation from log model evidences](/P/bma-lme)** <br>
+   3.1. Model evidence <br>
+   &emsp;&ensp; 3.1.1. *[Definition](/D/me)* <br>
+   &emsp;&ensp; 3.1.2. **[Derivation](/P/me-der)** <br>
+   
+   3.2. Log model evidence <br>
+   &emsp;&ensp; 3.2.1. *[Definition](/D/lme)* <br>
+   &emsp;&ensp; 3.2.2. **[Derivation](/P/lme-der)** <br>
+   &emsp;&ensp; 3.2.3. **[Expression using prior and posterior](/P/lme-pnp)** <br>
+   &emsp;&ensp; 3.2.4. **[Partition into accuracy and complexity](/P/lme-anc)** <br>
+   &emsp;&ensp; 3.2.5. *[Uniform-prior log model evidence](/D/uplme)* <br>
+   &emsp;&ensp; 3.2.6. *[Cross-validated log model evidence](/D/cvlme)* <br>
+   &emsp;&ensp; 3.2.7. *[Empirical Bayesian log model evidence](/D/eblme)* <br>
+   &emsp;&ensp; 3.2.8. *[Variational Bayesian log model evidence](/D/vblme)* <br>
+   
+   3.3. Family evidence <br>
+   &emsp;&ensp; 3.3.1. *[Definition](/D/fe)* <br>
+   &emsp;&ensp; 3.3.2. **[Derivation](/P/fe-der)** <br>
+   
+   3.4. Log family evidence <br>
+   &emsp;&ensp; 3.4.1. *[Definition](/D/lfe)* <br>
+   &emsp;&ensp; 3.4.2. **[Derivation](/P/lfe-der)** <br>
+   &emsp;&ensp; 3.4.3. **[Calculation from log model evidences](/P/lfe-lme)** <br>
+   
+   3.5. Bayes factor <br>
+   &emsp;&ensp; 3.5.1. *[Definition](/D/bf)* <br>
+   &emsp;&ensp; 3.5.2. **[Transitivity](/P/bf-trans)** <br>
+   &emsp;&ensp; 3.5.3. **[Computation using Savage-Dickey Density Ratio](/P/bf-sddr)** <br>
+   &emsp;&ensp; 3.5.4. **[Computation using Encompassing Prior Method](/P/bf-ep)** <br>
+   &emsp;&ensp; 3.5.5. *[Encompassing model](/D/encm)* <br>
+   
+   3.6. Log Bayes factor <br>
+   &emsp;&ensp; 3.6.1. *[Definition](/D/lbf)* <br>
+   &emsp;&ensp; 3.6.2. **[Derivation](/P/lbf-der)** <br>
+   &emsp;&ensp; 3.6.3. **[Calculation from log model evidences](/P/lbf-lme)** <br>
+   
+   3.7. Posterior model probability <br>
+   &emsp;&ensp; 3.7.1. *[Definition](/D/pmp)* <br>
+   &emsp;&ensp; 3.7.2. **[Derivation](/P/pmp-der)** <br>
+   &emsp;&ensp; 3.7.3. **[Calculation from Bayes factors](/P/pmp-bf)** <br>
+   &emsp;&ensp; 3.7.4. **[Calculation from log Bayes factor](/P/pmp-lbf)** <br>
+   &emsp;&ensp; 3.7.5. **[Calculation from log model evidences](/P/pmp-lme)** <br>
+   
+   3.8. Bayesian model averaging <br>
+   &emsp;&ensp; 3.8.1. *[Definition](/D/bma)* <br>
+   &emsp;&ensp; 3.8.2. **[Derivation](/P/bma-der)** <br>
+   &emsp;&ensp; 3.8.3. **[Calculation from log model evidences](/P/bma-lme)** <br>

P/betabin-cdf.md

@@ -0,0 +1,89 @@
+---
+layout: proof
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2022-10-22 05:28:00
+
+title: "Cumulative distribution function of the beta-binomial distribution"
+chapter: "Probability Distributions"
+section: "Univariate discrete distributions"
+topic: "Beta-binomial distribution"
+theorem: "Cumulative distribution function"
+
+sources:
+
+proof_id: "P366"
+shortcut: "betabin-cdf"
+username: "JoramSoch"
+---
+
+
+**Theorem:** Let $X$ be a [random variable](/D/rvar) following a [beta-binomial distribution](/D/betabin):
+
+$$ \label{eq:betabin}
+X \sim \mathrm{BetBin}(n,\alpha,\beta) \; .
+$$
+
+Then, the [cumulative distribution function](/D/cdf) of $X$ is
+
+$$ \label{eq:betabin-cdf}
+F_X(x) = \frac{1}{\mathrm{B}(\alpha,\beta)} \cdot \frac{\Gamma(n+1)}{\Gamma(\alpha+\beta+n)} \cdot \sum_{i=0}^{x} \frac{\Gamma(\alpha+i) \cdot \Gamma(\beta+n-i)}{\Gamma(i+1) \cdot \Gamma(n-i+1)}
+$$
+
+where $\mathrm{B}(x,y)$ is the beta function and $\Gamma(x)$ is the gamma function.
+
+
+**Proof:** The [cumulative distribution function](/D/cdf) is defined as
+
+$$ \label{eq:cdf}
+F_X(x) = \mathrm{Pr}(X \leq x)
+$$
+
+which, for a [discrete random variable](/D/rvar-disc), evaluates to
+
+$$ \label{eq:cdf-disc}
+F_X(x) = \sum_{i=-\infty}^{x} f_X(i) \; .
+$$
+
+With the [probability mass function of the beta-binomial distribution](/P/betabin-pmf), this becomes
+
+$$ \label{eq:betabin-cdf-s1}
+F_X(x) = \sum_{i=0}^{x} {n \choose i} \cdot \frac{\mathrm{B}(\alpha+i,\beta+n-i)}{\mathrm{B}(\alpha,\beta)} \; .
+$$
+
+Using the expression of binomial coefficients in terms of factorials
+
+$$ \label{eq:bincoeff-facts}
+{n \choose k} = \frac{n!}{k! \, (n-k)!} \; ,
+$$
+
+the relationship between factorials and the gamma function
+
+$$ \label{eq:facts-gamfct}
+n! = \Gamma(n+1)
+$$
+
+and the link between gamma function and beta function
+
+$$ \label{eq:betafct-gamfct}
+\mathrm{B}(\alpha,\beta) = \frac{\Gamma(\alpha) \, \Gamma(\beta)}{\Gamma(\alpha+\beta)} \; ,
+$$
+
+equation \eqref{eq:betabin-cdf-s1} can be further developped as follows:
+
+$$ \label{eq:betabin-cdf-s2}
+\begin{split}
+F_X(x) &\overset{\eqref{eq:bincoeff-facts}}{=} \frac{1}{\mathrm{B}(\alpha,\beta)} \cdot \sum_{i=0}^{x} \frac{n!}{i! \, (n-i)!} \cdot \mathrm{B}(\alpha+i,\beta+n-i) \\
+&\overset{\eqref{eq:betafct-gamfct}}{=} \frac{1}{\mathrm{B}(\alpha,\beta)} \cdot \sum_{i=0}^{x} \frac{n!}{i! \, (n-i)!} \cdot
+\frac{\Gamma(\alpha+i) \cdot \Gamma(\beta+n-i)}{\Gamma(\alpha+\beta+n)} \\
+&= \frac{1}{\mathrm{B}(\alpha,\beta)} \cdot \frac{n!}{\Gamma(\alpha+\beta+n)} \cdot \sum_{i=0}^{x}
+\frac{\Gamma(\alpha+i) \cdot \Gamma(\beta+n-i)}{i! \, (n-i)!} \\
+&\overset{\eqref{eq:facts-gamfct}}{=} \frac{1}{\mathrm{B}(\alpha,\beta)} \cdot \frac{\Gamma(n+1)}{\Gamma(\alpha+\beta+n)} \cdot \sum_{i=0}^{x}
+\frac{\Gamma(\alpha+i) \cdot \Gamma(\beta+n-i)}{\Gamma(i+1) \cdot \Gamma(n-i+1)} \; .
+\end{split}
+$$
+
+This completes the proof.