Skip to content

Commit 311edd3

Browse files
committed
2 parents 9e89ee3 + 986868f commit 311edd3

4 files changed

Lines changed: 392 additions & 5 deletions

File tree

I/ToC.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -322,6 +322,7 @@ title: "Table of Contents"
322322
&emsp;&ensp; 1.3.4. **[Variance](/P/bin-var)** <br>
323323
&emsp;&ensp; 1.3.5. **[Range of variance](/P/bin-varrange)** <br>
324324
&emsp;&ensp; 1.3.6. **[Shannon entropy](/P/bin-ent)** <br>
325+
&emsp;&ensp; 1.3.7. **[Conditional binomial](/P/bin-margcond)** <br>
325326

326327
1.4. Poisson distribution <br>
327328
&emsp;&ensp; 1.4.1. *[Definition](/D/poiss)* <br>
@@ -440,11 +441,12 @@ title: "Table of Contents"
440441

441442
3.9. Beta distribution <br>
442443
&emsp;&ensp; 3.9.1. *[Definition](/D/beta)* <br>
443-
&emsp;&ensp; 3.9.2. **[Probability density function](/P/beta-pdf)** <br>
444-
&emsp;&ensp; 3.9.3. **[Moment-generating function](/P/beta-mgf)** <br>
445-
&emsp;&ensp; 3.9.4. **[Cumulative distribution function](/P/beta-cdf)** <br>
446-
&emsp;&ensp; 3.9.5. **[Mean](/P/beta-mean)** <br>
447-
&emsp;&ensp; 3.9.6. **[Variance](/P/beta-var)** <br>
444+
&emsp;&ensp; 3.9.2. **[Relationship to chi-squared distribution](/P/beta-chi2)** <br>
445+
&emsp;&ensp; 3.9.3. **[Probability density function](/P/beta-pdf)** <br>
446+
&emsp;&ensp; 3.9.4. **[Moment-generating function](/P/beta-mgf)** <br>
447+
&emsp;&ensp; 3.9.5. **[Cumulative distribution function](/P/beta-cdf)** <br>
448+
&emsp;&ensp; 3.9.6. **[Mean](/P/beta-mean)** <br>
449+
&emsp;&ensp; 3.9.7. **[Variance](/P/beta-var)** <br>
448450

449451
3.10. Wald distribution <br>
450452
&emsp;&ensp; 3.10.1. *[Definition](/D/wald)* <br>
@@ -658,6 +660,9 @@ title: "Table of Contents"
658660
&emsp;&ensp; 4.2.1. *[Definition](/D/dir-data)* <br>
659661
&emsp;&ensp; 4.2.2. **[Maximum likelihood estimation](/P/dir-mle)** <br>
660662

663+
4.3. Beta-binomial data <br>
664+
&emsp;&ensp; 4.3.1. **[Method of moments](/P/betabin-mome)** <br>
665+
661666
5. Categorical data
662667

663668
5.1. Binomial observations <br>

P/beta-chi2.md

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2022-10-07 13:20:00
9+
10+
title: "Relationship between chi-squared distribution and beta distribution"
11+
chapter: "Probability Distributions"
12+
section: "Univariate continuous distributions"
13+
topic: "Beta distribution"
14+
theorem: "Relationship with chi-squared distribution"
15+
16+
sources:
17+
- authors: "Probability Fact"
18+
year: 2021
19+
title: "If X ~ chisq(m) and Y ~ chisq(n) are independent"
20+
in: "Twitter"
21+
pages: "retrieved on 2022-10-17"
22+
url: "https://twitter.com/ProbFact/status/1450492787854647300"
23+
24+
proof_id: "P356"
25+
shortcut: "beta-chi2"
26+
username: "JoramSoch"
27+
---
28+
29+
30+
**Theorem:** Let $X$ and $Y$ be [independent](/D/ind) [random variables](/D/rvar) following [chi-squared distributions](/D/chi2):
31+
32+
$$ \label{eq:chi2}
33+
X \sim \chi^2(m) \quad \text{and} \quad Y \sim \chi^2(n) \; .
34+
$$
35+
36+
Then, the quantity $X/(X+Y)$ follows a [beta distributiob](/D/beta):
37+
38+
$$ \label{eq:beta-chi2}
39+
\frac{X}{X+Y} \sim \mathrm{Bet}\left( \frac{m}{2}, \frac{n}{2} \right) \; .
40+
$$
41+
42+
43+
**Proof:** The [probability density function of the chi-squared distribution](/P/chi2-pdf) is
44+
45+
$$ \label{eq:chi2-pdf}
46+
X \sim \chi^2(u) \quad \Rightarrow \quad f_X(x) = \frac{1}{\Gamma\left( \frac{u}{2} \right) \cdot 2^{u/2}} \cdot x^{\frac{u}{2}-1} \cdot e^{-\frac{x}{2}} \; .
47+
$$
48+
49+
Define the random variables $Z$ and $W$ as functions of $X$ and $Y$
50+
51+
$$ \label{eq:ZW-XY}
52+
\begin{split}
53+
Z &= \frac{X}{X+Y} \\
54+
W &= Y \; ,
55+
\end{split}
56+
$$
57+
58+
such that the inverse functions $X$ and $Y$ in terms of $Z$ and $W$ are
59+
60+
$$ \label{eq:XY-ZW}
61+
\begin{split}
62+
X &= \frac{ZW}{1-Z} \\
63+
Y &= W \; .
64+
\end{split}
65+
$$
66+
67+
This implies the following Jacobian matrix and determinant:
68+
69+
$$ \label{eq:XY-ZW-jac}
70+
\begin{split}
71+
J &= \left[ \begin{matrix}
72+
\frac{\mathrm{d}X}{\mathrm{d}Z} & \frac{\mathrm{d}X}{\mathrm{d}W} \\
73+
\frac{\mathrm{d}Y}{\mathrm{d}Z} & \frac{\mathrm{d}Y}{\mathrm{d}W}
74+
\end{matrix} \right]
75+
= \left[ \begin{matrix}
76+
\frac{W}{(1-Z)^2} & \frac{Z}{1-Z} \\
77+
0 & 1
78+
\end{matrix} \right] \\
79+
\lvert J \rvert &= \frac{W}{(1-Z)^2} \; .
80+
\end{split}
81+
$$
82+
83+
Because $X$ and $Y$ are [independent](/D/ind), the [joint density](/D/dist-joint) of $X$ and $Y$ is [equal to the product](/P/prob-ind) of the [marginal densities](/D/dist-marg):
84+
85+
$$ \label{eq:f-XY}
86+
f_{X,Y}(x,y) = f_X(x) \cdot f_Y(y) \; .
87+
$$
88+
89+
With the [probability density function of an invertible function](/P/pdf-invfct), the [joint density](/D/dist-joint) of $Z$ and $W$ can be derived as:
90+
91+
$$ \label{eq:f-ZW-s1}
92+
f_{Z,W}(z,w) = f_{X,Y}(x,y) \cdot \lvert J \rvert \; .
93+
$$
94+
95+
Substituting \eqref{eq:XY-ZW} into \eqref{eq:chi2-pdf}, and then with \eqref{eq:XY-ZW-jac} into \eqref{eq:f-ZW-s1}, we get:
96+
97+
$$ \label{eq:f-ZW-s2}
98+
\begin{split}
99+
f_{Z,W}(z,w) &= f_X\left( \frac{zw}{1-z} \right) \cdot f_Y(w) \cdot \lvert J \rvert \\
100+
&= \frac{1}{\Gamma\left( \frac{m}{2} \right) \cdot 2^{m/2}} \cdot \left( \frac{zw}{1-z} \right)^{\frac{m}{2}-1} \cdot e^{-\frac{1}{2} \left( \frac{zw}{1-z} \right)} \cdot \frac{1}{\Gamma\left( \frac{n}{2} \right) \cdot 2^{n/2}} \cdot w^{\frac{n}{2}-1} \cdot e^{-\frac{w}{2}} \cdot \frac{w}{(1-z)^2} \\
101+
&= \frac{1}{\Gamma\left( \frac{m}{2} \right) \Gamma\left( \frac{n}{2} \right) \cdot 2^{m/2} 2^{n/2}} \cdot \left( \frac{z}{1-z} \right)^{\frac{m}{2}-1} \left( \frac{1}{(1-z)} \right)^2 \cdot w^{\frac{m}{2}+\frac{n}{2}-1} e^{-\frac{1}{2} \left( \frac{zw}{1-z} + \frac{w(1-z)}{1-z} \right)} \\
102+
&= \frac{1}{\Gamma\left( \frac{m}{2} \right) \Gamma\left( \frac{n}{2} \right) \cdot 2^{(m+n)/2}} \cdot z^{\frac{m}{2}-1} \cdot (1-z)^{-\frac{m}{2}-1} \cdot w^{\frac{m+n}{2}-1} \cdot e^{-\frac{1}{2} \left( \frac{w}{1-z} \right)} \; .
103+
\end{split}
104+
$$
105+
106+
The [marginal density](/D/dist-marg) of $Z$ can now be [obtained by integrating out](/D/dist-marg) $W$:
107+
108+
$$ \label{eq:f-Z-s1}
109+
\begin{split}
110+
f_Z(z) &= \int_{0}^{\infty} f_{Z,W}(z,w) \, \mathrm{d}w \\
111+
&= \frac{1}{\Gamma\left( \frac{m}{2} \right) \Gamma\left( \frac{n}{2} \right) \cdot 2^{(m+n)/2}} \cdot z^{\frac{m}{2}-1} \cdot (1-z)^{-\frac{m}{2}-1} \cdot \int_{0}^{\infty} w^{\frac{m+n}{2}-1} \cdot e^{-\frac{1}{2} \left( \frac{w}{1-z} \right)} \, \mathrm{d}w \\
112+
&= \frac{1}{\Gamma\left( \frac{m}{2} \right) \Gamma\left( \frac{n}{2} \right) \cdot 2^{(m+n)/2}} \cdot z^{\frac{m}{2}-1} \cdot (1-z)^{-\frac{m}{2}-1} \cdot \frac{\Gamma\left( \frac{m+n}{2} \right)}{\left( \frac{1}{2(1-z)} \right)^{\frac{m+n}{2}}} \cdot \\
113+
&\hphantom{=} \int_{0}^{\infty} \frac{\left( \frac{1}{2(1-z)} \right)^{\frac{m+n}{2}}}{\Gamma\left( \frac{m+n}{2} \right)} \cdot w^{\frac{m+n}{2}-1} \cdot e^{-\frac{1}{2(1-z)} \, w} \, \mathrm{d}w \; .
114+
\end{split}
115+
$$
116+
117+
At this point, we can recognize that the integrand is equal to the [probability density function of a gamma distribution](/P/gam-pdf) with
118+
119+
$$ \label{eq:f-W-gam-ab}
120+
a = \frac{m+n}{2} \quad \text{and} \quad b = \frac{1}{2(1-z)} \; ,
121+
$$
122+
123+
and [because a probability density function integrates to one](/D/pdf), we have:
124+
125+
$$ \label{eq:f-Z-s2}
126+
\begin{split}
127+
f_Z(z) &= \frac{1}{\Gamma\left( \frac{m}{2} \right) \Gamma\left( \frac{n}{2} \right) \cdot 2^{(m+n)/2}} \cdot z^{\frac{m}{2}-1} \cdot (1-z)^{-\frac{m}{2}-1} \cdot \frac{\Gamma\left( \frac{m+n}{2} \right)}{\left( \frac{1}{2(1-z)} \right)^{\frac{m+n}{2}}} \\
128+
&= \frac{\Gamma\left( \frac{m+n}{2} \right) \cdot 2^{(m+n)/2}}{\Gamma\left( \frac{m}{2} \right) \Gamma\left( \frac{n}{2} \right) \cdot 2^{(m+n)/2}} \cdot z^{\frac{m}{2}-1} \cdot (1-z)^{-\frac{m}{2}+\frac{m+n}{2}-1} \\
129+
&= \frac{\Gamma\left( \frac{m+n}{2} \right)}{\Gamma\left( \frac{m}{2} \right) \Gamma\left( \frac{n}{2} \right)} \cdot z^{\frac{m}{2}-1} \cdot (1-z)^{\frac{n}{2}-1} \; .
130+
\end{split}
131+
$$
132+
133+
With the [definition of the beta function](/P/beta-mean), this becomes
134+
135+
$$ \label{eq:f-Z-s3}
136+
f_Z(z) = \frac{1}{\mathrm{B}\left( \frac{m}{2}, \frac{n}{2} \right)} \cdot z^{\frac{m}{2}-1} \cdot (1-z)^{\frac{n}{2}-1}
137+
$$
138+
139+
which is the [probability density function of the beta distribution](/P/beta-pdf), such that
140+
141+
$$ \label{eq:beta-chi2-qed}
142+
Z \sim \mathrm{Bet}\left( \frac{m}{2}, \frac{n}{2} \right) \; .
143+
$$

P/betabin-mome.md

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
---
2+
layout: proof
3+
mathjax: true
4+
5+
author: "Joram Soch"
6+
affiliation: "BCCN Berlin"
7+
e_mail: "joram.soch@bccn-berlin.de"
8+
date: 2022-10-07 15:13:00
9+
10+
title: "Method of moments for beta-binomial data"
11+
chapter: "Statistical Models"
12+
section: "Probability data"
13+
topic: "Beta-binomial data"
14+
theorem: "Method of moments"
15+
16+
sources:
17+
- authors: "statisticsmatt"
18+
year: 2022
19+
title: "Method of Moments Estimation Beta Binomial Distribution"
20+
in: "YouTube"
21+
pages: "retrieved on 2022-10-07"
22+
url: "https://www.youtube.com/watch?v=18PWnWJsPnA"
23+
- authors: "Wikipedia"
24+
year: 2022
25+
title: "Beta-binomial distribution"
26+
in: "Wikipedia, the free encyclopedia"
27+
pages: "retrieved on 2022-10-07"
28+
url: "https://en.wikipedia.org/wiki/Beta-binomial_distribution#Method_of_moments"
29+
30+
proof_id: "P357"
31+
shortcut: "betabin-mome"
32+
username: "JoramSoch"
33+
---
34+
35+
36+
**Theorem:** Let $y = \left\lbrace y_1, \ldots, y_N \right\rbrace$ be a set of observed counts independent and identically distributed according to a [beta-binomial distribution](/D/betabin) with number of trials $n$ as well as parameters $\alpha$ and $\beta$:
37+
38+
$$ \label{eq:binbeta}
39+
y_i \sim \mathrm{BinBet}(n, \alpha, \beta), \quad i = 1, \ldots, N \; .
40+
$$
41+
42+
Then, the [method-of-moments estimates](/D/mome) for the parameters $\alpha$ and $\beta$ are given by
43+
44+
$$ \label{eq:binbeta-mome}
45+
\begin{split}
46+
\hat{\alpha} &= \frac{n m_1 - m_2}{n \left( \frac{m_2}{m_1} - m_1 - 1 \right) + m_1} \\
47+
\hat{\beta} &= \frac{\left( n - m_1 \right)\left( n - \frac{m_2}{m_1} \right)}{n \left( \frac{m_2}{m_1} - m_1 - 1 \right) + m_1}
48+
\end{split}
49+
$$
50+
51+
where $m_1$ and $m_2$ are the [first two raw sample moments](/D/mom-raw):
52+
53+
$$ \label{eq:y-m1-m2}
54+
\begin{split}
55+
m_1 &= \frac{1}{N} \sum_{i=1}^N y_i \\
56+
m_2 &= \frac{1}{N} \sum_{i=1}^N y_i^2 \; .
57+
\end{split}
58+
$$
59+
60+
61+
**Proof:** The first two [raw moments](/D/mom-raw) of the [beta-binomial distribution](/D/betabin) in terms of the parameters $\alpha$ and $\beta$ are given by
62+
63+
$$ \label{eq:binbeta-mu1-mu2}
64+
\begin{split}
65+
\mu_1 &= \frac{n \alpha}{\alpha + \beta} \\
66+
\mu_2 &= \frac{n \alpha (n \alpha + \beta + n)}{(\alpha + \beta)(n \alpha + \beta + 1)}
67+
\end{split}
68+
$$
69+
70+
Thus, [matching the moments](/D/mome) requires us to solve the following equation system for $\alpha$ and $\beta$:
71+
72+
$$ \label{eq:binbeta-m1-m2}
73+
\begin{split}
74+
m_1 &= \frac{n \alpha}{\alpha + \beta} \\
75+
m_2 &= \frac{n \alpha (n \alpha + \beta + n)}{(\alpha + \beta)(n \alpha + \beta + 1)} \; .
76+
\end{split}
77+
$$
78+
79+
From the first equation, we can deduce:
80+
81+
$$ \label{eq:beta-as-alpha}
82+
\begin{split}
83+
m_1(\alpha+\beta) &= n \alpha \\
84+
m_1 \alpha + m_1 \beta &= n \alpha \\
85+
m_1 \beta &= n \alpha - m_1 \alpha \\
86+
\beta &= \frac{n \alpha}{m_1} - \alpha \\
87+
\beta &= \alpha \left( \frac{n}{m_1} - 1 \right) \; .
88+
\end{split}
89+
$$
90+
91+
If we define $q = n/m_1 - 1$ and plug \eqref{eq:beta-as-alpha} into the second equation, we have:
92+
93+
$$ \label{eq:alpha-as-q}
94+
\begin{split}
95+
m_2 &= \frac{n \alpha (n \alpha + \alpha q + n)}{(\alpha + \alpha q)(\alpha + \alpha q + 1)} \\
96+
&= \frac{n \alpha (\alpha (n + q) + n)}{\alpha (1 + q)(\alpha (1 + q) + 1)} \\
97+
&= \frac{n (\alpha (n + q) + n)}{(1 + q)(\alpha (1 + q) + 1)} \\
98+
&= \frac{n (\alpha (n + q) + n)}{\alpha (1 + q)^2 + (1 + q)} \; .
99+
\end{split}
100+
$$
101+
102+
Noting that $1+q = n/m_1$ and expanding the fraction with $m_1$, one obtains:
103+
104+
$$ \label{eq:binbeta-mome-alpha}
105+
\begin{split}
106+
m_2 &= \frac{n \left(\alpha \left( \frac{n}{m_1} + n - 1 \right) + n \right)}{n \left( \alpha \frac{n}{m_1^2} + \frac{1}{m_1} \right)} \\
107+
m_2 &= \frac{\alpha \left( n + n m_1 - m_1 \right) + n m_1}{\alpha \frac{n}{m_1} + 1} \\
108+
m_2 \left( \frac{\alpha n}{m_1} + 1 \right) &= \alpha \left( n + n m_1 - m_1 \right) + n m_1 \\
109+
\alpha \left( n \frac{m_2}{m_1} - (n + n m_1 - m_1) \right) &= n m_1 - m_2 \\
110+
\alpha \left( n \left( \frac{m_2}{m_1} - m_1 - 1 \right) + m_1 \right) &= n m_1 - m_2 \\
111+
\alpha &= \frac{n m_1 - m_2}{n \left( \frac{m_2}{m_1} - m_1 - 1 \right) + m_1} \; .
112+
\end{split}
113+
$$
114+
115+
Plugging this into equation \eqref{eq:beta-as-alpha}, one obtains for $\beta$:
116+
117+
$$ \label{eq:binbeta-mome-beta}
118+
\begin{split}
119+
\beta &= \alpha \left( \frac{n}{m_1} - 1 \right) \\
120+
\beta &= \left( \frac{n m_1 - m_2}{n \left( \frac{m_2}{m_1} - m_1 - 1 \right) + m_1} \right) \left( \frac{n}{m_1} - 1 \right) \\
121+
\beta &= \frac{n^2 - n m_1 - n \frac{m_2}{m_1} + m_2}{n \left( \frac{m_2}{m_1} - m_1 - 1 \right) + m_1} \\
122+
\hat{\beta} &= \frac{\left( n - m_1 \right)\left( n - \frac{m_2}{m_1} \right)}{n \left( \frac{m_2}{m_1} - m_1 - 1 \right) + m_1} \; .
123+
\end{split}
124+
$$
125+
126+
Together, \eqref{eq:binbeta-mome-alpha} and \eqref{eq:binbeta-mome-beta} constitute the method-of-moment estimates of $\alpha$ and $\beta$.

0 commit comments

Comments
 (0)