Merge pull request #117 from StatProofBook/master

JoramSoch · web-flow · commit 7c131aace4b8 · 2023-11-17T15:21:48.000+01:00
update to master
diff --git a/D/ent.md b/D/ent.md
@@ -34,4 +34,4 @@ $$ \label{eq:ent}
 \mathrm{H}(X) = - \sum_{x \in \mathcal{X}} p(x) \cdot \log_b p(x)
 $$
 
-where $b$ is the base of the logarithm specifying in which unit the entropy is determined.
+where $b$ is the base of the logarithm specifying in which unit the entropy is determined. By convention, $0 \cdot \log 0$ is taken to be zero when calculating the entropy of $X$.
diff --git a/D/kl.md b/D/kl.md
@@ -43,4 +43,6 @@ $$ \label{eq:KL-cont}
 \mathrm{KL}[P||Q] = \int_{\mathcal{X}} p(x) \cdot \log \frac{p(x)}{q(x)} \, \mathrm{d}x
 $$
 
-where $p(x)$ and $q(x)$ are the [probability density functions](/D/pdf) of $P$ and $Q$.
+where $p(x)$ and $q(x)$ are the [probability density functions](/D/pdf) of $P$ and $Q$.
+
+[By convention](/D/ent), $0 \cdot \log 0$ is taken to be zero when calculating the divergence between $P$ and $Q$.
diff --git a/D/skew-samp.md b/D/skew-samp.md
@@ -0,0 +1,36 @@
+---
+layout: definition
+mathjax: true
+
+author: "Thomas J. Faulkenberry"
+affiliation: "Tarleton State University"
+e_mail: "faulkenberry@tarleton.edu"
+date: 2023-10-30 12:00:00
+
+title: "Sample skewness"
+chapter: "General Theorems"
+section: "Probability theory"
+topic: "Skewness"
+definition: "Sample skewness"
+
+sources:
+  - authors: "Joanes, D. N. and Gill, C. A."
+    year: 1998
+    title: "Comparing measures of sample skewness and kurtosis"
+    in: "The Statistician"
+    pages: "vol. 47, part 1, pp. 183-189"
+    url: "https://www.jstor.org/stable/2988433"
+
+def_id: "D190"
+shortcut: "skew-samp"
+username: "tomfaulkenberry"
+---
+
+
+**Definition:** Let $x = \left\lbrace x_1, \ldots, x_n \right\rbrace$ be a [sample](/D/samp) from a [random variable](/D/rvar) $X$. Then, the sample skewness of $x$ is given by
+
+$$ \label{eq:skew-samp}
+\hat{s} = \frac{\frac{1}{n}\sum_{i=1}^n (x_i-\bar{x})^3}{\left[\frac{1}{n}\sum_{i=1}^n(x_i-\bar{x})^2\right]^{3/2}} \; ,
+$$
+
+where $\bar{x}$ is the [sample mean](/D/mean-samp).
diff --git a/I/DbA.md b/I/DbA.md
@@ -204,10 +204,11 @@ title: "Definition by Author"
 
 - [Definition Template](/D/-temp-)
 
-### tomfaulkenberry (5 definitions)
+### tomfaulkenberry (6 definitions)
 
 - [Bayes factor](/D/bf)
 - [Encompassing model](/D/encm)
 - [ex-Gaussian distribution](/D/exg)
+- [Sample skewness](/D/skew-samp)
 - [Skewness](/D/skew)
 - [Wald distribution](/D/wald)
diff --git a/I/DbN.md b/I/DbN.md
@@ -196,3 +196,4 @@ title: "Definition by Number"
 | D187 | exg | [ex-Gaussian distribution](/D/exg) | tomfaulkenberry | 2023-04-18 |
 | D188 | skew | [Skewness](/D/skew) | tomfaulkenberry | 2023-04-20 |
 | D189 | bvn | [Bivariate normal distribution](/D/bvn) | JoramSoch | 2023-09-22 |
+| D190 | skew-samp | [Sample skewness](/D/skew-samp) | tomfaulkenberry | 2023-10-30 |
diff --git a/I/DbT.md b/I/DbT.md
@@ -212,6 +212,7 @@ title: "Definition by Topic"
 - [Sample covariance](/D/cov-samp)
 - [Sample covariance matrix](/D/covmat-samp)
 - [Sample mean](/D/mean-samp)
+- [Sample skewness](/D/skew-samp)
 - [Sample space](/D/samp-spc)
 - [Sample variance](/D/var-samp)
 - [Sampling distribution](/D/dist-samp)
diff --git a/I/PbA.md b/I/PbA.md
@@ -8,7 +8,7 @@ title: "Proof by Author"
 
 - [Covariance matrix of the multinomial distribution](/P/mult-cov)
 
-### JoramSoch (392 proofs)
+### JoramSoch (393 proofs)
 
 - [Accuracy and complexity for the univariate Gaussian](/P/ug-anc)
 - [Accuracy and complexity for the univariate Gaussian with known variance](/P/ugkv-anc)
@@ -140,6 +140,7 @@ title: "Proof by Author"
 - [Joint likelihood is the product of likelihood function and prior density](/P/jl-lfnprior)
 - [Kullback-Leibler divergence for the Bernoulli distribution](/P/bern-kl)
 - [Kullback-Leibler divergence for the binomial distribution](/P/bin-kl)
+- [Kullback-Leibler divergence for the continuous uniform distribution](/P/cuni-kl)
 - [Kullback-Leibler divergence for the Dirichlet distribution](/P/dir-kl)
 - [Kullback-Leibler divergence for the gamma distribution](/P/gam-kl)
 - [Kullback-Leibler divergence for the matrix-normal distribution](/P/matn-kl)
@@ -427,11 +428,13 @@ title: "Proof by Author"
 
 - [Proof Template](/P/-temp-)
 
-### tomfaulkenberry (16 proofs)
+### tomfaulkenberry (18 proofs)
 
 - [Encompassing prior method for computing Bayes factors](/P/bf-ep)
 - [Mean of the ex-Gaussian distribution](/P/exg-mean)
 - [Mean of the Wald distribution](/P/wald-mean)
+- [Method of moments for ex-Gaussian distributed data](/P/exg-mome)
+- [Method of moments for Wald distributed data](/P/wald-mome)
 - [Moment-generating function of the ex-Gaussian distribution](/P/exg-mgf)
 - [Moment-generating function of the exponential distribution](/P/exp-mgf)
 - [Moment-generating function of the Wald distribution](/P/wald-mgf)
diff --git a/I/PbN.md b/I/PbN.md
@@ -427,3 +427,6 @@ title: "Proof by Number"
 | P419 | bern-kl | [Kullback-Leibler divergence for the Bernoulli distribution](/P/bern-kl) | JoramSoch | 2023-10-13 |
 | P420 | bin-kl | [Kullback-Leibler divergence for the binomial distribution](/P/bin-kl) | JoramSoch | 2023-10-20 |
 | P421 | wald-skew | [Skewness of the Wald distribution](/P/wald-skew) | tomfaulkenberry | 2023-10-24 |
+| P422 | cuni-kl | [Kullback-Leibler divergence for the continuous uniform distribution](/P/cuni-kl) | JoramSoch | 2023-10-27 |
+| P423 | wald-mome | [Method of moments for Wald distributed data](/P/wald-mome) | tomfaulkenberry | 2023-10-30 |
+| P424 | exg-mome | [Method of moments for ex-Gaussian distributed data](/P/exg-mome) | tomfaulkenberry | 2023-10-30 |
diff --git a/I/PbT.md b/I/PbT.md
@@ -167,6 +167,7 @@ title: "Proof by Topic"
 
 - [Kullback-Leibler divergence for the Bernoulli distribution](/P/bern-kl)
 - [Kullback-Leibler divergence for the binomial distribution](/P/bin-kl)
+- [Kullback-Leibler divergence for the continuous uniform distribution](/P/cuni-kl)
 - [Kullback-Leibler divergence for the Dirichlet distribution](/P/dir-kl)
 - [Kullback-Leibler divergence for the gamma distribution](/P/gam-kl)
 - [Kullback-Leibler divergence for the matrix-normal distribution](/P/matn-kl)
@@ -252,6 +253,8 @@ title: "Proof by Topic"
 - [Median of the normal distribution](/P/norm-med)
 - [Method of moments for beta-binomial data](/P/betabin-mome)
 - [Method of moments for beta-distributed data](/P/beta-mome)
+- [Method of moments for ex-Gaussian distributed data](/P/exg-mome)
+- [Method of moments for Wald distributed data](/P/wald-mome)
 - [Mode of the continuous uniform distribution](/P/cuni-med)
 - [Mode of the exponential distribution](/P/exp-mode)
 - [Mode of the log-normal distribution](/P/lognorm-mode)
diff --git a/I/PwS.md b/I/PwS.md
@@ -46,6 +46,7 @@ title: "Proofs without Source"
 - [Gamma distribution is a special case of Wishart distribution](/P/gam-wish)
 - [Joint likelihood is the product of likelihood function and prior density](/P/jl-lfnprior)
 - [Kullback-Leibler divergence for the Bernoulli distribution](/P/bern-kl)
+- [Kullback-Leibler divergence for the continuous uniform distribution](/P/cuni-kl)
 - [Kullback-Leibler divergence for the matrix-normal distribution](/P/matn-kl)
 - [Kullback-Leibler divergence for the normal distribution](/P/norm-kl)
 - [Linear combination of independent normal random variables](/P/norm-lincomb)
@@ -84,6 +85,8 @@ title: "Proofs without Source"
 - [Median of the exponential distribution](/P/exp-med)
 - [Median of the log-normal distribution](/P/lognorm-med)
 - [Median of the normal distribution](/P/norm-med)
+- [Method of moments for ex-Gaussian distributed data](/P/exg-mome)
+- [Method of moments for Wald distributed data](/P/wald-mome)
 - [Mode of the continuous uniform distribution](/P/cuni-med)
 - [Mode of the exponential distribution](/P/exp-mode)
 - [Mode of the normal distribution](/P/norm-mode)
diff --git a/I/ToC.md b/I/ToC.md
@@ -136,7 +136,8 @@ title: "Table of Contents"
    
    1.12. Skewness <br>
    &emsp;&ensp; 1.12.1. *[Definition](/D/skew)* <br>
-   &emsp;&ensp; 1.12.2. **[Partition into expected values](/P/skew-mean)** <br>
+   &emsp;&ensp; 1.12.2. *[Sample skewness](/D/skew-samp)* <br>
+   &emsp;&ensp; 1.12.3. **[Partition into expected values](/P/skew-mean)** <br>
    
    1.13. Covariance <br>
    &emsp;&ensp; 1.13.1. *[Definition](/D/cov)* <br>
@@ -488,6 +489,7 @@ title: "Table of Contents"
    &emsp;&ensp; 3.10.4. **[Mean](/P/wald-mean)** <br>
    &emsp;&ensp; 3.10.5. **[Variance](/P/wald-var)** <br>
    &emsp;&ensp; 3.10.6. **[Skewness](/P/wald-skew)** <br>
+   &emsp;&ensp; 3.10.7. **[Method of moments](/P/wald-mome)** <br>
    
    3.11. ex-Gaussian distribution <br>
    &emsp;&ensp; 3.11.1. *[Definition](/D/exg)* <br>
@@ -496,6 +498,7 @@ title: "Table of Contents"
    &emsp;&ensp; 3.11.4. **[Mean](/P/exg-mean)** <br>
    &emsp;&ensp; 3.11.5. **[Variance](/P/exg-var)** <br>
    &emsp;&ensp; 3.11.6. **[Skewness](/P/exg-skew)** <br>
+   &emsp;&ensp; 3.11.7. **[Method of moments](/P/exg-mome)** <br>
 
 4. Multivariate continuous distributions
 
diff --git a/P/bvn-pdf.md b/P/bvn-pdf.md
@@ -40,7 +40,7 @@ $$ \label{eq:mvn-pdf}
 f_X(x) = \frac{1}{\sqrt{(2 \pi)^n |\Sigma|}} \cdot \exp \left[ -\frac{1}{2} (x-\mu)^\mathrm{T} \Sigma^{-1} (x-\mu) \right] \; .
 $$
 
-Plugging in $n = 2$ as well as $\mu$ and $\Sigma$ from equation \eqref{eq:bvn}, we obtain:
+Plugging in $n = 2$, $\mu = \left[ \begin{matrix} \mu_1 \\\\ \mu_2 \end{matrix} \right]$ and $\Sigma = \left[ \begin{matrix} \sigma_1^2 & \sigma_{12} \\\\ \sigma_{12} & \sigma_2^2 \end{matrix} \right]$, we obtain:
 
 $$ \label{eq:bvn-pdf-s1}
 \begin{split}
diff --git a/P/bvn-pdfcorr.md b/P/bvn-pdfcorr.md
@@ -27,7 +27,7 @@ username: "JoramSoch"
 ---
 
 
-**Theorem:** Let $X = \left[ \begin{matrix} X_1 & X_2 \end{matrix} \right]^\mathrm{T}$ follow a bivariate normal distribution:
+**Theorem:** Let $X = \left[ \begin{matrix} X_1 \\\\ X_2 \end{matrix} \right]$ follow a bivariate normal distribution:
 
 $$ \label{eq:bvn}
 X \sim \mathcal{N}\left( \mu = \left[ \begin{matrix} \mu_1 \\ \mu_2 \end{matrix} \right], \Sigma = \left[ \begin{matrix} \sigma_1^2 & \sigma_{12} \\ \sigma_{12} & \sigma_2^2 \end{matrix} \right] \right) \; .
@@ -42,7 +42,7 @@ $$
 where $\rho$ is the [correlation](/D/corr) between $X_1$ and $X_2$.
 
 
-**Proof:** Since $X$ follows a special case of the [multivariate normal distribution, its covariance matrix is](/P/mvn-corr)
+**Proof:** Since $X$ follows a special case of the [multivariate normal distribution, its covariance matrix is](/P/mvn-cov)
 
 $$ \label{eq:cov-X}
 \mathrm{Cov}(X) = \Sigma = \left[ \begin{matrix} \sigma_1^2 & \sigma_{12} \\ \sigma_{12} & \sigma_2^2 \end{matrix} \right]
diff --git a/P/cuni-kl.md b/P/cuni-kl.md
@@ -43,7 +43,7 @@ $$ \label{eq:KL-cont}
 \mathrm{KL}[P\,||\,Q] = \int_{\mathcal{X}} p(x) \, \ln \frac{p(x)}{q(x)} \, \mathrm{d}x \; .
 $$
 
-This means that the KL divergence of $P$ from $Q$ is only defined, if for all $x \in \mathcal{X}$, $q(x) = 0$ implies $p(x) = 0$. Thus, $\mathrm{KL}[P\,||\,Q]$ only exists, if $a_2 \leq a_1$ and $b_1 \leq b_2$, i.e. if $P$ only places non-zero probability where $Q$ also places non-zero probability, such that $q(x)$ is not zero for any $x \in \mathcal{X}$ where $p(x)$ is positive.
+This means that the KL divergence of $P$ from $Q$ is only defined, if for all $x \in \mathcal{X}$, $q(x) = 0$ implies $p(x) = 0$. Thus, $\mathrm{KL}[P\,\vert\vert\,Q]$ only exists, if $a_2 \leq a_1$ and $b_1 \leq b_2$, i.e. if $P$ only places non-zero probability where $Q$ also places non-zero probability, such that $q(x)$ is not zero for any $x \in \mathcal{X}$ where $p(x)$ is positive.
 
 If this requirement is fulfilled, we can write
 
@@ -57,7 +57,7 @@ $$ \label{eq:cuni-KL-s2}
 \mathrm{KL}[P\,||\,Q] = \int_{-\infty}^{a_1} 0 \cdot \ln \frac{0}{q(x)} \, \mathrm{d}x + \int_{a_1}^{b_1} p(x) \, \ln \frac{p(x)}{q(x)} \, \mathrm{d}x + \int_{b_1}^{+\infty} 0 \cdot \ln \frac{0}{q(x)} \, \mathrm{d}x \; .
 $$
 
-Now, $(0 \cdot \ln 0)$ is taken to be $0$ [by convention](/D/ent), such that
+Now, $(0 \cdot \ln 0)$ is taken to be [zero by convention](/D/ent), such that
 
 $$ \label{eq:cuni-KL-s3}
 \mathrm{KL}[P\,||\,Q] = \int_{a_1}^{b_1} p(x) \, \ln \frac{p(x)}{q(x)} \, \mathrm{d}x
diff --git a/P/exg-mome.md b/P/exg-mome.md
@@ -0,0 +1,113 @@
+---
+layout: proof
+mathjax: true
+
+author: "Thomas J. Faulkenberry"
+affiliation: "Tarleton State University"
+e_mail: "faulkenberry@tarleton.edu"
+date: 2023-10-30 12:00:00
+
+title: "Method of moments for ex-Gaussian distributed data"
+chapter: "Probability Distributions"
+section: "Univariate continuous distributions"
+topic: "ex-Gaussian distribution"
+theorem: "Method of moments"
+
+sources:
+  
+proof_id: "P424"
+shortcut: "exg-mome"
+username: "tomfaulkenberry"
+---
+  
+  
+
+**Theorem:** Let $y = \left\lbrace y_1, \ldots, y_n \right\rbrace$ be a set of observed data [independent and identically distributed](/D/iid) according to an [ex-Gaussian distribution](/D/exg) with parameters $\mu$, $\sigma$, and $\lambda$:
+
+$$ \label{eq:exq}
+y_i \sim \mathrm{ex-Gaussian}(\mu,\sigma,\lambda), \quad i = 1, \ldots, n \; .
+$$
+
+Then, the [method-of-moments estimates](/D/mome) for the parameters $\mu$, $\sigma$, and $\lambda$ are given by
+
+$$ \label{eq:exg-MoM}
+\begin{split}
+\hat{\mu} &= \bar{y} - \sqrt[3]{\frac{\bar{s}\cdot \bar{v}^{3/2}}{2}}\\
+\hat{\sigma} &= \sqrt{\bar{v}\cdot\left(1 - \sqrt[3]{\frac{\bar{s}^2}{4}}\right)}\\
+\hat{\lambda} &= \sqrt[3]{\frac{2}{\bar{s}\cdot \bar{v}^{3/2}}} \; ,
+\end{split}
+$$
+
+where $\bar{y}$ is the [sample mean](/D/mean-samp), $\bar{v}$ is the [sample variance](/D/var-samp) and $\bar{s}$ is the [sample skewness](/D/skew-samp)
+
+$$ \label{eq:y-mean-var-skew}
+\begin{split}
+\bar{y} &= \frac{1}{n} \sum_{i=1}^n y_i \\
+\bar{v} &= \frac{1}{n-1} \sum_{i=1}^n (y_i - \bar{y})^2 \\
+\bar{s} &= \frac{\frac{1}{n}\sum_{i=1}^n (y_i-\bar{y})^3}{\left[\frac{1}{n}\sum_{i=1}^n(y_i-\bar{y})^2\right]^{3/2}} \; .
+\end{split}
+$$
+
+
+**Proof:** The [mean](/P/exg-mean), [variance](/P/exg-var), and [skewness](/P/exg-skew) of the [ex-Gaussian distribution](/D/exg) in terms of the parameters $\mu$, $\sigma$, and $\lambda$ are given by
+
+$$ \label{eq:exg-E-Var-Skew}
+\begin{split}
+\mathrm{E}(X) &= \mu + \frac{1}{\lambda} \\
+\mathrm{Var}(X) &= \sigma^2 + \frac{1}{\lambda^2}\\
+\mathrm{Skew}(X) &= \frac{2}{\lambda^3\left(\sigma^2+\frac{1}{\lambda^2}\right)^{3/2}} \; .
+\end{split}
+$$
+
+Thus, [matching the moments](/D/mome) requires us to solve the following system of equations for $\mu$, $\sigma$, and $\lambda$:
+
+$$ \label{eq:exg-mean-var-skew}
+\begin{split}
+\bar{y} &= \mu + \frac{1}{\lambda} \\
+\bar{v} &= \sigma^2 + \frac{1}{\lambda^2}\\
+\bar{s} &= \frac{2}{\lambda^3\left(\sigma^2+\frac{1}{\lambda^2}\right)^{3/2}} \; .
+\end{split}
+$$
+
+To this end, our first step is to substitute the second equation of \eqref{eq:exg-mean-var-skew} into the third equation:
+
+$$ \label{eq:lambda-s1}
+\begin{split}
+\bar{s} &= \frac{2}{\lambda^3\left(\sigma^2+\frac{1}{\lambda^2}\right)^{3/2}} \\
+&= \frac{2}{\lambda^3\cdot \bar{v}^{3/2}} \; .
+\end{split}
+$$
+
+Re-expressing \eqref{eq:lambda-s1} in terms of $\lambda^3$ and taking the cube root gives:
+
+$$ \label{eq:lambda-s2}
+\lambda = \sqrt[3]{\frac{2}{\bar{s}\cdot \bar{v}^{3/2}}} \; .
+$$
+
+Next, we solve the first equation of \eqref{eq:exg-mean-var-skew} for $\mu$ and substitute \eqref{eq:lambda-s2}:
+
+$$ \label{eq:mu-s1}
+\begin{split}
+\mu &= \bar{y} - \frac{1}{\lambda}\\
+&= \bar{y} - \sqrt[3]{\frac{\bar{s}\cdot\bar{v}^{3/2}}{2}} \; .
+\end{split}
+$$
+
+Finally, we solve the second equation of \eqref{eq:exg-mean-var-skew} for $\sigma$:
+
+$$ \label{eq:sigma-s1}
+\sigma^2 = \bar{v} - \frac{1}{\lambda^2} \; .
+$$
+
+Taking the square root gives and substituting \eqref{eq:lambda-s2} gives:
+
+$$ \label{eq:sigma-s2}
+\begin{split}
+\sigma &= \sqrt{\bar{v}-\frac{1}{\lambda^2}} \\
+&= \sqrt{\bar{v} - \left(\sqrt[3]{\frac{\bar{s}\cdot \bar{v}^{3/2}}{2}}\right)^2} \\
+&= \sqrt{\bar{v} - \bar{v}\cdot\sqrt[3]{\frac{\bar{s}^2}{4}}} \\
+&= \sqrt{\bar{v}\cdot \left(1- \sqrt[3]{\frac{\bar{s}^2}{4}}\right)} \; .
+\end{split}
+$$
+
+Together, \eqref{eq:mu-s1}, \eqref{eq:sigma-s2}, and \eqref{eq:lambda-s2} constitute the method-of-moment estimates of $\mu$, $\sigma$, and $\lambda$.
diff --git a/P/f-pdf.md b/P/f-pdf.md
@@ -30,7 +30,7 @@ username: "JoramSoch"
 **Theorem:** Let $F$ be a [random variable](/D/rvar) following an [F-distribution](/D/f):
 
 $$ \label{eq:f}
-F \sim F(u,v) \; .
+F \sim \mathrm{F}(u,v) \; .
 $$
 
 Then, the [probability density function](/D/pdf) of $F$ is
@@ -43,7 +43,7 @@ $$
 **Proof:** An [F-distributed random variable](/D/f) is defined as the ratio of two [chi-squared random variables](/D/chi2), divided by their [degrees of freedom](/D/dof)
 
 $$ \label{eq:f-def}
-X \sim \chi^2(u), \; Y \sim \chi^2(v) \quad \Rightarrow \quad F = \frac{X/u}{Y/v} \sim F(u,v)
+X \sim \chi^2(u), \; Y \sim \chi^2(v) \quad \Rightarrow \quad F = \frac{X/u}{Y/v} \sim \mathrm{F}(u,v)
 $$
 
 where $X$ and $Y$ are [independent of each other](/D/ind).
diff --git a/P/wald-mome.md b/P/wald-mome.md
