Merge pull request #157 from StatProofBook/master

JoramSoch · web-flow · commit 6261cf949319 · 2024-11-15T12:01:36.000+01:00
update to master
diff --git a/D/est-bias.md b/D/est-bias.md
@@ -29,6 +29,6 @@ username: "JoramSoch"
 
 **Definition:** Let $\hat{\theta}: \mathcal{Y} \rightarrow \Theta$ be an [estimator](/D/est) of a [parameter](/D/para) $\theta \in \Theta$ from [data](/D/data) $y \in \mathcal{Y}$. Then,
 
-* $\hat{\theta}$ is called an unbiased estimator when its [expected value](/D/mean) is equal to the parameter that it is estimating: $\mathrm{E}_{\hat{\theta}}\left[ \hat{\theta} \right] = \theta$, where the expectation is calculated over all possible samples $y$ leading to values of $\hat{\theta}$.
+* $\hat{\theta}$ is called an unbiased estimator when its [expected value](/D/mean) is equal to the parameter that it is estimating: $\mathrm{E}_{\hat{\theta}}(\hat{\theta}) = \theta$, where the expectation is calculated over all possible samples $y$ leading to values of $\hat{\theta}$.
 
-* $\hat{\theta}$ is called a biased estimator otherwise, i.e. when $\mathrm{E}_{\hat{\theta}}\left[ \hat{\theta} \right] \neq \theta$.
+* $\hat{\theta}$ is called a biased estimator otherwise, i.e. when $\mathrm{E}_{\hat{\theta}}(\hat{\theta}) \neq \theta$.
diff --git a/D/est.md b/D/est.md
@@ -14,6 +14,12 @@ topic: "Basic concepts of estimation"
 definition: "Estimator"
 
 sources:
+  - authors: "Ostwald, Dirk"
+    year: 2023
+    title: "Punktschätzung"
+    in: "Wahrscheinlichkeitstheorie und Frequentistische Inferenz"
+    pages: "Einheit (9), Folie 7"
+    url: "https://www.ipsy.ovgu.de/ipsy_media/Methodenlehre+I/Wintersemester+2324/Wahrscheinlichkeitstheorie+und+Frequentistische+Inferenz/9_Punktsch%C3%A4tzung.pdf"
   - authors: "Wikipedia"
     year: 2024
     title: "Estimator"
diff --git a/D/nw.md b/D/nw.md
@@ -14,6 +14,12 @@ topic: "Normal-Wishart distribution"
 definition: "Definition"
 
 sources:
+  - authors: "Bishop, Christopher M."
+    year: 2006
+    title: "Appendix B. Probability Distributions"
+    in: "Pattern Recognition for Machine Learning"
+    pages: "p. 690, eq. B.53"
+    url: "http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf"
 
 def_id: "D175"
 shortcut: "nw"
diff --git a/I/DbA.md b/I/DbA.md
@@ -13,7 +13,7 @@ title: "Definition by Author"
 
 - [Sample covariance](/D/cov-samp)
 
-### JoramSoch (192 definitions)
+### JoramSoch (193 definitions)
 
 - [Akaike information criterion](/D/aic)
 - [Alternative hypothesis](/D/h1)
@@ -24,6 +24,7 @@ title: "Definition by Author"
 - [Beta-binomial data](/D/betabin-data)
 - [Beta-binomial distribution](/D/betabin)
 - [Beta-distributed data](/D/beta-data)
+- [Biased vs. unbiased estimator](/D/est-bias)
 - [Binomial distribution](/D/bin)
 - [Binomial observations](/D/bin-data)
 - [Bivariate normal distribution](/D/bvn)
diff --git a/I/DbN.md b/I/DbN.md
@@ -215,3 +215,4 @@ title: "Definition by Number"
 | D206 | dof | [Degrees of freedom](/D/dof) | JoramSoch | 2024-10-11 |
 | D207 | dist-uni | [Unimodal and multimodal probability distribution](/D/dist-uni) | JoramSoch | 2024-10-25 |
 | D208 | est | [Estimand, estimator and estimate](/D/est) | JoramSoch | 2024-11-01 |
+| D209 | est-bias | [Biased vs. unbiased estimator](/D/est-bias) | JoramSoch | 2024-11-08 |
diff --git a/I/DbT.md b/I/DbT.md
@@ -19,6 +19,7 @@ title: "Definition by Topic"
 - [Beta-binomial data](/D/betabin-data)
 - [Beta-binomial distribution](/D/betabin)
 - [Beta-distributed data](/D/beta-data)
+- [Biased vs. unbiased estimator](/D/est-bias)
 - [Binomial distribution](/D/bin)
 - [Binomial observations](/D/bin-data)
 - [Bivariate normal distribution](/D/bvn)
diff --git a/I/DwS.md b/I/DwS.md
@@ -19,7 +19,6 @@ title: "Definitions without Source"
 - [Maximum likelihood estimation](/D/mle)
 - [Maximum log-likelihood](/D/mll)
 - [Multinomial observations](/D/mult-data)
-- [Normal-Wishart distribution](/D/nw)
 - [Posterior distribution](/D/post)
 - [Posterior predictive distribution](/D/post-pred)
 - [Prior distribution](/D/prior)
diff --git a/I/PbA.md b/I/PbA.md
@@ -12,7 +12,7 @@ title: "Proof by Author"
 
 - [Posterior predictive distribution is a marginal distribution of the joint likelihood](/P/postpred-jl)
 
-### JoramSoch (441 proofs)
+### JoramSoch (442 proofs)
 
 - [Accuracy and complexity for Bayesian linear regression](/P/blr-anc)
 - [Accuracy and complexity for Bayesian linear regression with known covariance](/P/blrkc-anc)
@@ -248,6 +248,7 @@ title: "Proof by Author"
 - [Mode of the normal distribution](/P/norm-mode)
 - [Moment in terms of moment-generating function](/P/mom-mgf)
 - [Moment-generating function of a function of a random variable](/P/mgf-fct)
+- [Moment-generating function of a sum of independent random variables](/P/mgf-sumind)
 - [Moment-generating function of linear combination of independent random variables](/P/mgf-lincomb)
 - [Moment-generating function of the beta distribution](/P/beta-mgf)
 - [Moment-generating function of the gamma distribution](/P/gam-mgf)
diff --git a/I/PbN.md b/I/PbN.md
@@ -483,3 +483,4 @@ title: "Proof by Number"
 | P475 | bvn-lincomb | [Linear combination of bivariate normal random variables](/P/bvn-lincomb) | JoramSoch | 2024-10-25 |
 | P476 | bvn-mi | [Mutual information of the bivariate normal distribution](/P/bvn-mi) | JoramSoch | 2024-11-01 |
 | P477 | mvn-mi | [Mutual information of the multivariate normal distribution](/P/mvn-mi) | JoramSoch | 2024-11-01 |
+| P478 | mgf-sumind | [Moment-generating function of a sum of independent random variables](/P/mgf-sumind) | JoramSoch | 2024-11-08 |
diff --git a/I/PbT.md b/I/PbT.md
@@ -286,6 +286,7 @@ title: "Proof by Topic"
 - [Mode of the normal distribution](/P/norm-mode)
 - [Moment in terms of moment-generating function](/P/mom-mgf)
 - [Moment-generating function of a function of a random variable](/P/mgf-fct)
+- [Moment-generating function of a sum of independent random variables](/P/mgf-sumind)
 - [Moment-generating function of linear combination of independent random variables](/P/mgf-lincomb)
 - [Moment-generating function of the beta distribution](/P/beta-mgf)
 - [Moment-generating function of the ex-Gaussian distribution](/P/exg-mgf)
diff --git a/I/PwS.md b/I/PwS.md
@@ -127,7 +127,6 @@ title: "Proofs without Source"
 - [Probability density function of the multivariate normal distribution](/P/mvn-pdf)
 - [Probability density function of the multivariate t-distribution](/P/mvt-pdf)
 - [Probability density function of the normal distribution](/P/norm-pdf)
-- [Probability density function of the normal-Wishart distribution](/P/nw-pdf)
 - [Probability density function of the Wald distribution](/P/wald-pdf)
 - [Probability mass function of the Bernoulli distribution](/P/bern-pmf)
 - [Probability mass function of the binomial distribution](/P/bin-pmf)
diff --git a/P/mgf-lincomb.md b/P/mgf-lincomb.md
@@ -38,7 +38,7 @@ where $a_1, \ldots, a_n$ are $n$ real numbers.
 
 **Proof:** The [moment-generating function of a random variable](/D/mgf) $X_i$ is
 
-$$ \label{eq:mfg-vect}
+$$ \label{eq:mfg}
 M_{X_i}(t) = \mathrm{E} \left( \exp \left[ t X_i \right] \right)
 $$
 
diff --git a/P/mle-bias.md b/P/mle-bias.md
@@ -21,7 +21,7 @@ username: "JoramSoch"
 ---
 
 
-**Theorem:** [Maximum likelihood estimation](/D/mle) can result in [biased estimates](/D/est-unb) of model parameters, i.e. estimates whose long-term expected value is unequal to the quantities they estimate:
+**Theorem:** [Maximum likelihood estimation](/D/mle) can result in [biased estimates](/D/est-bias) of model parameters, i.e. estimates whose long-term expected value is unequal to the quantities they estimate:
 
 $$ \label{eq:aicc-aic}
 \mathrm{E}\left[ \hat{\theta}_\mathrm{MLE} \right] = \mathrm{E}\left[ \operatorname*{arg\,max}_\theta \mathrm{LL}_m(\theta) \right] \neq \theta \; .
diff --git a/P/mlr-olstr.md b/P/mlr-olstr.md
@@ -39,7 +39,7 @@ $$
 2) and, if the two regressors are orthogonal to each other, they simplify to
 
 $$ \label{eq:mlr-ols-tr-orth}
-\hat{\beta}_1 = \frac{x_1^\mathrm{T} y}{x_1^\mathrm{T} x_1} \quad \text{and} \quad \hat{\beta}_2 = \frac{x_2^\mathrm{T} y}{x_2^\mathrm{T} x_2}, \quad \text{if} \quad x_1 \perp x_2 \; .
+\hat{\beta}_1 = \frac{x_1^\mathrm{T} y}{x_1^\mathrm{T} x_1} \quad \text{and} \quad \hat{\beta}_2 = \frac{x_2^\mathrm{T} y}{x_2^\mathrm{T} x_2} \; .
 $$
 
 
diff --git a/P/nw-pdf.md b/P/nw-pdf.md
@@ -14,6 +14,12 @@ topic: "Normal-Wishart distribution"
 theorem: "Probability density function"
 
 sources:
+  - authors: "Bishop, Christopher M."
+    year: 2006
+    title: "Appendix B. Probability Distributions"
+    in: "Pattern Recognition for Machine Learning"
+    pages: "p. 690, eq. B.53"
+    url: "http://users.isr.ist.utl.pt/~wurmd/Livros/school/Bishop%20-%20Pattern%20Recognition%20And%20Machine%20Learning%20-%20Springer%20%202006.pdf"
 
 proof_id: "P323"
 shortcut: "nw-pdf"
diff --git a/P/resvar-bias.md b/P/resvar-bias.md
@@ -46,7 +46,7 @@ $$ \label{eq:mean-mle}
 \bar{y} = \frac{1}{n} \sum_{i=1}^{n} y_i
 $$
 
-2) and $\hat{\sigma}^2$ is a [biased estimator](/D/est-unb) of $\sigma^2$
+2) and $\hat{\sigma}^2$ is a [biased estimator](/D/est-bias) of $\sigma^2$
 
 $$ \label{eq:resvar-var}
 \mathrm{E}\left[ \hat{\sigma}^2 \right] \neq \sigma^2 \; ,
@@ -144,4 +144,4 @@ $$ \label{eq:E-resvar-mle-s3}
 \end{split}
 $$
 
-which proves the [bias](/D/est-unb) given by \eqref{eq:resvar-bias}.
+which proves the [bias](/D/est-bias) given by \eqref{eq:resvar-bias}.
diff --git a/P/resvar-biasp.md b/P/resvar-biasp.md
@@ -47,7 +47,7 @@ $$ \label{eq:beta-mle}
 \hat{\beta} = (X^\mathrm{T} V^{-1} X)^{-1} X^\mathrm{T} V^{-1} y
 $$
 
-2) and $\hat{\sigma}^2$ is a [biased estimator](/D/est-unb) of $\sigma^2$
+2) and $\hat{\sigma}^2$ is a [biased estimator](/D/est-bias) of $\sigma^2$
 
 $$ \label{eq:resvar-var}
 \mathrm{E}\left[ \hat{\sigma}^2 \right] \neq \sigma^2 \; ,
diff --git a/P/resvar-unb.md b/P/resvar-unb.md
@@ -32,7 +32,7 @@ $$ \label{eq:ug}
 y_i \overset{\text{i.i.d.}}{\sim} \mathcal{N}(\mu, \sigma^2), \quad i = 1,\ldots,n \; .
 $$
 
-An [unbiased estimator](/D/est-unb) of $\sigma^2$ is given by
+An [unbiased estimator](/D/est-bias) of $\sigma^2$ is given by
 
 $$ \label{eq:resvar-unb}
 \hat{\sigma}^2_{\mathrm{unb}} = \frac{1}{n-1} \sum_{i=1}^{n} \left( y_i - \bar{y} \right)^2 \; .
@@ -45,7 +45,7 @@ $$ \label{eq:resvar-mle}
 \hat{\sigma}^2_{\mathrm{MLE}} = \frac{1}{n} \sum_{i=1}^{n} \left( y_i - \bar{y} \right)^2
 $$
 
-is a [biased estimator](/D/est-unb) in the sense that
+is a [biased estimator](/D/est-bias) in the sense that
 
 $$ \label{eq:resvar-bias}
 \mathbb{E}\left[ \hat{\sigma}^2_{\mathrm{MLE}} \right] = \frac{n-1}{n} \sigma^2 \; .
@@ -61,7 +61,7 @@ $$ \label{eq:resvar-bias-adj}
 \end{split}
 $$
 
-such that an [unbiased estimator](/D/est-unb) can be constructed as
+such that an [unbiased estimator](/D/est-bias) can be constructed as
 
 $$ \label{eq:resvar-unb-qed}
 \begin{split}
diff --git a/P/resvar-unbp.md b/P/resvar-unbp.md
@@ -27,7 +27,7 @@ $$ \label{eq:mlr}
 y = X\beta + \varepsilon, \; \varepsilon \sim \mathcal{N}(0, \sigma^2 V) \; .
 $$
 
-An [unbiased estimator](/D/est-unb) of $\sigma^2$ is given by
+An [unbiased estimator](/D/est-bias) of $\sigma^2$ is given by
 
 $$ \label{eq:sigma-unb}
 \hat{\sigma}^2 = \frac{1}{n-p} (y-X\hat{\beta})^\mathrm{T} V^{-1} (y-X\hat{\beta})
@@ -46,7 +46,7 @@ $$ \label{eq:resvar-mle}
 \hat{\sigma}^2_{\mathrm{MLE}} = \frac{1}{n} (y-X\hat{\beta})^\mathrm{T} V^{-1} (y-X\hat{\beta})
 $$
 
-is a [biased estimator](/D/est-unb) in the sense that
+is a [biased estimator](/D/est-bias) in the sense that
 
 $$ \label{eq:resvar-bias}
 \mathbb{E}\left[ \hat{\sigma}^2_{\mathrm{MLE}} \right] = \frac{n-p}{n} \sigma^2 \; .
@@ -62,7 +62,7 @@ $$ \label{eq:resvar-bias-adj}
 \end{split}
 $$
 
-such that an [unbiased estimator](/D/est-unb) can be constructed as
+such that an [unbiased estimator](/D/est-bias) can be constructed as
 
 $$ \label{eq:resvar-unb-qed}
 \begin{split}
diff --git a/P/slr-olsmean.md b/P/slr-olsmean.md
@@ -48,7 +48,7 @@ $$ \label{eq:slr-ols-mean}
 \end{split}
 $$
 
-which means that the [ordinary least squares solution](/P/slr-ols) produces [unbiased estimators](/D/est-unb).
+which means that the [ordinary least squares solution](/P/slr-ols) produces [unbiased estimators](/D/est-bias).
 
 
 **Proof:** According to the simple linear regression model in \eqref{eq:slr}, the expectation of a single data point is
