Merge pull request #151 from StatProofBook/master

JoramSoch · web-flow · commit f15198fd3f65 · 2024-09-27T12:46:09.000+02:00
update to master
diff --git a/I/DbA.md b/I/DbA.md
@@ -13,7 +13,7 @@ title: "Definition by Author"
 
 - [Sample covariance](/D/cov-samp)
 
-### JoramSoch (186 definitions)
+### JoramSoch (187 definitions)
 
 - [Akaike information criterion](/D/aic)
 - [Alternative hypothesis](/D/h1)
@@ -51,6 +51,7 @@ title: "Definition by Author"
 - [Cross-validated log model evidence](/D/cvlme)
 - [Cumulant-generating function](/D/cgf)
 - [Cumulative distribution function](/D/cdf)
+- [Data](/D/data)
 - [Deviance](/D/dev)
 - [Deviance information criterion](/D/dic)
 - [Differential cross-entropy](/D/dent-cross)
diff --git a/I/DbN.md b/I/DbN.md
@@ -209,3 +209,4 @@ title: "Definition by Number"
 | D200 | iid | [independent and identically distributed](/D/iid) | JoramSoch | 2024-08-08 |
 | D201 | post-pred | [Posterior predictive distribution](/D/post-pred) | aloctavodia | 2024-08-18 |
 | D202 | prior-pred | [Prior predictive distribution](/D/prior-pred) | aloctavodia | 2024-08-19 |
+| D203 | data | [Data](/D/data) | JoramSoch | 2024-09-20 |
diff --git a/I/DbT.md b/I/DbT.md
@@ -54,6 +54,7 @@ title: "Definition by Topic"
 
 ### D
 
+- [Data](/D/data)
 - [Definition Template](/D/-temp-)
 - [Deviance](/D/dev)
 - [Deviance information criterion](/D/dic)
diff --git a/I/DwS.md b/I/DwS.md
@@ -8,6 +8,7 @@ title: "Definitions without Source"
 - [Beta-distributed data](/D/beta-data)
 - [Binomial observations](/D/bin-data)
 - [Conditional differential entropy](/D/dent-cond)
+- [Data](/D/data)
 - [Dirichlet-distributed data](/D/dir-data)
 - [Estimation matrix](/D/emat)
 - [Joint differential entropy](/D/dent-joint)
diff --git a/I/PbA.md b/I/PbA.md
@@ -12,7 +12,7 @@ title: "Proof by Author"
 
 - [Posterior predictive distribution is a marginal distribution of the joint likelihood](/P/postpred-jl)
 
-### JoramSoch (434 proofs)
+### JoramSoch (435 proofs)
 
 - [Accuracy and complexity for Bayesian linear regression](/P/blr-anc)
 - [Accuracy and complexity for Bayesian linear regression with known covariance](/P/blrkc-anc)
@@ -408,6 +408,7 @@ title: "Proof by Author"
 - [Scaling of the variance upon multiplication with a constant](/P/var-scal)
 - [Second central moment is variance](/P/momcent-2nd)
 - [Self-covariance equals variance](/P/cov-var)
+- [Self-independence of random event](/P/ind-self)
 - [Simple linear regression is a special case of multiple linear regression](/P/slr-mlr)
 - [Specific t-test for single regressor in multiple linear regression](/P/mlr-tsingle)
 - [Square of expectation of product is less than or equal to product of expectation of squares](/P/mean-prodsqr)
diff --git a/I/PbN.md b/I/PbN.md
@@ -475,3 +475,4 @@ title: "Proof by Number"
 | P467 | postpred-jl | [Posterior predictive distribution is a marginal distribution of the joint likelihood](/P/postpred-jl) | aloctavodia | 2024-09-11 |
 | P468 | mean-wlln | [Weak law of large numbers](/P/mean-wlln) | JoramSoch | 2024-09-13 |
 | P469 | mean-mse | [The expected value minimizes the mean squared error](/P/mean-mse) | salbalkus | 2024-09-13 |
+| P470 | ind-self | [Self-independence of random event](/P/ind-self) | JoramSoch | 2024-09-20 |
diff --git a/I/PbT.md b/I/PbT.md
@@ -476,6 +476,7 @@ title: "Proof by Topic"
 - [Scaling of the variance upon multiplication with a constant](/P/var-scal)
 - [Second central moment is variance](/P/momcent-2nd)
 - [Self-covariance equals variance](/P/cov-var)
+- [Self-independence of random event](/P/ind-self)
 - [Simple linear regression is a special case of multiple linear regression](/P/slr-mlr)
 - [Skewness of the ex-Gaussian distribution](/P/exg-skew)
 - [Skewness of the exponential distribution](/P/exp-skew)
diff --git a/I/ToC.md b/I/ToC.md
diff --git a/P/ind-self.md b/P/ind-self.md
@@ -20,7 +20,7 @@ sources:
     in: "Wikipedia, the free encyclopedia"
     pages: "retrieved on 2024-09-20"
     url: "https://en.wikipedia.org/wiki/Independence_(probability_theory)#Self-independence"
-  - authors: "JoramSoch"
+  - authors: "Soch, Joram"
     year: 2023
     title: "Suppose A is an event. Can A be independent of itself?"
     in: "X"
@@ -49,7 +49,7 @@ P(E)   &= \left( P(E) \right)^2 \; .
 \end{split}
 $$
 
-This is only fulfilled, if
+For $0 \leq P(E) \leq 1$, this is only fulfilled, if
 
 $$ \label{eq:ind-self-qed}
 P(E) = 0 \quad \text{or} \quad P(E) = 1 \; .
diff --git a/P/mean-mse.md b/P/mean-mse.md
@@ -43,7 +43,7 @@ $$
 Setting the first derivative
 
 $$ \label{eq:dmse-da}
-\frac{d}{da} \left[ a^2 - 2a\mu + \mathrm{E}(X_i^2) \right] = 2a - 2\mu
+\frac{\mathrm{d}}{\mathrm{d}a} \left[ a^2 - 2a\mu + \mathrm{E}(X_i^2) \right] = 2a - 2\mu
 $$
 
 to zero to perform a derivative test, we obtain:
diff --git a/P/med-mae.md b/P/med-mae.md
@@ -0,0 +1,92 @@
+---
+layout: proof
+mathjax: true
+
+author: "Salvador Balkus"
+affiliation: "Harvard T.H. Chan School of Public Health"
+e_mail: "sbalkus@g.harvard.edu"
+date: 2024-09-23 23:30:00
+
+title: "The median minimizes the mean absolute error"
+chapter: "General Theorems"
+section: "Probability theory"
+topic: "Measures of central tendency"
+theorem: "Median minimizes mean absolute error"
+
+sources:
+  - authors: "Wikipedia"
+    year: 2024
+    title: "Derivative test"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2024-09-23"
+    url: "https://en.wikipedia.org/wiki/Derivative_test"
+  - authors: "Wikipedia"
+    year: 2024
+    title: "Leibniz integral rule"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2024-09-23"
+    url: "https://en.wikipedia.org/wiki/Leibniz_integral_rule"
+  - authors: "Wikipedia"
+    year: 2024
+    title: "Jensen's inequality"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2024-09-23"
+    url: "https://en.wikipedia.org/wiki/Jensen%27s_inequality"
+  - authors: "Wikipedia"
+    year: 2024
+    title: "Convex function"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2024-09-23"
+    url: "https://en.wikipedia.org/wiki/Convex_function"
+
+proof_id: "P471"
+shortcut: "med-mae"
+username: "salbalkus"
+---
+
+
+**Theorem:** Let $X_1, \ldots, X_n$ be a collection of [continuous](/D/rvar-disc) [random variables](/D/rvar) drawn from a [probability distribution](/D/dist) with the [probability density function](/D/pdf) $f(x)$ supported on $(-\infty, \infty)$ with common [median](/D/med) $m$. Then, $m$ minimizes the mean absolute error:
+
+$$ \label{eq:med-mae}
+m = \operatorname*{arg\,min}_{a \in \mathbb{R}} \mathrm{E}\left[ \lvert X_i - a \rvert \right] \; .
+$$
+
+
+**Proof:** We can find the optimum by performing a derivative test. First, since an absolute value function is not differentaible at 0, we simplify the objective function by splitting it into two separate integrals:
+
+$$ \label{eq:med-mae-s1}
+E(\lvert X_i - a \rvert) = \int_{-\infty}^a (a - x) f(x) \, \mathrm{d}x + \int_{a}^\infty (x - a) f(x) \, \mathrm{d}x \; .
+$$
+
+Now note that $\lvert\frac{\partial}{\partial a}(a - x)f(x)\rvert = \lvert\frac{\partial}{\partial a}(x - a)f(x)\rvert = f(x)$. Consequently, $\int_{-\infty}^af(x) = P(X_i < a)$ and $\int_{a}^\infty f(x) = P(X_i > a)$, both of which must be finite by the [axioms of probability](/D/prob-ax). Therefore, these integrals meet the conditions for application of Leibniz's rule.
+
+Applying Leibniz's rule, we can differentiate the objective function as follows:
+
+$$ \label{eq:med-mae-s2}
+\begin{split}
+  & \frac{\partial}{\partial a} \left( \int_{-\infty}^a (a - x) f(x) \, \mathrm{d}x + \int_{a}^\infty (x - a) f(x) \, \mathrm{d}x \right) \\
+= & (a - x) f(x) + \int_{-\infty}^a f(x) \, \mathrm{d}x - (x - a) f(x) - \int_{a}^\infty f(x) \, \mathrm{d}x \; .
+\end{split}
+$$
+
+Canceling terms and setting this derivative to 0, it must be true that
+
+$$\label{eq:dmed-da}
+\int_{-\infty}^a f(x) \, \mathrm{d}x - \int_{a}^\infty f(x) \, \mathrm{d}x = 0
+\quad \Rightarrow \quad
+P(X_i < a) = P(X_i > a) \; .
+$$
+
+This yields the implication
+
+$$\label{eq:med-mae-qed}
+P(X_i < a) = P(X_i > a)
+\quad \Rightarrow \quad 
+P(X_i < a) = 1 - P(X_i < a)
+\quad \Rightarrow \quad
+P(X_i < a) = 0.5
+$$
+
+As a result, $a$ satisfies the [definition of a median](/D/med) at the critical point of the objective function.
+
+Finally, the absolute value is a convex function, and so is its expected value by Jensen's inequality. This implies, since the median is the sole critical point, it must be a global minimum. Therefore, the median must minimize the mean absolute error, completing the proof.
diff --git a/P/mlr-fomnibus.md b/P/mlr-fomnibus.md
@@ -62,7 +62,7 @@ under the [null hypothesis](/D/h0) that all [regression coefficients](/D/mlr) $\
 
 $$ \label{eq:mlr-f-omnibus-h0}
 H_0: \; \beta_1 = 0_{p_1} \quad \Leftrightarrow \quad
-\beta_j = 0 \quad \text{for all} \quad j = (p_0+1),\ldots,p \; .
+\beta_{1j} = 0 \quad \text{for all} \quad j=1,\ldots,p_1 \; .
 $$
 
 In \eqref{eq:mlr-f-omnibus}, $\hat{\varepsilon}$ and $\hat{\varepsilon}_0$ are the [residual vectors](/P/mlr-mat) when using either the full design matrix $X$ or the reduced design matrix $X_0$:
diff --git a/P/mlr-llr.md b/P/mlr-llr.md
@@ -15,7 +15,7 @@ theorem: "Log-likelihood ratio"
 
 sources:
   - authors: "Ostwald, Dirk"
-    year: 2024
+    year: 2023
     title: "F-Statistiken"
     in: "Allgemeines Lineares Modell"
     pages: "Einheit (8), Folien 20-22"
diff --git a/P/mlr-olsdist.md b/P/mlr-olsdist.md
@@ -27,6 +27,18 @@ sources:
     in: "Mathematics for Brain Imaging"
     pages: "ch. 1.5, pp. 39-41, eqs. 1.106-1.110"
     url: "https://ueapsylabs.co.uk/sites/wpenny/mbi/mbi_course.pdf"
+  - authors: "Ostwald, Dirk"
+    year: 2023
+    title: "Modellformulierung"
+    in: "Allgemeines Lineares Modell"
+    pages: "Einheit (5), Folie 14"
+    url: "https://www.ipsy.ovgu.de/ipsy_media/Methodenlehre+I/Sommersemester+2023/Allgemeines+Lineares+Modell/5_Modellformulierung.pdf"
+  - authors: "Ostwald, Dirk"
+    year: 2023
+    title: "Parameterschätzung"
+    in: "Allgemeines Lineares Modell"
+    pages: "Einheit (6), Folien 10-12"
+    url: "https://www.ipsy.ovgu.de/ipsy_media/Methodenlehre+I/Sommersemester+2023/Allgemeines+Lineares+Modell/6_Parametersch%C3%A4tzung.pdf"
 
 proof_id: "P400"
 shortcut: "mlr-olsdist"
diff --git a/_layouts/definition.html b/_layouts/definition.html
@@ -16,7 +16,7 @@ <h1 class="post-title">Definition: {{ page.title }}</h1>
 	<a href="https://StatProofBook.github.io">The Book of Statistical Proofs</a> &#x25B7;
 	<a href="https://statproofbook.github.io/I/ToC#{{ page.chapter }}">{{ page.chapter }}</a> &#x25B7;
 	<a href="https://statproofbook.github.io/I/ToC#{{ page.section }}">{{ page.section }}</a> &#x25B7;
-	{{ page.topic }} &#x25B7;
+	<a href="https://statproofbook.github.io/I/ToC#{{ page.topic }}">{{ page.topic }}</a> &#x25B7;
 	{{ page.definition }} <br><br>
 	
 	<!-- Definition -->
diff --git a/_layouts/proof.html b/_layouts/proof.html
@@ -16,7 +16,7 @@ <h1 class="post-title">Proof: {{ page.title }}</h1>
 	<a href="https://StatProofBook.github.io">The Book of Statistical Proofs</a> &#x25B7;
 	<a href="https://statproofbook.github.io/I/ToC#{{ page.chapter }}">{{ page.chapter }}</a> &#x25B7;
 	<a href="https://statproofbook.github.io/I/ToC#{{ page.section }}">{{ page.section }}</a> &#x25B7;
-	{{ page.topic }} &#x25B7;
+	<a href="https://statproofbook.github.io/I/ToC#{{ page.topic }}">{{ page.topic }}</a> &#x25B7;
 	{{ page.theorem }} <br><br>
 	
 	<!-- Theorem & Proof -->
