Merge pull request #270 from salbalkus/master

added "mean-ls"

Author: StatProofBook

I/ToC.md

@@ -122,6 +122,7 @@ title: "Table of Contents"
    &emsp;&ensp; 1.10.13. **[Weak law of large numbers](/P/mean-wlln)** <br>
    &emsp;&ensp; 1.10.14. *[Expected value of a random vector](/D/mean-rvec)* <br>
    &emsp;&ensp; 1.10.15. *[Expected value of a random matrix](/D/mean-rmat)* <br>
+   &emsp;&ensp; 1.10.16. **[Expected value minimizes expected squared error](/P/mean-ls)** <br>
 
    1.11. Variance <br>
    &emsp;&ensp; 1.11.1. *[Definition](/D/var)* <br>
@@ -907,4 +908,4 @@ title: "Table of Contents"
    3.5. Bayesian model averaging <br>
    &emsp;&ensp; 3.5.1. *[Definition](/D/bma)* <br>
    &emsp;&ensp; 3.5.2. **[Derivation](/P/bma-der)** <br>
-   &emsp;&ensp; 3.5.3. **[Calculation from log model evidences](/P/bma-lme)** <br>
+   &emsp;&ensp; 3.5.3. **[Calculation from log model evidences](/P/bma-lme)** <br>

P/mean-ls.md

@@ -0,0 +1,42 @@
+---
+layout: proof
+mathjax: true
+
+author: "Salvador Balkus"
+affiliation: "Harvard T.H. Chan School of Public Health"
+e_mail: "sbalkus@g.harvard.edu"
+date: 2024-09-13 23:30:00
+
+title: "Expected value minimizes expected squared error"
+chapter: "General Theorems"
+section: "Probability theory"
+topic: "Expected value"
+theorem: "Expected value minimizes expected squared error"
+
+sources:
+
+proof_id: "P468"
+shortcut: "mean-ls"
+username: "salbalkus"
+---
+
+
+**Theorem:** Let $X_1, X_2, \ldots, X_n$ be a collection of [random variables](/D/rvar) with common [mean](/D/mean) $E(X_i) = \mu$. Then, $\mu$ minimizes the mean squared error; that is,
+
+$$
+  \text{argmin}_{a \in \mathbb{R}} E\Big((X_i - a)^2\Big) = \mu
+$$
+
+**Proof:** Using the [linearity of expectation](/P/mean-lin) we can simplify the objective function like so:
+
+$$
+E\Big((X_i - a)^2\Big) = E\Big(X_i^2 - 2aX_i + a^2\Big) = a^2 - 2a\mu + E(X_i^2)
+$$
+
+Setting $\frac{d}{da}a^2 - 2a\mu + E(X_i^2) = 0$ to perform a [derivative test](https://en.wikipedia.org/wiki/Derivative_test), we can compute the derivative to obtain
+
+$$
+2a - 2\mu = 0 \implies a = \mu
+$$
+
+The second derivative is equal to 2, which is greater than 0; since this is the sole critical point, we can conclude by the second derivative test that this value is the unique global minimum. This completes the proof that $\text{argmin}_{a \in \mathbb{R}} E\Big((X_i - a)^2\Big) = \mu$.