Merge pull request #40 from StatProofBook/master

JoramSoch · web-flow · commit 3e33d83aa3a0 · 2020-11-19T09:00:44.000+01:00
update to master
diff --git a/D/cdf.md b/D/cdf.md
@@ -34,19 +34,15 @@ F_X(x) = \mathrm{Pr}(X \leq x) \; .
 $$
 
 <br>
-1) Let $X$ be a discrete [random variable](/D/rvar) with possible outcomes $\mathcal{X}$ and the [probability mass function](/D/pmf) $f_X(x)$. Then, the function $F_X(x): \mathbb{R} \to [0,1]$ with
+1) If $X$ is a [discrete](/D/rvar-disc) [random variable](/D/rvar) with possible outcomes $\mathcal{X}$ and the [probability mass function](/D/pmf) $f_X(x)$, then [the cumulative distribution function is the function](/P/cdf-pmf) $F_X(x): \mathbb{R} \to [0,1]$ with
 
 $$ \label{eq:cdf-disc}
-F_X(x) = \sum_{\overset{z \in \mathcal{X}}{z \leq x}} f_X(z)
+F_X(x) = \sum_{\overset{t \in \mathcal{X}}{t \leq x}} f_X(t) \; .
 $$
 
-is the cumulative distribution function of $X$.
-
 <br>
-2) Let $X$ be a scalar continuous [random variable](/D/rvar) with the [probability density function](/D/pdf) $f_X(x)$. Then, the function $F_X(x): \mathbb{R} \to [0,1]$ with
+2) If $X$ is a [continuous](/D/rvar-disc) [random variable](/D/rvar) with possible outcomes $\mathcal{X}$ and the [probability density function](/D/pdf) $f_X(x)$, then [the cumulative distribution function is the function](/P/cdf-pdf) $F_X(x): \mathbb{R} \to [0,1]$ with
 
 $$ \label{eq:cdf-cont}
-F_X(x) = \int_{-\infty}^{x} f_X(z) \, \mathrm{d}z
-$$
-
-is the cumulative distribution function of $X$.
+F_X(x) = \int_{-\infty}^{x} f_X(t) \, \mathrm{d}t \; .
+$$
diff --git a/D/pdf.md b/D/pdf.md
@@ -27,7 +27,7 @@ username: "JoramSoch"
 ---
 
 
-**Definition:** Let $X$ be a [continuous](/D/rvar-disc) [random variable](/D/rvar) with possible outcomes $\mathcal{X}$. Then, $f_X(x): \mathbb{R} \to \mathbb{R}$ is the probability density function of $X$, if
+**Definition:** Let $X$ be a [continuous](/D/rvar-disc) [random variable](/D/rvar) with possible outcomes $\mathcal{X}$. Then, $f_X(x): \mathbb{R} \to \mathbb{R}$ is the probability density function (PDF) of $X$, if
 
 $$ \label{eq:pdf-def-s0}
 f_X(x) \geq 0
diff --git a/D/pmf.md b/D/pmf.md
@@ -27,7 +27,7 @@ username: "JoramSoch"
 ---
 
 
-**Definition:** Let $X$ be a [discrete](/D/rvar-disc) [random variable](/D/rvar) with possible outcomes $\mathcal{X}$. Then, $f_X(x): \mathbb{R} \to [0,1]$ is the probability mass function of $X$, if
+**Definition:** Let $X$ be a [discrete](/D/rvar-disc) [random variable](/D/rvar) with possible outcomes $\mathcal{X}$. Then, $f_X(x): \mathbb{R} \to [0,1]$ is the probability mass function (PMF) of $X$, if
 
 $$ \label{eq:pmf-def-s0}
 f_X(x) = 0
diff --git a/D/qf.md b/D/qf.md
@@ -27,14 +27,8 @@ username: "JoramSoch"
 ---
 
 
-**Definition:** Let $X$ be a [random variable](/D/rvar) with the [cumulative distribution function](/D/cdf) (CDF) $F_X(x)$. Then, the function $Q_X(p): [0,1] \to \mathbb{R}$ which is the inverse CDF
+**Definition:** Let $X$ be a [random variable](/D/rvar) with the [cumulative distribution function](/D/cdf) (CDF) $F_X(x)$. Then, the function $Q_X(p): [0,1] \to \mathbb{R}$ which is the inverse CDF is the quantile function (QF) of $X$. More precisly, the QF is the function that, for a given quantile $p \in [0,1]$, returns the smallest $x$ for which $F_X(x) = p$:
 
 $$ \label{eq:qf}
-Q_X(p) = F_X^{-1}(x)
-$$
-
-is the quantile function (QF) of $X$. More precisly, the QF is the function that, for a given quantile $p \in [0,1]$, returns the smallest $x$ for which $F_X(x) = p$:
-
-$$ \label{eq:qf-prec}
 Q_X(p) = \min \left\lbrace x \in \mathbb{R} \, \vert \, F_X(x) = p \right\rbrace \; .
 $$
diff --git a/D/rfmat.md b/D/rfmat.md
@@ -30,5 +30,5 @@ username: "JoramSoch"
 **Definition:** In [multiple linear regression](/D/mlr), the residual-forming matrix is the matrix $R$ that results in the vector of residuals left over by [estimated parameters](/D/emat) when right-multiplied with the measured data:
 
 $$ \label{eq:pm}
-Ry = \hat{\varepsilon} = y - \hat{y} \; .
+Ry = \hat{\varepsilon} = y - \hat{y} = y - X \hat{\beta} \; .
 $$
diff --git a/I/Definition_by_Author.md b/I/Definition_by_Author.md
@@ -4,7 +4,7 @@ title: "Definition by Author"
 ---
 
 
-### JoramSoch (102 definitions)
+### JoramSoch (104 definitions)
 
 - [Akaike information criterion](/D/aic)
 - [Bayesian information criterion](/D/bic)
@@ -63,9 +63,11 @@ title: "Definition by Author"
 - [Marginal likelihood](/D/ml)
 - [Marginal probability distribution](/D/dist-marg)
 - [Matrix-normal distribution](/D/matn)
+- [Maximum](/D/max)
 - [Maximum likelihood estimation](/D/mle)
 - [Maximum log-likelihood](/D/mll)
 - [Median](/D/med)
+- [Minimum](/D/min)
 - [Mode](/D/mode)
 - [Moment](/D/mom)
 - [Moment-generating function](/D/mgf)
diff --git a/I/Definition_by_Number.md b/I/Definition_by_Number.md
@@ -113,3 +113,5 @@ title: "Definition by Number"
 | D104 | dir-data | [Dirichlet-distributed data](/D/dir-data) | JoramSoch | 2020-10-22 |
 | D105 | rvar-disc | [Discrete and continuous random variable](/D/rvar-disc) | JoramSoch | 2020-10-29 |
 | D106 | rvar-uni | [Univariate and multivariate random variable](/D/rvar-uni) | JoramSoch | 2020-11-06 |
+| D107 | min | [Minimum](/D/min) | JoramSoch | 2020-11-12 |
+| D108 | max | [Maximum](/D/max) | JoramSoch | 2020-11-12 |
diff --git a/I/Definition_by_Topic.md b/I/Definition_by_Topic.md
@@ -97,9 +97,11 @@ title: "Definition by Topic"
 - [Marginal likelihood](/D/ml)
 - [Marginal probability distribution](/D/dist-marg)
 - [Matrix-normal distribution](/D/matn)
+- [Maximum](/D/max)
 - [Maximum likelihood estimation](/D/mle)
 - [Maximum log-likelihood](/D/mll)
 - [Median](/D/med)
+- [Minimum](/D/min)
 - [Mode](/D/mode)
 - [Moment](/D/mom)
 - [Moment-generating function](/D/mgf)
diff --git a/I/Proof_by_Author.md b/I/Proof_by_Author.md
@@ -4,7 +4,7 @@ title: "Proof by Author"
 ---
 
 
-### JoramSoch (179 proofs)
+### JoramSoch (183 proofs)
 
 - [(Non-)Multiplicativity of the expected value](/P/mean-mult)
 - [Additivity of the Kullback-Leibler divergence for independent distributions](/P/kl-add)
@@ -25,6 +25,8 @@ title: "Proof by Author"
 - [Convexity of the Kullback-Leibler divergence](/P/kl-conv)
 - [Convexity of the cross-entropy](/P/entcross-conv)
 - [Covariance of independent random variables](/P/cov-ind)
+- [Cumulative distribution function in terms of probability density function of a continuous random variable](/P/cdf-pdf)
+- [Cumulative distribution function in terms of probability mass function of a discrete random variable](/P/cdf-pmf)
 - [Cumulative distribution function of a strictly decreasing function of a random variable](/P/cdf-sdfct)
 - [Cumulative distribution function of a strictly increasing function of a random variable](/P/cdf-sifct)
 - [Cumulative distribution function of the continuous uniform distribution](/P/cuni-cdf)
@@ -129,6 +131,7 @@ title: "Proof by Author"
 - [Posterior model probability in terms of log Bayes factor](/P/pmp-lbf)
 - [Posterior probability of the alternative hypothesis for Bayesian linear regression](/P/blr-pp)
 - [Probability and log-odds in logistic regression](/P/logreg-pnlo)
+- [Probability density function is first derivative of cumulative distribution function](/P/pdf-cdf)
 - [Probability density function of a strictly decreasing function of a continuous random variable](/P/pdf-sdfct)
 - [Probability density function of a strictly increasing function of a continuous random variable](/P/pdf-sifct)
 - [Probability density function of the Dirichlet distribution](/P/dir-pdf)
@@ -149,6 +152,7 @@ title: "Proof by Author"
 - [Probability mass function of the discrete uniform distribution](/P/duni-pmf)
 - [Probability mass function of the multinomial distribution](/P/mult-pmf)
 - [Projection matrix and residual-forming matrix are idempotent](/P/mlr-idem)
+- [Quantile function is inverse of strictly monotonically increasing cumulative distribution function](/P/qf-cdf)
 - [Quantile function of the continuous uniform distribution](/P/cuni-qf)
 - [Quantile function of the discrete uniform distribution](/P/duni-qf)
 - [Quantile function of the exponential distribution](/P/exp-qf)
diff --git a/I/Proof_by_Number.md b/I/Proof_by_Number.md
@@ -195,3 +195,7 @@ title: "Proof by Number"
 | P186 | cdf-sdfct | [Cumulative distribution function of a strictly decreasing function of a random variable](/P/cdf-sdfct) | JoramSoch | 2020-11-06 |
 | P187 | pmf-sdfct | [Probability mass function of a strictly decreasing function of a discrete random variable](/P/pmf-sdfct) | JoramSoch | 2020-11-06 |
 | P188 | pdf-sdfct | [Probability density function of a strictly decreasing function of a continuous random variable](/P/pdf-sdfct) | JoramSoch | 2020-11-06 |
+| P189 | cdf-pmf | [Cumulative distribution function in terms of probability mass function of a discrete random variable](/P/cdf-pmf) | JoramSoch | 2020-11-12 |
+| P190 | cdf-pdf | [Cumulative distribution function in terms of probability density function of a continuous random variable](/P/cdf-pdf) | JoramSoch | 2020-11-12 |
+| P191 | pdf-cdf | [Probability density function is first derivative of cumulative distribution function](/P/pdf-cdf) | JoramSoch | 2020-11-12 |
+| P192 | qf-cdf | [Quantile function is inverse of strictly monotonically increasing cumulative distribution function](/P/qf-cdf) | JoramSoch | 2020-11-12 |
diff --git a/I/Proof_by_Topic.md b/I/Proof_by_Topic.md
@@ -35,6 +35,8 @@ title: "Proof by Topic"
 - [Convexity of the Kullback-Leibler divergence](/P/kl-conv)
 - [Convexity of the cross-entropy](/P/entcross-conv)
 - [Covariance of independent random variables](/P/cov-ind)
+- [Cumulative distribution function in terms of probability density function of a continuous random variable](/P/cdf-pdf)
+- [Cumulative distribution function in terms of probability mass function of a discrete random variable](/P/cdf-pmf)
 - [Cumulative distribution function of a strictly decreasing function of a random variable](/P/cdf-sdfct)
 - [Cumulative distribution function of a strictly increasing function of a random variable](/P/cdf-sifct)
 - [Cumulative distribution function of the continuous uniform distribution](/P/cuni-cdf)
@@ -179,6 +181,7 @@ title: "Proof by Topic"
 - [Posterior model probability in terms of log Bayes factor](/P/pmp-lbf)
 - [Posterior probability of the alternative hypothesis for Bayesian linear regression](/P/blr-pp)
 - [Probability and log-odds in logistic regression](/P/logreg-pnlo)
+- [Probability density function is first derivative of cumulative distribution function](/P/pdf-cdf)
 - [Probability density function of a strictly decreasing function of a continuous random variable](/P/pdf-sdfct)
 - [Probability density function of a strictly increasing function of a continuous random variable](/P/pdf-sifct)
 - [Probability density function of the Dirichlet distribution](/P/dir-pdf)
@@ -204,6 +207,7 @@ title: "Proof by Topic"
 
 ### Q
 
+- [Quantile function is inverse of strictly monotonically increasing cumulative distribution function](/P/qf-cdf)
 - [Quantile function of the continuous uniform distribution](/P/cuni-qf)
 - [Quantile function of the discrete uniform distribution](/P/duni-qf)
 - [Quantile function of the exponential distribution](/P/exp-qf)
diff --git a/I/Proofs_without_Source.md b/I/Proofs_without_Source.md
@@ -6,6 +6,8 @@ title: "Proofs without Source"
 
 - [Chi-square distribution is a special case of gamma distribution](/P/chi2-gam)
 - [Conditional distributions of the normal-gamma distribution](/P/ng-cond)
+- [Cumulative distribution function in terms of probability density function of a continuous random variable](/P/cdf-pdf)
+- [Cumulative distribution function in terms of probability mass function of a discrete random variable](/P/cdf-pmf)
 - [Cumulative distribution function of the continuous uniform distribution](/P/cuni-cdf)
 - [Cumulative distribution function of the discrete uniform distribution](/P/duni-cdf)
 - [Cumulative distribution function of the exponential distribution](/P/exp-cdf)
diff --git a/I/Table_of_Contents.md b/I/Table_of_Contents.md
@@ -104,16 +104,20 @@ title: "Table of Contents"
    &emsp;&ensp; 1.10.1. *[Standard deviation](/D/std)* <br>
    &emsp;&ensp; 1.10.2. *[Full width at half maximum](/D/fwhm)* <br>
    
-   1.11. Further moments <br>
-   &emsp;&ensp; 1.11.1. *[Moment](/D/mom)* <br>
-   &emsp;&ensp; 1.11.2. **[Moment in terms of moment-generating function](/P/mom-mgf)** <br>
-   &emsp;&ensp; 1.11.3. *[Raw moment](/D/mom-raw)* <br>
-   &emsp;&ensp; 1.11.4. **[First raw moment is mean](/P/momraw-1st)** <br>
-   &emsp;&ensp; 1.11.5. **[Second raw moment and variance](/P/momraw-2nd)** <br>
-   &emsp;&ensp; 1.11.6. *[Central moment](/D/mom-cent)* <br>
-   &emsp;&ensp; 1.11.7. **[First central moment is zero](/P/momcent-1st)** <br>
-   &emsp;&ensp; 1.11.8. **[Second central moment is variance](/P/momcent-2nd)** <br>
-   &emsp;&ensp; 1.11.9. *[Standardized moment](/D/mom-stand)* <br>
+   1.11. Further summary statistics <br>
+   &emsp;&ensp; 1.11.1. *[Minimum](/D/min)* <br>
+   &emsp;&ensp; 1.11.2. *[Maximum](/D/max)* <br>
+   
+   1.12. Further moments <br>
+   &emsp;&ensp; 1.12.1. *[Moment](/D/mom)* <br>
+   &emsp;&ensp; 1.12.2. **[Moment in terms of moment-generating function](/P/mom-mgf)** <br>
+   &emsp;&ensp; 1.12.3. *[Raw moment](/D/mom-raw)* <br>
+   &emsp;&ensp; 1.12.4. **[First raw moment is mean](/P/momraw-1st)** <br>
+   &emsp;&ensp; 1.12.5. **[Second raw moment and variance](/P/momraw-2nd)** <br>
+   &emsp;&ensp; 1.12.6. *[Central moment](/D/mom-cent)* <br>
+   &emsp;&ensp; 1.12.7. **[First central moment is zero](/P/momcent-1st)** <br>
+   &emsp;&ensp; 1.12.8. **[Second central moment is variance](/P/momcent-2nd)** <br>
+   &emsp;&ensp; 1.12.9. *[Standardized moment](/D/mom-stand)* <br>
 
 2. Information theory
    
diff --git a/P/cdf-pmf.md b/P/cdf-pmf.md
@@ -28,7 +28,7 @@ F_X(x) = \sum_{\overset{t \in \mathcal{X}}{t \leq x}} f_X(t) \; .
 $$
 
 
-**Proof:** The [cumulative distribution function](/D/cdf) of a [random variable](/D/rvar) $X$ is defined as the probability that $X$ is smaller than $X$:
+**Proof:** The [cumulative distribution function](/D/cdf) of a [random variable](/D/rvar) $X$ is defined as the probability that $X$ is smaller than $x$:
 
 $$ \label{eq:cdf}
 F_X(x) = \mathrm{Pr}(X \leq x) \; .
diff --git a/P/cuni-qf.md b/P/cuni-qf.md
@@ -30,7 +30,12 @@ $$
 Then, the [quantile function](/D/qf) of $X$ is
 
 $$ \label{eq:cuni-qf}
-Q_X(p) = bp + a(1-p) \; .
+Q_X(p) = \left\{
+\begin{array}{rl}
+-\infty \; , & \text{if} \; p = 0 \\
+bp + a(1-p) \; , & \text{if} \; p > 0 \; .
+\end{array}
+\right.
 $$
 
 
@@ -46,9 +51,15 @@ F_X(x) = \left\{
 \right.
 $$
 
-Thus, the [quantile function](/D/qf) is:
+The quantile function $Q_X(p)$ [is defined as](/D/qf) the smallest $x$, such that $F_X(x) = p$:
+
+$$ \label{eq:qf}
+Q_X(p) = \min \left\lbrace x \in \mathbb{R} \, \vert \, F_X(x) = p \right\rbrace \; .
+$$
+
+Thus, we have $Q_X(p) = -\infty$, if $p = 0$. When $p > 0$, it holds that
 
-$$ \label{eq:cuni-qf-s1}
+$$ \label{eq:exp-qf-s1}
 Q_X(p) = F_X^{-1}(x) \; .
 $$
 
@@ -58,6 +69,6 @@ $$ \label{eq:cuni-cdf-s2}
 \begin{split}
 p &= \frac{x-a}{b-a} \\
 x &= p(b-a) + a \\
-x &= bp + a(1-p) = Q_X(p) \; .
+x &= bp + a(1-p) \; .
 \end{split}
 $$
diff --git a/P/duni-qf.md b/P/duni-qf.md
@@ -30,7 +30,12 @@ $$
 Then, the [quantile function](/D/qf) of $X$ is
 
 $$ \label{eq:duni-qf}
-Q_X(p) = a (1-p) + (b+1) p - 1 \quad \text{where} \quad p \in \left\lbrace \frac{1}{n}, \frac{2}{n}, \ldots, \frac{b-a}{n}, 1 \right\rbrace
+Q_X(p) = \left\{
+\begin{array}{rl}
+-\infty \; , & \text{if} \; p = 0 \\
+a (1-p) + (b+1) p - 1 \; , & \text{when} \; p \in \left\lbrace \frac{1}{n}, \frac{2}{n}, \ldots, \frac{b-a}{n}, 1 \right\rbrace \; .
+\end{array}
+\right.
 $$
 
 with $n = b - a + 1$.
@@ -48,7 +53,13 @@ F_X(x) = \left\{
 \right.
 $$
 
-Because [the CDF only returns](/P/duni-cdf) multiples of $1/n$ with $n = b - a + 1$, the [quantile function](/D/qf) is only defined for such values. Since [the cumulative probability increases step-wise](/P/duni-cdf) by $1/n$ at each integer between and including $a$ and $b$, the minimum $x$ at which
+The quantile function $Q_X(p)$ [is defined as](/D/qf) the smallest $x$, such that $F_X(x) = p$:
+
+$$ \label{eq:qf}
+Q_X(p) = \min \left\lbrace x \in \mathbb{R} \, \vert \, F_X(x) = p \right\rbrace \; .
+$$
+
+Because [the CDF only returns](/P/duni-cdf) multiples of $1/n$ with $n = b - a + 1$, the [quantile function](/D/qf) is only defined for such values. First, we have $Q_X(p) = -\infty$, if $p = 0$. Second, since [the cumulative probability increases step-wise](/P/duni-cdf) by $1/n$ at each integer between and including $a$ and $b$, the minimum $x$ at which
 
 $$ \label{eq:duni-cdf-p}
 F_X(x) = \frac{c}{n} \quad \text{where} \quad c \in \left\lbrace 1, \ldots, n \right\rbrace
diff --git a/P/exp-qf.md b/P/exp-qf.md
@@ -30,7 +30,12 @@ $$
 Then, the [quantile function](/D/qf) of $X$ is
 
 $$ \label{eq:exp-qf}
-Q_X(p) = -\frac{\ln(1-p)}{\lambda} \; .
+Q_X(p) = \left\{
+\begin{array}{rl}
+-\infty \; , & \text{if} \; p = 0 \\
+-\frac{\ln(1-p)}{\lambda} \; , & \text{if} \; p > 0 \; .
+\end{array}
+\right.
 $$
 
 
@@ -45,7 +50,13 @@ F_X(x) = \left\{
 \right.
 $$
 
-Thus, the [quantile function](/D/qf) is:
+The quantile function $Q_X(p)$ [is defined as](/D/qf) the smallest $x$, such that $F_X(x) = p$:
+
+$$ \label{eq:qf}
+Q_X(p) = \min \left\lbrace x \in \mathbb{R} \, \vert \, F_X(x) = p \right\rbrace \; .
+$$
+
+Thus, we have $Q_X(p) = -\infty$, if $p = 0$. When $p > 0$, it holds that
 
 $$ \label{eq:exp-qf-s1}
 Q_X(p) = F_X^{-1}(x) \; .
diff --git a/P/norm-qf.md b/P/norm-qf.md
@@ -48,7 +48,7 @@ $$ \label{eq:norm-cdf}
 F_X(x) = \frac{1}{2} \left[ 1 + \mathrm{erf}\left( \frac{x-\mu}{\sqrt{2} \sigma} \right) \right] \; .
 $$
 
-Thus, the [quantile function](/D/qf) is:
+Because the cumulative distribution function (CDF) is strictly monotonically increasing, the [quantile function is equal to the inverse of the CDF](/P/qf-cdf):
 
 $$ \label{eq:norm-qf-s1}
 Q_X(p) = F_X^{-1}(x) \; .
diff --git a/P/qf-cdf.md b/P/qf-cdf.md
@@ -40,7 +40,7 @@ $$ \label{eq:qf}
 Q_X(p) = \min \left\lbrace x \in \mathbb{R} \, \vert \, F_X(x) = p \right\rbrace \; .
 $$
 
-If $F_X(x)$ is continuous and strictly monotonically increasing, then there is only $x$ for which $F_X(x) = p$ and $F_X(x)$ is an invertible function, such that
+If $F_X(x)$ is continuous and strictly monotonically increasing, then there is exactly one $x$ for which $F_X(x) = p$ and $F_X(x)$ is an invertible function, such that
 
 $$ \label{eq:qf-cdf-qed}
 Q_X(p) = F_X^{-1}(x) \; .
