added 3 definitions

Author: JoramSoch

D/eblme.md

@@ -0,0 +1,46 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2020-11-25 07:43:00
+
+title: "Empirical Bayesian log model evidence"
+chapter: "Model Selection"
+section: "Bayesian model selection"
+topic: "Log model evidence"
+definition: "Empirical Bayesian log model evidence"
+
+sources:
+  - authors: "Wikipedia"
+    year: 2020
+    title: "Empirical Bayes method"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2020-11-25"
+    url: "https://en.wikipedia.org/wiki/Empirical_Bayes_method#Introduction"
+
+def_id: "D114"
+shortcut: "eblme"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let $m$ be a [generative model](/D/gm) with model parameters $\theta$ and hyper-parameters $\lambda$ implying the [likelihood function](/D/lf) $p(y \vert \theta, \lambda, m)$ and [prior distribution](/D/prior) $p(\theta \vert \lambda, m)$. Then, the [Empirical Bayesian](/D/eb) [log model evidence](/D/lme) is the logarithm of the [marginal likelihood](/D/ml), maximized with respect to the hyper-parameters:
+
+$$ \label{eq:ebLME}
+\mathrm{ebLME}(m) = \log p(y \vert \hat{\lambda}, m)
+$$
+
+where
+
+$$ \label{eq:ML}
+p(y \vert \lambda, m) = \int p(y \vert \theta, \lambda, m) \, (\theta \vert \lambda, m) \, \mathrm{d}\theta
+$$
+
+and
+
+$$ \label{eq:EB}
+\hat{\lambda} = \operatorname*{arg\,max}_{\lambda} \log p(y \vert \lambda, m) \; .
+$$

D/uplme.md

@@ -0,0 +1,34 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2020-11-25 07:28:00
+
+title: "Uniform-prior log model evidence"
+chapter: "Model Selection"
+section: "Bayesian model selection"
+topic: "Log model evidence"
+definition: "Uniform-prior log model evidence"
+
+sources:
+  - authors: "Wikipedia"
+    year: 2020
+    title: "Lindley's paradox"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2020-11-25"
+    url: "https://en.wikipedia.org/wiki/Lindley%27s_paradox#Bayesian_approach"
+
+def_id: "D113"
+shortcut: "uplme"
+username: "JoramSoch"
+---
+
+
+**Definition:** Assume a [generative model](/D/gm) $m$ with [likelihood function](/D/lf) $p(y \vert \theta, m)$ and a [uniform](/D/prior-uni) [prior distribution](/D/prior) $p_{\mathrm{uni}}(\theta \vert m)$. Then, the [log model evidence](/D/lme) of this model is called "log model evidence with uniform prior" or "uniform-prior log model evidence" (upLME):
+
+$$ \label{eq:upLME}
+\mathrm{upLME}(m) = \log \int p(y \vert \theta, m) \, p_{\mathrm{uni}}(\theta \vert m) \, \mathrm{d}\theta \; .
+$$

D/vblme.md

@@ -0,0 +1,52 @@
+---
+layout: definition
+mathjax: true
+
+author: "Joram Soch"
+affiliation: "BCCN Berlin"
+e_mail: "joram.soch@bccn-berlin.de"
+date: 2020-11-25 08:10:00
+
+title: "Variational Bayesian log model evidence"
+chapter: "Model Selection"
+section: "Bayesian model selection"
+topic: "Log model evidence"
+definition: "Variational Bayesian log model evidence"
+
+sources:
+  - authors: "Wikipedia"
+    year: 2020
+    title: "Variational Bayesian methods"
+    in: "Wikipedia, the free encyclopedia"
+    pages: "retrieved on 2020-11-25"
+    url: "https://en.wikipedia.org/wiki/Variational_Bayesian_methods#Evidence_lower_bound"
+  - authors: "Bishop CM"
+    year: 2006
+    title: "Variational Inference"
+    in: "Pattern Recognition for Machine Learning"
+    pages: "pp. 462-474, eqs. 10.2-10.4"
+    url: "https://www.springer.com/gp/book/9780387310732"
+
+def_id: "D115"
+shortcut: "vblme"
+username: "JoramSoch"
+---
+
+
+**Definition:** Let $m$ be a [generative model](/D/gm) with model parameters $\theta$ implying the [likelihood function](/D/lf) $p(y \vert \theta, m)$. Moreover, assume a [prior distribution](/D/prior) $p(\theta \vert m)$, a resulting [posterior distribution](/D/post) $p(\theta \vert y, m)$ and an [approximate](/D/vb) [posterior distribution](/D/post) $q(\theta)$. Then, the [Variational Bayesian](/D/vb) [log model evidence](/D/lme) is the expectation of the [log-likelihood function](/D/llf) with respect to the approximate posterior, minus the [Kullback-Leibler divergence](/D/kl) between approximate posterior and true posterior distribution:
+
+$$ \label{eq:vbLME}
+\mathrm{vbLME}(m) = \mathcal{L}\left[q(\theta)\right] - \mathrm{KL}\left[q(\theta) || p(\theta \vert y)\right]
+$$
+
+where
+
+$$ \label{eq:ELL}
+\mathcal{L}\left[q(\theta)\right] = \int q(\theta) \log \frac{p(y,\theta|m)}{q(\theta)} \, \mathrm{d}\theta
+$$
+
+and
+
+$$ \label{eq:KL}
+\mathrm{KL}\left[q(\theta) || p(\theta \vert y)\right] = \int q(\theta) \log \frac{q(\theta)}{p(\theta|y,m)} \, \mathrm{d}\theta  \; .
+$$