You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Definition and Proof by @tomfaulkenberry were integrated:
- "/D/em" renamed to "/D/enc"
- did cosmetic corrections on "/P/bf-sddr" and "/P/bf-ep"
- "/D/encm" and "/P/bf-ep" added to "/I/Table_of_Contents"
- added source to "/P/bf-sddr"
In other words, the Bayes factor can be viewed as the factor by which the prior model odds are updated (after observing data $y$) to posterior model odds (see also [Bayes' rule](/P/bayes-rule)).
49
+
In other words, the Bayes factor can be viewed as the factor by which the prior model odds are updated (after observing data $y$) to posterior model odds – which is also expressed by [Bayes' rule](/P/bayes-rule).
**Definition:** Consider a family $f$ of [generative models](/D/gm) $m$ on data $y$, where each $m \in f$ is defined by placing an inequality constraint on model parameter(s) $\theta$ (e.g., $m:\theta>0$. Then the encompassing model $m_e$ is constructed such that each $m$ is nested within $m_e$ and all inequality constraints on the parameter(s) $\theta$ are removed.
31
+
**Definition:** Consider a family $f$ of [generative models](/D/gm) $m$ on data $y$, where each $m \in f$ is defined by placing an inequality constraint on model parameter(s) $\theta$ (e.g., $m:\theta>0$). Then the encompassing model $m_e$ is constructed such that each $m$ is nested within $m_e$ and all inequality constraints on the parameter(s) $\theta$ are removed.
title: "A tutorial on generalizing the default Bayesian t-test via posterior sampling and encompassing priors"
@@ -36,32 +35,33 @@ username: "tomfaulkenberry"
36
35
---
37
36
38
37
39
-
**Theorem:** Consider two models $m_1$ and $m_e$, where $m_1$ is nested within an [encompassing model](/D/em) $m_e$ via an inequality constraint on some parameter $\theta$, and $\theta$ is unconstrained under $m_e$. Then
40
-
\[
41
-
B_{1e} = \frac{c}{d} = \frac{1/d}{1/c}
42
-
\]
38
+
**Theorem:** Consider two models $m_1$ and $m_e$, where $m_1$ is nested within an [encompassing model](/D/encm) $m_e$ via an inequality constraint on some parameter $\theta$, and $\theta$ is unconstrained under $m_e$. Then, the [Bayes factor](/D/bf) is
39
+
40
+
$$ \label{eq:bf-ep}
41
+
\text{BF}_{1e} = \frac{c}{d} = \frac{1/d}{1/c}
42
+
$$
43
+
43
44
where $1/d$ and $1/c$ represent the proportions of the posterior and prior of the encompassing model, respectively, that are in agreement with the inequality constraint imposed by the nested model $m_1$.
44
45
45
-
**Proof:**
46
-
Consider first that for any model $m_1$ on data $y$ with parameter $\theta$, [Bayes theorem](/P/bayes-th) implies
46
+
**Proof:** Consider first that for any model $m_1$ on data $y$ with parameter $\theta$, [Bayes' theorem](/P/bayes-th) implies
Now, both the constrained model $m_1$ and the [encompassing model](/D/em) $m_e$ contain the same parameter vector $\theta$. Choose a specific value of $\theta$, say $\theta'$, that exists in the support of both models $m_1$ and $m_e$ (we can do this because $m_1$ is nested within $m_e$). Then, for this parameter value $\theta'$, we have $p(y \mid \theta',m_1)=p(y \mid \theta',m_e)$, so the expression for the Bayes factor (Equation \eqref{eq:bayesfactor} above) reduces to an expression involving only the priors and posteriors for $\theta'$ under $m_1$ and $m_e$:
64
+
Now, both the constrained model $m_1$ and the [encompassing model](/D/encm) $m_e$ contain the same parameter vector $\theta$. Choose a specific value of $\theta$, say $\theta'$, that exists in the support of both models $m_1$ and $m_e$ (we can do this, because $m_1$ is nested within $m_e$). Then, for this parameter value $\theta'$, we have $p(y \mid \theta',m_1)=p(y \mid \theta',m_e)$, so the expression for the Bayes factor in equation \eqref{eq:bayesfactor} reduces to an expression involving only the priors and posteriors for $\theta'$ under $m_1$ and $m_e$:
which completes the proof. Note that by definition, $1/d$ represents the proportion of the posterior distribution for $\theta$ under the [encompassing model](/D/em) $m_e$ that agrees with the constraints imposed by $m_1$. Similarly, $1/c$ represents the proportion of the prior distribution for $\theta$ under the [encompassing model](/D/em) $m_e$ that agrees with the constraints imposed by $m_1$.
97
+
which completes the proof. Note that by definition, $1/d$ represents the proportion of the posterior distribution for $\theta$ under the [encompassing model](/D/encm) $m_e$ that agrees with the constraints imposed by $m_1$. Similarly, $1/c$ represents the proportion of the prior distribution for $\theta$ under the [encompassing model](/D/encm) $m_e$ that agrees with the constraints imposed by $m_1$.
By [definition](/D/bf), the Bayes factor $\text{BF}_{01}$ is the ratio of marginal likelihoods of data $y$ over $m_0$ and $m_1$, respectively. That is,
44
+
**Proof:** By [definition](/D/bf), the Bayes factor $\text{BF}_{01}$ is the ratio of marginal likelihoods of data $y$ over $m_0$ and $m_1$, respectively. That is,
The key idea in the proof is that we can use a "change of variables" technique to express $\text{BF}_{01}$ entirely in terms of the "encompassing" model $m_1$. This proceeds by first unpacking the [marginal likelihood](/D/ml) for $m_0$ over the nuisance parameter $\varphi$ and then using the fact that $m_0$ is a sharp hypothesis nested within $m_1$ to rewrite everything in terms of $\mathcal{H}_1$. Specifically,
50
+
The key idea in the proof is that we can use a "change of variables" technique to express $\text{BF}_{01}$ entirely in terms of the "encompassing" model $m_1$. This proceeds by first unpacking the [marginal likelihood](/D/ml) for $m_0$ over the nuisance parameter $\varphi$ and then using the fact that $m_0$ is a sharp hypothesis nested within $m_1$ to rewrite everything in terms of $m_1$. Specifically,
0 commit comments