|
30 | 30 | be a [general linear model](/D/glm) with measured $n \times v$ data matrix $Y$, known $n \times p$ design matrix $X$, known $n \times n$ [covariance structure](/D/matn) $V$ as well as unknown $p \times v$ regression coefficients $B$ and unknown $v \times v$ [noise covariance](/D/matn) $\Sigma$. Moreover, assume a [normal-Wishart prior distribution](/P/mblr-prior) over the model parameters $B$ and $T = \Sigma^{-1}$: |
31 | 31 |
|
32 | 32 | $$ \label{eq:GLM-NW-prior} |
33 | | -p(B,T) = \mathcal{MN}(B; M_0, \Lambda_0^{-1}, T^{-1}) \cdot \mathcal{W}(T; P_0^{-1}, \nu_0) \; . |
| 33 | +p(B,T) = \mathcal{MN}(B; M_0, \Lambda_0^{-1}, T^{-1}) \cdot \mathcal{W}(T; \Omega_0^{-1}, \nu_0) \; . |
34 | 34 | $$ |
35 | 35 |
|
36 | 36 | Then, the [log model evidence](/D/lme) for this model is |
37 | 37 |
|
38 | | -\begin{equation} \label{eq:GLM-NW-LME} |
| 38 | +$$ \label{eq:GLM-NW-LME} |
39 | 39 | \begin{split} |
40 | 40 | \log p(y|m) = & \frac{v}{2} \log |P| - \frac{nv}{2} \log (2 \pi) + \frac{v}{2} \log |\Lambda_0| - \frac{v}{2} \log |\Lambda_n| + \\ |
41 | | -& \frac{\nu_0}{2} \log\left| \frac{1}{2} P_0 \right| - \frac{\nu_n}{2} \log\left| \frac{1}{2} P_n \right| + \log \Gamma_v \left( \frac{\nu_n}{2} \right) - \log \Gamma_v \left( \frac{\nu_0}{2} \right) |
| 41 | +& \frac{\nu_0}{2} \log\left| \frac{1}{2} \Omega_0 \right| - \frac{\nu_n}{2} \log\left| \frac{1}{2} \Omega_n \right| + \log \Gamma_v \left( \frac{\nu_n}{2} \right) - \log \Gamma_v \left( \frac{\nu_0}{2} \right) |
42 | 42 | \end{split} |
43 | | -\end{equation} |
| 43 | +$$ |
44 | 44 |
|
45 | 45 | where the [posterior hyperparameters](/D/post) are given by |
46 | 46 |
|
47 | | -\begin{equation} \label{eq:GLM-NW-post-par} |
| 47 | +$$ \label{eq:GLM-NW-post-par} |
48 | 48 | \begin{split} |
49 | 49 | M_n &= \Lambda_n^{-1} (X^\mathrm{T} P Y + \Lambda_0 M_0) \\ |
50 | 50 | \Lambda_n &= X^\mathrm{T} P X + \Lambda_0 \\ |
51 | | -P_n &= P_0 + Y^\mathrm{T} P Y + M_0^\mathrm{T} \Lambda_0 M_0 - M_n^\mathrm{T} \Lambda_n M_n \\ |
| 51 | +\Omega_n &= \Omega_0 + Y^\mathrm{T} P Y + M_0^\mathrm{T} \Lambda_0 M_0 - M_n^\mathrm{T} \Lambda_n M_n \\ |
52 | 52 | \nu_n &= \nu_0 + n \; . |
53 | 53 | \end{split} |
54 | | -\end{equation} |
| 54 | +$$ |
55 | 55 |
|
56 | 56 |
|
57 | 57 | **Proof:** According to the [law of marginal probability](/D/prob-marg), the [model evidence](/D/ml) for this model is: |
@@ -83,48 +83,48 @@ using the $v \times v$ [precision matrix](/D/precmat) $T = \Sigma^{-1}$ and the |
83 | 83 | <br> |
84 | 84 | When [deriving the posterior distribution](/P/mblr-post) $p(B,T|Y)$, the joint likelihood $p(Y,B,T)$ is obtained as |
85 | 85 |
|
86 | | -\begin{equation} \label{eq:GLM-NW-LME-s1} |
| 86 | +$$ \label{eq:GLM-NW-LME-s1} |
87 | 87 | \begin{split} |
88 | | -p(Y,B,T) = \; & \sqrt{\frac{|T|^n |P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|T|^p |\Lambda_0|^v}{(2 \pi)^{pv}}} \sqrt{\frac{|P_0|^{\nu_0}}{2^{\nu_0 v}}} \frac{1}{\Gamma_v \left( \frac{\nu_0}{2} \right)} \cdot |T|^{(\nu_0-v-1)/2} \exp\left[ -\frac{1}{2} \mathrm{tr}\left( P_0 T \right) \right] \cdot \\ |
| 88 | +p(Y,B,T) = \; & \sqrt{\frac{|T|^n |P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|T|^p |\Lambda_0|^v}{(2 \pi)^{pv}}} \sqrt{\frac{|\Omega_0|^{\nu_0}}{2^{\nu_0 v}}} \frac{1}{\Gamma_v \left( \frac{\nu_0}{2} \right)} \cdot |T|^{(\nu_0-v-1)/2} \exp\left[ -\frac{1}{2} \mathrm{tr}\left( \Omega_0 T \right) \right] \cdot \\ |
89 | 89 | & \exp\left[ -\frac{1}{2} \mathrm{tr}\left( T \left[ (B-M_n)^\mathrm{T} \Lambda_n (B-M_n) + (Y^\mathrm{T} P Y + M_0^\mathrm{T} \Lambda_0 M_0 - M_n^\mathrm{T} \Lambda_n M_n) \right] \right) \right] \; . |
90 | 90 | \end{split} |
91 | | -\end{equation} |
| 91 | +$$ |
92 | 92 |
|
93 | 93 | Using the [probability density function of the matrix-normal distribution](/P/matn-pdf), we can rewrite this as |
94 | 94 |
|
95 | | -\begin{equation} \label{eq:GLM-NW-LME-s2} |
| 95 | +$$ \label{eq:GLM-NW-LME-s2} |
96 | 96 | \begin{split} |
97 | | -p(Y,B,T) = \; & \sqrt{\frac{|T|^n |P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|T|^p |\Lambda_0|^v}{(2 \pi)^{pv}}} \sqrt{\frac{(2 \pi)^{pv}}{|T|^p |\Lambda_n|^v}} \sqrt{\frac{|P_0|^{\nu_0}}{2^{\nu_0 v}}} \frac{1}{\Gamma_v \left( \frac{\nu_0}{2} \right)} \cdot |T|^{(\nu_0-v-1)/2} \exp\left[ -\frac{1}{2} \mathrm{tr}\left( P_0 T \right) \right] \cdot \\ |
| 97 | +p(Y,B,T) = \; & \sqrt{\frac{|T|^n |P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|T|^p |\Lambda_0|^v}{(2 \pi)^{pv}}} \sqrt{\frac{(2 \pi)^{pv}}{|T|^p |\Lambda_n|^v}} \sqrt{\frac{|\Omega_0|^{\nu_0}}{2^{\nu_0 v}}} \frac{1}{\Gamma_v \left( \frac{\nu_0}{2} \right)} \cdot |T|^{(\nu_0-v-1)/2} \exp\left[ -\frac{1}{2} \mathrm{tr}\left( \Omega_0 T \right) \right] \cdot \\ |
98 | 98 | & \mathcal{MN}(B; M_n, \Lambda_n^{-1}, T^{-1}) \cdot \exp\left[ -\frac{1}{2} \mathrm{tr}\left( T \left[ Y^\mathrm{T} P Y + M_0^\mathrm{T} \Lambda_0 M_0 - M_n^\mathrm{T} \Lambda_n M_n \right] \right) \right] \; . |
99 | 99 | \end{split} |
100 | | -\end{equation} |
| 100 | +$$ |
101 | 101 |
|
102 | 102 | Now, $B$ can be integrated out easily: |
103 | 103 |
|
104 | | -\begin{equation} \label{eq:GLM-NW-LME-s3} |
| 104 | +$$ \label{eq:GLM-NW-LME-s3} |
105 | 105 | \begin{split} |
106 | | -\int p(Y,B,T) \, \mathrm{d}B = \; & \sqrt{\frac{|T|^n |P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|\Lambda_0|^v}{|\Lambda_n|^v}} \sqrt{\frac{|P_0|^{\nu_0}}{2^{\nu_0 v}}} \frac{1}{\Gamma_v \left( \frac{\nu_0}{2} \right)} \cdot |T|^{(\nu_0-v-1)/2} \cdot \\ |
107 | | -& \exp\left[ -\frac{1}{2} \mathrm{tr}\left( T \left[ P_0 + Y^\mathrm{T} P Y + M_0^\mathrm{T} \Lambda_0 M_0 - M_n^\mathrm{T} \Lambda_n M_n \right] \right) \right] \; . |
| 106 | +\int p(Y,B,T) \, \mathrm{d}B = \; & \sqrt{\frac{|T|^n |P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|\Lambda_0|^v}{|\Lambda_n|^v}} \sqrt{\frac{|\Omega_0|^{\nu_0}}{2^{\nu_0 v}}} \frac{1}{\Gamma_v \left( \frac{\nu_0}{2} \right)} \cdot |T|^{(\nu_0-v-1)/2} \cdot \\ |
| 107 | +& \exp\left[ -\frac{1}{2} \mathrm{tr}\left( T \left[ \Omega_0 + Y^\mathrm{T} P Y + M_0^\mathrm{T} \Lambda_0 M_0 - M_n^\mathrm{T} \Lambda_n M_n \right] \right) \right] \; . |
108 | 108 | \end{split} |
109 | | -\end{equation} |
| 109 | +$$ |
110 | 110 |
|
111 | 111 | Using the [probability density function of the Wishart distribution](/P/wish-pdf), we can rewrite this as |
112 | 112 |
|
113 | 113 | $$ \label{eq:GLM-NW-LME-s4} |
114 | | -\int p(Y,B,T) \, \mathrm{d}B = \sqrt{\frac{|P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|\Lambda_0|^v}{|\Lambda_n|^v}} \sqrt{\frac{|P_0|^{\nu_0}}{2^{\nu_0 v}}} \sqrt{\frac{2^{\nu_n v}}{|P_n|^{\nu_n}}} \, \frac{\Gamma_v \left( \frac{\nu_n}{2} \right)}{\Gamma_v \left( \frac{\nu_0}{2} \right)} \cdot \mathcal{W}(T; P_n^{-1}, \nu_n) \; . |
| 114 | +\int p(Y,B,T) \, \mathrm{d}B = \sqrt{\frac{|P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|\Lambda_0|^v}{|\Lambda_n|^v}} \sqrt{\frac{|\Omega_0|^{\nu_0}}{2^{\nu_0 v}}} \sqrt{\frac{2^{\nu_n v}}{|\Omega_n|^{\nu_n}}} \, \frac{\Gamma_v \left( \frac{\nu_n}{2} \right)}{\Gamma_v \left( \frac{\nu_0}{2} \right)} \cdot \mathcal{W}(T; \Omega_n^{-1}, \nu_n) \; . |
115 | 115 | $$ |
116 | 116 |
|
117 | 117 | Finally, $T$ can also be integrated out: |
118 | 118 |
|
119 | 119 | $$ \label{eq:GLM-NW-LME-s5} |
120 | | -\iint p(Y,B,T) \, \mathrm{d}B \, \mathrm{d}T = \sqrt{\frac{|P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|\Lambda_0|^v}{|\Lambda_n|^v}} \sqrt{\frac{\left| \frac{1}{2} P_0 \right|^{\nu_0}}{\left| \frac{1}{2} P_n \right|^{\nu_n}}} \, \frac{\Gamma_v \left( \frac{\nu_n}{2} \right)}{\Gamma_v \left( \frac{\nu_0}{2} \right)} = p(y|m) \; . |
| 120 | +\iint p(Y,B,T) \, \mathrm{d}B \, \mathrm{d}T = \sqrt{\frac{|P|^v}{(2 \pi)^{nv}}} \sqrt{\frac{|\Lambda_0|^v}{|\Lambda_n|^v}} \sqrt{\frac{\left| \frac{1}{2} \Omega_0 \right|^{\nu_0}}{\left| \frac{1}{2} \Omega_n \right|^{\nu_n}}} \, \frac{\Gamma_v \left( \frac{\nu_n}{2} \right)}{\Gamma_v \left( \frac{\nu_0}{2} \right)} = p(y|m) \; . |
121 | 121 | $$ |
122 | 122 |
|
123 | 123 | Thus, the [log model evidence](/D/lme) of this model is given by |
124 | 124 |
|
125 | | -\begin{equation} \label{eq:GLM-NW-LME-s6} |
| 125 | +$$ \label{eq:GLM-NW-LME-s6} |
126 | 126 | \begin{split} |
127 | 127 | \log p(y|m) = & \frac{v}{2} \log |P| - \frac{nv}{2} \log (2 \pi) + \frac{v}{2} \log |\Lambda_0| - \frac{v}{2} \log |\Lambda_n| + \\ |
128 | | -& \frac{\nu_0}{2} \log\left| \frac{1}{2} P_0 \right| - \frac{\nu_n}{2} \log\left| \frac{1}{2} P_n \right| + \log \Gamma_v \left( \frac{\nu_n}{2} \right) - \log \Gamma_v \left( \frac{\nu_0}{2} \right) \; . |
| 128 | +& \frac{\nu_0}{2} \log\left| \frac{1}{2} \Omega_0 \right| - \frac{\nu_n}{2} \log\left| \frac{1}{2} \Omega_n \right| + \log \Gamma_v \left( \frac{\nu_n}{2} \right) - \log \Gamma_v \left( \frac{\nu_0}{2} \right) \; . |
129 | 129 | \end{split} |
130 | | -\end{equation} |
| 130 | +$$ |
0 commit comments