Wishart Distribution

  • Definition
    If $S=X^TX$, where $X_{np}\sim N(0,I_n \otimes \Sigma)$, here, $p$ is the dimensions of the data, and, $n$ is the number of the data. Then $S$ is positive definite, and is said to have the wishart distribution with $n$ degree of freedom and covariance matrix $\Sigma$. \[ \begin{split} X^TX &= S \sim W_p(\Sigma,n) \
    &\Updownarrow \text{equivalence} \
    X&\sim N(0,I_n \otimes \Sigma) \end{split} \]

  1. [pdf.]If $S$ is $W_p(\Sigma, r)$ , then the density function of $S$ is, \[ p(S)=\frac{ |S|^{\frac{r-p-1}{2}} \cdot exp(-\frac{1}{2} Tr(\Sigma^{-1}S)) }{ 2^{rp/2} \cdot \pi^{\frac{p(p-1)}{4}} \cdot |\Sigma |^{r/2} \cdot \prod_{i=1}^p \Gamma (\frac{r-i+1}{2}) } \]

  2. [Decompose Properties] Let $S\sim W_p(\Sigma,r)$,$S_{11.2}=S_{11}-S_{12}S_{22}^{-1}S_{21}$ (schur completement of $S_{22}$),$\Sigma_{11.2}=\Sigma_{11}-\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$ (schur completement of $\Sigma_{22}$), then
    • $S_{11}\sim W_p(\Sigma_{11},r)$
    • $S_{22}\sim W_p(\Sigma_{22},r)$
    • $S_{11.2}\sim W_p(\Sigma_{11.2},r-(p-q))$
    • $S_{12} | S_{22} \sim N_{q,p-q}(\Sigma_{12}\Sigma_{22}^{-1}S_{22}, \Sigma_{11.2} \otimes S_{22})$
  3. [Sample Wishart Distribution] Let $S\sim W_p(I_p,r)$ and $S=T^TT$ where $T=(t_{ij})$ is a upper triangle matrix, $t_{ii}>0$ then,
    • $t_{ij}, 1\leq j \leq i \leq p$ are independently distributed
    • $t_{ii}^2 \sim \chi^2_{r-i+1}$
    • $t_{ij}\sim N(0,1), 1\le j \le i \le p$

Inverse Wishart Distribution

  • [definition] $S^{-1}$ is said to have an inverse wishart distribution $W_p^{-1}(\Sigma,r)$, if its pdf ($M=S^{-1}$) \[ p(M)=\frac{|M|^{-\frac{r+p+1}{2}} \cdot eTr(-\frac{1}{2} \Sigma^{-1} M^{-1})}{2^{rp/2} \cdot \pi^{p(p-1)/4} \cdot |\Sigma |^{r/2} \cdot \prod_{i=1}^p \Gamma (\frac{r+1-i}{2}) } \]

  • [transformation]The inverse wishart distribution can be treated as the results after performing transformation $(M\rightarrow S^{-1})$.
    And the jacobin of the transformation $(M\rightarrow S^{-1})$ is (using outer product, or, wedge product),
    \[ (dM)=det(S)^{-2m} (dS) \] if the $S$ is symmetric, then, \[ (dM)=det(S)^{-(m+1)}(dS) \]