multinomial distribution

[brief introduce]
设某随机实验有$A_1,…,A_k$个结果，分别将它们出现的次数标记为随机变量$X_1,…,X_k$.它们的概率分布是$p_1,…,p_k$,那么在$n$次实验中，$A_1$出现$n_1$次，$A_k$出现$n_k$次的事件概率为 \[ p(X_1=n_1,X_2=n_2,…,X_k=n_k)= \frac{ \Gamma( \sum_i n_i)}{ \Gamma(n_1)… \Gamma(n_k)} \prod_{i=1}^k p_i^{n_i} \]

For the random variable result $n_1$, it represents the frequency of the result $A_1$ appeared in the experiment, also the same as the $n_2,…,n_k$. So the meaning of the $p(n_1,…,n_k)$ is the joint probability of that $A_1$ occurred $n_1$ times and $A_2$ occurred $n_2$ times etc.
[properties]
- $E(X_i)=np_i$
- $var(X_i)=np_i(1-p_i)$
- $cov(X_i,X_j)=-np_ip_j$
[marginal distribution]
The marginal distribution of $X^{m}=(X_1,…,X_m)^T, (m\le k)$ is the multinomial distribution, \[ M(x^{m}|(p_1,…,p_m)) \]
[conditional distribution]
The conditional distribution of $X^{m}$ given the remaining $X_i$’s is also multinomial, \[ p(x^{m}|x_{m+1},…,x_k) \sim M_{m-1}(x^{m}|p_1’,…,p_m’),n-s) \] where $p_i’=\frac{p_i}{\sum_{j=1}^mp_j},(1\leq i\leq m)$ and $s=\sum_{i=m+1}^k x_i$.

proof. prompt: using multinomial theorem.

[brief introduce] If we treat the parameters $p_1,…,p_k$ as a random variable vector, with which obey a distribution. Then we could palce a conjugate prior distribution to it, and the distribution could be dirichlet distribution
[pdf] \[ p(p_1,…,p_k;\alpha_1,…,\alpha_k)=\frac{\Gamma(\sum_{i=1}^k \alpha_i)}{\prod_{i=1}^k \Gamma(\alpha_i)} \prod_{i=1}^k p_i^{\alpha_i-1} \] when $k=1$, the pdf is beta distribution.
[properties]
- $E(p_i)=\frac{\alpha_i} {\sum_{i=1}^k \alpha_i}$
- $var(p_i)=\frac{E(p_i)(1-E(p_i))}{1+\sum_{j=1}^k \alpha_j}$
- $cov(p_i,p_j)=\frac{E(p_i)E(p_j)}{1+\sum_{j=1}^k \alpha_j}$
[marginal] The marginal distribution $P^{m}=(p_1,…,p_m)^T,(m\le k)$ is the dirichlet distribution, \[ Dir(P^{m}|(\alpha_1,…,\alpha_m),\sum_{i=m+1}^k \alpha_i) \]
[conditional] The conditional distribution of $P^{m}$ given the $(p_{m+1},…,p_k)$ of $Y_i=\frac{p_i}{1-\sum_{j=m+1}^k p_j}$ is also the dirichlet distribution, \[ Dir(Y^{m}|(\alpha_1,…,\alpha_m),\sum_{i=m+1}^k \alpha_i) \]

假设有$k$个桶，$n$个球，每个桶中的球的个数为$x_i$,概率为$p_i$.那么掉入每个桶中的球的个数服从以下分布，