multinomial

  • [brief introduce]
    设某随机实验有$A_1,…,A_k$个结果,分别将它们出现的次数标记为随机变量$X_1,…,X_k$.它们的概率分布是$p_1,…,p_k$,那么在$n$次实验中,$A_1$出现$n_1$次,$A_k$出现$n_k$次的事件概率为 \[ p(X_1=n_1,X_2=n_2,…,X_k=n_k)= \frac{ \Gamma( \sum_i n_i)}{ \Gamma(n_1)… \Gamma(n_k)} \prod_{i=1}^k p_i^{n_i} \]

    For the random variable result $n_1$, it represents the frequency of the result $A_1$ appeared in the experiment, also the same as the $n_2,…,n_k$. So the meaning of the $p(n_1,…,n_k)$ is the joint probability of that $A_1$ occurred $n_1$ times and $A_2$ occurred $n_2$ times etc.

  • [properties]
    • $E(X_i)=np_i$
    • $var(X_i)=np_i(1-p_i)$
    • $cov(X_i,X_j)=-np_ip_j$
  • [marginal distribution]
    The marginal distribution of $X^{m}=(X_1,…,X_m)^T, (m\le k)$ is the multinomial distribution, \[ M(x^{m}|(p_1,…,p_m)) \]
  • [conditional distribution]
    The conditional distribution of $X^{m}$ given the remaining $X_i$’s is also multinomial, \[ p(x^{m}|x_{m+1},…,x_k) \sim M_{m-1}(x^{m}|p_1’,…,p_m’),n-s) \] where $p_i’=\frac{p_i}{\sum_{j=1}^mp_j},(1\leq i\leq m)$ and $s=\sum_{i=m+1}^k x_i$.

    proof. prompt: using multinomial theorem.

dirichlet distribution###

  • [brief introduce] If we treat the parameters $p_1,…,p_k$ as a random variable vector, with which obey a distribution. Then we could palce a conjugate prior distribution to it, and the distribution could be dirichlet distribution

  • [pdf] \[ p(p_1,…,p_k;\alpha_1,…,\alpha_k)=\frac{\Gamma(\sum_{i=1}^k \alpha_i)}{\prod_{i=1}^k \Gamma(\alpha_i)} \prod_{i=1}^k p_i^{\alpha_i-1} \] when $k=1$, the pdf is beta distribution.

  • [properties]
    • $E(p_i)=\frac{\alpha_i} {\sum_{i=1}^k \alpha_i}$
    • $var(p_i)=\frac{E(p_i)(1-E(p_i))}{1+\sum_{j=1}^k \alpha_j}$
    • $cov(p_i,p_j)=\frac{E(p_i)E(p_j)}{1+\sum_{j=1}^k \alpha_j}$
  • [marginal] The marginal distribution $P^{m}=(p_1,…,p_m)^T,(m\le k)$ is the dirichlet distribution, \[ Dir(P^{m}|(\alpha_1,…,\alpha_m),\sum_{i=m+1}^k \alpha_i) \]

  • [conditional] The conditional distribution of $P^{m}$ given the $(p_{m+1},…,p_k)$ of $Y_i=\frac{p_i}{1-\sum_{j=m+1}^k p_j}$ is also the dirichlet distribution, \[ Dir(Y^{m}|(\alpha_1,…,\alpha_m),\sum_{i=m+1}^k \alpha_i) \]

multinomial,binomial ,categorical ,bernoulli###

假设有$k$个桶,$n$个球,每个桶中的球的个数为$x_i$,概率为$p_i$.那么掉入每个桶中的球的个数服从以下分布,

分布球的个数桶数掉入桶的概率
multinomial$n$$k$$p\_1,p\_2,...,p\_k$
binomial$n$2$p$(另一个为$1-p$)
categorical1$k$$p\_1,p\_2,...,p\_k$
bernolli12$p$(另一个为$1-p$)