Discrete Distributions

Summary

Distribution	pmf	Mean	Variance	mgf
Discrete Uniform $U(1,2,\dots ,n)$	$\frac{1}{n}$	$\frac{n+1}{2}$	$\frac{n^2-1}{12}$	$\frac{e^t(1-e^{nt})}{n(1-e^t)}$
Bernoulli $\text{Bernoulli}(p)$	$p^x(1-p{)}^{1-x}$	$p$	$p(1-p)$	$1-p+p{e}^t$
Binomial $B(n,p)$	$\left(\genfrac{}{}{0ex}{}{n}{x}\right)p^x(1-p{)}^{n-x}$	$np$	$np(1-p)$	$(1-p+p{e}^t{)}^n$
Poisson $\text{Poisson}(\lambda )$	$\frac{e^{-\lambda }{\lambda }^x}{x!}$	$\lambda$	$\lambda$	$e^{\lambda (e^t-1)}$
Negative Binomial $\text{NB}(r,p)$	$\left(\genfrac{}{}{0ex}{}{x+r-1}{r-1}\right)p^r(1-p{)}^x$	$\frac{r(1-p)}{p}$	$\frac{r(1-p)}{p^2}$	$p^r[1-(1-p)e^t{]}^{-r}$
Geometric $\text{Geom}(p)$	$(1-p{)}^{x}p$	$\frac{1-p}{p}$	$\frac{1-p}{p^2}$	$\frac{p}{1-(1-p)e^t}$
Hypergeometric $\text{Hypergeometric}(N,K,n)$	$\frac{\left(\genfrac{}{}{0ex}{}{K}{x}\right)\left(\genfrac{}{}{0ex}{}{N-K}{n-x}\right)}{\left(\genfrac{}{}{0ex}{}{N}{n}\right)}$	$n\frac{K}{N}$	$n\frac{K}{N}\left(1-\frac{K}{N}\right)\frac{N-n}{N-1}$	—

Discrete uniform distribution

$X\sim U(x_1,x_2,\dots ,x_n)$

Finite number of outcomes are equally likely

• pmf: $\frac{1}{n}$, $x\in x_1,x_2,\dots ,x_n$
• mean: $\frac{n+1}{2}$ (when $x_i=i$)
• var: $\frac{n^2-1}{12}$
• mgf: $\frac{e^t(1-e^{nt})}{n(1-e^t)}$

Bernoulli distribution

Definition

$X\sim \text{Bernoulli}(p)$

Models random experiment whose outcomes are “success” or “failure”.

• pmf: $p^x(1-p{)}^{1-x}$, $x\in 0,1$
• Mean: $p$
• Variance: $p(1-p)$
• mgf: $1-p+p{e}^t$

Link to original

Binomial distribution

Definition

$X\sim B(n,p)$

Models the number of successes in a fixed number of independent Bernoulli trials, each with the same probability of success

$x$ is the amount of success outcomes

• pmf: $\left(\genfrac{}{}{0ex}{}{n}{x}\right)p^x(1-p{)}^{n-x}$, $x\in 0,1,2,\dots ,n$
• Mean: $np$
• Variance: $np(1-p)$
• mgf: $(1-p+p{e}^t{)}^n$

Link to original

Trinomial distribution

Definition

$\mathbf{X}\sim \text{Trinomial}(n;p_1,p_2,p_3)$

Like binomial, but with 3 possible outcomes. Special case of multinomial with $k=3$.

• pmf: $\frac{n!}{x_1!x_2!x_3!}{p}_1^{x_1}{p}_2^{x_2}{p}_3^{x_3}$, where $x_1+x_2+x_3=n$
• mean: $E(X_i)=n{p}_i$
• var: $\text{Var}(X_i)=n{p}_i(1-p_i)$
• mgf: $(p_1{e}^{t_1}+p_2{e}^{t_2}+p_3{e}^{t_3}{)}^n$

Conditional distribution: $E(X_1|X_2=x_2)=(n-x_2)\left(\frac{p_1}{1-p_2}\right)$

Correlation coefficient: $\rho (X_1,X_2)=-\sqrt{\frac{p_1{p}_2}{(1-p_1)(1-p_2)}}$

Link to original

Multinomial distribution

Definition

$\mathbf{X}\sim \text{Multinomial}(n;p_1,p_2,\dots ,p_k)$

Extension of binomial distribution to $k$ categories.

$x_i$ is the number of success in the $i$-th variable.

• pmf: $\frac{n!}{x_1!\cdots x_k!}{p}_1^{x_1}\cdots p_k^{x_k}$
  • $x_k=n-(x_1+\cdots +x_{k-1})$
  • $p_k=1-(p_1+\cdots +p_{k-1})$
• Mean: $E(X_i)=n{p}_i$
• Variance: $\text{Var}(X_i)=n{p}_i(1-p_i)$
• Covariance: $\text{Cov}(X_i,X_j)=-n{p}_i{p}_j$

Link to original

Negative binomial distribution

Definition

$X\sim \text{NB}(r,p)$

Distribution of number of failures ($x$) needed to get the $r$-th success

• pmf: $\left(\genfrac{}{}{0ex}{}{x+r-1}{r-1}\right)p^r(1-p{)}^x$, $x\in 0,1,2,\dots$
• mean: $\frac{r(1-p)}{p}$
• var: $\frac{r(1-p)}{p^2}$
• mgf: $p^r[1-(1-p)e^t{]}^{-r}$, $t<-\ln (1-p)$

Link to original

Poisson distribution

Definition

$X\sim \text{Poisson}(\lambda )$

Models number of events occurring in fixed intervals of time/space when:

• Events occur at constant rate $\lambda$
• Events are independent of one another
• pmf: $\frac{e^{-\lambda }{\lambda }^x}{x!}$, $x\in 0,1,2,\dots$, $\lambda >0$
• mean: $\lambda$
• variance: $\lambda$
• mgf: $e^{\lambda (e^t-1)}$

Notes

When modeling count data, use negative binomial distribution instead of poisson distribution to handle overdispersion.

Count data refers to numerical data that represents the number of times an event occurs.

The Poisson distribution is suitable for count data when the mean and variance are approximately equal, assuming events occur independently and at a constant average rate.

However, in real-world count data, overdispersion often occurs, where the variance exceeds the mean due to factors like clustering or unobserved heterogeneity.

The negative binomial distribution accommodates this overdispersion by introducing an additional parameter to model the variance more flexibly.

Link to original

Geometric distribution

$X\sim \text{Geom}(p)$

Special case of negative binomial distribution with $r=1$. Distribution for number of failures needed for a single success.

• pmf: $(1-p{)}^{x}p$, $x\in 0,1,2,\dots$
• mean: $\frac{1-p}{p}$
• var: $\frac{1-p}{p^2}$
• mgf: $\frac{p}{1-(1-p)e^t}$, $t<-\ln (1-p)$

Link to original

Hypergeometric distribution

$X\sim \text{Hypergeometric}(N,K,n)$

Distribution for number of successes in $n$ draws from finite population of size $N$ containing exactly $K$ successes (sampling without replacement)

• pmf: $\frac{\left(\genfrac{}{}{0ex}{}{K}{x}\right)\left(\genfrac{}{}{0ex}{}{N-K}{n-x}\right)}{\left(\genfrac{}{}{0ex}{}{N}{n}\right)}$, $x\in \max (0,n-N+K),\dots ,\min (n,K)$
• mean: $n\frac{K}{N}$
• var: $n\frac{K}{N}\left(1-\frac{K}{N}\right)\frac{N-n}{N-1}$