web.archive.org

Multinomial distribution: Definition from Answers.com

Multinomial

Probability mass function
Cumulative distribution function
Parameters	n > 0 number of trials (integer) $p_1, \ldots p_k$ event probabilities (Σp_i = 1)
Support	$X_i \in \{0,\dots,n\}$ $\Sigma X_i = n\!$
Probability mass function (pmf)	$\frac{n!}{x_1!\cdots x_k!} p_1^{x_1} \cdots p_k^{x_k}$
Cumulative distribution function (cdf)
Mean	E{X_i} = np_i
Median
Mode
Variance	Var(X_i) = np_i(1 − p_i) Cov(X_i,X_j) = − np_ip_j ( $i\neq j$ )
Skewness
Excess kurtosis
Entropy
Moment-generating function (mgf)	$\left( \sum_{i=1}^k p_i e^{t_i} \right)^n$
Characteristic function

In probability theory, the multinomial distribution is a generalization of the binomial distribution.

The binomial distribution is the probability distribution of the number of "successes" in n independent Bernoulli trials, with the same probability of "success" on each trial. In a multinomial distribution, each trial results in exactly one of some fixed finite number k of possible outcomes, with probabilities p₁, ..., p_k (so that p_i ≥ 0 for i = 1, ..., k and $\sum_{i=1}^k p_i = 1$ ), and there are n independent trials. Then let the random variables X_i indicate the number of times outcome number i was observed over the n trials. $X=(X_1,\ldots,X_k)$ follows a multinomial distribution with parameters n and p, where p = (p₁, ..., p_k).

Specification

Probability mass function

The probability mass function of the multinomial distribution is:

$\begin{align} f(x_1,\ldots,x_k;n,p_1,\ldots,p_k) & {} = \Pr(X_1 = x_1\mbox{ and }\dots\mbox{ and }X_k = x_k) \\ \\ & {} = \begin{cases} { \displaystyle {n! \over x_1!\cdots x_k!}p_1^{x_1}\cdots p_k^{x_k}}, \quad & \mbox{when } \sum_{i=1}^k x_i=n \\ \\ 0 & \mbox{otherwise,} \end{cases} \end{align}$

for non-negative integers x₁, ..., x_k.

Properties

The expected value of draws in the ith bin is

$\operatorname{E}(X_i) = n p_i.$

The covariance matrix is as follows. Each diagonal entry is the variance of a binomially distributed random variable, and is therefore

$\operatorname{var}(X_i)=np_i(1-p_i).$

The off-diagonal entries are the covariances:

$\operatorname{cov}(X_i,X_j)=-np_i p_j$

for i, j distinct.

All covariances are negative because for fixed N, an increase in one component of a multinomial vector requires a decrease in another component.

This is a k × k nonnegative-definite matrix of rank k − 1.

The off-diagonal entries of the corresponding correlation matrix are

$\rho(X_i,X_j) = -\sqrt{\frac{p_i p_j}{ (1-p_i)(1-p_j)}}.$

Note that the sample size drops out of this expression.

Each of the k components separately has a binomial distribution with parameters n and p_i, for the appropriate value of the subscript i.

The support of the multinomial distribution is the set : $\{(n_1,\dots,n_k)\in \mathbb{N}^{k}| n_1+\cdots+n_k=n\}.$ Its number of elements is

${n+k-1 \choose k} = \left\langle \begin{matrix}n \\ k \end{matrix}\right\rangle,$

the number of n-combinations of a multiset with k types, or multiset coefficient.

Sampling from a multinomial distribution

First, reorder the parameters $p_1, \ldots p_k$ such that they are sorted descendingly (this is only to speed up computation and not strictly necessary). Now, for each trial, draw an auxiliary variable x from a uniform (0,1) distribution. The resulting outcome is the component $j = \arg \min_{j'=1}^{k} \left( \sum_{i=1}^{j'} p_i \ge x \right)$ .

Related distributions

When k = 2, the multinomial distribution is the binomial distribution.
The Dirichlet distribution is the conjugate prior of the multinomial in Bayesian statistics.
Multivariate Polya distribution
Beta-binomial model

External links

Discrete Probability Distribution - Multinomial Distribution

References

Evans, Merran; Nicholas Hastings, Brian Peacock (2000). Statistical Distributions. New York: Wiley, 134-136. 3rd ed.. ISBN 0-471-37124-6.

v • d • e

Probability distributions

Discrete univariate with finite support

Benford · Bernoulli · binomial · categorical · hypergeometric · Rademacher · discrete uniform · Zipf · Zipf-Mandelbrot

Discrete univariate with infinite support

Boltzmann · Conway-Maxwell-Poisson · compound Poisson · discrete phase-type · extended negative binomial · Gauss-Kuzmin · geometric · logarithmic · negative binomial · parabolic fractal · Poisson · Skellam · Yule-Simon · zeta

Continuous univariate supported on a bounded interval, e.g. [0,1]

Beta · Irwin-Hall · Kumaraswamy · raised cosine · triangular · U-quadratic · uniform · Wigner semicircle

Continuous univariate supported on a semi-infinite interval, usually [0,∞)

Beta prime · Burr · chi-square · Coxian · Erlang · exponential · F · Fermi-Dirac · folded normal · Fréchet · Gamma · generalized extreme value · generalized inverse Gaussian · half-logistic · half-normal · Hotelling's T-square · hyper-exponential · hypoexponential · inverse chi-square (scale inverse chi-square) · inverse Gaussian · inverse gamma · Lévy · log-normal · log-logistic · Maxwell-Boltzmann · Maxwell speed · Nakagami · noncentral chi-square · Pareto · phase-type · Rayleigh · relativistic Breit–Wigner · Rice · Rosin–Rammler · shifted Gompertz · truncated normal · type-2 Gumbel · Weibull · Wilks' lambda

Continuous univariate supported on the whole real line (-∞,∞)

Cauchy · extreme value · exponential power · Fisher's z · Fisher-Tippett · generalized hyperbolic · hyperbolic secant · Landau · Laplace · Lévy skew alpha-stable · logistic · normal (Gaussian) · normal inverse Gaussian · skew normal · Student's t · type-1 Gumbel · Variance-Gamma · Voigt

Multivariate (joint)

Discrete: Ewens · Beta-binomial · multinomial · multivariate Polya Continuous: Dirichlet · Generalized Dirichlet · multivariate normal · multivariate Student · normal-scaled inverse gamma · normal-gamma Matrix-valued: inverse-Wishart · matrix normal · Wishart

Directional, degenerate, and singular

Directional: Kent · von Mises · von Mises–Fisher Degenerate: discrete degenerate · Dirac delta function Singular: Cantor

Families

exponential · natural exponential · location-scale · maximum entropy · Pearson · Tweedie

This entry is from Wikipedia, the leading user-contributed encyclopedia. It may not have been reviewed by professional editors (see full disclaimer)