Negative binomial distribution 45177 225837632 2008-07-15T17:06:38Z DutchCanadian 7481073 /* Waiting time in a Bernoulli process */ Minor spelling {{Probability distribution| name =Negative binomial| type =mass| pdf_image =[[Image:Negative_binomial.svg|380px]]The red line represents the mean, and the green line has an approximate length of 2&sigma;.| cdf_image =| parameters =<math>r > 0\!</math> ([[real number|real]])<br /><math>0<p<1\!</math> (real)| support =<math>k \in \{0,1,2,\ldots\}\!</math>| pdf =<math>\frac{\Gamma(r+k)}{k!\,\Gamma(r)}\,p^r\,(1-p)^k \!</math>| cdf =<math>I_p(r,k+1)\text{ where }I_p(x,y)</math> is the [[regularized incomplete beta function]]| mean =<math>r\,\frac{1-p}{p}\!</math>| median =| mode =<math>\lfloor(r-1)\,(1-p)/p\rfloor\text{ if }r>1</math><br /><math>0\text{ if }r\leq 1</math>| variance =<math>r\,\frac{1-p}{p^2}\!</math>| skewness =<math>\frac{2-p}{\sqrt{r\,(1-p)}}\!</math>| kurtosis =<math>\frac{6}{r} + \frac{p^2}{r\,(1-p)}\!</math>| entropy =| mgf =<math>\left(\frac{p}{1-(1-p) e^t}\right)^r \!</math>| char =<math>\left(\frac{p}{1-(1-p) e^{i\,t}}\right)^r \!</math> }} In [[probability]] and [[statistics]] the '''negative binomial distribution''' is a [[discrete probability distribution]]. It can be used to describe the distribution arising from an experiment consisting of a sequence of independent trials, subject to several constraints. Firstly each trial results in success or failure, the probability of success for each trial, ''p'', is constant across the experiment and finally the experiment continues until a fixed number of successes have been achieved. The '''Pascal distribution''' and the '''Polya distribution''' are special cases of the negative binomial. There is a convention among engineers, climatologists, and others to reserve "negative binomial" in a strict sense or "Pascal" (after [[Blaise Pascal]]) for the case of an integer-valued parameter ''r'', and use "[[Polya]]" (for [[George Pólya]]) for the real-valued case, to the right. The Polya distribution more accurately models occurrences of "contagious" discrete events, like tornado outbreaks, than does the [[Poisson distribution]]. ==Specification of the negative binomial distribution== ===Probability mass function=== The family of negative binomial distributions is a two-parameter family; several parametrizations are in common use. One very common parameterization employs two [[real number|real]]-valued parameters ''p'' and ''r'' with 0 &lt; ''p'' &lt; 1 and ''r'' &gt; 0. Under this parameterization, the [[probability mass function]] of a [[random variable]] with a NegBin(''r'', ''p'') distribution takes the following form: :<math> f(k;r,p) = {k+r-1 \choose k}\cdot p^r \cdot (1-p)^k \!</math> for ''k'' = 0,1,2,... where :<math> {k+r-1 \choose k} = \frac{\Gamma(k+r)}{k!\cdot\Gamma(r)} = (-1)^k\cdot{-r \choose k}\!</math> and :<math> \Gamma(r) = (r-1)!</math> ===Limiting case=== Under an alternative parameterization :<math> \lambda = r\cdot(p^{-1}-1) \!</math> :<math> p = \frac{r}{r+\lambda} \!</math> the mass function becomes :<math> g(k) = \frac{\lambda^k}{k!} \cdot \frac{\Gamma(r+k)}{\Gamma(r)\;(r+\lambda)^k} \cdot \frac{1}{\left(1+\frac{\lambda}{r}\right)^{r}} \!</math> where ''&lambda;'' and ''r'' are nonnegative real parameters. Under this parameterization, we have :<math> \lim_{r\to\infty} g(k) = \frac{\lambda^k}{k!} \cdot 1 \cdot \frac{1}{\exp(\lambda)} \!</math> which is the mass function of a [[Poisson distribution|Poisson-distributed]] random variable with Poisson rate ''&lambda;''. In other words, the alternatively parameterized negative binomial distribution [[convergence of random variables|converges]] to the Poisson distribution and ''r'' controls the deviation from the Poisson. This makes the negative binomial distribution suitable as a robust alternative to the Poisson, which approaches the Poisson for large ''r'', but which has larger variance than the Poisson for small ''r''. ===Gamma-Poisson mixture=== Third, the negative binomial distribution arises as a continuous mixture of Poisson distributions where the mixing distribution of the Poisson rate is a [[gamma distribution]]. Formally, this means that the mass function of the negative binomial distribution can also be written as {|- |<math>f(k)\!\!\!\!</math> |<math>= \int_0^{\infty} \mathrm{Poisson}(k \,|\, \lambda) \cdot \mathrm{Gamma}(\lambda \,|\, r, (1-p)/p) \; \mathrm{d}\lambda \!</math> |- | |<math>= \int_0^{\infty} \frac{\lambda^k}{k!} \exp(-\lambda) \cdot \frac{\lambda^{r-1} \exp(-\lambda p/(1-p))}{\Gamma(r)\;((1-p)/p)^r} \; \mathrm{d}\lambda \!</math> |- | |<math>= \frac{1}{k!\;\Gamma(r)} \; p^r \; \frac{1}{(1-p)^r} \; \int_0^{\infty} \lambda^{(r+k)-1} \, \exp(-\lambda/(1-p)) \;\mathrm{d}\lambda \!</math> |- | |<math>= \frac{1}{k!\;\Gamma(r)} \; p^r \; \frac{1}{(1-p)^r} \; (1-p)^{r+k} \; \Gamma(r+k) \!</math> |- | |<math>= \frac{\Gamma(r+k)}{k!\;\Gamma(r)} \; p^r \, (1-p)^k. \!</math> |} Because of this, the negative binomial distribution is also known as the '''gamma-Poisson (mixture) distribution'''. ===Cumulative distribution function=== The [[cumulative distribution function]] can be expressed in terms of the [[regularized incomplete beta function]]: :<math> F(k) = I_{p}(r, k+1). \!</math> ==Occurrence== ===Waiting time in a Bernoulli process=== For the special case where ''r'' is an integer, the negative binomial distribution is known as the '''Pascal distribution'''. It is the probability distribution of a certain number of failures and successes in a series of [[Independent identically-distributed random variables|independent and identically distributed]] [[Bernoulli trial]]s. For ''k''+''r'' Bernoulli trials with success probability ''p'', the negative binomial gives the probability of ''k'' failures and ''r'' successes, with success on the last trial. In other words, the negative binomial distribution is the probability distribution of the number of failures before the ''r''th success in a [[Bernoulli process]], with probability ''p'' of success on each trial. A Bernoulli process is a [[discrete random variable|discrete]] time process, and so the number of trials, failures, and successes are integers. Consider the following example. Suppose we repeatedly throw a die, and consider a "1" to be a "success". The probability of success on each trial is 1/6. The number of trials needed to get three successes belongs to the infinite set { 3, 4, 5, 6, ... }. That number of trials is a (displaced) negative-binomially distributed random variable. The number of failures before the third success belongs to the infinite set { 0, 1, 2, 3, ... }. That number of failures is also a negative-binomially distributed random variable. When ''r'' = 1 we get the probability distribution of failures before the first success (i.e. the probability of success on the (''k''+1)<sup>th</sup> trial), which is a [[geometric distribution]]: :<math> f(k) = p \cdot (1-p)^k \!</math> ===Overdispersed Poisson=== The negative binomial distribution, especially in its alternative parameterization described above, can be used as an alternative to the Poisson distribution. It is especially useful for discrete data over an unbounded positive range whose sample [[variance]] exceeds the sample [[mean]]. If a Poisson distribution is used to model such data, the model mean and variance are equal. In that case, the observations are ''overdispersed'' with respect to the Poisson model. Since the negative binomial distribution has one more parameter than the Poisson, the second parameter can be used to adjust the variance independently of the mean. See [[Cumulant#Cumulants of some discrete probability distributions]]. ==Related distributions== *The [[geometric distribution]] is a special case of the negative binomial distribution, with ::<math>\mathrm{Geometric}(p) = \mathrm{Neg Bin}(1, p).\,</math> *The negative binomial distribution [[convergence in distribution|converges]] to the [[Poisson distribution]] in the following sense: ::<math>\mathrm{Poisson}(\lambda) = \lim_{r \to \infty} \mathrm{NegBin}(r, r/(\lambda+r)).\,</math> *The negative binomial distribution is a special case of the [[discrete phase-type distribution]]. ==Properties== ===Relation to other distributions=== If ''X''<sub>''r''</sub> is a random variable following the negative binomial distribution with parameters ''r'' and ''p'', then ''X''<sub>''r''</sub> is a sum of ''r'' [[statistical independence|independent]] variables following the [[geometric distribution]] with parameter ''p''. As a result of the [[central limit theorem]], ''X''<sub>''r''</sub> is therefore approximately [[normal distribution|normal]] for sufficiently large ''r''. Furthermore, if ''Y''<sub>''s''+''r''</sub> is a random variable following the [[binomial distribution]] with parameters ''s'' + ''r'' and ''p'', then : <math> \begin{align} \Pr(X_r \leq s) & {} = I_p(r, s+1) \\ & {} = 1 - I_{1-p}(s+1, r) \\ & {} = 1 - I_{1-p}((s+r)-(r-1), (r-1)+1) \\ & {} = 1 - \Pr(Y_{s+r} \leq r-1) \\ & {} = \Pr(Y_{s+r} \geq r) \\ & {} = \Pr(\text{after } s+r \text{ trials, there are at least } r \text{ successes}). \end{align} </math> In this sense, the negative binomial distribution is the "inverse" of the binomial distribution. The sum of independent negative-binomially distributed random variables with the same value of the parameter ''p'' but the "''r''-values" ''r''<sub>1</sub> and ''r''<sub>2</sub> is negative-binomially distributed with the same ''p'' but with "''r''-value" ''r''<sub>1</sub> + ''r''<sub>2</sub>. The negative binomial distribution is [[infinite divisibility|infinitely divisible]], i.e., if ''X'' has a negative binomial distribution, then for any positive integer ''n'', there exist independent identically distributed random variables ''X''<sub>1</sub>, ..., ''X''<sub>''n''</sub> whose sum has the same distribution that ''X'' has. These will not be negative-binomially distributed in the sense defined above unless ''n'' is a divisor of ''r'' (more on this below). ===Sampling and point estimation of ''p''=== Suppose ''p'' is unknown and an experiment is conducted where it is decided ahead of time that sampling will continue until ''r'' successes are found. The sufficient statistics for the experiment is ''k'', the number of failures. In estimating ''p'', the minimum variance unbiased point estimator is <math>\hat{p}=\frac{r-1}{r+k-1}</math>. One might think the estimator is <math>\tilde{p}=\frac{r}{r+k}</math>, but this is biased. [http://links.jstor.org/sici?sici=0006-3444%28194511%2933%3A3%3C222%3AOAMOEF%3E2.0.CO%3B2-W Haldane Article] ===Relation to the binomial theorem=== Suppose ''K'' is a random variable with a negative binomial distribution with parameters ''r'' and ''p''. The statement that the sum from ''k'' = 0 to infinity, of the probability Pr[''K'' = ''k''], is equal to 1, can be shown by a bit of algebra to be equivalent to the statement that (1 &minus; ''p'')<sup>&minus; ''r''</sup> is what [[binomial series|Newton's binomial theorem]] says it should be. Suppose ''Y'' is a random variable with a [[binomial distribution]] with parameters ''n'' and ''p''. The statement that the sum from ''y'' = 0 to ''n'', of the probability Pr[''Y'' = ''y''], is equal to 1, says that 1 = (''p'' + (1 &minus; ''p''))<sup>''n''</sup> is what the strictly finitary [[binomial theorem]] of rudimentary algebra says it should be. Thus the negative binomial distribution bears the same relationship to the negative-integer-exponent case of the binomial theorem that the binomial distribution bears to the positive-integer-exponent case. Assume ''p'' + ''q'' = 1. Then the [[binomial theorem]] of elementary algebra implies that :<math>1=1^n=(p+q)^n=\sum_{k=0}^n {n \choose k}\cdot p^k\cdot q^{n-k}.</math> This can be written in a way that may at first appear to some to be incorrect, and perhaps perverse even if correct: :<math>(p+q)^n=\sum_{k=0}^\infty {n \choose k}\cdot p^k\cdot q^{n-k},</math> in which the upper bound of summation is infinite. The [[binomial coefficient]] :<math>{n \choose k}={n\cdot(n-1)\cdot(n-2)\cdots(n-k+1) \over k! }.</math> is defined even when ''n'' is negative or is not an integer. But in our case of the binomial distribution it is zero when ''k'' > ''n''. So ''why'' would we write the result in that form, with a seemingly needless sum of infinitely many zeros? The answer comes when we generalize the binomial theorem of elementary algebra to [[Newton's binomial theorem]]. Then we can say, for example :<math>(p+q)^{8.3}=\sum_{k=0}^\infty {8.3 \choose k}\cdot p^k\cdot q^{8.3 - k}.</math> Now suppose ''r'' > 0 and we use a negative exponent: :<math>1=p^r\cdot p^{-r}=p^r\cdot (1-q)^{-r}=p^r\cdot\sum_{k=0}^\infty {-r \choose k}\cdot (-q)^k.</math> Then all of the terms are positive, and the term :<math>p^r {-r \choose k}\cdot (-q)^k</math> is just the probability that the number of failures before the ''r''th success is equal to ''k'', provided ''r'' is an integer. (If ''r'' is a negative non-integer, so that the exponent is a positive non-integer, then some of the terms in the sum above are negative, so we do not have a probability distribution on the set of all nonnegative integers.) Now we also allow non-integer values of ''r''. Then we have a proper negative binomial distribution, which is a generalization of the Pascal distribution, which coincides with the Pascal distribution when ''r'' happens to be a positive integer. Recall from above that :The sum of independent negative-binomially distributed random variables with the same value of the parameter ''p'' but the "''r''-values" ''r''<sub>1</sub> and ''r''<sub>2</sub> is negative-binomially distributed with the same ''p'' but with "''r''-value" ''r''<sub>1</sub> + ''r''<sub>2</sub>. This property persists when the definition is thus generalized, and affords a quick way to see that the negative binomial distribution is [[infinite divisibility|infinitely divisible]]. ==Examples== ''(After a problem by Dr. Diane Evans, professor of mathematics at [[Rose-Hulman Institute of Technology]])'' Pat is required to sell candy bars to raise money for the 6th grade field trip. There are thirty houses in the neighborhood, and Pat is not supposed to return home until five candy bars have been sold. So the child goes door to door, selling candy bars. At each house, there is a 0.4 probability of selling one candy bar and a 0.6 probability of selling nothing. ''What's the probability mass function for selling the last candy bar at the ''n''<sup>th</sup> house?'' Recall that the NegBin(''r'', ''p'') distribution describes the probability of ''k'' failures and ''r'' successes in ''k''+''r'' Bernoulli(''p'') trials with success on the last trial. Selling five candy bars means getting five successes. The number of trials (i.e. houses) this takes is therefore ''k''+5 = ''n''. The random variable we are interested in is the number of houses, so we substitute ''k'' = ''n'' &minus; 5 into a NegBin(5, 0.4) mass function and obtain the following mass function of the distribution of houses (for ''n'' &ge; 5): :<math> f(n) = {(n-5) + 5 - 1 \choose 5-1} \; 0.4^5 \; 0.6^{n-5} = {n-1 \choose 4} \; 2^5 \; \frac{3^{n-5}}{5^n}. </math> ''What's the probability that Pat finishes on the tenth house?'' :<math> f(10) = 0.1003290624. \, </math> ''What's the probability that Pat finishes on or before reaching the eighth house?'' To finish on or before the eighth house, Pat must finish at the fifth, sixth, seventh, or eighth house. Sum those probabilities: :<math> f(5) = 0.01024 \, </math> :<math> f(6) = 0.03072 \, </math> :<math> f(7) = 0.055296 \, </math> :<math> f(8) = 0.0774144 \, </math> :<math>\sum_{j=5}^8 f(j) = 0.17367.</math> ''What's the probability that Pat [[The Little Match Girl|exhausts]] all 30 houses in the neighborhood?'' :<math>1-\sum_{j=5}^{30} f(j) = 1 - I_{0.4}(5, 30-5+1) \approx 1 - 0.99849 = 0.00151. </math> == See also == * [[Coupon collector's problem]] [[Category:Discrete distributions]] [[Category:Factorial and binomial topics]] {{ProbDistributions|discrete-infinite}} ==References== * Hilbe, Joseph M., ''Negative Binomial Regression'', Cambridge, UK: Cambridge University Press (2007)<ref>[http://www.cambridge.org/uk/catalogue/catalogue.asp?isbn=9780521857727 Negative Binomial Regression - Cambridge University Press<!-- Bot generated title -->]</ref> [[cs:Negativně binomické rozdělení]] [[de:Negative Binomialverteilung]] [[es:Distribución binomial negativa]] [[fa:توزیع دوجمله‌ای منفی]] [[it:Variabile casuale binomiale negativa]] [[hu:Negatív binomiális eloszlás]] [[nl:Negatief-binomiale verdeling]] [[ja:負の二項分布]] [[nov:Negativ binomial distributione]] [[pl:Rozkład Pascala]] [[pt:Distribuição binomial negativa]] [[ru:Отрицательное биномиальное распределение]] [[su:Sebaran binomial négatip]] [[fi:Negatiivinen binomijakauma]] [[zh:负二项分布]]