Factorial moments of the Binomial and related distributions

Factorial moments are defined as the expected value of the falling factorial of a random variable. In this post, we are going to try to compute the (scaled) Factorial moments of certain distributions without a direct computation.

The Factorial moment of a random variable $X$ is given by

$$\mathbb{E}[(X)_r]=\mathbb{E}[X(X-1)(X-2)\cdots (X-r+1)]$$

For our purposes, we focussing on the following definition which differs from the above only by a constant factor.

$$\displaystyle\mathbb{E}\left[\binom{X}{r}\right]=\mathbb{E}\left[\frac{(X)_r}{r!}\right]$$

Let's start with the famous Binomial distribution. The Binomial random variable $X$, with parameters $n$ and $p$, is the number of successes in a sequence of $n$ independent draws (with replacement), each with a success probability $p$.

Taking a cue from Problem of 7 of Strategic Practice 8 of Stat 110, $\binom{X}{k}$ denotes the set of draws where all the draws in the set result in a success. Creating an Indicator random variable for each set and using the linearity of expectations, we have

$$\displaystyle\mathbb{E}\left[\binom{X}{k}\right]=\binom{n}{k}\cdot p^k$$

We now move on to the Hypergeometric distribution (which is exactly the Stat 110 problem quoted above). Let $X$ denote the number of white balls from an Urn containing $N$ balls of which $K$ are white in a sample of $n$ draws. This is exactly the same problem given in Stat 110 quoted above.

$$\displaystyle\mathbb{E}\left[\binom{X}{k}\right]=\binom{n}{k}\cdot \frac{\binom{K}{k}}{\binom{N}{k}}$$

That is, we've considered an Indicator random variable for each $k$-set of draws and used the Linearity of Expectation. Note that $X$ is the number of successes where a 'success' is viewed as draw that gives a white ball.

Alternately, we can view a 'success' as a white ball that gets drawn. This way, we can solve the problem by considering an Indicator random variable for each $k$-set of white balls. Then,

$$\displaystyle\mathbb{E}\left[\binom{X}{k}\right]=\binom{K}{k}\cdot \frac{\binom{k}{k}\binom{N-k}{n-k}}{\binom{N}{n}}$$

Again, using the Linearity of Expectation, we restrict our attention to only a particular $k$-set of white balls. The probability that we are interested is that this particular set gets drawn when drawing $n$ balls. This is again a Hypergeometric distribution where only this particular set constitute 'successes' and rest are considered failures.

We now get to the Negative Binomial distribution with parameter $p$ and $j$ where $X$ denotes the number of failures we encounter before the $r$'th success. It can be verified with direct calculation that

$$\displaystyle\mathbb{E}\left[\binom{X}{k}\right]=\binom{k+r-1}{k}\left(\frac{q}{p}\right)^k$$

While the calculation is itself is not hard, I could not get a nice 'story' proof for the above. Rather we come back to this problem after the next case.

We now move to the Negative Hypergeometric distribution which will be most interesting aspect of this post. Here, again in the context of Urn and balls, $X$ denotes the number of black balls (failures) we get before the $r$'th white ball (success) from a population of $N$ balls in which $K$ are white.

Here again, $\binom{X}{k}$ denotes the $k$-set of draws that are black balls (failures) that gets drawn before the $r$'th white ball (success). Let $I_j,j \in \{1,2,\cdots,N-K\}$ be $1$ if the $j$th black ball (failure) gets drawn before the $r$'th white ball (success) and $0$ otherwise. Let the indicator random variable $I_S$ be the product of all the indicator random variable in the set $S$. Then,

$\displaystyle\mathbb{E}\left[\binom{X}{k}\right] =\sum_{S_k \subset \{1,2,\cdots,N-K\}}\mathbb{E}(I_{S_k})=\binom{N-K}{k}\mathbb{E}(I_1I_2\cdots I_k)=\binom{N-K}{k}\mathbb{P}(E)$

where $S_k$ denotes a $k$-element subset and $E$ denotes the event that failures $1$ to $k$ occur before the $r$'th success.

In other words, the probability that we want is the probability that we draw (without replacement) $k$ specific black balls from a bag containing $K$ white balls and $N-K$ black balls before the $r$'th white ball. This is a nice question by itself.

The key is to realise that we are concerned only about the $k$ specific black balls. This means we can completely ignore the remaining $N-K-k$ black balls and focus only on the $K$ white balls and the specific $k$ black balls.

The probability that we want is then the probability of getting $k$ black balls before the $r$'th white ball from an Urn containing $K+k$ balls of which $K$ are white. But that is exactly Negative Hypergeometric. Therefore,

$\displaystyle \mathbb{E}\left[\binom{X}{k}\right]=\binom{N-K}{k}\binom{k+r-1}{k}\frac{\binom{K+k-k-r}{K-r}}{\binom{K+k}{K}}=\binom{k+r-1}{k}\frac{\binom{N-K}{k}}{\binom{K+k}{k}}$

Alternately, the required probability can be seen as the probability of drawing $k$ black balls and $j-1$ white balls from an Urn containing $K+k$ balls of which $K$ are white. Then,

$\displaystyle \mathbb{E}\left[\binom{X}{k}\right]=\binom{N-K}{k}\frac{\binom{k}{k}\binom{K}{r-1}}{\binom{k+K}{k+r-1}}=\binom{N-K}{k}\frac{\binom{K}{r-1}}{\binom{k+K}{k+r-1}}$

Now we get back to the Negative Binomial case. Though I can't get a nice story proof for it, we can note that the first expression of the Negative Hypergeometric can be alternatively written as

$\displaystyle \mathbb{E}\left[\binom{X}{k}\right]=\binom{k+r-1}{k}\frac{\binom{N}{K+k}}{\binom{N}{K}}$

Using the asymptotic expression we derived in an earlier post, we can see that it matches with the result that we got from a direct calculation.

Hope you enjoyed the discussion. See ya in the next post.

Until then
Yours Aye
Me