TANSCHE SMTPT31 Mathematical Statistics Unit 3

SMTPT31

Mathematical Statistics
(TANSCHE Syllabus)

Madurai Kamaraj University
II- B.Sc., Mathematics, III - Semester

ARO Study Circle

Chapter 3
Discrete Distribution

Syllabus

Binomial, Poisson Continuous distribution: and Normal.
Chapter 7 - sections 7.2- 7.3. and Chapter 8 - sections 8.2 – 8.2.7

3.1 Binomial Distribution

The Binomial distribution was discovered by James Bernoulli (1654-1705) in the year 1700 and was first published posthumously in 1713, eight years after his death. It is one of the most important discrete probability distributions and serves as the foundation for many other distributions.

3.1.1 Definition and Derivation

Consider a random experiment that is performed repeatedly. Let the occurrence of an event in a trial be called a success and its non-occurrence a failure. Consider a set of \( n \) independent Bernoulli trials (\( n \) being finite), in which the probability \( p \) of success in any trial is constant for each trial. Then \( q = 1 - p \) is the probability of failure in any trial. The probability of \( x \) successes and consequently \( (n - x) \) failures in \( n \) independent trials, in a specified order (say) \( SSFSFFFS \ldots FSF \) (where \( S \) represents success and \( F \) represents failure) is given by the compound probability theorem: \[ P(SSFSFFFS \ldots FSF) = p \cdot p \cdot q \cdot p \cdot q \cdot q \cdot q \cdot p \ldots q \cdot p \cdot q = p^x q^{n-x} \] But \( x \) successes in \( n \) trials can occur in \( \binom {n}{x} \) ways and the probability for each of these ways is \( p^x q^{n-x} \). Hence the probability of \( x \) successes in \( n \) trials in any order whatsoever is given by the addition theorem of probability: \[ P(X = x) = \binom {n}{x} p^x q^{n-x}, \quad x = 0, 1, 2, \ldots , n \]

Definition 3.1.1 (Binomial Distribution). A random variable \( X \) is said to follow a binomial distribution if it assumes only non-negative values and its probability mass function is given by: \[ P(X = x) = p(x) = \binom {n}{x} p^x q^{n-x}, \quad x = 0, 1, 2, \ldots , n; \quad q = 1 - p \] and \( p(x) = 0 \), otherwise.

Remark 3.1.2.

1.: The two independent constants \( n \) and \( p \) in the distribution are known as the parameters of the distribution. \( n \) is also sometimes known as the degree of the binomial distribution.
2.: Binomial distribution is a discrete distribution as \( X \) can take only the integral values \( 0, 1, 2, \ldots , n \).
3.: We use the notation \( X \sim B(n, p) \) to denote that the random variable \( X \) follows a binomial distribution with parameters \( n \) and \( p \).
4.: The sum of probabilities is 1 because: \[ \sum _{x=0}^{n} p(x) = \sum _{x=0}^{n} \binom {n}{x} p^x q^{n-x} = (q + p)^n = 1 \]

3.1.2 Physical Conditions for Binomial Distribution

We get the binomial distribution under the following experimental conditions:

1.: Each trial results in two mutually disjoint outcomes, termed as success and failure.
2.: The number of trials \( n \) is finite.
3.: The trials are independent of each other.
4.: The probability of success \( p \) is constant for each trial.

The problems relating to tossing of a coin or throwing of dice or drawing cards from a pack of cards with replacement lead to binomial probability distribution.

Example 3.1.3 (Example 7.1). Ten coins are thrown simultaneously. Find the probability of getting at least seven heads.

Solution. Here \( p = \) Probability of getting a head \( = \frac {1}{2} \), \( q = \frac {1}{2} \). The probability of getting \( x \) heads in a random throw of 10 coins is: \[ p(x) = \binom {10}{x} \left (\frac {1}{2}\right )^x \left (\frac {1}{2}\right )^{10-x} = \binom {10}{x} \left (\frac {1}{2}\right )^{10}, \quad x = 0, 1, 2, \ldots , 10 \] Probability of getting at least seven heads: \begin {align*} P(X \geq 7) &= p(7) + p(8) + p(9) + p(10) \\ &= \left (\frac {1}{2}\right )^{10} \left \{ \binom {10}{7} + \binom {10}{8} + \binom {10}{9} + \binom {10}{10} \right \} \\ &= \frac {120 + 45 + 10 + 1}{1024} \\ &= \frac {176}{1024} = \frac {11}{64} \end {align*}

Example 3.1.4 (Example 7.2). \( A \) and \( B \) play a game in which their chances of winning are in the ratio \( 3 : 2 \). Find \( A \)’s chance of winning at least three games out of the five games played.

Solution. Let \( p \) be the probability that \( A \) wins the game. Then \( p = \frac {3}{5} \) and \( q = 1 - p = \frac {2}{5} \). By binomial probability law, the probability that out of 5 games played, \( A \) wins \( r \) games is: \[ P(X = r) = \binom {5}{r} \left (\frac {3}{5}\right )^r \left (\frac {2}{5}\right )^{5-r}, \quad r = 0, 1, 2, \ldots , 5 \] The required probability that \( A \) wins at least three games: \begin {align*} P(X \geq 3) &= \sum _{r=3}^{5} \binom {5}{r} \frac {3^r \cdot 2^{5-r}}{5^5} \\ &= \frac {27}{3125} \left [ \binom {5}{3} \cdot 2^2 + \binom {5}{4} \cdot 3 \cdot 2 + 1 \cdot 3^2 \cdot 1 \right ] \\ &= \frac {27}{3125} (40 + 30 + 9) \\ &= \frac {27 \times 79}{3125} = \frac {2133}{3125} = 0.68256 \end {align*}

Example 3.1.5 (Example 7.3). If \( m \) things are distributed among \( a \) men and \( b \) women, show that the probability that the number of things received by men is odd is: \[ \frac {1}{2} \left [ \frac {(b + a)^m - (b - a)^m}{(b + a)^m} \right ] \]

Solution. Let \( p = \) Probability that a thing is received by a man \( = \frac {a}{a+b} \). Then \( q = 1 - p = \frac {b}{a+b} \) is the probability that a thing is received by a woman. The probability that out of \( m \) things exactly \( x \) are received by men is: \[ p(x) = \binom {m}{x} p^x q^{m-x}, \quad x = 0, 1, 2, \ldots , m \] The probability \( P \) that the number of things received by men is odd is: \[ P = p(1) + p(3) + p(5) + \cdots \] \[ = \binom {m}{1} q^{m-1} p + \binom {m}{3} q^{m-3} p^3 + \binom {m}{5} q^{m-5} p^5 + \cdots \] Now, \[ (q + p)^m = q^m + \binom {m}{1} q^{m-1} p + \binom {m}{2} q^{m-2} p^2 + \binom {m}{3} q^{m-3} p^3 + \cdots \] and \[ (q - p)^m = q^m - \binom {m}{1} q^{m-1} p + \binom {m}{2} q^{m-2} p^2 - \binom {m}{3} q^{m-3} p^3 + \cdots \] Subtracting: \[ (q + p)^m - (q - p)^m = 2 \left [ \binom {m}{1} q^{m-1} p + \binom {m}{3} q^{m-3} p^3 + \cdots \right ] = 2P \] But \( q + p = 1 \) and \( q - p = \frac {b - a}{b + a} \). Hence: \begin {align*} 1 - \left ( \frac {b - a}{b + a} \right )^m &= 2P \\ P &= \frac {1}{2} \left [ \frac {(b + a)^m - (b - a)^m}{(b + a)^m} \right ] \end {align*}

Example 3.1.6 (Example 7.4). An irregular six faced die is thrown and the expectation that in 10 throws it will give five even numbers is twice the expectation that it will give four even numbers. How many times in 10,000 sets of 10 throws each, would you expect it to give no even number?

Solution. Let \( p \) be the probability of getting an even number in a throw of a die. Then: \[ P(X = x) = \binom {10}{x} p^x q^{10-x}, \quad x = 0, 1, 2, \ldots , 10 \] Given that \( P(X = 5) = 2P(X = 4) \): \begin {align*} \binom {10}{5} p^5 q^5 &= 2 \binom {10}{4} p^4 q^6 \\ \frac {10!}{5!5!} p^5 q^5 &= 2 \cdot \frac {10!}{4!6!} p^4 q^6 \\ \frac {1}{5!5!} p &= 2 \cdot \frac {1}{4!6!} q \\ \frac {p}{5!} &= \frac {2q}{6 \cdot 5!} \\ 3p &= q = 1 - p \\ 4p &= 1 \Rightarrow p = \frac {1}{4}, \quad q = \frac {3}{4} \end {align*} Hence: \[ P(X = 0) = q^{10} = \left ( \frac {3}{4} \right )^{10} \] The required number of times in 10,000 sets: \[ 10000 \times \left ( \frac {3}{4} \right )^{10} \approx 10000 \times 0.0563 = 563 \]

Example 3.1.7 (Example 7.5). In a precision bombing attack there is a \( 50\% \) chance that any one bomb will strike the target. Two direct hits are required to destroy the target completely. How many bombs must be dropped to give a \( 99\% \) chance or better of completely destroying the target?

Solution. We have \( p = \) Probability that the bomb strikes the target \( = \frac {1}{2} \). Let \( n \) be the number of bombs to be dropped. Let \( X \) be the number of bombs striking the target. Then \( X \sim B(n, p = \frac {1}{2}) \) with: \[ p(x) = \binom {n}{x} \left ( \frac {1}{2} \right )^x \left ( \frac {1}{2} \right )^{n-x} = \binom {n}{x} \left ( \frac {1}{2} \right )^n \] We need: \begin {align*} P(X \geq 2) &\geq 0.99 \\ 1 - P(X \leq 1) &\geq 0.99 \\ 1 - [P(X = 0) + P(X = 1)] &\geq 0.99 \\ 1 - \left [ \binom {n}{0} + \binom {n}{1} \right ] \left ( \frac {1}{2} \right )^n &\geq 0.99 \\ 1 - (1 + n) \left ( \frac {1}{2} \right )^n &\geq 0.99 \\ (1 + n) \left ( \frac {1}{2} \right )^n &\leq 0.01 \end {align*} By trial, we find that \( n = 11 \) satisfies this inequality. Hence the minimum number of bombs needed is 11.

Example 3.1.8 (Example 7.6). A department in a works has 10 machines which may need adjustment from time to time during the day. Three of these machines are old, each having a probability of \( 1/11 \) of needing adjustment during the day, and 7 are new, having corresponding probabilities of \( 1/21 \). Assuming that no machine needs adjustment twice on the same day, determine the probabilities that on a particular day: (i) just 2 old and no new machines need adjustment. (ii) If just 2 machines need adjustment, they are of the same type.

Solution. Let: \[ p_1 = \text {Probability that an old machine needs adjustment} = \frac {1}{11}, \quad q_1 = \frac {10}{11} \] \[ p_2 = \text {Probability that a new machine needs adjustment} = \frac {1}{21}, \quad q_2 = \frac {20}{21} \] Then: \[ P_1(r) = \binom {3}{r} p_1^r q_1^{3-r} = \binom {3}{r} \left ( \frac {1}{11} \right )^r \left ( \frac {10}{11} \right )^{3-r} \] \[ P_2(r) = \binom {7}{r} p_2^r q_2^{7-r} = \binom {7}{r} \left ( \frac {1}{21} \right )^r \left ( \frac {20}{21} \right )^{7-r} \] (i) Probability that just 2 old and no new machines need adjustment: \[ P_1(2) \cdot P_2(0) = \binom {3}{2} \left ( \frac {1}{11} \right )^2 \left ( \frac {10}{11} \right ) \left ( \frac {20}{21} \right )^7 = 0.016 \] (ii) Probability that just 2 new and no old machines need adjustment: \[ P_1(0) \cdot P_2(2) = \left ( \frac {10}{11} \right )^3 \binom {7}{2} \left ( \frac {1}{21} \right )^2 \left ( \frac {20}{21} \right )^5 = 0.028 \] Required probability: \[ 0.016 + 0.028 = 0.044 \]

3.1.3 Moments of Binomial Distribution

Theorem 3.1.9 (First Four Moments about Origin). For the binomial distribution \( B(n, p) \): \begin {align*} \mu _1' &= np \\ \mu _2' &= n(n-1)p^2 + np \\ \mu _3' &= n(n-1)(n-2)p^3 + 3n(n-1)p^2 + np \\ \mu _4' &= n(n-1)(n-2)(n-3)p^4 + 6n(n-1)(n-2)p^3 + 7n(n-1)p^2 + np \end {align*}

Proof : \begin {align*} \mu _1' = E(X) &= \sum _{x=0}^{n} x \binom {n}{x} p^x q^{n-x} \\ &= np \sum _{x=1}^{n} \binom {n-1}{x-1} p^{x-1} q^{n-x} \\ &= np(q+p)^{n-1} = np \end {align*} \begin {align*} \mu _2' = E(X^2) &= \sum _{x=0}^{n} [x(x-1) + x] \binom {n}{x} p^x q^{n-x} \\ &= n(n-1)p^2 \sum _{x=2}^{n} \binom {n-2}{x-2} p^{x-2} q^{n-x} + np \\ &= n(n-1)p^2(q+p)^{n-2} + np \\ &= n(n-1)p^2 + np \end {align*} Similarly, we can derive: \[ \mu _3' = n(n-1)(n-2)p^3 + 3n(n-1)p^2 + np \] \[ \mu _4' = n(n-1)(n-2)(n-3)p^4 + 6n(n-1)(n-2)p^3 + 7n(n-1)p^2 + np \] □

Theorem 3.1.10 (Central Moments of Binomial Distribution). \begin {align*} \mu _2 &= npq \\ \mu _3 &= npq(q-p) \\ \mu _4 &= npq[1 + 3pq(n-2)] \end {align*}

Proof : \begin {align*} \mu _2 &= \mu _2' - (\mu _1')^2 \\ &= n(n-1)p^2 + np - n^2 p^2 \\ &= np(1-p) = npq \end {align*} \begin {align*} \mu _3 &= \mu _3' - 3\mu _2'\mu _1' + 2(\mu _1')^3 \\ &= [n(n-1)(n-2)p^3 + 3n(n-1)p^2 + np] - 3[n(n-1)p^2 + np]np + 2(np)^3 \\ &= npq(q-p) \end {align*} \begin {align*} \mu _4 &= \mu _4' - 4\mu _3'\mu _1' + 6\mu _2'(\mu _1')^2 - 3(\mu _1')^4 \\ &= npq[1 + 3pq(n-2)] \end {align*} □

Corollary 3.1.11 (Skewness and Kurtosis). \begin {align*} \beta _1 &= \frac {\mu _3^2}{\mu _2^3} = \frac {(q-p)^2}{npq} \\ \beta _2 &= \frac {\mu _4}{\mu _2^2} = 3 + \frac {1 - 6pq}{npq} \end {align*}

Example 3.1.12 (Example 7.7). Comment on the following: The mean of a binomial distribution is 3 and variance is 4.

Solution. If the binomial distribution has parameters \( n \) and \( p \), then: \[ \text {Mean} = np = 3 \quad \text {and} \quad \text {Variance} = npq = 4 \] Dividing, we get \( q = \frac {4}{3} \), which is impossible since probability cannot exceed 1. Hence the statement is wrong.

Example 3.1.13 (Example 7.8). The mean and variance of a binomial distribution are 4 and \( \frac {4}{3} \) respectively. Find \( P(X \geq 1) \).

Solution. Let \( X \sim B(n, p) \). Then: \[ np = 4 \quad \text {and} \quad npq = \frac {4}{3} \] Dividing, \( q = \frac {1}{3} \). Hence \( p = \frac {2}{3} \). \[ n = \frac {4}{p} = \frac {4}{2/3} = 6 \] \[ P(X \geq 1) = 1 - P(X = 0) = 1 - q^6 = 1 - \left ( \frac {1}{3} \right )^6 = 1 - \frac {1}{729} = \frac {728}{729} \]

Example 3.1.14 (Example 7.9). If \( X \sim B(n, p) \), show that: \[ E\left ( \frac {X}{n} - p \right )^2 = \frac {pq}{n}, \quad \text {and} \quad \operatorname {Cov}\left ( \frac {X}{n}, \frac {n-X}{n} \right ) = -\frac {pq}{n} \]

Solution. Since \( X \sim B(n, p) \), we have \( E(X) = np \) and \( \operatorname {Var}(X) = npq \). (i) \begin {align*} E\left ( \frac {X}{n} - p \right )^2 &= \operatorname {Var}\left ( \frac {X}{n} \right ) \\ &= \frac {1}{n^2} \operatorname {Var}(X) \\ &= \frac {npq}{n^2} = \frac {pq}{n} \end {align*} (ii) \begin {align*} \operatorname {Cov}\left ( \frac {X}{n}, \frac {n-X}{n} \right ) &= E\left [ \left ( \frac {X}{n} - p \right ) \left ( \frac {n-X}{n} - q \right ) \right ] \\ &= E\left [ \left ( \frac {X}{n} - p \right ) \left ( 1 - \frac {X}{n} - q \right ) \right ] \\ &= E\left [ \left ( \frac {X}{n} - p \right ) \left ( p - \frac {X}{n} \right ) \right ] \\ &= -E\left ( \frac {X}{n} - p \right )^2 = -\frac {pq}{n} \end {align*}

3.1.4 Recurrence Relation for Moments of Binomial Distribution (Renovsky Formula)

Theorem 3.1.15 (Renovsky Formula). For the binomial distribution: \[ \mu _{r+1} = pq \left [ nr\mu _{r-1} + \frac {d\mu _r}{dp} \right ] \]

Proof : By definition: \[ \mu _r = E[X - E(X)]^r = \sum _{x=0}^{n} (x - np)^r \binom {n}{x} p^x q^{n-x} \] Differentiating with respect to \( p \): \begin {align*} \frac {d\mu _r}{dp} &= \sum _{x=0}^{n} \binom {n}{x} \left [ r(x - np)^{r-1}(-n) p^x q^{n-x} + (x - np)^r \left ( x p^{x-1} q^{n-x} - (n-x) p^x q^{n-x-1} \right ) \right ] \\ &= -nr \sum _{x=0}^{n} (x - np)^{r-1} \binom {n}{x} p^x q^{n-x} + \sum _{x=0}^{n} (x - np)^r \binom {n}{x} p^x q^{n-x} \left ( \frac {x}{p} - \frac {n-x}{q} \right ) \\ &= -nr\mu _{r-1} + \frac {1}{pq} \sum _{x=0}^{n} (x - np)^{r+1} \binom {n}{x} p^x q^{n-x} \\ &= -nr\mu _{r-1} + \frac {1}{pq} \mu _{r+1} \end {align*} Therefore: \[ \mu _{r+1} = pq \left [ nr\mu _{r-1} + \frac {d\mu _r}{dp} \right ] \] □

Putting \( r = 1, 2, 3 \) successively, we get: \begin {align*} \mu _2 &= pq\left [ n \cdot 1 \cdot \mu _0 + \frac {d\mu _1}{dp} \right ] = pq[n \cdot 1 \cdot 1 + 0] = npq \\ \mu _3 &= pq\left [ n \cdot 2 \cdot \mu _1 + \frac {d\mu _2}{dp} \right ] = pq[2n \cdot 0 + n(q-p)] = npq(q-p) \\ \mu _4 &= pq\left [ n \cdot 3 \cdot \mu _2 + \frac {d\mu _3}{dp} \right ] = npq[1 + 3pq(n-2)] \end {align*}

Example 3.1.16 (Example 7.10). Show that the \( r \)th moment \( \mu _r' \) about the origin of the binomial distribution of degree \( n \) is given by: \[ \mu _r' = \left ( p \frac {\partial }{\partial p} \right )^r (q + p)^n \]

Solution. We have: \[ (q + p)^n = \sum _{x=0}^{n} \binom {n}{x} p^x q^{n-x} \] \[ \frac {\partial }{\partial p} (q + p)^n = \sum _{x=0}^{n} \binom {n}{x} q^{n-x} x p^{x-1} \] \[ p \frac {\partial }{\partial p} (q + p)^n = \sum _{x=0}^{n} \binom {n}{x} p^x q^{n-x} x = E(X) = \mu _1' \] Thus the result is true for \( r = 1 \). Assume it is true for \( r = k \): \[ \left ( p \frac {\partial }{\partial p} \right )^k (q + p)^n = \mu _k' = \sum _{x=0}^{n} \binom {n}{x} p^x q^{n-x} x^k \] Differentiating partially with respect to \( p \) and multiplying by \( p \): \[ p \frac {\partial }{\partial p} \left [ \left ( p \frac {\partial }{\partial p} \right )^k (q + p)^n \right ] = \sum _{x=0}^{n} \binom {n}{x} p^x q^{n-x} x^{k+1} = \mu _{k+1}' \] \[ \left ( p \frac {\partial }{\partial p} \right )^{k+1} (q + p)^n = \mu _{k+1}' \] Hence by induction, the result holds for all positive integers \( r \).

3.1.5 Factorial Moments of Binomial Distribution

Definition 3.1.17 (Factorial Moments). The \( r \)th factorial moment of a random variable \( X \) is defined as: \[ \mu _{(r)}' = E[X^{(r)}] = E[X(X-1)(X-2)\cdots (X-r+1)] \]

Theorem 3.1.18 (Factorial Moments of Binomial Distribution). For the binomial distribution \( B(n, p) \): \[ \mu _{(r)}' = n^{(r)} p^r = n(n-1)(n-2)\cdots (n-r+1)p^r \]

Proof : \begin {align*} \mu _{(r)}' &= E[X^{(r)}] = \sum _{x=0}^{n} x^{(r)} \binom {n}{x} p^x q^{n-x} \\ &= \sum _{x=r}^{n} x(x-1)\cdots (x-r+1) \frac {n!}{x!(n-x)!} p^x q^{n-x} \\ &= n^{(r)} p^r \sum _{x=r}^{n} \binom {n-r}{x-r} p^{x-r} q^{n-x} \\ &= n^{(r)} p^r (q+p)^{n-r} = n^{(r)} p^r \end {align*} □

In particular: \begin {align*} \mu _{(1)}' &= np = \text {Mean} \\ \mu _{(2)}' &= n(n-1)p^2 \\ \mu _{(3)}' &= n(n-1)(n-2)p^3 \end {align*}

3.1.6 Mean Deviation About Mean

Theorem 3.1.19 (Mean Deviation About Mean). The mean deviation \( \eta \) about the mean \( np \) of the binomial distribution is: \[ \eta = 2npq \binom {n-1}{\mu -1} p^{\mu -1} q^{n-\mu } \] where \( \mu \) is the greatest integer contained in \( np + 1 \).

3.1.7 Mode of the Binomial Distribution

Theorem 3.1.20 (Mode of Binomial Distribution). The mode of the binomial distribution \( B(n, p) \) is:

1.: The integral part of \( (n+1)p \), if \( (n+1)p \) is not an integer.
2.: Both \( (n+1)p - 1 \) and \( (n+1)p \), if \( (n+1)p \) is an integer.

Proof : We have: \[ \frac {p(x)}{p(x-1)} = \frac {(n-x+1)p}{xq} = 1 + \frac {(n+1)p - x}{xq} \] Case 1: When \( (n+1)p \) is not an integer. Let \( (n+1)p = m + f \), where \( m \) is an integer and \( 0 < f < 1 \). Then: \[ \frac {p(x)}{p(x-1)} > 1 \text { for } x = 0, 1, 2, \ldots , m \] and \[ \frac {p(x)}{p(x-1)} < 1 \text { for } x = m+1, m+2, \ldots , n \] Thus: \[ p(0) < p(1) < p(2) < \cdots < p(m) > p(m+1) > p(m+2) > \cdots > p(n) \] Hence the unique mode is \( m \), the integral part of \( (n+1)p \). Case 2: When \( (n+1)p = m \) is an integer. Then: \[ \frac {p(m)}{p(m-1)} = 1 \] Thus: \[ p(0) < p(1) < \cdots < p(m-1) = p(m) > p(m+1) > \cdots > p(n) \] Hence the distribution is bimodal with modes at \( m-1 \) and \( m \). □

Example 3.1.21 (Example 7.11). Determine the binomial distribution for which the mean is 4 and variance 3 and find its mode.

Solution. Let \( X \sim B(n, p) \). Then: \[ np = 4 \quad \text {and} \quad npq = 3 \] Dividing, \( q = \frac {3}{4} \), so \( p = \frac {1}{4} \). \[ n = \frac {4}{p} = 16 \] Thus \( X \sim B(16, \frac {1}{4}) \). Mode: \( (n+1)p = 17 \times \frac {1}{4} = 4.25 \), which is not an integer. Hence the unique mode is 4, the integral part of \( (n+1)p \).

Example 3.1.22 (Example 7.12). Show that for \( p = 0.50 \), the binomial distribution has a maximum probability at \( X = \frac {1}{2}n \), if \( n \) is even, and at \( X = \frac {1}{2}(n-1) \) as well as \( X = \frac {1}{2}(n+1) \), if \( n \) is odd.

Solution. We need to find the mode of the binomial distribution. (i) Let \( n \) be even \( = 2m \), \( m = 1, 2, \ldots \) \[ (n+1)p = (2m+1) \times \frac {1}{2} = m + 0.5 \] Hence the unique mode is at \( X = m = \frac {n}{2} \). (ii) Let \( n \) be odd \( = 2m+1 \). Then: \[ (n+1)p = (2m+2) \times \frac {1}{2} = m+1 \] Since \( (n+1)p \) is an integer, the distribution is bimodal with modes at \( m \) and \( m+1 \), i.e., at \( \frac {n-1}{2} \) and \( \frac {n+1}{2} \).

3.1.8 Moment Generating Function of Binomial Distribution

Theorem 3.1.23 (Moment Generating Function). The moment generating function of the binomial distribution \( B(n, p) \) is: \[ M_X(t) = (q + pe^t)^n \]

Proof : \begin {align*} M_X(t) = E(e^{tX}) &= \sum _{x=0}^{n} e^{tx} \binom {n}{x} p^x q^{n-x} \\ &= \sum _{x=0}^{n} \binom {n}{x} (pe^t)^x q^{n-x} \\ &= (q + pe^t)^n \end {align*} □

Theorem 3.1.24 (Moment Generating Function about Mean). The m.g.f. about the mean of the binomial distribution is: \[ M_{X-np}(t) = (qe^{-pt} + pe^{qt})^n \]

Proof : \begin {align*} M_{X-np}(t) &= e^{-tnp} M_X(t) \\ &= e^{-tnp} (q + pe^t)^n \\ &= [qe^{-pt} + pe^{t(1-p)}]^n \\ &= [qe^{-pt} + pe^{qt}]^n \end {align*} □

Example 3.1.25 (Example 7.13). \( X \) is binomially distributed with parameters \( n \) and \( p \). What is the distribution of \( Y = n - X \)?

Solution. \( X \sim B(n, p) \) represents the number of successes in \( n \) independent trials with constant probability \( p \) of success for each trial. \( Y = n - X \) represents the number of failures in \( n \) independent trials with constant probability \( q \) of failure for each trial. Hence \( Y \sim B(n, q) \). Aliter: Since \( X \sim B(n, p) \), \( M_X(t) = (q + pe^t)^n \). \[ M_Y(t) = E(e^{tY}) = E(e^{t(n-X)}) = e^{nt} E(e^{-tX}) = e^{nt} M_X(-t) \] \[ = e^{nt} (q + pe^{-t})^n = [e^t(q + pe^{-t})]^n = (p + qe^t)^n \] Hence by the uniqueness theorem of m.g.f., \( Y = n - X \sim B(n, q) \).

Example 3.1.26 (Example 7.14). The m.g.f. of a r.v. \( X \) is \( \left ( \frac {2}{3} + \frac {1}{3} e^t \right )^9 \). Show that: \[ P(\mu - 2\sigma < X < \mu + 2\sigma ) = \sum _{x=1}^{5} \binom {9}{x} \left ( \frac {1}{3} \right )^x \left ( \frac {2}{3} \right )^{9-x} \]

Solution. Since \( M_X(t) = \left ( \frac {2}{3} + \frac {1}{3} e^t \right )^9 = (q + pe^t)^n \), by uniqueness theorem of m.g.f., \( X \sim B(n=9, p=\frac {1}{3}) \). \[ E(X) = \mu _X = np = 3, \quad \sigma _X^2 = npq = 9 \times \frac {1}{3} \times \frac {2}{3} = 2 \] \[ \mu \pm 2\sigma = 3 \pm 2 \times \sqrt {2} = 3 \pm 2 \times 1.4 = (0.2, 5.8) \] \[ P(\mu - 2\sigma < X < \mu + 2\sigma ) = P(0.2 < X < 5.8) = P(1 \leq X \leq 5) \] \[ = \sum _{x=1}^{5} \binom {9}{x} \left ( \frac {1}{3} \right )^x \left ( \frac {2}{3} \right )^{9-x} \]

3.1.9 Additive Property of Binomial Distribution

Theorem 3.1.27 (Additive Property). If \( X_1, X_2, \ldots , X_k \) are independent binomial variates with parameters \( (n_i, p) \), \( i = 1, 2, \ldots , k \), then their sum: \[ \sum _{i=1}^{k} X_i \sim B \left ( \sum _{i=1}^{k} n_i, p \right ) \]

Proof : For \( X \sim B(n_1, p) \) and \( Y \sim B(n_2, p) \), independent: \[ M_{X+Y}(t) = M_X(t) M_Y(t) = (q + pe^t)^{n_1} (q + pe^t)^{n_2} = (q + pe^t)^{n_1 + n_2} \] Hence \( X + Y \sim B(n_1 + n_2, p) \). By induction, the result extends to \( k \) variables. □

Remark 3.1.28. If \( p_1 \neq p_2 \), then \( X + Y \) is not a binomial variate because: \[ M_{X+Y}(t) = (q_1 + p_1 e^t)^{n_1} (q_2 + p_2 e^t)^{n_2} \] which cannot be expressed as \( (q + pe^t)^n \). Thus, in general, the sum of two independent binomial variates is not a binomial variate.

Example 3.1.29 (Example 7.15). If the independent random variables \( X, Y \) are binomially distributed, respectively with \( n = 3 \), \( p = 1/3 \), and \( n = 5 \), \( p = 1/3 \), write down the probability that \( X + Y \geq 1 \).

Solution. We are given \( X \sim B(3, \frac {1}{3}) \) and \( Y \sim B(5, \frac {1}{3}) \). Since \( X \) and \( Y \) are independent binomial random variables with \( p_1 = p_2 = \frac {1}{3} \), by the additive property: \[ X + Y \sim B(3 + 5, \frac {1}{3}) = B(8, \frac {1}{3}) \] \[ P(X + Y = r) = \binom {8}{r} \left ( \frac {1}{3} \right )^r \left ( \frac {2}{3} \right )^{8-r} \] Hence: \[ P(X + Y \geq 1) = 1 - P(X + Y = 0) = 1 - \left ( \frac {2}{3} \right )^8 \]

3.1.10 Characteristic Function of Binomial Distribution

Theorem 3.1.30 (Characteristic Function). The characteristic function of the binomial distribution \( B(n, p) \) is: \[ \phi _X(t) = (q + pe^{it})^n \]

Proof : \begin {align*} \phi _X(t) = E(e^{itX}) &= \sum _{x=0}^{n} e^{itx} \binom {n}{x} p^x q^{n-x} \\ &= \sum _{x=0}^{n} \binom {n}{x} (pe^{it})^x q^{n-x} \\ &= (q + pe^{it})^n \end {align*} □

3.1.11 Cumulants of Binomial Distribution

Theorem 3.1.31 (Cumulants). The cumulants of the binomial distribution \( B(n, p) \) are: \begin {align*} \kappa _1 &= np \\ \kappa _2 &= npq \\ \kappa _3 &= npq(q-p) \\ \kappa _4 &= npq(1-6pq) \end {align*}

Proof : The cumulant generating function is: \[ K_X(t) = \log M_X(t) = \log (q + pe^t)^n = n \log (q + pe^t) \] Expanding \( \log (q + pe^t) \) in powers of \( t \): \[ \log (q + pe^t) = p\left ( t + \frac {t^2}{2!} + \frac {t^3}{3!} + \cdots \right ) - \frac {p^2}{2} \left ( t + \frac {t^2}{2!} + \cdots \right )^2 + \frac {p^3}{3} \left ( t + \frac {t^2}{2!} + \cdots \right )^3 - \cdots \] Comparing coefficients of \( t, \frac {t^2}{2!}, \frac {t^3}{3!}, \frac {t^4}{4!} \): \begin {align*} \kappa _1 &= np \\ \kappa _2 &= npq \\ \kappa _3 &= npq(q-p) \\ \kappa _4 &= npq(1-6pq) \end {align*} □

3.1.12 Recurrence Relation for Cumulants

Theorem 3.1.32 (Recurrence Relation for Cumulants). For the binomial distribution: \[ \kappa _{r+1} = pq \frac {d\kappa _r}{dp} \]

Proof : From the cumulant generating function: \[ K_X(t) = n \log (q + pe^t) \] Differentiating \( r \) times: \[ \kappa _r = \left [ \frac {d^r}{dt^r} K_X(t) \right ]_{t=0} \] It can be shown (by differentiation with respect to \( p \)) that: \[ \kappa _{r+1} = pq \frac {d\kappa _r}{dp} \] In particular: \begin {align*} \kappa _2 &= pq \frac {d\kappa _1}{dp} = pq \frac {d(np)}{dp} = npq \\ \kappa _3 &= pq \frac {d\kappa _2}{dp} = pq \frac {d(npq)}{dp} = npq(q-p) \\ \kappa _4 &= pq \frac {d\kappa _3}{dp} = npq(1-6pq) \end {align*} □

3.1.13 Probability Generating Function

Theorem 3.1.33 (Probability Generating Function). The probability generating function of the binomial distribution \( B(n, p) \) is: \[ P(s) = \sum _{x=0}^{n} p(x) s^x = (q + ps)^n \]

Proof : \[ P(s) = \sum _{x=0}^{n} \binom {n}{x} p^x q^{n-x} s^x = \sum _{x=0}^{n} \binom {n}{x} (ps)^x q^{n-x} = (q + ps)^n \] □

Remark 3.1.34. The factorial moments can be obtained from the probability generating function: \[ \mu _{(r)}' = \left [ \frac {d^r}{ds^r} P(s) \right ]_{s=1} \] For the binomial distribution: \[ \mu _{(1)}' = [n(q+ps)^{n-1}p]_{s=1} = np \] \[ \mu _{(2)}' = [n(n-1)(q+ps)^{n-2}p^2]_{s=1} = n(n-1)p^2 \]

3.1.14 Recurrence Relation for Probabilities (Fitting of Binomial Distribution)

Theorem 3.1.35 (Recurrence Relation for Probabilities). For the binomial distribution \( B(n, p) \): \[ p(x+1) = \frac {n-x}{x+1} \cdot \frac {p}{q} \cdot p(x), \quad x = 0, 1, 2, \ldots , n-1 \]

Proof : \begin {align*} p(x+1) &= \binom {n}{x+1} p^{x+1} q^{n-x-1} \\ p(x) &= \binom {n}{x} p^x q^{n-x} \end {align*} \[ \frac {p(x+1)}{p(x)} = \frac {\binom {n}{x+1}}{\binom {n}{x}} \cdot \frac {p}{q} = \frac {n-x}{x+1} \cdot \frac {p}{q} \] Hence: \[ p(x+1) = \frac {n-x}{x+1} \cdot \frac {p}{q} \cdot p(x) \] □

This formula provides a convenient method of fitting a binomial distribution to observed data. The only probability we need to calculate is: \[ p(0) = q^n \] where \( q \) is estimated by equating the sample mean \( \bar {x} \) to \( np \), i.e., \( \hat {p} = \bar {x}/n \). Then: \[ p(1) = \frac {n}{1} \cdot \frac {p}{q} \cdot p(0) \] \[ p(2) = \frac {n-1}{2} \cdot \frac {p}{q} \cdot p(1) \] \[ p(3) = \frac {n-2}{3} \cdot \frac {p}{q} \cdot p(2) \] and so on.

Example 3.1.36 (Example 7.17). Seven coins are tossed and number of heads noted. The experiment is repeated 128 times and the following distribution is obtained: \[ \begin {array}{c|cccccccc} \text {No. of heads} & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\ \hline \text {Frequencies} & 7 & 6 & 19 & 35 & 30 & 23 & 7 & 1 \end {array} \] Fit a Binomial distribution assuming: (i) The coin is unbiased, (ii) The nature of the coin is not known, (iii) Probability of a head for four coins is 0.5 and for the remaining three coins is 0.45.

Solution. In fitting Binomial distribution, first of all, the mean and variance of the data are equated to \( np \) and \( npq \) respectively. Then the expected frequencies are calculated. Here \( n = 7 \) and \( N = 128 \). Case I: When the coin is unbiased: \[ p = q = \frac {1}{2}, \quad \frac {p}{q} = 1 \] \[ p(0) = q^n = \left ( \frac {1}{2} \right )^7 = \frac {1}{128} \] \[ f(0) = Nq^n = 128 \times \frac {1}{128} = 1 \] Using the recurrence formula: \begin {align*} p(1) &= \frac {7}{1} \cdot 1 \cdot p(0) = 7p(0), \quad f(1) = 7 \times 1 = 7 \\ p(2) &= \frac {6}{2} \cdot 1 \cdot p(1) = 3p(1), \quad f(2) = 7 \times 3 = 21 \\ p(3) &= \frac {5}{3} \cdot 1 \cdot p(2) = \frac {5}{3}p(2), \quad f(3) = 21 \times \frac {5}{3} = 35 \\ p(4) &= \frac {4}{4} \cdot 1 \cdot p(3) = p(3), \quad f(4) = 35 \times 1 = 35 \\ p(5) &= \frac {3}{5} \cdot 1 \cdot p(4) = \frac {3}{5}p(4), \quad f(5) = 35 \times \frac {3}{5} = 21 \\ p(6) &= \frac {2}{6} \cdot 1 \cdot p(5) = \frac {1}{3}p(5), \quad f(6) = 21 \times \frac {1}{3} = 7 \\ p(7) &= \frac {1}{7} \cdot 1 \cdot p(6) = \frac {1}{7}p(6), \quad f(7) = 7 \times \frac {1}{7} = 1 \end {align*} Case II: When the nature of the coin is not known: \[ np = \frac {1}{N} \sum f_i x_i = \frac {433}{128} = 3.3828, \quad n = 7 \] \[ p = 0.48326, \quad q = 0.51674, \quad \frac {p}{q} = 0.93521 \] \[ f(0) = Nq^7 = 128(0.51674)^7 = 1.2593 \] Using the recurrence formula: \begin {align*} f(1) &= \frac {7}{1} \times 0.93521 \times 1.2593 = 8.9438 \approx 9 \\ f(2) &= \frac {6}{2} \times 0.93521 \times 8.9438 = 23.129 \approx 23 \\ f(3) &= \frac {5}{3} \times 0.93521 \times 23.129 = 36.05 \approx 36 \\ f(4) &= \frac {4}{4} \times 0.93521 \times 36.05 = 33.715 \approx 34 \\ f(5) &= \frac {3}{5} \times 0.93521 \times 33.715 = 18.918 \approx 19 \\ f(6) &= \frac {2}{6} \times 0.93521 \times 18.918 = 5.897 \approx 6 \\ f(7) &= \frac {1}{7} \times 0.93521 \times 5.897 = 0.788 \approx 1 \end {align*} Case III: When probability of a head for four coins is 0.5 and for the remaining three coins is 0.45. The probability generating functions for the 4 coins and the remaining 3 coins are: \[ P_X(s) = (0.50 + 0.50s)^4, \quad P_Y(s) = (0.55 + 0.45s)^3 \] The p.g.f. for the whole experiment is: \[ P_{X+Y}(s) = P_X(s)P_Y(s) = (0.50 + 0.50s)^4(0.55 + 0.45s)^3 \] Then \( f(x) = 128 \times \text {coefficient of } s^x \) in \( P_{X+Y}(s) \). The expected frequencies are obtained accordingly.

Example 3.1.37 (Example 7.18). Let \( X \) and \( Y \) be independent binomial variates, each with parameters \( n \) and \( p \). Find \( P(X - Y = k) \).

Solution. Since each of the variables \( X \) and \( Y \) takes the values \( 0, 1, 2, \ldots , n \), \( Z = X - Y \) takes the values \( -n, -(n-1), \ldots , -1, 0, 1, \ldots , n \). \[ P(Z = k) = \sum _{r=0}^{n} P(X = k + r \cap Y = r) \] \[ = \sum _{r=0}^{n} P(X = k + r) P(Y = r) \quad (\text {since } X \text { and } Y \text { are independent}) \] \[ = \sum _{r=0}^{n} \binom {n}{k+r} \binom {n}{r} p^{2r+k} q^{2n-2r-k} \] where \( k = -n, -(n-1), \ldots , -1, 0, 1, \ldots , n \) and \( q = 1 - p \). In particular: \[ P(Z = 0) = \sum _{r=0}^{n} \binom {n}{r}^2 p^{2r} q^{2n-2r} \] \[ P(Z = -n) = p^n q^n \]

Example 3.1.38 (Example 7.19). Find the m.g.f. of standard binomial variate \( (X - np)/\sqrt {npq} \) and obtain its limiting form as \( n \to \infty \). Also interpret the result.

Solution. We know that if \( X \sim B(n, p) \), then: \[ M_X(t) = (q + pe^t)^n \] The m.g.f. of the standard binomial variate \( Z = \frac {X - np}{\sqrt {npq}} \) is: \begin {align*} M_Z(t) &= e^{-npt/\sqrt {npq}} M_X(t/\sqrt {npq}) \\ &= e^{-npt/\sqrt {npq}} \left ( q + pe^{t/\sqrt {npq}} \right )^n \\ &= \left [ q e^{-pt/\sqrt {npq}} + p e^{qt/\sqrt {npq}} \right ]^n \end {align*} Expanding using Taylor series: \begin {align*} M_Z(t) &= \left [ q \left ( 1 - \frac {pt}{\sqrt {npq}} + \frac {p^2 t^2}{2npq} + O(n^{-3/2}) \right ) + p \left ( 1 + \frac {qt}{\sqrt {npq}} + \frac {q^2 t^2}{2npq} + O(n^{-3/2}) \right ) \right ]^n \\ &= \left [ 1 + \frac {t^2}{2n} + O(n^{-3/2}) \right ]^n \end {align*} Taking logarithm: \[ \log M_Z(t) = n \log \left [ 1 + \frac {t^2}{2n} + O(n^{-3/2}) \right ] = \frac {t^2}{2} + O(n^{-1/2}) \] Proceeding to the limit as \( n \to \infty \): \[ \lim _{n \to \infty } \log M_Z(t) = \frac {t^2}{2} \] \[ \lim _{n \to \infty } M_Z(t) = e^{t^2/2} \] Interpretation: \( e^{t^2/2} \) is the m.g.f. of a standard normal variate. Hence, by the uniqueness theorem of m.g.f., the standard binomial variate tends to the standard normal variate as \( n \to \infty \). In other words, the binomial distribution tends to the normal distribution as \( n \to \infty \).

Example 3.1.39 (Example 7.20). A drunk performs a random walk over positions \( 0, \pm 1, \pm 2, \ldots \) as follows. He starts at 0. He takes successive one unit steps, going to the right with probability \( p \) and to the left with probability \( (1-p) \). His steps are independent. Let \( X \) denote his position after \( n \) steps. Find the distribution of \( (X+n)/2 \) and find \( E(X) \).

Solution. With the \( i \)th step, associate a variable \( X_i \) defined as: \[ X_i = \begin {cases} 1 & \text {if he takes the step to the right} \\ -1 & \text {if he takes the step to the left} \end {cases} \] Then \( X = X_1 + X_2 + \cdots + X_n \) gives the position after \( n \) steps. Define \( Y_i = (X_i + 1)/2 \). Then: \[ Y_i = \begin {cases} 1 & \text {with probability } p \\ 0 & \text {with probability } q = 1-p \end {cases} \] Since the steps are independent, \( Y_i \)’s are i.i.d. Bernoulli variates with parameter \( p \). Hence: \[ \sum _{i=1}^{n} Y_i \sim B(n, p) \] But: \[ \sum _{i=1}^{n} Y_i = \sum _{i=1}^{n} \frac {X_i + 1}{2} = \frac {1}{2} \left ( \sum _{i=1}^{n} X_i + n \right ) = \frac {X + n}{2} \] Thus \( \frac {X + n}{2} \sim B(n, p) \). \[ E\left ( \frac {X + n}{2} \right ) = np \Rightarrow \frac {1}{2} E(X + n) = np \] \[ E(X) + n = 2np \Rightarrow E(X) = n(2p-1) \]

Example 3.1.40 (Example 7.21). Suppose that the r.v. \( X \) is uniformly distributed on \( (0, 1) \), i.e., \( f_X(x) = 1; 0 \leq x \leq 1 \). Assume that the conditional distribution of \( Y | X = x \) has a binomial distribution with parameters \( n \) and \( p = x \), i.e., \[ P(Y = y | X = x) = \binom {n}{y} x^y (1-x)^{n-y}, \quad y = 0, 1, 2, \ldots , n \] Find (a) \( E(Y) \), (b) the distribution of \( Y \).

Solution. (a) We are given that \( Y | X = x \sim B(n, x) \). Hence: \[ E(Y | X = x) = nx \] By the law of total expectation: \[ E(Y) = E[E(Y | X)] = E(nX) = nE(X) \] Since \( X \sim U[0, 1] \): \[ E(X) = \int _0^1 x \cdot 1 \, dx = \frac {1}{2} \] Thus: \[ E(Y) = n \times \frac {1}{2} = \frac {n}{2} \] (b) The joint distribution of \( X \) and \( Y \) is: \[ f_{X,Y}(x,y) = f_X(x) \cdot f_{Y|X}(y|x) = 1 \times \binom {n}{y} x^y (1-x)^{n-y} \] The marginal distribution of \( Y \) is: \begin {align*} f_Y(y) &= \int _0^1 f_{X,Y}(x,y) \, dx \\ &= \binom {n}{y} \int _0^1 x^y (1-x)^{n-y} \, dx \\ &= \binom {n}{y} \frac {\Gamma (y+1)\Gamma (n-y+1)}{\Gamma (n+2)} \\ &= \frac {n!}{y!(n-y)!} \cdot \frac {y!(n-y)!}{(n+1)!} \\ &= \frac {1}{n+1}, \quad y = 0, 1, 2, \ldots , n \end {align*} Thus \( Y \) has a discrete uniform distribution on \( \{0, 1, 2, \ldots , n\} \).

Example 3.1.41 (Example 7.22). If \( K(t) \) is the cumulant generating function about the origin of the Binomial Distribution of size \( n \), show that: (a) \( \frac {d}{dt} K(t) = n(1 + e^{z+t})^{-1} \), where \( z = \log _e(p/q) \). (b) By expanding the R.H.S. in powers of \( t \) by Taylor’s Theorem, show that: \[ \kappa _r = n \frac {d^{r-1}}{dz^{r-1}} p \] where \( \kappa _r \) is the \( r \)th cumulant. (c) Hence or otherwise obtain the recurrence relation: \[ \kappa _{r+1} = pq \frac {d\kappa _r}{dp}, \quad r \geq 1 \] (d) Prove that \( \kappa _{r+1} = \frac {d\kappa _r}{dz} \), where \( z = \log _e(p/q) \).

Solution. For binomial distribution: \[ K(t) = \log M(t) = n \log (q + pe^t) \] (a) \[ \frac {d}{dt} K(t) = \frac {npe^t}{q + pe^t} \] If \( z = \log _e(p/q) \), then \( \frac {p}{q} = e^z \Rightarrow \frac {q}{p} = e^{-z} \). \[ \frac {d}{dt} K(t) = \frac {n}{1 + \frac {q}{p} e^{-t}} = n[1 + e^{-(z+t)}]^{-1} \] (b) By definition: \[ \kappa _r = \left [ \frac {d^r}{dt^r} K(t) \right ]_{t=0} = \left [ \frac {d^{r-1}}{dt^{r-1}} \frac {d}{dt} K(t) \right ]_{t=0} \] \[ = n \left [ \frac {d^{r-1}}{dt^{r-1}} \left ( 1 + e^{-(z+t)} \right )^{-1} \right ]_{t=0} \] Since \( \frac {d}{dt} \left ( 1 + e^{-(z+t)} \right )^{-1} = \frac {d}{dz} \left ( 1 + e^{-(z+t)} \right )^{-1} \), we have: \[ \kappa _r = n \left [ \frac {d^{r-1}}{dz^{r-1}} \left ( 1 + e^{-(z+t)} \right )^{-1} \right ]_{t=0} \] \[ = n \frac {d^{r-1}}{dz^{r-1}} \left ( 1 + e^{-z} \right )^{-1} = n \frac {d^{r-1}}{dz^{r-1}} p \] (since \( p = \frac {1}{1 + q/p} = (1 + e^{-z})^{-1} \)) (c) \[ \frac {d\kappa _r}{dp} = n \frac {d}{dp} \left ( \frac {d^{r-1}p}{dz^{r-1}} \right ) = n \frac {d}{dz} \left ( \frac {d^{r-1}p}{dz^{r-1}} \right ) \frac {dz}{dp} \] \[ = n \frac {d^r p}{dz^r} \cdot \frac {1}{pq} = \frac {1}{pq} \kappa _{r+1} \] Thus: \[ \kappa _{r+1} = pq \frac {d\kappa _r}{dp} \] (d) \[ \frac {d\kappa _r}{dz} = \frac {d\kappa _r}{dp} \cdot \frac {dp}{dz} \] \[ \frac {dp}{dz} = \frac {d}{dz} \left ( \frac {e^z}{1 + e^z} \right ) = \frac {e^z}{(1 + e^z)^2} = pq \] Thus: \[ \frac {d\kappa _r}{dz} = \frac {d\kappa _r}{dp} \cdot pq = \kappa _{r+1} \]

Example 3.1.42 (Example 7.23). If \( b(r; n, p) = \binom {n}{r} p^r q^{n-r} \) is the binomial probability and if: \[ B(k; n, p) = P(X \leq k) = \sum _{r=0}^{k} b(r; n, p) \] then prove that: \[ B(k; n, p) = (n-k) \binom {n}{k} \int _0^q (1-u)^k u^{n-k-1} du \]

Solution. \begin {align*} \frac {d}{dq} B(k; n, p) &= \frac {d}{dq} \sum _{r=0}^{k} \binom {n}{r} p^r q^{n-r} \\ &= \sum _{r=0}^{k} \binom {n}{r} \left [ r p^{r-1} (-1) q^{n-r} + p^r (n-r) q^{n-r-1} \right ] \\ &= -n \sum _{r=0}^{k} \left [ \binom {n-1}{r-1} p^{r-1} q^{n-r} - \binom {n-1}{r} p^r q^{n-r-1} \right ] \\ &= -n \left [ \binom {n-1}{k} p^k q^{n-k-1} \right ] \\ &= -n \binom {n-1}{k} p^k q^{n-k-1} \end {align*} On integration: \[ B(k; n, p) = n \binom {n-1}{k} \int _0^q (1-u)^k u^{n-k-1} du \] Since: \[ n \binom {n-1}{k} = (n-k) \binom {n}{k} \] We get: \[ B(k; n, p) = (n-k) \binom {n}{k} \int _0^q (1-u)^k u^{n-k-1} du \] Remark: This result is of great practical utility. It enables us to represent cumulative binomial probabilities in terms of incomplete Beta functions, which are tabulated in Karl Pearson’s Tables of the Incomplete Beta Functions.

3.1.15 Summary of Key Results for Binomial Distribution


Property	Result

Probability Mass Function	\( p(x) = \binom {n}{x} p^x q^{n-x}, \quad x = 0, 1, \ldots , n \)
Mean	\( \mu _1' = np \)
Variance	\( \mu _2 = npq \)
Third Central Moment	\( \mu _3 = npq(q-p) \)
Fourth Central Moment	\( \mu _4 = npq[1 + 3pq(n-2)] \)
Skewness	\( \beta _1 = \frac {(q-p)^2}{npq} \)
Kurtosis	\( \beta _2 = 3 + \frac {1-6pq}{npq} \)
Moment Generating Function	\( M_X(t) = (q + pe^t)^n \)
Characteristic Function	\( \phi _X(t) = (q + pe^{it})^n \)
Probability Generating Function	\( P(s) = (q + ps)^n \)
Cumulants	\( \kappa _r = npq \frac {d^{r-2}}{dp^{r-2}}(q-p) \) for \( r \geq 2 \)
Mode	Integral part of \( (n+1)p \), or bimodal if integer
Additive Property	\( B(n_1, p) * B(n_2, p) = B(n_1 + n_2, p) \)
Recurrence Relation (Probabilities)	\( p(x+1) = \frac {n-x}{x+1} \cdot \frac {p}{q} \cdot p(x) \)
Recurrence Relation (Moments)	\( \mu _{r+1} = pq\left [ nr\mu _{r-1} + \frac {d\mu _r}{dp} \right ] \)

3.2 Poisson Distribution

The Poisson distribution was discovered by the French mathematician and physicist Simeon Denis Poisson (1781-1840) who published it in 1837. It is a limiting case of the binomial distribution and is widely used to model the number of occurrences of rare events in a fixed interval of time or space.

3.2.1 Poisson Distribution as a Limiting Case of Binomial Distribution

The Poisson distribution is obtained as a limiting case of the binomial distribution under the following conditions:

1.: \( n \), the number of trials, is indefinitely large, i.e., \( n \to \infty \).
2.: \( p \), the constant probability of success for each trial, is indefinitely small, i.e., \( p \to 0 \).
3.: \( np = \lambda \) is finite. Thus \( p = \lambda /n \) and \( q = 1 - \lambda /n \), where \( \lambda > 0 \).

The probability of \( x \) successes in a series of \( n \) independent trials is: \[ b(x; n, p) = \binom {n}{x} p^x q^{n-x}, \quad x = 0, 1, 2, \ldots , n \] Taking the limit as \( n \to \infty \): \begin {align*} \lim _{n \to \infty } b(x; n, p) &= \lim _{n \to \infty } \binom {n}{x} \left ( \frac {\lambda }{n} \right )^x \left ( 1 - \frac {\lambda }{n} \right )^{n-x} \\ &= \frac {\lambda ^x}{x!} \lim _{n \to \infty } \frac {n(n-1)\cdots (n-x+1)}{n^x} \cdot \lim _{n \to \infty } \left ( 1 - \frac {\lambda }{n} \right )^n \cdot \lim _{n \to \infty } \left ( 1 - \frac {\lambda }{n} \right )^{-x} \\ &= \frac {\lambda ^x}{x!} \cdot 1 \cdot e^{-\lambda } \cdot 1 \\ &= \frac {e^{-\lambda } \lambda ^x}{x!}, \quad x = 0, 1, 2, \ldots , \infty \end {align*}

Definition 3.2.1 (Poisson Distribution). A random variable \( X \) is said to follow a Poisson distribution if it assumes only non-negative values and its probability mass function is given by: \[ P(X = x) = p(x; \lambda ) = \frac {e^{-\lambda } \lambda ^x}{x!}, \quad x = 0, 1, 2, \ldots , \infty ; \quad \lambda > 0 \] and \( p(x; \lambda ) = 0 \), otherwise.

Remark 3.2.2.

1.: We use the notation \( X \sim P(\lambda ) \) to denote that \( X \) follows a Poisson distribution with parameter \( \lambda \).
2.: The sum of probabilities is 1 because: \[ \sum _{x=0}^{\infty } p(x; \lambda ) = e^{-\lambda } \sum _{x=0}^{\infty } \frac {\lambda ^x}{x!} = e^{-\lambda } \cdot e^{\lambda } = 1 \]
3.: The parameter \( \lambda \) is the mean number of occurrences in the given interval.

3.2.2 The Poisson Process

The Poisson distribution may also be obtained independently as follows: Let \( X_t \) be the number of telephone calls received in time interval \( t \). Consider the following experimental conditions:

1.: The probability of getting a call in a small time interval \( (t, t + dt) \) is \( \lambda dt \), where \( \lambda > 0 \).
2.: The probability of getting more than one call in this time interval is of the order \( (dt)^2 \), i.e., \( 0[(dt)^2] \).
3.: The probability of any particular call in \( (t, t + dt) \) is independent of the actual time \( t \) and of all previous calls.

Under these conditions, the probability of getting exactly \( x \) calls in time \( t \) is: \[ P_x(t) = \frac {e^{-\lambda t} (\lambda t)^x}{x!}, \quad x = 0, 1, 2, \ldots , \infty \]

3.2.3 Examples of Poisson Distribution

The Poisson distribution is successfully employed in the following situations:

1.: Number of deaths from a disease (not in the form of an epidemic) such as heart attack or cancer.
2.: Number of suicides reported in a particular city.
3.: Number of defective items in a packing manufactured by a good concern.
4.: Number of faulty blades in a packet of 100.
5.: Number of air accidents in some unit of time.
6.: Number of printing mistakes at each page of a book.
7.: Number of telephone calls received at a particular telephone exchange in some unit of time.
8.: Number of cars passing a crossing per minute during busy hours.
9.: Number of fragments received by a surface area from a fragment atom bomb.
10.: Emission of radioactive (alpha) particles.

3.2.4 Moments of Poisson Distribution

Theorem 3.2.3 (First Four Moments about Origin). For the Poisson distribution \( P(\lambda ) \): \begin {align*} \mu _1' &= \lambda \\ \mu _2' &= \lambda ^2 + \lambda \\ \mu _3' &= \lambda ^3 + 3\lambda ^2 + \lambda \\ \mu _4' &= \lambda ^4 + 6\lambda ^3 + 7\lambda ^2 + \lambda \end {align*}

Proof : \begin {align*} \mu _1' = E(X) &= \sum _{x=0}^{\infty } x \cdot \frac {e^{-\lambda } \lambda ^x}{x!} \\ &= \lambda e^{-\lambda } \sum _{x=1}^{\infty } \frac {\lambda ^{x-1}}{(x-1)!} \\ &= \lambda e^{-\lambda } \cdot e^{\lambda } = \lambda \end {align*} \begin {align*} \mu _2' = E(X^2) &= \sum _{x=0}^{\infty } [x(x-1) + x] \frac {e^{-\lambda } \lambda ^x}{x!} \\ &= e^{-\lambda } \sum _{x=2}^{\infty } \frac {\lambda ^x}{(x-2)!} + \lambda \\ &= \lambda ^2 e^{-\lambda } \sum _{x=2}^{\infty } \frac {\lambda ^{x-2}}{(x-2)!} + \lambda \\ &= \lambda ^2 e^{-\lambda } \cdot e^{\lambda } + \lambda = \lambda ^2 + \lambda \end {align*} Similarly: \begin {align*} \mu _3' & = \lambda ^3 + 3\lambda ^2 + \lambda \\ \mu _4' & = \lambda ^4 + 6\lambda ^3 + 7\lambda ^2 + \lambda \end {align*} □

Theorem 3.2.4 (Central Moments of Poisson Distribution). \begin {align*} \mu _2 &= \lambda \\ \mu _3 &= \lambda \\ \mu _4 &= \lambda + 3\lambda ^2 \end {align*}

Proof : \begin {align*} \mu _2 &= \mu _2' - (\mu _1')^2 = (\lambda ^2 + \lambda ) - \lambda ^2 = \lambda \end {align*} \begin {align*} \mu _3 &= \mu _3' - 3\mu _2'\mu _1' + 2(\mu _1')^3 \\ &= (\lambda ^3 + 3\lambda ^2 + \lambda ) - 3(\lambda ^2 + \lambda )\lambda + 2\lambda ^3 \\ &= \lambda \end {align*} \begin {align*} \mu _4 &= \mu _4' - 4\mu _3'\mu _1' + 6\mu _2'(\mu _1')^2 - 3(\mu _1')^4 \\ &= \lambda + 3\lambda ^2 \end {align*} □

Corollary 3.2.5 (Skewness and Kurtosis). \begin {align*} \beta _1 &= \frac {\mu _3^2}{\mu _2^3} = \frac {\lambda ^2}{\lambda ^3} = \frac {1}{\lambda } & \text { and } & \qquad \gamma _1 = \sqrt {\beta _1} = \frac {1}{\sqrt {\lambda }} \\ \beta _2 &= \frac {\mu _4}{\mu _2^2} = \frac {\lambda + 3\lambda ^2}{\lambda ^2} = 3 + \frac {1}{\lambda } & \text { and } & \qquad \gamma _2 = \beta _2 -3 =\frac {1}{\lambda } \end {align*}

Remark 3.2.6.

1.: The mean and variance of the Poisson distribution are both equal to \( \lambda \).
2.: The Poisson distribution is always positively skewed since \( \beta _1 = 1/\lambda > 0 \).
3.: As \( \lambda \to \infty \), \( \beta _1 \to 0 \) and \( \beta _2 \to 3 \), so the distribution tends to normality.

Example 3.2.7 (Example 7.24). A car hire firm has two cars which it hires out day by day. The number of demands for a car on each day is distributed as a Poisson variate with mean 1.5. Calculate the proportion of days on which (i) neither car is used, and (ii) some demand is refused.

Solution. Let \( X \) be the number of demands for a car on any day. Then \( X \sim P(\lambda = 1.5) \) with: \[ P(X = x) = \frac {e^{-1.5}(1.5)^x}{x!}, \quad x = 0, 1, 2, \ldots \] (i) Proportion of days on which neither car is used: \[ P(X = 0) = e^{-1.5} = 0.2231 \] (ii) Proportion of days on which some demand is refused means \( X > 2 \): \[ P(X > 2) = 1 - P(X \leq 2) = 1 - [P(X = 0) + P(X = 1) + P(X = 2)] \] \[ = 1 - e^{-1.5} \left [ 1 + 1.5 + \frac {(1.5)^2}{2!} \right ] = 1 - 0.2231 \times 3.625 = 0.19126 \]

Example 3.2.8 (Example 7.25). A manufacturer of cotter pins knows that \( 5\% \) of his product is defective. If he sells cotter pins in boxes of 100 and guarantees that not more than 10 pins will be defective, what is the approximate probability that a box will fail to meet the guaranteed quality?

Solution. Here \( n = 100 \), \( p = 0.05 \). Since \( p \) is small, we use Poisson approximation with: \[ \lambda = np = 100 \times 0.05 = 5 \] Probability of \( x \) defective pins in a box of 100: \[ P(X = x) = \frac {e^{-5}5^x}{x!}, \quad x = 0, 1, 2, \ldots \] Probability that a box will fail to meet the guaranteed quality is: \[ P(X > 10) = 1 - P(X \leq 10) = 1 - \sum _{x=0}^{10} \frac {e^{-5}5^x}{x!} \]

Example 3.2.9 (Example 7.26). Six coins are tossed 6,400 times. Using the Poisson distribution, find the approximate probability of getting six heads \( r \) times.

Solution. The probability of obtaining six heads in one throw of six coins is: \[ p = \left ( \frac {1}{2} \right )^6 = \frac {1}{64} \] \[ \lambda = np = 6400 \times \frac {1}{64} = 100 \] Hence, using the Poisson probability law: \[ P(X = r) = \frac {e^{-100}(100)^r}{r!}, \quad r = 0, 1, 2, \ldots \]

Example 3.2.10 (Example 7.27). In a book of 520 pages, 390 typographical errors occur. Assuming Poisson law for the number of errors per page, find the probability that a random sample of 5 pages will contain no error.

Solution. The average number of typographical errors per page: \[ \lambda = \frac {390}{520} = 0.75 \] Probability of \( x \) errors per page: \[ P(X = x) = \frac {e^{-0.75}(0.75)^x}{x!}, \quad x = 0, 1, 2, \ldots \] The probability that a random sample of 5 pages will contain no error: \[ [P(X = 0)]^5 = (e^{-0.75})^5 = e^{-3.75} \]

Example 3.2.11 (Example 7.28). Suppose that the number of telephone calls coming into a telephone exchange between 10 A.M. and 11 A.M., say \( X_1 \), is a random variable with Poisson distribution with parameter 2. Similarly, the number of calls arriving between 11 A.M. and 12 noon, say \( X_2 \), has a Poisson distribution with parameter 6. If \( X_1 \) and \( X_2 \) are independent, what is the probability that more than 5 calls come in between 10 A.M. and 12 noon?

Solution. Let \( X = X_1 + X_2 \). By the additive property of Poisson distribution, \( X \sim P(\lambda = 2 + 6 = 8) \). Probability of \( x \) calls between 10 A.M. and 12 noon: \[ P(X = x) = \frac {e^{-8}8^x}{x!}, \quad x = 0, 1, 2, \ldots \] Probability that more than 5 calls come in: \[ P(X > 5) = 1 - P(X \leq 5) = 1 - \sum _{x=0}^{5} \frac {e^{-8}8^x}{x!} = 1 - 0.1912 = 0.8088 \]

Example 3.2.12 (Example 7.29). A Poisson distribution has a double mode at \( x = 1 \) and \( x = 2 \). What is the probability that \( X \) will have one or the other of these two values?

Solution. If the Poisson distribution is bimodal, the two modes are at \( x = \lambda - 1 \) and \( x = \lambda \). Given the modes are at \( x = 1 \) and \( x = 2 \), we have \( \lambda = 2 \). \[ P(X = x) = \frac {e^{-2}2^x}{x!}, \quad x = 0, 1, 2, \ldots \] \[ P(X = 1) = e^{-2} \cdot 2 = 2e^{-2} \] \[ P(X = 2) = \frac {e^{-2} \cdot 2^2}{2!} = 2e^{-2} \] Required probability: \[ P(X = 1) + P(X = 2) = 2e^{-2} + 2e^{-2} = 4e^{-2} = 0.542 \]

Example 3.2.13 (Example 7.30). If \( X \) is a Poisson variate such that: \[ P(X = 2) = 9P(X = 4) + 90P(X = 6) \] Find (i) \( \lambda \), the mean of \( X \), (ii) \( \beta _1 \), the coefficient of skewness.

Solution. For a Poisson variate: \[ P(X = x) = \frac {e^{-\lambda } \lambda ^x}{x!}, \quad x = 0, 1, 2, \ldots \] The given equation: \[ \frac {e^{-\lambda } \lambda ^2}{2!} = e^{-\lambda } \left [ 9 \frac {\lambda ^4}{4!} + 90 \frac {\lambda ^6}{6!} \right ] \] \[ \frac {\lambda ^2}{2} = 9 \cdot \frac {\lambda ^4}{24} + 90 \cdot \frac {\lambda ^6}{720} \] \[ \frac {\lambda ^2}{2} = \frac {3\lambda ^4}{8} + \frac {\lambda ^6}{8} \] Multiplying by 8: \[ 4\lambda ^2 = 3\lambda ^4 + \lambda ^6 \] \[ \lambda ^6 + 3\lambda ^4 - 4\lambda ^2 = 0 \] \[ \lambda ^2(\lambda ^4 + 3\lambda ^2 - 4) = 0 \] Since \( \lambda > 0 \): \[ \lambda ^4 + 3\lambda ^2 - 4 = 0 \] Let \( u = \lambda ^2 \): \[ u^2 + 3u - 4 = 0 \] \[ (u + 4)(u - 1) = 0 \] \[ u = 1 \quad \text {or} \quad u = -4 \] Since \( \lambda > 0 \), \( \lambda ^2 = 1 \Rightarrow \lambda = 1 \). (i) Mean \( = \lambda = 1 \). (ii) \( \beta _1 = \frac {1}{\lambda } = 1 \).

Example 3.2.14 (Example 7.31). If \( X \) and \( Y \) are independent Poisson variates such that: \[ P(X = 1) = P(X = 2) \quad \text {and} \quad P(Y = 2) = P(Y = 3) \] Find the variance of \( X - 2Y \).

Solution. Let \( X \sim P(\lambda ) \) and \( Y \sim P(\mu ) \). For \( X \): \[ P(X = 1) = P(X = 2) \Rightarrow \frac {e^{-\lambda } \lambda ^1}{1!} = \frac {e^{-\lambda } \lambda ^2}{2!} \Rightarrow \lambda = \frac {\lambda ^2}{2} \Rightarrow \lambda = 2 \] For \( Y \): \[ P(Y = 2) = P(Y = 3) \Rightarrow \frac {e^{-\mu } \mu ^2}{2!} = \frac {e^{-\mu } \mu ^3}{3!} \Rightarrow \frac {1}{2} = \frac {\mu }{6} \Rightarrow \mu = 3 \] Now: \[ \operatorname {Var}(X - 2Y) = \operatorname {Var}(X) + 4\operatorname {Var}(Y) \quad (\text {since independent}) \] \[ = \lambda + 4\mu = 2 + 4(3) = 14 \]

Example 3.2.15 (Example 7.32). If \( X \) and \( Y \) are independent Poisson variates with means \( \lambda _1 \) and \( \lambda _2 \) respectively, find the probability that: (i) \( X + Y \neq k \), (ii) \( X = Y \).

Solution. (i) Since \( X \sim P(\lambda _1) \) and \( Y \sim P(\lambda _2) \) are independent, by the additive property: \[ X + Y \sim P(\lambda _1 + \lambda _2) \] \[ P(X + Y = k) = \frac {e^{-(\lambda _1 + \lambda _2)}(\lambda _1 + \lambda _2)^k}{k!}, \quad k = 0, 1, 2, \ldots \] Thus: \[ P(X + Y \neq k) = 1 - \frac {e^{-(\lambda _1 + \lambda _2)}(\lambda _1 + \lambda _2)^k}{k!} \] (ii) \[ P(X = Y) = \sum _{r=0}^{\infty } P(X = r \cap Y = r) = \sum _{r=0}^{\infty } P(X = r)P(Y = r) \] \[ = \sum _{r=0}^{\infty } \frac {e^{-\lambda _1} \lambda _1^r}{r!} \cdot \frac {e^{-\lambda _2} \lambda _2^r}{r!} = e^{-(\lambda _1 + \lambda _2)} \sum _{r=0}^{\infty } \frac {(\lambda _1 \lambda _2)^r}{(r!)^2} \]

Example 3.2.16 (Example 7.33). Show that in a Poisson distribution with unit mean, the mean deviation about the mean is \( (2/e) \) times the standard deviation.

Solution. Here \( \lambda = 1 \). So: \[ P(X = x) = \frac {e^{-1}}{x!}, \quad x = 0, 1, 2, \ldots \] Mean deviation about the mean \( 1 \): \[ E(|X - 1|) = \sum _{x=0}^{\infty } |x - 1| \frac {e^{-1}}{x!} = e^{-1} \sum _{x=0}^{\infty } \frac {|x - 1|}{x!} \] \[ = e^{-1} \left [ 1 + \frac {1}{2!} + \frac {2}{3!} + \frac {3}{4!} + \cdots \right ] \] Since \( \frac {n}{(n+1)!} = \frac {1}{n!} - \frac {1}{(n+1)!} \), we get: \[ E(|X - 1|) = e^{-1} \left [ 1 + \left ( 1 - \frac {1}{2!} \right ) + \left ( \frac {1}{2!} - \frac {1}{3!} \right ) + \left ( \frac {1}{3!} - \frac {1}{4!} \right ) + \cdots \right ] \] \[ = e^{-1}(1 + 1) = \frac {2}{e} \] Since \( \sigma = 1 \) for Poisson distribution with \( \lambda = 1 \): \[ \text {Mean deviation} = \frac {2}{e} \times \sigma \]

Example 3.2.17 (Example 7.34). Let \( X_1, X_2, \ldots , X_n \) be identically and independently distributed \( \text {Bin}(1, p) \) variates. Let \( S_n \equiv \sum _{j=1}^{n} X_j \) and \( M_n(t) \) be the m.g.f. of \( S_n \). Find \( \lim _{n \to \infty } M_n(t) \) using \( np = \lambda \) (constant).

Solution. Since \( X_i \)’s are i.i.d. \( B(1, p) \), \( S_n = \sum _{j=1}^{n} X_j \sim B(n, p) \). \[ M_n(t) = (q + pe^t)^n = \left [ 1 + (e^t - 1)p \right ]^n \] With \( np = \lambda \Rightarrow p = \lambda /n \): \[ M_n(t) = \left [ 1 + \frac {(e^t - 1)\lambda }{n} \right ]^n \] Taking the limit as \( n \to \infty \): \[ \lim _{n \to \infty } M_n(t) = \exp [\lambda (e^t - 1)] \] This is the m.g.f. of a Poisson distribution with parameter \( \lambda \). Hence, by the uniqueness theorem of m.g.f., \( S_n \to P(\lambda ) \) as \( n \to \infty \) with \( np = \lambda \) fixed.

Example 3.2.18 (Example 7.35). (a) If \( X \) is a Poisson variate with mean \( m \), show that: \[ E(e^{-kX}) = \exp [-m(1 - e^{-k})] \] Hence show that if \( \bar {X} \) is the arithmetic mean of \( n \) independent random variables \( X_1, X_2, \ldots , X_n \), each having Poisson distribution with parameter \( m \), then \( e^{-\bar {X}} \) as an estimate of \( e^{-m} \) is biased, although \( \bar {X} \) is an unbiased estimate of \( m \). (b) If \( X \) is a Poisson variate with mean \( m \), what would be the expectation of \( e^{-kX} kX \), \( k \) being a constant?

Solution. (a) \begin {align*} E(e^{-kX}) &= \sum _{x=0}^{\infty } e^{-kx} \frac {e^{-m} m^x}{x!} = e^{-m} \sum _{x=0}^{\infty } \frac {(me^{-k})^x}{x!} \\ &= e^{-m} \cdot e^{me^{-k}} = e^{-m(1 - e^{-k})} \end {align*} We have: \[ E(\bar {X}) = E\left ( \frac {1}{n} \sum _{i=1}^{n} X_i \right ) = \frac {1}{n} \sum _{i=1}^{n} E(X_i) = \frac {1}{n} \cdot n m = m \] Hence \( \bar {X} \) is an unbiased estimate of \( m \). Now: \[ E(e^{-\bar {X}}) = E\left ( e^{-\frac {1}{n} \sum X_i} \right ) = \prod _{i=1}^{n} E\left ( e^{-\frac {X_i}{n}} \right ) \] \[ = \left [ e^{-m(1 - e^{-1/n})} \right ]^n = e^{-mn(1 - e^{-1/n})} \] Since \( e^{-mn(1 - e^{-1/n})} \neq e^{-m} \), \( e^{-\bar {X}} \) is a biased estimate of \( e^{-m} \). (b) \begin {align*} E(e^{-kX} kX) &= k \sum _{x=0}^{\infty } e^{-kx} x \frac {e^{-m} m^x}{x!} = km e^{-m} e^{-k} \sum _{x=1}^{\infty } \frac {(me^{-k})^{x-1}}{(x-1)!} \\ &= km e^{-m-k} \cdot e^{me^{-k}} = km \exp [m(e^{-k} - 1) - k] \end {align*}

Example 3.2.19 (Example 7.36). If \( X \) and \( Y \) are independent Poisson variates with means \( m_1 \) and \( m_2 \) respectively, prove that the probability that \( X - Y \) has the value \( r \) is the coefficient of \( t^r \) in: \[ \exp \left \{ m_1 t + \frac {m_2}{t} - m_1 - m_2 \right \} \]

Solution. Since \( X \) and \( Y \) are independent Poisson variates: \[ P(X = x) = \frac {e^{-m_1} m_1^x}{x!}, \quad P(Y = y) = \frac {e^{-m_2} m_2^y}{y!} \] \[ P(X - Y = r) = \sum _{s=0}^{\infty } P(X = r + s \cap Y = s) \] \[ = \sum _{s=0}^{\infty } \frac {e^{-m_1} m_1^{r+s}}{(r+s)!} \cdot \frac {e^{-m_2} m_2^s}{s!} \] \[ = e^{-m_1 - m_2} \sum _{s=0}^{\infty } \frac {m_1^{r+s} m_2^s}{(r+s)! s!} \] This is the coefficient of \( t^r \) in: \[ e^{-m_1 - m_2} \sum _{r=-\infty }^{\infty } \sum _{s=0}^{\infty } \frac {m_1^{r+s} m_2^s}{(r+s)! s!} t^r = e^{-m_1 - m_2} \sum _{s=0}^{\infty } \frac {m_2^s}{s!} \sum _{r=-\infty }^{\infty } \frac {m_1^{r+s}}{(r+s)!} t^r \] \[ = e^{-m_1 - m_2} \sum _{s=0}^{\infty } \frac {m_2^s}{s!} \cdot m_1^s \sum _{r=-\infty }^{\infty } \frac {(m_1 t)^r}{(r+s)!} \] This simplifies to: \[ \text {Coefficient of } t^r \text { in } \exp \left \{ m_1 t + \frac {m_2}{t} - m_1 - m_2 \right \} \]

Example 3.2.20 (Example 7.37). If \( X \) is a Poisson variate with mean \( m \), show that \( Y = (X - m)/\sqrt {m} \) is a variate with mean zero and variance unity. Find the M.G.F. for this variable and show that it approaches \( e^{t^2/2} \) as \( m \to \infty \). Also interpret the result.

Solution. \[ E(Y) = E\left ( \frac {X - m}{\sqrt {m}} \right ) = \frac {1}{\sqrt {m}} E(X - m) = 0 \] \[ V(Y) = V\left ( \frac {X - m}{\sqrt {m}} \right ) = \frac {1}{m} V(X) = \frac {m}{m} = 1 \] The m.g.f. of \( Y \): \[ M_Y(t) = E(e^{tY}) = E\left ( e^{t(X-m)/\sqrt {m}} \right ) = e^{-t\sqrt {m}} M_X(t/\sqrt {m}) \] \[ = e^{-t\sqrt {m}} \cdot e^{m(e^{t/\sqrt {m}} - 1)} = \exp \left [ -t\sqrt {m} + m(e^{t/\sqrt {m}} - 1) \right ] \] \[ = \exp \left [ -t\sqrt {m} + m\left ( \frac {t}{\sqrt {m}} + \frac {t^2}{2m} + \frac {t^3}{6m^{3/2}} + \cdots \right ) \right ] \] \[ = \exp \left [ \frac {t^2}{2} + \frac {t^3}{6\sqrt {m}} + \cdots \right ] \] Proceeding to the limit as \( m \to \infty \): \[ \lim _{m \to \infty } M_Y(t) = e^{t^2/2} \] Interpretation: \( e^{t^2/2} \) is the m.g.f. of the standard normal variate. Hence, by the uniqueness theorem of m.g.f., the standard Poisson variate tends to the standard normal variate as \( m \to \infty \). Thus, the Poisson distribution tends to the normal distribution for large values of the parameter \( m \).

Example 3.2.21 (Example 7.38). Deduce the first four moments about the mean of the Poisson distribution from those of the Binomial distribution.

Solution. The first four central moments of the binomial distribution are: \[ \mu _1 = 0, \quad \mu _2 = npq, \quad \mu _3 = npq(q-p), \quad \mu _4 = npq(1-6pq) + 3n^2p^2q^2 \] Poisson distribution is a limiting form of binomial distribution under: \[ n \to \infty , \quad p \to 0 \quad (q \to 1), \quad np = \lambda \] Using these conditions: \[ \text {Mean} = \lim (np) = \lambda \] \[ \mu _2 = \lim (npq) = \lambda \cdot 1 = \lambda \] \[ \mu _3 = \lim [npq(q-p)] = \lambda \cdot 1(1-0) = \lambda \] \[ \mu _4 = \lim [npq(1-6pq) + 3n^2p^2q^2] = \lambda \cdot 1(1-6 \cdot 0 \cdot 1) + 3\lambda ^2 \cdot 1 = \lambda + 3\lambda ^2 \]

Example 3.2.22 (Example 7.39). If \( X \) is a Poisson variate with parameter \( m \) and \( Y \) is another discrete variable whose conditional distribution for a given \( X \) is: \[ P(Y = r | X = x) = \binom {x}{r} p^r (1-p)^{x-r}, \quad 0 < p < 1, \quad r = 0, 1, 2, \ldots , x \] then show that the unconditional distribution of \( Y \) is a Poisson distribution with parameter \( mp \).

Solution. We are given: \[ P(X = x) = \frac {e^{-m} m^x}{x!}, \quad x = 0, 1, 2, \ldots \] \[ P(Y = r | X = x) = \binom {x}{r} p^r (1-p)^{x-r}, \quad r \leq x \] \[ P(Y = r) = \sum _{x=r}^{\infty } P(X = x \cap Y = r) = \sum _{x=r}^{\infty } P(X = x) P(Y = r | X = x) \] \[ = \sum _{x=r}^{\infty } \frac {e^{-m} m^x}{x!} \binom {x}{r} p^r (1-p)^{x-r} \] \[ = \frac {e^{-m} (mp)^r}{r!} \sum _{x=r}^{\infty } \frac {[m(1-p)]^{x-r}}{(x-r)!} \] \[ = \frac {e^{-m} (mp)^r}{r!} \cdot e^{m(1-p)} = \frac {e^{-mp} (mp)^r}{r!}, \quad r = 0, 1, 2, \ldots \] Hence \( Y \sim P(mp) \).

Example 3.2.23 (Example 7.40). If \( X \) and \( Y \) are independent Poisson variates, show that the conditional distribution of \( X \) given \( X + Y \) is binomial.

Solution. Let \( X \sim P(\lambda ) \) and \( Y \sim P(\mu ) \). Then \( X + Y \sim P(\lambda + \mu ) \). \[ P(X = r | X + Y = n) = \frac {P(X = r \cap Y = n - r)}{P(X + Y = n)} \] \[ = \frac {\frac {e^{-\lambda } \lambda ^r}{r!} \cdot \frac {e^{-\mu } \mu ^{n-r}}{(n-r)!}}{\frac {e^{-(\lambda + \mu )}(\lambda + \mu )^n}{n!}} \] \[ = \frac {n!}{r!(n-r)!} \cdot \frac {\lambda ^r \mu ^{n-r}}{(\lambda + \mu )^n} \] \[ = \binom {n}{r} \left ( \frac {\lambda }{\lambda + \mu } \right )^r \left ( \frac {\mu }{\lambda + \mu } \right )^{n-r} \] This is a binomial distribution with parameters \( n \) and \( p = \frac {\lambda }{\lambda + \mu } \).

Example 3.2.24 (Example 7.41). If \( X \) is a Poisson variate with parameter \( m \) and \( \mu _r \) is the \( r \)th central moment, prove that: \[ m \left [ \binom {r}{1} \mu _{r-1} + \binom {r}{2} \mu _{r-2} + \cdots + \binom {r}{r} \mu _0 \right ] = \mu _{r+1} \]

Solution. By definition: \[ \mu _{r+1} = E[X - m]^{r+1} = \sum _{x=0}^{\infty } (x - m)^{r+1} \frac {e^{-m} m^x}{x!} \] \[ = \sum _{x=1}^{\infty } (x - m)^r \frac {e^{-m} m^x}{(x-1)!} - m \sum _{x=0}^{\infty } (x - m)^r \frac {e^{-m} m^x}{x!} \] Let \( y = x - 1 \): \[ \mu _{r+1} = m \sum _{y=0}^{\infty } (y - m + 1)^r \frac {e^{-m} m^y}{y!} - m\mu _r \] \[ = m \sum _{y=0}^{\infty } [(y - m) + 1]^r p(y) - m\mu _r \] \[ = m \sum _{y=0}^{\infty } \left [ (y - m)^r + \binom {r}{1} (y - m)^{r-1} + \binom {r}{2} (y - m)^{r-2} + \cdots + 1 \right ] p(y) - m\mu _r \] \[ = m \left [ \mu _r + \binom {r}{1} \mu _{r-1} + \binom {r}{2} \mu _{r-2} + \cdots + \binom {r}{r} \mu _0 \right ] - m\mu _r \] \[ = m \left [ \binom {r}{1} \mu _{r-1} + \binom {r}{2} \mu _{r-2} + \cdots + \binom {r}{r} \mu _0 \right ] \]

Example 3.2.25 (Example 7.42). If \( X \) has a Poisson distribution with parameter \( \lambda \), show that the distribution function of \( X \) is given by: \[ F(x) = \frac {1}{\Gamma (x+1)} \int _{\lambda }^{\infty } e^{-t} t^x dt, \quad x = 0, 1, 2, \ldots \]

Solution. Consider the incomplete gamma integral: \[ I_x = \frac {1}{x!} \int _{\lambda }^{\infty } e^{-t} t^x dt \] Integrating by parts: \[ I_x = \left [ -\frac {e^{-t} t^x}{x!} \right ]_{\lambda }^{\infty } + \frac {1}{(x-1)!} \int _{\lambda }^{\infty } e^{-t} t^{x-1} dt = \frac {e^{-\lambda } \lambda ^x}{x!} + I_{x-1} \] Repeated application: \[ I_x = \frac {e^{-\lambda } \lambda ^x}{x!} + \frac {e^{-\lambda } \lambda ^{x-1}}{(x-1)!} + \cdots + \frac {e^{-\lambda } \lambda }{1!} + I_0 \] But: \[ I_0 = \int _{\lambda }^{\infty } e^{-t} dt = e^{-\lambda } \] Thus: \[ I_x = e^{-\lambda } + \lambda e^{-\lambda } + \frac {\lambda ^2 e^{-\lambda }}{2!} + \cdots + \frac {\lambda ^x e^{-\lambda }}{x!} \] \[ = P(X = 0) + P(X = 1) + \cdots + P(X = x) = P(X \leq x) = F(x) \] Hence: \[ F(x) = \frac {1}{x!} \int _{\lambda }^{\infty } e^{-t} t^x dt = \frac {1}{\Gamma (x+1)} \int _{\lambda }^{\infty } e^{-t} t^x dt \]

3.2.5 Recurrence Relation for Probabilities (Fitting of Poisson Distribution)

Theorem 3.2.26 (Recurrence Relation for Probabilities). For the Poisson distribution \( P(\lambda ) \): \[ p(x+1) = \frac {\lambda }{x+1} \cdot p(x), \quad x = 0, 1, 2, \ldots \]

Proof : \[ p(x) = \frac {e^{-\lambda } \lambda ^x}{x!}, \quad p(x+1) = \frac {e^{-\lambda } \lambda ^{x+1}}{(x+1)!} \] \[ \frac {p(x+1)}{p(x)} = \frac {\lambda }{x+1} \Rightarrow p(x+1) = \frac {\lambda }{x+1} \cdot p(x) \] □

This formula provides a convenient method of fitting a Poisson distribution to observed data. The only probability we need to calculate is: \[ p(0) = e^{-\lambda } \] where \( \lambda \) is estimated from the data (usually by equating the sample mean to \( \lambda \)). Then: \[ p(1) = \lambda p(0), \quad p(2) = \frac {\lambda }{2} p(1), \quad p(3) = \frac {\lambda }{3} p(2), \ldots \]

Example 3.2.27 (Example 7.43). After correcting 50 pages of the proof of a book, the proof reader finds that there are, on the average, 2 errors per 5 pages. How many pages would one expect to find with 0, 1, 2, 3 and 4 errors, in 1000 pages of the first print of the book? (Given that \( e^{-0.4} = 0.6703 \))

Solution. The mean number of errors per page is: \[ \lambda = \frac {2}{5} = 0.4 \] Probability of \( x \) errors per page: \[ P(X = x) = \frac {e^{-0.4}(0.4)^x}{x!}, \quad x = 0, 1, 2, \ldots \] Expected number of pages with \( x \) errors in 1000 pages: \[ 1000 \times P(X = x) = 1000 \times \frac {e^{-0.4}(0.4)^x}{x!} \] Using the recurrence formula: \begin {align*} p(0) &= e^{-0.4} = 0.6703, \quad f(0) = 1000 \times 0.6703 = 670 \\ p(1) &= \frac {0.4}{1} \times 0.6703 = 0.2681, \quad f(1) = 1000 \times 0.2681 = 268 \\ p(2) &= \frac {0.4}{2} \times 0.2681 = 0.05362, \quad f(2) = 1000 \times 0.05362 = 54 \\ p(3) &= \frac {0.4}{3} \times 0.05362 = 0.00715, \quad f(3) = 1000 \times 0.00715 = 7 \\ p(4) &= \frac {0.4}{4} \times 0.00715 = 0.000715, \quad f(4) = 1000 \times 0.000715 = 1 \end {align*}

Example 3.2.28 (Example 7.44). Fit a Poisson distribution to the following data which gives the number of doddens in a sample of clover seeds. \[ \begin {array}{c|ccccccccc} \text {No. of doddens } (x) & 0 & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 \\ \hline \text {Observed frequency} & 56 & 156 & 132 & 92 & 37 & 22 & 4 & 0 & 1 \end {array} \]

Solution. First, compute the mean: \[ \sum f x = 56(0) + 156(1) + 132(2) + 92(3) + 37(4) + 22(5) + 4(6) + 0(7) + 1(8) \] \[ = 0 + 156 + 264 + 276 + 148 + 110 + 24 + 0 + 8 = 986 \] \[ N = \sum f = 56 + 156 + 132 + 92 + 37 + 22 + 4 + 0 + 1 = 500 \] \[ \lambda = \frac {986}{500} = 1.972 \] Now: \[ p(0) = e^{-1.972} = 0.1392 \] Using the recurrence formula: \[ p(1) = \frac {1.972}{1} \times 0.1392 = 0.2745, \quad f(1) = 500 \times 0.2745 = 137 \] \[ p(2) = \frac {1.972}{2} \times 0.2745 = 0.2706, \quad f(2) = 500 \times 0.2706 = 135 \] \[ p(3) = \frac {1.972}{3} \times 0.2706 = 0.1779, \quad f(3) = 500 \times 0.1779 = 89 \] \[ p(4) = \frac {1.972}{4} \times 0.1779 = 0.0877, \quad f(4) = 500 \times 0.0877 = 44 \] \[ p(5) = \frac {1.972}{5} \times 0.0877 = 0.0346, \quad f(5) = 500 \times 0.0346 = 17 \] \[ p(6) = \frac {1.972}{6} \times 0.0346 = 0.0114, \quad f(6) = 500 \times 0.0114 = 6 \] \[ p(7) = \frac {1.972}{7} \times 0.0114 = 0.0032, \quad f(7) = 500 \times 0.0032 = 2 \] \[ p(8) = \frac {1.972}{8} \times 0.0032 = 0.0008, \quad f(8) = 500 \times 0.0008 = 0 \]


\( x \)	Observed Frequency	Expected Frequency

0	56	70
1	156	137
2	132	135
3	92	89
4	37	44
5	22	17
6	4	6
7	0	2
8	1	0

Total	500	500

3.2.6 Moment Generating Function of Poisson Distribution

Theorem 3.2.29 (Moment Generating Function). The moment generating function of the Poisson distribution \( P(\lambda ) \) is: \[ M_X(t) = e^{\lambda (e^t - 1)} \]

Proof : \begin {align*} M_X(t) = E(e^{tX}) &= \sum _{x=0}^{\infty } e^{tx} \frac {e^{-\lambda } \lambda ^x}{x!} \\ &= e^{-\lambda } \sum _{x=0}^{\infty } \frac {(\lambda e^t)^x}{x!} \\ &= e^{-\lambda } \cdot e^{\lambda e^t} = e^{\lambda (e^t - 1)} \end {align*} □

Theorem 3.2.30 (Characteristic Function). The characteristic function of the Poisson distribution \( P(\lambda ) \) is: \[ \phi _X(t) = e^{\lambda (e^{it} - 1)} \]

Proof : \[ \phi _X(t) = E(e^{itX}) = e^{-\lambda } \sum _{x=0}^{\infty } \frac {(\lambda e^{it})^x}{x!} = e^{\lambda (e^{it} - 1)} \] □

3.2.7 Cumulants of Poisson Distribution

Theorem 3.2.31 (Cumulants). For the Poisson distribution \( P(\lambda ) \), all cumulants are equal to \( \lambda \): \[ \kappa _r = \lambda , \quad r = 1, 2, 3, \ldots \]

Proof : The cumulant generating function is: \[ K_X(t) = \log M_X(t) = \log [e^{\lambda (e^t - 1)}] = \lambda (e^t - 1) \] \[ = \lambda \left [ \left ( 1 + t + \frac {t^2}{2!} + \frac {t^3}{3!} + \cdots + \frac {t^r}{r!} + \cdots \right ) - 1 \right ] \] \[ = \lambda t + \lambda \frac {t^2}{2!} + \lambda \frac {t^3}{3!} + \cdots + \lambda \frac {t^r}{r!} + \cdots \] The coefficient of \( \frac {t^r}{r!} \) in \( K_X(t) \) is \( \lambda \). Hence: \[ \kappa _r = \lambda , \quad r = 1, 2, 3, \ldots \] In particular: \[ \kappa _1 = \lambda , \quad \kappa _2 = \lambda , \quad \kappa _3 = \lambda , \quad \kappa _4 = \lambda \] Thus: \[ \mu _2 = \kappa _2 = \lambda , \quad \mu _3 = \kappa _3 = \lambda , \quad \mu _4 = \kappa _4 + 3\kappa _2^2 = \lambda + 3\lambda ^2 \] □

3.2.8 Additive Property of Poisson Distribution

Theorem 3.2.32 (Additive Property). If \( X_1, X_2, \ldots , X_k \) are independent Poisson variates with parameters \( \lambda _1, \lambda _2, \ldots , \lambda _k \) respectively, then: \[ \sum _{i=1}^{k} X_i \sim P\left ( \sum _{i=1}^{k} \lambda _i \right ) \]

Proof : For \( X \sim P(\lambda _1) \) and \( Y \sim P(\lambda _2) \), independent: \[ M_{X+Y}(t) = M_X(t) M_Y(t) = e^{\lambda _1(e^t - 1)} \cdot e^{\lambda _2(e^t - 1)} = e^{(\lambda _1 + \lambda _2)(e^t - 1)} \] By the uniqueness theorem of m.g.f., \( X + Y \sim P(\lambda _1 + \lambda _2) \). By induction, the result extends to \( k \) variables. □

Remark 3.2.33.

1.: The converse of the additive property is also true. If \( X \) and \( Y \) are independent and \( X + Y \sim P(\lambda _1 + \lambda _2) \), then \( X \sim P(\lambda _1) \) and \( Y \sim P(\lambda _2) \).
2.: The difference of two independent Poisson variates is NOT a Poisson variate because: \[ M_{X-Y}(t) = e^{\lambda _1(e^t - 1)} \cdot e^{\lambda _2(e^{-t} - 1)} \] which cannot be expressed in the form \( e^{\lambda (e^t - 1)} \).

3.2.9 Probability Generating Function of Poisson Distribution

Theorem 3.2.34 (Probability Generating Function). The probability generating function of the Poisson distribution \( P(\lambda ) \) is: \[ P(s) = e^{\lambda (s - 1)} \]

Proof : \[ P(s) = \sum _{x=0}^{\infty } p(x) s^x = \sum _{x=0}^{\infty } \frac {e^{-\lambda } \lambda ^x}{x!} s^x = e^{-\lambda } \sum _{x=0}^{\infty } \frac {(\lambda s)^x}{x!} = e^{\lambda (s - 1)} \] □

3.2.10 Mode of the Poisson Distribution

Theorem 3.2.35 (Mode of Poisson Distribution). The mode of the Poisson distribution \( P(\lambda ) \) is:

1.: The integral part of \( \lambda \), if \( \lambda \) is not an integer.
2.: Both \( \lambda - 1 \) and \( \lambda \), if \( \lambda \) is an integer.

Proof : \[ \frac {p(x)}{p(x-1)} = \frac {\lambda }{x} \] Case 1: When \( \lambda \) is not an integer. Let \( S \) be the integral part of \( \lambda \). Then: \[ \frac {p(x)}{p(x-1)} > 1 \text { for } x \leq S \quad \text {and} \quad < 1 \text { for } x > S \] Thus: \[ p(0) < p(1) < p(2) < \cdots < p(S) > p(S+1) > p(S+2) > \cdots \] Hence the unique mode is \( S \), the integral part of \( \lambda \). Case 2: When \( \lambda = k \) is an integer. Then: \[ \frac {p(k)}{p(k-1)} = 1 \] Thus: \[ p(0) < p(1) < \cdots < p(k-1) = p(k) > p(k+1) > \cdots \] Hence the distribution is bimodal with modes at \( k-1 \) and \( k \), i.e., at \( \lambda - 1 \) and \( \lambda \). □

3.2.11 Summary of Key Results for Poisson Distribution


Property	Result

Probability Mass Function	\( p(x) = \frac {e^{-\lambda } \lambda ^x}{x!}, \quad x = 0, 1, 2, \ldots \)
Mean	\( \mu _1' = \lambda \)
Variance	\( \mu _2 = \lambda \)
Third Central Moment	\( \mu _3 = \lambda \)
Fourth Central Moment	\( \mu _4 = \lambda + 3\lambda ^2 \)
Skewness	\( \beta _1 = \frac {1}{\lambda } \)
Kurtosis	\( \beta _2 = 3 + \frac {1}{\lambda } \)
Moment Generating Function	\( M_X(t) = e^{\lambda (e^t - 1)} \)
Characteristic Function	\( \phi _X(t) = e^{\lambda (e^{it} - 1)} \)
Probability Generating Function	\( P(s) = e^{\lambda (s - 1)} \)
Cumulants	\( \kappa _r = \lambda , \quad r = 1, 2, 3, \ldots \)
Mode	Integral part of \( \lambda \), or bimodal if integer
Additive Property	\( P(\lambda _1) * P(\lambda _2) = P(\lambda _1 + \lambda _2) \)
Recurrence Relation	\( p(x+1) = \frac {\lambda }{x+1} p(x) \)

3.3 Normal Distribution

The normal distribution was first discovered in 1733 by the English mathematician De-Moivre, who obtained this continuous distribution as a limiting case of the binomial distribution and applied it to problems arising in games of chance. It was also known to Laplace, no later than 1774, but through a historical error it was credited to Gauss, who first made reference to it in the beginning of the 19th century (1809), as the distribution of errors in Astronomy. Gauss used the normal curve to describe the theory of accidental errors of measurements involved in the calculation of orbits of heavenly bodies. The normal model has become the most important probability model in statistical analysis.

3.3.1 Definition

Definition 3.3.1 (Normal Distribution). A random variable \( X \) is said to have a normal distribution with parameters \( \mu \) (called "mean") and \( \sigma ^2 \) (called "variance") if its probability density function is given by: \[ f(x; \mu , \sigma ) = \frac {1}{\sigma \sqrt {2\pi }} \exp \left [ -\frac {1}{2} \left ( \frac {x - \mu }{\sigma } \right )^2 \right ], \quad -\infty < x < \infty , \quad -\infty < \mu < \infty , \quad \sigma > 0 \] or equivalently: \[ f(x; \mu , \sigma ) = \frac {1}{\sigma \sqrt {2\pi }} e^{-(x-\mu )^2 / 2\sigma ^2}, \quad -\infty < x < \infty \]

Remark 3.3.2.

1.: A random variable \( X \) with mean \( \mu \) and variance \( \sigma ^2 \) following the normal law is denoted by \( X \sim N(\mu , \sigma ^2) \).
2.: If \( X \sim N(\mu , \sigma ^2) \), then \( Z = \frac {X - \mu }{\sigma } \) is a standard normal variate with \( E(Z) = 0 \) and \( \operatorname {Var}(Z) = 1 \), and we write \( Z \sim N(0, 1) \).
3.: The p.d.f. of the standard normal variate \( Z \) is: \[ \phi (z) = \frac {1}{\sqrt {2\pi }} e^{-z^2/2}, \quad -\infty < z < \infty \]
4.: The corresponding distribution function of \( Z \), denoted by \( \Phi (z) \), is: \[ \Phi (z) = P(Z \leq z) = \int _{-\infty }^{z} \phi (u) \, du = \frac {1}{\sqrt {2\pi }} \int _{-\infty }^{z} e^{-u^2/2} \, du \]

3.3.2 Properties of Standard Normal Distribution Function

Theorem 3.3.3 (Result 1). \[ \Phi (-z) = 1 - \Phi (z) \]

Proof : \begin {align*} \Phi (-z) = P(Z \leq -z) &= P(Z \geq z) \quad (\text {by symmetry of the normal distribution}) \\ &= 1 - P(Z \leq z) = 1 - \Phi (z) \end {align*} □

Theorem 3.3.4 (Result 2). If \( X \sim N(\mu , \sigma ^2) \), then: \[ P(a \leq X \leq b) = \Phi \left ( \frac {b - \mu }{\sigma } \right ) - \Phi \left ( \frac {a - \mu }{\sigma } \right ) \]

Proof : \begin {align*} P(a \leq X \leq b) &= P\left ( \frac {a - \mu }{\sigma } \leq Z \leq \frac {b - \mu }{\sigma } \right ), \quad Z = \frac {X - \mu }{\sigma } \\ &= P\left ( Z \leq \frac {b - \mu }{\sigma } \right ) - P\left ( Z \leq \frac {a - \mu }{\sigma } \right ) \\ &= \Phi \left ( \frac {b - \mu }{\sigma } \right ) - \Phi \left ( \frac {a - \mu }{\sigma } \right ) \end {align*} □

3.3.3 Normal Distribution as a Limiting Form of Binomial Distribution

The normal distribution is another limiting form of the binomial distribution under the following conditions:

1.: \( n \), the number of trials, is indefinitely large, i.e., \( n \to \infty \).
2.: Neither \( p \) nor \( q \) is very small.

The probability function of the binomial distribution is: \[ p(x) = \binom {n}{x} p^x q^{n-x} = \frac {n!}{x!(n-x)!} p^x q^{n-x}, \quad x = 0, 1, 2, \ldots , n \] Consider the standard binomial variate: \[ Z = \frac {X - E(X)}{\sqrt {V(X)}} = \frac {X - np}{\sqrt {npq}}, \quad X = 0, 1, 2, \ldots , n \] When \( X = 0 \), \( Z = -\sqrt {np/q} \), and when \( X = n \), \( Z = \sqrt {nq/p} \). Thus, in the limit as \( n \to \infty \), \( Z \) takes values from \( -\infty \) to \( \infty \). Using Stirling’s approximation \( r! \approx \sqrt {2\pi } e^{-r} r^{r+1/2} \), the limiting probability density function is: \[ g(z) = \frac {1}{\sqrt {2\pi }} e^{-z^2/2}, \quad -\infty < z < \infty \] If \( X \) is a normal variate with mean \( \mu \) and s.d. \( \sigma \), then \( Z = (X - \mu )/\sigma \) is a standard normal variate. The Jacobian of transformation is \( 1/\sigma \). Hence: \[ f(x) = \frac {1}{\sigma \sqrt {2\pi }} e^{-(x-\mu )^2 / 2\sigma ^2}, \quad -\infty < x < \infty \]

3.3.4 Chief Characteristics of the Normal Distribution and Normal Probability Curve

The normal probability curve with mean \( \mu \) and standard deviation \( \sigma \) is given by: \[ f(x) = \frac {1}{\sigma \sqrt {2\pi }} e^{-(x-\mu )^2 / 2\sigma ^2}, \quad -\infty < x < \infty \] It has the following properties:

1.: The curve is bell-shaped and symmetrical about the line \( x = \mu \).
2.: Mean, median, and mode of the distribution coincide.
3.: As \( x \) increases numerically, \( f(x) \) decreases rapidly, the maximum probability occurring at the point \( x = \mu \), given by: \[ [f(x)]_{\max } = \frac {1}{\sigma \sqrt {2\pi }} \]
4.: \( \beta _1 = 0 \) and \( \beta _2 = 3 \).
5.: Moments about the mean: \[ \mu _{2r+1} = 0, \quad r = 0, 1, 2, \ldots \] \[ \mu _{2r} = 1 \cdot 3 \cdot 5 \cdots (2r-1) \sigma ^{2r}, \quad r = 0, 1, 2, \ldots \]
6.: Since \( f(x) \) is a probability, it can never be negative; no portion of the curve lies below the \( x \)-axis.
7.: The \( x \)-axis is an asymptote to the curve.
8.: The points of inflexion of the curve are at \( x = \mu \pm \sigma \).
9.: Mean deviation about mean is: \[ \sqrt {\frac {2}{\pi }} \sigma \approx \frac {4}{5} \sigma \] Quartile deviation: \[ Q.D. = \frac {Q_3 - Q_1}{2} \approx \frac {2}{3} \sigma \] Thus: \[ Q.D. : M.D. : S.D. \approx \frac {2}{3} \sigma : \frac {4}{5} \sigma : \sigma = 10 : 12 : 15 \]
10.: Area Properties: \begin {align*} P(\mu - \sigma < X < \mu + \sigma ) &= 0.6826 \\ P(\mu - 2\sigma < X < \mu + 2\sigma ) &= 0.9544 \\ P(\mu - 3\sigma < X < \mu + 3\sigma ) &= 0.9973 \end {align*}

3.3.5 Mode of Normal Distribution

Theorem 3.3.5 (Mode of Normal Distribution). The mode of the normal distribution \( N(\mu , \sigma ^2) \) is \( \mu \).

Proof : \[ \log f(x) = -\frac {1}{2} \log (2\pi ) - \log \sigma - \frac {(x-\mu )^2}{2\sigma ^2} \] Differentiating with respect to \( x \): \[ \frac {f'(x)}{f(x)} = -\frac {x-\mu }{\sigma ^2} \Rightarrow f'(x) = -\frac {x-\mu }{\sigma ^2} f(x) \] For mode, \( f'(x) = 0 \): \[ -\frac {x-\mu }{\sigma ^2} f(x) = 0 \Rightarrow x = \mu \] Now: \[ f''(x) = -\frac {1}{\sigma ^2} \left [ f(x) + (x-\mu ) f'(x) \right ] \] At \( x = \mu \): \[ f''(\mu ) = -\frac {1}{\sigma ^2} f(\mu ) = -\frac {1}{\sigma ^3 \sqrt {2\pi }} < 0 \] Hence \( x = \mu \) is the mode of the normal distribution. □

3.3.6 Median of Normal Distribution

Theorem 3.3.6 (Median of Normal Distribution). The median of the normal distribution \( N(\mu , \sigma ^2) \) is \( \mu \).

Proof : If \( M \) is the median, then: \[ \int _{-\infty }^{M} f(x) \, dx = \frac {1}{2} \] Let \( z = \frac {x - \mu }{\sigma } \): \[ \frac {1}{\sqrt {2\pi }} \int _{-\infty }^{(M-\mu )/\sigma } e^{-z^2/2} \, dz = \frac {1}{2} \] By symmetry of the standard normal distribution: \[ \frac {1}{\sqrt {2\pi }} \int _{-\infty }^{0} e^{-z^2/2} \, dz = \frac {1}{2} \] Thus: \[ \frac {M - \mu }{\sigma } = 0 \Rightarrow M = \mu \] Hence for the normal distribution, mean = median = mode = \( \mu \). □

3.3.7 Moment Generating Function of Normal Distribution

Theorem 3.3.7 (Moment Generating Function). The moment generating function of the normal distribution \( N(\mu , \sigma ^2) \) is: \[ M_X(t) = e^{\mu t + \frac {1}{2} \sigma ^2 t^2} \]

Proof : \begin {align*} M_X(t) = E(e^{tX}) &= \int _{-\infty }^{\infty } e^{tx} \frac {1}{\sigma \sqrt {2\pi }} e^{-(x-\mu )^2 / 2\sigma ^2} \, dx \\ &= \frac {1}{\sigma \sqrt {2\pi }} \int _{-\infty }^{\infty } \exp \left [ tx - \frac {(x-\mu )^2}{2\sigma ^2} \right ] dx \end {align*} Let \( z = \frac {x - \mu }{\sigma } \), so \( x = \mu + \sigma z \), \( dx = \sigma dz \): \begin {align*} M_X(t) &= \frac {1}{\sqrt {2\pi }} \int _{-\infty }^{\infty } \exp \left [ t(\mu + \sigma z) - \frac {z^2}{2} \right ] dz \\ &= \frac {e^{\mu t}}{\sqrt {2\pi }} \int _{-\infty }^{\infty } \exp \left [ -\frac {z^2}{2} + \sigma t z \right ] dz \\ &= \frac {e^{\mu t}}{\sqrt {2\pi }} \int _{-\infty }^{\infty } \exp \left [ -\frac {1}{2}(z - \sigma t)^2 + \frac {1}{2} \sigma ^2 t^2 \right ] dz \\ &= e^{\mu t + \frac {1}{2} \sigma ^2 t^2} \cdot \frac {1}{\sqrt {2\pi }} \int _{-\infty }^{\infty } e^{-(z-\sigma t)^2/2} dz \\ &= e^{\mu t + \frac {1}{2} \sigma ^2 t^2} \end {align*} □

Corollary 3.3.8 (M.G.F. of Standard Normal Variate). For \( Z = \frac {X - \mu }{\sigma } \sim N(0, 1) \): \[ M_Z(t) = e^{t^2/2} \]

Proof : \[ M_Z(t) = e^{-\mu t/\sigma } M_X(t/\sigma ) = e^{-\mu t/\sigma } \cdot e^{\mu t/\sigma + \frac {1}{2} \sigma ^2 (t/\sigma )^2} = e^{t^2/2} \] □

3.3.8 Cumulant Generating Function of Normal Distribution

Theorem 3.3.9 (Cumulant Generating Function). The cumulant generating function of the normal distribution \( N(\mu , \sigma ^2) \) is: \[ K_X(t) = \mu t + \frac {1}{2} \sigma ^2 t^2 \]

Proof : \[ K_X(t) = \log M_X(t) = \log \left ( e^{\mu t + \frac {1}{2} \sigma ^2 t^2} \right ) = \mu t + \frac {1}{2} \sigma ^2 t^2 \] Comparing coefficients: \begin {align*} \kappa _1 &= \text {Coefficient of } t = \mu = \text {Mean} \\ \kappa _2 &= \text {Coefficient of } \frac {t^2}{2!} = \sigma ^2 = \text {Variance} \\ \kappa _r &= 0, \quad r = 3, 4, \ldots \end {align*} Thus: \[ \mu _3 = \kappa _3 = 0, \quad \mu _4 = \kappa _4 + 3\kappa _2^2 = 3\sigma ^4 \] \[ \beta _1 = \frac {\mu _3^2}{\mu _2^3} = 0, \quad \beta _2 = \frac {\mu _4}{\mu _2^2} = \frac {3\sigma ^4}{\sigma ^4} = 3 \] □

3.3.9 Moments of Normal Distribution

Theorem 3.3.10 (Moments about Mean). For the normal distribution \( N(\mu , \sigma ^2) \): \begin {align*} \mu _{2r+1} &= 0, \quad r = 0, 1, 2, \ldots \\ \mu _{2r} &= 1 \cdot 3 \cdot 5 \cdots (2r-1) \sigma ^{2r}, \quad r = 0, 1, 2, \ldots \end {align*}

Proof : Odd order moments: \begin {align*} \mu _{2n+1} &= \int _{-\infty }^{\infty } (x-\mu )^{2n+1} f(x) \, dx \\ &= \frac {1}{\sigma \sqrt {2\pi }} \int _{-\infty }^{\infty } (x-\mu )^{2n+1} e^{-(x-\mu )^2 / 2\sigma ^2} \, dx \end {align*} Let \( z = \frac {x - \mu }{\sigma } \): \[ \mu _{2n+1} = \frac {\sigma ^{2n+1}}{\sqrt {2\pi }} \int _{-\infty }^{\infty } z^{2n+1} e^{-z^2/2} \, dz \] The integrand \( z^{2n+1} e^{-z^2/2} \) is an odd function of \( z \). Therefore: \[ \mu _{2n+1} = 0 \] Even order moments: \begin {align*} \mu _{2n} &= \frac {\sigma ^{2n}}{\sqrt {2\pi }} \int _{-\infty }^{\infty } z^{2n} e^{-z^2/2} \, dz \\ &= \frac {2\sigma ^{2n}}{\sqrt {2\pi }} \int _{0}^{\infty } z^{2n} e^{-z^2/2} \, dz \end {align*} Let \( t = z^2/2 \), so \( z = \sqrt {2t} \), \( dz = \frac {dt}{\sqrt {2t}} \): \[ \mu _{2n} = \frac {2\sigma ^{2n}}{\sqrt {2\pi }} \int _{0}^{\infty } (2t)^n e^{-t} \frac {dt}{\sqrt {2t}} = \frac {2^n \sigma ^{2n}}{\sqrt {\pi }} \int _{0}^{\infty } t^{n-1/2} e^{-t} \, dt \] \[ = \frac {2^n \sigma ^{2n}}{\sqrt {\pi }} \Gamma \left ( n + \frac {1}{2} \right ) = 1 \cdot 3 \cdot 5 \cdots (2n-1) \sigma ^{2n} \] Aliter: Using the m.g.f. about mean: \[ M_{X-\mu }(t) = e^{-\mu t} M_X(t) = e^{\frac {1}{2} \sigma ^2 t^2} \] \[ = 1 + \frac {\sigma ^2 t^2}{2!} + \frac {3\sigma ^4 t^4}{4!} + \frac {15\sigma ^6 t^6}{6!} + \cdots \] The coefficient of \( \frac {t^r}{r!} \) gives \( \mu _r \). Hence: \[ \mu _{2n+1} = 0, \quad \mu _{2n} = 1 \cdot 3 \cdot 5 \cdots (2n-1) \sigma ^{2n} \] □

Corollary 3.3.11. In particular: \[ \mu _1 = 0, \quad \mu _2 = \sigma ^2, \quad \mu _3 = 0, \quad \mu _4 = 3\sigma ^4 \] \[ \beta _1 = \frac {\mu _3^2}{\mu _2^3} = 0, \quad \beta _2 = \frac {\mu _4}{\mu _2^2} = 3 \]

3.3.10 Area Properties of Normal Distribution

Theorem 3.3.12 (Area Properties). For a normal distribution \( N(\mu , \sigma ^2) \): \begin {align*} P(\mu - \sigma < X < \mu + \sigma ) &= 0.6826 \\ P(\mu - 2\sigma < X < \mu + 2\sigma ) &= 0.9544 \\ P(\mu - 3\sigma < X < \mu + 3\sigma ) &= 0.9973 \end {align*}

Proof : \[ P(\mu - k\sigma < X < \mu + k\sigma ) = P(-k < Z < k) = 2\Phi (k) - 1 \] For \( k = 1 \): \[ 2\Phi (1) - 1 = 2(0.8413) - 1 = 0.6826 \] For \( k = 2 \): \[ 2\Phi (2) - 1 = 2(0.9772) - 1 = 0.9544 \] For \( k = 3 \): \[ 2\Phi (3) - 1 = 2(0.99865) - 1 = 0.9973 \] □

3.3.11 Important Area Table for Standard Normal Variate

\[ \begin {array}{|c|c|} \hline \text {Distance from Mean in terms of } \sigma & \text {Area under the Curve} \\ \hline Z = \pm 0.6745 & 50\% = 0.50 \\ Z = \pm 1.00 & 68.26\% = 0.6826 \\ Z = \pm 1.96 & 95\% = 0.95 \\ Z = \pm 2.00 & 95.44\% = 0.9544 \\ Z = \pm 2.58 & 99\% = 0.99 \\ Z = \pm 3.00 & 99.73\% = 0.9973 \\ \hline \end {array} \]

3.3.12 Additional Important Properties

1.: If \( X \) and \( Y \) are independent standard normal variates, then \( U = X + Y \) and \( V = X - Y \) are independently distributed with \( U \sim N(0, 2) \) and \( V \sim N(0, 2) \).
2.: Bernstein’s Theorem: If \( X \) and \( Y \) are independent and identically distributed random variables with finite variance, and if \( U = X + Y \) and \( V = X - Y \) are independent, then all r.v.’s \( X, Y, U, V \) are normally distributed.
3.: Characterisation of Normal Distribution: If \( X_1, X_2, \ldots , X_n \) are i.i.d. r.v.’s with finite variance, then the common distribution is normal if and only if \( \bar {X} = \frac {1}{n} \sum X_i \) and \( s^2 = \frac {1}{n} \sum (X_i - \bar {X})^2 \) are independent.

3.3.13 Examples on Normal Distribution

Example 3.3.13 (Example 8.1). If \( X \sim U(a, b) \) with mean 1 and variance \( \frac {4}{3} \), find \( P(X < 0) \).

Solution. For uniform distribution: \[ \text {Mean} = \frac {a+b}{2} = 1 \Rightarrow a + b = 2 \] \[ \text {Variance} = \frac {(b-a)^2}{12} = \frac {4}{3} \Rightarrow (b-a)^2 = 16 \Rightarrow b - a = 4 \] Solving: \[ a + b = 2, \quad b - a = 4 \Rightarrow 2b = 6 \Rightarrow b = 3, \quad a = -1 \] Thus: \[ f(x) = \frac {1}{4}, \quad -1 < x < 3 \] \[ P(X < 0) = \int _{-1}^{0} \frac {1}{4} \, dx = \frac {1}{4} \]

Example 3.3.14 (Example 8.2). Subway trains at a certain line run every half hour between midnight and six in the morning. What is the probability that a man entering the station at a random time during this period will have to wait at least twenty minutes?

Solution. Let \( X \) be the waiting time for the next train. Then \( X \) is uniformly distributed on \( (0, 30) \): \[ f(x) = \frac {1}{30}, \quad 0 < x < 30 \] Probability of waiting at least 20 minutes: \[ P(X \geq 20) = \int _{20}^{30} \frac {1}{30} \, dx = \frac {10}{30} = \frac {1}{3} \]

Example 3.3.15 (Example 8.3). If \( X \) has a uniform distribution in \( [0, 1] \), find the distribution (p.d.f.) of \( Y = -2\log X \). Identify the distribution also.

Solution. For \( X \sim U[0, 1] \): \[ f_X(x) = 1, \quad 0 < x < 1 \] The distribution function of \( Y \): \[ G(y) = P(Y \leq y) = P(-2\log X \leq y) = P(\log X \geq -y/2) \] \[ = P(X \geq e^{-y/2}) = 1 - P(X \leq e^{-y/2}) = 1 - e^{-y/2}, \quad y > 0 \] The p.d.f. of \( Y \) is: \[ g(y) = \frac {d}{dy} G(y) = \frac {1}{2} e^{-y/2}, \quad 0 < y < \infty \] This is the p.d.f. of an exponential distribution with parameter \( \theta = \frac {1}{2} \). It is also the chi-square distribution with 2 degrees of freedom.

Example 3.3.16 (Example 8.4). Show that for the rectangular distribution: \[ f(x) = \frac {1}{2a}, \quad -a < x < a \] the m.g.f. about origin is \( \frac {\sinh at}{at} \). Also show that moments of even order are given by: \[ \mu _{2n} = \frac {a^{2n}}{2n+1} \]

Solution. The m.g.f. about origin is: \[ M_X(t) = E(e^{tX}) = \int _{-a}^{a} e^{tx} \frac {1}{2a} \, dx = \frac {1}{2a} \left [ \frac {e^{tx}}{t} \right ]_{-a}^{a} \] \[ = \frac {1}{2at} (e^{at} - e^{-at}) = \frac {\sinh at}{at} \] Expanding: \[ M_X(t) = \frac {1}{at} \left [ at + \frac {(at)^3}{3!} + \frac {(at)^5}{5!} + \cdots \right ] = 1 + \frac {a^2 t^2}{3!} + \frac {a^4 t^4}{5!} + \cdots \] Since there are no terms with odd powers of \( t \), all moments of odd order vanish. For even moments: \[ \mu _{2n} = \text {Coefficient of } \frac {t^{2n}}{(2n)!} \text { in } M(t) = \frac {a^{2n}}{2n+1} \]

Example 3.3.17 (Example 8.5). If \( X_1 \) and \( X_2 \) are independent rectangular variates on \( [0, 1] \), find the distributions of: (i) \( X_1 / X_2 \), (ii) \( X_1 X_2 \), (iii) \( X_1 + X_2 \), (iv) \( X_1 - X_2 \).

Solution. We are given: \[ f_{X_1}(x_1) = f_{X_2}(x_2) = 1, \quad 0 < x_1 < 1, \quad 0 < x_2 < 1 \] The joint p.d.f. is: \[ f(x_1, x_2) = 1 \] (i) Let \( U = \frac {X_1}{X_2} \) and \( V = X_2 \). Then \( x_1 = uv, x_2 = v \). \[ |J| = \left | \begin {array}{cc} \frac {\partial x_1}{\partial u} & \frac {\partial x_1}{\partial v} \\ \frac {\partial x_2}{\partial u} & \frac {\partial x_2}{\partial v} \end {array} \right | = \left | \begin {array}{cc} v & u \\ 0 & 1 \end {array} \right | = v \] The joint p.d.f. of \( U \) and \( V \) is: \[ g(u, v) = v, \quad 0 < u < \infty , \quad 0 < v < \infty \] The distribution of \( U \): \[ g_U(u) = \begin {cases} \int _0^1 v \, dv = \frac {1}{2}, & 0 < u < 1 \\ \int _0^{1/u} v \, dv = \frac {1}{2u^2}, & u > 1 \end {cases} \] (ii) Let \( U = X_1 X_2 \) and \( V = X_1 \). Then \( x_1 = v, x_2 = u/v \). \[ |J| = \left | \begin {array}{cc} 0 & 1 \\ 1/v & -u/v^2 \end {array} \right | = -\frac {1}{v} \] \[ g(u, v) = \frac {1}{v}, \quad 0 < u < 1, \quad u < v < 1 \] \[ g_U(u) = \int _u^1 \frac {1}{v} \, dv = -\log u, \quad 0 < u < 1 \] (iii) Let \( U = X_1 + X_2 \) and \( V = X_1 - X_2 \). Then \( x_1 = \frac {u+v}{2}, x_2 = \frac {u-v}{2} \). \[ |J| = \frac {1}{2} \] \[ g(u, v) = \frac {1}{2}, \quad 0 < u < 2, \quad -1 < v < 1 \] \[ g_U(u) = \begin {cases} \int _{-u}^{u} \frac {1}{2} \, dv = u, & 0 < u < 1 \\ \int _{u-2}^{2-u} \frac {1}{2} \, dv = 2 - u, & 1 < u < 2 \end {cases} \] (iv) Similarly, \( V = X_1 - X_2 \): \[ g_V(v) = \begin {cases} 1 + v, & -1 < v < 0 \\ 1 - v, & 0 < v < 1 \end {cases} \]

Example 3.3.18 (Example 8.6). If \( X \) is a random variable with a continuous distribution function \( F \), then \( F(X) \) has a uniform distribution on \( [0, 1] \).

Solution. Let \( Y = F(X) \). Then: \[ G_Y(y) = P(Y \leq y) = P[F(X) \leq y] = P[X \leq F^{-1}(y)] = F[F^{-1}(y)] = y \] Thus: \[ g_Y(y) = \frac {d}{dy} G_Y(y) = 1, \quad 0 \leq y \leq 1 \] Hence \( Y \sim U[0, 1] \).

Example 3.3.19 (Example 8.7). If \( X \) and \( Y \) are independent rectangular variates for the range \( -a \) to \( a \) each, then show that the sum \( U = X + Y \) has the probability density: \[ g_U(u) = \begin {cases} \frac {2a + u}{4a^2}, & -2a \leq u \leq 0 \\ \frac {2a - u}{4a^2}, & 0 \leq u \leq 2a \end {cases} \]

Solution. For \( X, Y \sim U[-a, a] \): \[ f_X(x) = \frac {1}{2a}, \quad -a < x < a, \quad f_Y(y) = \frac {1}{2a}, \quad -a < y < a \] The joint p.d.f. is: \[ f(x, y) = \frac {1}{4a^2}, \quad -a < x < a, \quad -a < y < a \] The distribution of \( U = X + Y \) is the convolution: \[ g_U(u) = \int _{-\infty }^{\infty } f_X(x) f_Y(u - x) \, dx \] For \( -2a \leq u \leq 0 \): \[ g_U(u) = \int _{-a}^{u+a} \frac {1}{4a^2} \, dx = \frac {u + 2a}{4a^2} \] For \( 0 \leq u \leq 2a \): \[ g_U(u) = \int _{u-a}^{a} \frac {1}{4a^2} \, dx = \frac {2a - u}{4a^2} \] Thus: \[ g_U(u) = \begin {cases} \frac {2a + u}{4a^2}, & -2a \leq u \leq 0 \\ \frac {2a - u}{4a^2}, & 0 \leq u \leq 2a \end {cases} \]

Example 3.3.20 (Example 8.8). On the \( x \)-axis, \( (n+1) \) points are taken independently between the origin and \( x = 1 \), all positions being equally likely. Show that the probability that the \( (k+1) \)th of these points, counted from the origin, lies in the interval \( x - \frac {1}{2} dx \) to \( x + \frac {1}{2} dx \) is: \[ \binom {n}{k} (n+1) x^k (1-x)^{n-k} dx \] Verify that the integral of this expression from \( x = 0 \) to \( x = 1 \) is unity.

Solution. Here \( X \sim U[0, 1] \) with \( f_X(x) = 1 \). \[ P(0 < X < x) = x, \quad P(X > x) = 1 - x \] \[ P\left ( x - \frac {dx}{2} < X < x + \frac {dx}{2} \right ) = dx \] Required probability: \[ p = \binom {n+1}{k} x^k \cdot \binom {n+1-k}{n-k} (1-x)^{n-k} \cdot dx \] \[ = \binom {n}{k} (n+1) x^k (1-x)^{n-k} dx \] Verification: \[ \int _0^1 \binom {n}{k} (n+1) x^k (1-x)^{n-k} dx = (n+1) \binom {n}{k} \frac {\Gamma (k+1)\Gamma (n-k+1)}{\Gamma (n+2)} \] \[ = (n+1) \cdot \frac {n!}{k!(n-k)!} \cdot \frac {k!(n-k)!}{(n+1)!} = 1 \]

Example 3.3.21 (Example 8.9). If \( X \) and \( Y \) are i.i.d. \( U[-a, a] \) variates, find the p.d.f. of \( Z = X + Y \) and identify the distribution.

Solution. For \( X \sim U[-a, a] \): \[ M_X(t) = \frac {e^{at} - e^{-at}}{2at} = \frac {\sinh at}{at} \] Since \( X \) and \( Y \) are independent: \[ M_Z(t) = M_X(t) M_Y(t) = \left ( \frac {\sinh at}{at} \right )^2 \] This is the m.g.f. of a triangular distribution on \( (-2a, 2a) \) with peak at \( x = 0 \). Hence: \[ f_Z(z) = \begin {cases} \frac {2a + z}{4a^2}, & -2a \leq z \leq 0 \\ \frac {2a - z}{4a^2}, & 0 \leq z \leq 2a \end {cases} \] Thus \( Z \sim \text {Trg}(-2a, 2a) \).

3.3.14 Summary of Key Results for Normal Distribution


Property	Result

Probability Density Function	\( f(x) = \dfrac {1}{\sigma \sqrt {2\pi }} e^{-(x-\mu )^2 / 2\sigma ^2}, \quad -\infty < x < \infty \)

Mean	\( \mu _1' = \mu \)

Variance	\( \mu _2 = \sigma ^2 \)

Third Central Moment	\( \mu _3 = 0 \)

Fourth Central Moment	\( \mu _4 = 3\sigma ^4 \)

Skewness	\( \beta _1 = 0 \)

Kurtosis	\( \beta _2 = 3 \)

Moment Generating Function	\( M_X(t) = e^{\mu t + \frac {1}{2} \sigma ^2 t^2} \)

Characteristic Function	\( \phi _X(t) = e^{i\mu t - \frac {1}{2} \sigma ^2 t^2} \)

Cumulants	\( \kappa _1 = \mu , \quad \kappa _2 = \sigma ^2, \quad \kappa _r = 0 \text { for } r \geq 3 \)

Mode	\( \mu \)

Median	\( \mu \)

Points of Inflexion	\( x = \mu \pm \sigma \)

Mean Deviation	\( \sqrt {\dfrac {2}{\pi }} \sigma \approx 0.7979\sigma \)

Quartile Deviation	\( 0.6745\sigma \)

Area within \( \mu \pm \sigma \)	\( 68.26\% \)

Area within \( \mu \pm 2\sigma \)	\( 95.44\% \)

Area within \( \mu \pm 3\sigma \)	\( 99.73\% \)

Additive Property	\( N(\mu _1, \sigma _1^2) * N(\mu _2, \sigma _2^2) = N(\mu _1 + \mu _2, \sigma _1^2 + \sigma _2^2) \)

ARO Study Circle

TANSCHE SMTPT31 Mathematical Statistics Unit 3

Mathematical Statistics (TANSCHE Syllabus) Madurai Kamaraj University II- B.Sc., Mathematics, III - Semester

Contents

Chapter 3Discrete Distribution