Mathematical Statistics
(TANSCHE Syllabus)
Madurai Kamaraj University
II- B.Sc., Mathematics, III - Semester
Contents
Chapter 1 Theory of Probability
Syllabus
Unit I : Probability
Definition of Sample Space - Events - Definition of Probability - Addition and Multiplication laws of probability - independence of events- Conditional Probability - Baye’s theorem - Simple Problems
Chapter 4 - sections 4.1 – 4.3 and sections 4.5 - 4.8
1.1 Introduction to the Theory of Probability
When an experiment is repeated under essentially homogeneous and similar conditions, we generally come across two types of situations:
-
1. The result or what is usually known as the ‘outcome’ is unique or certain.
-
2. The result is not unique but may be one of several possible outcomes.
1.1.1 Deterministic Phenomena
-
Example 1.1.3. For a number of situations, the deterministic model suffices. Examples include:
-
• Boyle’s Law: For a perfect gas, \(V \propto \frac {1}{P}\) i.e., \(PV = \text {constant}\), provided the temperature remains the same.
-
• Particle Velocity: The velocity \(v\) of a particle after time \(t\) is given by \(v = u + at\), where \(u\) is the initial velocity and \(a\) is the acceleration. This equation uniquely determines \(v\) if the right-hand quantities are known.
-
• Ohm’s Law: Given by \(C = \frac {E}{R}\), where \(C\) is the flow of current, \(E\) is the potential difference between the two ends of the conductor, and \(R\) is the resistance. This relationship uniquely determines the value of \(C\) as soon as \(E\) and \(R\) are given.
-
1.1.2 Probabilistic Phenomena
1.2 Definitions of Various Terms
1.2.1 Trial and Event
1.2.2 Exhaustive Events
-
-
1. Tossing a coin: There are 2 exhaustive cases (H or T).
-
2. Throwing a die: There are 6 exhaustive cases (1, 2, 3, 4, 5, 6).
-
3. Drawing two cards from a deck: The exhaustive number of cases is \(\binom {52}{2}\), because we choose 2 cards out of 52.
-
4. Throwing two dice: The exhaustive number of cases is \(6^2 = 36\).
-
5. In general, throwing \(n\) dice gives \(6^n\) exhaustive cases.
-
1.2.3 Favourable Events or Cases
-
-
1. Drawing a card from a deck:
-
• Favourable cases for drawing an ace = 4.
-
• Favourable cases for drawing a spade = 13.
-
• Favourable cases for drawing a red card = 26.
-
-
2. Throwing two dice: The favourable cases for getting a sum of 5 are: (1,4), (4,1), (2,3), (3,2). So, 4 favourable cases.
-
1.2.4 Mutually Exclusive Events
1.2.5 Equally Likely Events
1.3 Independent Events
-
-
1. Tossing a coin multiple times: Getting a head on the first toss does not affect getting a head on the second toss. These are independent events.
-
2. Drawing cards with replacement: If you draw a card, replace it, and then draw again, the two draws are independent.
-
3. Note: If you draw a card and do not replace it, the second draw depends on the first. These are dependent events.
-
1.3.1 Mathematical or Classical Probability
-
-
1. Odds in favour of \(E\) are \(m : (n - m)\).
-
2. Odds against \(E\) are \((n - m) : m\).
-
3. The probability that \(E\) does not happen is:
\[ q = \frac {n - m}{n} = 1 - \frac {m}{n} = 1 - p \]
\[ \text {So, } p + q = 1 \]
-
4. \(p\) and \(q\) are always between 0 and 1: \(0 \le p \le 1\), \(0 \le q \le 1\).
-
5. \(p\) is called probability of success, \(q\) is called probability of failure.
-
6. If \(P(E) = 1\), \(E\) is a certain event.
-
7. If \(P(E) = 0\), \(E\) is an impossible event.
-
Limitations of Classical Definition
-
1. It fails when outcomes are not equally likely. Example: The probability that a student passes an exam is not \(1/2\), because passing and failing are not equally likely.
-
2. It fails when the total number of outcomes is infinite.
1.3.2 Statistical or Empirical Probability
-
Definition 1.3.5 (Statistical Probability (Von Mises)). If an experiment is repeated \(n\) times under the same conditions, and an event \(E\) occurs \(m\) times, then the probability of \(E\) is the limiting value of \(m/n\) as \(n\) becomes very large:
\[ P(E) = \lim _{n \to \infty } \frac {m}{n} \]
(We assume this limit exists and is unique.)
1.3.3 Solved Examples
-
Solution. A leap year has 366 days = 52 weeks + 2 extra days. The possible pairs of extra days are:
(Sun, Mon), (Mon, Tue), (Tue, Wed), (Wed, Thu), (Thu, Fri), (Fri, Sat), (Sat, Sun)
Out of these, 2 pairs contain a Sunday: (Sun, Mon) and (Sat, Sun). So, favourable cases = 2, total cases = 7.
\[ P = \frac {2}{7} \]
-
Example 1.3.8. (a) Two cards are drawn at random from a well-shuffled pack of 52 cards. Show that the chance of drawing two aces is \(1/221\).
(b) From a pack of 52 cards, three are drawn at random. Find the chance that they are a king, a queen, and a knave (jack).
(c) Four cards are drawn from a pack. Find the probability that (i) all are diamonds, (ii) there is one card of each suit, (iii) there are two spades and two hearts.
-
Solution. (a) Total ways = \(\binom {52}{2}\). Favourable ways = \(\binom {4}{2}\).
\[ P = \frac {\binom {4}{2}}{\binom {52}{2}} = \frac {6}{1326} = \frac {1}{221} \]
(b) Total ways = \(\binom {52}{3}\). Favourable ways: Choose 1 king from 4, 1 queen from 4, 1 jack from 4 = \(4 \times 4 \times 4 = 64\).
\[ P = \frac {64}{\binom {52}{3}} = \frac {64}{22100} = \frac {16}{5525} \]
(c) Total ways = \(\binom {52}{4}\).
(i) All diamonds: Choose 4 diamonds from 13 = \(\binom {13}{4}\).\[ P = \frac {\binom {13}{4}}{\binom {52}{4}} \]
(ii) One card of each suit: Choose 1 from each of the 4 suits = \(\binom {13}{1}^4 = 13^4\).
\[ P = \frac {13^4}{\binom {52}{4}} \]
(iii) Two spades and two hearts: Choose 2 spades from 13 = \(\binom {13}{2}\), choose 2 hearts from 13 = \(\binom {13}{2}\).
\[ P = \frac {\binom {13}{2} \times \binom {13}{2}}{\binom {52}{4}} \]
-
Solution. Total ways = \(\binom {52}{13}\). Number of ways to get 9 cards of one specific suit: Choose 9 from that suit = \(\binom {13}{9}\), and the remaining 4 cards from the other 39 cards = \(\binom {39}{4}\). There are 4 suits, so total favourable = \(4 \times \binom {13}{9} \times \binom {39}{4}\).
\[ P = \frac {4 \times \binom {13}{9} \times \binom {39}{4}}{\binom {52}{13}} \]
-
Example 1.3.10. (a) Among the digits 1,2,3,4,5, first one digit is chosen, and then a second digit is chosen from the remaining four. All 20 outcomes are equally likely. Find the probability that an odd digit is selected (i) the first time, (ii) the second time, (iii) both times.
(b) From 25 tickets numbered 1 to 25, one ticket is drawn at random. Find the chance that it is (i) a multiple of 5 or 7, (ii) a multiple of 3 or 7.
-
Solution. (a) Total outcomes = \(5 \times 4 = 20\).
(i) First digit odd: Odd digits are 1,3,5 (3 digits). For each, second digit can be any of the remaining 4. Favourable = \(3 \times 4 = 12\).\[ P = \frac {12}{20} = \frac {3}{5} \]
(ii) Second digit odd: Count outcomes where second digit is odd. By symmetry, same as (i): \(12\) outcomes.
\[ P = \frac {12}{20} = \frac {3}{5} \]
(iii) Both digits odd: Choose first from 1,3,5 (3 ways), then second from the remaining 2 odd digits (2 ways). Favourable = \(3 \times 2 = 6\).
\[ P = \frac {6}{20} = \frac {3}{10} \]
(b) (i) Multiples of 5: 5,10,15,20,25 (5 numbers). Multiples of 7: 7,14,21 (3 numbers). No overlap. Favourable = \(5+3=8\).
\[ P = \frac {8}{25} \]
(ii) Multiples of 3: 3,6,9,12,15,18,21,24 (8 numbers). Multiples of 7: 7,14,21 (3 numbers). Common: 21 (1 number). Favourable = \(8+3-1=10\).
\[ P = \frac {10}{25} = \frac {2}{5} \]
-
Example 1.3.11. A committee of 4 people is to be appointed from 3 officers of the production department, 4 officers of the purchase department, 2 officers of the sales department, and 1 chartered accountant. Find the probability of forming the committee in the following manner: (i) There must be one from each category. (ii) It should have at least one from the purchase department. (iii) The chartered accountant must be in the committee.
-
Solution. Total persons = \(3+4+2+1 = 10\). Total committees = \(\binom {10}{4} = 210\).
(i) One from each category: \(\binom {3}{1} \times \binom {4}{1} \times \binom {2}{1} \times \binom {1}{1} = 3 \times 4 \times 2 \times 1 = 24\).
\[ P = \frac {24}{210} = \frac {4}{35} \]
(ii) At least one from purchase = \(1 - P(\text {no purchase})\). No purchase: choose all 4 from the other \(3+2+1=6\) persons: \(\binom {6}{4} = 15\).
\[ P(\text {no purchase}) = \frac {15}{210} = \frac {1}{14} \]
\[ P(\text {at least one purchase}) = 1 - \frac {1}{14} = \frac {13}{14} \]
(iii) CA is in committee: Choose CA (1 way), then choose remaining 3 from the other 9: \(\binom {9}{3} = 84\).
\[ P = \frac {84}{210} = \frac {2}{5} \]
-
Solution. (a) Total letters = 11. Total ways to place R and E in two different positions = \(11 \times 10 = 110\) (ordered). Number of position pairs with exactly 4 letters between them: If R is in position 1, E must be in position 6. If R in 2, E in 7. If R in 3, E in 8. If R in 4, E in 9. If R in 5, E in 10. If R in 6, E in 11. So 6 pairs. R and E can swap places: \(6 \times 2 = 12\) favourable arrangements.
\[ P = \frac {12}{110} = \frac {6}{55} \]
(b) Total permutations of ’MISSISSIPPI’: Letters: M(1), I(4), S(4), P(2).
\[ \text {Total} = \frac {11!}{4! \, 4! \, 2! \, 1!} \]
Treat 4 S’s as a single block. Then we have: [SSSS], M, I(4), P(2) → total \(1+1+4+2 = 8\) objects, with I repeated 4 times, P repeated 2 times. Arrangements of these 8 objects = \(\frac {8!}{4! \, 2!}\). The block [SSSS] can start at 8 possible positions (1st to 8th). So favourable = \(8 \times \frac {8!}{4! \, 2!}\).
\[ P = \frac {8 \times \frac {8!}{4! \, 2!}}{\frac {11!}{4! \, 4! \, 2!}} = \frac {8 \times 8! \times 4!}{11!} = \frac {4}{165} \]
-
Solution. Real roots \(\Rightarrow b^2 \geq 4ac\). Each of \(a, b, c\) can be 1 to 6. Total outcomes = \(6^3 = 216\). We count favourable cases where \(b^2 \geq 4ac\).
Since maximum \(b^2 = 36\), we need \(4ac \leq 36 \Rightarrow ac \leq 9\).
List all pairs \((a,c)\) with \(a,c \in \{1,\dots ,6\}\) and \(ac \leq 9\):
\[ \begin {array}{c|c|c|c} a & c & ac & \text {Minimum } b \text { such that } b^2 \geq 4ac \\ \hline 1 & 1 & 1 & b \geq 2 \quad (b=2,3,4,5,6) \Rightarrow 5 \text { values} \\ 1 & 2 & 2 & b \geq 3 \quad (3,4,5,6) \Rightarrow 4 \\ 1 & 3 & 3 & b \geq 4 \quad (4,5,6) \Rightarrow 3 \\ 1 & 4 & 4 & b \geq 4 \quad (4,5,6) \Rightarrow 3 \\ 1 & 5 & 5 & b \geq 5 \quad (5,6) \Rightarrow 2 \\ 1 & 6 & 6 & b \geq 5 \quad (5,6) \Rightarrow 2 \\ 2 & 1 & 2 & b \geq 3 \quad (3,4,5,6) \Rightarrow 4 \\ 2 & 2 & 4 & b \geq 4 \quad (4,5,6) \Rightarrow 3 \\ 2 & 3 & 6 & b \geq 5 \quad (5,6) \Rightarrow 2 \\ 2 & 4 & 8 & b \geq 6 \quad (6) \Rightarrow 1 \\ 2 & 5 & 10 & \text {not possible ( }10 > 9\text {)} \\ 2 & 6 & 12 & \text {not possible} \\ 3 & 1 & 3 & b \geq 4 \quad (4,5,6) \Rightarrow 3 \\ 3 & 2 & 6 & b \geq 5 \quad (5,6) \Rightarrow 2 \\ 3 & 3 & 9 & b \geq 6 \quad (6) \Rightarrow 1 \\ 3 & 4 & 12 & \text {not possible} \\ 4 & 1 & 4 & b \geq 4 \quad (4,5,6) \Rightarrow 3 \\ 4 & 2 & 8 & b \geq 6 \quad (6) \Rightarrow 1 \\ 4 & 3 & 12 & \text {not possible} \\ 5 & 1 & 5 & b \geq 5 \quad (5,6) \Rightarrow 2 \\ 5 & 2 & 10 & \text {not possible} \\ 6 & 1 & 6 & b \geq 5 \quad (5,6) \Rightarrow 2 \\ 6 & 2 & 12 & \text {not possible} \end {array} \]
Now sum the number of \(b\) values for each \((a,c)\) pair. Note that \((a,c)\) and \((c,a)\) are different because \(a\) and \(c\) are different coefficients. We count each ordered pair.
Let us count systematically:
- \(a=1\): \(c=1(5),2(4),3(3),4(3),5(2),6(2)\) → sum = \(5+4+3+3+2+2 = 19\) - \(a=2\): \(c=1(4),2(3),3(2),4(1)\) → sum = \(4+3+2+1 = 10\) (c=5,6 not allowed) - \(a=3\): \(c=1(3),2(2),3(1)\) → sum = \(3+2+1 = 6\) (c=4,5,6 not allowed) - \(a=4\): \(c=1(3),2(1)\) → sum = \(3+1 = 4\) (c=3,4,5,6 not allowed) - \(a=5\): \(c=1(2)\) → sum = \(2\) (c=2,3,4,5,6 not allowed) - \(a=6\): \(c=1(2)\) → sum = \(2\) (c=2,3,4,5,6 not allowed)
Total favourable = \(19 + 10 + 6 + 4 + 2 + 2 = 43\).
\[ P = \frac {43}{216} \]
-
Solution. Let \(x \geq 0\), \(y \geq 0\) such that \(x + y = 2n\). The product \(xy\) is maximum when \(x = y = n\), so maximum product = \(n \cdot n = n^2\). We want \(xy \geq \frac {3}{4} n^2\).
Substitute \(y = 2n - x\):
\[ x(2n - x) \geq \frac {3}{4} n^2 \]
\[ 2nx - x^2 \geq \frac {3}{4} n^2 \]
Multiply by 4: \(8nx - 4x^2 \geq 3n^2\)
\[ 4x^2 - 8nx + 3n^2 \leq 0 \]
\[ (2x - 3n)(2x - n) \leq 0 \]
So \(x\) lies between \(\frac {n}{2}\) and \(\frac {3n}{2}\).
Since \(x\) can range from \(0\) to \(2n\), the favourable length = \(\frac {3n}{2} - \frac {n}{2} = n\). Total length = \(2n - 0 = 2n\).
\[ P = \frac {n}{2n} = \frac {1}{2} \]
-
Solution. Total tickets: \(1, 2, 3, \dots , 2n+1\). Total ways to choose 3 tickets = \(\binom {2n+1}{3} = \frac {(2n+1)(2n)(2n-1)}{6} = \frac {n(4n^2-1)}{3}\).
We count the number of 3-term APs possible. Let the middle term be \(m\) and common difference be \(d \geq 1\). Then the three terms are \(m-d\), \(m\), \(m+d\). We need \(1 \leq m-d\) and \(m+d \leq 2n+1\).
For a fixed \(d\), \(m\) can range from \(d+1\) to \(2n+1-d\). Number of possible \(m\) values = \((2n+1-d) - (d+1) + 1 = 2n+1-2d\).
Now sum over \(d = 1\) to \(n\) (since \(m+d \leq 2n+1\) gives \(d \leq n\)):
\(\seteqnumber{0}{1.}{0}\)\begin{align*} \text {Favourable} &= \sum _{d=1}^{n} (2n+1-2d) = \sum _{d=1}^{n} (2n+1) - 2\sum _{d=1}^{n} d \\ &= n(2n+1) - 2 \cdot \frac {n(n+1)}{2} = n(2n+1) - n(n+1) = n^2 \\ P& = \frac {n^2}{\frac {n(4n^2-1)}{3}} = \frac {3n}{4n^2-1} \end{align*}
1.4 Axiomatic Approach to Probability
In this section, we will learn the modern, mathematical approach to probability. This approach was proposed by the Russian mathematician A.N. Kolmogorov in 1933. It includes both the classical and statistical definitions as special cases and overcomes their limitations.
1.4.1 Random Experiment and Sample Space
-
-
1. The sample space serves as the universal set for all questions related to the experiment.
-
2. A sample space is finite if the number of elements is finite, and infinite if the number of elements is infinite.
-
3. A sample space is discrete if its points can be arranged into a sequence (finite or countably infinite). It is continuous if it contains uncountably many points.
-
1.4.2 Examples of Sample Spaces
1.4.3 Event
1.4.4 Algebra of Events
Since events are subsets of \(S\), we can use set operations to create new events.
1.4.5 Laws of Set Theory (for Events)
All the usual laws of set theory apply to events:
-
• Commutative Laws: \(A \cup B = B \cup A\), \(A \cap B = B \cap A\)
-
• Associative Laws: \((A \cup B) \cup C = A \cup (B \cup C)\), \((A \cap B) \cap C = A \cap (B \cap C)\)
-
• Distributive Laws: \(A \cup (B \cap C) = (A \cup B) \cap (A \cup C)\), \(A \cap (B \cup C) = (A \cap B) \cup (A \cap C)\)
-
• De Morgan’s Laws: \(\overline {A \cup B} = \overline {A} \cap \overline {B}\), \(\overline {A \cap B} = \overline {A} \cup \overline {B}\)
1.4.6 Table: Event Descriptions in Set Theory
| Verbal Description | Set Theory Notation |
| At least one of \(A\) or \(B\) occurs | \(A \cup B\) |
| Both \(A\) and \(B\) occur | \(A \cap B\) |
| Neither \(A\) nor \(B\) occurs | \(\overline {A} \cap \overline {B}\) |
| \(A\) occurs but \(B\) does not | \(A \cap \overline {B}\) |
| Exactly one of \(A\) or \(B\) occurs | \(A \triangle B\) |
| If \(A\) occurs then \(B\) occurs | \(A \subset B\) |
| \(A\) and \(B\) are mutually exclusive | \(A \cap B = \phi \) |
| Complementary event of \(A\) | \(\overline {A}\) |
-
Example 1.4.25. \(A, B, C\) are three arbitrary events. Find expressions for the following events: (i) only \(A\) occurs, (ii) both \(A\) and \(B\) occur but not \(C\), (iii) all three occur, (iv) at least one occurs, (v) at least two occur, (vi) exactly one occurs, (vii) exactly two occur, (viii) none occurs.
-
-
1. only \(A \text { occurs} = A \cap \overline {B} \cap \overline {C} \)
-
2. \(A \text { and } B \text { but not } C = A \cap B \cap \overline {C} \)
-
3. all three occur \(= A \cap B \cap C \)
-
4. at least one occurs \(= A \cup B \cup C \)
-
5. at least two occur\(= (A \cap B) \cup (B \cap C) \cup (C \cap A) \)
-
6. exactly one occurs \(= (A \cap \overline {B} \cap \overline {C}) \cup (\overline {A} \cap B \cap \overline {C}) \cup (\overline {A} \cap \overline {B} \cap C) \)
-
7. exactly two occur \(= (A \cap B \cap \overline {C}) \cup (A \cap \overline {B} \cap C) \cup (\overline {A} \cap B \cap C) \)
-
8. none occurs \(= \overline {A} \cap \overline {B} \cap \overline {C}\)
-
1.4.7 Probability Function and Axioms
-
Definition 1.4.27 (Axioms of Probability (Kolmogorov)).
-
1. Axiom of Positiveness: For every event \(A\), \(P(A) \geq 0\).
-
2. Axiom of Certainty: \(P(S) = 1\) (the sample space is certain to occur).
-
3. Axiom of Additivity (Union): If \(\{A_n\}\) is a finite or infinite sequence of mutually exclusive events (pairwise disjoint), then:
\[ P\left (\bigcup _{n} A_n\right ) = \sum _{n} P(A_n) \]
-
1.4.8 Extended Axiom of Addition and Axiom of Continuity
Two additional important properties are equivalent to each other:
-
Proof : We prove both directions.
Part (a): Extended Axiom of Addition \(\Rightarrow \) Axiom of Continuity
Let \(\{B_n\}\) be a decreasing sequence of events such that \(B_1 \supset B_2 \supset B_3 \supset \cdots \) and \(\bigcap _{k=1}^{\infty } B_k = \phi \).
Define \(C_k = B_k - B_{k+1} = B_k \cap \overline {B_{k+1}}\) for \(k = 1, 2, \dots \). These events are pairwise disjoint. Also, for any \(n\):
\[ B_n = \left (\bigcup _{k=n}^{\infty } C_k\right ) \cup \left (\bigcap _{k=n}^{\infty } B_k\right ) \]
Since \(\bigcap _{k=1}^{\infty } B_k = \phi \), we have \(\bigcap _{k=n}^{\infty } B_k = \phi \) for all \(n\).
Thus:
\[ B_n = \bigcup _{k=n}^{\infty } C_k \]
Now, by the extended axiom of addition:
\[ P(B_n) = \sum _{k=n}^{\infty } P(C_k) \]
The series \(\sum _{k=1}^{\infty } P(C_k) = P(B_1) \leq 1\) converges. Hence the tail sum:
\[ \lim _{n \to \infty } P(B_n) = \lim _{n \to \infty } \sum _{k=n}^{\infty } P(C_k) = 0 \]
Part (b): Axiom of Continuity \(\Rightarrow \) Extended Axiom of Addition
Let \(\{A_n\}\) be a sequence of pairwise disjoint events. Define \(B_n = \bigcup _{k=n}^{\infty } A_k\). Then \(B_1 \supset B_2 \supset B_3 \supset \cdots \) and \(\bigcap _{n=1}^{\infty } B_n = \phi \) (since any \(\omega \) belongs to at most finitely many \(A_n\)).
By the axiom of continuity:
\[ \lim _{n \to \infty } P(B_n) = 0 \]
Now, for any \(n\):
\[ \bigcup _{k=1}^{\infty } A_k = \left (\bigcup _{k=1}^{n} A_k\right ) \cup B_{n+1} \]
Since these are disjoint:
\[ P\left (\bigcup _{k=1}^{\infty } A_k\right ) = \sum _{k=1}^{n} P(A_k) + P(B_{n+1}) \]
Taking limit as \(n \to \infty \):
\[ P\left (\bigcup _{k=1}^{\infty } A_k\right ) = \lim _{n \to \infty } \sum _{k=1}^{n} P(A_k) + \lim _{n \to \infty } P(B_{n+1}) = \sum _{k=1}^{\infty } P(A_k) \]
Thus the extended axiom of addition holds. □
1.4.9 Important Theorems on Probability
-
Proof : \(S\) and \(\phi \) are mutually exclusive, and \(S \cup \phi = S\). So:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(S \cup \phi ) &= P(S) \\ P(S) + P(\phi ) &= P(S) \quad \text {(by Axiom 3)} \\ P(\phi ) &= 0 \end{align*} □
-
Proof : \(A\) and \(\overline {A}\) are disjoint, and \(A \cup \overline {A} = S\). So:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A \cup \overline {A}) &= P(A) + P(\overline {A}) = P(S) = 1 \\ \therefore P(\overline {A}) &= 1 - P(A) \end{align*} □
-
Proof : \(B = (A \cap B) \cup (\overline {A} \cap B)\), and these two are disjoint. So:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(B) &= P(A \cap B) + P(\overline {A} \cap B) \\ \therefore P(\overline {A} \cap B) &= P(B) - P(A \cap B) \end{align*} The second formula is proved similarly. □
-
Proof : \(A \cup B = A \cup (\overline {A} \cap B)\), and these are disjoint. So:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A \cup B) &= P(A) + P(\overline {A} \cap B) \\ &= P(A) + [P(B) - P(A \cap B)] \quad \text {(by Theorem 4.4)} \\ &= P(A) + P(B) - P(A \cap B) \end{align*} □
-
Proof : (i) \(A = B \cup (A \cap \overline {B})\) and these are disjoint. So:
\[ P(A) = P(B) + P(A \cap \overline {B}) \Rightarrow P(A \cap \overline {B}) = P(A) - P(B) \]
(ii) Since \(P(A \cap \overline {B}) \geq 0\), we have \(P(A) - P(B) \geq 0\), so \(P(B) \leq P(A)\). □
-
Theorem 1.4.40. For any \(n\) events \(A_1, A_2, \dots , A_n\) defined on a probability space, the probability of their union is given by:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P\left (\bigcup _{i=1}^n A_i\right ) = \sum _{i=1}^n P(A_i) &- \sum _{1 \leq i < j \leq n} P(A_i \cap A_j) \\ &+ \sum _{1 \leq i < j < k \leq n} P(A_i \cap A_j \cap A_k) \\ &- \cdots + (-1)^{n-1} P(A_1 \cap A_2 \cap \cdots \cap A_n) \end{align*}
-
Proof : We will prove this theorem using the principle of mathematical induction.
Step 1: Base Case (\(n = 2\))
For two events \(A_1\) and \(A_2\), the addition theorem (Theorem 4.5) gives:
\[ P(A_1 \cup A_2) = P(A_1) + P(A_2) - P(A_1 \cap A_2) \]
This matches the formula with:
\[ \sum _{i=1}^2 P(A_i) = P(A_1) + P(A_2), \quad \sum _{1 \leq i < j \leq 2} P(A_i \cap A_j) = P(A_1 \cap A_2) \]
and all higher sums are empty (zero). So the formula holds for \(n = 2\).
Step 2: Induction Hypothesis
Assume that the formula holds for any \(r\) events (where \(r \geq 2\)). That is:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P\left (\bigcup _{i=1}^r A_i\right ) = \sum _{i=1}^r P(A_i) &- \sum _{1 \leq i < j \leq r} P(A_i \cap A_j) \\ &+ \sum _{1 \leq i < j < k \leq r} P(A_i \cap A_j \cap A_k) \\ &- \cdots + (-1)^{r-1} P(A_1 \cap A_2 \cap \cdots \cap A_r) \end{align*}
Step 3: Prove for \(n = r + 1\)
Consider \(r+1\) events \(A_1, A_2, \dots , A_r, A_{r+1}\). We can write:
\[ \bigcup _{i=1}^{r+1} A_i = \left (\bigcup _{i=1}^r A_i\right ) \cup A_{r+1} \]
Using the addition theorem for two events (with \(X = \bigcup _{i=1}^r A_i\) and \(Y = A_{r+1}\)), we get:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P\left (\bigcup _{i=1}^{r+1} A_i\right ) &= P\left (\bigcup _{i=1}^r A_i\right ) + P(A_{r+1}) - P\left [\left (\bigcup _{i=1}^r A_i\right ) \cap A_{r+1}\right ] \\ &= P\left (\bigcup _{i=1}^r A_i\right ) + P(A_{r+1}) - P\left [\bigcup _{i=1}^r (A_i \cap A_{r+1})\right ] \quad \text {(Distributive Law)} \end{align*}
Now, apply the induction hypothesis to the first term (with \(r\) events) and also to the last term (with \(r\) events \(A_1 \cap A_{r+1}, A_2 \cap A_{r+1}, \dots , A_r \cap A_{r+1}\)):
First term: \(P\left (\bigcup _{i=1}^r A_i\right )\)
\[ = \sum _{i=1}^r P(A_i) - \sum _{1 \leq i < j \leq r} P(A_i \cap A_j) + \sum _{1 \leq i < j < k \leq r} P(A_i \cap A_j \cap A_k) - \cdots + (-1)^{r-1} P(A_1 \cap \cdots \cap A_r) \]
Last term: \(P\left [\bigcup _{i=1}^r (A_i \cap A_{r+1})\right ]\)
\[ = \sum _{i=1}^r P(A_i \cap A_{r+1}) - \sum _{1 \leq i < j \leq r} P(A_i \cap A_j \cap A_{r+1}) + \sum _{1 \leq i < j < k \leq r} P(A_i \cap A_j \cap A_k \cap A_{r+1}) - \cdots + (-1)^{r-1} P(A_1 \cap A_2 \cap \cdots \cap A_r \cap A_{r+1}) \]
Step 4: Combine the terms
Substitute these into the expression for \(P\left (\bigcup _{i=1}^{r+1} A_i\right )\):
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P\left (\bigcup _{i=1}^{r+1} A_i\right ) &= \left [\sum _{i=1}^r P(A_i) - \sum _{1 \leq i < j \leq r} P(A_i \cap A_j) + \cdots + (-1)^{r-1} P(A_1 \cap \cdots \cap A_r)\right ] \\ &\quad + P(A_{r+1}) \\ &\quad - \left [\sum _{i=1}^r P(A_i \cap A_{r+1}) - \sum _{1 \leq i < j \leq r} P(A_i \cap A_j \cap A_{r+1}) + \cdots + (-1)^{r-1} P(A_1 \cap \cdots \cap A_r \cap A_{r+1})\right ] \end{align*}
Now, collect like terms:
-
• First-order terms (single events):
\[ \sum _{i=1}^r P(A_i) + P(A_{r+1}) = \sum _{i=1}^{r+1} P(A_i) \]
-
• Second-order terms (intersections of two events):
\(\seteqnumber{0}{1.}{0}\)\begin{align*} &- \sum _{1 \leq i < j \leq r} P(A_i \cap A_j) - \sum _{i=1}^r P(A_i \cap A_{r+1}) \\ &= - \sum _{1 \leq i < j \leq r} P(A_i \cap A_j) - \sum _{1 \leq i < r+1} P(A_i \cap A_{r+1}) \\ &= - \sum _{1 \leq i < j \leq r+1} P(A_i \cap A_j) \end{align*}
-
• Third-order terms (intersections of three events):
\(\seteqnumber{0}{1.}{0}\)\begin{align*} &+ \sum _{1 \leq i < j < k \leq r} P(A_i \cap A_j \cap A_k) + \sum _{1 \leq i < j \leq r} P(A_i \cap A_j \cap A_{r+1}) \\ &= + \sum _{1 \leq i < j < k \leq r+1} P(A_i \cap A_j \cap A_k) \end{align*}
-
• This pattern continues. The signs alternate: \(+\) for odd-order intersections, \(-\) for even-order intersections.
-
• The last term (intersection of all \(r+1\) events) will have sign \((-1)^{(r+1)-1} = (-1)^r\):
\[ + (-1)^r P(A_1 \cap A_2 \cap \cdots \cap A_r \cap A_{r+1}) \]
Step 5: Conclusion
Thus, we have shown that if the formula holds for \(r\) events, it also holds for \(r+1\) events. Since the formula holds for \(n = 2\) (base case), by the principle of mathematical induction, it holds for all positive integers \(n \geq 2\).
Therefore,
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P\left (\bigcup _{i=1}^n A_i\right ) = \sum _{i=1}^n P(A_i) &- \sum _{1 \leq i < j \leq n} P(A_i \cap A_j) \\ &+ \sum _{1 \leq i < j < k \leq n} P(A_i \cap A_j \cap A_k) \\ &- \cdots + (-1)^{n-1} P(A_1 \cap A_2 \cap \cdots \cap A_n) \end{align*} This completes the proof. □
-
-
Proof : We prove both inequalities.
Part (a): \(P\left (\bigcup _{i=1}^n A_i\right ) \leq \sum _{i=1}^n P(A_i)\)
We prove this by induction on \(n\).
Base case (\(n = 2\)): From Theorem 4.5, \(P(A_1 \cup A_2) = P(A_1) + P(A_2) - P(A_1 \cap A_2)\). Since \(P(A_1 \cap A_2) \geq 0\), we have:
\[ P(A_1 \cup A_2) \leq P(A_1) + P(A_2) \]
Induction hypothesis: Assume the inequality holds for \(n = r\), i.e.,
\[ P\left (\bigcup _{i=1}^r A_i\right ) \leq \sum _{i=1}^r P(A_i) \]
Induction step: For \(n = r+1\), we have:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P\left (\bigcup _{i=1}^{r+1} A_i\right ) &= P\left (\left (\bigcup _{i=1}^r A_i\right ) \cup A_{r+1}\right ) \\ &\leq P\left (\bigcup _{i=1}^r A_i\right ) + P(A_{r+1}) \quad \text {(by base case with two events)} \\ &\leq \sum _{i=1}^r P(A_i) + P(A_{r+1}) \quad \text {(by induction hypothesis)} \\ &= \sum _{i=1}^{r+1} P(A_i) \end{align*} Thus, by induction, the inequality holds for all \(n\).
Part (b): \(P\left (\bigcap _{i=1}^n A_i\right ) \geq \sum _{i=1}^n P(A_i) - (n-1)\)
Apply the inequality from part (a) to the complementary events \(\overline {A_1}, \overline {A_2}, \dots , \overline {A_n}\):
\[ P\left (\bigcup _{i=1}^n \overline {A_i}\right ) \leq \sum _{i=1}^n P(\overline {A_i}) \]
Now, by De Morgan’s law:
\[ \bigcup _{i=1}^n \overline {A_i} = \overline {\bigcap _{i=1}^n A_i} \]
So:
\[ P\left (\overline {\bigcap _{i=1}^n A_i}\right ) \leq \sum _{i=1}^n [1 - P(A_i)] = n - \sum _{i=1}^n P(A_i) \]
But \(P\left (\overline {\bigcap A_i}\right ) = 1 - P\left (\bigcap _{i=1}^n A_i\right )\). Therefore:
\[ 1 - P\left (\bigcap _{i=1}^n A_i\right ) \leq n - \sum _{i=1}^n P(A_i) \]
Rearranging:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} - P\left (\bigcap _{i=1}^n A_i\right ) &\leq n - \sum _{i=1}^n P(A_i) - 1 \\ - P\left (\bigcap _{i=1}^n A_i\right ) &\leq (n-1) - \sum _{i=1}^n P(A_i) \\ P\left (\bigcap _{i=1}^n A_i\right ) &\geq \sum _{i=1}^n P(A_i) - (n-1) \end{align*} This completes the proof. □
-
Proof :
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A \cup B \mid C) &= \frac {P((A \cup B) \cap C)}{P(C)} = \frac {P((A \cap C) \cup (B \cap C))}{P(C)} \\ &= \frac {P(A \cap C) + P(B \cap C) - P(A \cap B \cap C)}{P(C)} \\ &= \frac {P(A \cap C)}{P(C)} + \frac {P(B \cap C)}{P(C)} - \frac {P(A \cap B \cap C)}{P(C)} \\ &= P(A \mid C) + P(B \mid C) - P(A \cap B \mid C) \end{align*} □
-
Proof :
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A \cap \overline {B} \mid C) + P(A \cap B \mid C) &= \frac {P(A \cap \overline {B} \cap C)}{P(C)} + \frac {P(A \cap B \cap C)}{P(C)} \\ &= \frac {P[(A \cap \overline {B} \cap C) \cup (A \cap B \cap C)]}{P(C)} \\ &= \frac {P(A \cap C)}{P(C)} = P(A \mid C) \end{align*} □
-
Proof : We verify the three axioms:
Axiom 1 (Positiveness): \(P(A \mid B) = \frac {P(A \cap B)}{P(B)} \geq 0\) since \(P(A \cap B) \geq 0\) and \(P(B) > 0\).
Axiom 2 (Certainty): \(P(S \mid B) = \frac {P(S \cap B)}{P(B)} = \frac {P(B)}{P(B)} = 1\).
Axiom 3 (Additivity): If \(\{A_n\}\) are mutually exclusive, then \(\{A_n \cap B\}\) are also mutually exclusive. So:
\[ P\left (\bigcup _{n} A_n \mid B\right ) = \frac {P\left (\bigcup _{n} (A_n \cap B)\right )}{P(B)} = \frac {\sum _n P(A_n \cap B)}{P(B)} = \sum _n P(A_n \mid B) \]
Thus, \(P(\cdot \mid B)\) satisfies all axioms. □
-
Proof : Since \(B \subset C\), we have \(B \cap A \subset C \cap A\). By Theorem 4.6(ii):
\[ P(B \cap A) \leq P(C \cap A) \]
Dividing both sides by \(P(A) > 0\):
\[ \frac {P(B \cap A)}{P(A)} \leq \frac {P(C \cap A)}{P(A)} \Rightarrow P(B \mid A) \leq P(C \mid A) \]
□
1.5 Conditional Probability and Multiplication Law
-
Proof : From the definition of conditional probability:
\[ P(B \mid A) = \frac {P(A \cap B)}{P(A)} \Rightarrow P(A \cap B) = P(A) \cdot P(B \mid A) \]
Similarly, \(P(A \mid B) = \frac {P(A \cap B)}{P(B)} \Rightarrow P(A \cap B) = P(B) \cdot P(A \mid B)\). □
-
Proof : This follows by repeatedly applying the multiplication law. For example, for \(n=3\):
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A_1 \cap A_2 \cap A_3) &= P(A_1) \cdot P(A_2 \cap A_3 \mid A_1) \\ &= P(A_1) \cdot P(A_2 \mid A_1) \cdot P(A_3 \mid A_1 \cap A_2) \end{align*} The general case follows by induction. □
1.5.1 Independent Events
-
Proof :
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A \cap \overline {B}) &= P(A) - P(A \cap B) \\ &= P(A) - P(A)P(B) \quad (\text {since independent}) \\ &= P(A)[1 - P(B)] = P(A)P(\overline {B}) \end{align*} So \(A\) and \(\overline {B}\) are independent. □
-
Proof :
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(\overline {A} \cap \overline {B}) &= P(\overline {A \cup B}) = 1 - P(A \cup B) \\ &= 1 - [P(A) + P(B) - P(A \cap B)] \\ &= 1 - P(A) - P(B) + P(A)P(B) \\ &= [1 - P(A)][1 - P(B)] = P(\overline {A})P(\overline {B}) \end{align*} □
-
Proof : We need to show \(P((A \cup B) \cap C) = P(A \cup B)P(C)\).
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P((A \cup B) \cap C) &= P((A \cap C) \cup (B \cap C)) \\ &= P(A \cap C) + P(B \cap C) - P(A \cap B \cap C) \\ &= P(A)P(C) + P(B)P(C) - P(A)P(B)P(C) \quad \text {(mutual independence)} \\ &= P(C)[P(A) + P(B) - P(A)P(B)] \\ &= P(C)[P(A) + P(B) - P(A \cap B)] \quad \text {(since $A$ and $B$ are independent)} \\ &= P(C) \cdot P(A \cup B) \end{align*} Thus \((A \cup B)\) and \(C\) are independent. □
-
Proof : Since \(A \cap B \subset A \subset A \cup B\), by Theorem 4.6(ii):
\[ P(A \cap B) \leq P(A) \leq P(A \cup B) \]
Also, from Theorem 4.5:
\[ P(A \cup B) = P(A) + P(B) - P(A \cap B) \leq P(A) + P(B) \]
Combining, we get the desired result. □
-
Definition 1.5.11 (Mutual Independence). A set of events \(A_1, A_2, \dots , A_n\) is mutually independent if for every subset \(\{i_1, i_2, \dots , i_k\}\):
\[ P(A_{i_1} \cap A_{i_2} \cap \cdots \cap A_{i_k}) = P(A_{i_1}) P(A_{i_2}) \cdots P(A_{i_k}) \]
The total number of conditions for mutual independence is \(2^n - 1 - n\).
1.5.2 Probability of At Least One of n Independent Events
For \(n\) independent events \(A_1, A_2, \dots , A_n\) with \(P(A_i) = p_i\):
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(\text {at least one occurs}) &= 1 - P(\text {none occurs}) \\ &= 1 - P(\overline {A_1} \cap \overline {A_2} \cap \cdots \cap \overline {A_n}) \\ &= 1 - P(\overline {A_1})P(\overline {A_2}) \cdots P(\overline {A_n}) \\ &= 1 - (1-p_1)(1-p_2)\cdots (1-p_n) \end{align*}
1.5.3 Solved Examples
-
Example 1.5.13. Two dice, one green and one red, are thrown. Let \(A\) be the event that the sum is odd, and \(B\) the event of at least one ace (number 1).
-
1. Describe the complete sample space and events \(A\), \(B\), \(\overline {B}\), \(A \cap B\), \(A \cup B\), \(A \cap \overline {B}\). Find their probabilities.
-
2. Find the probabilities of various combinations.
-
-
Solution. Sample space \(S = \{(i,j): i=1..6, j=1..6\}\), \(n(S)=36\).
\(A = \{(i,j): i+j \text { is odd}\}\). There are 18 such outcomes. So \(P(A)=\frac {18}{36}=\frac 12\).
\(B = \{(i,j): i=1 \text { or } j=1\}\). Count: \(6+6-1=11\). So \(P(B)=\frac {11}{36}\).
\(\overline {B}\) = no ace: \(36-11=25\) outcomes. \(P(\overline {B})=\frac {25}{36}\).
\(A \cap B\) = sum odd and at least one ace:
\[ \{(1,2),(2,1),(1,4),(4,1),(1,6),(6,1)\} \quad \Rightarrow \quad n=6, \quad P=\frac {6}{36}=\frac 16 \]
\(A \cup B\): \(n(A \cup B) = n(A) + n(B) - n(A \cap B) = 18+11-6=23\), \(P=\frac {23}{36}\).
\(A \cap \overline {B}\) = sum odd and no ace: \(n(A) - n(A \cap B) = 18-6=12\), \(P=\frac {12}{36}=\frac 13\).
Now for part (b):
\(\seteqnumber{0}{1.}{0}\)\begin{align*} \text {(i) } P(\overline {A \cup B}) &= 1 - P(A \cup B) = 1 - \frac {23}{36} = \frac {13}{36} \\ \text {(ii) } P(\overline {A \cap B}) &= 1 - P(A \cap B) = 1 - \frac 16 = \frac 56 \\ \text {(iii) } P(\overline {A} \cap B) &= P(B) - P(A \cap B) = \frac {11}{36} - \frac {6}{36} = \frac {5}{36} \\ \text {(iv) } P(A \cap \overline {B}) &= \frac {12}{36} = \frac 13 \\ \text {(v) } P(\overline {A} \cap \overline {B}) &= P(\overline {A \cup B}) = \frac {13}{36} \\ \text {(vi) } P(A \cup \overline {B}) &= P(A) + P(\overline {B}) - P(A \cap \overline {B}) = \frac {18}{36} + \frac {25}{36} - \frac {12}{36} = \frac {31}{36} \\ \text {(vii) } P(\overline {A \cup B}) &= \frac {13}{36} \\ \text {(viii) } P(A \cap (A \cup B)) &= P(A) = \frac 12 \\ \text {(ix) } P(A \cup (A \cap B)) &= P(A) = \frac 12 \\ \text {(x) } P(A \mid B) &= \frac {P(A \cap B)}{P(B)} = \frac {6/36}{11/36} = \frac {6}{11}, \quad P(B \mid A) = \frac {6/36}{18/36} = \frac 13 \\ \text {(xi) } P(A \mid \overline {B}) &= \frac {P(A \cap \overline {B})}{P(\overline {B})} = \frac {12/36}{25/36} = \frac {12}{25}, \quad P(\overline {B} \mid A) = \frac {P(A \cap \overline {B})}{P(A)} = \frac {12/36}{18/36} = \frac {12}{18} = \frac 23 \end{align*}
-
Solution. (a) \(P(S > 8) = P(S=9) + P(S=10) + P(S=11) + P(S=12)\).
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(S=9) &= \frac {4}{36}, \quad P(S=10) = \frac {3}{36}, \quad P(S=11) = \frac {2}{36}, \quad P(S=12) = \frac {1}{36} \\ P(S>8) &= \frac {4+3+2+1}{36} = \frac {10}{36} = \frac {5}{18} \end{align*}
(b) \(P(S=7) = \frac {6}{36} = \frac 16\), \(P(S=11) = \frac {2}{36} = \frac {1}{18}\). Since these are mutually exclusive:
\[ P(\text {neither } 7 \text { nor } 11) = 1 - \left (\frac 16 + \frac {1}{18}\right ) = 1 - \frac {3+1}{18} = 1 - \frac {4}{18} = \frac {14}{18} = \frac 79 \]
-
Solution. Let \(U_1\) = first urn chosen, \(U_2\) = second urn chosen. \(P(U_1)=P(U_2)=\frac 12\).
(i) \(P(2 \text { or } 4) = P(U_1) \cdot P(2\text { or }4 \mid U_1) + P(U_2) \cdot P(2\text { or }4 \mid U_2)\)
\[ = \frac 12 \times \frac {2}{4} + \frac 12 \times \frac {2}{6} = \frac 12 \times \frac 12 + \frac 12 \times \frac 13 = \frac 14 + \frac 16 = \frac {3+2}{12} = \frac {5}{12} \]
(ii) \(P(3) = \frac 12 \times \frac 14 + \frac 12 \times 0 = \frac 18\)
(iii) \(P(1 \text { or } 9) = \frac 12 \times \frac 14 + \frac 12 \times \frac 16 = \frac 18 + \frac {1}{12} = \frac {3+2}{24} = \frac {5}{24}\)
-
Solution. The event can happen in three mutually exclusive ways:
-
• 1R, 1W, 2B
-
• 2R, 1W, 1B
-
• 1R, 2W, 1B
Total ways = \(\binom {15}{4} = 1365\).
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P &= \frac {\binom {6}{1}\binom {4}{1}\binom {5}{2} + \binom {6}{2}\binom {4}{1}\binom {5}{1} + \binom {6}{1}\binom {4}{2}\binom {5}{1}}{1365} \\ &= \frac {6\times 4\times 10 + 15\times 4\times 5 + 6\times 6\times 5}{1365} \\ &= \frac {240 + 300 + 180}{1365} = \frac {720}{1365} = \frac {48}{91} \approx 0.5275 \end{align*}
-
-
Solution. For one die: \(P(\text {6 in one throw}) = \frac 16\), \(P(\text {no 6}) = \frac 56\).
\[ P(\text {at least one 6 in 4 throws}) = 1 - \left (\frac 56\right )^4 = 1 - \frac {625}{1296} = \frac {671}{1296} \approx 0.5177 > 0.5 \]
For two dice: \(P(\text {double 6}) = \frac {1}{36}\), \(P(\text {no double 6}) = \frac {35}{36}\).
\[ P(\text {at least one double 6 in 24 throws}) = 1 - \left (\frac {35}{36}\right )^{24} \approx 0.4914 < 0.5 \]
So the first bet is favourable (probability > 0.5), the second is not.
-
Solution. The problem is solved if at least one solves it.
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(\text {solved}) &= 1 - P(\text {none solves}) \\ &= 1 - P(\overline {A})P(\overline {B})P(\overline {C}) \\ &= 1 - \left (1-\frac 12\right )\left (1-\frac 34\right )\left (1-\frac 14\right ) \\ &= 1 - \left (\frac 12\right )\left (\frac 14\right )\left (\frac 34\right ) = 1 - \frac {3}{32} = \frac {29}{32} \end{align*}
-
Solution. Since \(A \subset A \cup B\), we have \(P(A) \leq P(A \cup B)\). So \(P(A \cup B) \geq \frac 34\).
Also \(P(A \cap B) \leq P(B) = \frac 58\). And:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A \cup B) &\leq 1 \\ P(A) + P(B) - P(A \cap B) &\leq 1 \\ \frac 34 + \frac 58 - P(A \cap B) &\leq 1 \\ \frac {6+5}{8} - P(A \cap B) &\leq 1 \\ \frac {11}{8} - P(A \cap B) &\leq 1 \\ \frac {11}{8} - 1 &\leq P(A \cap B) \\ \frac {3}{8} &\leq P(A \cap B) \end{align*} Thus \(\frac 38 \leq P(A \cap B) \leq \frac 58\).
-
Solution. The probability that a number is divisible by a prime \(r\) is \(1/r\). So the probability that neither is divisible by \(r\) is \((1 - 1/r)^2\). The required probability over all primes is:
\[ P = \prod _{r \text { prime}} \left (1 - \frac {1}{r^2}\right ) = \frac {1}{\zeta (2)} = \frac {6}{\pi ^2} \approx 0.6079 \]
-
Solution. Let \(A\) = first draw gives 4 gold, \(B\) = second draw gives 4 silver.
(i) With replacement: \(A\) and \(B\) are independent.
\[ P(A) = \frac {\binom {10}{4}}{\binom {18}{4}}, \quad P(B) = \frac {\binom {8}{4}}{\binom {18}{4}} \]
\[ P(A \cap B) = \frac {\binom {10}{4}}{\binom {18}{4}} \times \frac {\binom {8}{4}}{\binom {18}{4}} \]
(ii) Without replacement: \(A\) and \(B\) are dependent.
\[ P(A) = \frac {\binom {10}{4}}{\binom {18}{4}}, \quad P(B \mid A) = \frac {\binom {8}{4}}{\binom {14}{4}} \]
\[ P(A \cap B) = \frac {\binom {10}{4}}{\binom {18}{4}} \times \frac {\binom {8}{4}}{\binom {14}{4}} \]
-
Solution. Let \(A\) = exactly 3 defectives in the first 8 examined. Let \(B\) = the 9th examined is defective. We want \(P(A \cap B) = P(A) \cdot P(B \mid A)\).
\[ P(A) = \frac {\binom {4}{3} \binom {11}{5}}{\binom {15}{8}} \]
After 8 draws with 3 defectives, remaining: \(15-8=7\) players, with \(4-3=1\) defective left.
\[ P(B \mid A) = \frac {1}{7} \]
\[ P(A \cap B) = \frac {\binom {4}{3} \binom {11}{5}}{\binom {15}{8}} \times \frac 17 \]
-
Solution. Let \(E_i\) = \(A_i\) dies in a year. \(P(E_i)=p\), \(P(\overline {E_i})=1-p\). Probability that at least one dies = \(1 - (1-p)^n\). By symmetry, the probability that \(A_1\) is the first to die = \(\frac {1}{n}\) (since all are equally likely to be first).
\[ P = \frac {1}{n} \left [1 - (1-p)^n\right ] \]
-
Solution. \(P(X) = \frac {6}{14} = \frac {3}{7}\), \(P(Y) = \frac {14}{30} = \frac {7}{15}\). (i) \(P(\overline {X} \cap \overline {Y}) = P(\overline {X})P(\overline {Y}) = \left (1-\frac {3}{7}\right )\left (1-\frac {7}{15}\right ) = \frac {4}{7} \times \frac {8}{15} = \frac {32}{105}\). (ii) \(P(\text {settled}) = 1 - P(\overline {X} \cap \overline {Y}) = 1 - \frac {32}{105} = \frac {73}{105}\).
-
Solution. \(P(X) = \frac {3}{5}\), \(P(\overline {X}) = \frac {2}{5}\), \(P(Y) = \frac {5}{8}\), \(P(\overline {Y}) = \frac {3}{8}\). They contradict if one tells truth and the other lies:
\[ P = P(X \cap \overline {Y}) + P(\overline {X} \cap Y) = \frac {3}{5} \times \frac {3}{8} + \frac {2}{5} \times \frac {5}{8} = \frac {9}{40} + \frac {10}{40} = \frac {19}{40} = 0.475 = 47.5\% \]
-
Solution. Sum 9 can occur as: \((3,6), (6,3), (4,5), (5,4)\).
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P &= P(3)P(6) + P(6)P(3) + P(4)P(5) + P(5)P(4) \\ &= 2\left [\frac {1-k}{6} \cdot \frac {1+k}{6}\right ] + 2\left [\frac {1+k}{6} \cdot \frac {1-2k}{6}\right ] \\ &= \frac {2(1-k^2)}{36} + \frac {2(1 - k - 2k^2)}{36} \\ &= \frac {2(1-k^2 + 1 - k - 2k^2)}{36} = \frac {2(2 - k - 3k^2)}{36} = \frac {2 - k - 3k^2}{18} \end{align*}
-
Example 1.5.29. (a) \(A\) and \(B\) alternately cut a pack of cards, shuffling after each cut. \(A\) starts. Find their respective chances of first cutting a diamond.
(b) Three guns fire independently. \(P(E_1)=0.5\), \(P(E_2)=0.6\), \(P(E_3)=0.8\). Find probability that (i) exactly one hit, (ii) at least two hits.
-
Solution. (a) \(P(\text {diamond}) = \frac {13}{52} = \frac 14\), \(P(\text {not diamond}) = \frac 34\). \(A\) wins if he gets diamond on his first turn, or both miss and he gets on second, etc.
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A\text { wins}) &= \frac 14 + \left (\frac 34\right )^2\frac 14 + \left (\frac 34\right )^4\frac 14 + \cdots \\ &= \frac 14 \left [1 + \left (\frac 34\right )^2 + \left (\frac 34\right )^4 + \cdots \right ] = \frac 14 \cdot \frac {1}{1 - \frac {9}{16}} = \frac 14 \cdot \frac {1}{\frac {7}{16}} = \frac 14 \cdot \frac {16}{7} = \frac {4}{7} \\ P(B\text { wins}) &= 1 - \frac {4}{7} = \frac {3}{7} \end{align*}
(b) \(P(E_1)=0.5\), \(P(E_2)=0.6\), \(P(E_3)=0.8\). Let \(\overline {E_1}=0.5\), \(\overline {E_2}=0.4\), \(\overline {E_3}=0.2\). (i) Exactly one hit:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P &= P(E_1 \overline {E_2} \overline {E_3}) + P(\overline {E_1} E_2 \overline {E_3}) + P(\overline {E_1} \overline {E_2} E_3) \\ &= 0.5\times 0.4\times 0.2 + 0.5\times 0.6\times 0.2 + 0.5\times 0.4\times 0.8 \\ &= 0.04 + 0.06 + 0.16 = 0.26 \end{align*} (ii) At least two hits:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P &= P(E_1 E_2 \overline {E_3}) + P(E_1 \overline {E_2} E_3) + P(\overline {E_1} E_2 E_3) + P(E_1 E_2 E_3) \\ &= 0.5\times 0.6\times 0.2 + 0.5\times 0.4\times 0.8 + 0.5\times 0.6\times 0.8 + 0.5\times 0.6\times 0.8 \\ &= 0.06 + 0.16 + 0.24 + 0.24 = 0.70 \end{align*}
-
Solution. The event can happen in three ways:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(\text {G,B,B}) &= \frac 34 \times \frac 24 \times \frac 34 = \frac {9}{32} \\ P(\text {B,G,B}) &= \frac 14 \times \frac 24 \times \frac 34 = \frac {3}{32} \\ P(\text {B,B,G}) &= \frac 14 \times \frac 24 \times \frac 14 = \frac {1}{32} \\ \text {Total} &= \frac {9+3+1}{32} = \frac {13}{32} \end{align*}
1.6 Bayes’ Theorem
In this section, we will learn about Bayes’ Theorem, which is one of the most important results in probability theory. It allows us to revise our probabilities when we get new information. This theorem was discovered by the Reverend Thomas Bayes (1701-1761) and published after his death.
Often, we know the probabilities of certain events, and then we observe that some event has occurred. We want to find the probability that a particular cause was responsible. Bayes’ Theorem provides a way to calculate these posterior probabilities from the prior probabilities.
1.6.1 Statement of Bayes’ Theorem
-
Theorem 1.6.1 (Bayes’ Theorem). Let \(E_1, E_2, \dots , E_n\) be mutually disjoint (pairwise mutually exclusive) events such that:
\[ P(E_i) \neq 0 \quad \text {for } i = 1, 2, \dots , n \]
and let \(A\) be any arbitrary event which is a subset of \(\bigcup _{i=1}^n E_i\) such that \(P(A) > 0\). Then for any \(i = 1, 2, \dots , n\):
\[ P(E_i \mid A) = \frac {P(E_i) \cdot P(A \mid E_i)}{\displaystyle \sum _{j=1}^n P(E_j) \cdot P(A \mid E_j)} \]
-
Proof : Since \(E_1, E_2, \dots , E_n\) are mutually disjoint and \(A \subset \bigcup _{i=1}^n E_i\), we can write:
\[ A = A \cap \left (\bigcup _{i=1}^n E_i\right ) = \bigcup _{i=1}^n (A \cap E_i) \]
Since the events \(A \cap E_i\) are also mutually disjoint, by the addition theorem of probability (Axiom 3), we get:
\[ P(A) = \sum _{i=1}^n P(A \cap E_i) \]
Now, using the multiplication law of probability:
\[ P(A \cap E_i) = P(E_i) \cdot P(A \mid E_i) \]
Therefore:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A) &= \sum _{i=1}^n P(E_i) \cdot P(A \mid E_i) \end{align*}
Also, by the definition of conditional probability:
\[ P(E_i \mid A) = \frac {P(A \cap E_i)}{P(A)} = \frac {P(E_i) \cdot P(A \mid E_i)}{P(A)} \]
Substituting the expression for \(P(A)\), we get:
\[ P(E_i \mid A) = \frac {P(E_i) \cdot P(A \mid E_i)}{\displaystyle \sum _{j=1}^n P(E_j) \cdot P(A \mid E_j)} \]
This completes the proof. □
1.6.2 Important Terminology
1.6.3 Total Probability Theorem
Before stating Bayes’ Theorem, we often use the following important result:
-
Proof : Since \(E_1, E_2, \dots , E_n\) form a partition of \(S\), we have:
\[ A = A \cap S = A \cap \left (\bigcup _{i=1}^n E_i\right ) = \bigcup _{i=1}^n (A \cap E_i) \]
The events \(A \cap E_i\) are mutually disjoint because the \(E_i\)’s are disjoint. Hence:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A) &= \sum _{i=1}^n P(A \cap E_i) \\ &= \sum _{i=1}^n P(E_i) \cdot P(A \mid E_i) \quad \text {(by multiplication law)} \end{align*} □
1.6.4 Bayes’ Theorem for Future Events
-
Corollary 1.6.6 (Bayes’ Theorem for Future Events). Let \(E_1, E_2, \dots , E_n\) be mutually disjoint events with \(P(E_i) \neq 0\) for all \(i\). Suppose that after event \(A\) has occurred, we want the probability of another event \(C\) (a future event). Then:
\[ P(C \mid A) = \frac {\displaystyle \sum _{i=1}^n P(E_i) \cdot P(A \mid E_i) \cdot P(C \mid E_i \cap A)}{\displaystyle \sum _{i=1}^n P(E_i) \cdot P(A \mid E_i)} \]
If \(C\) is independent of \(A\) given \(E_i\), i.e., \(P(C \mid E_i \cap A) = P(C \mid E_i)\), then:
\[ P(C \mid A) = \frac {\displaystyle \sum _{i=1}^n P(E_i) \cdot P(A \mid E_i) \cdot P(C \mid E_i)}{\displaystyle \sum _{i=1}^n P(E_i) \cdot P(A \mid E_i)} \]
1.6.5 Solved Examples
-
Example 1.6.7. In 1989 there were three candidates for the position of principal - Mr. Chatterji, Mr. Aiyangar and Dr. Singh - whose chances of getting the appointment are in the proportion \(4:2:3\) respectively. The probability that Mr. Chatterji if selected would introduce co-education in the college is \(0.3\). The probabilities of Mr. Aiyangar and Dr. Singh doing the same are respectively \(0.5\) and \(0.8\). What is the probability that there was co-education in the college in 1990?
-
Solution. Let us define the following events:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} E_1 &: \text {Mr. Chatterji is selected as principal} \\ E_2 &: \text {Mr. Aiyangar is selected as principal} \\ E_3 &: \text {Dr. Singh is selected as principal} \\ A &: \text {Introduction of co-education} \end{align*}
We are given:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(E_1) &= \frac {4}{4+2+3} = \frac {4}{9}, \quad P(E_2) = \frac {2}{9}, \quad P(E_3) = \frac {3}{9} = \frac {1}{3} \\ P(A \mid E_1) &= 0.3 = \frac {3}{10}, \quad P(A \mid E_2) = 0.5 = \frac {5}{10}, \quad P(A \mid E_3) = 0.8 = \frac {8}{10} \end{align*}
We want \(P(A)\). Since \(E_1, E_2, E_3\) form a partition of the sample space, by the theorem of total probability:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(A) &= P(E_1)P(A \mid E_1) + P(E_2)P(A \mid E_2) + P(E_3)P(A \mid E_3) \\ &= \frac {4}{9} \times \frac {3}{10} + \frac {2}{9} \times \frac {5}{10} + \frac {3}{9} \times \frac {8}{10} \\ &= \frac {12}{90} + \frac {10}{90} + \frac {24}{90} \\ &= \frac {46}{90} = \frac {23}{45} \end{align*} Thus, the probability that co-education was introduced in 1990 is \(\dfrac {23}{45}\).
-
Example 1.6.8. The contents of urns I, II and III are as follows:
-
• Urn I: 1 white, 2 black and 3 red balls
-
• Urn II: 2 white, 1 black and 1 red balls
-
• Urn III: 4 white, 5 black and 3 red balls
One urn is chosen at random and two balls are drawn. They happen to be white and red. What is the probability that they come from urns I, II or III?
-
-
Solution. Let us define the following events:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} E_1 &: \text {Urn I is chosen} \\ E_2 &: \text {Urn II is chosen} \\ E_3 &: \text {Urn III is chosen} \\ A &: \text {The two balls drawn are white and red} \end{align*}
Since one urn is chosen at random:
\[ P(E_1) = P(E_2) = P(E_3) = \frac {1}{3} \]
Now we compute the likelihoods \(P(A \mid E_i)\):
For Urn I: Total balls = \(1+2+3 = 6\). Number of ways to choose 2 balls = \(\binom {6}{2} = 15\). Number of ways to choose 1 white and 1 red = \(\binom {1}{1} \times \binom {3}{1} = 1 \times 3 = 3\).
\[ P(A \mid E_1) = \frac {3}{15} = \frac {1}{5} \]
For Urn II: Total balls = \(2+1+1 = 4\). Number of ways to choose 2 balls = \(\binom {4}{2} = 6\). Number of ways to choose 1 white and 1 red = \(\binom {2}{1} \times \binom {1}{1} = 2 \times 1 = 2\).
\[ P(A \mid E_2) = \frac {2}{6} = \frac {1}{3} \]
For Urn III: Total balls = \(4+5+3 = 12\). Number of ways to choose 2 balls = \(\binom {12}{2} = 66\). Number of ways to choose 1 white and 1 red = \(\binom {4}{1} \times \binom {3}{1} = 4 \times 3 = 12\).
\[ P(A \mid E_3) = \frac {12}{66} = \frac {2}{11} \]
Now, by Bayes’ Theorem:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(E_1 \mid A) &= \frac {P(E_1)P(A \mid E_1)}{P(E_1)P(A \mid E_1) + P(E_2)P(A \mid E_2) + P(E_3)P(A \mid E_3)} \\ &= \frac {\frac {1}{3} \times \frac {1}{5}}{\frac {1}{3} \times \frac {1}{5} + \frac {1}{3} \times \frac {1}{3} + \frac {1}{3} \times \frac {2}{11}} \\ &= \frac {\frac {1}{15}}{\frac {1}{15} + \frac {1}{9} + \frac {2}{33}} \end{align*}
Find the common denominator of \(15, 9, 33\). LCM = \(495\).
\(\seteqnumber{0}{1.}{0}\)\begin{align*} \frac {1}{15} &= \frac {33}{495}, \quad \frac {1}{9} = \frac {55}{495}, \quad \frac {2}{33} = \frac {30}{495} \\ \text {Denominator} &= \frac {33+55+30}{495} = \frac {118}{495} \\ P(E_1 \mid A) &= \frac {33/495}{118/495} = \frac {33}{118} \end{align*}
Similarly:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(E_2 \mid A) &= \frac {\frac {1}{3} \times \frac {1}{3}}{\frac {118}{495}} = \frac {\frac {1}{9}}{\frac {118}{495}} = \frac {1}{9} \times \frac {495}{118} = \frac {55}{118} \\ P(E_3 \mid A) &= \frac {\frac {1}{3} \times \frac {2}{11}}{\frac {118}{495}} = \frac {\frac {2}{33}}{\frac {118}{495}} = \frac {2}{33} \times \frac {495}{118} = \frac {30}{118} \end{align*}
We can verify that \(P(E_1 \mid A) + P(E_2 \mid A) + P(E_3 \mid A) = \frac {33+55+30}{118} = \frac {118}{118} = 1\).
Thus:
\[ P(E_1 \mid A) = \frac {33}{118}, \quad P(E_2 \mid A) = \frac {55}{118}, \quad P(E_3 \mid A) = \frac {30}{118} \]
-
Example 1.6.9. In answering a question on a multiple choice test, a student either knows the answer or he guesses. Let \(p\) be the probability that he knows the answer and \(1-p\) the probability that he guesses. Assume that a student who guesses at the answer will be correct with probability \(1/m\), where \(m\) is the number of multiple-choice alternatives. What is the conditional probability that a student knew the answer to a question given that he answered it correctly?
-
Solution. Let us define the following events:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} E_1 &: \text {The student knew the right answer} \\ E_2 &: \text {The student guesses the answer} \\ A &: \text {The student gets the right answer} \end{align*}
We are given:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(E_1) &= p, \quad P(E_2) = 1-p \\ P(A \mid E_1) &= 1 \quad (\text {if he knows, he is always correct}) \\ P(A \mid E_2) &= \frac {1}{m} \quad (\text {if he guesses, probability of correct is } 1/m) \end{align*}
We want \(P(E_1 \mid A)\). By Bayes’ Theorem:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(E_1 \mid A) &= \frac {P(E_1) \cdot P(A \mid E_1)}{P(E_1)P(A \mid E_1) + P(E_2)P(A \mid E_2)} \\ &= \frac {p \times 1}{p \times 1 + (1-p) \times \frac {1}{m}} \\ &= \frac {p}{p + \frac {1-p}{m}} \\ &= \frac {p}{\frac {mp + 1 - p}{m}} \\ &= \frac {mp}{mp + 1 - p} \\ &= \frac {mp}{1 + (m-1)p} \end{align*}
Thus, the probability that the student knew the answer given that he answered correctly is \(\dfrac {mp}{1 + (m-1)p}\).
-
Example 1.6.10. In a bolt factory, machines \(A\), \(B\) and \(C\) manufacture respectively \(25\%\), \(35\%\) and \(40\%\) of the total output. Of their output, \(5\%\), \(4\%\), \(2\%\) are defective bolts. A bolt is drawn at random from the product and is found to be defective. What are the probabilities that it was manufactured by machines \(A\), \(B\) and \(C\)?
-
Solution. Let us define the following events:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} E_1 &: \text {Bolt is manufactured by machine } A \\ E_2 &: \text {Bolt is manufactured by machine } B \\ E_3 &: \text {Bolt is manufactured by machine } C \\ D &: \text {Bolt is defective} \end{align*}
We are given:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(E_1) &= 0.25 = \frac {25}{100}, \quad P(E_2) = 0.35 = \frac {35}{100}, \quad P(E_3) = 0.40 = \frac {40}{100} \\ P(D \mid E_1) &= 0.05 = \frac {5}{100}, \quad P(D \mid E_2) = 0.04 = \frac {4}{100}, \quad P(D \mid E_3) = 0.02 = \frac {2}{100} \end{align*}
First, we compute \(P(D)\) using the theorem of total probability:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(D) &= P(E_1)P(D \mid E_1) + P(E_2)P(D \mid E_2) + P(E_3)P(D \mid E_3) \\ &= \frac {25}{100} \times \frac {5}{100} + \frac {35}{100} \times \frac {4}{100} + \frac {40}{100} \times \frac {2}{100} \\ &= \frac {125}{10000} + \frac {140}{10000} + \frac {80}{10000} \\ &= \frac {345}{10000} = \frac {69}{2000} \end{align*}
Now, by Bayes’ Theorem:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(E_1 \mid D) &= \frac {P(E_1)P(D \mid E_1)}{P(D)} = \frac {\frac {25}{100} \times \frac {5}{100}}{\frac {345}{10000}} = \frac {\frac {125}{10000}}{\frac {345}{10000}} = \frac {125}{345} = \frac {25}{69} \\[6pt] P(E_2 \mid D) &= \frac {P(E_2)P(D \mid E_2)}{P(D)} = \frac {\frac {35}{100} \times \frac {4}{100}}{\frac {345}{10000}} = \frac {\frac {140}{10000}}{\frac {345}{10000}} = \frac {140}{345} = \frac {28}{69} \\[6pt] P(E_3 \mid D) &= \frac {P(E_3)P(D \mid E_3)}{P(D)} = \frac {\frac {40}{100} \times \frac {2}{100}}{\frac {345}{10000}} = \frac {\frac {80}{10000}}{\frac {345}{10000}} = \frac {80}{345} = \frac {16}{69} \end{align*}
We can verify that \(P(E_1 \mid D) + P(E_2 \mid D) + P(E_3 \mid D) = \frac {25+28+16}{69} = \frac {69}{69} = 1\).
Thus:
\(\seteqnumber{0}{1.}{0}\)\begin{align*} P(\text {from machine } A \mid \text {defective})& = \frac {25}{69}, \\ P(\text {from machine } B \mid \text {defective}) & = \frac {28}{69}, \\ P(\text {from machine } C \mid \text {defective}) &= \frac {16}{69} \end{align*}
Note that even though machine \(C\) produces the most bolts (\(40\%\)), it has the lowest probability of being the source of a defective bolt because its defect rate is the lowest (\(2\%\)).
0 Comments