Mathematical Statistics
(TANSCHE Syllabus)
Madurai Kamaraj University
II- B.Sc., Mathematics, III - Semester
Contents
5.1 Chi-Square Distribution
5.2 Derivation of Chi-Square Distribution
5.3 Moment Generating Function of Chi-Square Distribution
5.3.1 Cumulant Generating Function of Chi-Square Distribution
5.3.2 Limiting Form of Chi-Square Distribution for Large Degrees of Freedom
5.3.3 Mode and Skewness of Chi-Square Distribution
5.3.4 Additive Property of Chi-Square Variates
5.3.5 Chi-Square Probability Curve
5.3.6 Summary of Key Results for Chi-Square Distribution
5.3.7 Important Remarks
5.4 Student’s t-Distribution
5.4.1 Derivation of Student’s t-Distribution
5.4.2 Fisher’s Definition of t
5.4.3 Distribution of Fisher’s t
5.4.4 Constants of t-Distribution
5.4.5 Limiting Form of t-Distribution
5.4.6 Graph of t-Distribution
5.4.7 Critical Values of t
5.4.8 Applications of t-Distribution
5.4.9 t-Test for Single Mean
5.4.10 t-Test for Difference of Means
5.4.11 Paired t-Test
5.4.12 Summary of Key Results for t-Distribution
5.4.13 Important Remarks
5.5 F-Distribution
5.5.1 Derivation of Snedecor’s F-Distribution
5.5.2 Constants of F-Distribution
5.5.3 Mode and Points of Inflexion of F-Distribution
5.5.4 Important Results on F-Distribution
5.5.5 Applications of F-Distribution
5.5.6 F-Test for Equality of Population Variances
5.5.7 Critical Values of F-Distribution
5.5.8 Summary of Key Results for F-Distribution
5.5.9 Important Remarks
Chapter 5
Tests of Significance for Small Samples
Syllabus
Using the chi-square distribution - Student’s t- distribution - F- distribution
Chapter 13 sections 13-13.5, Chapter 14 sections 14-14.2.10 and Sections 14.5 – 14.5.5
5.1 Chi-Square Distribution
The Chi-square distribution (pronounced as "Ki-square") is one of the most important distributions in statistics. It is widely used for testing hypotheses, especially when dealing with categorical data. The chi-square distribution was first discovered by Karl Pearson in 1900.
5.2 Derivation of Chi-Square Distribution
Method of Moment Generating Function
Let \( X_i, (i = 1, 2, \dots , n) \) be independent \( N(\mu _i, \sigma _i^2) \) variates. We want the distribution of: \[ \chi ^2 = \sum _{i=1}^{n} \left ( \frac {X_i - \mu _i}{\sigma _i} \right )^2 = \sum _{i=1}^{n} U_i^2 \] where \( U_i = \frac {X_i - \mu _i}{\sigma _i} \sim N(0,1) \). Since \( X_i \)’s are independent, \( U_i \)’s are also independent. Therefore: \[ M_{\chi ^2}(t) = M_{\sum U_i^2}(t) = \prod _{i=1}^{n} M_{U_i^2}(t) = [M_{U_i^2}(t)]^n \] Now: \begin {align*} M_{U_i^2}(t) &= E(e^{t U_i^2}) \\ &= \int _{-\infty }^{\infty } e^{t u_i^2} \frac {1}{\sqrt {2\pi }} e^{-u_i^2/2} du_i \\ &= \frac {1}{\sqrt {2\pi }} \int _{-\infty }^{\infty } e^{-\frac {1}{2}(1-2t)u_i^2} du_i \\ &= (1 - 2t)^{-1/2} \end {align*} Therefore: \[ M_{\chi ^2}(t) = (1 - 2t)^{-n/2} \] This is the m.g.f. of a Gamma variate with parameters \( \frac {1}{2} \) and \( \frac {n}{2} \). Hence, by the uniqueness theorem of m.g.f.’s: \[ \chi ^2 = \sum _{i=1}^{n} \left ( \frac {X_i - \mu _i}{\sigma _i} \right )^2 \] is a Gamma variate with parameters \( \frac {1}{2} \) and \( \frac {n}{2} \). Thus, the p.d.f. of chi-square distribution with \( n \) degrees of freedom is: \[ f(\chi ^2) = \frac {1}{2^{n/2} \Gamma (n/2)} e^{-\chi ^2/2} (\chi ^2)^{(n/2)-1}, \quad 0 \leq \chi ^2 < \infty \]
- 1.
- If \( X_i; i = 1, 2, \dots , n \) are \( n \) independent normal variates with mean \( \mu _i \) and S.D. \( \sigma _i \), then \( \sum _{i=1}^{n} \left ( \frac {X_i - \mu _i}{\sigma _i} \right )^2 \) is a chi-square variate with \( n \) d.f.
- 2.
- Normal distribution is a particular case of chi-square distribution when \( n = 1 \).
5.3 Moment Generating Function of Chi-Square Distribution
Let \( X \sim \chi ^2_{(n)} \). Then: \[ M_X(t) = E(e^{tX}) = \int _0^{\infty } e^{tx} f(x) \, dx \] \begin {align*} M_X(t) &= \frac {1}{2^{n/2} \Gamma (n/2)} \int _0^{\infty } e^{tx} e^{-x/2} x^{(n/2)-1} \, dx \\ &= \frac {1}{2^{n/2} \Gamma (n/2)} \int _0^{\infty } e^{-\left ( \frac {1-2t}{2} \right )x} x^{(n/2)-1} \, dx \\ &= \frac {1}{2^{n/2} \Gamma (n/2)} \cdot \frac {\Gamma (n/2)}{\left ( \frac {1-2t}{2} \right )^{n/2}} \\ &= (1 - 2t)^{-n/2}, \quad |t| < \frac {1}{2} \end {align*}
5.3.1 Cumulant Generating Function of Chi-Square Distribution
If \( X \sim \chi ^2_{(n)} \), then: \[ K_X(t) = \log M_X(t) = -\frac {n}{2} \log (1 - 2t) \] Expanding: \[ K_X(t) = \frac {n}{2} \left [ 2t + \frac {(2t)^2}{2} + \frac {(2t)^3}{3} + \frac {(2t)^4}{4} + \cdots \right ] \] Comparing coefficients: \begin {align*} \kappa _1 &= n \quad \text {(Mean)} \\ \kappa _2 &= 2n \quad \text {(Variance)} \\ \kappa _3 &= 8n \\ \kappa _4 &= 48n \end {align*} In general: \[ \kappa _r = n \cdot 2^{r-1} (r-1)! \] Therefore: \begin {align*} \text {Mean} &= n \\ \text {Variance} &= 2n \\ \beta _1 &= \frac {\mu _3^2}{\mu _2^3} = \frac {(8n)^2}{(2n)^3} = \frac {8}{n} \\ \beta _2 &= \frac {\mu _4}{\mu _2^2} = \frac {48n + 12n^2}{4n^2} = 3 + \frac {12}{n} \end {align*}
5.3.2 Limiting Form of Chi-Square Distribution for Large Degrees of Freedom
If \( X \sim \chi ^2_{(n)} \), then \( M_X(t) = (1 - 2t)^{-n/2} \). Let \( Z = \frac {X - \mu }{\sigma } \) be the standard chi-square variate, where \( \mu = n \) and \( \sigma ^2 = 2n \). Then: \[ M_Z(t) = e^{-\mu t / \sigma } M_X(t/\sigma ) = e^{-nt/\sqrt {2n}} (1 - 2t/\sqrt {2n})^{-n/2} \] Taking logarithm: \begin {align*} K_Z(t) &= \log M_Z(t) \\ &= -t\sqrt {\frac {n}{2}} - \frac {n}{2} \log \left ( 1 - t\sqrt {\frac {2}{n}} \right ) \\ &= -t\sqrt {\frac {n}{2}} + \frac {n}{2} \left [ t\sqrt {\frac {2}{n}} + \frac {t^2}{2} \cdot \frac {2}{n} + O(n^{-3/2}) \right ] \\ &= \frac {t^2}{2} + O(n^{-1/2}) \end {align*} Taking the limit as \( n \to \infty \): \[ \lim _{n \to \infty } M_Z(t) = e^{t^2/2} \] This is the m.g.f. of the standard normal distribution. Hence, as \( n \to \infty \), the chi-square distribution tends to the normal distribution.
5.3.3 Mode and Skewness of Chi-Square Distribution
The p.d.f. of chi-square distribution is: \[ f(x) = \frac {1}{2^{n/2} \Gamma (n/2)} e^{-x/2} x^{(n/2)-1}, \quad 0 \leq x < \infty \] Taking logarithm: \[ \log f(x) = \text {constant} - \frac {x}{2} + \left ( \frac {n}{2} - 1 \right ) \log x \] Differentiating: \[ \frac {f'(x)}{f(x)} = -\frac {1}{2} + \frac {n/2 - 1}{x} = \frac {n - 2 - x}{2x} \] For the mode, set \( f'(x) = 0 \): \[ x = n - 2 \] Thus, the mode of the chi-square distribution with \( n \) d.f. is \( (n - 2) \). Karl Pearson’s Coefficient of Skewness: \[ \text {Skewness} = \frac {\text {Mean} - \text {Mode}}{\text {S.D.}} = \frac {n - (n-2)}{\sqrt {2n}} = \sqrt {\frac {2}{n}} \] Since the coefficient of skewness is positive, the chi-square distribution is positively skewed. As \( n \) increases, the skewness decreases, and the distribution tends to symmetry.
5.3.4 Additive Property of Chi-Square Variates
- 1.
- The converse of the additive property is also true.
- 2.
- Another useful version: If \( X \) and \( Y \) are independent non-negative variates such that \( X + Y \) follows chi-square with \( n_1 + n_2 \) d.f., and \( X \) is chi-square with \( n_1 \) d.f., then \( Y \) is chi-square with \( n_2 \) d.f.
5.3.5 Chi-Square Probability Curve
The chi-square probability curve has the following characteristics: From the derivative: \[ f'(x) = \frac {n - 2 - x}{2x} f(x) \]
- 1.
- For \( n = 1 \) and \( n = 2 \), the curve is monotonically decreasing.
- 2.
-
For \( n > 2 \):
- \( f'(x) > 0 \) for \( 0 < x < n-2 \) (curve rises)
- \( f'(x) < 0 \) for \( x > n-2 \) (curve falls)
- 3.
- The curve attains its maximum at \( x = n-2 \).
- 4.
- The curve is positively skewed.
- 5.
- The \( x \)-axis is an asymptote to the curve.
5.3.6 Summary of Key Results for Chi-Square Distribution
| Property | Result |
| Probability Density Function | \( f(x) = \dfrac {1}{2^{n/2} \Gamma (n/2)} e^{-x/2} x^{(n/2)-1}, \quad 0 \leq x < \infty \) |
| Mean | \( n \) |
| Variance | \( 2n \) |
| Mode | \( n - 2 \text { (for } n \geq 2\text {)} \) |
| Moment Generating Function | \( M_X(t) = (1 - 2t)^{-n/2}, \quad |t| < \frac {1}{2} \) |
| Cumulants | \( \kappa _r = n \cdot 2^{r-1} (r-1)! \) |
| Skewness | \( \beta _1 = \dfrac {8}{n} \) |
| Kurtosis | \( \beta _2 = 3 + \dfrac {12}{n} \) |
| Additive Property | \( \chi ^2_{n_1} + \chi ^2_{n_2} = \chi ^2_{n_1 + n_2} \) |
| Limiting Form | As \( n \to \infty \), tends to normal distribution |
| Test Statistic | \( \chi ^2 = \displaystyle \sum _{i=1}^{k} \dfrac {(O_i - E_i)^2}{E_i} \) |
| Degrees of Freedom for Goodness of Fit | \( k - 1 \) (or \( k - p - 1 \) if parameters are estimated) |
5.3.7 Important Remarks
- 1.
- The chi-square test is a non-parametric test because it does not make any assumptions about the population parameters.
- 2.
- The test is always a one-tailed test because the chi-square distribution is positively skewed and the critical region lies in the right tail.
- 3.
- When applying the chi-square test, the total of observed frequencies must equal the total of expected frequencies.
- 4.
- The degrees of freedom are reduced by the number of parameters estimated from the data.
- 5.
-
For the validity of the chi-square test:
- The sample observations should be independent.
- The total frequency \( N \) should be reasonably large (usually greater than 50).
- No theoretical cell frequency should be less than 5.
- If any cell has expected frequency less than 5, it should be pooled with adjacent cells.
5.4 Student’s t-Distribution
The entire large sample theory was based on the application of the "Normal Test" (cf. Section 12.9). However, if the sample size \( n \) is small, the distribution of various statistics, e.g., \( Z = \frac {\bar {x} - \mu }{\sigma / \sqrt {n}} \) or \( Z = \frac {X - nP}{\sqrt {nPQ}} \), etc., are far from normality and as such the ’normal test’ cannot be applied if \( n \) is small. In such cases, exact sample tests, pioneered by W.S. Gosset (1908) who wrote under the pen name of Student, and later developed and extended by Prof. R.A. Fisher (1926), are used.
- 1.
- A statistic \( t \) following Student’s t-distribution with \( n \) d.f. is denoted as \( t \sim t_n \).
- 2.
- If we take \( \nu = 1 \), the t-distribution reduces to the standard Cauchy distribution: \[ f(t) = \frac {1}{\pi (1+t^2)}, \quad -\infty < t < \infty \]
- 3.
- The t-distribution is symmetric about zero.
5.4.1 Derivation of Student’s t-Distribution
The expression for \( t \) can be rewritten as: \[ \frac {t^2}{n-1} = \frac {(\bar {x} - \mu )^2 / (\sigma ^2/n)}{(n s^2 / \sigma ^2)} \] Since \( x_i \)’s are from a normal population: \[ \bar {x} \sim N(\mu , \sigma ^2/n) \Rightarrow \frac {(\bar {x} - \mu )}{\sigma /\sqrt {n}} \sim N(0,1) \] Thus, \( \frac {(\bar {x} - \mu )^2}{\sigma ^2/n} \) is a chi-square variate with 1 d.f. Also, \( \frac {n s^2}{\sigma ^2} \) is a chi-square variate with \( (n-1) \) d.f. (by Theorem 13.5). Since \( \bar {x} \) and \( s^2 \) are independently distributed, \( \frac {t^2}{n-1} \) is the ratio of two independent chi-square variates with 1 and \( (n-1) \) d.f. respectively. Hence, it is a \( \beta _2\left (\frac {1}{2}, \frac {n-1}{2}\right ) \) variate. Therefore, the p.d.f. of \( t \) is: \[ f(t) = \frac {1}{\sqrt {\nu } B\left (\frac {1}{2}, \frac {\nu }{2}\right )} \cdot \frac {1}{\left [ 1 + \frac {t^2}{\nu } \right ]^{(\nu +1)/2}}, \quad -\infty < t < \infty \]
5.4.2 Fisher’s Definition of t
5.4.3 Distribution of Fisher’s t
Since \( \xi \) and \( \chi ^2 \) are independent, their joint probability differential is: \[ dF(\xi , \chi ^2) = \frac {1}{\sqrt {2\pi }} e^{-\xi ^2/2} \cdot \frac {e^{-\chi ^2/2} (\chi ^2)^{(n/2)-1}}{2^{n/2} \Gamma (n/2)} d\xi d\chi ^2 \] Let: \[ t = \frac {\xi }{\sqrt {\chi ^2/n}}, \quad u = \chi ^2 \] Then \( \xi = t\sqrt {u/n} \). The Jacobian is: \[ |J| = \sqrt {\frac {u}{n}} \] The joint distribution becomes: \[ dG(t,u) = \frac {1}{\sqrt {2\pi } \, 2^{n/2} \Gamma (n/2) \sqrt {n}} e^{-u(1+t^2/n)/2} u^{(n-1)/2} du \, dt \] Integrating over \( u \): \[ dG_1(t) = \frac {1}{\sqrt {n} B\left (\frac {1}{2}, \frac {n}{2}\right )} \cdot \frac {dt}{\left [ 1 + \frac {t^2}{n} \right ]^{(n+1)/2}}, \quad -\infty < t < \infty \] This is the same as the p.d.f. of Student’s t-distribution with \( n \) d.f.
5.4.4 Constants of t-Distribution
Since \( f(t) \) is symmetric about \( t = 0 \), all moments of odd order vanish: \[ \mu _{2r+1} = 0, \quad r = 0, 1, 2, \dots \] In particular, the mean is 0. Hence central moments coincide with moments about origin. The moments of even order are given by: \[ \mu _{2r} = n^r \frac {(2r-1)(2r-3)\cdots 3 \cdot 1}{(n-2)(n-4)\cdots (n-2r)}, \quad n > 2r \] In particular: \[ \mu _2 = \frac {n}{n-2}, \quad n > 2 \] \[ \mu _4 = \frac {3n^2}{(n-2)(n-4)}, \quad n > 4 \] Thus: \[ \beta _1 = 0 \quad \text {and} \quad \beta _2 = \frac {3(n-2)}{(n-4)} \]
- 1.
- As \( n \to \infty \), \( \beta _1 = 0 \) and \( \beta _2 = 3 \), so the t-distribution tends to the normal distribution.
- 2.
- The moment generating function of the t-distribution does not exist because moments of order \( 2r \geq n \) do not exist.
5.4.5 Limiting Form of t-Distribution
As \( n \to \infty \), the p.d.f. of the t-distribution tends to the standard normal p.d.f.: \[ f(t) = \frac {1}{\sqrt {2\pi }} e^{-t^2/2}, \quad -\infty < t < \infty \]
5.4.6 Graph of t-Distribution
The t-distribution has the following characteristics:
- 1.
- It is symmetrical about \( t = 0 \).
- 2.
- As \( t \) increases, \( f(t) \) decreases rapidly and tends to zero as \( t \to \infty \).
- 3.
- The \( t \)-axis is an asymptote to the curve.
- 4.
- For small \( n \), the tails of the t-distribution have greater probability than the tails of the standard normal distribution.
- 5.
- As \( n \to \infty \), the t-distribution tends to the standard normal distribution.
5.4.7 Critical Values of t
The critical values of \( t \) at level of significance \( \alpha \) and d.f. \( \nu \) for a two-tailed test are given by: \[ P(|t| > t_\nu (\alpha )) = \alpha \] Since the t-distribution is symmetric: \[ P(t > t_\nu (\alpha )) = \alpha /2 \] \[ P(t > t_\nu (2\alpha )) = \alpha \] Thus, the significant value of \( t \) for a single-tailed test at level \( \alpha \) is the same as the value for a two-tailed test at level \( 2\alpha \).
5.4.8 Applications of t-Distribution
The t-distribution has many applications in statistics:
- 1.
- To test if the sample mean differs significantly from a hypothetical population mean.
- 2.
- To test the significance of the difference between two sample means.
- 3.
- To test the significance of an observed sample correlation coefficient.
- 4.
- To test the significance of regression coefficients.
5.4.9 t-Test for Single Mean
Suppose we want to test whether a random sample \( x_i, (i = 1, 2, \dots , n) \) has been drawn from a normal population with a specified mean \( \mu _0 \). Under the null hypothesis \( H_0: \mu = \mu _0 \), the statistic: \[ t = \frac {\bar {x} - \mu _0}{S / \sqrt {n}} \] follows Student’s t-distribution with \( (n-1) \) d.f. where: \[ S^2 = \frac {1}{n-1} \sum _{i=1}^{n} (x_i - \bar {x})^2 \]
| \( X \) | \( d = X - 90 \) | \( d^2 \) |
| 70 | -20 | 400 |
| 120 | 30 | 900 |
| 110 | 20 | 400 |
| 101 | 11 | 121 |
| 88 | -2 | 4 |
| 83 | -7 | 49 |
| 95 | 5 | 25 |
| 98 | 8 | 64 |
| 107 | 17 | 289 |
| 100 | 10 | 100 |
| Total | 72 | 2352 |
5.4.10 t-Test for Difference of Means
Suppose we want to test if two independent samples \( x_i \) and \( y_j \) have been drawn from two normal populations with means \( \mu _X \) and \( \mu _Y \). Under \( H_0: \mu _X = \mu _Y \) and assuming \( \sigma _X^2 = \sigma _Y^2 = \sigma ^2 \), the statistic: \[ t = \frac {\bar {x} - \bar {y}}{S \sqrt {\frac {1}{n_1} + \frac {1}{n_2}}} \] follows Student’s t-distribution with \( (n_1 + n_2 - 2) \) d.f. where: \[ S^2 = \frac {1}{n_1 + n_2 - 2} \left [ \sum (x_i - \bar {x})^2 + \sum (y_j - \bar {y})^2 \right ] \]
- 1.
- Parent populations are normal.
- 2.
- Population variances are equal and unknown.
- 3.
- Samples are random and independent.
5.4.11 Paired t-Test
When the sample sizes are equal and observations are paired, we use the paired t-test. Let \( d_i = x_i - y_i \) be the differences. Under \( H_0: \mu _d = 0 \), the statistic: \[ t = \frac {\bar {d}}{S/\sqrt {n}} \] follows Student’s t-distribution with \( (n-1) \) d.f. where: \[ \bar {d} = \frac {1}{n} \sum d_i, \quad S^2 = \frac {1}{n-1} \sum (d_i - \bar {d})^2 \]
| Pig number | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | Total |
| Food A | 49 | 53 | 51 | 52 | 47 | 50 | 52 | 53 | 407 |
| Food B | 52 | 55 | 52 | 53 | 50 | 54 | 54 | 53 | 423 |
5.4.12 Summary of Key Results for t-Distribution
| Property | Result |
| Probability Density Function | \( f(t) = \dfrac {1}{\sqrt {\nu } B\left (\frac {1}{2}, \frac {\nu }{2}\right )} \cdot \dfrac {1}{\left [ 1 + \frac {t^2}{\nu } \right ]^{(\nu +1)/2}}, \quad -\infty < t < \infty \) |
| Mean | \( 0 \) |
| Variance | \( \dfrac {\nu }{\nu - 2}, \quad \nu > 2 \) |
| Skewness | \( \beta _1 = 0 \) |
| Kurtosis | \( \beta _2 = \dfrac {3(\nu - 2)}{(\nu - 4)}, \quad \nu > 4 \) |
| Moment Generating Function | Does not exist |
| Limiting Form | As \( \nu \to \infty \), tends to \( N(0,1) \) |
| Test Statistic (Single Mean) | \( t = \dfrac {\bar {x} - \mu }{S/\sqrt {n}} \sim t_{(n-1)} \) |
| Test Statistic (Difference of Means) | \( t = \dfrac {\bar {x} - \bar {y}}{S\sqrt {\frac {1}{n_1} + \frac {1}{n_2}}} \sim t_{(n_1+n_2-2)} \) |
| Test Statistic (Paired Observations) | \( t = \dfrac {\bar {d}}{S/\sqrt {n}} \sim t_{(n-1)} \) |
5.4.13 Important Remarks
- 1.
- The t-test is used when the sample size is small (usually \( n < 30 \)).
- 2.
- The basic assumption of the t-test is that the parent population is normal.
- 3.
- The t-distribution is symmetric and has heavier tails than the normal distribution.
- 4.
- For single-tailed tests, use the critical value at \( 2\alpha \) from the two-tailed tables.
- 5.
- The sample variance \( S^2 \) is an unbiased estimate of the population variance \( \sigma ^2 \).
- 6.
- Before applying the t-test for difference of means, it is desirable to test the equality of variances using the F-test.
5.5 F-Distribution
The F-distribution was discovered by Sir Ronald A. Fisher and is named after him. It plays a very important role in statistical inference, particularly in the analysis of variance (ANOVA) and in testing the equality of variances from two normal populations.
- 1.
- The sampling distribution of \( F \) does not involve any population parameters and depends only on the degrees of freedom \( \nu _1 \) and \( \nu _2 \).
- 2.
- A statistic \( F \) following Snedecor’s F-distribution with \( (\nu _1, \nu _2) \) d.f. is denoted as \( F \sim F(\nu _1, \nu _2) \).
- 3.
- \( \nu _1 \) is called the numerator degrees of freedom and \( \nu _2 \) is called the denominator degrees of freedom.
5.5.1 Derivation of Snedecor’s F-Distribution
Since \( X \) and \( Y \) are independent chi-square variates with \( \nu _1 \) and \( \nu _2 \) d.f. respectively, their joint probability differential is: \[ dF(x,y) = \frac {1}{2^{(\nu _1+\nu _2)/2} \Gamma (\nu _1/2) \Gamma (\nu _2/2)} e^{-(x+y)/2} x^{(\nu _1/2)-1} y^{(\nu _2/2)-1} dx \, dy \] for \( 0 \leq x, y < \infty \). Let us make the following transformation: \[ F = \frac {x / \nu _1}{y / \nu _2}, \quad u = y \] Then: \[ x = \frac {\nu _1}{\nu _2} F u, \quad y = u \] The Jacobian of transformation is: \[ |J| = \frac {\nu _1}{\nu _2} u \] The joint distribution of the transformed variables is: \[ dG(F,u) = \frac {(\nu _1/\nu _2)^{\nu _1/2}}{2^{(\nu _1+\nu _2)/2} \Gamma (\nu _1/2) \Gamma (\nu _2/2)} e^{-u(1+\nu _1 F/\nu _2)/2} u^{(\nu _1+\nu _2)/2-1} F^{(\nu _1/2)-1} du \, dF \] Integrating out \( u \) over the range 0 to \( \infty \): \[ g(F) = \frac {(\nu _1/\nu _2)^{\nu _1/2}}{B(\nu _1/2, \nu _2/2)} \cdot \frac {F^{(\nu _1/2)-1}}{\left [ 1 + \frac {\nu _1}{\nu _2} F \right ]^{(\nu _1+\nu _2)/2}}, \quad 0 \leq F < \infty \] This is the probability density function of the F-distribution with \( (\nu _1, \nu _2) \) degrees of freedom.
5.5.2 Constants of F-Distribution
The \( r \)-th moment about origin of the F-distribution is: \[ \mu _r' = E(F^r) = \left ( \frac {\nu _2}{\nu _1} \right )^r \frac {\Gamma \left ( r + \frac {\nu _1}{2} \right ) \Gamma \left ( \frac {\nu _2}{2} - r \right )}{\Gamma \left ( \frac {\nu _1}{2} \right ) \Gamma \left ( \frac {\nu _2}{2} \right )}, \quad \nu _2 > 2r \] In particular: \[ \mu _1' = \frac {\nu _2}{\nu _2 - 2}, \quad \nu _2 > 2 \] Thus, the mean of the F-distribution is independent of \( \nu _1 \). \[ \mu _2' = \frac {\nu _2^2(\nu _1 + 2)}{\nu _1(\nu _2 - 2)(\nu _2 - 4)}, \quad \nu _2 > 4 \] \[ \mu _2 = \mu _2' - (\mu _1')^2 = \frac {2\nu _2^2(\nu _1 + \nu _2 - 2)}{\nu _1(\nu _2 - 2)^2(\nu _2 - 4)}, \quad \nu _2 > 4 \]
5.5.3 Mode and Points of Inflexion of F-Distribution
Taking logarithm of the p.d.f.: \[ \log f(F) = C + \left ( \frac {\nu _1}{2} - 1 \right ) \log F - \frac {\nu _1 + \nu _2}{2} \log \left ( 1 + \frac {\nu _1}{\nu _2} F \right ) \] Differentiating and setting to zero: \[ \frac {\nu _1 - 2}{2F} - \frac {\nu _1(\nu _1 + \nu _2)}{2(\nu _2 + \nu _1 F)} = 0 \] \[ F = \frac {\nu _2(\nu _1 - 2)}{\nu _1(\nu _2 + 2)} \] Thus, the mode of the F-distribution is: \[ \text {Mode} = \frac {\nu _2(\nu _1 - 2)}{\nu _1(\nu _2 + 2)} \]
- 1.
- Since \( F > 0 \), the mode exists if and only if \( \nu _1 > 2 \).
- 2.
- The mode is always less than unity.
- 3.
- The points of inflexion of the F-distribution exist when \( \nu _1 > 4 \) and are equidistant from the mode.
- 4.
- The F-distribution is positively skewed since the mean is greater than the mode.
5.5.4 Important Results on F-Distribution
Thus, the \((1-\alpha )\) significant points of \( F(n_2, n_1) \) distribution are the reciprocals of the \(\alpha \)-significant points of \( F(n_1, n_2) \) distribution.
5.5.5 Applications of F-Distribution
The F-distribution has the following applications in statistical theory:
- 1.
- To test the equality of population variances (F-test for homogeneity of variances).
- 2.
- In analysis of variance (ANOVA) to test the equality of several population means.
- 3.
- In regression analysis to test the overall significance of the regression model.
- 4.
- To test the significance of the multiple correlation coefficient.
5.5.6 F-Test for Equality of Population Variances
Suppose we want to test:
- 1.
- Whether two independent samples have been drawn from normal populations with the same variance \( \sigma ^2 \).
- 2.
- Whether two independent estimates of the population variance are homogeneous or not.
Under the null hypothesis \( H_0: \sigma _X^2 = \sigma _Y^2 = \sigma ^2 \), the statistic: \[ F = \frac {S_X^2}{S_Y^2} \] follows Snedecor’s F-distribution with \( (n_1-1, n_2-1) \) degrees of freedom. where: \[ S_X^2 = \frac {1}{n_1-1} \sum _{i=1}^{n_1} (x_i - \bar {x})^2, \quad S_Y^2 = \frac {1}{n_2-1} \sum _{j=1}^{n_2} (y_j - \bar {y})^2 \]
- 1.
- The greater of the two variances should be placed in the numerator.
- 2.
- \( n_1 \) corresponds to the greater variance.
- 3.
- The F-test assumes that the populations are normal.
5.5.7 Critical Values of F-Distribution
The available F-tables give the critical values for the right-tailed test. The significant value \( F_\alpha (n_1, n_2) \) at level of significance \( \alpha \) and \( (n_1, n_2) \) d.f. is determined by: \[ P[F > F_\alpha (n_1, n_2)] = \alpha \] The reciprocal relation between upper and lower significant points: \[ F_\alpha (n_1, n_2) = \frac {1}{F_{1-\alpha }(n_2, n_1)} \]
| Sample | Size | Sample Mean | Sum of squares of deviations from the mean |
| 1 | 10 | 15 | 90 |
| 2 | 12 | 14 | 108 |
- 1.
- Equality of population means
- 2.
- Equality of population variances
5.5.8 Summary of Key Results for F-Distribution
| Property | Result |
| Probability Density Function | \( f(F) = \dfrac {(\nu _1/\nu _2)^{\nu _1/2}}{B(\nu _1/2, \nu _2/2)} \cdot \dfrac {F^{(\nu _1/2)-1}}{\left [ 1 + \frac {\nu _1}{\nu _2} F \right ]^{(\nu _1+\nu _2)/2}}, \quad 0 \leq F < \infty \) |
| Mean | \( \dfrac {\nu _2}{\nu _2 - 2}, \quad \nu _2 > 2 \) |
| Variance | \( \dfrac {2\nu _2^2(\nu _1 + \nu _2 - 2)}{\nu _1(\nu _2 - 2)^2(\nu _2 - 4)}, \quad \nu _2 > 4 \) |
| Mode | \( \dfrac {\nu _2(\nu _1 - 2)}{\nu _1(\nu _2 + 2)}, \quad \nu _1 > 2 \) |
| Reciprocal Property | \( \dfrac {1}{F} \sim F(\nu _2, \nu _1) \) |
| Test Statistic (Variance Ratio) | \( F = \dfrac {S_1^2}{S_2^2} \sim F(n_1-1, n_2-1) \) |
| Critical Values | \( F_\alpha (\nu _1, \nu _2) = \dfrac {1}{F_{1-\alpha }(\nu _2, \nu _1)} \) |
5.5.9 Important Remarks
- 1.
- The F-test is used to test the equality of variances from two normal populations.
- 2.
- The greater variance should always be placed in the numerator.
- 3.
- The F-distribution is positively skewed.
- 4.
- The F-test is a one-tailed test for testing equality of variances.
- 5.
- For a two-tailed test, use \( F > F_{\alpha /2} \) and \( F < F_{1-\alpha /2} \).
- 6.
- Before applying the t-test for difference of means, the F-test should be used to test the equality of variances.
0 Comments