Fisher’s exact test is a statistical significant test often used in the analysis of \(2\times2\) contingency table. Based on the marginal sums, the Fisher’s eact test calculates the association between 2 binary factors as a value \(p\). The null hypothesis is that these factors are independent, whereas the research hypothesis is that they are not independent. If \(p<\alpha\), the null hypothesis is rejected and vice versa. The Fisher’s exact test has no assumption of the chi-square test, because it is not based on the chi-square distribution.
Agresti (1990) was interested in whether a British woman as she claimed can truly tell the pouring order tea and milk into a cup. To test, she was given 8 cups of milk tea, in four of which milk was added first. The null hypothesis is that there is no association between this woman’s guess and the true order, while the research hypothesis is that there is a positive association. Since the research hypothesis is that there is a positive association, we use one-tailed test by setting up the argument alternative=“greater” in the function fisher.test( ). The result shows that \(p=.2429\) and we cannot reject the null hypothesis.
dta<-matrix(c(3,1,1,3),2,2,dimnames=list(Guess=c("Milk","Tea"),
Truth=c("Milk","Tea")))
dta## Truth
## Guess Milk Tea
## Milk 3 1
## Tea 1 3fisher.test(dta,alternative="greater")##
## Fisher's Exact Test for Count Data
##
## data: dta
## p-value = 0.2429
## alternative hypothesis: true odds ratio is greater than 1
## 95 percent confidence interval:
## 0.3135693 Inf
## sample estimates:
## odds ratio
## 6.408309For chi-square test, it is required that the observed frequency in each cell should be at least 5. For a given sample size, there are often a limited number of different contingency tables that you could obtain and thus a limited number of different values of chi-square. If only a few different values of \(\chi^2\) are possible, then the \(\chi^2\) distribution, which is continuous, cannot provide a reasonable approximation to the distribution of our statistic, which is discrete. In this case with a \(2\times2\) contingent table, you can substitute Fisher’s exact test for chi-square test.
An alternative approach to analyzing categorical data is based on likelihood ratios. For large sample sizes, the chi-square test is equivalent to the likelihood ratio test. Likelihood ratio tests are heavily used in log-linear models for analyzing contingency tables because of their additive properties. The general idea of a likelihood ratio can be described quite simply. Suppose we collect data and calculate the probability or likelihood of the data occurring given that the null hypothesis is true. We also calculate the likelihood that the data would occur under some alternative hypothesis. If the data are much more likely for some alternative hypothesis than for \(H_o\), we would be inclined to reject \(H_o\). However, if the data are almost as likely under \(H_o\) as they are for some other alternatives, we would be inclined to retain \(H_o\). Thus, the likelihood ratio (the ratio of these two likelihoods) forms a basis for evaluating the null hypothesis.
For the one-dimensional goodness-of-fit test,
\(\chi^2_{C-1}=2\sum O_i ln(\frac{O_i}{E_i})\),
where \(O_i\) and \(E_i\) are the observed and expected frequencies for each cell.
For analyzing contingency tables, we can use essentially the same formula,
\(\chi^2_{(R-1)(C-1)}=2\sum O_{ij} ln(\frac{O_{ij}}{E_{ij}})\),
where \(O_{ij}\) and \(E_{ij}\) are the observed and expected frequencies for each cell of the \(i^{th}\) row and the \(j^{th}\) column. This statistic is evaluated with respect to the \(\chi^2\) distribution on \((R-1)(C-1)\) degrees of freedom. See the below table for the death sentenced case.
| Defendant’s Race | Yes | No | Total | |
|---|---|---|---|---|
| Nonwhite | 33 | 251 | 284 | |
| White | 33 | 508 | 541 | |
| Total | 66 | 759 | 825 |
Applying this formula to these frequency data to get the likelihood ratio value, which is 7.358. The probability for a value larger than it is .0059, which is far smaller than .05. Thus, the null hypothesis is rejected.
counts<-matrix(c(33,33,251,508),2,2)
rows<-rowSums(counts)
cols<-colSums(counts)
e.counts<-sapply(1:2,function(i){
sapply(1:2,function(j){
return(rows[j]*cols[i]/sum(counts))
})
})
lrvalue<-2*sum(counts*log(counts/e.counts))
1-pchisq(lrvalue,df=1)## [1] 0.006676647There are many situations in which we wish to control for a third variable Z (e.g., environments), while we are looking at the relationship between X (e.g., stress; high/low) and Y (e.g., mental status; normal/disturbed). The Mantel-Haenszel statistic or called Cochran-Mantel-Haenszel statistic is designed to deal with just these situations.
In 1973 the University of California at Berkeley investigated gender discrimination in graduate admission. A superficial examination of admissions for that year revealed that approximately 45% of male applicants were admitted compared with only about 30% of female applicants. On the surface, this would appear to be a clear case of gender discrimination. However, graduate admissions are made by departments, not by a university admission office, and it is appropriate and necessary to look at admissions data at the departmental level. The data of graduate admissions (i.e., Table 6.9) are listed below.
| Major | Male | Female | ||
|---|---|---|---|---|
| Depts | Admit | Reject | Admit | Reject |
| A | 512 | 313 | 89 | 19 |
| B | 333 | 207 | 17 | 8 |
| C | 120 | 205 | 202 | 391 |
| D | 138 | 279 | 131 | 244 |
| E | 53 | 138 | 94 | 299 |
| F | 22 | 351 | 24 | 317 |
| Total B-F | 686 | 1180 | 468 | 1259 |
| % of Total B-F | 36.8% | 63.2% | 28.8% | 71.2% |
If Department A is excluded, there were more males admitted than females (36.8% vs. 28.8%). The \(\chi^2\) test for these data shows a significant association between the admission outcome and the gender, \(\chi^2_{(1)}=37.97\), \(p<.01\).
dd<-matrix(c(686,1180,468,1259),2,2)
chisq.test(dd)##
## Pearson's Chi-squared test with Yates' continuity correction
##
## data: dd
## X-squared = 37.979, df = 1, p-value = 7.151e-10However, if we break the data down by departments, we see that in three of these departments (C, E, and F) women were admitted at ta higher rate, and in the remaining two the differences in favor of men were quite small. The Mantel-Haenszel statistic is designed to deal with the data from each department separately. We then sum the results across departments. For each department, there is a \(2\times2\) contingency table (i.e., gender \(\times\) admission result). The computation of the Mantel-Haenszel statistic is based on the fact that for any \(2\times 2\) table, the entry in any one cell, given the marginal totals, determines the entry in every other cell. This means that we can create a statistic using only the data in cell\(_{11}\) of the table for each department. The Mantel-Haenszel statistic can be computed as
\(M^2=\frac{(|\sum O_{11,k}-\sum E_{11,k}|-0.5)^2}{\sum n_{1+,k}n_{2+,k}n_{+1,k}n_{+2,k}/(n^2_{++,k}(n_{++,k}-1))}\).
The numerator is simply the difference between the observed counts and the expected counts in the entry\(_{11}\) of the \(k^{th}\) table (for the \(k^{th}\) department). The denominator is the sum of the variances of \(k\) tables. The variance of each table is the product of the 4 marginal sums (two row marginal sums and two column marginal sums) divided by \(N^2(N-1)\), where \(N\) is the total sum of the \(k^{th}\) table. This statistic \(M^2\) can be evaluated as a chi-square on 1 \(df\). The below codes are used to declare a 3-dimensional matrix (or called array) to store the graduate admission data, and to compute the observed and expected counts in each cell\(_{11}\).
# Declare an array to store all data
all.dta<-array(c(512,313,89,19,
353,207,17,8,
120,205,202,391,
138,279,131,244,
53,138,94,299,
22,351,24,317),c(2,2,6))
all.dta## , , 1
##
## [,1] [,2]
## [1,] 512 89
## [2,] 313 19
##
## , , 2
##
## [,1] [,2]
## [1,] 353 17
## [2,] 207 8
##
## , , 3
##
## [,1] [,2]
## [1,] 120 202
## [2,] 205 391
##
## , , 4
##
## [,1] [,2]
## [1,] 138 131
## [2,] 279 244
##
## , , 5
##
## [,1] [,2]
## [1,] 53 94
## [2,] 138 299
##
## , , 6
##
## [,1] [,2]
## [1,] 22 24
## [2,] 351 317# Compute the expected value of cell11
Obs<-all.dta[1,1,]
Eps<-sapply(1:6,function(k){
temp<-all.dta[,,k]
return((colSums(temp)[1]*rowSums(temp)[1])/sum(temp))
})Now we compute the variance for each cell\(_{11}\). The below shows exactly the same content of Table 6.10.
variances<-sapply(1:6,function(k){
temp<-all.dta[,,k]
numerator<-prod(c(colSums(temp),rowSums(temp)))
denominator<-sum(temp)^2*(sum(temp)-1)
return(numerator/denominator)
})
cbind(Obs,Eps,variances)## Obs Eps variances
## [1,] 512 531.43087 21.913465
## [2,] 353 354.18803 5.572417
## [3,] 120 113.99782 47.861410
## [4,] 138 141.63258 44.339873
## [5,] 53 48.07705 24.251044
## [6,] 22 24.03081 10.752552We can compute the Mantel-Haenszel statistic \(M^2\). The \(p\) value is larger than .05. Thus, we cannot reject \(H_o\) that there is no gender discrimination on graduate admission.
n.m<-(abs(sum(Obs)-sum(Eps))-0.5)^2
d.m<-sum(variances)
m<-n.m/d.m
1-pchisq(m,1)## [1] 0.2322635The frequency distribution of all cells in Department A is different from other departments. However, the assumption of the Mantel_Haensel statistic is that departments are homogeneous with respect to the pattern of admissions. Thus, we compute \(M^2\) without Department A.
n.m2<-(abs(sum(Obs[-1])-sum(Eps[-1]))-0.5)^2
d.m2<-sum(variances[-1])
m2<-n.m2/d.m2
1-pchisq(m2,1)## [1] 0.7564553Back to the case of UC Berkeley, what is wrong with the data? If you calculate the admission rate for each department, you will find Department A and Department B have an admission rate over 50%. These departments happen to have more male applicants, thus creating a false association between gender and admission descision.