Fisher's Exact Test

Fisher's exact test is a statistical test used to determine if there are nonrandom associations between two categorical variables.

Let there exist two such variables  and , with  and  observed states, respectively. Now form an  matrix in which the entries  represent the number of observations in which  and . Calculate the row and column sums  and , respectively, and the total sum

(1)

of the matrix. Then calculate the conditional probability of getting the actual matrix given the particular row and column sums, given by

(2)

which is a multivariate generalization of the hypergeometric probability function. Now find all possible matrices of nonnegative integers consistent with the row and column sums  and . For each one, calculate the associated conditional probability using (2), where the sum of these probabilities must be 1.

To compute the P-value of the test, the tables must then be ordered by some criterion that measures dependence, and those tables that represent equal or greater deviation from independence than the observed table are the ones whose probabilities are added together. There are a variety of criteria that can be used to measure dependence. In the  case, which is the one Fisher looked at when he developed the exact test, either the Pearson chi-square or the difference in proportions (which are equivalent) is typically used. Other measures of association, such as the likelihood-ratio-test, -squared, or any of the other measures typically used for association in contingency tables, can also be used.

The test is most commonly applied to  matrices, and is computationally unwieldy for large  or . For tables larger than , the difference in proportion can no longer be used, but the other measures mentioned above remain applicable (and in practice, the Pearson statistic is most often used to order the tables). In the case of the  matrix, the P-value of the test can be simply computed by the sum of all -values which are .

For an example application of the  test, let  be a journal, say either Mathematics Magazine or Science, and let  be the number of articles on the topics of mathematics and biology appearing in a given issue of one of these journals. If Mathematics Magazine has five articles on math and one on biology, and Science has none on math and four on biology, then the relevant matrix would be

(3)

Computing  gives

(4)

and the other possible matrices and their s are

			(5)
			(6)
			(7)
			(8)

which indeed sum to 1, as required. The sum of -values less than or equal to  is then 0.0476 which, because it is less than 0.05, is significant. Therefore, in this case, there would be a statistically significant association between the journal and type of article appearing.