Fisher's Exact Test

Fisher's exact test takes every table arrangement that could arise once the row and column totals of a 2x2 table are held fixed and calculates the probability of each from the hypergeometric distribution. The p value is then the sum of the probabilities of arrangements as extreme as the observed one or more so, which means nothing is approximated by a continuous distribution and the probability is counted directly. That construction is what makes the test the preferred choice when totals are small or when one or two cells hold very few people: there is no approximation left to deteriorate. As a by-product the odds ratio is reported, and it carries the direction and the magnitude of the association. The direction of the test can be set, with two-sided as the default and the one-sided options reserved for cases where the expected direction was fixed in advance. YouReply Analyze builds the table from row-level data and stops with the actual table size stated whenever the table is not 2x2.

• In a pilot-scale trial, does one of two invitation wordings raise the rate of survey completion?
• Is a rarely chosen response option ticked at different rates by two participant groups?
• In a small sample of firms, is holding a certificate related to whether the firm exports?
• Among focus group participants, does noticing a new feature vary with how long they have used the app?

• When the table is 2x2, because each of the two variables carries exactly two levels.
• When the total number of observations stays small or a cell's expected count falls below the familiar threshold of 5.
• When the expected frequency table returned by the chi-square test leaves you doubting the approximation.
• When you want the divergence between the groups reported as an odds ratio, which this method supplies directly.
• When your theoretical expectation points one way and you wrote it down beforehand, so one of the one-sided options is defensible.

• First variable: a column with exactly two levels, such as a condition or group indicator.
• Second variable: again a two-level column, usually the field marking whether the outcome occurred.
• Whether the two levels are stored as text labels or coded as 0 and 1 makes no difference; what counts is the number of levels.
• One row per participant, since two records for the same person break the independence the table relies on.
• If one of the variables carries more than two categories, it has to be reduced to two levels with a derived column on the Variable tab before the test will run.

Independence of observations

All four cells must fill from separate participants. A matched design in which the same person is measured under two conditions calls for a different test.

A 2x2 table

The exact calculation is defined over a two-row, two-column arrangement whose row and column totals are held fixed.

Binary coding with meaningful levels

Both levels have to be conceptually defensible. Collapsing a multi-category variable into two levels requires justifying where the cut falls.

Fixed marginal totals

The exact probability rests on treating the row and column totals as settled before the data were gathered. In designs where the margins are themselves subject to chance, the calculation leans conservative and the p value can come out larger than it should.

Sample size

The exact calculation is defined at any size, yet with very small totals even the smallest attainable p value can sit above the threshold, leaving the test unable to reveal a real association.

• Independence of observations: The panel does not check this assumption automatically; the researcher evaluates it.
• A 2x2 table: The condition is enforced during the run: a larger table stops the analysis, and the error message states the actual table size and directs you to the chi-square test for larger tables.
• Binary coding with meaningful levels: The panel reads the number of levels from the cleaned data and dims the method with a stated reason when a selection does not fit; it cannot evaluate the collapsing decision itself.
• Fixed marginal totals: The panel knows nothing about the sampling design and performs no check here; how the marginal totals arose is for the researcher to judge.
• Sample size: NO MINIMUM SAMPLE SIZE IS REQUIRED. However few observations there are, the analysis runs and prints a result, so whether the design had enough power is something you weigh from the table and the observation count.

1. 1Drag your file onto the panel; the upload area takes both CSV and XLSX.
2. 2In the editable grid on the Data tab, confirm that each of the two columns holds only two distinct values, since an unexpected third value usually traces back to spelling or to a missing-value code.
3. 3On the Variable tab mark the columns as nominal, declare the missing-value codes, and build a derived column if something needs reducing to two levels.
4. 4Search for the method on the Analysis tab. Turning on the switch for methods that fit your data narrows the list while unsuitable ones stay visible.
5. 5In the parameter form choose the two columns forming the table and set the direction of the test. Two-sided is the default, and a one-sided choice is defensible only against an expectation recorded in advance.
6. 6Run the analysis. The stacked bar chart of the contingency table is drawn above the result sections.
7. 7After reading the odds ratio and the p value, return to the 2x2 table and carry the cell counts into your write-up.
8. 8Export the result to Excel and the chart to PNG; the computation credits card names the library call and its version.

Odds ratio

The odds of the event in the first row divided by the odds in the second. A value of 1 points to no association, and values above 1 to the event being more likely in the first row.

Exact p value

The probability summed from the hypergeometric distribution. Since no distribution is approximated, it remains valid in sparse tables.

Direction used

Whether the test ran two-sided or under one of the one-sided options is stated in the result, which makes it easy to carry into a report.

Significance flag

A boolean field saying whether the p value falls below the customary 0.05 threshold.

2x2 contingency table

The four-cell table built from the cleaned data, reported with the number of people in each cell.

Column names and observation count

The names of the two columns forming the table and the number of valid observations entering the analysis both appear in the result.

Stacked bar chart

The contingency table is drawn as a stacked bar chart and can be saved as a PNG.

Odds ratio

Here the odds ratio does the work of an effect size, and no other measure is returned. A value equal to 1 means the odds match across the two rows, values above 1 mean the event is more likely in the first row, and values below 1 say the reverse. Because the scale is multiplicative, 2 and 0.5 describe associations of equal strength running in opposite directions. When the event is common the odds ratio looks markedly larger than the gap between the two proportions, so adding the cell percentages to the report guards against misreading. A zero in one cell can send the odds ratio to infinity or leave it undefined, and the interpretation then rests on the table alone.

No confidence interval is returned for this method. The odds ratio comes as a point estimate with nothing computed around it, and no interval is produced for a difference in proportions or for relative risk either. Reporting that requires an interval alongside the odds ratio has to obtain it outside the panel. Bear in mind as well that a zero cell makes such an interval problematic in its own right.

The numbers below are a representative example, not data from a real study or a real user.

Research question

In an illustrative pilot study, does one of two invitation wordings raise the chance that the survey is completed?

Variables

First variable: invitation wording (short wording, detailed wording) · Second variable: survey status (completed, abandoned)

Example result

Of 24 people who received the short wording, 18 completed the survey and 6 abandoned it; of 24 who received the detailed wording, 11 completed and 13 abandoned, for a pilot sample of 48. A two-sided test was chosen: odds ratio = 3.55, p = 0.075.

Interpretation

Completion stands at 75.0 per cent under the short wording and 45.8 per cent under the detailed one, and the odds ratio of 3.55 marks a noticeable divergence in favour of the short wording, yet the exact p value stays above the 0.05 threshold so the significance flag comes back negative. That is not evidence that the effect is absent: in a table of 48 people even an odds ratio of this size may fail to clear the threshold, so the finding counts as inconclusive and needs retesting on a larger sample in the main study. Since no interval is printed around the odds ratio, the breadth of the uncertainty can only be sensed from the cell counts. The figures are illustrative.

Research question: Does the purchase rate differ between women and men?

First variable

gender

Second variable

purchased

Computation credits: scipy 1.18.0 · statsmodels 0.14.6 · scikit-learn 1.9.0 · numpy 2.5.1 · pandas 3.0.5 · semopy 2.3.11 · 89fc29a · Data seed: 20260914

The association between survey completion and invitation wording was examined with Fisher's exact test and was not significant, two-sided p = .075, odds ratio = 3.55 (completion 75.0 per cent under the short wording and 45.8 per cent under the detailed wording; N = 48).

An example sentence close to APA style; the numbers are representative.

• It runs on 2x2 tables only, and feeding in a variable with three or more levels stops the analysis with the table size stated.
• With very small totals even the smallest attainable p value may not clear the threshold, so a non-significant result is no proof that the effect is absent.
• No interval is printed around the odds ratio, which leaves the breadth of the uncertainty out of the report and the calculation to another tool.
• A zero in one of the cells can send the odds ratio to infinity or leave it undefined, and in such tables the interpretation should not rest on that number alone.
• Treating the margins as fixed makes the calculation conservative in designs where the margins vary by chance, and the p value can then be larger than it need be.
• It does not apply to paired binary data where the same participants are measured under two conditions; that design requires a matched test.

• Chi-Square Test of IndependenceWhy: When the table exceeds 2x2 or the totals are comfortably large, it also offers an extra strength measure such as Cramer's V.
• Proportion TestWhy: When the question is framed as a comparison of two proportions and the reporting favours the language of rates over odds.
• McNemar TestWhy: When the binary outcome was measured twice on the same people, since independence fails and a matched test is required.
• Logistic RegressionWhy: When the binary outcome has to be modelled with several variables at once and odds ratios estimated while other effects are held constant.

• Chi-Square Test of Independence
• Cross-Tabulation
• Proportion Test

I ran the chi-square test and the expected counts were small. Will the panel move me to this test?

It will not. The test of independence in the panel never inspects expected cell counts and under no circumstances switches to the exact test by itself, so the two methods operate quite separately. If you read the expected frequency table and see the criterion failing, you have to pick this method on the Analysis tab yourself and run it again. That works when your table is 2x2; if it is larger the exact test will not run at all and merging categories is what you have to weigh.

When is a one-sided test the right choice?

Choosing a direction is defensible only if you wrote down which way you expected the divergence to run before seeing the data. Narrowing the direction after looking at the result shrinks the p value artificially and undermines the finding. The direction that was used appears in the result, so reporting it is straightforward, and staying with the two-sided default is the more cautious move whenever you are unsure.

My sample is tiny. Will this test still run?

Yes. No minimum observation count is required, so the analysis prints a result no matter how small the total. The real constraint here is practical rather than statistical: in a table of a handful of people, the smallest probability the hypergeometric distribution can yield may itself sit above the threshold. In that situation, reporting the result as inconclusive is more accurate than declaring no effect.

The odds ratio came out at 3.55. Can I read that as a difference in rates?

You cannot. An odds ratio is a ratio of odds, and when the event is common it looks markedly larger than the gap between the two rates. In the worked table the 29-point gap between 75.0 and 45.8 per cent turns into the far more dramatic-looking figure of 3.55 on the odds scale. To forestall that misreading you are expected to report the cell percentages as well and to use the odds ratio for direction and relative magnitude only.

• Agresti, A. (2018). An Introduction to Categorical Data Analysis
• Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics
• Fisher, R. A. (1934). Statistical Methods for Research Workers
• scipy.stats.fisher_exact documentation