McNemar Test

The McNemar test compares two paired binary measurements, typically yes and no answers given by the same people before and after something happened, or two matched binary items. Four outcomes are possible: yes twice, no twice, yes on the first measurement only, and yes on the second measurement only. The first two groups held their position and play no part in the statistic; all the information sits in the two discordant cells that hold the people who moved. The question is whether the count of those who went from yes to no departs meaningfully from the count of those who went from no to yes, once you assume movement has no preferred direction. Built this way, the test addresses the symmetry of change rather than the absolute level of either proportion: forty per cent can say yes at both measurements while a great deal of traffic passes in both directions underneath. YouReply Analyze builds the 2x2 table from row-level data, reports the discordant cell counts separately, and states in the result which version of the calculation was applied.

• Did the intention to recommend a brand shift among participants after they watched an explanatory video?
• Has awareness of a campaign among the same panel members changed between two measurement waves?
• Did the share of staff who knew a safety rule correctly rise after the training programme?
• Do acceptance rates differ between two product features presented side by side in the same questionnaire?

• When the answer is binary and the same participant is measured at two points in time or under two conditions.
• In matched designs, where each participant is paired with a counterpart of similar age and gender and the binary answers of the two members are compared.
• When the question concerns a proportion rising or falling between measurements rather than a comparison between two separate groups of people.
• When the two measurements sit in two columns of the same file, with one row per participant.
• When what matters first is whether the movement is lopsided rather than how large the shift is.

• First column: the binary answer at the first measurement. It must hold two distinct values, such as yes and no, 1 and 0, or present and absent.
• Second column: the same answer at the second measurement, coded identically. Using different labels across the two columns pushes the table beyond two categories and makes the analysis meaningless.
• Each row stands for one participant, that is one matched pair. Data kept in long form, with one row per measurement, have to be reshaped into wide form first.
• A response with more than two categories must be reduced to a binary split before it enters this test, and the derived column feature on the Variable tab keeps a record of that reduction.
• Once there are three binary measurements this test no longer applies; three or more paired binary columns call for Cochran's Q test.

Measurements are genuinely paired

The values in the two columns must belong to the same person or to a pair deliberately matched together. Proportions from two independent samples cannot be compared this way, because without pairing the agreement and disagreement cells carry no meaning.

Independence between pairs

The two measurements of one participant are related by design, but different participants should be independent of each other. Where several members of a household or a cluster of classmates appear, the significance looks more impressive than it is.

A two-level response

Both columns must carry exactly two levels so that agreement and disagreement can be gathered into a 2x2 table.

Enough discordant pairs

How far the chi-square approximation can be trusted depends on the number of discordant pairs. When few people switch sides, an exact probability calculation is needed instead of the approximate distribution.

Handling of missing values

A participant without an answer at one of the two measurements forms no pair and leaves the analysis. If those gaps are not random, the change read from the surviving pairs can be biased.

• Measurements are genuinely paired: The panel cannot verify that rows really belong to the same person, since the pairing comes from the layout of your file. Checking that the columns are correctly aligned in wide form is your job on the Data tab grid.
• Independence between pairs: The panel does not check this assumption automatically; the researcher evaluates it.
• A two-level response: The table is built during the run; the analysis will not proceed when more than two levels are present or when cleaning leaves a table that is not 2x2.
• Enough discordant pairs: This decision belongs to the panel and is automatic: fewer than 25 discordant pairs triggers the exact binomial version, and 25 or more the chi-square approximation with a continuity correction. Which one ran appears as a field of its own in the result, and you are expected to pass that on when you report.
• Handling of missing values: Incomplete pairs are dropped and the number of pairs entering the table is visible in the result; no assessment is made of why they went missing.

1. 1Drag your data file onto the upload area. CSV and XLSX are both read, and the file opens in the editable grid on the Data tab.
2. 2In the grid, confirm that the two measurement columns use the same coding; Yes in one column and Y in the other count as two separate categories.
3. 3On the Variable tab mark both columns as nominal, declare the missing-value codes, and build the derived column that reduces a multi-category answer to binary if you need one.
4. 4On the Analysis tab pick the method from the categorical group, using the search box or the 'Only ones that fit my data' switch. Methods that do not fit appear dimmed with a plain-language reason beside them.
5. 5In the parameter form name the first and second measurement columns in order; the order affects only how the table reads, not the statistic.
6. 6Run the analysis and read the result from the collapsible sections. No chart is produced for this method, so the tables sit at the top.
7. 7Open the 2x2 table, compare the discordant cells, check which version was used, and export the tables to Excel; the computation credits card names the call and its version.

Test statistic

Its meaning follows the version used: the continuity-corrected chi-square value in the approximate version, and the statistic underlying the binomial calculation in the exact one.

Significance value

The p value computed under the hypothesis that movement is balanced in both directions. In the exact version the probability comes straight from the binomial distribution, in the approximate version from the chi-square distribution.

The 2x2 agreement table

The joint distribution of the two measurements: yes twice, no twice, and the two discordant cells. Everything the test uses lies off the diagonal.

Discordant cell counts

The numbers who moved from yes to no and from no to yes are reported separately, and the gap between them shows the direction of the shift.

Which version ran

The result states whether the exact binomial calculation or the corrected chi-square approximation produced the numbers, a choice made automatically from the count of discordant pairs.

No effect size is computed

This method returns no separate measure of magnitude: no difference between proportions, no odds ratio and no coefficient suited to paired data. To convey how large the shift is you have to report the yes percentages at both measurements and the discordant counts yourself. The most readable presentation in practice puts the two percentages side by side and adds the number of participants who switched in each direction.

No confidence interval is returned for this method. The test statistic and the p value arrive as point results, and no interval is computed for the difference between the two proportions or for a paired odds ratio. Where a publication requires one, it has to be produced outside the panel.

The numbers below are a representative example, not data from a real study or a real user.

Research question

In an illustrative communication study, did watching an explanatory video change participants' intention to recommend the product?

Variables

First column: intention to recommend before watching (yes, no) · Second column: intention to recommend after watching (yes, no)

Example result

240 participants who answered at both measurements entered the analysis. In the table 64 said yes twice and 95 said no twice, while 29 moved from yes to no and 52 from no to yes. With 81 discordant pairs the corrected chi-square version was selected automatically: the statistic came to 5.98 with p = 0.015. The yes share stood at 38.8 per cent at the first measurement and 48.3 per cent at the second.

Interpretation

Movement was not evenly split: roughly twice as many people crossed from no to yes as the other way. The significance says that this degree of lopsidedness would be uncommon among 81 discordant pairs if direction were a coin flip. What deserves attention is that the 159 participants who held their position contribute nothing to the calculation, so the result rests on the imbalance among movers rather than on the sample as a whole. Since no effect size is returned, the weight of the shift has to be conveyed through the nine and a half point gap between 38.8 and 48.3 per cent. With a single group and no comparison condition, attributing the rise to the video alone is not defensible. The figures are illustrative.

Research question: Did the usage rate of the same people change between the two measurements?

First variable

use_before

Second variable

use_after

Computation credits: scipy 1.18.0 · statsmodels 0.14.6 · scikit-learn 1.9.0 · numpy 2.5.1 · pandas 3.0.5 · semopy 2.3.11 · 89fc29a · Data seed: 20260914

Intention to recommend changed significantly after the video, McNemar test, corrected chi-square(1, N = 240) = 5.98, p = .015, with 52 participants moving from no to yes and 29 from yes to no.

An example sentence close to APA style; the numbers are representative.

• Because participants who kept their answer are set aside, the test stays insensitive whenever discordant pairs are few, however large the sample.
• The result speaks to the imbalance in direction of change, not to the size of the rise in the proportion, so a p value read without the percentages tells a reader very little.
• In a single-group before and after design, a significant result is no proof that an intervention worked, since everything else that happened over the same period can produce the same imbalance.
• It cannot be applied once there are three paired binary columns; three or more repeated measurements need a different test.
• With neither an effect size nor an interval returned, completing a reporting table is work you do by hand.
• Sweeping several binary items pair by pair through this test inflates the chance of a false positive as the number of comparisons grows, and no multiple-comparison correction is applied for you.

• Cochran's Q TestWhy: When there are three or more paired binary measurements; it tests several yes or no items, or several time points, in the same people at once.
• Chi-Square Test of IndependenceWhy: When the two groups compared are independent, meaning the same people were not measured twice; independence is then tested on a contingency table.
• Wilcoxon Signed-Rank TestWhy: When the paired measurement is ordinal or continuous rather than binary; it tests the difference between measurements through signed ranks.
• Paired Samples t-TestWhy: When the difference between the two measurements is asked at the level of means and the response is scored.

• Cochran's Q Test
• Chi-Square Test of Independence
• Wilcoxon Signed-Rank Test

Can I choose the exact binomial version myself?

The choice belongs to the panel and follows the number of discordant pairs: below 25 the exact binomial calculation runs, at 25 and above the chi-square approximation with a continuity correction. No switch in the parameter form overrides this. You can see which version ran from its own field in the result and carry that into your report; because the two versions can diverge slightly near the boundary, stating it belongs in the methods section.

Why are people who gave the same answer twice left out?

The question being asked is whether movement favours one direction. Those who held their position carry no information about that, because only the switchers reveal how much traffic went each way. The diagonal of the 2x2 table is therefore reported but kept out of the statistic. When reading the result, always state the number of discordant pairs: twenty switchers in a file of 500 people tell you no more than twenty switchers in a file of 60.

The result is significant. Can I say the intervention worked?

The test only says that switching was lopsided between two measurements. When a before and after measurement is taken in a single group, a change in the news agenda, a seasonal effect or a tendency created by being measured a second time can all sit behind the shift. A causal claim needs a comparison condition or an assignment mechanism, and no statistic supplies that for you.

How should I report magnitude?

Since the panel returns no effect size here, build the reporting out of descriptive numbers: the yes percentages at both measurements, the gap in percentage points between them, and the counts in the two discordant cells. Those three pieces give a reader both the direction and the practical weight of the change. If you want a confidence interval for the difference between proportions, compute it outside the panel.

• Agresti, A. (2018). An Introduction to Categorical Data Analysis
• Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics
• Siegel, S., & Castellan, N. J. (1988). Nonparametric Statistics for the Behavioral Sciences
• statsmodels.stats.contingency_tables.mcnemar documentation