Friedman Test

The Friedman test ranks every participant's answers across the conditions within that participant, then asks whether the average ranks of the conditions stay close to one another. Ranking inside each row removes differences in overall level between people: if one respondent is generally generous and another generally stingy, that has no bearing on the result. The null hypothesis is that the expected mean ranks of the conditions are equal. Because only the order of the values is used and not the spacing between them, the test is preferred over repeated measures analysis of variance on short rating scales and in small samples where normality cannot be defended. YouReply Analyze reports the chi-square statistic alongside a table of mean rank and median per condition, with Kendall's W as a measure of concordance.

• Do the same panel members rate three packaging designs differently on how much they like them?
• Do self-efficacy ratings collected at the start, middle and end of a training programme shift?
• Do respondents find four advertising headlines equally easy to understand?
• Does reported usage intensity fluctuate across three weeks of a reminder experiment?

• When a measurement is repeated under three or more conditions and every run comes from the same people.
• When the answers sit on an ordinal rating scale whose intervals cannot be treated as equal.
• When an overall verdict is wanted before pairwise Wilcoxon comparisons on three or more conditions.
• When the number of subjects is small and nothing can be assumed about the shape of the within-condition distributions.

• One column per condition: at least three columns are chosen and a row holds one participant across all of them.
• Every selected column is coded numerically and measured at least at the ordinal level.
• Any row with a missing value is dropped entirely, and at least six complete rows must remain afterwards.

Independence of blocks

Participants must be independent of each other, so that one person's answers do not influence another's. Dependence across conditions, by contrast, is exactly what the test expects.

Fully matched rows

For the interpretation to hold, every row needs a measurement under all conditions. Participants who answered only some conditions cannot enter the analysis.

At least ordinal measurement

The values must be meaningfully orderable within a row. Ranking inside a row carries no meaning for unordered categorical codes.

Three or more conditions

With two conditions the test adds nothing: within-row ranking collapses into sign information, and the Wilcoxon signed-rank test answers the same question more directly.

• Independence of blocks: The panel does not check this assumption automatically; the researcher evaluates it.
• Fully matched rows: The engine drops rows with missing values outright and refuses to run if fewer than six complete rows are left.
• At least ordinal measurement: The parameter form asks for a list of at least three columns, and column names with no counterpart in the data set are reported by name.
• Three or more conditions: The engine rejects the analysis when fewer than three columns are selected and states why.

1. 1Upload your data file and confirm on the Data tab that the repeated measurements arrive as separate columns.
2. 2On the Variable tab mark the measurement level of the condition columns as ordinal and declare your missing value codes.
3. 3Find the Friedman test in the method picker on the Analysis tab; the data requirements box lists which columns can serve as matched measurements.
4. 4In the parameter form select the condition columns to compare, at least three of them, in order.
5. 5Run the analysis. The chi-square value, degrees of freedom, p value, the table of mean rank and median per condition and Kendall's W open as separate sections.
6. 6If the overall test is significant, run the Wilcoxon signed-rank test separately to see which pairs of conditions differ, and apply a multiple comparison correction yourself.

Chi-square statistic

A single value summarising how far the mean ranks of the conditions move away from their common expected rank.

Degrees of freedom

The number of conditions compared minus one, written next to the chi-square value when reporting.

p value

The probability of a chi-square value at least this large if the expected ranks of the conditions were equal.

Number of subjects and conditions

How many participants entered the analysis after incomplete rows were dropped, and how many conditions were compared.

Mean rank and median per condition

Each condition's average rank from the within-row rankings together with its raw median; this table carries the direction.

Kendall's W and its label

The coefficient of concordance is returned with a label describing its magnitude.

Kendall's coefficient of concordance (W)

Obtained by dividing the chi-square value by the number of subjects times one less than the number of conditions, so it stays between zero and one. It expresses how far participants agree on the ordering of the conditions: below 0.1 counts as negligible, around 0.3 as weak, around 0.5 as moderate and 0.7 or more as strong agreement. As the value grows, the ordering becomes consistent from one respondent to the next.

This method returns no confidence interval, neither for the condition medians nor for Kendall's W; every value is reported as a point estimate. An interval has to come from a separate calculation or from a resampling procedure of your own.

The numbers below are a representative example, not data from a real study or a real user.

Research question

In a message test, do the same panel members rate three information texts differently for clarity on a seven-point item?

Variables

First condition: clarity of the short text (1-7, ordinal) · Second condition: clarity of the bulleted text (1-7, ordinal) · Third condition: clarity of the worked-example text (1-7, ordinal)

Example result

Complete rows: 48; conditions: 3. Mean ranks were 1.59 (median 4), 2.05 (median 5) and 2.36 (median 5). Chi-square(2) = 14.41, p = 0.001, Kendall's W = 0.150 (weak agreement).

Interpretation

The three texts separate overall, with the short text sitting at the bottom of the order. Because Kendall's W stays low, that ordering is only loosely shared: most respondents lean the same way, but individual rankings vary a good deal. Naming the pair of texts responsible calls for Wilcoxon comparisons run on the side.

Research question: Do the same respondents' scores change across the three measurement times?

Analyzed columns

measure_1,measure_2,measure_3

Computation credits: scipy 1.18.0 · statsmodels 0.14.6 · scikit-learn 1.9.0 · numpy 2.5.1 · pandas 3.0.5 · semopy 2.3.11 · 89fc29a · Data seed: 20260914

Clarity ratings differed significantly across the three information texts, chi-square(2) = 14.41, p = .001, Kendall's W = .150 (n = 48), with mean ranks of 1.59, 2.05 and 2.36 respectively.

An example sentence close to APA style; the numbers are representative.

• Not for matched designs with only two conditions, where the Wilcoxon signed-rank test answers directly.
• Not for independent groups made up of different people, because within-row ranking only means something for matched measurements.
• A significant result does not name the conditions that differ, and this method produces no pairwise comparisons; you run and correct them yourself.
• A participant who skipped one condition is dropped completely, so scattered missingness erodes the usable number of subjects quickly.
• As ties within a row multiply on short scales, the ranking carries less information and the test loses power.
• It speaks about differences in order rather than in magnitude, so reporting a difference in mean scores needs another method.

• Repeated Measures ANOVAWhy: When the measurements are continuous and the distributional assumptions can be defended; it gives condition means and sphericity information directly.
• Wilcoxon Signed-Rank TestWhy: When only two conditions are compared, or when pairwise comparisons are needed after the overall test.
• Kruskal-Wallis H TestWhy: When the conditions come from separate groups rather than the same people, so the ranking is built across the whole sample.

• Wilcoxon Signed-Rank Test
• Repeated Measures ANOVA
• Kruskal-Wallis H Test

Does the panel run pairwise comparisons after the Friedman test on its own?

It does not. This method produces only the overall verdict and the per-condition rank table. For pairwise work you run the Wilcoxon signed-rank test separately on each pair of conditions, then apply a Bonferroni or comparable correction to the p values you obtain.

How many participants do I need?

After incomplete rows are dropped the engine requires six complete rows and will not run below that. That floor is a technical threshold rather than a guarantee of power: detecting small shifts on a seven-point scale usually takes several dozen participants.

Kendall's W is low but the p value is significant, is that a contradiction?

No. The p value says whether chance can account for the ordering, and with more subjects even faint orderings reach significance. Kendall's W measures how far participants share the same ranking. Reporting both is how a significant but weak pattern gets described honestly.

How are tied ratings handled?

When a participant gives two conditions the same value, those values receive the average rank within that row. The engine computes and reports no further tie correction, so on short scales with many ties the chi-square value can come out smaller than the pattern deserves.

• Hollander, M., Wolfe, D. A., & Chicken, E. (2014). Nonparametric Statistical Methods
• Conover, W. J. (1999). Practical Nonparametric Statistics
• Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics
• scipy.stats.friedmanchisquare documentation