Runs Test

A runs test looks at the ORDER of the values rather than their size. The panel splits each column at its own median: values above it carry one sign and values below it the other. Every unbroken stretch of the same sign then counts as one run. If the order really is random, the number of runs to expect follows from how many observations sit above and below the median. Markedly fewer runs than expected means values of the same sign have bunched together; markedly more means the signs alternate more regularly than chance would produce. The result turns that comparison into a z value and a p value through the standard normal approximation. In survey research the method mostly serves as a check on the fieldwork: answers that drift with the order of collection can reveal a pattern tied to the interviewers or to how the data were entered.

• Do the answers drift systematically with the order of entry, or is the sequence random?
• In fieldwork spread over several days, do later respondents differ structurally from earlier ones?
• Do records entered consecutively by the same interviewer resemble one another and cluster?
• Is there a time-related drift in consecutive readings taken from an instrument?

• When the order of the rows carries meaning: collection time, record number or measurement sequence has been preserved.
• When each column holds at least 10 observations; with fewer the engine does not run the test and says why.
• Before an analysis that rests on independence, to see whether any order-related pattern is present.
• During data quality checks, when looking for response patterns that repeat in blocks.
• When the question is whether consecutive readings or daily measurements drift over time.

• The columns tested must be numeric, since a median has to be computable on the scale.
• No grouping or time variable is selected. The sequence is simply the row order of the file, so sort the data the way you want before uploading it.
• A binary column can be tested too, but because the split still happens at the median, an unbalanced binary column can leave one side empty.
• Each row is one observation, and reordering the rows changes the result, which follows naturally from a test that reads sequence.

A meaningful sequence

The test takes the given order of the rows at face value. If the file is sorted alphabetically by name or by a score, the result reflects that sorting criterion instead of the data collection process.

Enough observations on both sides of the median

The expected number of runs follows from how many observations lie above and below the median. When one side holds very few cases the expectation becomes unstable and the normal approximation weakens.

Enough observations for the normal approximation

The distribution of the number of runs approaches normal only with a reasonable sample. In small samples the p value from the approximation is inexact and departs from the true value.

Ties not swamping the median

A pile of values exactly equal to the median blurs the split into two signs. In columns with few distinct values, a five-point Likert item for example, a large share of cases can share the median value.

• A meaningful sequence: The panel does not check this assumption automatically; the researcher evaluates it.
• Enough observations on both sides of the median: Where every value sits on the same side of the median the test does not run, and the situation appears as a message. Columns whose variance in the runs calculation comes out as zero are reported with an explanation in place of a result.
• Enough observations for the normal approximation: The engine does not test a column with fewer than 10 observations and reports the count as insufficient. Clearing that floor does not make the approximation trustworthy, it only makes the computation possible.
• Ties not swamping the median: The panel does not separately report how many cases equal the median. Since the counts above and below are printed, you can see from them whether the two add up to the number of observations.

1. 1Sort the data by whatever ordering you want preserved and save it as CSV or XLSX; the panel uses the row order exactly as uploaded.
2. 2Drop the file onto the panel and check in the Data tab grid that the sequence looks the way you expect.
3. 3On the Variable tab confirm that the columns to be tested are numeric and declare any missing value codes as missing.
4. 4Open the method picker on the Analysis tab and take this method from the randomness category; methods that do not fit are dimmed with a plain-language reason.
5. 5Tick the columns in the parameter form and change the alpha level if needed; the cut point is each column's own median and cannot be chosen.
6. 6Start the run. For each column the result lists the observation count, the number of runs, the expected number of runs, the median and the z and p values in collapsible sections.
7. 7Download the tables as Excel. No chart is drawn for this method; the computation credits card names the library call and its version.

Observation count and median

The number of complete observations used in the column and the median value taken as the cut point for the split.

Counts above and below the median

How many observations fall on either side of the cut point. The expected number of runs is derived from these two counts.

Observed number of runs

The count of unbroken stretches of the same sign. Every change of sign starts a new run.

Expected number of runs

How many runs would appear on average if the order were random. The gap to the observed count is what the test rests on.

z value and p value

The difference between observed and expected runs expressed through the standard normal approximation, with the two-tailed p value.

Randomness verdict

A label saying whether the sequence counts as random at the chosen alpha. Columns that could not be tested carry a message with the reason instead.

Ratio of observed to expected runs

A ratio near one says the sequence matches expectation. Values clearly below one indicate that same-sign values gather in blocks, and values above one a regular pattern where the signs take turns.

Magnitude of the z value

The distance of the departure in standard error units. A z beyond 2 in absolute value draws attention at conventional levels, but the direction is what matters here: negative points to clustering and positive to excessive alternation.

Length of the longest run

The panel does not return this value, yet it can be computed from the raw column and shows where the clustering concentrates. Most sequence problems gather in one part of the dataset.

This method returns no confidence interval. The number of runs, the expected number, the z and the p value are point estimates, and neither an interval for the run count nor the standard error itself is printed separately. The test is implemented only through the large-sample approximation: no exact p value is computed for small samples and no continuity correction is applied. So when the observation count stays in the tens the p value can depart from its true value, and in borderline cases hovering around the conventional threshold the decision should not rest on it alone. No effect size is produced either; the size of the departure is read by comparing the observed and expected run counts.

The numbers below are a representative example, not data from a real study or a real user.

Research question

Across three weeks of fieldwork, do satisfaction scores drift with the order in which responses were collected? (Illustrative example, not real data.)

Variables

Satisfaction score: continuous from 0 to 100, 240 responses · Rows ordered by survey completion time, alpha 0.05

Example result

The median was 68.0, with 120 observations above it and 120 below. Against an expected 121.0 runs, 84 runs were observed: z = -4.79, p < 0.001, and the sequence was not judged random.

Interpretation

With far fewer runs than expected, high and low scores have gathered into blocks along the order of entry. That points either to satisfaction drifting over time or to records from the same area or the same interviewer having been entered consecutively. The finding is a clue rather than an error: the next step is to compare means by week and by interviewer and locate the source of the clustering. It also weakens the assumption that observations are independent, since answers within a block resemble each other, which can make the standard errors of later group comparisons smaller than they should be.

Research question: Are the answers random in file order, or did the collection order leave a pattern?

Analyzed columns

satisfaction,income

Significance level (alpha)

0.05

Computation credits: scipy 1.18.0 · statsmodels 0.14.6 · scikit-learn 1.9.0 · numpy 2.5.1 · pandas 3.0.5 · semopy 2.3.11 · 89fc29a · Data seed: 20260914

The order in which satisfaction scores were collected was not random: across 240 observations, 84 runs were observed against an expected 121.0, z = -4.79, p < .001.

An example sentence close to APA style; the numbers are representative.

• The test depends on the given row order, and no ordering variable can be chosen in the panel; getting the sequence right before upload is the researcher's job.
• The cut point is always the column's own median. There is no option to split at the mean, at a fixed threshold or at a boundary set in advance.
• Only the large-sample approximation is implemented. With no exact p value and no continuity correction, the result is approximate in small samples.
• It does not say why the sequence departs from randomness. Separating a time effect from an interviewer effect or a data entry effect needs further analysis.
• In columns carrying many values tied to the median the split becomes ambiguous, which makes Likert items with few distinct values a poor fit.
• No chart visualises the runs, so the panel cannot show which part of the dataset the clustering sits in.

• Descriptive StatisticsWhy: When splitting the data into periods and comparing each period's mean and spread is enough, without testing the sequence itself.
• Normality TestsWhy: When the question concerns the shape of the distribution rather than the order; it reports skewness, kurtosis and formal tests.
• Independent Samples t-TestWhy: When two specific periods or two interviewers are the suspected source of the drift; it compares those means directly.

• Descriptive Statistics
• Normality Tests

The sequence came out non-random, is my data broken?

Not necessarily. The test says only that the order is not random, not why. In most survey work the reason is innocent: records are sorted by completion time and the composition of the sample shifted over the period, or records collected in one area sit next to each other in the file. The value of the finding is as a warning, and the task is to work out which variable the clustering travels with. If the drift follows the interviewer it is a data quality matter; if it follows the collection date it is a note about how representative the sample is.

Can I use a cut point other than the median?

The cut point is fixed in this method and each column is split at its own median. To split at the mean, at a particular score or at a theoretical threshold, define a derived column on the Variable tab that encodes your rule as a two-valued variable. The median of that new column coincides with your threshold by construction, so the test then works on the distinction you wanted. The route costs an extra step but it is the only way to tie the cut point to the research question.

How many observations are enough?

The engine refuses to run below 10 observations, but that is only the floor at which the computation goes through. The p value becomes trustworthy as the distribution of the run count approaches normal, which needs a reasonable number of cases on each side; a common recommendation is roughly 10 or more both above and below the median. Since no exact p value is computed here and no continuity correction is applied, borderline results in small samples call for caution.

What does it mean when there are more runs than expected?

More runs than expected means the signs alternate more regularly than chance allows, so high and low values have arrived almost in turn. In survey data that pattern usually traces back to the order of data entry or to records merged alternately from two sources. In instrument readings it suggests a process that overcorrects itself. Either direction leads to the same conclusion: the sequence holds a structure that randomness does not explain, and its source is worth finding.

• Wald, A., & Wolfowitz, J. (1940). On a Test Whether Two Samples are from the Same Population
• Hollander, M., Wolfe, D. A., & Chicken, E. (2014). Nonparametric Statistical Methods
• Field, A. (2018). Discovering Statistics Using IBM SPSS Statistics
• scipy.stats.norm documentation