Customer Metrics (NPS): Method

NPS, CSAT and CES: Intervals, Comparisons, Trends, Sample Size and Key Drivers

What the tool takes

Survey responses in two layouts. Respondent rows: a first line of column names, then one line per respondent with the cells separated by commas, tabs or semicolons. The score column is the first one named score, ltr, nps, csat, ces, rating, recommend, likelihood, satisfaction or effort (else the first column); a column named segment (or group, region, product, channel, plan, country, tier) gives the segments; one named period (or wave, quarter, month, year, date, week) gives the periods; weight carries a survey weight and count repeats a line that many times. Every other column whose non-blank cells are numbers is a driver rating (a sub-question such as support, price or quality). Blank cells and NA are missing. A tally is the other layout: lines of score and count, or the counts alone in scale order; a tally carries no segments, periods or drivers.

The metric select says how the score column is read. NPS is Reichheld's (2003) 0–10 "how likely are you to recommend" item: promoters 9–10, passives 7–8, detractors 0–6, NPS = 100 (p_P − p_D) between −100 and +100. CSAT is a 1–5 satisfaction item summarised as the top-two-box share (4 and 5), the convention of Sauro and Lewis (2016). CES is the Customer Effort Score of Dixon, Freeman and Toman (2010) in its later 7-point agreement form ("the company made it easy for me to handle my issue", 1 strongly disagree to 7 strongly agree), summarised as the share answering 5 to 7; the original 1–5 effort item (1 = very low effort) runs the other way and should be reversed before pasting. For CSAT and CES the mean rating with its t interval is shown as well.

The score and its interval

The promoter and detractor shares come from one multinomial sample, so the variance of their difference is Var(p_P − p_D) = (p_P + p_D − (p_P − p_D)²)/n (Rocks 2016 §1.4; the variance of a difference of two multinomial proportions, Agresti 2002), and SE(NPS) = 100 √Var. The Wald interval NPS ± z SE is the one most calculators print. Rocks (2016) compared it with the Goodman, Adjusted Wald and score-test intervals by simulation over 10,000 trinomial distributions and the Satmetrix data of 236 companies and found the Wald interval's coverage poor at small n; his recommended closed form is the Adjusted Wald AW(3, T): add ¾ of a response to the promoters and to the detractors, 1½ to the passives (3 in all, triangular shape) and 3 to n, then compute the Wald interval of the adjusted shares. The adjustment pulls the centre toward zero by a fraction of a point and widens the interval slightly; above n ≈ 100 the two intervals differ little, and at n = 30 the adjusted one keeps its 95 % coverage where the Wald interval falls to about 90 %. The page prints both and reports the adjusted one. It is the method Sauro and Lewis publish for NPS intervals ("add 3 to n, ¾ to the promoters and ¾ to the detractors"); their seventeen printed 90 % widths are reproduced by the test script.

For a share (CSAT, CES) the Wilson score interval (Wilson 1927) (p + z²/2n ± z √(p(1 − p)/n + z²/4n²))/(1 + z²/n) is reported, the method Newcombe (1998) recommended after comparing seven, with the Agresti and Coull (1998) adjusted Wald interval beside it: add z²/2 successes and z² trials, then p̃ ± z √(p̃(1 − p̃)/ñ). Sauro and Lewis's rule "add two successes and four trials" is this interval at z = 2. With survey weights the shares are weighted and every standard error uses the Kish effective sample size n/deff with deff = 1 + CV²(w) (Kish 1965).

Comparisons

An entered benchmark (an industry figure or a target) is treated as a number without sampling error: z = (score − B)/SE and the two-sided p-value from the standard normal; the score differs from the benchmark at level α when |z| > z_α/2. The tool ships no benchmark tables, because the published industry tables are proprietary; the analyst enters the figure and its source. A prior-period score entered with its sample size n₁ carries its own sampling error, which the page takes at the largest value its score allows, the no-passive bound Var ≤ (1 − (B/100)²)/n₁ (for a share, B(1 − B)/n₁), so the test of the change is slightly conservative; without n₁ the prior score is treated as exact.

Two segments (or two periods) in the data are compared by the difference of their scores with SE = √(SE_A² + SE_B²), the two samples being independent, with the z test and the Wald interval of the difference. All segments at once are tested by Pearson's X² = Σ (n_ij − μ_ij)²/μ_ij on the k × 3 table of promoter / passive / detractor counts (k × 2 of favourable / other for a share), on (k − 1)(c − 1) degrees of freedom, with Cramér's V = √(X²/(n (min(k, c) − 1))) for the strength of the association (Agresti 2002 §3.2); expected counts below 5 are flagged.

Trend over periods

Periods are sorted naturally (Q1 before Q2, 2024-01 before 2024-02, 3 before 10) and each gets its score and interval. The trend line is the weighted least-squares fit of the score on the period index t = 1 … m with weights 1/SE²: slope b = Σw(t − t̄)(y − ȳ)/Σw(t − t̄)², SE(b) = 1/√Σw(t − t̄)², the average change per period with its z test. The change-point test looks for a single step: for every split after period k the periods before and after are pooled, their scores compared by the two-sample z, and the split with the largest |z| is reported with its p-value multiplied by the number of splits (the Bonferroni correction for having chosen the best of m − 1 tests). A second change inside either half is not tested; with many periods a CUSUM or a segmented regression is the next step.

Sample size

The half-width of the Wald interval is h = z SE. At the observed shares the respondents needed for a target half-width h₀ are n = (100 z)² (p_P + p_D − (p_P − p_D)²)/h₀² for NPS and n = (100 z)² p(1 − p)/h₀² for a share (the formula of Cochran 1977 §4.4 with the NPS variance in place of p(1 − p)). The requirement scales with 1/h₀²; a more polarised population (more promoters and detractors, fewer passives) needs more responses for the same precision, since the NPS variance is largest when the sample is split between the two extremes.

Key drivers

Each driver rating is correlated with the score over the respondents who answered both: Pearson r (linear) with the t test t = r √((n − 2)/(1 − r²)) on n − 2 degrees of freedom, and Spearman ρ (the Pearson correlation of the ranks, ties given their mean rank), which is robust to the unequal spacing of rating scales. The correlation is the driver's derived importance, the mean rating its performance; the importance–performance chart of Martilla and James (1977) splits the drivers at the mean importance and the mean performance into "concentrate here" (important, under-performing), "keep up the good work", "possible overkill" and "low priority". The multiple regression of the score on every driver (least squares by Householder QR, over the respondents complete in all drivers) gives each driver's coefficient with its t test, the standardized coefficient b_j s_xj/s_y, and Pratt's (1987) importance measure, the standardized coefficient times the driver's correlation with the score, which splits R² into one share per driver. Correlated drivers share their importance, so a near-zero coefficient beside a sizeable correlation means the effect is carried by a correlated driver; the Pratt shares can be negative in that case and are then not an allocation. A driver analysis ranks what moves with the score in one sample; whether moving the driver moves the score is a causal question for an experiment or a longitudinal reading.

Worked example on the page

The default file is an illustrative seeded set of 400 responses in three segments (Consumer, Enterprise, SMB) over four quarters with three driver ratings (support, price, quality on 1–10), made with a recorded script so that every section has something to show; it is not survey data, and the benchmark (+30), prior score (+20 from 350 responses) and target half-width (5 points) are illustrative entries. It gives NPS +26.5 (191 promoters, 124 passives, 85 detractors), SE 3.94, Adjusted Wald 95 % interval +18.6 to +34.0 (Wald +18.8 to +34.2); not distinguishable from the +30 benchmark (z = −0.89) nor from the prior +20 (z = 0.99); Enterprise +66.9 against Consumer −8.2 (z = 8.27), X² = 71.05 on 4 degrees of freedom; a step after Q2 (+5.9 to +52.0, z = 6.19); 953 responses for a ± 5-point interval; quality (r = 0.65) and support (r = 0.62) the key drivers, price (r = 0.26) low priority, R² = 0.529. The test script checks the library against Rocks (2016), the seventeen Adjusted Wald widths printed by Sauro and Lewis, their adjusted-Wald proportion examples, Newcombe's Wilson intervals, Agresti's gender × party table (X² = 7.01), the NIST StRD Norris and Longley regressions and a numpy / scipy run on the default file.

Limits of the tool

• Simple random sampling with optional weights; clustering and stratification enter only through the weights' design effect.
• Intervals are Wald, Adjusted Wald AW(3, T) and Wilson / Agresti–Coull; the iterative score interval and the bootstrap of Rocks (2016) are not offered.
• One change point; no seasonal adjustment or time-series model of the periods.
• Driver analysis by correlation and ordinary least squares; no relative weights (Johnson), dominance analysis, ordinal regression or structural model.
• Comparisons between segments and periods assume independent samples (different respondents); a panel re-surveyed each period is correlated and its change test is conservative here.
• No benchmark tables: the analyst enters the benchmark and its source.