Tools › Finance and Business › Marketing › Customer Metrics (NPS)


Customer Metrics (NPS)

Net Promoter Score, CSAT and Customer Effort Score from pasted survey responses (one respondent per line with optional segment, period and driver ratings, or a tally of counts per score point): the score with its standard error and confidence interval (NPS by the multinomial variance of Rocks 2016 with his recommended Adjusted Wald interval; CSAT top-two-box and CES favourable share by the Wilson score and Agresti–Coull intervals), the promoter / passive / detractor distribution, comparison tests against an industry benchmark and a prior period, between segments (difference with its standard error, X² test of the category distributions) and between periods, the trend over periods with a weighted-least-squares slope and a single change-point test, the sample size for a target interval half-width, and a key-driver analysis of the driver ratings (Pearson and Spearman correlations with the score, importance–performance quadrant chart, multiple regression with standardized coefficients and Pratt importance). No benchmark tables are shipped: benchmarks are entered by the analyst. The default case is an illustrative file of 400 responses in three segments over four quarters with three driver ratings (NPS +26.5, 95 % interval +18.6 to +34.0); the library is checked against Rocks (2016), the seventeen published Adjusted Wald widths of Sauro and Lewis, Newcombe's Wilson intervals, Agresti's X² table and the NIST StRD regression datasets.

  • NPS with the multinomial standard error and the Adjusted Wald interval of Rocks (2016); CSAT and CES favourable shares with Wilson and Agresti–Coull intervals; mean score with its t interval

  • Comparisons: against an entered benchmark or prior period, between two segments or periods (difference, SE, z), across all segments (X² test of promoters / passives / detractors)

  • Trend over periods with intervals, weighted-least-squares slope and a single change-point test; sample size for a target half-width

  • Key drivers: Pearson and Spearman correlation of each driver with the score, importance–performance quadrant chart, multiple regression with standardized coefficients and Pratt importance

Customer Metrics


Unit System: /

Responses (first line = column names; cells separated by commas or tabs; or a tally of counts per score point)




Benchmarks and precision (scores in the metric's unit, points for NPS and percent for CSAT / CES; blank = not used)






Comparisons (names as they appear in the segment and period columns; blank = the first and last)





×

Customer Metrics (NPS)

NPS, CSAT and CES: Intervals, Comparisons, Trends, Sample Size and Key Drivers

What the tool takes

Survey responses in two layouts. Respondent rows: a first line of column names, then one line per respondent with the cells separated by commas, tabs or semicolons. The score column is the first one named score, ltr, nps, csat, ces, rating, recommend, likelihood, satisfaction or effort (else the first column); a column named segment (or group, region, product, channel, plan, country, tier) gives the segments; one named period (or wave, quarter, month, year, date, week) gives the periods; weight carries a survey weight and count repeats a line that many times. Every other column whose non-blank cells are numbers is a driver rating (a sub-question such as support, price or quality). Blank cells and NA are missing. A tally is the other layout: lines of score and count, or the counts alone in scale order; a tally carries no segments, periods or drivers.

The metric select says how the score column is read. NPS is Reichheld's (2003) 0–10 "how likely are you to recommend" item: promoters 9–10, passives 7–8, detractors 0–6, NPS = 100 (pP − pD) between −100 and +100. CSAT is a 1–5 satisfaction item summarised as the top-two-box share (4 and 5), the convention of Sauro and Lewis (2016). CES is the Customer Effort Score of Dixon, Freeman and Toman (2010) in its later 7-point agreement form ("the company made it easy for me to handle my issue", 1 strongly disagree to 7 strongly agree), summarised as the share answering 5 to 7; the original 1–5 effort item (1 = very low effort) runs the other way and should be reversed before pasting. For CSAT and CES the mean rating with its t interval is shown as well.

The score and its interval

The promoter and detractor shares come from one multinomial sample, so the variance of their difference is Var(pP − pD) = (pP + pD − (pP − pD)²)/n (Rocks 2016 §1.4; the variance of a difference of two multinomial proportions, Agresti 2002), and SE(NPS) = 100 √Var. The Wald interval NPS ± z SE is the one most calculators print. Rocks (2016) compared it with the Goodman, Adjusted Wald and score-test intervals by simulation over 10,000 trinomial distributions and the Satmetrix data of 236 companies and found the Wald interval's coverage poor at small n; his recommended closed form is the Adjusted Wald AW(3, T): add ¾ of a response to the promoters and to the detractors, 1½ to the passives (3 in all, triangular shape) and 3 to n, then compute the Wald interval of the adjusted shares. The adjustment pulls the centre toward zero by a fraction of a point and widens the interval slightly; above n ≈ 100 the two intervals differ little, and at n = 30 the adjusted one keeps its 95 % coverage where the Wald interval falls to about 90 %. The page prints both and reports the adjusted one. It is the method Sauro and Lewis publish for NPS intervals ("add 3 to n, ¾ to the promoters and ¾ to the detractors"); their seventeen printed 90 % widths are reproduced by the test script.

For a share (CSAT, CES) the Wilson score interval (Wilson 1927) (p + z²/2n ± z √(p(1 − p)/n + z²/4n²))/(1 + z²/n) is reported, the method Newcombe (1998) recommended after comparing seven, with the Agresti and Coull (1998) adjusted Wald interval beside it: add z²/2 successes and z² trials, then p̃ ± z √(p̃(1 − p̃)/ñ). Sauro and Lewis's rule "add two successes and four trials" is this interval at z = 2. With survey weights the shares are weighted and every standard error uses the Kish effective sample size n/deff with deff = 1 + CV²(w) (Kish 1965).

Comparisons

An entered benchmark (an industry figure or a target) is treated as a number without sampling error: z = (score − B)/SE and the two-sided p-value from the standard normal; the score differs from the benchmark at level α when |z| > zα/2. The tool ships no benchmark tables, because the published industry tables are proprietary; the analyst enters the figure and its source. A prior-period score entered with its sample size n1 carries its own sampling error, which the page takes at the largest value its score allows, the no-passive bound Var ≤ (1 − (B/100)²)/n1 (for a share, B(1 − B)/n1), so the test of the change is slightly conservative; without n1 the prior score is treated as exact.

Two segments (or two periods) in the data are compared by the difference of their scores with SE = √(SEA² + SEB²), the two samples being independent, with the z test and the Wald interval of the difference. All segments at once are tested by Pearson's X² = Σ (nij − μij)²/μij on the k × 3 table of promoter / passive / detractor counts (k × 2 of favourable / other for a share), on (k − 1)(c − 1) degrees of freedom, with Cramér's V = √(X²/(n (min(k, c) − 1))) for the strength of the association (Agresti 2002 §3.2); expected counts below 5 are flagged.

Trend over periods

Periods are sorted naturally (Q1 before Q2, 2024-01 before 2024-02, 3 before 10) and each gets its score and interval. The trend line is the weighted least-squares fit of the score on the period index t = 1 … m with weights 1/SE²: slope b = Σw(t − t̄)(y − ȳ)/Σw(t − t̄)², SE(b) = 1/√Σw(t − t̄)², the average change per period with its z test. The change-point test looks for a single step: for every split after period k the periods before and after are pooled, their scores compared by the two-sample z, and the split with the largest |z| is reported with its p-value multiplied by the number of splits (the Bonferroni correction for having chosen the best of m − 1 tests). A second change inside either half is not tested; with many periods a CUSUM or a segmented regression is the next step.

Sample size

The half-width of the Wald interval is h = z SE. At the observed shares the respondents needed for a target half-width h₀ are n = (100 z)² (pP + pD − (pP − pD)²)/h₀² for NPS and n = (100 z)² p(1 − p)/h₀² for a share (the formula of Cochran 1977 §4.4 with the NPS variance in place of p(1 − p)). The requirement scales with 1/h₀²; a more polarised population (more promoters and detractors, fewer passives) needs more responses for the same precision, since the NPS variance is largest when the sample is split between the two extremes.

Key drivers

Each driver rating is correlated with the score over the respondents who answered both: Pearson r (linear) with the t test t = r √((n − 2)/(1 − r²)) on n − 2 degrees of freedom, and Spearman ρ (the Pearson correlation of the ranks, ties given their mean rank), which is robust to the unequal spacing of rating scales. The correlation is the driver's derived importance, the mean rating its performance; the importance–performance chart of Martilla and James (1977) splits the drivers at the mean importance and the mean performance into "concentrate here" (important, under-performing), "keep up the good work", "possible overkill" and "low priority". The multiple regression of the score on every driver (least squares by Householder QR, over the respondents complete in all drivers) gives each driver's coefficient with its t test, the standardized coefficient bj sxj/sy, and Pratt's (1987) importance measure, the standardized coefficient times the driver's correlation with the score, which splits R² into one share per driver. Correlated drivers share their importance, so a near-zero coefficient beside a sizeable correlation means the effect is carried by a correlated driver; the Pratt shares can be negative in that case and are then not an allocation. A driver analysis ranks what moves with the score in one sample; whether moving the driver moves the score is a causal question for an experiment or a longitudinal reading.

Worked example on the page

The default file is an illustrative seeded set of 400 responses in three segments (Consumer, Enterprise, SMB) over four quarters with three driver ratings (support, price, quality on 1–10), made with a recorded script so that every section has something to show; it is not survey data, and the benchmark (+30), prior score (+20 from 350 responses) and target half-width (5 points) are illustrative entries. It gives NPS +26.5 (191 promoters, 124 passives, 85 detractors), SE 3.94, Adjusted Wald 95 % interval +18.6 to +34.0 (Wald +18.8 to +34.2); not distinguishable from the +30 benchmark (z = −0.89) nor from the prior +20 (z = 0.99); Enterprise +66.9 against Consumer −8.2 (z = 8.27), X² = 71.05 on 4 degrees of freedom; a step after Q2 (+5.9 to +52.0, z = 6.19); 953 responses for a ± 5-point interval; quality (r = 0.65) and support (r = 0.62) the key drivers, price (r = 0.26) low priority, R² = 0.529. The test script checks the library against Rocks (2016), the seventeen Adjusted Wald widths printed by Sauro and Lewis, their adjusted-Wald proportion examples, Newcombe's Wilson intervals, Agresti's gender × party table (X² = 7.01), the NIST StRD Norris and Longley regressions and a numpy / scipy run on the default file.

Limits of the tool

  • Simple random sampling with optional weights; clustering and stratification enter only through the weights' design effect.
  • Intervals are Wald, Adjusted Wald AW(3, T) and Wilson / Agresti–Coull; the iterative score interval and the bootstrap of Rocks (2016) are not offered.
  • One change point; no seasonal adjustment or time-series model of the periods.
  • Driver analysis by correlation and ordinary least squares; no relative weights (Johnson), dominance analysis, ordinal regression or structural model.
  • Comparisons between segments and periods assume independent samples (different respondents); a panel re-surveyed each period is correlated and its change test is conservative here.
  • No benchmark tables: the analyst enters the benchmark and its source.



References

Reichheld, F. F. (2003). "The One Number You Need to Grow." Harvard Business Review, Vol. 81, No. 12, pp. 46–54.

Rocks, B. (2016). "Interval Estimation for the 'Net Promoter Score'." The American Statistician, Vol. 70, No. 4, pp. 365–372.

Agresti, A., and Coull, B. A. (1998). "Approximate Is Better than 'Exact' for Interval Estimation of Binomial Proportions." The American Statistician, Vol. 52, No. 2, pp. 119–126.

Agresti, A. (2002). Categorical Data Analysis, 2nd ed. Wiley, Hoboken, NJ.

Wilson, E. B. (1927). "Probable Inference, the Law of Succession, and Statistical Inference." Journal of the American Statistical Association, Vol. 22, No. 158, pp. 209–212.

Newcombe, R. G. (1998). "Two-Sided Confidence Intervals for the Single Proportion: Comparison of Seven Methods." Statistics in Medicine, Vol. 17, No. 8, pp. 857–872.

Sauro, J., and Lewis, J. R. (2016). Quantifying the User Experience: Practical Statistics for User Research, 2nd ed. Morgan Kaufmann, Cambridge, MA.

Dixon, M., Freeman, K., and Toman, N. (2010). "Stop Trying to Delight Your Customers." Harvard Business Review, Vol. 88, No. 7–8, pp. 116–122.

Martilla, J. A., and James, J. C. (1977). "Importance-Performance Analysis." Journal of Marketing, Vol. 41, No. 1, pp. 77–79.

Pratt, J. W. (1987). "Dividing the Indivisible: Using Simple Symmetry to Partition Variance Explained." In Pukkila, T., and Puntanen, S. (eds.), Proceedings of the Second International Conference in Statistics, University of Tampere, Tampere, pp. 245–260.

Spearman, C. (1904). "The Proof and Measurement of Association between Two Things." The American Journal of Psychology, Vol. 15, No. 1, pp. 72–101.

Kish, L. (1965). Survey Sampling. Wiley, New York.

Cochran, W. G. (1977). Sampling Techniques, 3rd ed. Wiley, New York.

National Institute of Standards and Technology (2003). Statistical Reference Datasets: Linear Regression (Norris, Longley). NIST Information Technology Laboratory, Gaithersburg, MD.

Worked Examples

Published examples this tool reproduces; each page lists the inputs to enter and the printed answers.

All worked examples


DISCLAIMER: The methods and calculation on aNOWysis.com are provided as means to educate the reader. There is no warranty, expessed or implied, as to the relevance or accuracy of the methods or calculations. All calculations should be checked by a qualified analyst for appropriateness and accuracy. We will not be held liable for any damages, injuries or deaths caused by using methods or calculations from this site.