Warranty Forecasting

Warranty Forecasting

Warranty Forecasting: Nevada Chart, Censored Likelihood, Conditional Forecast, Bounds and Back-Test

What the tool answers

How many units will come back in each of the next periods, how many before the warranty runs out, and what that costs. The answer is built from two things: the distribution of the age at which a unit fails, estimated from the returns seen so far, and the ages of the units now in the field. The method is the risk analysis of the Air Force Weibull Analysis Handbook (AFWAL-TR-83-2079, chapter 3) applied to shipment and return records.

The Nevada chart

Warranty records are usually kept as a table with one row per shipment period and one column per calendar period, each cell holding the units of that shipment that were returned in that period. The filled cells form a staircase, which gave the chart its name. A cell says two things about each unit in it: its age when it failed lay between the age the shipment had at the start of the return period and the age it had at the end; and nothing more precise than that. A unit that appears in no cell has not failed by the age its shipment has reached at the end of the chart: it is a suspension (a right-censored life). The tool makes this conversion and prints it as a table of returns by age interval and unfailed units by age.

Two conventions for the first column are offered. When the first column of a row is the period after the shipment, units are counted in service from the end of their shipment period: returns in the next period have ages 0 to 1, and a shipment i periods before the end of the chart has unfailed units of age i. When the first column is the shipment period itself, units are taken to enter service at the middle of that period: the first cell covers ages 0 to 0.5, the following cells 0.5 to 1.5, 1.5 to 2.5 and so on. If the units reach customers some periods after they ship, shift the shipment rows accordingly before pasting; the lag between shipment and sale is not estimated.

The life-table column of the first table is a check that needs no distribution: for each age interval, returns divided by the units that were under observation at that age; the running product of the complements gives the fraction returned by each age.

Fitting the life distribution

Two distributions are fitted. The Weibull, F(t) = 1 − exp[−(t/η)^β], has the shape β (below 1 for early failures, about 1 for failures at a constant rate, above 1 for wear-out) and the characteristic life η, the age by which 63.2 % fail. The lognormal, F(t) = Φ[ln(t / median)/σ], has the median life and the standard deviation σ of the logarithm of life. Both are fitted by maximum likelihood. The likelihood is the product of one term per unit (NIST/SEMATECH e-Handbook, section 8.4.1.2): F(t_hi) − F(t_lo) for a unit returned in the age interval from t_lo to t_hi, the density f(t) for a failure at a known age, and 1 − F(t) for a unit unfailed at age t. The tool maximizes its logarithm by a simplex search followed by Newton steps on the exact gradient.

The option "Exact: failed at the end of the period" replaces the interval term by the density at the end of the interval. It is the convention of analyses that list a Nevada chart as failures at 1, 2, 3 ... periods. With many periods the two treatments nearly agree. With few they do not: nine, seven and five returns at one, two and three months among 390 units give β = 2.49, η = 6.70 months as exact times and β = 1.39, η = 15.4 months as intervals, because "failed during the first month" is far weaker evidence of a late failure than "failed at exactly one month". The interval treatment is the default because it is what the data say.

The 2 × 2 covariance matrix of the estimates is the inverse of the negative second-derivative matrix of the log-likelihood at its maximum (the observed Fisher information). Bounds on the parameters are formed on the logarithms, β exp(± z s_β/β) and likewise for η, σ and the median, with z the standard normal value for the chosen two-sided confidence. The table also gives each distribution's log-likelihood and AIC = 2 × 2 − 2 ln L. Warranty data are heavily censored: typically a few percent of the units have failed, all at young ages. Two distributions that fit those ages equally well can disagree by a factor of two on the fraction failed at the end of a long warranty, and the AIC cannot settle that. Engineering knowledge of the failure mode has to.

The forecast

A unit that has reached age t without failing fails between the later ages a and b with probability

[F(b) − F(a)] / [1 − F(t)]

(the handbook's equation 3.2 for a = t). For each shipment period the tool multiplies this by the number of unfailed units and adds the shipments together; that is the expected number of returns in each future period. Summed over the periods the terms telescope to [F(t + H) − F(t)] / [1 − F(t)] per unit, the probability of failing within the whole horizon. Planned shipments are added as units of age zero in their period and contribute F(b) − F(a). A warranty length W removes a unit from the population at risk when it reaches age W; in the fit, a unit older than W that never came back is counted as unfailed at W, since a failure after that would not have been reported. The line "to the end of the warranty" is the same sum taken until every unit has reached W.

The fleet age table is the same calculation with the ages given directly: N units of age t each, a usage of u per period, and a distribution that is either entered (for example from a Weibull plot) or fitted to the listed failure ages with the table as suspensions. Its first table is the handbook's present risk, the sum of N F(t), and its second the future risk in the handbook's layout. Units added in a future period are taken to enter at the middle of that period.

Failed units are not followed after repair or replacement. The handbook gives the range of validity: the simple sum is adequate while the probability of failure of a unit within the horizon is below about 0.5; beyond that, second failures of replaced units matter and a renewal calculation or a simulation is needed.

Bounds on the forecast

Two sources of variation are combined. The first is the uncertainty of the fitted parameters: the forecast N is a function of the two parameters, its gradient g is found by small central differences, and its parameter variance is g^T V g with V the covariance matrix above (the first-order, or delta, method). The second is chance: even with the true distribution known, the number of failures among n units each failing with probability p is binomial with variance n p (1 − p). The bounds printed are quantiles of the gamma distribution with mean N and variance s², the sum of the two variances (shape N²/s², scale s²/N), so they are bounds on the count that will be observed, not only on its expected value. The gamma distribution is used because a count cannot be negative and is skewed when small: with chance variation alone it has the mean and variance of the Poisson count, and for large counts it becomes the normal band N ± z s. The bounds are approximate. In a simulation of 200 charts of fourteen shipment periods drawn from a known Weibull distribution, the nominal 90 % bounds on the returns of the next three periods contained the actual count in about nine cases of ten (the figure is recorded with the tool's reference calculations). They do not cover a wrong choice of distribution, a second failure mode, a design change part-way through the chart or a change in how customers use the product. When the parameters are entered instead of fitted, only the chance variation is in the bounds.

The reserve is the cost per claim times the one-sided upper bound at the chosen confidence: the amount that covers the claims of the forecast horizon with that confidence, under the same assumptions.

Seasonal and reporting adjustments

Both are entered factors; the tool does not estimate them. Seasonal factors are multipliers on the forecast returns, one per period of a repeating cycle (twelve for monthly data), with the position of the first forecast period in the cycle; they do not enter the fit. Reporting completeness corrects for claims that occurred but have not yet reached the database: the fraction already reported is entered for the latest period, the one before it, and so on, and the counts of those calendar periods are divided by these fractions before the fit. Fractions of this kind are obtained by comparing the same chart extracted at successive dates. Without the correction the latest periods look better than they are, and the fitted shape is biased low.

Back-test

The best test of a forecast is one whose answer is already known. With k periods held out, the chart is cut k columns short, the distribution is fitted to what was known then, and the k periods are forecast and set beside the returns that actually arrived, shipments made during those periods being taken as known. Errors inside the bounds are what the method promises. Errors that are consistently of one sign point to a cause the model lacks: a mixed population, reporting delay, seasonality, or a distribution that bends the wrong way.

What is not modeled

Two-dimensional warranties, limited by age and by usage (36 months or 36,000 miles, whichever comes first), need the distribution of usage rates among customers; the tool forecasts in one scale only, time in service for a Nevada chart or accumulated usage for a fleet table. Other things left out: repeated claims on the same unit, more than one failure mode (fit each mode separately, counting the returns for other modes as unfailed units), sub-populations by design level or plant, the sales lag, and discounting of future cost.

Checks

The fleet calculations reproduce the handbook's Tables 3.1, 3.2, 3.3, 3.5 and 3.6. The tables were computed by the handbook's authors with unrounded parameters, so with the rounded values printed on its figures (β = 2.3, η = 6,487 hours for the bearing cage) the tool gives 11.56 failures in twelve months where Table 3.3 prints 11.613; with β = 2.2975, η = 6,485 every cell of the table agrees within one unit of its last printed digit. The handbook's line is a rank regression read from its Weibull plot. Fitting the same six failures and 1,703 unfailed bearings by maximum likelihood (leave the shape and scale blank) gives β = 2.03, η = 11,855 hours and 5.1 failures in the twelve months instead of 11.6, with 90 % bounds of 1.1 to 11.5: with six failures among 1,703 units the two estimation methods disagree by a factor of two, and the bounds say as much. The maximum-likelihood fit reproduces the Weibull example of the NIST/SEMATECH e-Handbook, section 8.4.1.3 (β = 1.7208, η = 606.53 hours, ln L = −75.135) and, with the exact-age option, the parameters of the ReliaWiki Nevada chart example (β = 2.4928, η = 6.6951 months, about 29 returns forecast for the next month). Fits, covariance matrices, forecasts and bounds are compared with an independent implementation on twenty simulated charts.