Uncertainty Budget: Method

Uncertainty Budgets, Coverage Factors and Conformity Decisions

What the budget is

A calibration certificate states a value and an expanded uncertainty, y ± U. The budget is the table behind that statement: every influence that can move the result is listed as an input quantity X_i with its estimate x_i, the standard uncertainty u(x_i) assigned to it, the probability distribution that assignment rests on, the sensitivity coefficient c_i = ∂f/∂x_i of the measurement model and the resulting contribution u_i(y) = c_i u(x_i) to the uncertainty of the result (EA-4/02 M:2022 clause 4, Table 4.1). This tool is that table. It takes the components as typed lines, does the arithmetic of EA-4/02 and the GUM, and writes the statement in the form the calibration certificate needs. It does not propagate uncertainty through a formula: when the sensitivity coefficients have to be computed from a model, use the Uncertainty Propagation tool, which also runs the Monte Carlo method of JCGM 101.

Type A and Type B evaluations

A Type A evaluation uses a statistical analysis of repeated observations (EA-4/02 3.2): for n readings the estimate is the arithmetic mean, the standard uncertainty is s/√n with s the experimental standard deviation of the single readings, and the degrees of freedom are ν = n − 1. When the laboratory knows the repeatability of its procedure from earlier work it uses that pooled standard deviation s_p, divided by √n of the readings actually averaged, and the degrees of freedom are those of the pooled estimate (large, so the tool takes them as infinite unless a value is given). The tool's data rows do the Type A arithmetic from the readings themselves; typeA rows take s and n; pooled rows take s_p and n.

A Type B evaluation uses other knowledge (EA-4/02 3.3): a calibration certificate gives an expanded uncertainty U with its coverage factor k, so u = U/k (normal rows, k = 2 when the certificate does not say); a quoted standard uncertainty is entered as it is (std rows, with its degrees of freedom when known); limits ± a within which the quantity is known to lie are a rectangular distribution with u = a/√3 (rect), a triangular one with u = a/√6 when values near the middle are more likely (tri), a U-shaped (arcsine) one with u = a/√2 for a quantity that swings sinusoidally between its limits, such as a mismatch phase (U-shaped, EA-4/02 S6.8); any other divisor can be entered (custom). Limits set so that the quantity is very unlikely to lie outside them carry infinite degrees of freedom (EA-4/02 Annex E, E2 b); a less reliable limit can be given a finite ν in the last column, as GUM H.1 does.

Combination

For uncorrelated inputs the combined standard uncertainty is the root sum of squares of the contributions, u_c(y) = √Σ u_i(y)² (EA-4/02 eq. 4.1), and the share of each component in u_c² is what the budget is read for: the component with the largest share is the one worth improving. The estimate of the measurand is either entered (the model may be a product, a ratio or anything else: in EA-4/02 S3 the resistor value R_X = 10 000.178 Ω is typed) or, for a direct comparison whose model is a sum of the estimates, left blank and taken as Σ c_i x_i (S2: m_X = m_S + δm_D + δm + δm_C + δB). The tool treats the inputs as uncorrelated; EA-4/02 Annex D and S3.12 show how a correlated pair is reduced to one component before it enters the table (the ratio of two readings on the same instrument, for example), which is how the published budgets are written.

Effective degrees of freedom and the coverage factor

EA-4/02 5.1 states the expanded uncertainty U = k u_c with k = 2, which for a normal distribution of the result corresponds to a coverage probability of 95.45 %, close enough to be called "approximately 95 %". That is justified when at least three comparable, well-behaved contributions make the result normal (the Central Limit Theorem, 5.2) and the standard uncertainty is reliable, which 5.3 takes for granted when no Type A component rests on fewer than ten observations. When few observations enter, the reliability of u_c is measured by the effective degrees of freedom of Welch and Satterthwaite (Annex E, GUM G.4.1):

ν_eff = u_c⁴ / Σ u_i(y)⁴/ν_i

where the Type B terms with infinite ν_i contribute nothing to the sum. ν_eff is truncated to the next lower integer and the coverage factor is the two-sided t-distribution quantile for the coverage probability, k = t_p(ν_eff) (EA-4/02 Table E.1 for 95.45 %, GUM Table G.2 and NIST TN 1297 Table B.1 for 68.27, 90, 95, 95.45, 99 and 99.73 %). With ν_eff = 8 the 95 % factor is 2.31 instead of 1.96, which is why TN 1297 B.2 warns that k = 2 may be inadequate below about ν_eff = 11. The tool reports ν_eff, the t factor and the normal factor and applies the one the k rule asks for: t at ν_eff, the dominant-term rule below, a fixed k = 2 or an entered value.

When one component dominates (EA-4/02 S9.14, S10.13)

The normal distribution of the result is an assumption that fails when a single rectangular contribution outweighs the rest, as the resolution of a low-resolution instrument does (S9, hand-held voltmeter): the result is then essentially rectangular, and for a rectangular distribution the coverage probability is proportional to U, so the 95 % factor is k = p√3 = 1.65, not 2. EA-4/02 S9.14 applies this when the root sum of squares of the other contributions is at most 0.3 of the dominant one. When two rectangular terms dominate (S10, calliper: resolution and mechanical play) their convolution is a symmetric trapezoid with edge parameter β = (a₁ − a₂)/(a₁ + a₂) and k = [1 − √((1 − p)(1 − β²))] / √((1 + β²)/6) (eq. S10.10), 1.83 for β = ⅓. The tool tests both cases on every budget, extends the single-term rule to a dominant triangular (k = √6(1 − √(1 − p)) = 1.90) or U-shaped term (k = √2 sin(pπ/2) = 1.41), reports the factor and applies it when the k rule says so. The 0.3 criterion is applied as written; EA-4/02 S11 uses the trapezoid for a budget whose ratio is 0.34, which the tool therefore treats as normal.

Rounding and the statement

EA-4/02 6.3: the uncertainty is given to at most two significant figures, the result is rounded to the same decimal place, and when rounding down would reduce U by more than 5 % it is rounded up instead. The certificate then carries the statement of 6.1 (k = 2, normal) or 6.2 (k and ν_eff from the t distribution), and the tool prints the matching wording; for a dominant-term factor it uses the wording of S9.12.

Conformity: ratios, guard bands and decision rules

When the result is compared with a specification, the measurement uncertainty decides how sure the pass or fail is. The classic rule of thumb is the test uncertainty ratio of ANSI/NCSL Z540.3, TUR = T/(2U) with T the tolerance span and U the 95 % expanded uncertainty, required to be at least 4:1 when the risk itself is not computed; JCGM 106 7.6 writes the same quantity as the measurement capability index C_m = T/(4u) and notes that such ratios control the risk only loosely. The test accuracy ratio TAR compares the item's tolerance with the reference standard's tolerance or expanded uncertainty.

The decision rule (ISO/IEC 17025:2017 7.1.3, 7.8.6) is agreed with the customer before the work. With simple acceptance (shared risk) the acceptance limits are the tolerance limits and an item measured exactly at a limit has a 50 % probability of being out of tolerance. A guard band w moves the acceptance limits inward, A = T ∓ w; JCGM 106 8.3.2 and ILAC-G8 take w = rU with U at k = 2: r = 1 (the ILAC-G8 and ISO 14253-1 default) keeps the probability of falsely accepting an item measured at the acceptance limit below 2.5 % (2.3 % for a normal distribution), r = 1.5 below 0.16 %, r = 3 below 1 ppm, r = 0.83 below 5 % (ILAC-G8 Table 1). The tool can also set w from a target false-accept probability directly (the confidence-level guard bands of NASA-HDBK-8739.19-4 5.5, which it reproduces to four figures), or move the limit outward for guarded rejection so that an item declared non-conforming is non-conforming with the stated probability (JCGM 106 8.3.3: the nandrolone screening limit A = T + t_0.95;9 s). The non-binary statement of ILAC-G8 4.2.3 reports pass, conditional pass (inside the tolerance but within the guard band), conditional fail (outside the tolerance by less than the guard band) or fail.

Conformance probability and the two kinds of risk

Given the measured value y and standard uncertainty u, JCGM 106 7.2 describes the measurand by a normal distribution about y (a scaled t distribution with ν_eff degrees of freedom when the budget rests on few observations, 7.2.3) and the conformance probability is the part of it inside the tolerance, p_c = Φ((T_U − y)/u) − Φ((T_L − y)/u) (eq. 11; eq. 8 and 9 for one limit). For an accepted item 1 − p_c is the specific consumer's risk, the probability that this particular accepted item does not conform; for a rejected item p_c is the specific producer's risk (9.3.2). These are the figures a laboratory with "meagre prior information" about the item can give. When the population of items is known, a normal process distribution N(y₀, u₀) can be entered: the measured value is then combined with it into the posterior of JCGM 106 A.13 and A.14 (mean weighted by the reciprocal variances, u = u₀u_m/√(u₀² + u_m²)), the guard band from a target risk becomes the Bayesian guard band of NASA-HDBK-8739.19-4 5.4, and the tool integrates the global consumer's and producer's risks R_C and R_P of eq. 17 and 18 (A.15–A.17), the fractions of all items produced that are accepted though non-conforming and rejected though conforming. JCGM 106 9.5.3 (precision resistors, R_C = 1 %, R_P = 7 %) is reproduced in the test script.

The CMC note

A laboratory's calibration and measurement capability is the smallest uncertainty it can state for a nearly ideal item (EA-4/02 1.3). Marking the contributions that belong to the item under calibration (its repeatability, resolution, drift) with a trailing * on the component name lets the tool recompute the budget without them and report the expanded uncertainty the standards, procedure and environment alone would give.

Reading the results

The budget table is the EA-4/02 table with two columns added, the degrees of freedom and the share of u_c²; the last row is the measurand with u_c and ν_eff. Figure 1 shows the shares. Below it come u_c, the Welch–Satterthwaite arithmetic, the coverage factor with its source, U and the rounded statement. With tolerance limits the conformity section gives the ratios, the rule with its guard band and acceptance limits, p_c, the specific risk of the decision, the worst-case risks of the rule at its acceptance limits and the statement of conformity; Figure 2 draws the probability density of the measurand against the tolerance and acceptance limits so that the risk is visible as the area beyond the limits. The CMC note and a summary table close the page.

Limits

Inputs are treated as uncorrelated (reduce correlated pairs to one component as EA-4/02 Annex D and S3.12 do). Every value in the budget must be in the measurand's unit (a certificate's 45 mg is 0.045 when the unit is g). The conformity part assumes a normal (or t) distribution of the measurand and, with a prior, a normal process distribution; other priors (the gamma prior of the JCGM 106 ball-bearing example) are not offered. Second-order terms (a product of two zero-mean quantities, EA-4/02 S4.13) are entered as a ready-made standard uncertainty row. The tool does not store budgets between sessions; the report export keeps the page.