Risk Register and Quantitative Risk: Method

Risk Register and Quantitative Risk

What the tool does

A risk register lists the events that are not in the project's baseline plan but could land on it: each with a probability of occurring, an impact on cost and on schedule if it does, and a planned response. The tool takes such a register as pasted text and works it three ways. First the qualitative scoring that programme reviews use: a likelihood level and a consequence level for every risk, its cell on a 5 × 5 reporting matrix, and a ranking. Second the expected monetary value (EMV) of every risk before and after its response, which says what the register is expected to cost and whether each response is worth its price. Third a Monte Carlo simulation of the whole register, which gives the distribution of the total cost impact and of the total delay, the contingency at a chosen confidence level, the risks that drive that level, and the joint behaviour of cost and schedule. The scales, the matrix and the Monte Carlo rule follow public U.S. government guidance: the DoD Risk, Issue, and Opportunity Management Guide for Defense Acquisition Programs (2017), the GAO cost and schedule assessment guides and the NASA Cost Estimating Handbook.

The register

One risk per line, fields separated by vertical bars or tabs, in the order id, risk statement, category, probability, cost impact, schedule impact, response, response cost, probability after the response, cost impact after, schedule impact after, performance consequence level and expected closure month. A line that names the columns (id, risk, probability, cost, days, response, …) may come first and then sets their order. A probability is written as 0.30, 30 % or 30; with the "likelihood level" entry mode an integer 1 to 5 is converted to the midpoint of its band. An impact is three numbers low/likely/high under the chosen shape, two numbers for a uniform range, one number for a fixed value, and blank or 0 for none. The response is one of the four options of the DoD guide, accept, avoid, transfer or control (mitigate is read as control): an accepted risk keeps its values and costs nothing, an avoided risk has probability 0 unless a residual probability is given, and blank post-response fields keep the pre-response values. Money is a plain number in one currency unit; days are whatever days the baseline duration is counted in.

Impact distributions

A three-point impact a / m / b is a triangular distribution (mean (a + m + b)/3, variance (a² + m² + b² − am − ab − mb)/18), a PERT distribution (a beta distribution on [a, b] with shape parameters 1 + 4(m − a)/(b − a) and 1 + 4(b − m)/(b − a), mean (a + 4m + b)/6, variance (μ − a)(b − μ)/7; it puts less weight on the extremes than the triangle) or a uniform distribution between a and b. The quantile and distribution functions of the triangle and the uniform are closed forms; the PERT quantile is tabulated from the regularized incomplete beta function. Within one risk the cost and the schedule impact are drawn from the same uniform number, so a delay at its 90th percentile comes with a cost at its 90th percentile; the two are not independent in practice, and treating them so would understate the joint tail.

Qualitative scoring on the DoD scales

The DoD guide's Table 3-2 sets five likelihood levels by probability bands: 1 not likely (above 1 % to 20 %), 2 low likelihood (to 40 %), 3 likely (to 60 %), 4 highly likely (to 80 %), 5 near certainty (to 99 %); a probability of 100 % is an issue, not a risk. The bands are entered as five upper limits, so another agency's scale can be used. Its Table 3-1 sets five consequence levels for cost (level 5 a 10 % or greater increase over the baseline, level 4 a 5 to 10 % increase, level 3 1 to 5 %, level 2 under 1 %, level 1 minimal), with qualitative criteria for schedule and performance. The tool applies the cost criteria to both cost and schedule as the most likely impact divided by the baseline cost or duration, with four entered lower limits (the default 0.1, 1, 5 and 10 %; the guide gives no number between minimal and minor, so 0.1 % is the tool's choice), and takes an entered performance level per risk. As the guide requires, a risk's consequence is the greatest of its cost, schedule and performance levels, and its risk level comes from the reporting matrix of Figure 3-5, a fixed pattern of green, yellow and red cells (likelihood 5: low, moderate, high, high, high across consequence 1 to 5; likelihood 4: low, moderate, moderate, high, high; 3: low, low, moderate, moderate, high; 2: low, low, low, moderate, moderate; 1: low, low, low, low, moderate). The tool reproduces the guide's Figure 3-6 count of 80 register risks (14 red, 29 yellow, 37 green) from that pattern. A second option colours the matrix by the product of the two levels against two entered thresholds, the simpler rule that many organisations use. The score L × C ranks the register, ties broken by EMV, and the ranking is only a first ordering: the guide prioritises by the cost-effectiveness of the responses, the time frame and the interrelation of risks as well.

Expected monetary value

The EMV of a risk is its probability times its mean impact, EMV = p (μ_cost + μ_days × c_day), where c_day is the entered standing cost per day of delay. The DoD guide asks that a schedule risk be burdened with a reasonable share of the programme's burn rate; with c_day = 0 the cost and schedule sides stay separate. The guide's Table 3-3 works the method for four risks: a 20 % risk of $10 M has a risk-weighted consequence of $2 M, costs $1 M to mitigate and returns $1 M (1 : 1); a 70 % risk of $10 M returns $6 M for $1 M (6 : 1); a 40 % risk of $36 M returns $12.4 M for $2 M; a 60 % risk of $5 M returns $2.5 M for $0.5 M (5 : 1). The tool prints the same quantities for every treated risk: the benefit EMV − EMV′ from the post-response probability and impacts, the net value benefit − response cost, and the return ratio net value / response cost; a negative net value flags a response that costs more than it saves. The sum of the EMVs is the expected cost of the register, which is what a management reserve covers on average; it is not a contingency, because the register's total is far from its mean in most iterations.

Monte Carlo simulation

Each iteration decides, for every risk, whether it occurs and what it costs. The occurrence rule is the one NASA's Cost Estimating Handbook describes in Appendix G: a random number per risk, the consequence added when the number falls below the likelihood. The tool draws a standard-normal score z for each risk and lets it occur when Φ(z) < p; a risk that occurs draws its impacts from one uniform number through the quantile functions; the total cost impact is the sum over the occurring risks of cost plus days × c_day, and the total delay the sum of their days. The post-response case re-uses the same scores and uniform numbers with p′ and the post-response impacts, so the two S-curves differ only through the responses, not through sampling noise. The iterations are sorted into the cumulative distribution (the S-curve) of GAO-20-195G Chapter 12, and the percentiles P10 to P95 are read from it.

Correlated occurrences. Risks that share a cause (one supplier, one technology) tend to occur together, and ignoring that understates the tail (GAO-20-195G: correlation matters at the 80th percentile, little at the 50th). For entered pairs the scores are made correlated with the given coefficients through the Cholesky factor of the correlation matrix, a Gaussian copula on the occurrences: the joint probability of two risks with marginal probabilities p₁, p₂ and score correlation ρ is the bivariate normal probability Φ₂(Φ⁻¹p₁, Φ⁻¹p₂; ρ), and the tool reports the achieved score correlation and joint frequency of the first pair. A set of pairs that is not positive definite is refused rather than repaired. The coefficient is the correlation of the latent scores, not of the 0/1 occurrences, whose correlation is smaller (0.33 for p = 0.40, 0.35 and ρ = 0.5).

Schedule. Summing the delays of all occurring risks treats every risk as sitting on the critical path, which is an upper bound; the delay of a risk on a path with float may not reach the finish date at all. A schedule risk analysis proper puts the risks on the network and simulates it (GAO-16-89G Best Practice 8, where the house-construction example's deterministic February 10 finish had a 7 % chance with three-point durations and under 1 % with risk drivers); the Project Schedule tool does that. The register's delay distribution is still the right input for a risk-driver analysis and a fair reserve for a register whose risks fall on one chain of work.

Contingency and reserve

GAO-20-195G defines the contingency as the difference between the cost at management's confidence level and the point estimate, read off the S-curve (its Figure 14: $908,000 at 50 % against an $825,000 point estimate gives $83,000; $1,096,000 at 70 % gives $271,000). The register's impact distribution starts at zero, so the total impact at the chosen level is the contingency and the baseline plus it the risk-adjusted budget at that level; the part of it above the EMV is the allowance for the tail. The tool prints the same for the schedule and, when responses are entered, the budget after the responses including their cost, so the plan can be judged at the P-level as well as by EMV: a response that removes a large, unlikely impact changes the EMV little and the P90 a lot. GAO names no best-practice level: budgeting to the mean is common, 70 or 80 % is used for high-risk programmes, and funding every project of a portfolio at a high level can make the portfolio unaffordable.

Drivers

Two sensitivity measures are printed. The first is the tornado of the chosen level: the P-level of the total with each risk removed from the register, every other risk kept; the reduction is the most that treating the risk could buy at that level. The removals do not add up to the P-level because a percentile is not additive. The second is the correlation (Pearson and Spearman) of each risk's contribution, occurrence × impact, with the total, the index that GAO-16-89G shows in its Figures 39 and 40 and that risk software reports. A risk with a modest EMV can rank high in the tornado because the tail, not the mean, sets the contingency. The last figure, when closure months are entered, is the expected-exposure burn-down: the EMV of the risks still open at the end of each month, which the DoD guide expects to fall early for high-consequence risks.

Limits

Probabilities and impacts are the analyst's; the tool cannot judge them, and a register with optimistic ranges gives an optimistic S-curve (GAO: collect the data anonymously, and expect experts to cover about 70 % of the true range). Risk interdependence is modelled only through the occurrence correlation of entered pairs; one risk causing another, or an impact that depends on which other risks occurred, is not. Opportunities (negative impacts) are not entered. The schedule total is a critical-path upper bound. The matrix classification and the levels are ordinal, and the guide warns against fractional scores; the EMV and the simulation are where the numbers belong.