Multi-Criteria Decision (AHP, TOPSIS): Method

Multi-Criteria Decision: AHP, TOPSIS, SAW, WPM, ELECTRE, PROMETHEE, Weight Generators and Sensitivity

The decision matrix

A multi-criteria decision compares n alternatives against m criteria. The decision matrix holds the performance x_ij of alternative i on criterion j in whatever unit the criterion is measured in (dollars, kilograms, a 1 to 10 score). Each criterion is a benefit (more is better, written max) or a cost (less is better, written min), and carries a weight w_j, the weights summing to 1. Every method below turns the matrix, the types and the weights into one score or one order of the alternatives; they differ in how the raw numbers are normalised, how the criteria are combined and whether an alternative is scored on its own or only against the others. The NASA Systems Engineering Handbook (SP-2016-6105, section 6.8) describes this as the weighted trade-off matrix of a trade study and asks that the criteria, the weights, the scales and the sensitivity of the outcome be recorded in the decision report; the Air Force Analysis of Alternatives Handbook asks the same of an AoA. The tool is written for that use: every intermediate table is printed so that the report can be assembled from the page.

The default case is an illustrative trade study of four concepts against cost, mass, range, reliability and lead time with entered weights; it is not a published example. The published cases the tool is checked against are listed at the end.

The analytic hierarchy process

The AHP (Saaty 1980) derives the weights from pairwise comparisons instead of asking for them directly. For every pair of criteria the decision maker states how many times more important criterion i is than criterion j on the fundamental scale: 1 equal importance, 3 moderate, 5 strong, 7 very strong, 9 extreme importance, with 2, 4, 6 and 8 between (the scale is reproduced in NISTIR 7385 Table 3.1). The judgements fill a matrix A with a_ij on the scale, a_ji = 1/a_ij and a_ii = 1, so only the m(m − 1)/2 entries above the diagonal are independent; the tool accepts the upper triangle alone or the full matrix, and fractions such as 1/3.

If the judgements were perfectly consistent, a_ij = w_i/w_j for some weight vector w, the matrix would have rank one, A w = m w, and w would be read from any column. Real judgements are not consistent, and Saaty's prescription is the principal right eigenvector: A w = λ_max w with Σ w = 1. The tool solves it two ways. The power method iterates w ← A w / Σ(A w) until the vector stops changing (the matrix is positive, so the iteration converges to the Perron vector; λ_max follows from the mean of (A w)_i/w_i). The exact solve finds λ_max as the largest real root of the characteristic polynomial det(A − λI) = 0, bracketed between m (λ_max ≥ m for any positive reciprocal matrix) and the largest row sum (a bound on the Perron root) and closed by bisection to machine precision, the determinant being evaluated by LU factorisation; the vector then comes from inverse iteration with that shift. The two agree to the printed digits on every case, and both are tabulated beside the row geometric mean (Π_j a_ij)^1/m, normalised, which NASA's own AHP paper (Ivanco, Hou and Michaeli) uses for its ease of implementation, and the column-normalisation average, the approximate method of the older AHP programs (the INEL manual calls it the Saaty normalisation). For a consistent matrix all four coincide; the geometric mean and the eigenvector differ in the third decimal for ordinary inconsistency, and NISTIR 7385's Level 1 priorities (0.3339, 0.1289, 0.4688, 0.0684 in its Table 5.4) are in fact the geometric-mean values of its Table 5.1 matrix. Either vector can be carried forward with the Priorities select.

Consistency is measured by how far λ_max exceeds m: the consistency index CI = (λ_max − m)/(m − 1) is compared with the random index RI, the mean CI of randomly filled reciprocal matrices of the same size, and the consistency ratio CR = CI/RI should not exceed 0.10 (Saaty); NISTIR 7385 adds that 0.10 to 0.15 is tolerable but worth a review of the judgements. Saaty's RI table (0.58, 0.90, 1.12, 1.24, 1.32, 1.41, 1.45, 1.49 for m = 3 to 10) is the one reprinted in NISTIR 7385 and the INEL manual and is the default; the NASA paper prints Dijkstra's simulated set for both the eigenvector and the geometric-mean methods, offered as the alternative. The tool also prints λ_max in the column-sum form Σ_j w_j Σ_i a_ij of NISTIR 7385 Appendix A.3.2, which equals the eigenvalue exactly when w is the eigenvector. When CR exceeds 0.10 the table of a_ij w_j/w_i shows which judgements depart most from the ratios implied by the whole matrix; those are the ones to reconsider.

Several judges are combined by the geometric mean of their judgements element by element (aggregation of individual judgements), which keeps the matrix reciprocal; each judge's own priorities and CR are listed so that a judge whose comparisons are inconsistent or far from the group can be seen. The NASA paper describes this aggregation and the alternative of averaging the priorities instead.

The alternatives are then prioritised under each criterion. Where a criterion is measured, the local priorities are the normalised values, x_ij/Σ_i x_ij for a benefit and (1/x_ij)/Σ_i(1/x_ij) for a cost, the scoring NISTIR 7385 uses for the investment costs in its Table 5.3. Where a criterion is judged, a comparison matrix of the alternatives is entered under the criterion's name and treated like the criteria matrix (its CR is printed). The synthesis is the weighted sum of the local priorities with the criteria priorities as weights (the distributive mode, every column summing to 1, as in NISTIR 7385 Table 5.4 and the INEL output). The tool also prints the ideal mode, each column divided by its largest entry before weighting, because the distributive mode can reverse the order of two alternatives when a third is added or removed (Belton and Gear 1983): re-normalising the columns lets an irrelevant alternative shift the others' priorities, and the ideal mode does not. The consistency of the whole hierarchy, (CI_criteria + Σ w_j CI_j)/(RI_criteria + Σ w_j RI_j), is the figure the INEL manual prints as the consistency of the hierarchy.

Weighted sum and weighted product

The simple additive weighting (SAW) score is S_i = Σ_j w_j r_ij with the linear normalisation r_ij = x_ij/max_i x_ij for a benefit and min_i x_ij/x_ij for a cost, so the best alternative on every criterion scores 1. It is the weighted trade-off matrix of the handbooks and the most transparent method, and it assumes the criteria are mutually preferentially independent and that a shortfall on one can be compensated by a surplus on another. The weighted product (WPM) replaces the sum by the product, P_i = Π_j r_ij^w_j; it is dimensionless by construction (the ratio form P_k/P_l = Π (x_kj/x_lj)^±w_j does not depend on the normalisation) and punishes a very poor score on any one criterion more than the sum does, which is often the behaviour wanted of a trade study.

TOPSIS

The technique for order preference by similarity to ideal solution (Hwang and Yoon 1981) normalises each column by its Euclidean norm, r_ij = x_ij/√Σ_i x_ij², multiplies by the weights, v_ij = w_j r_ij, and forms the ideal solution A⁺ from the best v in each column (largest for a benefit, smallest for a cost) and the anti-ideal A⁻ from the worst. The separations S_i⁺ and S_i⁻ are the Euclidean distances of alternative i from the two, and the closeness coefficient C_i = S_i⁻/(S_i⁺ + S_i⁻) ranks the alternatives: 1 for the ideal, 0 for the anti-ideal. The NASA Langley study of propulsion-airframe aeroacoustics technologies (Burg, Hill, Brown and Geiselhart 2004) prints these steps for a subset of its Pugh matrix, and the tool reproduces its normalised values, weighted values, ideal vectors and closeness to the printed digits. Because the ideal and anti-ideal are taken from the alternatives present, TOPSIS too can reorder two alternatives when a third is added.

ELECTRE I

The outranking methods of Roy abandon a single score. For every ordered pair (a, b) ELECTRE I forms the concordance c(a, b), the total weight of the criteria on which a is at least as good as b (a measure of the evidence for "a is at least as good as b"), and the discordance d(a, b), the largest weighted shortfall of a against b on any criterion divided by the largest weighted difference between the two on any criterion, in the Hwang-Yoon form on the TOPSIS-weighted values (a measure of the evidence against). a outranks b when c(a, b) ≥ c* and d(a, b) ≤ d*; the thresholds are entered, or by default are the means of the two matrices. The kernel is the set of alternatives that no other alternative outranks: ELECTRE I selects a short list rather than ranking, and alternatives in the kernel are incomparable at the chosen thresholds. Tightening c* or loosening d* changes the relation, so the tool prints both matrices with the cells that pass in bold. The comparison table orders the alternatives by the net count of outranking relations (outranks minus outranked by), which is coarse; when the relation has a cycle the page says so.

PROMETHEE II

PROMETHEE (Brans and Vincke 1985) compares every pair on every criterion through a preference function P_j(d) of the oriented difference d = ±(x_aj − x_bj), positive when a is better: the usual criterion (any advantage counts fully), the U-shape (an advantage above an indifference threshold q), the V-shape (linear up to a preference threshold p), the level criterion (one half between q and p, one above p), the linear criterion (zero to q, linear to p) and the Gaussian (1 − exp(−d²/2s²)). The tool takes q, p and s as fractions of each criterion's range across the alternatives so that one pair of numbers serves every criterion. The aggregated preference π(a, b) = Σ_j w_j P_j(a, b) gives the leaving flow φ⁺(a), the mean of π(a, b) over the other alternatives, the entering flow φ⁻(a), the mean of π(b, a), and the net flow φ = φ⁺ − φ⁻ between −1 and 1; PROMETHEE II ranks the alternatives by φ (a complete order; PROMETHEE I, which keeps a and b incomparable when the two flows disagree, is not printed).

Weight generators

The entered weights are normalised to 1. The entropy weights (Shannon's measure applied to the matrix, Hwang and Yoon) use p_ij = x_ij/Σ_i x_ij, E_j = −(1/ln n) Σ_i p_ij ln p_ij (1 when every alternative scores the same) and w_j = (1 − E_j)/Σ_k(1 − E_k): a criterion that separates the alternatives more is given more weight, which is a property of the data rather than of the decision maker's values and should be read as such; the dispersion does not depend on whether the criterion is a benefit or a cost, so cost columns are used as entered. The rank-order centroid weights (Barron and Barrett 1996) need only the order of importance: for the criterion ranked k of m, w_k = (1/m) Σ_i=k^m 1/i, the centroid of the simplex of all weight vectors consistent with that order; the tool takes the order in which the criteria are listed. Equal weights complete the table. The set in use is chosen with the Weights From select; all five are printed.

Comparison and sensitivity

The methods are compared by the rank each gives every alternative and by Spearman's rank correlation with the weighted-sum ranking. Agreement across methods is the usual evidence that the choice is robust to the method; disagreement means the choice turns on a feature the methods treat differently (compensation between criteria, the ideal point, the thresholds), which the report should say. The sensitivity section varies the weight of one criterion from 0 to 1 with the other weights scaled in proportion. The weighted-sum score of each alternative is a straight line in that weight, so every crossing of two alternatives is found exactly and the intervals in which each alternative leads are stated; the TOPSIS closeness is sampled at 101 points with its crossings interpolated. The distance from the current weight to the nearest change of leader is the margin the decision has against an argument about that weight.

Checks

The library is checked against an independent numpy implementation on seeded consistent and inconsistent comparison matrices (eigenvector and eigenvalue against numpy.linalg.eig, CR = 0 for a consistent matrix, group aggregation) and seeded decision matrices (SAW, WPM, TOPSIS, ELECTRE, PROMETHEE with all six preference functions, entropy and centroid weights, sensitivity crossings), and against the published public cases: NISTIR 7385 (Lippiatt and Fuller 2007) Tables 5.1 to 5.5 and A.4, the Forest Service paper of Schmoldt, Peterson and Smith (its 5 × 5 and 3 × 3 matrices), the Idaho National Engineering Laboratory AHP 5.1 manual's car example (every priority, λ_max, CI, CR and the consistency of the hierarchy to the printed four decimals), the NASA Langley TOPSIS subset and the NASA paper's User 6 matrix and group synthesis table. The misprints found in those sources are recorded in the test script and the references README.