Meta-Analysis: Method

Meta-Analysis

What is pooled

A meta-analysis combines the results of several studies of the same question into one estimate with its uncertainty. Each study contributes an effect y_i and the variance v_i of that effect; the page computes both from the numbers on each data line. Ratio measures are analysed on the logarithmic scale and correlations on Fisher's z scale, where their sampling distributions are close to normal with a variance that does not depend on the effect, and every result is transformed back for display.

With a events among n₁ participants in group 1 (treated) and c among n₂ in group 2 (control), b = n₁ − a, d = n₂ − c: the risk ratio is (a/n₁)/(c/n₂) with var(ln RR) = 1/a − 1/n₁ + 1/c − 1/n₂; the odds ratio is ad/(bc) with var(ln OR) = 1/a + 1/b + 1/c + 1/d; the risk difference is a/n₁ − c/n₂ with variance ab/n₁³ + cd/n₂³. For continuous outcomes the mean difference is m₁ − m₂ with variance s₁²/n₁ + s₂²/n₂ (the standard deviations are not pooled). The standardized mean difference is Hedges' g = J (m₁ − m₂)/s_p, where s_p is the pooled standard deviation and J removes the small-sample bias of Cohen's d. Two published forms are offered: the exact factor J = Γ(ν/2)/[√(ν/2) Γ((ν − 1)/2)] with ν = n₁ + n₂ − 2 and variance 1/n₁ + 1/n₂ + g²/(2N), and the Review Manager form J = 1 − 3/(4N − 9) with variance N/(n₁n₂) + g²/(2(N − 3.94)). They differ in the third decimal for all but the smallest studies. A correlation r from n pairs becomes z = ½ ln[(1 + r)/(1 − r)] with variance 1/(n − 3). The generic input takes any estimate with its standard error, or with its 95 % limits, from which the standard error is (upper − lower)/(2 × 1.96) on the scale analysed; this is the route for hazard ratios, adjusted estimates and results of cluster or crossover trials.

Empty cells

A study with no events in one group has an infinite or undefined log ratio. The usual remedy adds 0.5 to each of its four cells. The page offers that rule (Review Manager, metafor), the same increment for every study as soon as one has an empty cell, the variant of R's meta package for the risk ratio (0.5 added to the events and once to each total), or no correction, in which case such studies drop out of the inverse-variance and random-effects analyses. Studies with no events in either group, or events in every participant, say nothing about a ratio and are always left out of risk ratios and odds ratios; they are kept for the risk difference, whose variance alone uses the corrected cells. Continuity corrections bias the result toward no effect when events are rare; the Mantel–Haenszel and Peto methods need them less or not at all.

Fixed-effect estimates

The fixed-effect (common-effect) model assumes one true effect. The inverse-variance estimate is θ = Σ w_iy_i/Σ w_i with w_i = 1/v_i and standard error 1/√Σ w_i. For 2 × 2 tables the Mantel–Haenszel estimates pool the tables themselves: OR_MH = Σ(ad/N)/Σ(bc/N), RR_MH = Σ(a n₂/N)/Σ(c n₁/N) and RD_MH = Σ[(a n₂ − c n₁)/N]/Σ(n₁n₂/N), N = n₁ + n₂, with the variance formulas of Robins, Breslow and Greenland (odds ratio) and of Greenland and Robins (risk ratio, risk difference) as printed in the Cochrane algorithms document. They remain well behaved when the individual tables are small. Peto's one-step odds ratio is exp[Σ(O − E)/Σ V] with O − E = a − n₁(a + c)/N and the hypergeometric variance V = n₁n₂(a + c)(b + d)/[N²(N − 1)]; it is good for rare events in balanced trials with small effects and biased otherwise. The results table lists the chosen method first and the others beside it.

Random-effects estimate

The random-effects model lets the true effects of the studies differ, normally distributed about a mean μ with variance τ². The estimate of μ is the same weighted mean with w_i = 1/(v_i + τ²), so that small studies gain weight as τ² grows. Four estimators of τ² are implemented. DerSimonian–Laird: τ² = max[0, (Q − (k − 1))/C] with C = Σ w − Σ w²/Σ w from the fixed-effect weights. Paule–Mandel: the value at which the generalized statistic Q(τ²) = Σ(y_i − μ)²/(v_i + τ²) equals its expectation k − 1. Maximum likelihood and restricted maximum likelihood: the maximum of the (restricted) normal log-likelihood over τ² ≥ 0, found here by locating the zero of its derivative; REML allows for the estimation of μ and is the default of Review Manager and of metafor. The page uses the chosen estimator and tabulates the result under all four.

The Wald interval is μ ± z SE with SE = 1/√Σ w_i. The Hartung–Knapp (Sidik–Jonkman) interval multiplies SE² by q = Σ w_i(y_i − μ)²/(k − 1) and uses a t distribution on k − 1 degrees of freedom; it has better coverage when studies are few or τ² is uncertain, but with τ² estimated as zero it can be narrower than the Wald interval, and with two or three studies very wide. The prediction interval μ ± c √(τ² + SE²) describes where the true effect of a new similar study is expected to lie; c is a t quantile with k − 1 degrees of freedom (Cochrane Handbook), with k − 2 (Higgins, Thompson and Spiegelhalter) or the normal quantile, by choice.

Heterogeneity

Cochran's Q = Σ w_i(y_i − θ)² with inverse-variance weights follows χ² on k − 1 degrees of freedom when all studies share one effect. I² = (Q − df)/Q, truncated at zero, is the share of the variation among the estimates that is not chance; H = √(Q/df). The page also gives I² = τ²/(τ² + s²), where s² = (k − 1)Σ w/[(Σ w)² − Σ w²] is the typical within-study variance; with the DerSimonian–Laird τ² the two coincide. The confidence interval of τ² is the Q-profile interval: the values of τ² at which the generalized Q equals the upper and lower χ² quantiles. Q has little power with few studies and too much with many large ones, so the size of τ and the prediction interval say more than its p value.

Subgroups

With a subgroup label on each line the analysis is repeated within the subgroups, either with a between-study variance of its own for each subgroup or with one variance pooled over them (estimated from the scatter of the studies about their subgroup means, which is the meta-regression on a categorical moderator). The test for subgroup differences is the heterogeneity statistic of the subgroup estimates, Q_between = Σ(θ_g − θ̄)²/SE_g², on G − 1 degrees of freedom, given for the fixed-effect and for the random-effects subgroup estimates.

Small-study effects

The funnel plot shows each effect against its standard error. Without selection and without heterogeneity the points scatter symmetrically inside the funnel θ ± z SE. Egger's test regresses the effect on its standard error with weights 1/v and a multiplicative dispersion and tests the slope with a t distribution on k − 2 degrees of freedom (the same test as the intercept of the regression of y/SE on 1/SE). Begg and Mazumdar's test is Kendall's rank correlation between the standardized deviates (y_i − θ)/√(v_i − 1/Σ w) and the variances, with the exact null distribution when there are no ties. Both tests are weak with fewer than about ten studies, and for odds ratios the estimate and its standard error are correlated by construction, which inflates false positives when effects are large.

Trim and fill (Duval and Tweedie) estimates how many studies are missing from one side of the funnel. The effects are centred on the pooled estimate and ranked by absolute value; with S the sum of the ranks on the long side, L₀ = [4S − k(k + 1)]/(2k − 1), and R₀ = γ − 1 with γ the length of the run of largest ranks on that side. The k₀ most extreme studies are trimmed, the centre is re-estimated and the count repeated until it settles; the trimmed studies are then restored together with their mirror images about the final centre, and the analysis is repeated on the augmented set. The side is chosen from the sign of Egger's slope unless given. The result is a sensitivity analysis, not a correction.

Leave-one-out and cumulative analyses

The leave-one-out table repeats the whole analysis k times, each time without one study, and shows how much any single study drives the estimate and the heterogeneity. The cumulative analysis adds the studies one at a time, in the order of entry, of the sort column (usually the year) or of precision, and shows when the evidence settled.

Checks

The numbers agree with R's metafor and meta packages on their published data sets. For the BCG vaccine trials that open the page: random-effects log risk ratio −0.7145 (SE 0.1798), τ² = 0.3132, Q = 152.23, the leave-one-out table, Egger's t = −1.40 and four studies filled by trim and fill, as printed in Viechtbauer (2010); Mantel–Haenszel risk ratio 0.635 (0.588 to 0.686) and Peto odds ratio 0.622, as Stata's metan gives them. Further checks use the examples of Normand (1999), Raudenbush (2009), DerSimonian and Kacker (2007), Yusuf and others (1985) and the tutorial of Balduzzi, Rücker and Schwarzer (2019), whose data include studies with empty cells and subgroups.

Limits

Study-level data only, one effect per study, two groups. Not included: meta-regression on continuous moderators, multivariate and network meta-analysis, generalized linear mixed models for sparse binary data, single-arm proportions and rates, selection models, and the Kenward–Roger or bootstrap intervals. A meta-analysis is no better than the studies in it: pooling does not remove bias in the studies or in their selection.