Meta-analysis

It is hard to judge all the apples at a market stall by tasting just one. If you taste several apples from several stalls and take an average, you can have more confidence in the result. Meta-analysis works in a similar way. It combines the effect sizes of individual studies into a single pooled value, weighting them according to sample size and precision. Studies with more participants receive greater weight, while those with fewer participants receive less.

“Meta” means “one level above.” The method gets its name because it studies existing studies rather than studying patients directly.

• It improves precision. Combining several small studies produces a sample approaching that of one large study. This narrows the confidence interval around the pooled value.
• It helps make sense of conflicting results. When some studies find a difference and others do not, it provides an overall picture of which conclusion the evidence leans toward.
• It allows researchers to examine differences between studies. Results can be examined separately to see whether they vary by age or symptom severity.

It is usually conducted as one step in a systematic review. Researchers first use predefined rules to find and select all relevant studies, then conduct a meta-analysis when the results can be combined numerically.

Results are often presented in a forest plot. Each study has a horizontal line: the square in the middle of the line represents that study’s estimate, and the length of the line represents its confidence interval. The diamond at the bottom represents the pooled estimate. The farther the diamond is from the vertical line indicating “no effect,” and the narrower it is, the clearer the pooled result is considered to be.

For example, if the pooled result is “pain scores decreased by an average of 1 point,” you also need to consider whether that 1-point change is large enough for a person to notice. The minimal clinically important difference is used as a benchmark for this.

• Heterogeneity — This refers to how much the results of the studies differ from one another. It is often expressed as I², a value between 0 and 100; higher values mean greater differences between study results. Values of roughly 75% or higher are often considered substantial. In such cases, the explanation of why the studies differ is more important than a single pooled value.
• Subgroup analysis — This analysis divides studies into groups, such as adolescents versus adults or mild versus severe symptoms, and pools them separately. It is used to explore the reasons for heterogeneity.
• Sensitivity analysis — This involves removing particular studies and recalculating the result to see how much it changes.
• Publication bias — This is the tendency for striking results to be more likely to be published. Combining only published studies can make an effect appear larger than it really is, so researchers check for this using methods such as funnel plots. See Bias for details.

The size of a pooled estimate and how much confidence you can place in it are different questions. Certainty depends on whether the included studies were well designed, whether their results agree, and whether the sample is large enough. The GRADE approach is an international method for classifying this certainty into levels. When reading an article about a meta-analysis, check both the pooled estimate and what it says about the certainty of the evidence.

A good meta-analysis also discloses how studies were found and selected, and which studies were excluded. If many low-quality studies are included, the pooled number may look precise but still be difficult to trust.

1. Check how many studies and participants were included in the synthesis. 2. Look at whether the studies included in the analysis were randomized controlled trials or observational studies. 3. Check the lower and upper limits of the pooled estimate’s confidence interval. 4. Examine the level of heterogeneity using indicators such as I². 5. Consider whether the size of the effect exceeds the minimal clinically important difference and how the certainty was assessed.

Meta-analysis and systematic review do not mean the same thing. Many systematic reviews describe the results in words without combining the numbers.