Methods used to select results to include in meta-analyses of nutrition research: A meta-research study

Abstract

Objectives: To investigate how often review authors encounter multiple results from included studies that are eligible for inclusion in a particular meta-analysis, and how often methods to select results are specified. Methods: MEDLINE and Epistemonikos were searched (January 2018–June 2019) to identify systematic reviews with meta-analysis of the association between food/diet and health-related outcomes. A random sample of these reviews was selected, and for the first presented (index) meta-analysis, rules used to select effect estimates to include in this meta-analysis were extracted from the reviews and their protocols. All effect estimates from the primary studies that were eligible for inclusion in the index meta-analyses were extracted (e.g., when a study report presented effect estimates for blood pressure at 3 weeks and 6 weeks, both unadjusted and adjusted for covariates, and all were eligible for inclusion in a meta-analysis of the effect of red meat consumption on blood pressure, we extracted all estimates, and classified the study as having “multiplicity of results”). Results: Forty-two systematic reviews with 325 studies (104 randomized, 221 non-randomized) were included; 14 reviews had a protocol. In 29% of review protocols and 69% of reviews, authors specified at least one decision rule to select effect estimates when multiple were available. In 68% of studies included in the index meta-analyses, there was at least one type of multiplicity of results. Conclusions: Authors of systematic reviews of nutrition studies should anticipate encountering multiplicity of results in the included primary studies. Specification of methods to handle multiplicity when designing reviews is therefore recommended.