Please use this identifier to cite or link to this item:
https://hdl.handle.net/2440/140398
Citations | ||
Scopus | Web of Science® | Altmetric |
---|---|---|
?
|
?
|
Type: | Journal article |
Title: | Variation observed in consensus judgements between pairs of reviewers when assessing the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research |
Author: | Kanukula, R. McKenzie, J.E. Cashin, A.G. Korevaar, E. McDonald, S. Mello, A.T. Nguyen, P.-Y. Saldanha, I.J. Wewege, M.A. Page, M.J. |
Citation: | Journal of Clinical Epidemiology, 2023; 166:111244-1-111244-10 |
Publisher: | Elsevier BV |
Issue Date: | 2023 |
ISSN: | 0895-4356 1878-5921 |
Statement of Responsibility: | Raju Kanukula, Joanne E. McKenzie, Aidan G. Cashin, Elizabeth Korevaar, Sally McDonald, Arthur T. Mello, Phi-Yen Nguyen, Ian J. Saldanha, Michael A. Wewege, Matthew J. Page |
Abstract: | Objectives: To evaluate the risk of bias due to missing evidence in a sample of published meta-analyses of nutrition research using the Risk Of Bias due to Missing Evidence (ROB-ME) tool and determine inter-rater agreement in assessments. Study Design and Setting: We assembled a random sample of 42 meta-analyses of nutrition research. Eight assessors were randomly assigned to one of four pairs. Each pair assessed 21 randomly assigned meta-analyses, and each meta-analysis was assessed by two pairs. We calculated raw percentage agreement and chance corrected agreement using Gwet’s Agreement Coefficient (AC) in consensus judgments between pairs. Results: Across the eight signaling questions in the ROB-ME tool, raw percentage agreement ranged from 52% to 100%, and Gwet’s AC ranged from 0.39 to 0.76. For the risk-of-bias judgment, the raw percentage agreement was 76% (95% confidence interval 60% to 92%) and Gwet’s AC was 0.47 (95% confidence interval 0.14 to 0.80). In seven (17%) meta-analyses, either one or both pairs judged the risk of bias due to missing evidence as ‘‘low risk’’. Conclusion: Our findings indicated substantial variation in assessments in consensus judgments between pairs for the signaling questions and overall risk-of-bias judgments. More tutorials and training are needed to help researchers apply the ROB-ME tool more consistently. |
Keywords: | Bias; Reporting bias; Meta-analysis; Nutritional sciences; Systematic review; Reliability |
Rights: | © 2024 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/ 4.0/). |
DOI: | 10.1016/j.jclinepi.2023.111244 |
Grant ID: | http://purl.org/au-research/grants/nhmrc/1139997 http://purl.org/au-research/grants/nhmrc/GNT2009612 http://purl.org/au-research/grants/nhmrc/GNT2010088 http://purl.org/au-research/grants/arc/DE200101618 |
Published version: | http://dx.doi.org/10.1016/j.jclinepi.2023.111244 |
Appears in Collections: | Public Health publications |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
hdl_140398.pdf | Published version | 445.93 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.