Interpretation of the risk of bias in network meta-analysis (RoB NMA)_Chinese Journal of Evidence-Based Medicine

Authors：

ZHANG Zheng ¹ , TIAN Chen ^2,3,4 , ZHENG Qingyong ^3,5 , XIONG Jing ¹ , TIAN Jinhui ^3,5 ,  SHI Lei ¹ ,  GE Long ^2,3,4

1. School of Nursing, Southern Medical University, Guangzhou 510515, P. R. China;
2. Department of Health Policy and Management, School of Public Health, Lanzhou University, Lanzhou 730000, P. R. China;
3. Key Laboratory of Evidence Based Medicine of Gansu Provincial, Lanzhou 730000, P. R. China;
4. Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences (2021RU017), School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
5. Evidence-based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;

Corresponding author：

SHI Lei, Email: 35508897@qq.com; GE Long, Email: gelong2009@163.com

Keywords：

Network meta-analysis; Systematic review; Risks of bias; Interpretation

DOI：

10.7507/1672-2531.202503195

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Network meta-analysis aims to integrate direct and indirect evidence, make a comprehensive comparison and in-depth analysis of three or more interventions and treatments, compare and rank the advantages and disadvantages of different treatment measures, so as to provide strong evidence for decision-making. However, there may be some bias in the process of making network meta-analysis, analyzing data and interpreting results. Therefore, accurate assessment and proper handling of the risks of bias in network meta-analysis is conductive to improve the quality of decision-making and promote the achievement of good clinical outcomes. At present, the number of published network meta-analysis has increased significantly globally, but the quality remains to be improved, and there is a lack of assessment tools for risks of bias in network meta-analysis. In 2025, Canadian scholar Carole Lunny and colleagues developed The Risk of Bias in Network Meta-Analysis tool for evaluating the risk of bias in network meta-analysis and published it in the BMJ, which is important to reduce the bias in network meta-analysis and promote optimal clinical decision-making. This study will interpret it with examples, aiming to help researchers better understand and apply this evaluation tool.

1.	Ahn E, Kang H. Concepts and emerging issues of network meta-analysis. Korean J Anesthesiol, 2021, 74(5): 371-382.
2.	Page MJ, McKenzie JE, Kirkham J, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev, 2014, 2014(10): MR000035.
3.	Psaty BM, Lumley T, Furberg CD, et al. Health outcomes associated with various antihypertensive therapies used as first-line agents: a network meta-analysis. JAMA, 2003, 289(19): 2534-2544.
4.	郑卿勇, 周泳佳, 许建国, 等. 诊断试验准确性网状Meta分析方法进展与挑战. 中国循证医学杂志, 2025, 25(3): 349-357.
5.	郑卿勇, 许建国, 刘明, 等. 网状Meta分析中的偏倚风险评估工具概览与思考. 中国循证医学杂志, 2025, 25(5): 584-594.
6.	Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 2017, 358: j4008.
7.	Whiting P, Savović J, Higgins JP, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol, 2016, 69: 225-234.
8.	Marques-Cruz M, Vieira RJ, Martinho-Dias D, et al. Reported methodological quality of medical systematic reviews: development of an assessment tool (ReMarQ) and meta-research study. Res Synth Methods, 2025, 16(1): 175-193.
9.	Jansen JP, Trikalinos T, Cappelleri JC, et al. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health, 2014, 17(2): 157-173.
10.	Whiting P, Wolff R, Mallett S, et al. A proposed framework for developing quality assessment tools. Syst Rev, 2017, 6(1): 204.
11.	Lunny C, Higgins JPT, White IR, et al. Risk of bias in network meta-analysis (RoB NMA) tool. BMJ, 2025, 388: e079839.
12.	Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ, 2019, 366: l4898.
13.	Efthimiou O, Mavridis D, Debray TP, et al. Combining randomized and non-randomized evidence in network meta-analysis. Stat Med, 2017, 36(8): 1210-1226.
14.	Bujkiewicz S, Singh J, Wheaton L, et al. Bridging disconnected networks of first and second lines of biologic therapies in rheumatoid arthritis with registry data: bayesian evidence synthesis with target trial emulation. J Clin Epidemiol, 2022, 150: 171-178.
15.	Brignardello-Petersen R, Murad MH, Walter SD, et al. GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks. J Clin Epidemiol, 2019, 105: 60-67.
16.	Nikolakopoulou A, Higgins JPT, Papakonstantinou T, et al. CINeMA: an approach for assessing confidence in the results of a network meta-analysis. PLoS Med, 2020, 17(4): e1003082.
17.	Ge L, Sadeghirad B, Ball GDC, et al. Comparison of dietary macronutrient patterns of 14 popular named dietary programmes for weight and cardiovascular risk factor reduction in adults: systematic review and network meta-analysis of randomised trials. BMJ, 2020, 369: m696.
18.	Lunny C, Tricco AC, Veroniki AA, et al. Methodological review to develop a list of bias items used to assess reviews incorporating network meta-analysis: protocol and rationale. BMJ Open, 2021, 11(6): e045987.
19.	Lunny C, Veroniki AA, Hutton B, et al. Knowledge user survey and Delphi process to inform development of a new risk of bias tool to assess systematic reviews with network meta-analysis (RoB NMA tool). BMJ Evid Based Med, 2023, 28(1): 58-67.
20.	Lunny C, Veroniki AA, Higgins JPT, et al. Methodological review of NMA bias concepts provides groundwork for the development of a list of concepts for potential inclusion in a new risk of bias tool for network meta-analysis (RoB NMA Tool). Syst Rev, 2024, 13(1): 25.
21.	Lunny C, Kanji S, Thabet P, et al. Assessing the methodological quality and risk of bias of systematic reviews: primer for authors of overviews of systematic reviews. BMJ Med, 2024, 3(1): e000604.
22.	王欣怡, 张珺, 葛龙, 等. 网状Meta分析方法在护理领域中的应用探讨. 循证护理, 2017, 3(5): 469-472.

1. Ahn E, Kang H. Concepts and emerging issues of network meta-analysis. Korean J Anesthesiol, 2021, 74(5): 371-382.
2. Page MJ, McKenzie JE, Kirkham J, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev, 2014, 2014(10): MR000035.
3. Psaty BM, Lumley T, Furberg CD, et al. Health outcomes associated with various antihypertensive therapies used as first-line agents: a network meta-analysis. JAMA, 2003, 289(19): 2534-2544.
4. 郑卿勇, 周泳佳, 许建国, 等. 诊断试验准确性网状Meta分析方法进展与挑战. 中国循证医学杂志, 2025, 25(3): 349-357.
5. 郑卿勇, 许建国, 刘明, 等. 网状Meta分析中的偏倚风险评估工具概览与思考. 中国循证医学杂志, 2025, 25(5): 584-594.
6. Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ, 2017, 358: j4008.
7. Whiting P, Savović J, Higgins JP, et al. ROBIS: a new tool to assess risk of bias in systematic reviews was developed. J Clin Epidemiol, 2016, 69: 225-234.
8. Marques-Cruz M, Vieira RJ, Martinho-Dias D, et al. Reported methodological quality of medical systematic reviews: development of an assessment tool (ReMarQ) and meta-research study. Res Synth Methods, 2025, 16(1): 175-193.
9. Jansen JP, Trikalinos T, Cappelleri JC, et al. Indirect treatment comparison/network meta-analysis study questionnaire to assess relevance and credibility to inform health care decision making: an ISPOR-AMCP-NPC Good Practice Task Force report. Value Health, 2014, 17(2): 157-173.
10. Whiting P, Wolff R, Mallett S, et al. A proposed framework for developing quality assessment tools. Syst Rev, 2017, 6(1): 204.
11. Lunny C, Higgins JPT, White IR, et al. Risk of bias in network meta-analysis (RoB NMA) tool. BMJ, 2025, 388: e079839.
12. Sterne JAC, Savović J, Page MJ, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ, 2019, 366: l4898.
13. Efthimiou O, Mavridis D, Debray TP, et al. Combining randomized and non-randomized evidence in network meta-analysis. Stat Med, 2017, 36(8): 1210-1226.
14. Bujkiewicz S, Singh J, Wheaton L, et al. Bridging disconnected networks of first and second lines of biologic therapies in rheumatoid arthritis with registry data: bayesian evidence synthesis with target trial emulation. J Clin Epidemiol, 2022, 150: 171-178.
15. Brignardello-Petersen R, Murad MH, Walter SD, et al. GRADE approach to rate the certainty from a network meta-analysis: avoiding spurious judgments of imprecision in sparse networks. J Clin Epidemiol, 2019, 105: 60-67.
16. Nikolakopoulou A, Higgins JPT, Papakonstantinou T, et al. CINeMA: an approach for assessing confidence in the results of a network meta-analysis. PLoS Med, 2020, 17(4): e1003082.
17. Ge L, Sadeghirad B, Ball GDC, et al. Comparison of dietary macronutrient patterns of 14 popular named dietary programmes for weight and cardiovascular risk factor reduction in adults: systematic review and network meta-analysis of randomised trials. BMJ, 2020, 369: m696.
18. Lunny C, Tricco AC, Veroniki AA, et al. Methodological review to develop a list of bias items used to assess reviews incorporating network meta-analysis: protocol and rationale. BMJ Open, 2021, 11(6): e045987.
19. Lunny C, Veroniki AA, Hutton B, et al. Knowledge user survey and Delphi process to inform development of a new risk of bias tool to assess systematic reviews with network meta-analysis (RoB NMA tool). BMJ Evid Based Med, 2023, 28(1): 58-67.
20. Lunny C, Veroniki AA, Higgins JPT, et al. Methodological review of NMA bias concepts provides groundwork for the development of a list of concepts for potential inclusion in a new risk of bias tool for network meta-analysis (RoB NMA Tool). Syst Rev, 2024, 13(1): 25.
21. Lunny C, Kanji S, Thabet P, et al. Assessing the methodological quality and risk of bias of systematic reviews: primer for authors of overviews of systematic reviews. BMJ Med, 2024, 3(1): e000604.
22. 王欣怡, 张珺, 葛龙, 等. 网状Meta分析方法在护理领域中的应用探讨. 循证护理, 2017, 3(5): 469-472.

Chinese Journal of Evidence-Based Medicine

Latest ArticlesInterpretation of the risk of bias in network meta-analysis (RoB NMA)

Abstract Full text Figures/Tables Video References Cited by

Format

Content