Research Group: Evidence Synthesis Methods

The Evidence Synthesis Methods research group develops and advances methodology for the synthesis of clinical, pre-clinical and public health evidence, with a focus on efficacy and safety of healthcare interventions. Our work spans pairwise and network meta-analysis, addressing key challenges such as publication bias, integration of non-randomized studies in humans, analysis and integration of animal studies, multivariate statistical modelling, and the synthesis of rare safety outcomes.

In addition, we develop prognostic models to improve risk prediction and patient stratification in treatment decision-making, and we lead methodological innovation in the synthesis of evidence from human and animal studies.

Group members

Group leader

Group members

GALENOS

GALENOS: Living Evidence for Mental Health

Mental Health problems, and specifically anxiety, depression and psychosis, are amongst the leading causes of global disease burden. While treatment and prevention strategies are available these do not work for everyone and there have been no major advances in treatments for decades. 

GALENOS aims to address these issues through a process of living evidence synthesis. The project will produce a robust and regularly updated synthesis of the literature on the underlying mechanisms, prevention and treatment of anxiety, depression and psychosis. We are leading the methodology underpinning GALENOS, developing an innovative framework to synthesize clinical data with pre-clinical evidence from in vitro and animal studies. Evidence from human and non-human studies is then triangulated and assessed for bias and credibility. 

Our work is supported by The Wellcome Trust.

PPT

Evidence synthesis methods to Predict Personalized Treatment effects

We develop advanced statistical methods to improve personalized decision-making in medicine. Building on our pioneering work in network meta-analysis and individual participant data approaches, Our research focuses on predicting treatment efficacy and safety at the individual level, estimating dose–response relationships, and developing robust methods to validate model performance.

By applying these approaches across diverse clinical areas—including mental health, cardiology, neurology, and rheumatology—we aim to generate reliable, patient-centred evidence that supports more precise, effective, and safe healthcare decision-making.

Our work is funded by the Swiss National Science Foundation (Project funding 10002053)

CINeMA

CINeMA: Confidence in Network Meta-Analysis

Policy makers and guideline developers face challenges in evaluating the quality of evidence from systematic reviews with multiple interventions. We previously developed a framework to judge the confidence that can be placed in results obtained from a network meta-analysis (NMA) based on the GRADE domains: study limitations, indirectness, inconsistency, imprecision and publication bias. The framework combines judgments about direct evidence with their statistical contribution to network meta-analysis results, enabling evaluation of the credibility of NMA treatment effects and treatment rankings. However, the process is cumbersome and time-consuming for large networks.

Our user-friendly web application CINeMA (Confidence In Network Meta-Analysis) will greatly simplify the evaluation of NMA results, guiding reviewers through a structured process, with semi-automation of several steps that should decrease workload considerably. Only study outcome data and study-level risk of bias assessments are required as input; then CINeMA produces graphical and numerical summaries of NMA output, indicating likely judgments for the five credibility domains.

A ‘proof-of-concept’ version of CINeMA is available in cinema.ispm.unibe.ch

Salanti G, Del Giovane C, Chaimani A, Caldwell DM, Higgins JP. Evaluating the quality of evidence from a network meta-analysis. PLoS One. 2014 Jul 3;9(7):e99682.

Working Papers on CINeMA

Evaluation of confidence in the results of network meta-analysis: Within-study bias, across-studies bias and indirectness (PDF, 1.0 MB)
Evaluation of confidence in the results of network meta-analysis: Imprecision, heterogeneity and incoherence (PDF, 655KB)

Videos

In these videos from a Cochrane Learning Live webinar, Georgia Salanti and Theodore Papakonstantinou present the CINeMA (Confidence in Network Meta-analysis) framework and web aplication developed to judge the confidence that can be placed in results obtained from a network meta-analysis by adapting and extending the GRADE domains (study limitations, inconsistency, indirectness, imprecision and publication bias).

Below you will find the videos covering:

Part 1: Introduction to CINeMA framework
Part 2: Within-study bias, indirectness
Part 3: Imprecison, heterogeneity/incoherence, reporting bias
Part 4: Upcoming developments of CINeMA app, questions and answers

RoB-MEN

Assessing Risk of Bias from Missing Evidence in NMA with ROB-MEN

Selective outcome reporting and publication bias threaten the validity of systematic reviews and network meta-analyses (NMA). To address this, we developed ROB-MEN (Risk Of Bias due to Missing Evidence in NMA), the first tool to systematically evaluate such bias. ROB-MEN assesses risk within pairwise comparisons by considering unavailable results and unpublished studies, and across the network by integrating direct evidence contributions, small-study effects, and unobserved comparisons. Each estimate is classified as “low risk,” “some concerns,” or “high risk.” We illustrate ROB-MEN using examples in cardiology and psychiatry. An R Shiny app supports implementation: RoB-Men app.

Methodology article: Chiocchia V, Nikolakopoulou A, Higgins JPT, Page MJ, Papakonstantinou T, Cipriani A, Furukawa TA, Siontis GCM, Egger M, Salanti G. ROB-MEN: a tool to assess risk of bias due to missing evidence in network meta-analysis.BMC Med (2021) 19:304

RoB-MEN tutorial article: Chiocchia V, Holloway A & Salanti, G. Semi-automated assessment of the risk of bias due to missing evidence in network meta-analysis: a guidance paper for the ROB-MEN web-application. BMC Med Res Methodol (2023) 23:223

For more information on how to use the RoB-MEN app, watch the Cochrane Learning Live webinar videos.

Videos

Georgia’s YouTube channel has several educational videos about our research.

crossnma: A new R package to synthesize cross-design evidence and cross-format data

 

Confidence in Network Meta-Analysis: How to evaluate study limitations (theory)

This video explains how to evaluate the impact of study limitations (risk of bias) in the results of network meta-analysis.

Confidence in Network Meta-Analysis: How to evaluate study limitations (practical)

This video explains how to use the web-application CINeMA to evaluate the impact of study limitations (risk of bias) in the results of network meta-analysis.

Evidence-based designing of clinical trials using Living Network Meta-analysis

This is a talk presented at the 24th Cochrane Colloquium in 2016 in Seoul as part of the Methods Symposium. It presents, partly, work done within the EU funded project EBAR (MSCA-IF-703254)

A 10 minutes introduction to Network Meta analysis

Network meta-analysis (NMA) has emerged as the new evidence synthesis tool. Clinical papers that use NMA are increasingly published in the medical literature. This is a 10-minutes non-technical introduction to the concept of indirect comparison and NMA.

Combining randomised and non-randomised evidence in network meta-analysis

Observational studies convey valuable information about the effectiveness of interventions in real-life clinical practice and there is a growing interest for methods to include non-randomized evidence in the decision-making process. We then present three alternative methods that allow the inclusion of observational studies in an NMA of RCTs: the design-adjusted synthesis, the use of observational evidence as prior information and the use of three-level hierarchical models.

Network meta-anlysis (NMA)

Mathematical models

Individual Patient Data IPD meta analysis - Matthias Egger

Systematic reviews, meta analysis and real world evidence - Matthias Egger

Completed projects

  1. What works best? Methods for ranking competing treatments in network meta-analysis
  2. Next Generation Health Technology Assessment
  3. Predicting the real-world effectiveness and safety of medical interventions
  4. MHCOVID: Scientific evidence about changes in the prevalence of mental health issues due to the COVID-19 pandemic and containment measures.
  5. Evidence-based planning of clinical research (EBAR)
    (funded by EU research grant MSCA-IF-703254)
    This project aims to develop a theoretical framework for efficient and sustainable research planning, with the potential to direct resource allocation after considering the existing evidence.
  6. Enhancing methods for evaluating the comparative safety of medical interventions
    (funded by the Swiss National Science Foundation)
    This project aims to advance the methods for synthesizing evidence from randomised trials on the safety of interventions by developing and exploring meta-analytical models for correlated rare events and network meta-analysis of adverse events.
  7. Comparative effectiveness and safety of disease modifying drugs in early treatment of multiple sclerosis
    (funded by the Swiss Multiple Sclerosis Society)
    This project aims to answer a) What happens when people who receive a diagnosis of multiple sclerosis decide to start treatment with a disease-modifying drug? b) Which disease modifying drugs have the best efficacy-safety profile?