EMA “Practical Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons”

The “Practical Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons” was adopted on March 8, 2024 by the HTA Coordination Group in accordance with EU Regulation 2021/2282. It provides practical guidance for reviewers in making direct and indirect comparisons in joint clinical assessments (JCAs). It is important for manufacturers of innovative products to know which criteria are used to assess and evaluate their clinical studies.

Direct comparisons are defined as comparisons between interventions within the same study, while indirect comparisons compare interventions using common comparators in different studies.

Meta-analyses are described as statistical methods for summarizing the results of several studies, with network meta-analyses being presented as an extended form for comparing multiple interventions. The guide underlines its anchoring in EU Regulation 2021/2282, in particular Article 9, which regulates the conduct of joint clinical evaluations. It is stressed that the guide aims to provide reviewers with practical assistance in evaluating various comparative methods without making specific recommendations for individual member states. The focus is on the practical application of methods for direct and indirect comparisons, including meta-analyses and network meta-analyses.

Objective of the guide

The guide aims to provide reviewers with a tool to assess the validity and uncertainty of various comparative methods. It is clarified that the guide does not provide specific recommendations for individual member states, but rather serves as a guide for evaluating evidence syntheses. The importance of a thorough assessment of the methods used and their appropriateness for each clinical issue is emphasized and the need to interpret the results in the context of all available evidence is emphasized.

Fundamental Aspects of Evidence Synthesis

The systematic literature review as a basis for robust comparisons and the selection of suitable studies is emphasized as a critical step, with the evaluation of the interchangeability of studies playing a key role. Three main concepts are identified: Similarity, Homogeneity and Consistency.

• Similarity refers to the comparability of study characteristics
• Homogeneity on the consistency of treatment effects in similar studies,
• Consistency on the agreement between direct and indirect comparisons.

These aspects must be carefully reviewed to ensure the validity of the evidence synthesis. Potential sources of heterogeneity should be identified and considered in order to increase the robustness of the conclusions.

Methods for direct and indirect comparisons

In direct comparisons, standard meta-analysis techniques such as fixed effect and random effects models are presented.

The Bucher Approach is described as a basic method for indirect comparisons.

Network meta-analyses are explained as an advanced technique for the simultaneous comparison of multiple interventions. Specific challenges are also addressed, such as dealing with various effect measures and the analysis of temporal data, such as the meta-analysis of hazard ratios. The use of individual patient data is discussed and the importance of choosing the right method based on available evidence and specific research questions is emphasized. In addition, the importance of sensitivity analyses is emphasized in order to verify the robustness of the results.

Evaluation of population-adjusted methods in evidence syntheses

It highlights the importance of covariate selection, stressing that selection should be based on clinical and methodological knowledge. The guide discusses various approaches such as Matching-Adjusted Indirect Comparison (MAIC) and Simulated Treatment Comparison (STC). It is emphasized that these methods are particularly useful when there are differences in study populations. The interpretation of adjusted results is emphasized as a critical issue, pointing out the need to take into account the assumptions and limitations of the methods used. The Guide recommends that the results of population-adjusted analyses be considered in the context of the non-adjusted results and that potential biases be discussed. In addition, the importance of transparent reporting on the methods and assumptions used is emphasized.

Evaluating comparisons based on non-randomized data

The use of probability values (propensity scores) is presented as an important method for reducing biases in observational studies. The guide explains various applications of probability values, including attribution, stratification, and weighting. The choice of the specific method depends on the research question and the available data. Interpreting the results is particularly important, pointing out the need to consider potential residual biases and unobserved confounding factors. The Guide recommends that the results of comparisons based on non-randomized data be interpreted with caution and that they be viewed in the context of all available evidence. In addition, the importance of sensitivity analyses to verify the robustness of the results is emphasized.

For healthcare technology manufacturers, this knowledge is crucial as it enables them to design and conduct their clinical trials and data analyses in such a way that they optimally meet the requirements of joint clinical evaluations (JCAs) in the EU. This can ultimately make it easier to access the market and reimburse their technologies.

‍

Source: https://health .ec. europa.eu/document/download/1f6b8a70-5ce0-404e-9066- 120dc9a8df75_en? filename=hta_practical -guideline_direct-and-indirect-comparisons_en.pdf (delete space for link)

Can we also help you through the regulatory jungle for your healthcare technology? Contact us for an initial consultation.