The value of clinical investigations - why evidence levels are as exciting as your favourite TV series

The scientific significance (evidence) of clinical studies is categorised hierarchically according to internationally valid conventions. This emphasises the key role that evidence plays in the market launch of innovative medical devices.

It is therefore important that developers of innovative products for the healthcare market understand the evidence requirements of both the regulatory authorities and all other stakeholders in the authorisation process so that evidence generation meets all requirements.

The following article aims to provide an initial insight and overview - with the necessary simplifications - as this area is essential for laypersons developing innovative medical devices, including DiGA or DiPA.

Understanding the hierarchical classification of evidence categorisations

Both internationally and within Germany, different terms and scales are used for the evaluation of evidence for clinical-scientific questions. This is an unnecessarily complicated confusion for scientists and a downright imposition for manufacturers of medical devices.

Different institutions, federal authorities, national and international professional societies and associations, European and national authorities, notified bodies or legal texts use different formulations that all mean something similar, such as evidence level, level of evidence, level of evidence, evidence class, degree of evidence or scientific evidence.

The better the study design, the higher the quality of a clinical investigation

What all classifications and formulations have in common is that they categorise reports or scientific publications on clinical studies according to certain quality criteria. In general, higher evidence classes have a broader basis and grade Ia has the highest evidence quality, while grade IV or V have the lowest quality. The requirements for clinical evidence for the respective classifications are quite similar, but there are differences in the choice of comparators, the preferred efficacy endpoints or the target population. This makes it difficult to compare studies and therapy recommendations.

In recent years, there have been increasing efforts in evidence-based medicine to achieve a standardised approach with comparable classifications and uniform interpretations. The GRADE methodology (Grading of Recommendations, Assessment, Development and Evaluation), which has been developed over 20 years ago and is recognised worldwide, is increasingly being recommended by professional associations as a method of classifying the quality of evidence and strength of recommendations for guidelines.

Why should manufacturers be supported in their knowledge of evidence grades?

If experts are involved in the development of the study design from the outset, this has a positive effect on the quality of the evidence.

It is important to know these levels of evidence and how they are achieved, as certain levels of evidence (or possibly also "scientific findings") are required as part of an authorisation procedure or the procedure for inclusion in the DiGA Directory, for example, and these must be achieved through a clinical investigation or a performance study. In order for a therapeutic or diagnostic medical device not only to be authorised, but also later recommended by the guidelines of the professional societies, the highest possible level of evidence must be achieved.

Variables influencing the assessment of the quality of evidence

Grading evidence according to GRADE is a complex process that indicates four levels of confidence in the results of one or more clinical studies. Factors that influence the classification are Choice of study design to avoid systematic errors and biased results, validity (internal validity), quality of implementation, size and precision, and transferability and applicability of results (external validity).

The term levels of evidence refers to the strength, robustness and/or quality of the evidence. These levels reflect the source of evidence, statistical validity, clinical relevance, and acceptance through peer review processes in peer-reviewed journals. Evidence levels can be downgraded due to: study quality, imprecision, indirectness (lack of agreement between actual study PICO (population, intervention, comparator and endpoint) and question PICO), inconsistency between studies or because the absolute effect size is very small or too large. This must be avoided through experience and scientific rigour.

In accordance with the available levels of evidence, the respective professional societies subsequently issue recommendation grades in the guidelines, taking into account the medical benefits, risks and side effects based on the consistency of the study situation. This is crucial for manufacturers of medical devices.

Mastering specifications for evidence criteria for medical devices and in-vitro diagnostics

In the field of clinical and other clinical investigations of medical devices, DiGA and / or in vitro diagnostics, high requirements are placed on evidence. However, (EU) 2017/745 (MDR), the Medical Devices Implementation Act (MPDG), the Digital Healthcare Act (DiGAV) and Regulation (EU) 2017/746 on in vitro diagnostic medical devices (IVDR) do not specify the actual requirements for the quality of the evidence in any detail.

Knowledge of the legal information on the scientific quality of clinical investigations of medical devices, DiGA or performance studies of IVDs has a positive effect

With regard to scientific quality, the MDR states that clinical data and clinical evaluation results for a device must be available in "sufficient quantity and quality" to enable a qualified assessment of the safety of the device and the achievement of the intended clinical benefit when used as intended by the manufacturer. These requirements currently leave a great deal of room for discretion and do not provide any guidance for interpretation by manufacturers.

In the IVDR, the manufacturer should define the "level of clinical evidence" required to demonstrate compliance with the relevant general safety and performance requirements. The level of "scientific evidence" should be appropriate with regard to the characteristics of the device and its intended purpose. It is therefore the manufacturer's responsibility to decide what is appropriate for their product based on the intended use and risk class.

This places a considerable burden on manufacturers, as it is difficult for them to anticipate - as our experience shows - which quality criteria will be assessed in the requirements or in the final assessment and according to which interpretation. In addition, there is currently the fact that, depending on the personal characteristics of the assessors in the various committees, different interpretations may arise. Therefore, there is currently an urgent need to develop guidelines and unambiguous specifications for the preparation of this clinical evidence and the respective levels of evidence required, as well as a detailed and binding communication on how far and on what basis, for example in a DiGA fast-track procedure, the discretionary scope is interpreted and how.

Consultation and support for the evidence for DiGAs

For DiGAs, the BfArM provides information on the evidence levels required in its guidance on the fast-track procedure for digital health applications (DiGA) in accordance with Section 139e SGB V. However, the discretionary scope addressed in the DiGA fast-track procedure is not explained in more detail and, in our experience from recent years, further information and more precise details are only available to manufacturers and physicians working in science in direct dialogue with the BfArm, although these are often not binding. This potential can certainly be further consolidated.

In its position paper, the Medical Device Coordination Group (MDCG) has named scientific validity, analytical performance and clinical performance as components of the evidence for clinical evidence for in vitro diagnostic medical devices in accordance with IVDR 2017/746.

If you are more interested in who uses which terms:

• Level of evidence: Association of the Scientific Medical Societies in Germany (AWMF), Institute for Quality and Efficiency in Healthcare (IQWiG), Oxford Centre for Evidence-Based Medicine (OCEBM), "level of evidence"
• Evidence class: Agency for Healthcare Research and Quality (AHCR)
• Scientific knowledge: in the German version of Regulation (EU) 2017/745 (MDR), Regulation (EU) 2017/746 on in vitro diagnostic medical devices (IVDR) and in the Medical Devices Implementation Act (MPDG), "clinical evidence"
• Evidence: is equated with "study types"
• Medical findings: in the Digital Healthcare Act (DiGAV)
• Evidence levels: Federal Joint Committee (G-BA), Federal Institute for Drugs and Medical Devices (BfArM), National Association of Statutory Health Insurance Funds (GKV-Spitzenverband)

Summary

In summary of the current situation, although there are several commonalities in terms of the need for clinical evidence from clinical studies, the understanding of their needs by manufacturers and researchers is currently suboptimal, so that the studies designed and conducted and their reports cannot always meet the needs.

This leads to expected delays in decision-making, which has a negative impact on access to innovation in Europe. A standardised and uniform formulation of the evidence requirements of all European authorities and institutions would make sense in order to optimise cooperation with regard to evidence generation, evidence use and evidence presentation.

Until then, the early involvement of experts in the field to support manufacturers and the very early consultation and joint setting of standards with authorities is essential for a successful process.

References and specialist articles can be requested from MEDIACC.

(as of spring 2024)

We are also available for a personal consultation. At MEDIACC, we offer comprehensive support in developing the right type of study. Pleasecontact us for an initial non-binding discussion!