Clinical relevance in medical studies: What does that actually mean?

In the world of medical research and clinical trials, we often come across the term “clinical relevance”; this is required in particular by trials of medical devices when it is to be reimbursed by health insurance companies and, if necessary, the price should also be negotiated.

A key question is at what point a change can be considered significant for the individual patient. In other words: What threshold value must be reached in order to assume a response to an intervention and to use the difference in response rates between two groups as an effect measure for the benefit assessment?

In particular, it is required for digital health applications. It will therefore be discussed below on the basis of a statement from the independent German Institute for Quality and Efficiency in Health Care (IQWiG), sources see below.

But what is behind this term and why is it so important? Put simply, it is a question of whether the results of a study actually make a significant difference for patients. It is therefore not enough that a clinical trial was able to show that a treatment is statistically significantly superior to another or no treatment — it must also have a measurable and noticeable positive effect on health or quality of life.

The problem with clinical relevance

The challenge is that there is no universal formula for determining clinical relevance. What is relevant for one disease or patient may be meaningless for others. For example, a slight improvement for a severe, life-threatening illness could make a big difference, while the same improvement would barely be noticeable for a mild, self-limiting illness.

When is an effect even considered clinically relevant?

The answer to this question depends on various factors:

• Severity of illness: The more serious the disease, the more likely even a small effect is considered relevant.
• Incidence of illness: In the case of frequent illnesses, even a minor effect can have a major impact on the entire population.
• Benefits and risks of treatment: A treatment with significant side effects must have greater benefits to be considered clinically relevant.

How is clinical relevance assessed?

There are various approaches to assess the clinical relevance of study results. Some common methods include:

• Effect size: Here, the magnitude of the effect of a treatment is quantified based on the value of the so-called Cohen d or effect size correlation r. Cohen originally proposed a division of three, in which 0.2 should be interpreted as a small effect, up to 0.5 a medium effect and above 0.8 a large effect. However, it is important to note that a large effect size does not automatically mean clinical relevance.
• Significance test for relevant superiority: This method tests whether a new treatment is better than an existing one, to an extent that is considered clinically relevant.
• Evaluation based on the observed effect: Here, the actual difference between treatment groups is considered and whether this difference is significant in practice is assessed.
• Responder analyses: This method looks at the proportion of patients who respond to treatment, i.e. experience a certain improvement.
• Relative effect: This approach looks at the likelihood that a patient will have a better outcome with a specific treatment than with another treatment

From this list, it can already be seen that there is (too) much room for interpretation here, which can be exploited depending on one's own interests. The literature on this subject is very extensive.

The importance of pre-specification

An important point is that the method for evaluating clinical relevance, relevance threshold, and type of decision (whether to consider the observed effect or confidence interval) should be determined before starting the study. This helps prevent bias and ensures that results are interpreted in an objective way.

IQWiG's further approach

For a long time, the methodological discussion revolved around so-called Minimal Important Differences (MID). The basic idea was to identify a threshold value for each questionnaire that represents the smallest significant change for patients. However, methodological problems with this approach have recently become apparent. The MID is not a fixed value, but variable and depends, for example, on the type and severity of the disease, the direction of change investigated (improvement or deterioration) or the methodology. In addition, many studies to determine an MID no longer meet today's methodological standards or do not adequately describe the methodology used.

In this context, IQWiG developed a further approach in 2020 to determine a threshold value. The aim of this definition was to create clarity for manufacturers and make arbitrary responder analyses based on incomprehensible responder definitions unattractive. IQWiG defined a value of 15% of the range of the respective scales as a plausible threshold for a small but sufficiently noticeable change. In the General Methods 6.0 published in November 2020, the Institute then stipulated that in future it would use responder analyses for the assessment from a threshold value of at least 15 % of the scale range of the survey instrument used.

‍

If you are interested in clinical trials for approval or reimbursement purposes in compliance with clinical relevance requirements, please feel free to arrange a first, non-binding consultation with us. We're here for you!

‍

Source: https://www. iqwig.de/ events/11-11-25_ kieser_bewertung_klinischer_relevanz.pdf, https://www. iqwig.de /methoden /allgemeine-methoden-v6-1.pdf (please remove spaces for links)