Validating therapy decisions by reasons for therapy switch in relapsing-remitting multiple sclerosis

Models for individualized treatment recommendations in relapsing–remitting multiple sclerosis often select therapies based on predicted risks of relapse and/or 3-month confirmed disability progression (3m-CDP). Recommendations should be interpretable, align with guidelines, and reflect patient perspectives. We validated recently developed prognostic algorithms in eight subgroups defined by reason for therapy switch: intolerance, lack of efficacy, pregnancy desire, serious adverse events, programmed stop, personal convenience, therapy initiation, drug-holiday end.

From the OFSEP registry, we analysed 3768 therapy cycles from 3768 patients treated between 2017 and 2021 with interferon beta, glatiramer acetate, teriflunomide, dimethyl fumarate, fingolimod, natalizumab. One therapy cycle was randomly selected per patient. The algorithms produced ranked therapy lists per treatment cycle as switching option; certainty of the provided recommendation was quantified by entropy (0 = clear; 4.78 = maximal uncertainty).

Recommendations were more consistent for relapse (median entropy 0.54, IQR 0.39–0.61) than for 3m-CDP (0.95, IQR 0.88–1.01). Across subgroups, natalizumab most often ranked highest for relapse, whereas teriflunomide or interferon beta frequently ranked highest for 3m-CDP. Calibration varied across switcher subgroups and outcomes, while discrimination was comparable to overall set: C-index 0.776 (95% CI 0.753–0.797) for 3m-CDP and 0.638 (95% CI 0.615–0.661) for relapse.

Guideline and relapse-based recommendations did not always align, notably in the pregnancy desire subgroup. For 3m-CDP, guideline-based recommendations are scarce, precluding systematic comparison with model outputs. Agreement between the best-ranked therapy for relapse and 3m-CDP was zero (κ = 0), underscoring outcome-dependent divergence.

Algorithm-generated recommendations depend on the chosen outcome and may diverge from guideline-based practice or patient perspectives. Transparent communication of uncertainty and outcome trade-offs is essential for shared decision-making. Confirmed disability progression at 3 months Deutsche Gesellschaft für Neurologie, German neurological society European Committee of Treatment of Research in Multiple Sclerosis/European Academy of Neurology Observatoire Français de la Sclérose en Plaques Progression independent of relapse activity Progressive multifocal leukoencephalopathy Persons with relapsing-remitting multiple sclerosis Transparent reporting of a multivariable prediction model for individual Prognosis or Diagnosis + Artificial Intelligence OFSEP Data collection has been supported by a grant provided by the French State and handled by the “Agence Nationale de la Recherche,” within the framework of the “France 2030” programme, under the reference ANR-10-COHO-002, Observatoire Français de la Sclérose en Plaques (OFSEP)/Eugène Devic EDMUS Foundation against multiple sclerosis.

We also thank Nikolaus von Bomhard and Marcel Müller for their technical support in providing a secure server for publishing the Shiny app. ChatGPT was used to polish the text and R code. Open Access funding enabled and organized by Projekt DEAL.