Matters arising: Methodological concerns and interpretative limitations of assessing subjective effects of ketamine

The letter addresses critical issues in the methodology and conclusions of the meta-analysis, including inconsistencies in study design, methodological flaws in comparing correlation coefficients, and discrepancies in reported results and their interpretation. It contributes significantly to the ongoing discussion on psychedelic-assisted therapies and highlights the need for more standardized methodologies in future research. Dahan et al.1 have made a commendable effort to synthesize existing literature on the subjective effects of ketamine and psilocybin and their relationship to therapeutic outcome.

However, several critical methodological and interpretative issues—such as inconsistencies in study design, flawed comparisons of correlation coefficients, and discrepancies between reported results and their interpretations—warrant further discussion and careful consideration. Firstly, for the ketamine treatment group, Dahan et al. selected articles that did not include psychotherapy alongside ketamine administration, thereby overlooking the potential synergistic effects of combined ketamine and psychotherapy interventions, known as ketamine-assisted therapy (KAP)2. This approach was found to be beneficial, improving outcomes and delaying relapse, as demonstrated in a recent systematic review by Kew et al.3.

There is substantial evidence that psychotherapy provided before, during, and following ketamine sessions can maximize and prolong therapeutic benefits4,5,6,7. Moreover, there are well-established integrative approaches such as the Montreal Model, which combines ketamine infusions with structured psychotherapy and supportive environments to produce more durable antidepressant effects in treatment-resistant depression8. In contrast, all psilocybin studies included in the meta-analysis included psychotherapy as an integral part of the treatment process.

This discrepancy creates a challenge in comparing the two groups in the discussion since the therapeutic outcomes for psilocybin could be influenced not only by the drug itself but also by the synergistic effects of psychotherapy. This inconsistency in study design might contribute to the observed differences in correlation strengths between subjective effects and therapeutic outcomes for ketamine and psilocybin. Secondly, Dahan et al. attempt to compare the correlation coefficients between subjective effects and therapeutic outcomes for ketamine vs. psilocybin in their “Ketamine versus psilocybin” section.

However, this comparison is methodologically flawed for several key reasons: (a) Different questionnaires were used to measure subjective effects for ketamine (Clinician-Administered Dissociative State Scale (CADSS) or the Brief Psychiatric Rating Scale (BPRS)) versus psilocybin (Mystical Experience Questionnaire (MEQ30), other mystical experience measures, such as States of Consciousness Questionnaire (SOCQ), 5-Dimension Altered States of Consciousness questionnaire (5D-ASC)). These questionnaires capture fundamentally different aspects of subjective experience and measure essentially different constructs; (b) The ketamine studies predominantly used general psychiatric symptom scales (BPRS) or dissociation measures (CADSS) not specifically designed for ketamine effects. In contrast, the psilocybin studies used questionnaires specifically developed and validated to measure mystical-type psychedelic experiences, therefore psilocybin questionnaires are likely be more sensitive and specific to psilocybin’s psychedelic effects, potentially leading to stronger correlations with outcomes compared to the more general scales used for ketamine.

Therefore, discrepancy in measurement approaches likely contributes to the observed differences in correlation strengths between the two treatments, rather than reflecting true differences in the relationship between subjective effects and therapeutic outcomes. This observation is further strengthened by the fact that two ketamine studies of substance use disorder (SUD) used Hood’s mystical scale (HMS), which measures similar core dimensions of mystical experiences, and these studies showed comparable R2-values (54% for ketamine and 60% for psilocybin), which suggest that subjective effects may play a crucial role in certain therapeutic contexts. Moreover, this difference in measurement tools likely contributes to the higher heterogeneity observed in the ketamine studies (I2 = 55%) compared to the psilocybin studies (I2 = 28%).