Study Reveals Variable Reliability in Mental Health Diagnostic Interviews

The Lead

Diagnostic interviews for mental health conditions, commonly used to diagnose disorders including depression, anxiety, bipolar, and personality disorders, show significant variation in reliability according to a new study published in Jama Network Open. The research challenges the long-held assumption that these interviews serve as a definitive "gold standard" for mental health assessment.

The Study's Findings on Diagnostic Reliability

Laura Duncan, a psychiatry professor at McMaster University in Ontario, Canada and one of the study's authors, pointed out that diagnostic interviews "continue to be widely viewed as the best available approach, possibly due to the lack of better alternatives." The review study brings together evidence from studies on "test-retest reliability" of diagnostic interviews from February 2024 to September 2025.

The study's authors used Cohen's kappa coefficient to estimate reliability, measuring how often patients would receive the same diagnosis when given the same diagnostic interview twice, accounting for chance agreement. The average reliability was generally better for substance use disorders, with opioid use disorder showing the highest overall reliability. Duncan attributed this to substance use disorder criteria being largely behavior-based, making them easier to quantify than symptoms like sadness or anxiety.

The Data Analysis: Interview Types and Their Limitations

The review included papers on various diagnostic tools including the Structured Clinical Interview for DSM 5 (SCID) and Mini International Neuropsychiatric Interview (Mini), as well as tools for specific disorders like the Clinically Administered PTSD Scale (Caps).

Dr. Michael First, a psychiatrist and professor at Columbia University who authored the SCID, criticized the study for lumping "fully structured" and "semi-structured" interviews together. Fully structured interviews follow a strict script and are more likely to yield consistent results, while semi-structured interviews allow clinicians to ask follow-up questions based on patient responses, potentially leading to more accurate diagnoses but also more variability between sessions.

Despite these limitations, both experts agree that more objective laboratory tests for mental conditions are needed, though First noted that psychiatrists have been hoping for such tests "for 50 years" without success.

The Impact Analysis: Shaping the Future of Psychiatric Diagnosis

The study highlights a critical need for more rigor in psychiatric diagnosis methods. While diagnostic interviews remain the primary tool for assessment, their variable reliability raises questions about the consistency of mental health diagnoses across different settings and providers.

The research underscores the challenges in mental health assessment, where subjective reporting of symptoms often forms the basis of diagnosis. This variability can have significant implications for treatment decisions, research outcomes, and patient care across healthcare systems.

The criticism from experts like Dr. First also points to methodological challenges in studying diagnostic tools themselves, including inconsistent reporting of interview formats and designs in research literature.

The Prediction: Toward a New Diagnostic Paradigm

Looking forward, Duncan suggested an alternative approach where clinicians "move away from strict diagnostic categories, where a condition is either present or absent, and think about symptoms on a spectrum or continuum." This shift could potentially lead to more nuanced understanding and treatment of mental health conditions.

As the field continues to evolve, there's a clear need for both improved diagnostic instruments and more comprehensive research comparing different interview methodologies. The study's authors emphasize that the limitations identified in current diagnostic approaches should motivate further development of more reliable assessment tools in psychiatry.