Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients
- Publikationstyp:
- Zeitschriftenaufsatz
- Metadaten:
-
- Autoren
- Severin Hennemann
- Sebastian Kuhn
- Michael Witthoeft
- Stefanie M Jungmann
- Autoren-URL
- https://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=fis-test-1&SrcAuth=WosAPI&KeyUT=WOS:000767333000001&DestLinkType=FullRecord&DestApp=WOS_CPL
- DOI
- 10.2196/32832
- Externe Identifier
- Clarivate Analytics Document Solution ID: ZQ8FB
- PubMed Identifier: 35099395
- ISSN
- 2368-7959
- Ausgabe der Veröffentlichung
- 1
- Zeitschrift
- JMIR MENTAL HEALTH
- Schlüsselwörter
- mHealth
- symptom checker
- diagnostics
- mental disorders
- psychotherapy
- mobile phone
- Artikelnummer
- ARTN e32832
- Datum der Veröffentlichung
- 2022
- Status
- Published
- Titel
- Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients
- Sub types
- Article
- Ausgabe der Zeitschrift
- 9
Datenquelle: Web of Science (Lite)
- Andere Metadatenquellen:
-
- Abstract
- <jats:sec> <jats:title>Background</jats:title> <jats:p>Digital technologies have become a common starting point for health-related information-seeking. Web- or app-based symptom checkers aim to provide rapid and accurate condition suggestions and triage advice but have not yet been investigated for mental disorders in routine health care settings.</jats:p> </jats:sec> <jats:sec> <jats:title>Objective</jats:title> <jats:p>This study aims to test the diagnostic performance of a widely available symptom checker in the context of formal diagnosis of mental disorders when compared with therapists’ diagnoses based on structured clinical interviews.</jats:p> </jats:sec> <jats:sec> <jats:title>Methods</jats:title> <jats:p>Adult patients from an outpatient psychotherapy clinic used the app-based symptom checker Ada–check your health (ADA; Ada Health GmbH) at intake. Accuracy was assessed as the agreement of the first and 1 of the first 5 condition suggestions of ADA with at least one of the interview-based therapist diagnoses. In addition, sensitivity, specificity, and interrater reliabilities (Gwet first-order agreement coefficient [AC1]) were calculated for the 3 most prevalent disorder categories. Self-reported usability (assessed using the System Usability Scale) and acceptance of ADA (assessed using an adapted feedback questionnaire) were evaluated.</jats:p> </jats:sec> <jats:sec> <jats:title>Results</jats:title> <jats:p>A total of 49 patients (30/49, 61% women; mean age 33.41, SD 12.79 years) were included in this study. Across all patients, the interview-based diagnoses matched ADA’s first condition suggestion in 51% (25/49; 95% CI 37.5-64.4) of cases and 1 of the first 5 condition suggestions in 69% (34/49; 95% CI 55.4-80.6) of cases. Within the main disorder categories, the accuracy of ADA’s first condition suggestion was 0.82 for somatoform and associated disorders, 0.65 for affective disorders, and 0.53 for anxiety disorders. Interrater reliabilities ranged from low (AC1=0.15 for anxiety disorders) to good (AC1=0.76 for somatoform and associated disorders). The usability of ADA was rated as high in the System Usability Scale (mean 81.51, SD 11.82, score range 0-100). Approximately 71% (35/49) of participants would have preferred a face-to-face over an app-based diagnostic.</jats:p> </jats:sec> <jats:sec> <jats:title>Conclusions</jats:title> <jats:p>Overall, our findings suggest that a widely available symptom checker used in the formal diagnosis of mental disorders could provide clinicians with a list of condition suggestions with moderate-to-good accuracy. However, diagnostic performance was heterogeneous between disorder categories and included low interrater reliability. Although symptom checkers have some potential to complement the diagnostic process as a screening tool, the diagnostic performance should be tested in larger samples and in comparison with further diagnostic instruments.</jats:p> </jats:sec>
- Autoren
- Severin Hennemann
- Sebastian Kuhn
- Michael Witthöft
- Stefanie M Jungmann
- DOI
- 10.2196/32832
- eISSN
- 2368-7959
- Ausgabe der Veröffentlichung
- 1
- Zeitschrift
- JMIR Mental Health
- Sprache
- en
- Online publication date
- 2022
- Paginierung
- e32832 - e32832
- Status
- Published online
- Herausgeber
- JMIR Publications Inc.
- Herausgeber URL
- http://dx.doi.org/10.2196/32832
- Datum der Datenerfassung
- 2022
- Titel
- Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients
- Ausgabe der Zeitschrift
- 9
Datenquelle: Crossref
- Abstract
- <h4>Background</h4>Digital technologies have become a common starting point for health-related information-seeking. Web- or app-based symptom checkers aim to provide rapid and accurate condition suggestions and triage advice but have not yet been investigated for mental disorders in routine health care settings.<h4>Objective</h4>This study aims to test the diagnostic performance of a widely available symptom checker in the context of formal diagnosis of mental disorders when compared with therapists' diagnoses based on structured clinical interviews.<h4>Methods</h4>Adult patients from an outpatient psychotherapy clinic used the app-based symptom checker Ada-check your health (ADA; Ada Health GmbH) at intake. Accuracy was assessed as the agreement of the first and 1 of the first 5 condition suggestions of ADA with at least one of the interview-based therapist diagnoses. In addition, sensitivity, specificity, and interrater reliabilities (Gwet first-order agreement coefficient [AC1]) were calculated for the 3 most prevalent disorder categories. Self-reported usability (assessed using the System Usability Scale) and acceptance of ADA (assessed using an adapted feedback questionnaire) were evaluated.<h4>Results</h4>A total of 49 patients (30/49, 61% women; mean age 33.41, SD 12.79 years) were included in this study. Across all patients, the interview-based diagnoses matched ADA's first condition suggestion in 51% (25/49; 95% CI 37.5-64.4) of cases and 1 of the first 5 condition suggestions in 69% (34/49; 95% CI 55.4-80.6) of cases. Within the main disorder categories, the accuracy of ADA's first condition suggestion was 0.82 for somatoform and associated disorders, 0.65 for affective disorders, and 0.53 for anxiety disorders. Interrater reliabilities ranged from low (AC1=0.15 for anxiety disorders) to good (AC1=0.76 for somatoform and associated disorders). The usability of ADA was rated as high in the System Usability Scale (mean 81.51, SD 11.82, score range 0-100). Approximately 71% (35/49) of participants would have preferred a face-to-face over an app-based diagnostic.<h4>Conclusions</h4>Overall, our findings suggest that a widely available symptom checker used in the formal diagnosis of mental disorders could provide clinicians with a list of condition suggestions with moderate-to-good accuracy. However, diagnostic performance was heterogeneous between disorder categories and included low interrater reliability. Although symptom checkers have some potential to complement the diagnostic process as a screening tool, the diagnostic performance should be tested in larger samples and in comparison with further diagnostic instruments.
- Addresses
- Department of Clinical Psychology, Psychotherapy and Experimental Psychopathology, University of Mainz, Mainz, Germany.
- Autoren
- Severin Hennemann
- Sebastian Kuhn
- Michael Witthöft
- Stefanie M Jungmann
- DOI
- 10.2196/32832
- eISSN
- 2368-7959
- Externe Identifier
- PubMed Identifier: 35099395
- PubMed Central ID: PMC8844983
- Open access
- true
- ISSN
- 2368-7959
- Ausgabe der Veröffentlichung
- 1
- Zeitschrift
- JMIR mental health
- Sprache
- eng
- Medium
- Electronic
- Online publication date
- 2022
- Open access status
- Open Access
- Paginierung
- e32832
- Datum der Veröffentlichung
- 2022
- Status
- Published
- Publisher licence
- CC BY
- Datum der Datenerfassung
- 2022
- Titel
- Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients.
- Sub types
- research-article
- Journal Article
- Ausgabe der Zeitschrift
- 9
Datenquelle: Europe PubMed Central
- Abstract
- BACKGROUND: Digital technologies have become a common starting point for health-related information-seeking. Web- or app-based symptom checkers aim to provide rapid and accurate condition suggestions and triage advice but have not yet been investigated for mental disorders in routine health care settings. OBJECTIVE: This study aims to test the diagnostic performance of a widely available symptom checker in the context of formal diagnosis of mental disorders when compared with therapists' diagnoses based on structured clinical interviews. METHODS: Adult patients from an outpatient psychotherapy clinic used the app-based symptom checker Ada-check your health (ADA; Ada Health GmbH) at intake. Accuracy was assessed as the agreement of the first and 1 of the first 5 condition suggestions of ADA with at least one of the interview-based therapist diagnoses. In addition, sensitivity, specificity, and interrater reliabilities (Gwet first-order agreement coefficient [AC1]) were calculated for the 3 most prevalent disorder categories. Self-reported usability (assessed using the System Usability Scale) and acceptance of ADA (assessed using an adapted feedback questionnaire) were evaluated. RESULTS: A total of 49 patients (30/49, 61% women; mean age 33.41, SD 12.79 years) were included in this study. Across all patients, the interview-based diagnoses matched ADA's first condition suggestion in 51% (25/49; 95% CI 37.5-64.4) of cases and 1 of the first 5 condition suggestions in 69% (34/49; 95% CI 55.4-80.6) of cases. Within the main disorder categories, the accuracy of ADA's first condition suggestion was 0.82 for somatoform and associated disorders, 0.65 for affective disorders, and 0.53 for anxiety disorders. Interrater reliabilities ranged from low (AC1=0.15 for anxiety disorders) to good (AC1=0.76 for somatoform and associated disorders). The usability of ADA was rated as high in the System Usability Scale (mean 81.51, SD 11.82, score range 0-100). Approximately 71% (35/49) of participants would have preferred a face-to-face over an app-based diagnostic. CONCLUSIONS: Overall, our findings suggest that a widely available symptom checker used in the formal diagnosis of mental disorders could provide clinicians with a list of condition suggestions with moderate-to-good accuracy. However, diagnostic performance was heterogeneous between disorder categories and included low interrater reliability. Although symptom checkers have some potential to complement the diagnostic process as a screening tool, the diagnostic performance should be tested in larger samples and in comparison with further diagnostic instruments.
- Date of acceptance
- 2021
- Autoren
- Severin Hennemann
- Sebastian Kuhn
- Michael Witthöft
- Stefanie M Jungmann
- Autoren-URL
- https://www.ncbi.nlm.nih.gov/pubmed/35099395
- DOI
- 10.2196/32832
- Externe Identifier
- PubMed Central ID: PMC8844983
- ISSN
- 2368-7959
- Ausgabe der Veröffentlichung
- 1
- Zeitschrift
- JMIR Ment Health
- Schlüsselwörter
- diagnostics
- mHealth
- mental disorders
- mobile phone
- psychotherapy
- symptom checker
- Sprache
- eng
- Country
- Canada
- Paginierung
- e32832
- PII
- v9i1e32832
- Datum der Veröffentlichung
- 2022
- Status
- Published online
- Titel
- Diagnostic Performance of an App-Based Symptom Checker in Mental Disorders: Comparative Study in Psychotherapy Outpatients.
- Sub types
- Journal Article
- Ausgabe der Zeitschrift
- 9
Datenquelle: PubMed
- Beziehungen:
- Eigentum von