Improving the Quality of Clinical Assessments in Talking Therapies Through Machine Learning
Key Takeaways
Limbic Access identifies common mental health conditions with 93% accuracy
Background
We developed a Clinical AI Assessment Assistant called Limbic Access, equipped with a clinical brain that uses specialised clinical machine learning models to inform clinical assessments and improve diagnostic quality. In this white paper, we collate three studies with a total of over 20,000 patients, and we show that Limbic Access' Machine Learning (ML) model identifies the appropriate additional outcome measures (known as Anxiety Disorder Specific Measures) with an accuracy of over 93% across the eight most common diagnostic categories, matching the performance of trained clinicians. Collecting additional, relevant outcome measures before assessment helps clinicians deliver a more informed assessment.
Methods
The ML model operates on a set of preprocessed input vectors (free text, questionnaire scores, behavioural indicators and demographic variables). The model transforms these preprocessed inputs into a set of probabilities across a Ranked Consideration Set over eight common diagnostic categories. The ranking is sorted in descending order of probability. The main evaluation metric of interest is the accuracy with which the algorithm would administer the relevant ADSM for a diagnosis, if that diagnosis was present. Formally that means the percentage of times with which the actual diagnosis is within the top two problems in Ranked Consideration Set selected and ranked by our machine learning model. With additional data collected by the selected outcome measures, Clinical Logic can be applied to generate a set of primary and secondary presenting problems, which can then be used by Clinicians to inform a more effective assessment.
Results
On historical data, the model achieved an overall accuracy of 93.5%. Moreover, it achieved good accuracy for each of the individual diagnostic categories as well, indicating that it has good performance. The model achieved a similar level of accuracy on our prospective evaluation - achieving an overall accuracy of 94.2%. On the live dataset, the Limbic Access correctly detected 92.47% of diagnoses, comparable to the performance on historical and prospective datasets.
93% accurate
Limbic Access machine learning accuracy when selecting the correct ADSMs
Conclusions
Together, this evidence supports our approach of using ML to
administer relevant ADSMs, and presenting clinicians with the entire set of model-informed Primary and Secondary Presenting Problems to assist in their assessments. By appropriately administering ADSMs pre-assessment and accurately identifying presenting problems, Limbic Access can save clinicians time and support the quality of their assessments, as well as offer patients an enhanced referral experience and potentially better outcomes.
