Our commitment to safety
In an era where technology and artificial intelligence (AI) is reshaping the landscape of healthcare and beyond, it is essential that the products we build are safe, reliable, and beneficial to all users. This guide outlines the core principles and commitments that drive our approach when developing Clinical AI, ensuring they meet the highest standards of safety, compliance, and efficacy.
In this post we will cover the following five key pillars that define our values and standards of AI safety. They are unwavering commitments to:
Predictability
Explainability
Accountability
Security
Real World Evaluation
1. Predictability
We believe that the foundation of a safe product lies in its ability to perform its intended function reliably and consistently. For our AI products, this commitment is underpinned by strong machine learning principles, task-specific application development and predictability through excellence. Let’s break this down.
Strong Data Science Principles: We prioritise the legal and appropriate collection of large, complete, and representative data sets tailored to the user population our products serve. By doing so, we ensure and validate the accuracy of our AI algorithms and the reliability of their outcomes. For example, when evaluating the effectiveness of our Machine Learning (ML) model used in Limbic Access, we tested performance across 20,000+ data points. Read more in the white paper.
Task-Specific Application Development: We design algorithms and features with clear, defined, and often, narrow goals in mind. This approach allows us to assess the accuracy of our system as a whole and understand the performance of individual components within it. Often Large Language Models (LLMs) enable developers to be vague about the purpose of the model–which can be risky. For example, if we instruct an LLM to simply ‘act like a therapist’, the LLM will attempt to conduct clinical sub-tasks such as diagnosing, treatment allocation, and treatment planning, which the LLM may not have the context for. With vague instructions, it can become difficult to assess the performance and potential shortcomings on these specialised clinical tasks. At Limbic, we build algorithms that are rigorously trained for specific clinical tasks which are evaluated on specific and proprietary data. This specificity enables tighter control over the behaviour of models and their performance against each task.
Predictability Through Excellence: The predictability of our products is a direct result of our meticulous development process, powered by a world-class team, technology stack, and a foundational design process. For instance, our ISO 13485:2016 certifies our commitment and ability to build our products to medical device standards in a controlled, reproducible, and methodical manner.
2. Explainability
Transparency and accountability are key to building trust and ensuring the clinical safety of our products. We are committed to building explainable AI and designing rigorous testing processes.
Building explainable AI: Our AI models are designed to be transparent, allowing us to understand and explain the reasoning behind their outputs. This philosophy ensures our products are not ‘black boxes’ and are instead designed to be audited. Each clinical task, such as identifying presenting problems or selecting additional questionnaires to administer, is supported by a rigorously trained and dedicated Machine Learning model or clinical logic. This allows us to understand ‘why’ our AI made a particular decision or arrived at a specific output. Moreover, our clinical, research, and product teams regularly conduct interviews with mental health practitioners to understand how we can improve the implementation of AI systems to build safer, explainable AI. These conversations highlight the importance of utilising AI systems as decision support tools that amplify clinicians, making their touch points with patients as high value as possible. In October 2022, we conducted a foundational study on this topic.
Rigorous testing processes: We employ a comprehensive suite of tests, including automatic and manual unit, regression, end-to-end, integration, accuracy, safety, and usability tests. This extensive and multistage stage testing process runs against all of our features, which leads to tens of thousands of individual tests before any feature is deployed in the product with our patients. Our testing platform is compliant with ISO 14971, an international standard for rigorous risk management of management devices throughout their lifecycle, and IEC 62304, the international standard for ensuring that software used in medical devices is safe and performs as intended.
3. Accountability
We position ourselves as leaders in clinical scrutiny and excellence, constantly striving to exceed the standards of compliance and integrity. Our accountability is demonstrated through our commitment to regulatory leadership and open collaboration. Let’s dive in:
Regulatory Leadership: We take pride in pioneering the regulation of AI-driven tools for mental health, setting new benchmarks for compliance, safety, and performance. We will never compromise on seeking out and meeting the highest clinical standards to ensure patients and services have confidence that our products are safe and rigorously tested for clinical settings. To this end, we are the only mental healthcare company with a Class IIa UKCA medical device status for an AI-enabled clinical decision support product.
Open Collaboration: We maintain a transparent and open relationship with national bodies, healthcare services, patients, and the broader ecosystem, ensuring our operations and products are open for scrutiny. Working closely with NICE, NHSE, and other national bodies ensures we build AI responsibly. Our numerous accreditations and high ORCHA score of 92% reflect our commitment to safety and compliance, surpassing industry standards. Currently, our Research team is collaborating with the AI Safety Institute on an exciting new study, understanding the impacts of AI models on user well-being and mental health. In the past we have collaborated with Society Generale (SGS), one of the world’s leading, testing, inspection and certification companies, to audit our medical device practices. In 2023, we hosted demystify_Ai, a series of short talks from our research & clinical team focused on demystifying artificial intelligence technologies to demonstrate how they can be used in Mental Health Services.
4. Security
Recognising the sensitivity of health information, we embed security in every aspect of our operation. Our commitment to data security is reflected in comprehensive compliance measures and rigorous security practices.
Comprehensive Compliance: Our infrastructure is compliant with ISO 27001, GDPR, HIPAA, and NHS Data Security and Protection Toolkit, ensuring the lawful and ethical handling of data. We also have maintained an Information Security Management System in place since 2020 and hold a Cyber Essentials certification.
Rigorous Security Practices: Through regular penetration testing, internal auditing, and business continuity management, we maintain the highest standards of data security and protect patient information diligently. As no system is perfect, we will continuously seek out areas of vulnerability and hold ourselves accountable.
5. Real World Evaluation
Continuous evaluation in the real world is crucial for maintaining and enhancing the safety and performance of our products. With a commitment to extensive monitoring and research and seeking out real-world impact, our team looks beyond AI performance to see how, when implemented, Limbic improves patients' lives.
Extensive monitoring and research: Our Clinical and Research teams regularly publish robust, peer-reviewed studies in respected journals. In 2024 alone, we published well-respected papers in the British Medical Journal, the Journal of Medical Internet Research, and Nature Medicine. These papers not only validate the performance of our products, but that when implemented in the ‘real world’ in clinical settings, our products genuinely improve the lives of patients and clinicians across over 250,000 data points. We have developed a world-class Clinical Evaluation team to drive forward such efforts.
Real World Impact: In addition to published research, we seek validation from industry peers and bodies to understand more about the impact Limbic makes. We have demonstrated our ability to deliver impact to clinicians and patients’ lives with the following nomination and awards from the respected Health Service Journal. Limbic was shortlisted for the HSJ Partnership Award (2023), Limbic won Improving Mental Health through Digital Award (2023), and finally, Limbic won the Best Mental Health Partnership with the NHS Award (2023).
Conclusion
Through adherence to these core principles, we not only aim to develop products that are safe and beneficial for our users but also aspire to set new standards for excellence and trustworthiness in the AI and healthcare industries. Our journey is one of continuous improvement, driven by a commitment to accuracy, transparency, security, and above all, the well-being of those we serve.
