Passive Evaluation in Operational Environment of the AI Clinician Decision Support System for Sepsis Treatment
Study Details
Study Description
Brief Summary
Sepsis, or systemic infection, is a common reason for ICU admission and death throughout the world. Despite advances in the way we treat this condition, it remains a significant economic and healthcare burden. A key part of the treatment of sepsis is the administration of IV fluids and blood pressure medication. However, there is huge uncertainty around dosing of these drugs in an individual patient. A tool to personalise these medications could improve patient survival. The study team has developed a new method to automatically and continuously review and recommend the correct medication doses to doctors, which was created using artificial intelligence (AI) techniques applied to large medical databases. The researchers' previous work has shown it has the potential to improve patient survival rates. The tool will be capable of processing patient data within the electronic patient record of NHS hospitals in real-time to suggest a course of action. This tool will be evaluated and refined in simulation studies and then be tested in two NHS Trusts in "shadow mode" (results not provided to duty clinicians). This will allow comparison of actual decisions made and recommended decisions from the AI system. The second stage of this clinical evaluation will display the recommendations to clinicians to assess the acceptability of the tool and confirm technical feasibility to inform future clinical trials. The long-term expected benefits of this project are numerous: improved patient survival, reduced use of precious intensive care resources and reduction in healthcare costs.
| Condition or Disease | Intervention/Treatment | Phase |
|---|---|---|
|
|
Detailed Description
Sepsis is life-threatening organ dysfunction due to severe infection and affects 250,000 patients annually in the UK (pre-COVID-19), of whom 48,000 die. In addition, virtually all COVID-19 intensive care unit (ICU) deaths had sepsis. It is a leading cause of death and the most expensive condition treated in hospitals. It was recognised as a top research priority by the James Lind Alliance, a partnership of patients and clinicians to prioritise the most pressing unanswered questions facing the NHS.
The cornerstone of sepsis resuscitation is the administration of intravenous fluids and/or vasopressors (drugs that squeeze the blood vessels to increase blood pressure) to maintain blood flow to prevent organ failure. However, there is huge uncertainty around the individual dosing of these drugs in an individual patient, partially due to high sepsis heterogeneity. The current guidelines provide recommendations at a population-level but fail to individualise the decisions. Wrong decisions lead to poorer outcomes and increased ICU-resource use. A tool to personalise these medications could improve patient survival.
The study team has developed a new method to automatically and continuously review and recommend the correct dose of these medications to doctors, which was created using artificial intelligence (AI) techniques applied to large medical databases. The method used is called reinforcement learning. In this framework, the study models patients with sepsis in the ICU as belonging to a large number of possible disease states, and analyses what interventions are likely to help them transition to healthier states, and eventually to survival. The researchers demonstrated in their initial publication that the value of the AI selected strategy was on average reliably higher than human clinicians. In a large validation cohort independent from the training data, mortality was lowest in patients where clinicians' actual doses matched the AI decisions: mortality rates rose, in a dose dependent manner, as the clinicians' actual decisions diverged from the AI decisions. The study team has estimated that their AI algorithm could reduce mortality by 10% (in relative terms), which represents over 1,000 lives saved annually in the UK and would scale to hundreds of thousands of lives worldwide. Now, the study team intends to start clinical testing of this AI technology in the UK.
The envisioned end-product will be a piece of software that will be accessible by clinicians (ICU doctors initially, then eventually to ICU nurses as well) at the bedside in intensive care. This software will be connected to the electronic patient record, which will be fed to the AI algorithm. In return, the AI will identify where the patient sits in the array of possible disease states, and which actions (a dose of intravenous fluids and vasopressors) are most likely to be beneficial.
First, the study team will develop this software tool, capable of processing patient data within the electronic patient record of NHS hospitals in real-time to suggest a course of action. The study will start by evaluating and refining this tool in simulation studies. The study team will then test the AI tool in two NHS Trusts in a "shadow mode" when the result is not provided to duty clinicians in charge of patient care. This will allow comparison of actual decisions made and recommended decisions from the AI system. In the second stage of the clinical evaluation, the study team will display the recommendations to clinicians to assess the acceptability of the tool to clinicians and also confirm the technical feasibility to inform future large scale clinical trials.
The long-term expected benefits of this project are numerous: improved patient survival, reduced use of precious intensive care resources and reduction in healthcare costs.
Study Design
Outcome Measures
Primary Outcome Measures
- Data Availability [18 months]
Data availability: what percentage of essential and optional data fields are available 24/7.
- Anonymised patients' data [18 months]
Patient demographics (age in years, gender, primary diagnosis) Outcomes: organ function (hourly SOFA), ICU and hospital mortality
- Rate of intravenous fluids administered to patients [18 months]
Rate of intravenous fluids (millilitres per hour) and vasopressors (noradrenaline equivalent, in micrograms per kilogram per minute) administered to patients.
- Evaluators' data (the doctors assessing the AI in the background) [18 months]
The evaluator's grade and seniority (i.e. A custom-made interface linked to a database will capture and record the following: What rate of intravenous fluids (millilitres per hour) and vasopressors (noradrenaline equivalent, in micrograms per kilogram per minute) they would recommend, before and after seeing the AI suggested dose. Agreement with the AI suggested dose on a Likert scale including the following labels: "the AI suggested dose is:" "certainly too low", "possibly too low","likely appropriate","possibly too high", "certainly too high". After collecting this data, we will report the median difference (with interquartile range) between the AI suggested doses and the evaluator suggested doses, as well as the proportion of AI decisions in each agreement category ("certainly too low", "possibly too low","likely appropriate","possibly too high", "certainly too high").
- System Availability [18 months]
System availability: delays in generating response 24/7. Number and nature of technical issues (drop-outs, freezes).
Eligibility Criteria
Criteria
Inclusion Criteria:
For patients:
-
Adult patient > 18 years old
-
Admitted to an intensive care unit
-
Likely or confirmed diagnosis of sepsis as per sepsis-3 definition (as defined in the glossary)
-
ICU length of stay > 24h
For Evaluators:
- ICU doctors at the senior registrar, ICU fellow or consultant level
Exclusion Criteria:
For patients:
-
Not for full active care, e.g. not for vasopressors
-
Not expected to survive more than 24h
-
Elective surgical admission (these patients are regularly on antibiotics but given as a prophylaxis, with no sepsis)
-
Opted-out for use of their data for research (NHS and NHS-X website)
For both patients and evaluators:
Declined participation No patient consent is required
Contacts and Locations
Locations
| Site | City | State | Country | Postal Code | |
|---|---|---|---|---|---|
| 1 | Univeristy College London Hospitals NHS Foundation Trust | London | United Kingdom | NW1 2PG | |
| 2 | Imperial College Hospitals NHS Trust | London | United Kingdom | W2 1BL |
Sponsors and Collaborators
- Imperial College London
- National Institute for Health Research, United Kingdom
Investigators
None specified.Study Documents (Full-Text)
More Information
Publications
None provided.- 20HH6297