VALIDATE: Validate: Trustworthy AI to Improve Acute Stroke Outcomes
Study Details
Study Description
Brief Summary
Artificial intelligence (AI)-powered prognostic tools and clinical decision support systems can predict the outcome of certain diseases based on a multitude of patient data at high speed, facilitating decisions by healthcare professionals. In acute ischemic stroke, the overall treatment effect and population-wide outcome benefit of treatments such as IV thrombolysis and mechanical thrombectomy are well established. However, in individual patients it is difficult to predict the prognosis in the acute phase of stroke: some patients are candidates for these treatments, but may have poor clinical outcomes (no improvement of stroke or even worsening) Our aim in this study is to validate an artificial intelligence (AI)-based prognostic tool to provide accurate real-time outcome prediction in patients with acute ischemic stroke.
During the study, all patients admitted to the emergency room with an acute ischemic stroke will receive the usual treatment for acute stroke in accordance with the stroke neurologists in charge. A "shadow" clinical researcher, without interaction with treating physicians, will collect the data required by the AI model in vivo. These data will be obtained by filling in clinical data through an App on a hospital mobile/tablet, and by a connection with your electronic medical record.
The AI models will estimate the outcome of the acute stroke patient, and this prediction will be compared with the real outcome of the patient after 3 months of follow-up.
| Condition or Disease | Intervention/Treatment | Phase |
|---|---|---|
|
Detailed Description
Artificial intelligence (AI)-powered prognostic tools and clinical decision support systems have the ability to predict the outcome of certain diseases based on a multitude of patient data at high speed, facilitating decisions by healthcare professionals.
In acute ischemic stroke, the overall treatment effect and population-wide outcome benefit of IV thrombolysis and mechanical thrombectomy are well established. However, the outcome still differs significantly for individual patients, where some are eligible for treatment but may have catastrophic outcomes. Multiple prognostic variables and their combination in a single patient make it difficult to predict individual outcomes after stroke treatment. We aim to validate an AI prognostic tool to provide accurate outcome prediction in patients with acute ischemic stroke in a prospective observational follow-up study.
Hypothesis AI-based models can be applied in real-time in acute stroke patients and provide an early accurate prediction of their outcome.
Methodology The study complies two phases. Phase 1: retrospective study. While technological readiness will be achieved for the clinical validation study further model refinement on heterogeneous data will be performed based on existing models that have been developed on extensive high-quality medical data. VALIDATE will analyse retrospective databases from the 3 clinical sites involved in the study to test and validate the previously generated AI models. Encrypted data of all acute ischemic stroke patients admitted to the centres during 2018-2021 period will fed the AI models to validate the model's accuracy comparing the outcomes predicted by the AI modelling with those of the actual patients. The dataset includes demographics, baseline clinical characteristics, risk factors, neuroimaging data, acute treatments, clinical evaluation (National Institute of Health Stroke Scale (NIHSS)), functional evaluation at 3-6 months (mRS), patient reported outcome measures (PROMs), etc. The interaction between these data sets and the AI models will be done through a federated learning procedure, that is, the data will be analyzed on our hospital servers, and they will not be transferred to any other center.
Grading the contribution of the progressively complex diagnostic procedures to the AI models and establishing a set of the minimum relevant variables for the AI model able to accurately predict functional outcome will also be achieved.
Phase 3: Prospective multicenter observational shadowing study. consecutive acute stroke patients admitted to 3 high-volume comprehensive stroke centres will be evaluated. All patients will receive the usual stroke work-up and standard of care treatment according to the treating physicians. A shadow clinical researcher with no interaction with the treating physicians will recollect in vivo the data required by the AI modelling. These data will be obtained by filling of clinical data through an app and by connection with the electronical medical record of the patient to obtain additional baseline and neuroimaging data. The real outcomes of the patients will be measured through clinician and patient reported outcomes measurements (CROMs and PROMs), and they will be compared with the estimated outcomes according to the AI model. PROMS after 7 days, 1 and 3 months will be obtained through the implementation of an innovative nudging-based digital platform (NORA) to improve patient-clinician communication and follow-up. An electronic case report form (eCRF) will be designed to recollect key process indicators (KPI) and CROMs, which will be integrated in the NORA platform.
The sample size calculation has been based on the results of a clinical dataset of consecutive code stroke patients admitted to Hospital Vall d'Hebron during the year 2020. It has been used as an example of the usual mRS distribution at 3 months in a cohort of consecutive acute stroke patients.
In a test for agreement between two raters using the Kappa statistic, a sample size of 182 subjects achieves 95% power to detect a true Kappa value of 0,7 in a test of H0: Kappa = κ0 vs. H1: Kappa ≠ κ0 when there are three categories with frequencies equal to 0,58, 0,34, and 0,08. This power calculation is based on a significance level of 0,05 and a minimum acceptable kappa (κ0) of 0.6 (moderate agreement) and an expected kappa (κ1) of 0.8 (substantial agreement). Assuming a drop-out rate of 20% for the 90-day follow-up the Dropout-Inflated Expected Enrolment (DIEE) Number would be 218 patients with acute ischemic stroke.
Study Design
Outcome Measures
Primary Outcome Measures
- Distribution of functional outcome predicted by real-time AI models measured by mRS [3 months]
Distribution of functional outcome measured by the modified Rankin scale (mRS) (trichotomized: 0-2, 3-4 and 5-6) predicted by the real-time AI models compared with the real patient's mRS distribution
Secondary Outcome Measures
- Additional clinician reported (CRO) and patient reported (PRO) outcomes prediction [1 week, 1 month and 3 months]
Evaluate the accuracy of AI models to predict additional predefined clinician and patient reported outcomes (CROs/PROs)
- Demonstrate the feasibility of applying AI models in real life [1 day]
AI-based forecast synchronization, problems with system integration, ..
Eligibility Criteria
Criteria
Inclusion Criteria:
-
Subject is 18 years of age or older, or of legal age to give informed consent per state or national law
-
Informed consent for the use of data, obtained from patient or his or her legally designated representative (if locally required)
Exclusion Criteria:
-
Neuroimaging (CT/MRI) with signs of acute intracranial haemorrhage
-
Serious, advanced, or terminal illness with anticipated life expectancy of less than 3 months
-
Unlikely to be available for 90-day follow-up (e.g., no fixed home address, no telephone, etc.)
Contacts and Locations
Locations
| Site | City | State | Country | Postal Code | |
|---|---|---|---|---|---|
| 1 | Universität Heidelberg | Heidelberg | Germany | ||
| 2 | Hadassah Medical Center | Jerusalem | Israel | ||
| 3 | Hospital Vall d'Hebron - VHIR | Barcelona | Catalonia | Spain | 08035 |
Sponsors and Collaborators
- Hospital Universitari Vall d'Hebron Research Institute
- VALIDATE CONSORTIUM
Investigators
None specified.Study Documents (Full-Text)
None provided.More Information
Publications
None provided.- PR(AG)310-2022
- 101057263