Multi-Reader Retrospective Study Examining Carebot AI CXR 2.0.21-v2.01 Implementation in Everyday Radiology Clinical Practice

Sponsor

Carebot s.r.o. (Industry)

Overall Status

Completed

CT.gov ID

NCT05963945

Collaborator

(none)

956

Enrollment

Location

5.1

Actual Duration (Months)

188.9

Patients Per Site Per Month

Study Details

Study Description

Brief Summary

The primary objective is to evaluate the performance parameters of the proposed DLAD (Carebot AI CXR) in comparison to individual radiologists.

Condition or Disease	Intervention/Treatment	Phase
Pneumothorax Pulmonary Nodule, Solitary Atelectasis Subcutaneous Emphysema Cardiomegaly Consolidation Pleural Effusion	Device: Carebot AI CXR

Detailed Description

In the period between October 18th, 2022, and November 17th, 2022, anonymized chest X-ray images of patients were collected at the Radiodiagnostic Department of the Havířov Hospital, p.o. The collection process involved utilizing the CloudPACS imaging and archiving system provided by OR-CZ spol. s r.o.

The collected X-ray images were subjected to the proposed DLAD (Carebot AI CXR) for analysis. Subsequently, the DLAD's performance was compared with the standard clinical practice, where radiologists assessed the CXR images in the simulated hospital setting with access to standard viewing tools (e.g., pan, zoom, WW/WL) and were given an unlimited amount of time to complete the review. Each radiologist determined the presence or absence of 7 indicated radiological findings, including atelectasis (ATE), consolidation (CON), pleural effusion (EFF), pulmonary lesion (LES), subcutaneous emphysema (SCE), cardiomegaly (CMG), and pneumothorax (PNO).

Study Design

Study Type:

Observational

Actual Enrollment :

956 participants

Observational Model:

Cohort

Time Perspective:

Retrospective

Official Title:

Multi-Reader Retrospective Study Examining Carebot AI CXR 2.0.21-v2.01 Implementation in Everyday Radiology Clinical Practice

Actual Study Start Date :

Oct 18, 2022

Actual Primary Completion Date :

Nov 17, 2022

Actual Study Completion Date :

Mar 21, 2023

Arms and Interventions

Arm	Intervention/Treatment
Retrospective collection for the period October 18th, 2022, and November 17th, 2022 A total of 1,073 chest X-rays were acquired within the specified period at the department. The data collection remained intact and unaffected throughout the testing phase, ensuring the integrity of the dataset. The collected sample accurately represents the prevalence of findings within the observed population. After excluding ineligible studies such as X-rays from patients under 18 years of age, lateral projection X-rays, and scans of insufficient quality, a total of 956 relevant CXRs were identified for further assessment.	Device: Carebot AI CXR The proposed DLAD (Carebot AI CXR) is a deep learning-based medical device designed to assist radiologists in interpreting chest X-ray images acquired in anteroposterior (AP) or posteroanterior (PA) projection. By employing advanced deep learning algorithms, this solution enables automatic detection of abnormal findings by analyzing visual patterns associated with specific conditions. The targeted abnormalities include atelectasis (ATE), consolidation (CON), pleural effusion (EFF), pulmonary lesion (LES), subcutaneous emphysema (SCE), cardiomegaly (CMG), and pneumothorax (PNO). The DLAD functions as a prediction algorithm complemented by various application peripherals, such as web-based communication tools, DICOM file conversion capabilities, and storage and reporting libraries supporting both DICOM Structured Report and DICOM Presentation State formats.

Arm

Intervention/Treatment

Retrospective collection for the period October 18th, 2022, and November 17th, 2022

A total of 1,073 chest X-rays were acquired within the specified period at the department. The data collection remained intact and unaffected throughout the testing phase, ensuring the integrity of the dataset. The collected sample accurately represents the prevalence of findings within the observed population. After excluding ineligible studies such as X-rays from patients under 18 years of age, lateral projection X-rays, and scans of insufficient quality, a total of 956 relevant CXRs were identified for further assessment.

Device: Carebot AI CXR

The proposed DLAD (Carebot AI CXR) is a deep learning-based medical device designed to assist radiologists in interpreting chest X-ray images acquired in anteroposterior (AP) or posteroanterior (PA) projection. By employing advanced deep learning algorithms, this solution enables automatic detection of abnormal findings by analyzing visual patterns associated with specific conditions. The targeted abnormalities include atelectasis (ATE), consolidation (CON), pleural effusion (EFF), pulmonary lesion (LES), subcutaneous emphysema (SCE), cardiomegaly (CMG), and pneumothorax (PNO). The DLAD functions as a prediction algorithm complemented by various application peripherals, such as web-based communication tools, DICOM file conversion capabilities, and storage and reporting libraries supporting both DICOM Structured Report and DICOM Presentation State formats.

Outcome Measures

Primary Outcome Measures

Performance test [March 2023]
The primary objective is to evaluate the performance parameters of the proposed DLAD (Carebot AI CXR) in comparison to individual radiologists. The performance test includes sensitivity and specificity, positive and negative likelihood ratio, and positive and negative predictive value. The aforementioned parameters are statistically compared using confidence intervals (CI) and p-Values. The comparison procedure consists of two steps: a global hypothesis test is conducted to determine whether there are significant differences between DLAD and radiologists. If the global hypothesis test yields a significant result, individual hypothesis tests are performed. Additionally, multiple comparison methods, such as McNemar with continuity correction for Se and Sp, Holm method for LRs, and weighted generalized score statistics for PVs, are applied to control the overall error rate. All tests are performed as two-tailed tests at the 5% significance level.

Eligibility Criteria

Criteria

Ages Eligible for Study:

18 Years and Older

Sexes Eligible for Study:

All

Inclusion Criteria:

Hospital patients > 18 years who were referred for chest radiography October 18th, 2022, and November 17th, 2022 at the Radiodiagnostic Department of the Havířov Hospital, p.o.