Using AI as a Diagnostic Decision Support Tool to Help the Diagnosis of Skin Disease in Primary Healthcare in Catalonia
Study Details
Study Description
Brief Summary
Background: Dermatological conditions are a relevant health problem. Machine learning models are increasingly being applied to dermatology as a diagnostic decision support tool using image analysis, specially for skin cancer detection and classification.
Objective: The objective of this study is to perform a prospective validation of an image analysis ML model, which is capable of screening 44 different skin disease types, comparing its diagnostic capacity with that of General Practitioners (GPs) and dermatologists.
Methods: In this prospective study 100 consecutive patients who visit a participant GP with a skin problem in central Catalonia will be recruited, data collection is planned to last 7 months. Skin diseases anonymized pictures will be taken and introduced in the ML model interface, which will return top 5 accuracy diagnosis. The same image will be also sent as a teledermatology consultation, following the current workflow. GP, ML model and dermatologist/s assessments will be compared to calculate the precision, sensitivity, specificity and accuracy of the ML model.
| Condition or Disease | Intervention/Treatment | Phase |
|---|---|---|
|
N/A |
Detailed Description
A secure anonymous stand alone web interface that is compatible to any mobile device will be integrated with the Autoderm API. The study conducted in this project will consist in a prospective study aimed to evaluate the ML model performance, comparing its diagnostic capacity with GPs and dermatologists.
To conduct the study the following procedure will be executed until the required number of samples is reached:
-
A suitable patient with skin concern is asked to participate and sign the patient's study agreement.
-
GP will diagnose the skin condition.
-
GP (or nurse) will take one good quality image of the skin condition.
-
GP will send the photograph as a teledermatology consultation following the current workflow.
-
The image is entered in the Autoderm ML interface.
-
Dermatologist will diagnose the skin condition.
The study will be conducted in primary care centers managed by the Catalan Health Institute. Participant PCP will be located in rural and metropolitan areas in Central Catalonia, which includes the regions of Anoia, Bages, Moianès, Berguedà and Osona. The reference population included in the study will be about 512,050. The recruitment of prospective subjects will consist on a consecutive basis.
General practitioners will collect data from consecutive patients who meet the inclusion criteria after obtaining written informed consent. Collected data will be reported exclusively in case report form (attached at Annex V and VI).
The GP will diagnose the skin condition and will fill the "Face-to-face assessment by GP". For each patient, the GP using a smartphone camera will take a close up good quality image of the skin problem. The image will be anonymous and it will be not possible to identify patients. The GP will use the Autoderm ML interface to upload the anonymized image and will fill the "Assessment provided by the ML model" questionnaire with the top 3 diagnoses generated by the ML model.
In order to get a second opinion, the GP will incorporate the anonymized image and an accurate description of the skin lesion into the patient's medical history following the current teledermatology flow. The GP will fill "Assessment by teledermatology" questionnaire after receiving the information, being response time about 2-7 days.
In case of dermatology referral, the GP will fill "Assessment by in person dermatologist", by accessing electronic health records as they become available, being the average waiting time for referral from 30 to 90 days.
Questionnaire case number will be the same for all questionnaires and it will not be possible to identify the patient, as case number will be predefined before the initiation of the data collection phase.
To compare the performance of the ML model with that of the GPs and dermatologists, it will be required a sample size of 100 images of skin diseases from patients who meet the inclusion criteria. The proposed sample size is based on sample size calculation used in similar research.
Study Design
Arms and Interventions
| Arm | Intervention/Treatment |
|---|---|
| Experimental: Diagnostic Test: ML model The diagnostic capacity of the ML model will be compared with that of the general practitioners and with dermatologist. |
Diagnostic Test: Autoderm® dermatology search engine (ML model) testing
GP using a smartphone camera will take an image of the skin problem and will use the Autoderm ML interface to upload the anonymized image. The obtained predicted diagnosis will be recorded in case report form.
|
Outcome Measures
Primary Outcome Measures
- Sensitivity of the ML model [1 year]
True positive rate of the ML model
- Specificity of the ML model [1 year]
True negative rate of the ML model
- Accuracy of the ML model [1 year]
Ratio of number of correct predictions to the total number of input samples
- Area under the receiver operating characteristic curve of the ML model [1 year]
Diagnostic ability of the ML model
Secondary Outcome Measures
- Rate of the eligible participants who agree to participate in the study [1 year]
Frequency of patients who agree to participate in the clinical trial and are eligible.
Eligibility Criteria
Criteria
Inclusion Criteria:
-
Patients who have cutaneous disease reason-for-visit.
-
Patients who provide written informed consent.
-
Patients who are 18 years of age or older.
Exclusion Criteria:
-
Patients with advanced dementia.
-
Patients with a cutaneous lesion which can't be photographed with a smartphone and images with poor quality.
-
Patients who have conditions associated with risk of poor protocol compliance.
Contacts and Locations
Locations
| Site | City | State | Country | Postal Code | |
|---|---|---|---|---|---|
| 1 | CAP Navàs | Navàs | Barcelona | Spain | 08670 |
Sponsors and Collaborators
- Jordi Gol i Gurina Foundation
- iDoc24
- Institut Català de la Salut
Investigators
None specified.Study Documents (Full-Text)
None provided.More Information
Publications
None provided.- P20/159-P