AIDMel: AI-Augmented Skin Cancer Diagnosis in Teledermatoscopy

Sponsor

Karolinska University Hospital (Other)

Overall Status

Enrolling by invitation

CT.gov ID

NCT06080711

Collaborator

Karolinska Institutet (Other), Medical University of Vienna (Other), Stockholm School of Economics (Other)

Enrollment

Location

Arms

20.5

Anticipated Duration (Months)

1.5

Patients Per Site Per Month

Study Details

Study Description

Brief Summary

In this study an artificial intelligence (AI) tool for skin cancer diagnosis is implemented in a teleldermatoscopy platform. The aim is to study the effects on clinician diagnostic accuracy, management decisions, and confidence. Furthermore, this prospective randomized study investigates the role of human factors in determining clinician reliance on AI tools and the consequent accuracy in a real-world setting.

Condition or Disease	Intervention/Treatment	Phase
Skin Cancer Melanoma	Other: AI assistance	N/A

Detailed Description

Deep-learning algorithms can potentially benefit many areas in healthcare, including the diagnosis of skin cancer using teledermatoscopy. However, there is a dearth of clinical, prospective research on human-AI interaction in diagnostic tasks that take human factors into account.

In this study we will examine the impact of such factors in a real-world setting where we integrate an algorithm in an existing teledermatoscopy platform that is used clinically at a tertiary hospital in Sweden. We will investigate what impact various implementations of AI tool output in relation to human factors have on diagnostic accuracy and management decisions.

Study subjects are recruited at the Department of Dermatology at Karolinska University Hospital and will be asked to rate prospective teledermatoscopic consults with and without AI-support. Each consult will be randomized into one of three workflows with or without one pre-defined implementation of the AI tool. Study subjects are also asked to complete two surveys with demographic information and questions relating to various human factors. Patients participating in the study will be diagnosed outside the study prior to inclusion without any involvement of an AI tool, notably by two experienced dermatologists who do not participate as study subjects.

Study Design

Study Type:

Interventional

Anticipated Enrollment :

30 participants

Allocation:

Randomized

Intervention Model:

Crossover Assignment

Masking:

None (Open Label)

Primary Purpose:

Diagnostic

Official Title:

AI-Augmented Skin Cancer Diagnosis in Teledermatoscopy: A Prospective Randomized Study

Actual Study Start Date :

Feb 15, 2023

Anticipated Primary Completion Date :

Jun 30, 2024

Anticipated Study Completion Date :

Oct 30, 2024

Arms and Interventions

Arm	Intervention/Treatment
No Intervention: Workflow 1 Standard of care
Experimental: Workflow 2 Consult with AI assistance	Other: AI assistance Participants will be informed of the diagnostic probabilities for each of ten differential diagnoses according to the AI tool
Experimental: Workflow 3 First workflow 1, then workflow 2	Other: AI assistance Participants will be informed of the diagnostic probabilities for each of ten differential diagnoses according to the AI tool

Arm

Intervention/Treatment

No Intervention: Workflow 1

Standard of care

Experimental: Workflow 2

Consult with AI assistance

Other: AI assistance

Participants will be informed of the diagnostic probabilities for each of ten differential diagnoses according to the AI tool

Experimental: Workflow 3

First workflow 1, then workflow 2

Other: AI assistance

Participants will be informed of the diagnostic probabilities for each of ten differential diagnoses according to the AI tool

Outcome Measures

Primary Outcome Measures

Diagnostic accuracy [1 year]
Determine sensitivity, specificity, accuracy and AUROC in terms of diagnostic accuracy for dermatologists with vs without AI advice. Further, to investigate the role of the different workflows (diagnosis with or without AI with varying sequencing) and the influence of demographics and human factors (e.g. level of experience) on diagnostic accuracy
Accuracy of management decisions [1 year]
Determine sensitivity, specificity, accuracy and AUROC in terms of accuracy for management decisions for dermatologists with vs without AI and investigate the role of the different workflows (with or without AI with varying sequencing) and the influence of demographics and human factors (e.g. level of experience) on management decisions (biopsy/surgery, no intervention, or follow-up)
Tendency to change initial diagnosis or management decision [1 year]
Evaluate which factors affect the likelihood of a physician changing their evaluation after receiving algorithmic input
Self-reported confidence in diagnosis and management decisions [1 year]
Investigate whether AI or other factors affect the physician's confidence in their diagnosis and management decisions

Eligibility Criteria

Criteria

Ages Eligible for Study:

18 Years and Older

Sexes Eligible for Study:

All

Accepts Healthy Volunteers:

Yes

Inclusion Criteria:

Licensed physician
Working at a dermatology clinic
Sufficient knowledge in Swedish
Written consent to participate

Exclusion Criteria:

No experience of using dermatoscopy
Does not wish to participate
Incomplete answers
Physicians that are involved in the patients' clinical care relating to the teledermoscopical consult