A Single-center, Retrospective Study to Evaluate the Clinical Performance of Artificial Intelligence Medical Assisted Diagnostic Software (VeriSee DR) for Screening of Diabetic Retinopathy in Patients With Diabetes Mellitus

• Sponsor: Acer Being Health Inc. (Industry)
• Overall Status: Completed
• CT.gov ID: NCT04160988
• Collaborator: National Taiwan University Hospital (Other)
• Enrollment: 703
• Location: 1
• Actual Duration (Months): 4.8
• Patients Per Site Per Month: 145.6

Study Details

Study Description

Brief Summary

This study is to evaluate the clinical performance of VeriSee DR for DR screening from color fundus photography images in patients with diabetes mellitus. The sensitivity and specificity of VeriSee DR's automated image analysis for screening the diabetes retinopathy will be determined.

Condition or Disease	Intervention/Treatment	Phase
Diabetic Retinopathy

Study Design

• Study Type: Observational
• Actual Enrollment: 703 participants
• Observational Model: Case-Only
• Time Perspective: Retrospective
• Official Title: A Single-center, Retrospective Study to Evaluate the Clinical Performance of Artificial Intelligence Medical Assisted Diagnostic Software (VeriSee DR) for Screening of Diabetic Retinopathy in Patients With Diabetes Mellitus
• Actual Study Start Date: Dec 16, 2019
• Actual Primary Completion Date: Jan 31, 2020
• Actual Study Completion Date: May 11, 2020

Outcome Measures

Primary Outcome Measures

1. Sensitivity [2 months]

   To evaluate the clinical performance of VeriSee DR by determining the sensitivity. Sensitivity = 100% x TP/(TP+FN)

2. Specificity [2 months]

   To evaluate the clinical performance of VeriSee DR by determining the specificity. Specificity = 100% x TN/(TN+FP)

Secondary Outcome Measures

1. Positive Predictive Values (PPV) [2 months]

   To evaluate the clinical performance of VeriSee DR by determining the positive predictive values (PPV). Positive predictive value (PPV) =100% x TP/(TP+FP)

2. Negative Predictive Values (NPV) [2 months]

   To evaluate the clinical performance of VeriSee DR by determining the negative predictive values (NPV). Negative predictive value (NPV) = 100% x TN/(FN+TN)

3. Percentage of Participant Images With Insufficient Quality as Judged by VeriSee DR [2 months]

   To determine the percentage of subjects' images with insufficient quality as judged by VeriSee DR. The percentage of subjects who have the images with insufficient quality as determined by VeriSee DR will be presented.

Eligibility Criteria

Criteria

• Ages Eligible for Study: 20 Years and Older
• Sexes Eligible for Study: All
• Accepts Healthy Volunteers: No

Inclusion Criteria:

• Subjects enrolled in this study should meet all the following criteria.

1. Subject with age ≥ 20 years old

2. Subject with documented diagnosis of diabetes mellitus

3. Subject with image taken by color fundus photography that meet the following requirement:

• The resolution of image is 1024×1024 pixels or higher;

• The angle view of image is 45 or 50 degree.

1. Subject's image includes macula and optic nerve as judged by the ophthalmologist.

Exclusion Criteria:

• Subjects will be excluded if they meet any of the following criteria.

1. The color fundus photography image previously used by VeriSee DR during the development process and pre-clinical test

2. The macula, optic nerve, or other part in the image of color fundus photography is unclear to determine the disease condition as judged by the ophthalmologist.

Contacts and Locations

Locations

	Site	City	State	Country	Postal Code
1	National Taiwan University Hospital	Taipei		Taiwan	100

Sponsors and Collaborators

• Acer Being Health Inc.
• National Taiwan University Hospital

Investigators

None specified.

Study Documents (Full-Text)

• Study Protocol - Jul 30, 2019

More Information

Publications

None provided.

• Responsible Party: Acer Being Health Inc.
• ClinicalTrials.gov Identifier: NCT04160988
• Other Study ID Numbers: QCR19002
• First Posted: Nov 13, 2019
• Last Update Posted: Jan 29, 2021
• Last Verified: Jan 1, 2021

• Studies a U.S. FDA-regulated Drug Product: No
• Studies a U.S. FDA-regulated Device Product: No
• Additional relevant MeSH terms: Retinal Diseases; Diabetic Retinopathy; Diabetes Mellitus

Study Results

Participant Flow

Arm/Group Title	Test Group
Arm/Group Description	A total of 703 subjects with at least one qualified color fundus photography image as determined by the ophthalmologist were enrolled in this study. All 703 subjects enrolled were included in the intended use population. A total of 703 qualified color fundus photography images from all enrolled subjects were included for effectiveness analysis. No subjects were excluded from the intended use population.
Period Title: Overall Study
STARTED	703
COMPLETED	675
NOT COMPLETED	28

Baseline Characteristics

Arm/Group Title	Intended Use Population
Arm/Group Description	Intended Use Population
Overall Participants	703
Age (years) [Mean (Standard Deviation) ]
Mean (Standard Deviation) [years]	58.5 (13.6)
Sex: Female, Male (Count of Participants)
Female	354 50.4%
Male	349 49.6%
Race and Ethnicity Not Collected (Count of Participants)
Region of Enrollment (Count of Participants)
Taiwan	703 100%

Outcome Measures

1. Primary Outcome

Title	Sensitivity
Description	To evaluate the clinical performance of VeriSee DR by determining the sensitivity. Sensitivity = 100% x TP/(TP+FN)
Time Frame	2 months

Outcome Measure Data

Analysis Population Description

Of 703 subjects enrolled, VeriSee DR judged the images of 28 subjects (3.98%) as with insufficient image quality.

Arm/Group Title	Intended Use Population
Arm/Group Description	Intended Use Population
Measure Participants	675
Number (95% Confidence Interval) [percent]	95

2. Primary Outcome

Title	Specificity
Description	To evaluate the clinical performance of VeriSee DR by determining the specificity. Specificity = 100% x TN/(TN+FP)
Time Frame	2 months

Outcome Measure Data

Analysis Population Description

Of 703 subjects enrolled, VeriSee DR judged the images of 28 subjects (3.98%) as with insufficient image quality.

Arm/Group Title	Test Group
Arm/Group Description	A total of 703 subjects with at least one qualified color fundus photography image as determined by the ophthalmologist were enrolled in this study. All 703 subjects enrolled were included in the intended use population. A total of 703 qualified color fundus photography images from all enrolled subjects were included for effectiveness analysis. No subjects were excluded from the intended use population.
Measure Participants	675
Number (95% Confidence Interval) [percent]	89.9

3. Secondary Outcome

Title	Positive Predictive Values (PPV)
Description	To evaluate the clinical performance of VeriSee DR by determining the positive predictive values (PPV). Positive predictive value (PPV) =100% x TP/(TP+FP)
Time Frame	2 months

Outcome Measure Data

Analysis Population Description

Of 703 subjects enrolled, VeriSee DR judged the images of 28 subjects (3.98%) as with insufficient image quality.

Arm/Group Title	Test Group
Arm/Group Description	A total of 703 subjects with at least one qualified color fundus photography image as determined by the ophthalmologist were enrolled in this study. All 703 subjects enrolled were included in the intended use population. A total of 703 qualified color fundus photography images from all enrolled subjects were included for effectiveness analysis. No subjects were excluded from the intended use population.
Measure Participants	675
Number (95% Confidence Interval) [percent]	94.5

4. Secondary Outcome

Title	Negative Predictive Values (NPV)
Description	To evaluate the clinical performance of VeriSee DR by determining the negative predictive values (NPV). Negative predictive value (NPV) = 100% x TN/(FN+TN)
Time Frame	2 months

Outcome Measure Data

Analysis Population Description

Of 703 subjects enrolled, VeriSee DR judged the images of 28 subjects (3.98%) as with insufficient image quality.

Arm/Group Title	Intended Use Population
Arm/Group Description	Intended Use Population
Measure Participants	675
Number (95% Confidence Interval) [percent]	90.7

5. Secondary Outcome

Title	Percentage of Participant Images With Insufficient Quality as Judged by VeriSee DR
Description	To determine the percentage of subjects' images with insufficient quality as judged by VeriSee DR. The percentage of subjects who have the images with insufficient quality as determined by VeriSee DR will be presented.
Time Frame	2 months

Outcome Measure Data

Analysis Population Description

[Not Specified]

Arm/Group Title	Intended Use Population
Arm/Group Description	Intended Use Population
Measure Participants	703
Count of Participants [Participants]	28 4%

Adverse Events

Time Frame	this study is a retrospective study so there is no need to collect adverse event data.
Adverse Event Reporting Description	this study is a retrospective study so there is no need to collect adverse event data.
Arm/Group Title	Test Group
Arm/Group Description	A total of 703 subjects with at least one qualified color fundus photography image as determined by the ophthalmologist were enrolled in this study. All 703 subjects enrolled were included in the intended use population. A total of 703 qualified color fundus photography images from all enrolled subjects were included for effectiveness analysis. No subjects were excluded from the intended use population.
All Cause Mortality
	Test Group
	Affected / at Risk (%)	# Events
Total	0/0 (NaN)
Serious Adverse Events
	Test Group
	Affected / at Risk (%)	# Events
Total	0/0 (NaN)
Other (Not Including Serious) Adverse Events
	Test Group
	Affected / at Risk (%)	# Events
Total	0/0 (NaN)

Limitations/Caveats

[Not Specified]

More Information

Certain Agreements

Principal Investigators are NOT employed by the organization sponsoring the study.

There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.