Transferability and Cost-effectiveness of PROSPECT in Surgical Trainees

Sponsor
University Ghent (Other)
Overall Status
Completed
CT.gov ID
NCT01965860
Collaborator
(none)
39
Enrollment
1
Location
3
Arms
18
Actual Duration (Months)
2.2
Patients Per Site Per Month

Study Details

Study Description

Brief Summary

To design, validate and prove cost-effectiveness of a comprehensive endovascular simulation curriculum consisting of cognitive and psychomotor skills training for endovascular management of symptomatic vascular disease in the lower limbs.

Condition or DiseaseIntervention/TreatmentPhase
  • Other: additional curriculum (E-learning + simulation)
  • Other: additional E-learning
N/A

Detailed Description

The main goal is to design a comprehensive endovascular simulation curriculum consisting of cognitive and psychomotor skills training for endovascular management of symptomatic vascular disease in the lower limbs (Rutherford classification 2-5; stenosis or occlusion in the iliac, superficial femoral and popliteal arteries). The curriculum will consist of E-learning, video-based learning and simulation exercises on the Virtual Reality simulator on which an endovascular procedure will be simulated.

Skills transferability to real life practice will be verified by means of a RCT. In this RCT the investigators will compare the cognitive knowledge and technical performance of curricular trained surgical trainees with conventionally trained trainees during treatment of patients under supervision. The hypothesis states that surgeons trained within the PROSPECT curriculum will show improved technical knowledge of endovascular treatment of atherosclerotic disease in the iliac, superficial femoral and popliteal arteries and will demonstrate increased technical proficiency in the angiosuite in comparison with surgeons who received only conventional training. Subjects will be surgical trainees (N=32). They will be randomized into a three groups: a control group (N=11), a group that only has access to E-learning (N=10) and a PROSPECT group (N=11). Both groups will continue their traditional clinical education and the PROSPECT group will additionally be trained within this endovascular curriculum. The investigators hope that this study may ultimately lead to an improvement in the quality of patient care by standardizing competencies of endovascular operators in training and practice.

Additionally a cost-effectiveness analysis of PROSPECT compared to traditional training modalities will be executed. A successful training program is expected to lead to a shortening of the learning curve, reduction in number of errors during real life procedures and more efficient use of hybrid angio suites. The financial impact of potentially shorter duration of endovascular interventions will be studied and compared to time investment and additional costs associated with stepwise, supervised training using VR simulators.

Study Design

Study Type:
Interventional
Actual Enrollment :
39 participants
Allocation:
Randomized
Intervention Model:
Parallel Assignment
Intervention Model Description:
Initially 39 participants were enrolled, 7 excluded because they declined to participate. 32 people were randomized.Initially 39 participants were enrolled, 7 excluded because they declined to participate. 32 people were randomized.
Masking:
Single (Outcomes Assessor)
Primary Purpose:
Health Services Research
Official Title:
The Study of Transferability and Cost-effectiveness of a Proficiency-based Stepwise Endovascular Curricular Training Program (PROSPECT) in Surgical Trainees
Actual Study Start Date :
Aug 1, 2014
Actual Primary Completion Date :
Feb 1, 2016
Actual Study Completion Date :
Feb 1, 2016

Arms and Interventions

ArmIntervention/Treatment
Experimental: PROSPECT

the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module). After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator.

Other: additional curriculum (E-learning + simulation)

No Intervention: CONTROL

The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.

Experimental: E-LEARNING

the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module). After studying each module the trainee will complete a Multiple Choice Questionnaire, no simulation.

Other: additional E-learning

Outcome Measures

Primary Outcome Measures

  1. Technical and Global Performance of the Surgical Trainee in Real Life Procedures [6 weeks after intervention]

    The consultant will assess the trainee during the intervention. Global Rating Scale of Endovascular Performance (Range 11-55): to rate the global performance - modified OSATS (Objective Structured Assessment of Technical Skills) scale Examiner Checklist for Diagnostic Angiography, Angioplasty and Stenting (Range 17-85): to rate the technical performance For both scales, the higher the score, the better. Surrogate measures of performance are also evaluated (total procedure and fluoroscopy time, radiation dose, contrast volume and number of endovascular tools used). Post-hoc videos of hand movements and the fluoroscopy imaging recorded during the real procedure will be rated using the same scoring systems. Two real life endovascular procedures will be performed within six weeks after completing the curriculum. Evaluation will occur during the intervention and videos of hand movements and fluoroscopy imaging will be evaluated afterwards.

Secondary Outcome Measures

  1. Improved Knowledge and Technical Performance of the Surgical Trainee on a Simulated Complex Module [Pre curriculum, 6 weeks and 3 months after completion of the curriculum (3 months only for intervention arms)]

    The cognitive and technical skills retention of the surgical trainee is measured by a MCQ test and a simulation based exercise, pre and post PROSPECT. MCQ test: validated Multiple Choice Test - Range 0-20 GRS: Global Rating Scale of Endovascular Performance, modified OSATS (Objective Structured Assessment of Technical Skills) scale: to assess global performance (same as primary outcome) - Range: 11-55 Examiners checklist: Examiner Checklist for Diagnostic Angiography, Angioplasty and Stenting: to assess technical performance (same as primary objective) - Range: 17-85 For al outcome measures: the higher the score, the better

Other Outcome Measures

  1. PROSPECT Proficiency [After RCT]

    After the RCT, a registry was started with all participants that started PROSPECT. Outcome: do PROSPECT participants pass the curriculum, do they become proficient?

Eligibility Criteria

Criteria

Ages Eligible for Study:
18 Years to 65 Years
Sexes Eligible for Study:
All
Accepts Healthy Volunteers:
Yes
Inclusion Criteria:
  • Surgical trainee

  • Knowledge and technical skill level evaluated

  • Demographics questionnaire and general MCQ test completed

  • 2 endovascular simulated exercises completed after familiarization with the simulator

Exclusion Criteria:

Contacts and Locations

Locations

SiteCityStateCountryPostal Code
1Ghent University HospitalGhentBelgium9000

Sponsors and Collaborators

  • University Ghent

Investigators

  • Principal Investigator: Isabelle Van Herzeele, MD, PhD, University Ghent

Study Documents (Full-Text)

None provided.

More Information

Publications

Responsible Party:
University Ghent
ClinicalTrials.gov Identifier:
NCT01965860
Other Study ID Numbers:
  • 2013/783
First Posted:
Oct 18, 2013
Last Update Posted:
Mar 16, 2021
Last Verified:
Jan 1, 2021
Additional relevant MeSH terms:

Study Results

Participant Flow

Recruitment DetailsRecruitment for RCT (before 02/2016): general surgery trainees from Ghent University Recruitment for registry (after 02/2016): general surgery trainees from Ghent University, Leuven University, Nancy University, Lille University, CAMES Copenhagen
Pre-assignment Detail7 people declined to participate before randomisation during the RCT
Arm/Group TitlePROSPECTE-LEARNINGCONTROL
Arm/Group Descriptionthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculumthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
Period Title: Overall Study
STARTED111011
COMPLETED91010
NOT COMPLETED201

Baseline Characteristics

Arm/Group TitlePROSPECTE-LEARNINGCONTROLTotal
Arm/Group Descriptionthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculumthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.Total of all reporting groups
Overall Participants11101132
Age (Count of Participants)
<=18 years
0
0%
0
0%
0
0%
0
0%
Between 18 and 65 years
11
100%
10
100%
11
100%
32
100%
>=65 years
0
0%
0
0%
0
0%
0
0%
Sex: Female, Male (Count of Participants)
Female
5
45.5%
6
60%
5
45.5%
16
50%
Male
6
54.5%
4
40%
6
54.5%
16
50%
Postgraduate year (Count of Participants)
Level 1
2
18.2%
3
30%
4
36.4%
9
28.1%
Level 2
2
18.2%
3
30%
1
9.1%
6
18.8%
Level 3
4
36.4%
1
10%
3
27.3%
8
25%
Level 4
0
0%
2
20%
1
9.1%
3
9.4%
Level 5
1
9.1%
1
10%
1
9.1%
3
9.4%
Level 6
2
18.2%
0
0%
1
9.1%
3
9.4%
No. endovascular cases assisted (Count of Participants)
0-5
1
9.1%
2
20%
2
18.2%
5
15.6%
5-10
2
18.2%
1
10%
1
9.1%
4
12.5%
10-50
6
54.5%
5
50%
1
9.1%
12
37.5%
50-100
1
9.1%
1
10%
5
45.5%
7
21.9%
100-200
1
9.1%
0
0%
0
0%
1
3.1%
> 200
0
0%
1
10%
2
18.2%
3
9.4%

Outcome Measures

1. Primary Outcome
TitleTechnical and Global Performance of the Surgical Trainee in Real Life Procedures
DescriptionThe consultant will assess the trainee during the intervention. Global Rating Scale of Endovascular Performance (Range 11-55): to rate the global performance - modified OSATS (Objective Structured Assessment of Technical Skills) scale Examiner Checklist for Diagnostic Angiography, Angioplasty and Stenting (Range 17-85): to rate the technical performance For both scales, the higher the score, the better. Surrogate measures of performance are also evaluated (total procedure and fluoroscopy time, radiation dose, contrast volume and number of endovascular tools used). Post-hoc videos of hand movements and the fluoroscopy imaging recorded during the real procedure will be rated using the same scoring systems. Two real life endovascular procedures will be performed within six weeks after completing the curriculum. Evaluation will occur during the intervention and videos of hand movements and fluoroscopy imaging will be evaluated afterwards.
Time Frame6 weeks after intervention

Outcome Measure Data

Analysis Population Description
[Not Specified]
Arm/Group TitlePROSPECTE-LEARNINGCONTROL
Arm/Group Descriptionthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculumthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
Measure Participants91010
GRS (real life simulation)
39.36
(2.05)
28.42
(2.15)
23.09
(2.18)
Checklist (real life simulation)
63.51
(3.18)
53.63
(3.34)
38.72
(3.38)
2. Secondary Outcome
TitleImproved Knowledge and Technical Performance of the Surgical Trainee on a Simulated Complex Module
DescriptionThe cognitive and technical skills retention of the surgical trainee is measured by a MCQ test and a simulation based exercise, pre and post PROSPECT. MCQ test: validated Multiple Choice Test - Range 0-20 GRS: Global Rating Scale of Endovascular Performance, modified OSATS (Objective Structured Assessment of Technical Skills) scale: to assess global performance (same as primary outcome) - Range: 11-55 Examiners checklist: Examiner Checklist for Diagnostic Angiography, Angioplasty and Stenting: to assess technical performance (same as primary objective) - Range: 17-85 For al outcome measures: the higher the score, the better
Time FramePre curriculum, 6 weeks and 3 months after completion of the curriculum (3 months only for intervention arms)

Outcome Measure Data

Analysis Population Description
[Not Specified]
Arm/Group TitlePROSPECTE-LEARNINGCONTROL
Arm/Group Descriptionthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculumthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
Measure Participants91010
Pre-test MCQ
14
(3.39)
14.88
(1.13)
14
(2.49)
Post test MCQ 6w
18.11
(1.36)
17.88
(0.99)
15.2
(2.7)
Post test MCQ 3m
18.25
(1.98)
17.71
(0.75)
NA
(NA)
Pre GRS
19.78
(7.24)
16.25
(4.1)
18
(7.75)
Post GRS 6w
49.56
(3.78)
32.75
(13.51)
21.2
(7.48)
Post GRS 3m
51.13
(3.44)
24.14
(6.65)
NA
(NA)
Pre Examiner Checklist
43.56
(15.76)
35.63
(11.38)
37.6
(13.98)
Post Examiner Checklist 6w
83.22
(1.99)
59.25
(16.73)
42.8
(15.39)
Post Examiner Checklist 3m
82.25
(1.75)
51.71
(16.31)
NA
(NA)
Statistical Analysis 1
Statistical Analysis Overview Comparison Group Selection PROSPECT, E-LEARNING, CONTROL
Comments Prior to the study a power analysis was performed to calculate the number of participants required in each of the three groups. Previous work comparing OSATS derived scores in endovascular interventions shows a Cohen D of two. Using a of .05, a power of .80, and an expected dropout rate of 10%, the minimum number of surgical trainees required per group was seven. Null hypothesis for primary outcome: no difference in technical performance during real life procedures between the three groups
Type of Statistical Test Superiority
Comments
Statistical Test of Hypothesisp-Value0.001
Comments
MethodANOVA
Comments
Statistical Analysis 2
Statistical Analysis Overview Comparison Group Selection PROSPECT
Comments Null Hypothesis: There is no difference in Pre and Post test MCQ score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.003
Comments
MethodANOVA
Comments
Statistical Analysis 3
Statistical Analysis Overview Comparison Group Selection E-LEARNING
Comments Null Hypothesis: There is no difference in Pre and Post test MCQ score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.001
Comments
MethodANOVA
Comments
Statistical Analysis 4
Statistical Analysis Overview Comparison Group Selection CONTROL
Comments Null Hypothesis: There is no difference in Pre and Post test MCQ score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.228
Comments
MethodANOVA
Comments
Statistical Analysis 5
Statistical Analysis Overview Comparison Group Selection PROSPECT
Comments Null Hypothesis: There is no difference in Pre and Post test GRS score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.001
Comments
MethodANOVA
Comments
Statistical Analysis 6
Statistical Analysis Overview Comparison Group Selection E-LEARNING
Comments Null Hypothesis: There is no difference in Pre and Post test GRS score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.008
Comments
MethodANOVA
Comments
Statistical Analysis 7
Statistical Analysis Overview Comparison Group Selection CONTROL
Comments Null Hypothesis: There is no difference in Pre and Post test GRS score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.164
Comments
MethodANOVA
Comments
Statistical Analysis 8
Statistical Analysis Overview Comparison Group Selection PROSPECT
Comments Null Hypothesis: There is no difference in Pre and Post test Examiner's checklist score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.001
Comments
MethodANOVA
Comments
Statistical Analysis 9
Statistical Analysis Overview Comparison Group Selection E-LEARNING
Comments Null Hypothesis: There is no difference in Pre and Post test Examiner's checklist score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.001
Comments
MethodANOVA
Comments
Statistical Analysis 10
Statistical Analysis Overview Comparison Group Selection CONTROL
Comments Null Hypothesis: There is no difference in Pre and Post test Examiner's checklist score
Type of Statistical Test Other
Comments
Statistical Test of Hypothesisp-Value0.190
Comments
MethodANOVA
Comments
3. Other Pre-specified Outcome
TitlePROSPECT Proficiency
DescriptionAfter the RCT, a registry was started with all participants that started PROSPECT. Outcome: do PROSPECT participants pass the curriculum, do they become proficient?
Time FrameAfter RCT

Outcome Measure Data

Analysis Population Description
[Not Specified]
Arm/Group Title
Arm/Group Description

Adverse Events

Time Frame
Adverse Event Reporting Description Participants were not at risk of adverse events as they were not patients but healthy trainees. There was monitoring/assesment for all-cause mortality, serious and other (not including serious) adverse events, but none occured.
Arm/Group TitlePROSPECTE-LEARNINGCONTROL
Arm/Group Descriptionthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculumthe trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
All Cause Mortality
PROSPECTE-LEARNINGCONTROL
Affected / at Risk (%)# EventsAffected / at Risk (%)# EventsAffected / at Risk (%)# Events
Total0/11 (0%) 0/10 (0%) 0/11 (0%)
Serious Adverse Events
PROSPECTE-LEARNINGCONTROL
Affected / at Risk (%)# EventsAffected / at Risk (%)# EventsAffected / at Risk (%)# Events
Total0/11 (0%) 0/10 (0%) 0/11 (0%)
Other (Not Including Serious) Adverse Events
PROSPECTE-LEARNINGCONTROL
Affected / at Risk (%)# EventsAffected / at Risk (%)# EventsAffected / at Risk (%)# Events
Total0/11 (0%) 0/10 (0%) 0/11 (0%)

Limitations/Caveats

Single centre, results may not be applicable to other centres/countries. Randomisation status was not blinded for participants, consultant can unintentionally know randomisation group. Long-term skills retention was not assessed beyond 3 months post-training. As the procedures were closely supervised, operative metrics cannot be used to assess the operative performance.

More Information

Certain Agreements

All Principal Investigators ARE employed by the organization sponsoring the study.

There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.

Results Point of Contact

Name/TitleProf. Dr. Isabelle Van Herzeele
OrganizationGhent University
Phone93325108 ext +32
Emailisabelle.vanherzeele@ugent.be
Responsible Party:
University Ghent
ClinicalTrials.gov Identifier:
NCT01965860
Other Study ID Numbers:
  • 2013/783
First Posted:
Oct 18, 2013
Last Update Posted:
Mar 16, 2021
Last Verified:
Jan 1, 2021