Transferability and Cost-effectiveness of PROSPECT in Surgical Trainees

• Sponsor: University Ghent (Other)
• Overall Status: Completed
• CT.gov ID: NCT01965860
• Collaborator: (none)

Study Details

Study Description

Brief Summary

To design, validate and prove cost-effectiveness of a comprehensive endovascular simulation curriculum consisting of cognitive and psychomotor skills training for endovascular management of symptomatic vascular disease in the lower limbs.

Condition or Disease	Intervention/Treatment	Phase
Vascular Disease	Other: additional curriculum (E-learning + simulation); Other: additional E-learning	N/A

Detailed Description

The main goal is to design a comprehensive endovascular simulation curriculum consisting of cognitive and psychomotor skills training for endovascular management of symptomatic vascular disease in the lower limbs (Rutherford classification 2-5; stenosis or occlusion in the iliac, superficial femoral and popliteal arteries). The curriculum will consist of E-learning, video-based learning and simulation exercises on the Virtual Reality simulator on which an endovascular procedure will be simulated.

Skills transferability to real life practice will be verified by means of a RCT. In this RCT the investigators will compare the cognitive knowledge and technical performance of curricular trained surgical trainees with conventionally trained trainees during treatment of patients under supervision. The hypothesis states that surgeons trained within the PROSPECT curriculum will show improved technical knowledge of endovascular treatment of atherosclerotic disease in the iliac, superficial femoral and popliteal arteries and will demonstrate increased technical proficiency in the angiosuite in comparison with surgeons who received only conventional training. Subjects will be surgical trainees (N=32). They will be randomized into a three groups: a control group (N=11), a group that only has access to E-learning (N=10) and a PROSPECT group (N=11). Both groups will continue their traditional clinical education and the PROSPECT group will additionally be trained within this endovascular curriculum. The investigators hope that this study may ultimately lead to an improvement in the quality of patient care by standardizing competencies of endovascular operators in training and practice.

Additionally a cost-effectiveness analysis of PROSPECT compared to traditional training modalities will be executed. A successful training program is expected to lead to a shortening of the learning curve, reduction in number of errors during real life procedures and more efficient use of hybrid angio suites. The financial impact of potentially shorter duration of endovascular interventions will be studied and compared to time investment and additional costs associated with stepwise, supervised training using VR simulators.

Study Design

• Study Type: Interventional
• Actual Enrollment: 39 participants
• Allocation: Randomized
• Intervention Model: Parallel Assignment
• Intervention Model Description: Initially 39 participants were enrolled, 7 excluded because they declined to participate. 32 people were randomized.Initially 39 participants were enrolled, 7 excluded because they declined to participate. 32 people were randomized.
• Masking: Single (Outcomes Assessor)
• Primary Purpose: Health Services Research
• Official Title: The Study of Transferability and Cost-effectiveness of a Proficiency-based Stepwise Endovascular Curricular Training Program (PROSPECT) in Surgical Trainees
• Actual Study Start Date: Aug 1, 2014
• Actual Primary Completion Date: Feb 1, 2016
• Actual Study Completion Date: Feb 1, 2016

Arms and Interventions

Arm	Intervention/Treatment
Experimental: PROSPECT the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module). After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator.	Other: additional curriculum (E-learning + simulation)
No Intervention: CONTROL The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
Experimental: E-LEARNING the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module). After studying each module the trainee will complete a Multiple Choice Questionnaire, no simulation.	Other: additional E-learning

Outcome Measures

Primary Outcome Measures

1. Technical and Global Performance of the Surgical Trainee in Real Life Procedures [6 weeks after intervention]

   The consultant will assess the trainee during the intervention. Global Rating Scale of Endovascular Performance (Range 11-55): to rate the global performance - modified OSATS (Objective Structured Assessment of Technical Skills) scale Examiner Checklist for Diagnostic Angiography, Angioplasty and Stenting (Range 17-85): to rate the technical performance For both scales, the higher the score, the better. Surrogate measures of performance are also evaluated (total procedure and fluoroscopy time, radiation dose, contrast volume and number of endovascular tools used). Post-hoc videos of hand movements and the fluoroscopy imaging recorded during the real procedure will be rated using the same scoring systems. Two real life endovascular procedures will be performed within six weeks after completing the curriculum. Evaluation will occur during the intervention and videos of hand movements and fluoroscopy imaging will be evaluated afterwards.

Secondary Outcome Measures

1. Improved Knowledge and Technical Performance of the Surgical Trainee on a Simulated Complex Module [Pre curriculum, 6 weeks and 3 months after completion of the curriculum (3 months only for intervention arms)]

   The cognitive and technical skills retention of the surgical trainee is measured by a MCQ test and a simulation based exercise, pre and post PROSPECT. MCQ test: validated Multiple Choice Test - Range 0-20 GRS: Global Rating Scale of Endovascular Performance, modified OSATS (Objective Structured Assessment of Technical Skills) scale: to assess global performance (same as primary outcome) - Range: 11-55 Examiners checklist: Examiner Checklist for Diagnostic Angiography, Angioplasty and Stenting: to assess technical performance (same as primary objective) - Range: 17-85 For al outcome measures: the higher the score, the better

Other Outcome Measures

1. PROSPECT Proficiency [After RCT]

   After the RCT, a registry was started with all participants that started PROSPECT. Outcome: do PROSPECT participants pass the curriculum, do they become proficient?

Eligibility Criteria

Criteria

• Ages Eligible for Study: 18 Years to 65 Years
• Sexes Eligible for Study: All
• Accepts Healthy Volunteers: Yes

Inclusion Criteria:

• Surgical trainee

• Knowledge and technical skill level evaluated

• Demographics questionnaire and general MCQ test completed

• 2 endovascular simulated exercises completed after familiarization with the simulator

Exclusion Criteria:

Contacts and Locations

Locations

	Site	City	State	Country	Postal Code
1	Ghent University Hospital	Ghent		Belgium	9000

Sponsors and Collaborators

• University Ghent

Investigators

• Principal Investigator: Isabelle Van Herzeele, MD, PhD, University Ghent

Study Documents (Full-Text)

None provided.

More Information

Publications

• Maertens H, Aggarwal R, Desender L, Vermassen F, Van Herzeele I. Development of a PROficiency-Based StePwise Endovascular Curricular Training (PROSPECT) Program. J Surg Educ. 2016 Jan-Feb;73(1):51-60. doi: 10.1016/j.jsurg.2015.07.009. Epub 2015 Aug 11.

• Responsible Party: University Ghent
• ClinicalTrials.gov Identifier: NCT01965860
• Other Study ID Numbers: 2013/783
• First Posted: Oct 18, 2013
• Last Update Posted: Mar 16, 2021
• Last Verified: Jan 1, 2021

• Additional relevant MeSH terms: Vascular Diseases

Study Results

Participant Flow

Recruitment Details	Recruitment for RCT (before 02/2016): general surgery trainees from Ghent University Recruitment for registry (after 02/2016): general surgery trainees from Ghent University, Leuven University, Nancy University, Lille University, CAMES Copenhagen
Pre-assignment Detail	7 people declined to participate before randomisation during the RCT

Arm/Group Title	PROSPECT	E-LEARNING	CONTROL
Arm/Group Description	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculum	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.	The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
Period Title: Overall Study
STARTED	11	10	11
COMPLETED	9	10	10
NOT COMPLETED	2	0	1

Baseline Characteristics

Arm/Group Title	PROSPECT	E-LEARNING	CONTROL	Total
Arm/Group Description	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculum	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.	The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.	Total of all reporting groups
Overall Participants	11	10	11	32
Age (Count of Participants)
<=18 years	0 0%	0 0%	0 0%	0 0%
Between 18 and 65 years	11 100%	10 100%	11 100%	32 100%
>=65 years	0 0%	0 0%	0 0%	0 0%
Sex: Female, Male (Count of Participants)
Female	5 45.5%	6 60%	5 45.5%	16 50%
Male	6 54.5%	4 40%	6 54.5%	16 50%
Postgraduate year (Count of Participants)
Level 1	2 18.2%	3 30%	4 36.4%	9 28.1%
Level 2	2 18.2%	3 30%	1 9.1%	6 18.8%
Level 3	4 36.4%	1 10%	3 27.3%	8 25%
Level 4	0 0%	2 20%	1 9.1%	3 9.4%
Level 5	1 9.1%	1 10%	1 9.1%	3 9.4%
Level 6	2 18.2%	0 0%	1 9.1%	3 9.4%
No. endovascular cases assisted (Count of Participants)
0-5	1 9.1%	2 20%	2 18.2%	5 15.6%
5-10	2 18.2%	1 10%	1 9.1%	4 12.5%
10-50	6 54.5%	5 50%	1 9.1%	12 37.5%
50-100	1 9.1%	1 10%	5 45.5%	7 21.9%
100-200	1 9.1%	0 0%	0 0%	1 3.1%
> 200	0 0%	1 10%	2 18.2%	3 9.4%

Outcome Measures

1. Primary Outcome

Title	Technical and Global Performance of the Surgical Trainee in Real Life Procedures
Description	The consultant will assess the trainee during the intervention. Global Rating Scale of Endovascular Performance (Range 11-55): to rate the global performance - modified OSATS (Objective Structured Assessment of Technical Skills) scale Examiner Checklist for Diagnostic Angiography, Angioplasty and Stenting (Range 17-85): to rate the technical performance For both scales, the higher the score, the better. Surrogate measures of performance are also evaluated (total procedure and fluoroscopy time, radiation dose, contrast volume and number of endovascular tools used). Post-hoc videos of hand movements and the fluoroscopy imaging recorded during the real procedure will be rated using the same scoring systems. Two real life endovascular procedures will be performed within six weeks after completing the curriculum. Evaluation will occur during the intervention and videos of hand movements and fluoroscopy imaging will be evaluated afterwards.
Time Frame	6 weeks after intervention

Outcome Measure Data

Analysis Population Description

[Not Specified]

Arm/Group Title	PROSPECT	E-LEARNING	CONTROL
Arm/Group Description	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculum	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.	The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
Measure Participants	9	10	10
GRS (real life simulation)	39.36 (2.05)	28.42 (2.15)	23.09 (2.18)
Checklist (real life simulation)	63.51 (3.18)	53.63 (3.34)	38.72 (3.38)

2. Secondary Outcome

Title	Improved Knowledge and Technical Performance of the Surgical Trainee on a Simulated Complex Module
Description	The cognitive and technical skills retention of the surgical trainee is measured by a MCQ test and a simulation based exercise, pre and post PROSPECT. MCQ test: validated Multiple Choice Test - Range 0-20 GRS: Global Rating Scale of Endovascular Performance, modified OSATS (Objective Structured Assessment of Technical Skills) scale: to assess global performance (same as primary outcome) - Range: 11-55 Examiners checklist: Examiner Checklist for Diagnostic Angiography, Angioplasty and Stenting: to assess technical performance (same as primary objective) - Range: 17-85 For al outcome measures: the higher the score, the better
Time Frame	Pre curriculum, 6 weeks and 3 months after completion of the curriculum (3 months only for intervention arms)

Outcome Measure Data

Analysis Population Description

[Not Specified]

Arm/Group Title	PROSPECT	E-LEARNING	CONTROL
Arm/Group Description	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculum	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.	The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
Measure Participants	9	10	10
Pre-test MCQ	14 (3.39)	14.88 (1.13)	14 (2.49)
Post test MCQ 6w	18.11 (1.36)	17.88 (0.99)	15.2 (2.7)
Post test MCQ 3m	18.25 (1.98)	17.71 (0.75)	NA (NA)
Pre GRS	19.78 (7.24)	16.25 (4.1)	18 (7.75)
Post GRS 6w	49.56 (3.78)	32.75 (13.51)	21.2 (7.48)
Post GRS 3m	51.13 (3.44)	24.14 (6.65)	NA (NA)
Pre Examiner Checklist	43.56 (15.76)	35.63 (11.38)	37.6 (13.98)
Post Examiner Checklist 6w	83.22 (1.99)	59.25 (16.73)	42.8 (15.39)
Post Examiner Checklist 3m	82.25 (1.75)	51.71 (16.31)	NA (NA)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	PROSPECT, E-LEARNING, CONTROL
	Comments	Prior to the study a power analysis was performed to calculate the number of participants required in each of the three groups. Previous work comparing OSATS derived scores in endovascular interventions shows a Cohen D of two. Using a of .05, a power of .80, and an expected dropout rate of 10%, the minimum number of surgical trainees required per group was seven. Null hypothesis for primary outcome: no difference in technical performance during real life procedures between the three groups
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.001
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	PROSPECT
	Comments	Null Hypothesis: There is no difference in Pre and Post test MCQ score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.003
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	E-LEARNING
	Comments	Null Hypothesis: There is no difference in Pre and Post test MCQ score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.001
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 4

Statistical Analysis Overview	Comparison Group Selection	CONTROL
	Comments	Null Hypothesis: There is no difference in Pre and Post test MCQ score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.228
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 5

Statistical Analysis Overview	Comparison Group Selection	PROSPECT
	Comments	Null Hypothesis: There is no difference in Pre and Post test GRS score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.001
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 6

Statistical Analysis Overview	Comparison Group Selection	E-LEARNING
	Comments	Null Hypothesis: There is no difference in Pre and Post test GRS score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.008
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 7

Statistical Analysis Overview	Comparison Group Selection	CONTROL
	Comments	Null Hypothesis: There is no difference in Pre and Post test GRS score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.164
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 8

Statistical Analysis Overview	Comparison Group Selection	PROSPECT
	Comments	Null Hypothesis: There is no difference in Pre and Post test Examiner's checklist score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.001
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 9

Statistical Analysis Overview	Comparison Group Selection	E-LEARNING
	Comments	Null Hypothesis: There is no difference in Pre and Post test Examiner's checklist score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.001
	Comments
	Method	ANOVA
	Comments

Statistical Analysis 10

Statistical Analysis Overview	Comparison Group Selection	CONTROL
	Comments	Null Hypothesis: There is no difference in Pre and Post test Examiner's checklist score
	Type of Statistical Test	Other
	Comments
Statistical Test of Hypothesis	p-Value	0.190
	Comments
	Method	ANOVA
	Comments

3. Other Pre-specified Outcome

Title	PROSPECT Proficiency
Description	After the RCT, a registry was started with all participants that started PROSPECT. Outcome: do PROSPECT participants pass the curriculum, do they become proficient?
Time Frame	After RCT

Outcome Measure Data

Analysis Population Description

[Not Specified]

Arm/Group Title

Arm/Group Description

Adverse Events

Adverse Event Reporting Description	Participants were not at risk of adverse events as they were not patients but healthy trainees. There was monitoring/assesment for all-cause mortality, serious and other (not including serious) adverse events, but none occured.
Arm/Group Title	PROSPECT		E-LEARNING		CONTROL
Arm/Group Description	the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire and perform a simple and a complex exercise on the simulator. additional curriculum		the trainee will study four modules of E-learning (basic endovascular skills module, iliac artery module, superficial femoral artery module and postoperative care module. After studying each module the trainee will complete a Multiple Choice Questionnaire. No simulation exercises.		The trainee will continue clinical education without additional curriculum, but will be allowed to study independently.
All Cause Mortality
	PROSPECT		E-LEARNING		CONTROL
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/11 (0%)		0/10 (0%)		0/11 (0%)
Serious Adverse Events
	PROSPECT		E-LEARNING		CONTROL
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/11 (0%)		0/10 (0%)		0/11 (0%)
Other (Not Including Serious) Adverse Events
	PROSPECT		E-LEARNING		CONTROL
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/11 (0%)		0/10 (0%)		0/11 (0%)

Limitations/Caveats

Single centre, results may not be applicable to other centres/countries. Randomisation status was not blinded for participants, consultant can unintentionally know randomisation group. Long-term skills retention was not assessed beyond 3 months post-training. As the procedures were closely supervised, operative metrics cannot be used to assess the operative performance.

More Information

Certain Agreements

All Principal Investigators ARE employed by the organization sponsoring the study.

There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.