Educational Study of Multimedia in Surgical Skills Training in Colorectal Surgery
Study Details
Study Description
Brief Summary
To practice independently surgeons require competency in surgical skills, encompassing a combination of technical and non-technical skills. Cognitive skills, aspects of non-technical skills, represent an integral component of surgical competency. Cognitive skills comprise factual knowledge and decision-making.
Changing work patterns in the United Kingdom, as specified by the European Working Time Directive (EWTD), have had a profound impact on the delivery of surgical skills training. Surgical trainees are now increasingly removed from normal working hours in which the majority of traditional operative training and experience is gained, leading to a net reduction in trainees' operative exposure. This reduction in operative experience means that surgical competence can no longer be assured on the basis of experience alone.
Although there is no educational technology that can replace the craft apprenticeship required to train a competent surgeon, reduction in training hours has led to rapid development of educational tools to augment surgical skills training outside the operating room environment. These tools tend to concentrate on technical skills performance without emphasis on cognitive skills.
Trainees in today's era have grown up in a multimedia environment; multimedia is media that uses a combination of text, voiceover, animation and video. Multimedia is an underdeveloped educational resource that can supplement cognitive skills training in operative surgery.
The purpose of this study was to design and develop an online multimedia educational tool in a common colorectal surgery procedure ("Anterior Resection") and determine the effectiveness of this tool in teaching and assessment of cognitive skills.
Study hypothesis: Multimedia learning is equivalent to conventional teaching "Study Day" in improving scores in cognitive surgical skills.
| Condition or Disease | Intervention/Treatment | Phase |
|---|---|---|
|
N/A |
Detailed Description
This study involved two main objectives:
- Develop and design multimedia educational tools for open and laparoscopic anterior resection
-
The multimedia tools were developed in collaboration with a professional multimedia company (Digimed). Initially 31 open and 35 laparoscopic (key-hole) procedures were filmed after informed, written consent was obtained from patients. Key procedural steps/ subtasks were identified and integrated onto navigational interactive interface platforms developed using Adobe® Flash® Professional CS5 10.1.
-
Each subtask was composited into succinct video clips using Final Cut Pro®, supplemented with annotation, animation, text and voiceover. Content addressed cognitive aspects including factual and anatomical knowledge, decision-making points, instrumentation required, team positioning and use of assistance. The multimedia educational tools are available online at following the link http://www.colorectaltraining.co.uk. The colorectal training website has 1.24GB including 69 video clips in the open tool and 67 video clips in the laparoscopic tool.
- Evaluate the effectiveness of multimedia in teaching/assessment of cognitive surgical skills
-
A randomised control study was conducted to evaluate the effectiveness of online multimedia in comparison to conventional "Study day" teaching in cognitive surgical skills acquisition. Enrolled participants were randomly allocated into either the intervention (Multimedia) or control (Study Day) Group. The methodology of the assessments was online timed (written) assessments to test cognitive surgical skills. All trainees were assessed before and after the study period.
-
Following ethics approval, permission was obtained from surgical training programme directors (TPDs) to recruit trainees into the study. The study was conducted between 6th October 2011 and 23rd December 2011. Trainees invited to participate were contacted by email. A study flyer attachment and hyperlink (http://www.colorectaltraining.co.uk) were provided for trainees to access relevant study information on a 'Participation Information Sheet' and an online consent form. After submission of the consent form, trainees were directed to separate web pages on the secure web-based service containing a Trainee Proforma (to obtain demographic participant information) and Pre-Assessment Test.
-
Participants in the multimedia group were provided with unrestricted access for 30 days and the study day was held at the end of this period for the "Study Day" group. Multimedia group participants were each emailed an individual login and password to access the multimedia tools on the colorectal training website: http://www.colorectaltraining.co.uk. The login details were requested each time the website was accessed.
-
The study day involved a series of detailed interactive lectures covering all the steps of open and laparoscopic anterior resection surgery. The content delivered was identical to that in the multimedia tool. After a period of one week, all study participants were required to complete the online Post-Assessment test. Multimedia group participants emailed a separate hyperlink to complete an online Evaluation form. All completed forms were stored securely for analysis.
- Sample size
-
Sample size was calculated on the basis of a pilot study of 50 surgical trainees taking part in an anterior resection study day in January 2009. Trainees were asked to complete a pre-study day written assessment, followed by a post-study day assessment immediately after the lectures had been completed. The mean (SD) pre- and post-assessments scores were 12.4 (4.56) and 26.8 (3.99) respectively.
-
The sample size was calculated with the mean change under the assumption that the baseline (pre-assessment) score for both arms is the same. Mean change in score in the control is 14.4 (4.9) and an expected mean change in score of 24.4 (SD 4.9) in the intervention group is assumed. The sample size required to test the hypothesis at a type I error of 5% and power of 80% was calculated (per group) to be 10. So a total of 20 trainees need to be recruited. If the response rate is 20% and assuming a drop off rate of 20% ((20/20%)/ (1-20%)), a total of 125 trainees needed to be approached.
- Statistical analysis
-
The baseline pre-assessment test ensured comparability of knowledge base and cognitive skills between study groups and helped to assess construct validity of the assessment tool. Basic descriptive statistics include the numbers and percentages for categorical data, mean and standard deviation for normally distributed continuous data; median and range for skewed data was also included. All data were analysed on an intention to treat basis.
-
Differences in assessment scores between multimedia and control (study day) groups were compared using either the unpaired Students' t-test (for normally distributed data) or the Mann Whitney U test (for skewed data). To analyse post-scores versus pre-test scores within the same group, paired t-test (for normally distributed data) or the Mann Whitney U test (for skewed data) was used. Data was analysed using SPSS (statistical package for social sciences version 16.0, Chicago IL). Significance levels were set at P<0.05.
Study Design
Arms and Interventions
| Arm | Intervention/Treatment |
|---|---|
| Experimental: Multimedia group The multimedia group is the interventional arm of the study |
Other: Multimedia group
Participants are provided with unrestricted access to the online multimedia educational tools for self-directed study (during the study period)
Other Names:
|
| Experimental: Study Day Group The study day group are the control arm of the study |
Other: Study Day Group
Participants in the control arm of the study attend a conventional teaching "Study Day" involving a series of lectures on Open and Laparoscopic Anterior Resection
|
Outcome Measures
Primary Outcome Measures
- Improvement in assessment scores following implementation of the teaching modality (Multimedia and Study day). [Participants will be followed for the duration of the educational study, a period of 11 weeks]
All participants were assessed using a timed 30 minute online tool (sent via a hyperlink) to test cognitive skills both before and after the teaching modality. The assessment tool comprised a random assortment of 30 multiple choice and short answer questions. A similar test had been previously validated. A large bank of two hundred questions was designed to comprehensively cover cognitive skills relevant to all the procedural steps in open and laparoscopic 'anterior resection' surgery. The question content was germane to the information delivered in the multimedia tools and study day. Completed forms were stored securely on Smart Survey software. The purpose of the on-line pre-assessment test score was to establish the baseline level knowledge of all participants, prior to randomisation.
Secondary Outcome Measures
- The association between change in scores and level of training and acceptability of multimedia as an educational resource. [Participants will be followed for the duration of the educational study, a period of 11 weeks]
Eligibility Criteria
Criteria
Inclusion Criteria:
- Postgraduate specialist general surgical trainees at Speciality Training Year 3 (ST3) level/ Specialist Registrar (SpR) Year 1 or above
Exclusion Criteria:
-
Postgraduate general surgical trainees below ST3 level/ SPR Year 1
-
ALL non general surgical postgraduate trainees
-
Medical students
Contacts and Locations
Locations
| Site | City | State | Country | Postal Code | |
|---|---|---|---|---|---|
| 1 | Academic Surgical Oncology Unit, University of Sheffield | Sheffield | South Yorkshire | United Kingdom | S10 2RX |
Sponsors and Collaborators
- University of Sheffield
- Ethicon Endo-Surgery
- HCA International Limited
- Digimed
Investigators
- Principal Investigator: Umar Shariff, MBChB, MRCS, University of Sheffield
- Principal Investigator: Saba Balasubramanian, PhD, FRCS, University of Sheffield
Study Documents (Full-Text)
None provided.More Information
Additional Information:
Publications
- Beard JD. Assessment of surgical competence. Br J Surg. 2007 Nov;94(11):1315-6. Erratum in: Br J Surg. 2008 Mar;95(3):402.
- Fitzgerald JE, Caesar BC. The European Working Time Directive: a practical review for surgical trainees. Int J Surg. 2012;10(8):399-403. doi: 10.1016/j.ijsu.2012.08.007. Epub 2012 Aug 24.
- Kneebone RL. Skills training using multimedia and models. Hosp Med. 2001 Jul;62(7):428-30.
- Moonesinghe SR, Lowery J, Shahi N, Millen A, Beard JD. Impact of reduction in working hours for doctors in training on postgraduate medical education and patients' outcomes: systematic review. BMJ. 2011 Mar 22;342:d1580. doi: 10.1136/bmj.d1580. Review.
- Reznick RK. Surgical simulation: a vital part of our future. Ann Surg. 2005 Nov;242(5):640-1.
- SMBRER186