Evaluation of Implementation of a National Point-of-Care Ultrasound Training Program

• Sponsor: VA Office of Research and Development (U.S. Fed)
• Overall Status: Completed
• CT.gov ID: NCT03296280
• Collaborator: US Department of Veterans Affairs (U.S. Fed)
• Enrollment: 116
• Location: 1
• Actual Duration (Months): 47.9
• Patients Per Site Per Month: 2.4

Study Details

Study Description

Brief Summary

This VA QUERI Partnered Evaluation Initiative will evaluate the impact of an immersive Point-of-care Ultrasound (POCUS) Training Course on provider skill acquisition and retention; the frequency of POCUS use by trained providers; and the barriers/facilitators to POCUS in the VHA. Data sources include pre- and post-course assessment tools, medical coding data, and course evaluations. Providers that participate in the POCUS Training Course will be compared to control providers from wait-listed facilities. Additionally, participating facilities vs. wait-listed facilities for the POCUS Training Course will be compared. Findings from this project will guide ongoing efforts of the investigators' operating partners, VA Specialty Care Centers of Innovation (SCCI) and the VA Simulation Learning and Research Network (SimLEARN), to develop a national POCUS training program and facilitate implementation of POCUS use system-wide in the VA healthcare system.

Condition or Disease	Intervention/Treatment	Phase
Other Acute Illnesses Presenting to the Hospital; Heart Failure; Pneumonia; Deep Venous Thrombosis; Cellulitis; Abdominal Aortic Aneurysm; Acute Cholecystitis; Peritoneal Free Fluid; Hydronephrosis; Pleural Effusion; Urinary Retention

Detailed Description

Background: Point-of-care ultrasonography (POCUS) has been shown to reduce procedure-related complications from invasive bedside procedures, reduce time to diagnosis, and reduce ancillary testing, which ultimately reduces patient radiation exposure and healthcare costs. Despite its potential advantages, POCUS has not been universally adopted in healthcare due to limited numbers of providers trained in use of POCUS. The VA's Simulation Learning, Education and Research Network (SimLEARN) and Specialty Care Centers of Innovation (SCCI) have launched a collaborative initiative to develop a national POCUS training program. The goal of this collaborative training program is to teach VA providers basic diagnostic and procedural applications of POCUS. The proposed project will evaluate the effectiveness of an immersive POCUS training course developed by SimLEARN on provider skill acquisition and retention, frequency of use, and identify barriers/facilitators to POCUS use at participating VHA facilities.

Methods: Approximately 120 participants from a convenience sample of 20 diverse VA facilities will participate in a 2.5-day immersive POCUS training course at the SimLEARN National Simulation Center in Orlando during fiscal year 2017. The investigators' evaluation plan will collect data from facility Chiefs of Staff, providers from facilities participating in POCUS training, and providers from wait-listed facilities at different time points using different assessment tools: VA Facility POCUS Survey, Provider POCUS Survey, Brief Provider POCUS Survey, Pre-/Post-course Knowledge and Skills Test, and Coding Data.

Objective 1: Evaluate provider skill acquisition and retention, and frequency of POCUS use after participation in the POCUS Training Course. Pre- and post-course testing will be used to assess acquisition of knowledge and technical skills to perform POCUS exams. Post-course testing for knowledge and skill retention, and frequency of use, will be performed 6-9 months after the training course.

Objective 2: Determine the effect of the POCUS Training Course and implementation facilitation on facility-level frequency of POCUS use. Facilities with providers that infrequently use POCUS will be eligible to participate in the POCUS Training Course. However, not all facilities will be able to be accommodated in the 1st year of the training program, requiring a facility waiting list. Facilities with providers participating in the POCUS Training Course will be compared to wait-listed facilities with regard to frequency of POCUS use. Using the Brief Provider POCUS Survey, frequency of POCUS use by providers in both participating and waitlisted facilities will be compared. Additionally, coding data will be reviewed to assess frequency of procedures performed with and without imaging guidance and procedural complication rates comparing the two groups of facilities.

Objective 3: Determine provider and facility-level barriers and facilitators to POCUS use. Provider and facility-level barriers will be assessed using 3 tools: Provider POCUS Survey (trained providers), Brief Provider POCUS Survey (wait-listed and participating facilities), and Facility POCUS Survey (all facilities). Differences in barriers reported and their relationship to frequency of POCUS use will be compared.

Objective 4: Development of POCUS Champions course to facilitate local implementation by addressing facility-level barriers. The POCUS Champions course will address several facility-level barriers to implementation that have been identified by the PEI: lack of image archiving, limited access to ultrasound machines, undefined credentialing/privileging processes, need for a quality assurance process, and lack of trained providers. The POCUS Champions course will include an annual 3-day boot camp, bi-monthly conference calls, and ongoing mentorship. The goal of the 3-day boot camp will be for Champions to prepare a local implementation plan that can be presented to their facility's leadership. During the bimonthly conference calls, a brief didactic on a focused topic will be presented followed by an update on local progress from each site. The initial goal of the POCUS Champions course is to facilitate implementation of POCUS use in VA facilities, starting with 4-6 facilities that can serve as models for other facilities. The research to date has identified these elements as core components of a successful POCUS program. The POCUS Champions course will support progress on intermediate milestones towards ensuring these core components are in place, to facilitate adoption, reach, and sustainment. This provides a roadmap to sustainable POCUS use.

Deliverables: The primary deliverable will be report that will summarize the effect of training on provider skill acquisition and retention, frequency of use of POCUS, and barriers and facilitators to POCUS implementation. This project will guide SimLEARN's and SCCI's efforts to develop a National POCUS Training Program and implement POCUS use nationally in the VA healthcare system. Additionally, this report may serve as a roadmap for the VHA for implementation of similar simulation-based educational innovations.

Study Design

• Study Type: Observational
• Actual Enrollment: 116 participants
• Observational Model: Cohort
• Time Perspective: Prospective
• Official Title: Evaluation of Implementation of a National Point-of-Care Ultrasound Training Program
• Actual Study Start Date: Oct 3, 2016
• Actual Primary Completion Date: Jun 29, 2018
• Actual Study Completion Date: Sep 30, 2020

Arms and Interventions

Arm	Intervention/Treatment
Early Intervention This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17
Late Intervention This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.

Outcome Measures

Primary Outcome Measures

1. Change in Frequency of Point-of-care Ultrasound Use for Diagnostic and Procedural Applications at Baseline and 6-9 Months [Baseline compared to 6-9 months later.]

   Provider self-reported data pre- and post-intervention on frequency of point-of-care ultrasound use, based on number of exams per week.

Secondary Outcome Measures

1. Change in Point-of-care Ultrasound Knowledge From Baseline to 6-9 Months [Baseline compared to 6-9 months later.]

   Provider knowledge will be tested pre- and post-intervention (at baseline [pre-course] and 6-9 months post-course). Knowledge Exam has a minimum of 0 points and a maximum of 30 points (1 pt per question). A higher score indicates more knowledge about POCUS. A positive change in score means a better outcome post-course (net increase in knowledge) compared to baseline.

2. Change in Point-of-care Ultrasound Cardiac Skills From Baseline to 6-9 Months [Baseline compared to 6-9 months later.]

   Provider hands-on cardiac skills will be tested pre- and post-intervention (at baseline [pre-course], and 6-9 months post-course). The minimum score on the POCUS Cardiac Skills scale is 0 points and the maximum is 100 points. A higher score indicates more skill. A positive change in score means a better outcome post-course (net increase in skills) compared to baseline.

3. Change in Point-of-care Ultrasound Lung Skills From Baseline to 6-9 Months [Baseline compared to 6-9 months later.]

   Provider hands-on lung skills will be tested pre- and post-intervention (at baseline [pre-course], and 6-9 months post-course). The minimum score on the POCUS Lung Skills scale is 0 points and the maximum is 100 points. A higher score indicates more skill. A positive change in score means a better outcome post-course (net increase in skills) compared to baseline.

4. Change in Point-of-care Ultrasound Peripheral IV Skills From Baseline to 6-9 Months [Baseline compared to 6-9 months later.]

   Provider hands-on peripheral IV skills will be tested pre- and post-intervention (at baseline [pre-course], and 6-9 months post-course). The minimum score on the POCUS Peripheral IV Skills scale is 0 points and the maximum is 100 points. A higher score indicates more skill. A positive change in score means a better outcome post-course (net increase in skills) compared to baseline.

5. Change in Point-of-care Ultrasound Abdominal Skills From Baseline to 6-9 Months [Baseline compared to 6-9 months later.]

   Provider hands-on abdominal skills will be tested pre- and post-intervention (at baseline [pre-course], and 6-9 months post-course). The minimum score on the POCUS Abdominal Skills scale is 0 points and the maximum is 100 points. A higher score indicates more skill. A positive change in score means a better outcome post-course (net increase in skills) compared to baseline.

6. Change in Barrier to Point-of-care Ultrasound Use (Ability to Operate) From Baseline to 6-9 Months Post-Course [Baseline compared to 6-9 months later.]

   Change in percentage of providers who reported not knowing how to operate POCUS 6-9 months post-course compared to pre-course. A positive percentage indicates a better outcome post-course.

7. Change in Barriers to Point-of-care Ultrasound Use (Skill Maintenance) From Baseline to 6-9 Months Post-Course [Baseline compared to 6-9 months later.]

   Change in percentage of providers who reported experiencing difficulty with maintaining POCUS skills pre-course compared to 6-9 months post-course. A positive percentage indicates more providers are experiencing this barrier post-course, which is not unexpected as more providers acquire POCUS skills.

Eligibility Criteria

Criteria

• Ages Eligible for Study: N/A and Older
• Sexes Eligible for Study: All
• Accepts Healthy Volunteers: Yes

Inclusion Criteria:

Selection of VA facilities invited to participate in the national POCUS training course was based on data from a national needs assessment conducted by the VA's Health Analysis and Information Group. VA facilities that meet all of the following inclusion criteria are eligible to send physicians to participate in the POCUS training course:

1. Chief of staff desires POCUS training for his/her facility

2. =2 Service Chiefs from emergency medicine, critical care medicine, or hospital medicine desire training for his/her physicians, AND

3. =1 portable ultrasound machine(s) available to physicians within these services

Exclusion Criteria:

1. Chief of staff does not desire POCUS training for his/her facility

2. <2 Service Chiefs from emergency medicine, critical care medicine, or hospital medicine desire training for his/her physicians, OR

3. No portable ultrasound machine(s) available to physicians within these services

Contacts and Locations

Locations

	Site	City	State	Country	Postal Code
1	South Texas Health Care System, San Antonio, TX	San Antonio	Texas	United States	78229

Sponsors and Collaborators

• VA Office of Research and Development
• US Department of Veterans Affairs

Investigators

• Principal Investigator: Nilam Jayant Soni, MD MSc, South Texas Health Care System, San Antonio, TX

Study Documents (Full-Text)

• Study Protocol and Statistical Analysis Plan - Sep 9, 2016

More Information

Additional Information:

• VA Quality Enhancemen Research Initiative (QUERI)

Publications

None provided.

• Responsible Party: VA Office of Research and Development
• ClinicalTrials.gov Identifier: NCT03296280
• Other Study ID Numbers: PEX 16-003; I50HX002263-01A1
• First Posted: Sep 28, 2017
• Last Update Posted: Nov 23, 2020
• Last Verified: Nov 1, 2020

• Individual Participant Data (IPD) Sharing Statement: No
• Plan to Share IPD: No
• Studies a U.S. FDA-regulated Drug Product: No
• Studies a U.S. FDA-regulated Device Product: No
• Product Manufactured in and Exported from the U.S.: No
• Keywords provided by VA Office of Research and Development: ultrasound; ultrasonography; point of care; Doppler ultrasound; ultrasound imaging; diagnostic ultrasound
• Additional relevant MeSH terms: Cellulitis; Cholecystitis; Cholecystitis, Acute; Pleural Effusion; Urinary Retention; Hydronephrosis; Thrombosis; Aortic Aneurysm; Aortic Aneurysm, Abdominal; Venous Thrombosis

Study Results

Participant Flow

Recruitment Details	10 facilities were to participate in the first 6 POCUS course session during FY17. 10 facilities were to participate in the last 6 POCUS course sessions during FY17. Participants, rather than facilities, were recruited and assigned to to the early or late intervention arms.

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17. Although facilities are being recruited the intervention is taking place at the individual level. All reporting will be at the individual level.	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17. Although facilities are being recruited the intervention is taking place at the individual level. All reporting will be at the individual level.
Period Title: Overall Study
STARTED	73	43
COMPLETED	67	36
NOT COMPLETED	6	7

Baseline Characteristics

Arm/Group Title	Early Intervention	Late Intervention	Total
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.	Total of all reporting groups
Overall Participants	73	43	116
Age (years) [Mean (Standard Deviation) ]
Mean (Standard Deviation) [years]	45.7 (10.6)	43.5 (7.7)	44.9 (9.7)
Sex: Female, Male (Count of Participants)
Female	27 37%	18 41.9%	45 38.8%
Male	43 58.9%	21 48.8%	64 55.2%
Race and Ethnicity Not Collected (Count of Participants)
Count of Participants [Participants]	0 0%
Provider Specialty (participants) [Number]
Anesthesiology	6 8.2%	7 16.3%	13 11.2%
Critical Care	10 13.7%	8 18.6%	18 15.5%
Emergency Medicine	13 17.8%	9 20.9%	22 19%
Endocrinology	1 1.4%	0 0%	1 0.9%
General Surgery	0 0%	3 7%	3 2.6%
Internal Medicine	31 42.5%	10 23.3%	41 35.3%
Pulmonary	8 11%	2 4.7%	10 8.6%
Other	1 1.4%	0 0%	1 0.9%
Facility Location (participants) [Number]
Continental	13 17.8%	0 0%	13 11.2%
Midwest	11 15.1%	4 9.3%	15 12.9%
Northeast	8 11%	12 27.9%	20 17.2%
Pacific	11 15.1%	11 25.6%	22 19%
Southeast	30 41.1%	16 37.2%	46 39.7%
Years of practice since residency (years) [Mean (Standard Deviation) ]
Mean (Standard Deviation) [years]	14.8 (10.4)	11.5 (6.9)	13.6 (9.5)
Percent of time performing direct patient care (% of time performing direct pt care) [Mean (Standard Deviation) ]
Mean (Standard Deviation) [% of time performing direct pt care]	57 (30)	58 (23)	57 (28)

Outcome Measures

1. Primary Outcome

Title	Change in Frequency of Point-of-care Ultrasound Use for Diagnostic and Procedural Applications at Baseline and 6-9 Months
Description	Provider self-reported data pre- and post-intervention on frequency of point-of-care ultrasound use, based on number of exams per week.
Time Frame	Baseline compared to 6-9 months later.

Outcome Measure Data

Analysis Population Description

Participants who completed a provider survey both at the start and about 6-9 months later

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
Measure Participants	62	33
Mean (95% Confidence Interval) [Number of POCUS Exams/week]	12.9	4.0

2. Secondary Outcome

Title	Change in Point-of-care Ultrasound Knowledge From Baseline to 6-9 Months
Description	Provider knowledge will be tested pre- and post-intervention (at baseline [pre-course] and 6-9 months post-course). Knowledge Exam has a minimum of 0 points and a maximum of 30 points (1 pt per question). A higher score indicates more knowledge about POCUS. A positive change in score means a better outcome post-course (net increase in knowledge) compared to baseline.
Time Frame	Baseline compared to 6-9 months later.

Outcome Measure Data

Analysis Population Description

Participants who completed a knowledge survey both at the start and about 6-9 months later were analyzed. Those who did not provide both surveys are not reported. Row populations may therefore differ from total population counts.

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
Measure Participants	57	27
Mean (95% Confidence Interval) [change in score on POCUS Knowledge scale]	5.3	6.5

3. Secondary Outcome

Title	Change in Point-of-care Ultrasound Cardiac Skills From Baseline to 6-9 Months
Description	Provider hands-on cardiac skills will be tested pre- and post-intervention (at baseline [pre-course], and 6-9 months post-course). The minimum score on the POCUS Cardiac Skills scale is 0 points and the maximum is 100 points. A higher score indicates more skill. A positive change in score means a better outcome post-course (net increase in skills) compared to baseline.
Time Frame	Baseline compared to 6-9 months later.

Outcome Measure Data

Analysis Population Description

Participants who completed a skills survey both at the start and about 6-9 months later were analyzed. Those who did not provide both surveys are not reported. Row populations may therefore differ from total population counts.

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
Measure Participants	58	23
Mean (95% Confidence Interval) [change in score on Cardiac Skills scale]	41.6	46.1

4. Secondary Outcome

Title	Change in Point-of-care Ultrasound Lung Skills From Baseline to 6-9 Months
Description	Provider hands-on lung skills will be tested pre- and post-intervention (at baseline [pre-course], and 6-9 months post-course). The minimum score on the POCUS Lung Skills scale is 0 points and the maximum is 100 points. A higher score indicates more skill. A positive change in score means a better outcome post-course (net increase in skills) compared to baseline.
Time Frame	Baseline compared to 6-9 months later.

Outcome Measure Data

Analysis Population Description

Participants who completed a skills survey both at the start and about 6-9 months later were analyzed. Those who did not provide both surveys are not reported. Row populations may therefore differ from total population counts.

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
Measure Participants	58	24
Mean (95% Confidence Interval) [change in score on Lung Skills scale]	49.2	60.5

5. Secondary Outcome

Title	Change in Point-of-care Ultrasound Peripheral IV Skills From Baseline to 6-9 Months
Description	Provider hands-on peripheral IV skills will be tested pre- and post-intervention (at baseline [pre-course], and 6-9 months post-course). The minimum score on the POCUS Peripheral IV Skills scale is 0 points and the maximum is 100 points. A higher score indicates more skill. A positive change in score means a better outcome post-course (net increase in skills) compared to baseline.
Time Frame	Baseline compared to 6-9 months later.

Outcome Measure Data

Analysis Population Description

Participants who completed a skills survey both at the start and about 6-9 months later were analyzed. Those who did not provide both surveys are not reported. Row populations may therefore differ from total population counts.

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
Measure Participants	54	21
Mean (95% Confidence Interval) [change in score on Peri-IV Skills scale]	31.6	40.5

6. Secondary Outcome

Title	Change in Point-of-care Ultrasound Abdominal Skills From Baseline to 6-9 Months
Description	Provider hands-on abdominal skills will be tested pre- and post-intervention (at baseline [pre-course], and 6-9 months post-course). The minimum score on the POCUS Abdominal Skills scale is 0 points and the maximum is 100 points. A higher score indicates more skill. A positive change in score means a better outcome post-course (net increase in skills) compared to baseline.
Time Frame	Baseline compared to 6-9 months later.

Outcome Measure Data

Analysis Population Description

Participants who completed a skills survey both at the start and about 6-9 months later were analyzed. Those who did not provide both surveys are not reported. Row populations may therefore differ from total population counts.

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
Measure Participants	58	24
Mean (95% Confidence Interval) [change in score on Abdom Skills scale]	40.8	54.8

7. Secondary Outcome

Title	Change in Barrier to Point-of-care Ultrasound Use (Ability to Operate) From Baseline to 6-9 Months Post-Course
Description	Change in percentage of providers who reported not knowing how to operate POCUS 6-9 months post-course compared to pre-course. A positive percentage indicates a better outcome post-course.
Time Frame	Baseline compared to 6-9 months later.

Outcome Measure Data

Analysis Population Description

Participants who reported not knowing how to use POCUS machine in survey at the beginning of the course

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
Measure Participants	22	9
Number [percentage of participants]	68 93.2%	67 155.8%

8. Secondary Outcome

Title	Change in Barriers to Point-of-care Ultrasound Use (Skill Maintenance) From Baseline to 6-9 Months Post-Course
Description	Change in percentage of providers who reported experiencing difficulty with maintaining POCUS skills pre-course compared to 6-9 months post-course. A positive percentage indicates more providers are experiencing this barrier post-course, which is not unexpected as more providers acquire POCUS skills.
Time Frame	Baseline compared to 6-9 months later.

Outcome Measure Data

Analysis Population Description

Participants who reported at least one barrier 6-9 months after course

Arm/Group Title	Early Intervention	Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17	This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
Measure Participants	30	19
Number [percentage of participants]	23 31.5%	37 86%

Adverse Events

Time Frame	Adverse events were not monitored. Not applicable.
Adverse Event Reporting Description	Adverse events were not monitored. Not applicable.
Arm/Group Title	Early Intervention		Late Intervention
Arm/Group Description	This cohort of 10 facilities will undergo early training by participating in the first 6 POCUS course sessions during FY17		This cohort of 10 facilities will undergo late training by participating in the last 6 POCUS courses during FY17.
All Cause Mortality
	Early Intervention		Late Intervention
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)
Serious Adverse Events
	Early Intervention		Late Intervention
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)
Other (Not Including Serious) Adverse Events
	Early Intervention		Late Intervention
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)

Limitations/Caveats

Maintaining early or late intervention groups due to schedules/space limitations. Equipment not readily available and inability to receive IT approvals limited follow-up scheduling. Participants with limited clinical time may skew scores.

More Information

Certain Agreements

All Principal Investigators ARE employed by the organization sponsoring the study.

There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.