Increasing Evidence-based Clinical Practices in VA
Study Details
Study Description
Brief Summary
Evidence-based clinical practices (EBCPs) are often not widely adopted, despite extensive efforts to influence individual practitioners to use them. The aim of the project is to work with Veterans Integrated Service Networks (VISN) 1, 10, and 23 and their medical centers to create and test organizations that facilitate the use of EBCPs. The research objectives are to: Test the effectiveness of the proposed organizational model in comparison with a more limited data-feedback strategy in improving system use of a selected EBCP; Identify and analyze organizational factors that affect model implementation; Test the feasibility of intervention activities to introduce and support the model.
Condition or Disease | Intervention/Treatment | Phase |
---|---|---|
|
N/A |
Detailed Description
Background:
The Veterans Health Administration (VHA) needs to develop efficient ways to broadly implement evidence based practices and foster a learning organization culture that systematically and continuously applies research to improve VA healthcare. Recognizing this need, VHA Health Services Research and Development (HSR&D) invited applications in the fall of 2003 for collaboration HSR&D investigators and Integrated Service Networks (VISNs) on a) implementing and evaluating an evidenced-based interventions or b) undergoing and evaluating an organizational or structural change to transform the VISN in to a learning organization that can efficiently implement evidence-based practices. Collaborations are intended to help improve clinical services locally within participating VISNs and provide templates for expanding successful changes nationwide.
Objectives:
Despite recognition that successful implementation of evidence-based clinical practices (EBCPs) usually depends on the on the structure and processes of the larger healthcare organization in which new clinical practices are introduced, the processes and dynamics of implementation are not well understood. The aim of this project was to deepen that understanding by testing an organizational model that we hypothesized would strengthen the ability of healthcare organizations to implement evidence-based clinical practices. The research objectives were to: - Test the hypothesis that medical centers with high fidelity to the organizational model would be more successful in improving system use of a selected EBCP;
- Identify and analyze organizational factors that affect model implementation; - Test the feasibility of intervention activities to introduce and support the model.
Methods:
The three-year study used a mixed-methods pre-post comparison-group design to implement and evaluate the organizational model in medical centers in 3 VISNs in the Department of Veterans Affairs. The model posits that the implementation of evidence-based practices will be enhanced through the presence of three interacting components in the organization: 1) active leadership commitment to quality, 2) robust clinical process redesign to incorporate evidence-based practices into routine operations, and 3) use of management structures and processes to support and align redesign. The target clinical practice was hand-hygiene compliance. One VISN was randomly assigned to the intervention arm that implemented the organizational model; two VISNs were assigned to a comparison arm that participated in a more limited data feedback strategy. Measures included: 1) ratings of implementation fidelity, as measured on a 0-4 scale at the site level supported by narrative evidence by research team; 2) percent compliance with national hand-hygiene guidelines for each site, as measured through structured observations by medical center staff; 3) staff ratings of team effectiveness and facility emphasis on quality, as measured through a written survey; and 4) factors affecting model implementation, as identified qualitatively through interviews and quantitatively through staff surveys
Status:
Data collection and analysis is complete. Manuscript preparation is ongoing.
Study Design
Arms and Interventions
Arm | Intervention/Treatment |
---|---|
Other: Organizational Model Organizational Model: Participants in this arm of the study will receive information regarding the organizational model and work closely with the research team throughout the project to implement various aspects of the model. Participants in this arm of the study will be interviewed and participate in the data feedback portion of the study as well. |
Behavioral: Organization Model
The organizational model contains three components: leadership support, a multidisciplinary redesign team, and management structures and processes to link the two.
Behavioral: Data Feedback Model
The research team will periodically interview the facilities and provide them with reported hand hygiene data.
|
Other: Data Feedback Data Feedback Only: Participants in this arm will be interviewed periodically and participate in the data feedback portion of the study. |
Behavioral: Data Feedback Model
The research team will periodically interview the facilities and provide them with reported hand hygiene data.
|
Outcome Measures
Primary Outcome Measures
- Fidelity to the Organizational Model [Fidelity was assessed at the end of the 3 year study.]
Final fidelity to the Organizational Model was assessed by averaging scores for each component of the model. Scores ranged from 0 (no evidence of that factor present) to 4 (factor fully present and used as intended). The 3 main components of the model included 1) active leadership commitment to quality, 2) robust clinical process redesign to incorporate evidence-based practices into routine operations, and 3) use of management structures and processes to support and align redesign. Scores for each component of the model were measured by the study team using structured rating instruments based on data collected during interviews. Sites with an overall fidelity score above 3.0 were considered to have high fidelity to the organizational model.
- Effect Size of Improvement in Hand Hygiene Compliance [3 months pre and post study intervention]
The effect size of improvement in hand-hygiene compliance was calculated by comparing the baseline three-month periods to the last three-month periods of the study. To evaluate the statistical significance of changes in proportion adherence over time, we ran a weighted least squares regression model with time (i.e. month) as the independent variable and adherence proportion as the dependent variable. The sample size in each data collection period was used as the weight. Our interest is in the statistical significance of the coefficient associated with time. To evaluate the practical significance of the change pre and post intervention, we examined the effect size associated with the change in proportion adherence in the first 3-month period of data collection and the last 3-month period. Effect size was calculated as 2*arcsin(sqr(p2)) - 2*arcsin(sqr(p1)). Using Cohen's criteria, an effect size of .2 is interpreted as small, .5 as medium and .8 as large.
Eligibility Criteria
Criteria
Inclusion Criteria:
VA employees of VISNs 1, 10 and 23 working in participating clinical units or holding a leadership position in medical centers identified by facility leadership for participation in the study.
Exclusion Criteria:
Must have operational proficiency in English. All to be interviewed and surveyed will hold professional positions in VA, so this is not expected to be a major barrier to inclusion.
Contacts and Locations
Locations
Site | City | State | Country | Postal Code | |
---|---|---|---|---|---|
1 | VA Boston Healthcare System Jamaica Plain Campus, Jamaica Plain, MA | Boston | Massachusetts | United States | 02130 |
Sponsors and Collaborators
- US Department of Veterans Affairs
Investigators
- Principal Investigator: Carol VanDeusen-Lukas, EdD, VA Boston Healthcare System Jamaica Plain Campus, Jamaica Plain, MA
Study Documents (Full-Text)
None provided.More Information
Publications
None provided.- IMV 04-055
Study Results
Participant Flow
Recruitment Details | |
---|---|
Pre-assignment Detail |
Arm/Group Title | Organizational Model | Data Feedback |
---|---|---|
Arm/Group Description | This arm is considered the intervention arm. Participants in this arm of the study will receive information regarding the organizational model and work closely with the research team throughout the project to implement various aspects of the model. Organization Model: The organizational model contains three components: leadership support, a multidisciplinary redesign team, and management structures and processes to link the two. | This arm is considered the control arm. Participants in this arm will be interviewed periodically and participate in the data feedback portion of the study but will not undergo any active intervention pertaining to the organizational model. Data Feedback Model: This is considered the non-intervention arm. The research team will periodically interview the facilities and provide them with reported hand hygiene data. |
Period Title: Overall Study | ||
STARTED | 889 | 735 |
COMPLETED | 889 | 735 |
NOT COMPLETED | 0 | 0 |
Baseline Characteristics
Arm/Group Title | Organizational Model | Data Feedback | Total |
---|---|---|---|
Arm/Group Description | Participants in this arm of the study will receive information regarding the organizational model and work closely with the research team throughout the project to implement various aspects of the model. Organization Model: The organizational model contains three components: leadership support, a multidisciplinary redesign team, and management structures and processes to link the two. The research team will periodically interview the facilities and provide them with reported hand hygiene data. | Participants in this arm will be interviewed periodically and participate in the data feedback portion of the study but will not undergo any active intervention pertaining to the organizational model. Data Feedback Only Model: The research team will periodically interview the facilities and provide them with reported hand hygiene data. | Total of all reporting groups |
Overall Participants | 889 | 735 | 1624 |
Age (Count of Participants) | |||
<=18 years |
NA
NaN
|
NA
NaN
|
NA
NaN
|
Between 18 and 65 years |
NA
NaN
|
NA
NaN
|
NA
NaN
|
>=65 years |
NA
NaN
|
NA
NaN
|
NA
NaN
|
Sex: Female, Male (Count of Participants) | |||
Female |
NA
NaN
|
NA
NaN
|
NA
NaN
|
Male |
NA
NaN
|
NA
NaN
|
NA
NaN
|
Outcome Measures
Title | Fidelity to the Organizational Model |
---|---|
Description | Final fidelity to the Organizational Model was assessed by averaging scores for each component of the model. Scores ranged from 0 (no evidence of that factor present) to 4 (factor fully present and used as intended). The 3 main components of the model included 1) active leadership commitment to quality, 2) robust clinical process redesign to incorporate evidence-based practices into routine operations, and 3) use of management structures and processes to support and align redesign. Scores for each component of the model were measured by the study team using structured rating instruments based on data collected during interviews. Sites with an overall fidelity score above 3.0 were considered to have high fidelity to the organizational model. |
Time Frame | Fidelity was assessed at the end of the 3 year study. |
Outcome Measure Data
Analysis Population Description |
---|
This outcome measure was only assessed at the site (facility) level. |
Arm/Group Title | Organizational Model | Data Feedback |
---|---|---|
Arm/Group Description | Participants in this arm of the study received information regarding the organizational model and worked closely with the research team throughout the project to implement various aspects of the model. Organization Model: The organizational model contains three components: leadership support, a multidisciplinary redesign team, and management structures and processes to link the two. The Intervention Arm included 889 individual participants across 7 study sites (medical centers). | Participants in this arm were interviewed periodically and participated in the data feedback portion of the study but did not undergo any active intervention pertaining to the organizational model. Data Feedback Model: This is considered the non-intervention arm. The research team periodically interviewed the facilities and provided them with reported hand hygiene data. The Control Arm included 735 individual participants across 9 study sites (medical centers) in total. Only 5 of the study sites are included in the reporting of outcome data due to incomplete data at 4 of the study sites. |
Measure Participants | 889 | 735 |
Measure Facilities | 7 | 5 |
Site A |
3.95
|
NA
|
Site B |
3.38
|
NA
|
Site C |
3.23
|
NA
|
Site D |
3.17
|
NA
|
Site E |
2.15
|
NA
|
Site F |
1.98
|
NA
|
Site G |
1.42
|
NA
|
Site H |
NA
|
2.30
|
Site I |
NA
|
1.62
|
Site J |
NA
|
1.52
|
Site K |
NA
|
1.40
|
Site L |
NA
|
1.30
|
Title | Effect Size of Improvement in Hand Hygiene Compliance |
---|---|
Description | The effect size of improvement in hand-hygiene compliance was calculated by comparing the baseline three-month periods to the last three-month periods of the study. To evaluate the statistical significance of changes in proportion adherence over time, we ran a weighted least squares regression model with time (i.e. month) as the independent variable and adherence proportion as the dependent variable. The sample size in each data collection period was used as the weight. Our interest is in the statistical significance of the coefficient associated with time. To evaluate the practical significance of the change pre and post intervention, we examined the effect size associated with the change in proportion adherence in the first 3-month period of data collection and the last 3-month period. Effect size was calculated as 2*arcsin(sqr(p2)) - 2*arcsin(sqr(p1)). Using Cohen's criteria, an effect size of .2 is interpreted as small, .5 as medium and .8 as large. |
Time Frame | 3 months pre and post study intervention |
Outcome Measure Data
Analysis Population Description |
---|
This outcome measure was only assessed at the site (facility) level. |
Arm/Group Title | Organizational Model | Data Feedback |
---|---|---|
Arm/Group Description | Participants in this arm of the study received information regarding the organizational model and worked closely with the research team throughout the project to implement various aspects of the model. Organization Model: The organizational model contains three components: leadership support, a multidisciplinary redesign team, and management structures and processes to link the two. The Intervention Arm included 7 study sites (medical centers). Of the 7 sites there were 4 with high fidelity to the Organizational Model and 3 with low fidelity to the Organizational Model. | Participants in this arm were interviewed periodically and participated in the data feedback portion of the study but did not undergo any active intervention pertaining to the organizational model. Data Feedback Model: This is considered the non-intervention arm. The research team periodically interviewed the facilities and provided them with reported hand hygiene data. The Control Arm included 9 study sites (medical centers) in total. Only 5 of the study sites are included in analysis of this outcome measure due to incomplete hand hygiene data at 4 of the study sites. Of the 5 study sites included, there were 2 sites with high fidelity to the Organization Model and 3 sites with low fidelity to the Organizational model, even though the model was not specifically introduced to the control arm. |
Measure Participants | 7 | 5 |
Measure Facilities | 7 | 5 |
Site A |
0.67
|
NA
|
Site B |
0.48
|
NA
|
Site C |
0.92
|
NA
|
Site D |
0.52
|
NA
|
Site E |
0.14
|
NA
|
Site F |
0.14
|
NA
|
Site G |
-0.22
|
NA
|
Site H |
NA
|
0.77
|
Site I |
NA
|
0.61
|
Site J |
NA
|
-0.26
|
Site K |
NA
|
0.05
|
Site L |
NA
|
0.24
|
Adverse Events
Time Frame | ||||
---|---|---|---|---|
Adverse Event Reporting Description | No adverse events were collected. | |||
Arm/Group Title | Organizational Model | Data Feedback | ||
Arm/Group Description | This arm is considered the intervention arm. Participants in this arm of the study will receive information regarding the organizational model and work closely with the research team throughout the project to implement various aspects of the model. Organization Model: The organizational model contains three components: leadership support, a multidisciplinary redesign team, and management structures and processes to link the two. Data Feedback Model: This is considered the non-intervention arm. The research team will periodically interview the facilities and provide them with reported hand hygiene data. | This arm is considered the control arm. Participants in this arm will be interviewed periodically and participate in the data feedback portion of the study but will not undergo any active intervention pertaining to the organizational model. Data Feedback Model: This is considered the non-intervention arm. The research team will periodically interview the facilities and provide them with reported hand hygiene data. | ||
All Cause Mortality |
||||
Organizational Model | Data Feedback | |||
Affected / at Risk (%) | # Events | Affected / at Risk (%) | # Events | |
Total | / (NaN) | / (NaN) | ||
Serious Adverse Events |
||||
Organizational Model | Data Feedback | |||
Affected / at Risk (%) | # Events | Affected / at Risk (%) | # Events | |
Total | 0/0 (NaN) | 0/0 (NaN) | ||
Other (Not Including Serious) Adverse Events |
||||
Organizational Model | Data Feedback | |||
Affected / at Risk (%) | # Events | Affected / at Risk (%) | # Events | |
Total | 0/0 (NaN) | 0/0 (NaN) |
Limitations/Caveats
More Information
Certain Agreements
All Principal Investigators ARE employed by the organization sponsoring the study.
There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.
Results Point of Contact
Name/Title | Dr. Carol VanDeusen Lukas |
---|---|
Organization | Department of Veterans Affairs |
Phone | 857-364-5685 |
carol.vandeusenlukas@va.gov |
- IMV 04-055