A Technology-Enhanced Approach for Implementing Evidence-Based Practices in Child Welfare
Study Details
Study Description
Brief Summary
In this study, a computer-assisted adaptation of SafeCare, an evidence-based home visiting program, will be developed and tested in child welfare. This approach will assist home visitors with the delivery of SafeCare to families, with the goal of improving provider fidelity and implementation success. In the computer-assisted SafeCare sessions, a portion of the session will be delivered on a tablet computer that the home visitor brings to the family's home. The computer software will deliver the educational piece, or explain a piece of each session, along with modeling videos of the skills. The home visitor will follow up with the practice and feedback portions of the session. A randomized feasibility trial will be conducted to test the technology-based approach against the standard SafeCare implementation approach in terms of feasibility of implementation, provider job demands, and resources, as well as client outcomes such as skill acquisition and mental health.
| Condition or Disease | Intervention/Treatment | Phase |
|---|---|---|
|
N/A |
Detailed Description
SafeCare contains three, six-session modules that target risk factors for child physical abuse and neglect, including (1) child health, (2) home safety, and (3) parent-child interaction. The focus of the child health module is to train parents to make effective decisions related to child health, to prevent child illness and injury, and assist parents in seeking appropriate health treatment for children when an illness or injury occurs. Parents are also taught to use health reference materials, including a validated SafeCare health manual. Home safety focuses on making the home a safe place where the risk for child injury is reduced. Parents identify and eliminate common household hazards and learn the appropriate levels of supervision for children across developmental stages. The parent- child interaction module focuses on the parent-child relationship, with the goal of improving interactions between the child and parent as well as child behavior by teaching parents structured skills to use in a consistent and predictable way with children across daily routine activities. Each of the three SafeCare modules is structured according to an assess-train-assess approach, with six sessions each. Session 1 is a pretest of baseline skills and knowledge of the particular module. Sessions 2-5 are training sessions, which include explanation of target skills, modeling of target skills by the provider, practice of the target skills by the parent, and feedback from the provider about the parent's mastery and competence in skills. Session 6 is a second assessment session, which allows the provider to examine parent mastery of target skills across each module. Providers train the parents in each module to a level of mastery, which is the demonstration of at least 80% of target skills at the Session 6 assessment.
Conditions:
SafeCare-Implementation as Usual (SC-IU): The SC-IU condition followed the protocol used in standard SafeCare implementation, which includes the following phases: (1) Agency readiness - NSTRC reviews the requisite organizational communication, service system and SafeCare funding plan, implementation requirements, and budgetary information with interested agencies to ensure fit of SafeCare within the organizational context of the agency, (2) SafeCare workshop training -This includes a 4-day classroom-based training that involves didactic presentations, modeling of the instruction, and role-playing for SafeCare trainees as well as structured assessments of the skills taught, (3) SafeCare provider certification process -Providers receive support from NSTRC as they begin SafeCare delivery with families. The support includes the SafeCare trainers listening to audio recordings of the providers' SafeCare sessions, assessing these sessions for fidelity, and then providing a follow-up coaching session to provide feedback to the provider. Once trainees demonstrate mastery of SafeCare, defined as meeting session fidelity of _ 85% , in three sessions per module (nine total), they are certified as a SafeCare provider, (4) SafeCare postcertification support and sustainability - Once a provider achieves certification, the provider moves into this phase that consists of monthly submissions of one SafeCare session recording monitored for fidelity by an NSTRC trainer or trained SafeCare coach at the implementing agency.
SafeCare-Tech-Assisted (SC-TA). This implementation followed the four phases described above, but the delivery of SafeCare was adapted to include technology assistance delivered on a tablet via a web-based SafeCare program entitled SafeCare Takes Care. SafeCare Takes Care includes a combination of video, audio narration, and engaging questions and was developed through an alpha and beta testing process with parenting experts and parents similar in education and socioeconomic status to the parents served with the SafeCare program. The videos are presented in a manner similar to a talk show. For each module and session, the host of SafeCare Takes Care presents a new topic (i.e., the session content for that day) with video modeling of the skills from "at-home viewers." For example, in the parent-child interaction module, a video begins with the talk show host explaining the skills being covered in the session, followed by a video of a parent (i.e., an at-home viewer) modeling these skills, and the host may take some questions from studio audience members or from fans on the "street cam." SafeCare Takes Care uses an open-source systems and languages to input text, picture, and video-related content into website interventions. All text was narrated to minimize literacy requirements. The architecture is based on the Python programming language using a Django web framework and Foundation an advanced responsive front-end framework, to ensure mobile friendliness. This framework consists of Cascading Style Sheets and Javascript to ensure proper display of the web application across multiple devices with differing screen sizes and resolutions. SafeCare Takes Care was hosted at Oregon Research Institute (ORI) on a Linux server with MySQL, an open-source language for relational database development. Data collection components were securely transmitted to ORI servers using Secure Sockets Layer protocol. This platform has been successfully used for the delivery of other evidence-based parenting programs to highrisk parents (see Baggett et al., 2010). SafeCare providers assigned to the SC-TA condition participated in the standard SafeCare workshop and also received training in the technology-mediated approach to SafeCare delivery. The technology training took approximately 2 hr and focused on how the provider utilizes the technology in each session. Specifically, after greeting the parent, the provider was instructed to connect the parent to the web-based program, during which the parent participates in the multimodal learning (e.g., explanation and modeling of skills) of SafeCare target skills. When the parent completes the web-directed portion of the session, the provider is prompted by the web-based program to take over the session delivery, revisit any explanation and modeling the parent has questions about, and then engage the parent in live practice of the skills presented in the web program. Lastly, the provider offers positive and constructive feedback about the practice and closes the SafeCare session. In addition to the technology-mediated delivery, there were some slight adaptations to the scoring instructions of the Safe- Care Fidelity checklist for the SC-TA to accommodate the use of the web-based program into the session. Specifically, fidelity items pertaining to explanation and modeling were scored as completed by a coach if it was clear in the audio recording of the session that the provider connected the parentparticipant to the web-based program. All other fidelity items on the SafeCare Fidelity Checklist remained the same. That is, SafeCare providers were fully expected to deliver the session opening, SafeCare target skills practice and feedback, and session closing. Additionally, if parents had questions about the explanation or modeling components reviewed in the webbased program, providers were trained to address these concerns, and fidelity was rated as it would be in standard implementation for these items. Lastly, SafeCare coaches were instructed to include the providers' technology equipment under their scoring of "has materials ready" on the fidelity checklist. Providers in both groups participated in coaching calls with their assigned SafeCare coach following the coach's scoring of fidelity, as is the protocol for SC-IU. These calls serve as an opportunity for the coach to provide positive and constructive feedback to the provider regarding their session delivery.
Provider demographics and professional background factors were measured by a form developed for the project and asked questions regarding provider age, education, race/ethnicity, and field experience. SafeCare delivery time demands: Time diaries were completed by providers who were delivering SafeCare to families. Providers were instructed to complete a time diary following each Safe- Care session and submit them to the research team on a monthly basis. Information reported on the time diary form included the specific amount of time in minutes spent on SafeCare-related activities prior to, during, and following each session. These activities were determined by the research team in consultation with SafeCare trainers who are familiar with the common activities conducted by SafeCare providers. Provider fidelity: Fidelity was measured utilizing the SafeCare Provider Fidelity Checklist. The checklist includes a number of concrete behaviors providers should perform during the SafeCare session. Providers submit audio recordings to NSTRC and expert coders rate fidelity using this checklist based on the verbal behaviors performed by the provider in the SafeCare session. Provider implementation progress: SafeCare provider implementation progress was documented by research team members based on the implementation record review. Records were maintained by the NSTRC trainer who noted the progress of each provider through the training and certification process. To meet certification, the provider has to achieve 85% fidelity on three sessions in each SafeCare module (parent-child interaction, child health, and child safety), for a total of nine sessions. Provider implementation progress was coded as "workshop only" if the provider completed the training workshop but did not begin working with families. Providers were coded as "Began certification, SafeCare inactive," if they began certification after workshop training but discontinued SafeCare delivery before reaching certification. Providers coded as "Began certification, SafeCare active" were still delivering SafeCare services at the end of the study period but had not yet achieved certification. Lastly, providers coded as SafeCare certified achieved at least 85% fidelity on nine SafeCare sessions during the study period. SC-TA qualitative interview: A semi-structured qualitative interview was conducted upon completion of the study to gather feedback on satisfaction and recommendations for SC-TA and learn how the SC-TA implementation delivery approach compared to SC-IU.
Data Analysis Plan: Quantitative and qualitative data were analyzed using a convergence mixed-methods approach. Quantitative data were analyzed using Fisher's exact tests, chi-square tests, and independent samples t tests. Data for qualitative analyses included transcripts of audio recorded semi-structured interviews among providers. Thematic analysis was used to analyze all transcripts by the principal investigator and two other members of the research team. Derived codes from these transcripts were compared for consistency and overlap. Codes were grouped into themes. Differences in coding were explored and discussed until 95% agreement was reached across all transcripts. Periodic checks were made for intercoder agreement. Interrater discrepancies in coding were reviewed and discussed until 100% consensus was reached.
Study Design
Arms and Interventions
| Arm | Intervention/Treatment |
|---|---|
| Experimental: Technology-Assisted SafeCare A modified version of SafeCare, using a tablet and online program to complete a portion of the session |
Behavioral: Technology-Assisted SafeCare
A technology-enhanced version of SafeCare
Other Names:
|
| Active Comparator: SafeCare-as-usual SafeCare as it is usually delivered |
Behavioral: SafeCare
SafeCare, an evidence-based home visiting program
|
Outcome Measures
Primary Outcome Measures
- Number of Providers Reporting Satisfaction With Technology [Approximately six months after completing the baseline assessment]
The investigators examined provider qualitative feedback to count the number of participants reporting that the Technology-Assisted SafeCare was acceptable and feasible. Interview data was transcribed and summarized to examine themes reported by the Tech-Assisted SafeCare providers' at the time of their exit interview (approximately six months after the baseline assessment).
- SafeCare Fidelity [The average of up to 9 scores across 6 months]
The SafeCare fidelity checklist measures 30+ clinical behaviors that SafeCare providers are expected to deliver during an hour long SafeCare session. Fidelity on each behavior is scored as a + if occurs and a - if does not occur. Percentages are computed for the number of +'s divided by the number of total behaviors. Scores range from 0 to 100%, with higher numbers indicating higher fidelity. A score of 85% or greater is indicative of high competence with SafeCare delivery. SafeCare providers receive fidelity monitoring for nine sessions (three in each of three modules) until they reach certification and once per month thereafter. All fidelity scores for the participants will be collected from baseline assessment to six-month follow-up in order to assess the effect of implementation condition on fidelity scores. The frequency of fidelity assessment will vary depending on the caseload of each home visitor participant.
- SafeCare Provider Implementation Status [Status of provider at end of study participation]
The investigators used 4 categories to indicate the status of the SafeCare providers at the end of their study participation, these groups: completed were workshop only, started SafeCare but became inactive, started and continued SafeCare active (but not certified), SafeCare certified (completed 9 family sessions with 85% fidelity)
- SafeCare Time Diary [average of provider time preparing for and completing SafeCare sessions during the study period (6 months)]
Provider participants documented their time spent on SafeCare session preparation, completion, and follow-up using a time diary. This form was used to document the time home visitors spend on job-related activities such as preparing for home visits, conducting home visits, and completing notes, among other activities. All participants completed the form following each SafeCare session they complete during the six months of the study. The total number of time diaries completed will vary depending on the home visitor's caseload. Means were computed based on an average of the time diaries submitted per study arm and wer compared for differences.
Eligibility Criteria
Criteria
Inclusion Criteria:
-
Home Visitors: Home Visitors who are being trained in SafeCare
-
Parents: English-speaking SafeCare parents with a child between the ages of 1.5-5 years who are receiving services from a Home Visitor who is in this study
Exclusion Criteria:
-
Home Visitors: Home Visitors not being trained in SafeCare; Home Visitors who exclusively serve Spanish-speaking families
-
Parents: Parents not receiving SafeCare; parents whose Home Visitor is not a participant in this study; parents under age 18 or who have a child outside of the ages of 1.5-5 years
Contacts and Locations
Locations
| Site | City | State | Country | Postal Code | |
|---|---|---|---|---|---|
| 1 | Georgia State University, School of Public Health, National SafeCare Training & Research Center | Atlanta | Georgia | United States | 30303 |
Sponsors and Collaborators
- Georgia State University
Investigators
- Principal Investigator: Shannon Self-Brown, PhD, Georgia State University
Study Documents (Full-Text)
None provided.More Information
Publications
None provided.- 5R21MH098244-02
Study Results
Participant Flow
| Recruitment Details | |
|---|---|
| Pre-assignment Detail |
| Arm/Group Title | SafeCare- Intervention as Usual (SC-IU) | SafeCare Technology-Assisted (SC-TA) |
|---|---|---|
| Arm/Group Description | The SC-IU condition followed the protocol used in standard SafeCare implementation, which includes the following phases: (1) Agency readiness-NSTRC reviews the requisite organizational communication, service system and SafeCare funding plan, implementation requirements, and budgetary information with interested agencies to ensure fit of SafeCare within the organizational context of the agency, (2) SafeCare workshop training-This includes a 4-day classroom-based training that involves didactic presentations, modeling of the instruction, and role-playing for SafeCare trainees as well as structured assessments of the skills taught, (3) SafeCare provider certification process-Providers receive support from NSTRC as they begin SafeCare delivery with families. The support includes the SafeCare trainers listening to audio recordings of the providers' SafeCare sessions, assessing these sessions for fidelity, and then providing a follow-up coaching session to provide feedback to | SafeCare providers assigned to the SC-TA condition participated in the standard SafeCare workshop and also received training in the technology-mediated approach to SafeCare delivery. The technology training took approximately 2 hr and focused on how the provider utilizes the technology in each session. Specifically, after greeting the parent, the provider was instructed to connect the parent to the web-based program, during which the parent participates in the multimodal learning (e.g., explanation and modeling of skills) of SafeCare target skills. When the parent completes the web-directed portion of the session, the provider is prompted by the web-based program to take over the session delivery, revisit any explanation and modeling the parent has questions about, and then engage the parent in live practice of the skills presented in the web program. Lastly, the provider offers positive and constructive feedback about the practice and closes the SafeCare session. |
| Period Title: Overall Study | ||
| STARTED | 11 | 20 |
| COMPLETED | 11 | 20 |
| NOT COMPLETED | 0 | 0 |
Baseline Characteristics
| Arm/Group Title | SafeCare-as-usual | Technology-Assisted SafeCare | Total |
|---|---|---|---|
| Arm/Group Description | SafeCare as it is usually delivered SafeCare: SafeCare, an evidence-based home visiting program | A modified version of SafeCare, using a tablet and online program to complete a portion of the session Technology-Assisted SafeCare: A technology-enhanced version of SafeCare | Total of all reporting groups |
| Overall Participants | 11 | 20 | 31 |
| Age (Count of Participants) | |||
| <=18 years |
0
0%
|
0
0%
|
0
0%
|
| Between 18 and 65 years |
11
100%
|
20
100%
|
31
100%
|
| >=65 years |
0
0%
|
0
0%
|
0
0%
|
| Sex: Female, Male (Count of Participants) | |||
| Female |
8
72.7%
|
19
95%
|
27
87.1%
|
| Male |
3
27.3%
|
1
5%
|
4
12.9%
|
| Ethnicity (NIH/OMB) (Count of Participants) | |||
| Hispanic or Latino |
5
45.5%
|
4
20%
|
9
29%
|
| Not Hispanic or Latino |
6
54.5%
|
16
80%
|
22
71%
|
| Unknown or Not Reported |
0
0%
|
0
0%
|
0
0%
|
| Race (NIH/OMB) (Count of Participants) | |||
| American Indian or Alaska Native |
2
18.2%
|
2
10%
|
4
12.9%
|
| Asian |
0
0%
|
0
0%
|
0
0%
|
| Native Hawaiian or Other Pacific Islander |
0
0%
|
0
0%
|
0
0%
|
| Black or African American |
2
18.2%
|
5
25%
|
7
22.6%
|
| White |
4
36.4%
|
10
50%
|
14
45.2%
|
| More than one race |
1
9.1%
|
1
5%
|
2
6.5%
|
| Unknown or Not Reported |
2
18.2%
|
2
10%
|
4
12.9%
|
| Region of Enrollment (participants) [Number] | |||
| United States |
11
100%
|
20
100%
|
31
100%
|
Outcome Measures
| Title | Number of Providers Reporting Satisfaction With Technology |
|---|---|
| Description | The investigators examined provider qualitative feedback to count the number of participants reporting that the Technology-Assisted SafeCare was acceptable and feasible. Interview data was transcribed and summarized to examine themes reported by the Tech-Assisted SafeCare providers' at the time of their exit interview (approximately six months after the baseline assessment). |
| Time Frame | Approximately six months after completing the baseline assessment |
Outcome Measure Data
| Analysis Population Description |
|---|
| Qualitative data were summarized for 14 of the 20 SafeCare Technology-Assisted Providers, who completed an exit interview regarding the technology usability |
| Arm/Group Title | Technology-Assisted SafeCare |
|---|---|
| Arm/Group Description | A modified version of SafeCare, using a tablet and online program to complete a portion of the session Technology-Assisted SafeCare: A technology-enhanced version of SafeCare |
| Measure Participants | 14 |
| Count of Participants [Participants] |
14
127.3%
|
| Title | SafeCare Fidelity |
|---|---|
| Description | The SafeCare fidelity checklist measures 30+ clinical behaviors that SafeCare providers are expected to deliver during an hour long SafeCare session. Fidelity on each behavior is scored as a + if occurs and a - if does not occur. Percentages are computed for the number of +'s divided by the number of total behaviors. Scores range from 0 to 100%, with higher numbers indicating higher fidelity. A score of 85% or greater is indicative of high competence with SafeCare delivery. SafeCare providers receive fidelity monitoring for nine sessions (three in each of three modules) until they reach certification and once per month thereafter. All fidelity scores for the participants will be collected from baseline assessment to six-month follow-up in order to assess the effect of implementation condition on fidelity scores. The frequency of fidelity assessment will vary depending on the caseload of each home visitor participant. |
| Time Frame | The average of up to 9 scores across 6 months |
Outcome Measure Data
| Analysis Population Description |
|---|
| [Not Specified] |
| Arm/Group Title | Technology-Assisted SafeCare | SafeCare-as-usual |
|---|---|---|
| Arm/Group Description | A modified version of SafeCare, using a tablet and online program to complete a portion of the session Technology-Assisted SafeCare: A technology-enhanced version of SafeCare | SafeCare as it is usually delivered SafeCare: SafeCare, an evidence-based home visiting program |
| Measure Participants | 20 | 11 |
| Mean (Standard Deviation) [percentage of behaviors implemented] |
91.94
(5.98)
|
91.14
(8.81)
|
| Title | SafeCare Provider Implementation Status |
|---|---|
| Description | The investigators used 4 categories to indicate the status of the SafeCare providers at the end of their study participation, these groups: completed were workshop only, started SafeCare but became inactive, started and continued SafeCare active (but not certified), SafeCare certified (completed 9 family sessions with 85% fidelity) |
| Time Frame | Status of provider at end of study participation |
Outcome Measure Data
| Analysis Population Description |
|---|
| [Not Specified] |
| Arm/Group Title | SafeCare-as-usual | Technology-Assisted SafeCare |
|---|---|---|
| Arm/Group Description | SafeCare as it is usually delivered SafeCare: SafeCare, an evidence-based home visiting program | A modified version of SafeCare, using a tablet and online program to complete a portion of the session Technology-Assisted SafeCare: A technology-enhanced version of SafeCare |
| Measure Participants | 11 | 20 |
| Number [percentage of providers certified] |
36.8
|
36.4
|
| Title | SafeCare Time Diary |
|---|---|
| Description | Provider participants documented their time spent on SafeCare session preparation, completion, and follow-up using a time diary. This form was used to document the time home visitors spend on job-related activities such as preparing for home visits, conducting home visits, and completing notes, among other activities. All participants completed the form following each SafeCare session they complete during the six months of the study. The total number of time diaries completed will vary depending on the home visitor's caseload. Means were computed based on an average of the time diaries submitted per study arm and wer compared for differences. |
| Time Frame | average of provider time preparing for and completing SafeCare sessions during the study period (6 months) |
Outcome Measure Data
| Analysis Population Description |
|---|
| [Not Specified] |
| Arm/Group Title | SafeCare-as-usual | Technology-Assisted SafeCare |
|---|---|---|
| Arm/Group Description | SafeCare as it is usually delivered SafeCare: SafeCare, an evidence-based home visiting program | A modified version of SafeCare, using a tablet and online program to complete a portion of the session Technology-Assisted SafeCare: A technology-enhanced version of SafeCare |
| Measure Participants | 11 | 20 |
| Mean (Standard Deviation) [Minutes] |
14.66
(4.56)
|
13.23
(4.29)
|
Adverse Events
| Time Frame | 1 year, 8 months | |||
|---|---|---|---|---|
| Adverse Event Reporting Description | Participants were trained to disseminate a parenting intervention training. No participants died, were injured, or were at risk for serious adverse events as a result of this training. | |||
| Arm/Group Title | SafeCare-as-usual | Technology-Assisted SafeCare | ||
| Arm/Group Description | SafeCare as it is usually delivered SafeCare: SafeCare, an evidence-based home visiting program | A modified version of SafeCare, using a tablet and online program to complete a portion of the session Technology-Assisted SafeCare: A technology-enhanced version of SafeCare | ||
All Cause Mortality |
||||
| SafeCare-as-usual | Technology-Assisted SafeCare | |||
| Affected / at Risk (%) | # Events | Affected / at Risk (%) | # Events | |
| Total | 0/11 (0%) | 0/20 (0%) | ||
Serious Adverse Events |
||||
| SafeCare-as-usual | Technology-Assisted SafeCare | |||
| Affected / at Risk (%) | # Events | Affected / at Risk (%) | # Events | |
| Total | 0/11 (0%) | 0/20 (0%) | ||
Other (Not Including Serious) Adverse Events |
||||
| SafeCare-as-usual | Technology-Assisted SafeCare | |||
| Affected / at Risk (%) | # Events | Affected / at Risk (%) | # Events | |
| Total | 0/11 (0%) | 0/20 (0%) | ||
Limitations/Caveats
More Information
Certain Agreements
Principal Investigators are NOT employed by the organization sponsoring the study.
There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.
Results Point of Contact
| Name/Title | Shannon Self Brown, PhD |
|---|---|
| Organization | Georgia State University |
| Phone | 404-413-1283 |
| sselfbrown@gsu.edu |
- 5R21MH098244-02