VillageWhere: Using Mobile Technology to Enhance Multisystemic Therapy (MST) Outcomes

• Sponsor: Evidence-Based Practice Institute, Seattle, WA (Industry)
• Overall Status: Completed
• CT.gov ID: NCT02320903
• Collaborator: National Institute of Mental Health (NIMH) (NIH), Medical University of South Carolina (Other)
• Enrollment: 88
• Location: 1
• Arm: 1
• Duration (Months): 23
• Patients Per Site Per Month: 3.8

Study Details

Study Description

Brief Summary

The goal of the current trial is to assess the feasibility of a newly developed prototype of a cell phone app called VillageWhere. VillageWhere is designed to improve clinical outcomes of youth with externalizing behavioral problems by improving parental monitoring of youth whereabouts and activities while increasing the delivery of rewards to teens for good behavior. This Phase I Small Business Innovation Research (SBIR) proposal seeks to develop and conduct initial feasibility testing of the VillageWhere prototype. The primary features of the prototype include location monitoring via smartphones' global positioning system (GPS) functionality as well as facilitating teen self-appraisal of meeting behavioral expectations. Good behavior is rewarded with points that are tracked by the app and can be used to redeem for real-world rewards. The investigators will conduct an 45-day open trial (N=20 caregiver/teen dyads) to assess use, feasibility and acceptability for teens involved in the juvenile justice system or otherwise struggling with externalizing behavioral problems. Caregivers and teens will be assessed at baseline, before receiving the app, and at the end of the app use period (45 days). The therapist of each caregiver/teen dyad will also be assessed at the end of the app use period. Primary outcomes include: use of app features, monitoring of teen behavior and whereabouts, delivery of real-world rewards, and feasibility and acceptability. Secondary outcomes include: parenting behaviors, externalizing teen behaviors and internalizing teen behaviors.

Condition or Disease	Intervention/Treatment	Phase
Child Behavior Disorders	Device: VillageWhere App	N/A

Detailed Description

VillageWhere was pilot tested with a sample of 16 youth and their primary caregiver participating in an EST for a serious conduct problem (Mean age 14.6 years, 52% male, 56% African American, 33% were previously arrested). After completing baseline self-report measures, dyads used the app as desired for 4 weeks, then participated in a post-assessment and qualitative interviews. App usage by both parents and youth was high with little diminished use over the four weeks of the trial. All parents set and used reinforcers for functional behavior and used the "find my teen" feature; 84% set geolocation expectations for youth's whereabouts at specific times; and 88% set other expectations for the youth (e.g., finish homework), viewed notifications, and regularly used the "Find My Teen" feature. As expected, parents interacted with the app frequently: 2.4 times per day. Youth used the app on average four times per day. All youth earned and redeemed points for complying with their parent's expectations and routinely viewed their behavior plans (93.3%) and feedback notification messages (86.7%). Parents did not on average use coaching videos (20%).

Although the study was not powered to detect statistical significance, a series of paired t-tests were conducted to examine pre- to post-changes on self-report measures. Changes for parents and youth all were in predicted directions, with effect sizes ranging from small to high (average d = .42, range .21-.80). Parents reported greater use of positive parenting behaviors (i.e., rewards and praise), greater discipline consistency, lower overall perceived stress, a higher feeling of influence over youth behavior, and greater parenting efficacy. Youth reported increases in parent use of positive behaviors, consistency, and knowledge of whereabouts. Parent- and youth-reported change in youth aggression and rule-breaking (as measured by the CBCL/YSR).

Qualitative results were also positive. Using 7-point scales, parents and youth reported on app satisfaction three times/week. Parent ratings were high (M = 6.0) across items (e.g., "The app was helpful today," "I have ways to intervene with my teen"). Youth satisfaction scores were also high (M = 6.1; e.g., "My parent is noticing & rewarding my good behavior;" "I liked getting app prompts & reminders"). Parents reported that the app helped them trust their youth, increase their use of rewards, and made giving rewards easier; they also reported a reduction in youths' "excuse-making." Youth perceived that the app allowed them more freedom, because parents had an objective way of verifying their whereabouts. Nearly all youth (89%) liked that their parent was more consistent in paying attention to their positive behaviors and rewarding them for it (versus only being punitive). Both parents and youth described significant improvements in their relationship.

In addition, 16 therapists working with the parent-youth dyads also gave qualitative feedback that was highly positive. Therapists stated that they liked the extra accountability the app afforded to teens and the reduction in parent-youth communication issues.

Study Design

• Study Type: Interventional
• Actual Enrollment: 88 participants
• Allocation: N/A
• Intervention Model: Single Group Assignment
• Masking: None (Open Label)
• Primary Purpose: Treatment
• Official Title: Phase I - Using Mobile Technology to Enhance MST Outcomes (VillageWhere)
• Study Start Date: Apr 1, 2014
• Actual Primary Completion Date: Mar 1, 2016
• Actual Study Completion Date: Mar 1, 2016

Arms and Interventions

Arm	Intervention/Treatment
Experimental: Use of VillageWhere App Prototype In this single-arm study design, all enrolled caregivers and teens will use the VillageWhere App Prototype that has been developed for this study. They are requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day.	Device: VillageWhere App

Outcome Measures

Primary Outcome Measures

1. App Usage - Parent Set up / Modified an App-based Behavior Plan for the Teen [up to 45-days]

   Percent of parents who set up a behavior plan for teen using app features.

2. App Usage - Set up and Allocated / Earned Points (Rewards) for Positive Teen Behavior [up to 45-days]

   How many parents/caregivers set up a reward system and redeemed at least some points for the teen.

3. App Usage - Parent Set Geolocation Expectations for the Teen. [up to 45-days]

   Percentage of parent/caregivers who set either required or off-limits locations for their teens.

4. App Usage - Viewed Notifications Regarding Behavioral Expectations Met/Not Met [up to 45 days]

   The percentage of participants who viewed notifications that the teen was or was not meeting a behavioral expectation. Both parents and teens had this capability.

5. App Usage - Viewed Parent Coaching Videos [up to 45 days]

   The app included 5 brief video vignettes depicting an effective parenting technique.

6. App Usage - View Progress Graph (Points Earned Over Time) [up to 45 days]

   Youth/teen points earned over time were depicted on a graph for each day of use. To view the graph, caregivers/parents and youth/teens had to click on an app button.

7. App Satisfaction Rating - the App Was Helpful Today [up to 45 days]

   Parent/caregiver was prompted randomly 3 times each week to rate on a scale of 1 (strongly disagree) to 7 (completely agree), how helpful was the app to you today?

8. App Satisfaction Rating - I Have Ways to Intervene With my Teen. [up to 45 days]

   Parent/caregiver was prompted randomly 3 times each week to rate on a scale of 1 (strongly disagree) to 7 (completely agree), the app gives me new ways of intervening with my teen.

9. App Satisfaction Rating - Overall Satisfaction [up to 45 days]

   Parent/caregiver and teen was prompted randomly 3 times each week to rate on a scale of 1 (totally unsatisfied) to 7 (completely satisfied), overall, how satisfied are you with the app today?

10. App Satisfaction Rating - Parent is Noticing and Rewarding Good Behavior [up to 45 days]

    Three times each week, at random, youth/teens were asked to rate on a scale of 1 (strongly disagree) to 7 (strongly agree), my parent is noticing and rewarding good behavior. All ratings were averaged across full period of app use.

11. App Satisfaction - I Like Getting Prompts and Reminders [up to 45 days]

    Youth/teens in the study were prompted 3 times/week during period of app usage to rate, on a scale of 1 (strongly disagree) to 7 (strongly agree), "I like getting prompts and reminders from the app."

Secondary Outcome Measures

1. Achenbach System of Empirically-Based Assessment, Aggression Subscale [Baseline and Post assessment (end of 45-day app use period)]

   The Achenbach System of Empirically-Based Assessment is a measure of youth problem behavior and psychiatric symptomatology that has parallel parent (report of teen) and youth (self-report) versions to allow for different perspectives on the problem. The parent version is called the Child Behavior Checklist (CBCL) and the youth version is called the Youth Self Report (YSR). Selected subscales from the CBCL and the YSR were administered before and after the 4 weeks of app usage, in this case, the aggression subscale. Higher values indicate worse symptoms. Possible range on this subcale is 0 - 36; a score of 13 or higher indicates clinically significant aggression, defined as aggression that is higher than that of 93% of teens in the United States.

2. Achenbach System of Empirically-Based Assessment, Depression Subscale [Baseline and Post assessment (end of 45-day app use period)]

   The Achenbach System of Empirically-Based Assessment is a measure of youth problem behavior and psychiatric symptomatology that has parallel parent (report of teen) and youth (self-report) versions to allow for different perspectives on the problem. The parent version is called the Child Behavior Checklist (CBCL) and the youth version is called the Youth Self Report (YSR). Selected subscales from the CBCL and the YSR were administered before and after the 4 weeks of app usage, in this case, the depression subscale. Higher values indicate worse symptoms. Possible range on this subcale is 0 - 26; a score of 8 or higher indicates clinically significant depression, defined as depressive symptoms higher than that of 93% of teens in the United States.

3. Achenbach System of Empirically-Based Assessment, Rule-breaking Subscale [Baseline and Post assessment (end of 45-day app use period)]

   The Achenbach System of Empirically-Based Assessment is a measure of youth problem behavior and psychiatric symptomatology that has parallel parent (report of teen) and youth (self-report) versions to allow for different perspectives on the problem. The parent version is called the Child Behavior Checklist (CBCL) and the youth version is called the Youth Self Report (YSR). Selected subscales from the CBCL and the YSR were administered before and after the 4 weeks of app usage, in this case, the rule-breaking subscale. Higher values indicate worse symptoms. Possible range on this subcale is 0 - 34; a score of 9 or higher indicates clinically significant rule-breaking, defined as rule-breaking that is higher than that of 93% of teens in the United States.

4. Loeber Parental Supervision Scale [Baseline and Post assessment (end of 45-day app use period)]

   This measure of extent to which parents supervise and monitor youth whereabouts was administered to parents/caregivers and to youth/teens before and after the 4 weeks of app usage.This 8-item scale has a scored range of 0 (minimum score) - 16 (maximum score), with high scores indicating more or better parental supervision/monitoring.

5. Loeber Positive Parenting Scale [Baseline and Post assessment (end of 45-day app use period)]

   This measure of extent to which parents express warmth and encouragement to youth was administered to parents/caregivers and to youth/teens before and after the 4 weeks of app usage.This 9-item scale has a scored range of 0 (minimum score) - 18 (maximum score), with high scores indicating more parental positivity.

6. Loeber Parental Consistency Scale [Baseline and Post assessment (end of 45-day app use period)]

   This measure of extent to which parents are consistent in their rules, expectations, and discipline techniques with youth was administered to parents/caregivers and to youth/teens before and after the 4 weeks of app usage. This 5-item scale has a scored range of 0 (minimum score) - 10 (maximum score), with high scores indicating more consistent / better parental discipline follow-through.

7. Loeber Parental Effectiveness Scale [Change over 30 days of app usage (baseline to post-assessment)]

   Degree to which parents feel that their parenting techniques are working in terms of managing teen behavior challenges. This 3-item scale has a scored range of 0 (minimum score) - 8 (maximum score), with high scores indicating better discipline effectiveness (e.g., "I feel like the punishment I give works in curbing youth behavior").

8. Perceived Stress Scale (PSS) [Change over 30 days of app usage (baseline to post-assessment)]

   Degree to which parents are experiencing stress related to parenting. This 10-item scale has a scored range of 0 (minimum score) - 40 (maximum score), with high scores indicating more stress related to parenting.

9. Parental Locus of Control Scale [Change over 30 days of app usage (baseline to post-assessment)]

   Degree to which parents feel efficacious and confident in managing parenting challenges. This 20-item scale has a scored range of 0 (minimum score) - 40 (maximum score), with high scores indicating that the parent has a higher sense of feeling efficacious in their role as a parent.

10. Loeber Parental Knowledge Scale [over 30 days of app usage]

    Youth report of how much parents actually know about the youth's whereabouts/activities. This 5-item scale has a scored range of 0 (minimum score) - 15 (maximum score), with high scores indicating that the parent knows more about what teens are actually doing.

Eligibility Criteria

Criteria

• Ages Eligible for Study: 13 Years and Older
• Sexes Eligible for Study: All
• Accepts Healthy Volunteers: Yes

Inclusion Criteria:

• Teen: English speaking, between ages of 13 and 17, juvenile justice involvement and/or externalizing behavioral problems, currently in treatment and will remain in treatment for duration of trial

• Caregiver: English speaking, owns Android smart phone, currently in treatment with their teen and will remain in treatment for duration of trial

Exclusion Criteria:

• N/A

Contacts and Locations

Locations

	Site	City	State	Country	Postal Code
1	Evidence-Based Practice Institute	Seattle	Washington	United States	98144

Sponsors and Collaborators

• Evidence-Based Practice Institute, Seattle, WA
• National Institute of Mental Health (NIMH)
• Medical University of South Carolina

Investigators

• Principal Investigator: Linda A. Dimeff, PhD, Evidence-Based Practice Institute
• Principal Investigator: Cindy M Schaeffer, PhD, University of Maryland, Baltimore

Study Documents (Full-Text)

• Study Protocol and Statistical Analysis Plan - Sep 12, 2014

More Information

Publications

None provided.

• Responsible Party: Evidence-Based Practice Institute, Seattle, WA
• ClinicalTrials.gov Identifier: NCT02320903
• Other Study ID Numbers: 1R43MH097349-01; 1R43MH097349-01
• First Posted: Dec 19, 2014
• Last Update Posted: Feb 14, 2018
• Last Verified: Nov 1, 2017

• Keywords provided by Evidence-Based Practice Institute, Seattle, WA: Criminal Justice; Juvenile Delinquency
• Additional relevant MeSH terms: Mental Disorders; Child Behavior Disorders

Study Results

Participant Flow

Recruitment Details	We completed a four-week within-subjects pre-post trial to determine the usability, acceptability, and feasibility. MST therapists in Seattle, New York City, Hartford, and Washington, DC referred families to the study. Parents contacted the study team to indicate their interest. Therapists of referred families also were enrolled in the study.
Pre-assignment Detail	Initial App Set Up: Clinicians were solely responsible for assisting families to set up their app; however, many clinicians were uncomfortable with this because they were not "tech savvy". Solution: Phase II will include a self-guided setup tool that will empower users (parents, youth) to complete the set up independently.

Arm/Group Title	Therapists	Parents/Caregivers	Youth/Teens
Arm/Group Description	In this study, all enrolled caregivers and teens will be referred to the study by their participating therapists. All caregiver/teen dyads will use the VillageWhere App Prototype that has been developed for this study. They are requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	In this study, all enrolled caregivers and teens will be referred to the study by their participating therapists. All caregiver/teen dyads will use the VillageWhere App Prototype that has been developed for this study. They are requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	In this study, all enrolled caregivers and teens will be referred to the study by their participating therapists. All caregiver/teen dyads will use the VillageWhere App Prototype that has been developed for this study. They are requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Period Title: Overall Study
STARTED	24	32	32
COMPLETED	16	16	16
NOT COMPLETED	8	16	16

Baseline Characteristics

Arm/Group Title	Therapists	Parents/Caregivers	Youth/Teens	Total
Arm/Group Description	In this study, all enrolled caregivers and teens will be referred to the study by their participating therapists. All caregiver/teen dyads will use the VillageWhere App Prototype that has been developed for this study. They are requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	In this study, all enrolled caregivers and teens will be referred to the study by their participating therapists. All caregiver/teen dyads will use the VillageWhere App Prototype that has been developed for this study. They are requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	In this study, all enrolled caregivers and teens will be referred to the study by their participating therapists. All caregiver/teen dyads will use the VillageWhere App Prototype that has been developed for this study. They are requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	Total of all reporting groups
Overall Participants	24	32	32	88
Age (Years) [Mean (Standard Deviation) ]
Age	41.7 (0)	14.6 (1.6)	24.15 (0)
Sex: Female, Male (Count of Participants)
Female	22 91.7%	15 46.9%	9 28.1%	46 52.3%
Male	2 8.3%	1 3.1%	7 21.9%	10 11.4%
Ethnicity (NIH/OMB) (Count of Participants)
Hispanic or Latino	2 8.3%	3 9.4%	3 9.4%	8 9.1%
Not Hispanic or Latino	20 83.3%	12 37.5%	11 34.4%	43 48.9%
Unknown or Not Reported	2 8.3%	17 53.1%	18 56.3%	37 42%
Race (NIH/OMB) (Count of Participants)
American Indian or Alaska Native	0 0%	0 0%	0 0%	0 0%
Asian	1 4.2%	0 0%	0 0%	1 1.1%
Native Hawaiian or Other Pacific Islander	0 0%	0 0%	0 0%	0 0%
Black or African American	1 4.2%	5 15.6%	5 15.6%	11 12.5%
White	5 20.8%	9 28.1%	8 25%	22 25%
More than one race	0 0%	1 3.1%	1 3.1%	2 2.3%
Unknown or Not Reported	17 70.8%	17 53.1%	18 56.3%	52 59.1%
Region of Enrollment (Count of Participants)
United States	24 100%	32 100%	32 100%	88 100%

Outcome Measures

1. Primary Outcome

Title	App Usage - Parent Set up / Modified an App-based Behavior Plan for the Teen
Description	Percent of parents who set up a behavior plan for teen using app features.
Time Frame	up to 45-days

Outcome Measure Data

Analysis Population Description

All parent/caregiver and teen participants analyzed.

Arm/Group Title	Parents/Caregivers
Arm/Group Description	All caregivers who used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16
Count of Participants [Participants]	16 66.7%

2. Primary Outcome

Title	App Usage - Set up and Allocated / Earned Points (Rewards) for Positive Teen Behavior
Description	How many parents/caregivers set up a reward system and redeemed at least some points for the teen.
Time Frame	up to 45-days

Outcome Measure Data

Analysis Population Description

Percent of caregivers who set and allocated points to the teen (teens who had points redeemed).

Arm/Group Title	Parents/Caregivers
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16
Count of Participants [Participants]	16 66.7%

3. Primary Outcome

Title	App Usage - Parent Set Geolocation Expectations for the Teen.
Description	Percentage of parent/caregivers who set either required or off-limits locations for their teens.
Time Frame	up to 45-days

Outcome Measure Data

Analysis Population Description

[Not Specified]

Arm/Group Title	Parents/Caregivers
Arm/Group Description	In this single-arm study design, all enrolled caregivers and teens will use the VillageWhere App Prototype that has been developed for this study. They are requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16
Count of Participants [Participants]	14 58.3%

4. Primary Outcome

Title	App Usage - Viewed Notifications Regarding Behavioral Expectations Met/Not Met
Description	The percentage of participants who viewed notifications that the teen was or was not meeting a behavioral expectation. Both parents and teens had this capability.
Time Frame	up to 45 days

Outcome Measure Data

Analysis Population Description

All parent/caregiver and teen participants analyzed

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Count of Participants [Participants]	14 58.3%	14 43.8%

5. Primary Outcome

Title	App Usage - Viewed Parent Coaching Videos
Description	The app included 5 brief video vignettes depicting an effective parenting technique.
Time Frame	up to 45 days

Outcome Measure Data

Analysis Population Description

Only the parent/caregiver version of the app had this feature.

Arm/Group Title	Parents/Caregivers Only
Arm/Group Description	Parent/caregiver users of the linked parent-youth phone app system.
Measure Participants	16
Count of Participants [Participants]	3 12.5%

6. Primary Outcome

Title	App Usage - View Progress Graph (Points Earned Over Time)
Description	Youth/teen points earned over time were depicted on a graph for each day of use. To view the graph, caregivers/parents and youth/teens had to click on an app button.
Time Frame	up to 45 days

Outcome Measure Data

Analysis Population Description

Analyses performed separately for parents and teens.

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Count of Participants [Participants]	10 41.7%	8 25%

7. Primary Outcome

Title	App Satisfaction Rating - the App Was Helpful Today
Description	Parent/caregiver was prompted randomly 3 times each week to rate on a scale of 1 (strongly disagree) to 7 (completely agree), how helpful was the app to you today?
Time Frame	up to 45 days

Outcome Measure Data

Analysis Population Description

Only parent/caregivers were administered this measure.

Arm/Group Title	Parents/Caregivers
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16
Mean (Standard Deviation) [units on a scale]	5.95 (.999)

8. Primary Outcome

Title	App Satisfaction Rating - I Have Ways to Intervene With my Teen.
Description	Parent/caregiver was prompted randomly 3 times each week to rate on a scale of 1 (strongly disagree) to 7 (completely agree), the app gives me new ways of intervening with my teen.
Time Frame	up to 45 days

Outcome Measure Data

Analysis Population Description

Only parent/caregivers were given this question.

Arm/Group Title	Parents/Caregivers Only
Arm/Group Description	Parent/caregiver users of the linked parent-youth phone app system.
Measure Participants	16
Mean (Standard Deviation) [units on a scale]	6.00 (1.09)

9. Primary Outcome

Title	App Satisfaction Rating - Overall Satisfaction
Description	Parent/caregiver and teen was prompted randomly 3 times each week to rate on a scale of 1 (totally unsatisfied) to 7 (completely satisfied), overall, how satisfied are you with the app today?
Time Frame	up to 45 days

Outcome Measure Data

Analysis Population Description

Parent/caregiver and teen participants were analyzed separately on this measure.

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Mean (Standard Deviation) [units on a scale]	5.91 (1.21)	6.14 (0.72)

10. Primary Outcome

Title	App Satisfaction Rating - Parent is Noticing and Rewarding Good Behavior
Description	Three times each week, at random, youth/teens were asked to rate on a scale of 1 (strongly disagree) to 7 (strongly agree), my parent is noticing and rewarding good behavior. All ratings were averaged across full period of app use.
Time Frame	up to 45 days

Outcome Measure Data

Analysis Population Description

Only youth/teens were provided this question.

Arm/Group Title	Youth/Teens Only
Arm/Group Description	Youth/teens who used the app.
Measure Participants	16
Mean (Standard Deviation) [units on a scale]	6.19 (1.15)

11. Primary Outcome

Title	App Satisfaction - I Like Getting Prompts and Reminders
Description	Youth/teens in the study were prompted 3 times/week during period of app usage to rate, on a scale of 1 (strongly disagree) to 7 (strongly agree), "I like getting prompts and reminders from the app."
Time Frame	up to 45 days

Outcome Measure Data

Analysis Population Description

Only teens were provided with this question.

Arm/Group Title	Youth/Teens Only
Arm/Group Description	Youth/teens who used the app.
Measure Participants	16
Mean (Standard Deviation) [units on a scale]	5.62 (1.27)

12. Secondary Outcome

Title	Achenbach System of Empirically-Based Assessment, Aggression Subscale
Description	The Achenbach System of Empirically-Based Assessment is a measure of youth problem behavior and psychiatric symptomatology that has parallel parent (report of teen) and youth (self-report) versions to allow for different perspectives on the problem. The parent version is called the Child Behavior Checklist (CBCL) and the youth version is called the Youth Self Report (YSR). Selected subscales from the CBCL and the YSR were administered before and after the 4 weeks of app usage, in this case, the aggression subscale. Higher values indicate worse symptoms. Possible range on this subcale is 0 - 36; a score of 13 or higher indicates clinically significant aggression, defined as aggression that is higher than that of 93% of teens in the United States.
Time Frame	Baseline and Post assessment (end of 45-day app use period)

Outcome Measure Data

Analysis Population Description

Parent/caregivers and youth/teens results were analyzed separately.

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Mean at Time 1 (baseline)	13.75 (8.80)	11.29 (4.82)
Mean at Time 2 (post-usage)	12.31 (8.80)	10.14 (5.43)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was 0.16.

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Youth/Teens
	Comments	Paired sample T tests were used to measure change over time for teens.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for youth report was d = 0.22.

13. Secondary Outcome

Title	Achenbach System of Empirically-Based Assessment, Depression Subscale
Description	The Achenbach System of Empirically-Based Assessment is a measure of youth problem behavior and psychiatric symptomatology that has parallel parent (report of teen) and youth (self-report) versions to allow for different perspectives on the problem. The parent version is called the Child Behavior Checklist (CBCL) and the youth version is called the Youth Self Report (YSR). Selected subscales from the CBCL and the YSR were administered before and after the 4 weeks of app usage, in this case, the depression subscale. Higher values indicate worse symptoms. Possible range on this subcale is 0 - 26; a score of 8 or higher indicates clinically significant depression, defined as depressive symptoms higher than that of 93% of teens in the United States.
Time Frame	Baseline and Post assessment (end of 45-day app use period)

Outcome Measure Data

Analysis Population Description

Parent/caregivers and youth/teens results were analyzed separately. Least squares means show amount of difference from pre- to post-testing in paired samples T tests.

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Pre-test mean	6.93 (5.36)	6.29 (4.81)
Post-test mean	5.80 (5.17)	4.71 (3.91)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was d = 0.22.

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Youth/Teens
	Comments	Paired sample T tests were used to measure change for teens over time.
	Type of Statistical Test	Other
	Comments	What was the within-group change over time effect size?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for teen report on this measure was d = 0.22.

14. Secondary Outcome

Title	Achenbach System of Empirically-Based Assessment, Rule-breaking Subscale
Description	The Achenbach System of Empirically-Based Assessment is a measure of youth problem behavior and psychiatric symptomatology that has parallel parent (report of teen) and youth (self-report) versions to allow for different perspectives on the problem. The parent version is called the Child Behavior Checklist (CBCL) and the youth version is called the Youth Self Report (YSR). Selected subscales from the CBCL and the YSR were administered before and after the 4 weeks of app usage, in this case, the rule-breaking subscale. Higher values indicate worse symptoms. Possible range on this subcale is 0 - 34; a score of 9 or higher indicates clinically significant rule-breaking, defined as rule-breaking that is higher than that of 93% of teens in the United States.
Time Frame	Baseline and Post assessment (end of 45-day app use period)

Outcome Measure Data

Analysis Population Description

Parent/caregivers and youth/teens results were analyzed separately.

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Mean at Time 1 (baseline)	9.71 (6.34)	10.07 (4.20)
Mean at Time 2 (post-usage)	9.07 (6.49)	9.21 (4.61)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was d = 0.10.

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Youth/Teens
	Comments	Paired samples T tests were used to measure change over time for teens.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size? (Cohen's d)
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for youth report was d = 0.28.

15. Secondary Outcome

Title	Loeber Parental Supervision Scale
Description	This measure of extent to which parents supervise and monitor youth whereabouts was administered to parents/caregivers and to youth/teens before and after the 4 weeks of app usage.This 8-item scale has a scored range of 0 (minimum score) - 16 (maximum score), with high scores indicating more or better parental supervision/monitoring.
Time Frame	Baseline and Post assessment (end of 45-day app use period)

Outcome Measure Data

Analysis Population Description

Parent/caregivers and youth/teens results were analyzed separately.

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Mean at Time 1 (baseline)	13.00 (2.25)	10.86 (3.32)
Mean at Time 2 (post-usage)	13.38 (2.45)	11.50 (2.44)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was d = 0.16.

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Youth/Teens
	Comments	Paired sample T tests were used to measure change for teens over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size? (Cohen's d)
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for youth report was d = 0.21.

16. Secondary Outcome

Title	Loeber Positive Parenting Scale
Description	This measure of extent to which parents express warmth and encouragement to youth was administered to parents/caregivers and to youth/teens before and after the 4 weeks of app usage.This 9-item scale has a scored range of 0 (minimum score) - 18 (maximum score), with high scores indicating more parental positivity.
Time Frame	Baseline and Post assessment (end of 45-day app use period)

Outcome Measure Data

Analysis Population Description

Parent/caregivers and youth/teens results were analyzed separately.

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Mean at Time 1 (baseline)	8.87 (4.97)	8.57 (2.24)
Mean at Time 2 (post-usage)	11.39 (2.95)	8.93 (2.79)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was d = 0.62.

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Youth/Teens
	Comments	Paired sample T tests were used to measure change over time for teens.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for youth report was d = 0.23.

17. Secondary Outcome

Title	Loeber Parental Consistency Scale
Description	This measure of extent to which parents are consistent in their rules, expectations, and discipline techniques with youth was administered to parents/caregivers and to youth/teens before and after the 4 weeks of app usage. This 5-item scale has a scored range of 0 (minimum score) - 10 (maximum score), with high scores indicating more consistent / better parental discipline follow-through.
Time Frame	Baseline and Post assessment (end of 45-day app use period)

Outcome Measure Data

Analysis Population Description

Parent/caregivers and youth/teens results were analyzed separately.

Arm/Group Title	Parents/Caregivers	Youth/Teens
Arm/Group Description	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App	All caregiver/teen dyads used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. VillageWhere App
Measure Participants	16	16
Mean at Time 1 (baseline)	1.87 (1.64)	5.21 (2.01)
Mean at Time 2 (post-usage)	2.27 (2.02)	5.57 (2.87)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was d = 0.22.

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Youth/Teens
	Comments	Paired samples T tests were used to measure within-groups change over time for teens.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for youth report was d = 0.26.

18. Secondary Outcome

Title	Loeber Parental Effectiveness Scale
Description	Degree to which parents feel that their parenting techniques are working in terms of managing teen behavior challenges. This 3-item scale has a scored range of 0 (minimum score) - 8 (maximum score), with high scores indicating better discipline effectiveness (e.g., "I feel like the punishment I give works in curbing youth behavior").
Time Frame	Change over 30 days of app usage (baseline to post-assessment)

Outcome Measure Data

Analysis Population Description

All parents/caregivers who completed the study.

Arm/Group Title	Parents/Caregivers Only
Arm/Group Description	Parent/caregiver users of the linked parent-youth phone app system.
Measure Participants	16
Mean at Time 1 (baseline)	4.20 (1.90)
Mean at Time 2 (post-usage)	4.40 (1.88)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was d =.12.

19. Secondary Outcome

Title	Perceived Stress Scale (PSS)
Description	Degree to which parents are experiencing stress related to parenting. This 10-item scale has a scored range of 0 (minimum score) - 40 (maximum score), with high scores indicating more stress related to parenting.
Time Frame	Change over 30 days of app usage (baseline to post-assessment)

Outcome Measure Data

Analysis Population Description

All parents/caregivers who completed the study.

Arm/Group Title	Parents/Caregivers Only
Arm/Group Description	Parent/caregiver users of the linked parent-youth phone app system.
Measure Participants	16
Mean at Time 1 (baseline)	18.09 (8.14)
Mean at Time 2 (post-usage)	12.73 (4.45)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was d = 0.81.

20. Secondary Outcome

Title	Parental Locus of Control Scale
Description	Degree to which parents feel efficacious and confident in managing parenting challenges. This 20-item scale has a scored range of 0 (minimum score) - 40 (maximum score), with high scores indicating that the parent has a higher sense of feeling efficacious in their role as a parent.
Time Frame	Change over 30 days of app usage (baseline to post-assessment)

Outcome Measure Data

Analysis Population Description

All parents/caregivers who completed the study.

Arm/Group Title	Parents/Caregivers Only
Arm/Group Description	Parent/caregiver users of the linked parent-youth phone app system.
Measure Participants	16
Mean at Time 1 (baseline)	15.25 (6.22)
Mean at Time 2 (post-usage)	18.63 (8.72)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for parents over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for parent report on this measure was d = 0.43.

21. Secondary Outcome

Title	Loeber Parental Knowledge Scale
Description	Youth report of how much parents actually know about the youth's whereabouts/activities. This 5-item scale has a scored range of 0 (minimum score) - 15 (maximum score), with high scores indicating that the parent knows more about what teens are actually doing.
Time Frame	over 30 days of app usage

Outcome Measure Data

Analysis Population Description

Youth/teens who used app only

Arm/Group Title	Youth/Teens Only
Arm/Group Description	Youth/teens who used the app.
Measure Participants	16
Mean at Time 1 (baseline)	8.07 (2.73)
Mean at Time 2 (post-usage)	9.00 (3.09)

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Parents/Caregivers
	Comments	Paired sample T tests were used to measure change for teens over time.
	Type of Statistical Test	Equivalence
	Comments	What was the within-group change over time effect size (Cohen's d)?
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Other Statistical Analysis		Cohen's d for youth report on this measure was d = 24.

Adverse Events

Time Frame	Adverse events were informally monitored throughout the 4-week period during which time parents and teens were using the app. Also, most families had their baseline assessment sessions 1-2 weeks prior to the start of app use, and their post-assessment 1-2 weeks after finishing their use of the app. Thus, adverse events were informally monitored for the entire study period, for most families, approximately 8 weeks.
Adverse Event Reporting Description	Since the study was non-medical, participants were NOT at risk of a mortality event, i.e., the intervention involved use of a phone app only. Participants were also not at risk for a serious adverse event involving "a life-threatening event, hospitalization, or incapacity/disability." As a social science study, our participants may have had some risk of Other including Not Serious Adverse events, such as embarrassment or discomfort answering psychologically- and behaviorally-oriented questions.
Arm/Group Title	Use of VillageWhere App Prototype
Arm/Group Description	In this single-arm study design, all enrolled caregivers and teens used the VillageWhere App Prototype that was developed for this study. They were requested to use it as often as they would like throughout the duration of the trial. The app is designed to be used several times throughout each day. Thus, the risk was identical for both parents and teens, since both experienced the app as the only intervention (i.e., there was no comparison group who did not use the app).
All Cause Mortality
	Use of VillageWhere App Prototype
	Affected / at Risk (%)	# Events
Total	0/32 (0%)
Serious Adverse Events
	Use of VillageWhere App Prototype
	Affected / at Risk (%)	# Events
Total	0/32 (0%)
Other (Not Including Serious) Adverse Events
	Use of VillageWhere App Prototype
	Affected / at Risk (%)	# Events
Total	0/32 (0%)

Limitations/Caveats

[Not Specified]

More Information

Certain Agreements

Principal Investigators are NOT employed by the organization sponsoring the study.

There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.