TAP-P: Can a Novel Telemedicine Tool Reduce Disparities Related to the Identification of Preschool Children With Autism?
Study Details
Study Description
Brief Summary
Families seeking evaluation for autism spectrum disorder (ASD) often face barriers such as low availability of specialists, lengthy waitlists, and long distances to tertiary care diagnostic centers. This is especially true for children from traditionally underserved groups and communities. Without innovative approaches for enhanced identification of ASD, families and clinicians will continue to struggle with accessing and providing care. Telemedicine offers tremendous potential for addressing this need, but there are few psychometrically sound, validated tools that can be administered remotely, via telehealth platforms. This team of investigators developed and conducted a preliminary evaluation of a novel parent-administered, clinician-guided tele-diagnostic tool, the TAP (TELE-ASD-PEDS), designed specifically for direct-to-home and community clinic use with toddlers. Remote administration of the TAP yielded a very high level of agreement with blinded comprehensive evaluation regarding ASD risk classification. Subsequently, the unanticipated broad dissemination of the TAP during COVID-19 demonstrated its value for traditionally underserved groups, spanning broad geographies. Although promising, this work was limited by its specific focus on toddlers with ASD concerns. A telemedicine tool designed for the unique context and population of preschool-aged children referred for diagnostic assessment could have tremendous value in terms of both accurate identification as well as family engagement with service. In the current work, the investigators will now evaluate the performance, usability, and utility of the TAP-Preschool, a new telemedicine tool for ASD risk assessment in preschoolers, through a clinical trial. The TAP-Preschool was developed through a computationally informed co-production in which the targeted population were recruited as active partners in designing the tool. The investigators will gather critical data not only regarding its structure and accuracy, but also its potential deployment across systems responsible for engaging children and families from underserved groups in meaningful service. This work has potential to transform the ASD evaluation process and dramatically improve care access for traditionally underserved groups.
| Condition or Disease | Intervention/Treatment | Phase |
|---|---|---|
|
N/A |
Detailed Description
Innovative telemedicine practices could address many existing traditional barriers to ASD identification. Telemedicine has the potential to put linguistically and culturally competent clinical expert virtual providers directly into communities during a critical window in which many vulnerable families may approach, or re-approach, care systems about developmental concerns that become more prominent with age. In the current proposal, the investigators will evaluate the potential clinical and familial value of a telemedicine-based ASD assessment tool, the TAP-Preschool (TELE-ASD-PEDS-Preschool), designed to overcome traditional barriers to diagnosis and service access in underserved preschool populations that may not be readily identified or engaged by early screening and intervention systems.
A growing body of literature supports the use of telemedicine-based approaches to ASD assessment and intervention. This includes remote activities to assess infant social communication skills in the first year of life, coaching parents through administration of gold standard diagnostic tests, and provider coding and analysis of interactions that are video-recorded by caregivers. Results reflect high levels of caregiver and provider satisfaction and satisfactory agreement with traditional in-person evaluations, illustrating both the promise and feasibility of tele-assessment. However, most existing approaches are limited by a focus on screening rather than diagnosis, protocols requiring specific materials, or asynchronous analysis of submitted videos, which require resources that preclude broader use. Moreover, in current form, these tools are not designed or intended to yield a quantitative formal assessment of core ASD symptoms to support diagnostic decision-making. Further, although providers are increasingly exploring telemedicine approaches to ASD assessment in the context of the COVID-19 pandemic, there is limited published work to date on the use of real-time, caregiver-led ASD assessment in home settings with diverse populations. Prior work from the investigators' team has focused almost exclusively on toddlers in this regard. Ultimately, very few viable tools with validated psychometric properties are available for use in current models of telehealth assessment and care.
In earlier work, we applied machine learning models and the principles of feature engineering to a phenotypically rich clinical research data set of preschoolers with ASD and other developmental concerns (N=914: 594 ASD and 320 Non-ASD) to identify key behavioral targets for tool development. We then engaged in a rigorous adaptation and translation approach to optimize design of the TAP-Preschool. We included leading ASD assessment experts, providers dedicated to caring for underserved populations, and parents of preschool children with ASD from underserved racial/ethnic and linguistic groups. We lead these groups in a collaborative design process in order to yield a set of interactive and play-based, parent-coached tele-assessment activities designed to 1) elicit observations tied to key computational features, 2) be deployed within a 30-minute timeframe, 3) employ inexpensive materials found in most homes, and 4) use accessible assessment instructions for real-time coaching of parents.
The investigators will now deploy the preliminary TAP-Preschool with a small sample (n = 30) to assess acceptability/feasibility, potential clinical value for remote observation, and challenges that warrant revision. These data will then be used to modify the TAP-Preschool and the refined tool will be deployed with a new sample of clinically referred children (n=120). The investigators will evaluate its ability to facilitate accurate telemedicine supported diagnostic decision-making.
Initial deployment, evaluation, and refinement of the TAP-Preschool:
Participants: Investigators will recruit 30 parent/child dyads (children ages 36 to 72 months) with existing diagnoses of ASD (n = 20) or other developmental concerns (n = 10) from a clinical research database. Each participant will have recent data available from comprehensive evaluation tools (ADOS-2, cognitive functioning, adaptive skills). Children with and without ASD are included to provide information about TAP-Preschool usage across diverse phenotypic profiles. The sample size of 30 is deemed adequate for gathering detection of feasibility/acceptability issues and key feedback regarding measure modification for further validation.
Assessment on psychometrics, clinical, and familial value of the TAP-Preschool:
Participants: Investigators will recruit a novel sample (n = 120) parent/child dyads (children 36 - 72 months of age) referred for evaluation of ASD or developmental delays. These children and a primary caregiver will participate in a home-based tele-assessment session and a subsequent blinded in-person evaluation. English/Spanish speaking families with access to a device that will support Zoom will be included.
Initial deployment, evaluation, and refinement of the TAP-Preschool:
Consenting families will be scheduled for a single tele-assessment session with a consented licensed psychologist ("remote clinician") from our clinical research center (n=10). This clinician will coach parents through the TAP-Preschool procedures. Although families will receive standard, basic support regarding tele-assessment procedures, they will not receive extensive training on the TAP-P prior to the session in order to mimic real-world use. All remote clinicians will be experts in ASD with training on the original TAP and ADOS-2 research reliability. Each clinician will participate in 3 sessions (2 children with ASD, 1 other developmental concerns). Clinicians will be blinded to child diagnostic status. Participating telemedicine clinicians in this aim will not be aware of the ratio (2:1 ASD vs. other DD) or recruitment status prior to evaluation ratings. As children will have existing diagnoses, no diagnostic feedback will be provided.
Investigators will collect user data (caregiver, clinicians) on satisfaction, ease of implementation, and diagnostic certainty (clinicians only). Based on this feedback, the preliminary TAP-Preschool instructions and procedures will be modified as needed. To systematically measure acceptability and feasibility of use, investigators will utilize an adapted Acceptability, Likely Effectiveness, Feasibility, and Appropriateness Questionnaire (ALFA-Q).54 The ALFA-Q asks caregivers and clinicians to use a 5-point Likert scale to rate the instrument acceptability, effectiveness, feasibility, and appropriateness for ASD decision making. We will also solicit free-form input. Team leads will briefly interview each clinician and caregiver about their experiences. After each telemedicine evaluation, participating clinicians will view data from previous comprehensive evaluations. Clinicians will provide concrete task evaluation data regarding whether they were able to elicit such behaviors or information in the telemedicine evaluation process. It is important to note that diagnostic agreement rankings will include dichotomous (agree/disagree) and uncertainty data (Likert ratings). As in previous preliminary feasibility/effectiveness trial, investigators will target >60% of providers agreeing with existing risk classification for ASD, <10% of ASD inaccurately identified with ASD, and >50% non ASD DD with certainty as key benchmarks for understanding potential meaningful clinical value. Based on this data, the investigative team will collaborate with the clinical design team to suggest instrument modifications. The tele-assessment will take <45 minutes to complete. The parent forms will take approximately 20 minutes to complete.
Assessment on psychometrics, clinical, and familial value of the TAP-Preschool (Novel
Sample):
All consented families will complete an initial home-based tele-assessment visit via Zoom that includes the TAP-Preschool and a brief symptom-focused developmental interview with a consented clinician. The initial tele-assessment session includes interviewing and developmental assessment to mimic real-world use of TAP-Preschool. The visit is designed to take less than 90 minutes, with TAP-Preschool evaluation lasting <30 minutes. All sessions will be timed. After the session, the examiner will record the clinical diagnosis issued (ASD, other developmental concerns, or typical development) and complete two diagnostic certainty ratings, one dichotomous, the other continuous. All initial TAP-Preschool administrations will be recorded via Zoom, and 50% of administrations (randomly selected) will be co-scored by a blinded examiner, unaware of prior assessment results and diagnostic decision, to evaluate inter-rater reliability. Within 7 days of the remote assessment, families will participate in an in-person diagnostic assessment including common comprehensive measures of ASD, cognitive skills, and adaptive behavior. Twenty-five percent of families will be randomly selected to participate in a remote home-based re-test with TAP-Preschool two weeks following initial administration.
Tele-assessment measures: Prior to the appointment, caregivers will complete a Demographics Questionnaire (i.e., age, gender race/ethnicity, zip code, parent education) and a Medical History Form through a secure HIPAA-compliant web portal (REDCap). To reduce technology-related disruptions, research staff will talk to families before the visit about connectivity settings and recommended materials. These will take approximately 20 minutes to complete.
In a single tele-assessment session modeled after our previous work, licensed clinicians will administer the TAP-Preschool, the Developmental Profile, 4th edition (DP-4), and a DSM-5 ASD interview. The DP-4 is a caregiver interview (birth - 21 years) that identifies developmental strengths and weaknesses in five core areas. The DSM-5 interview provides symptom-focused questions with developmental anchors pertinent to this age range. Immediately after the session, the clinician will complete a DSM-5 checklist indicating symptoms present and clinical diagnosis issued (ASD, other developmental concerns, or typical development). This form also contains dichotomous (certain/uncertain) and continuous (10-point Likert scale) certainty ratings. Clinicians will describe their satisfaction with the tele-assessment process (CSQ: Clinician Satisfaction Questionnaire). The initial tele-assessment appointment with the psychologist will last approximately 90 minutes.
Blinded in-person assessment measures: The in-person examiner will be blinded to the tele-assessment diagnosis until after the in-person assessment. The diagnostic confirmation battery will include the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) (one module is chosen based on language ability), a cognitive measure (Mullen Scales of Early Learning or Differential Ability Scales 2nd Edition, the Vineland Adaptive Behavior Scales, Third Edition, and a DSM-5 ASD Interview. Immediately after the in-person session, the examiner will also complete a DSM-5 checklist and certainty ratings. In-person appointments will last approximately 2-3 hours and will consist of traditional diagnostic evaluations for autism spectrum disorder.
Parent measures: Parents will complete the Parent Perceptions of Telehealth (PPT) and the Parent Service Satisfaction (PSS) surveys, used in our previous work, to assess perceptions of tele-assessment procedures after the telemedicine and in-person evaluations. These parent forms will take approximately 10 minutes to complete.
Study Design
Arms and Interventions
| Arm | Intervention/Treatment |
|---|---|
| Experimental: Tele-assessment + In-person assessment All families will receive an in-person tele-assessment appointment and an in-person evaluation. |
Behavioral: Tele-assessment + In-person assessment
All consented families will complete an initial home-based tele-assessment visit via Zoom that includes the TAP-Preschool and a brief symptom-focused developmental interview with a consented clinician. The initial tele-assessment session includes interviewing and developmental assessment to mimic real-world use of TAP-Preschool. After the session, the examiner will record the clinical diagnosis issued (ASD, other developmental concerns, or typical development) and complete two diagnostic certainty ratings. All initial TAP-Preschool administrations will be recorded via Zoom, and 50% of administrations (randomly selected) will be co-scored by a blinded examiner, unaware of prior assessment results and diagnostic decision, to evaluate inter-rater reliability. Within 7 days of the remote assessment, families will participate in an in-person diagnostic assessment including common comprehensive measures of ASD, cognitive skills, and adaptive behavior.
|
Outcome Measures
Primary Outcome Measures
- Positive Predictive Value (PPV) of the TAP-Preschool [PPV will be calculated immediately after the last participant completes their in-person assessment]
Positive predictive value is the proportion of participants who screen at-risk for ASD on the TAP-Preschool that have a diagnosis of autism.
- Negative Predictive Value (NPV) of the TAP-Preschool [NPV will be calculated immediately after the last participant completes their in-person assessment]
Negative predictive value is the proportion of the participants who do not screen at-risk for ASD on the TAP-Preschool who do not have a diagnosis of autism.
- Sensitivity of the TAP-Preschool [Sensitivity will be calculated immediately after the last participant completes their in-person assessment]
Sensitivity will measure how often the TAP-Preschool correctly generated a positive result for participants who have autism.
- Specificity of the TAP-Preschool [Specificity will be calculated immediately after the last participant completes their in-person assessment]
Specificity will measure the TAP-Preschool's ability to correctly generate a negative result for people who don't have a diagnosis of autism.
- Diagnostic Certainty: Tele-assessment [Through study completion, an average of 2 years]
Immediately after completing the TAP-Preschool, clinicians will rate their diagnostic certainty in two ways: (1) on a 4-point Likert Scale from "Completely Uncertain" to "Completely Certain" and (2) on a dichotomous scale from "Certain" to "Uncertain"
- Family Satisfaction [Through study completion, an average of 2 years]
Parents will complete the Parent Service Satisfaction survey to assess perceptions of tele-assessment procedures. The survey includes six questions with three response options per questions (Very True, Somewhat True, Not True)
- Diagnostic Certainty: In-person assessment [Through study completion, an average of 2 years]
Immediately after completing the in-person assessment, clinicians will rate their diagnostic certainty in two ways: (1) on a 4-point Likert Scale from "Completely Uncertain" to "Completely Certain" and (2) on a dichotomous scale from "Certain" to "Uncertain"
Eligibility Criteria
Criteria
Inclusion Criteria:
-
Initial deployment (n = 30):
-
English/Spanish Speaking families
-
Children 36-72 months of age
-
access to a device capable of supporting Zoom
-
already has participated in a diagnostic evaluation
Novel sample (n = 120):
Inclusion
-
English/Spanish Speaking families
-
Children 36-72 months of age
-
access to a device capable of supporting Zoom
-
has not participated in a diagnostic evaluation
Exclusion Criteria:
-
Initial deployment (n = 30):
-
severe sensorimotor impairments
Novel sample (n = 120):
- severe sensorimotor impairments
Contacts and Locations
Locations
No locations specified.Sponsors and Collaborators
- Vanderbilt University Medical Center
- National Institute of Mental Health (NIMH)
Investigators
- Principal Investigator: Zachary Warren, PhD, Vanderbilt University Medical Center
Study Documents (Full-Text)
None provided.More Information
Publications
None provided.- 220273
- R21MH128790