EPI-SET: Early Psychosis Intervention - Spreading Evidence-based Treatment
Study Details
Study Description
Brief Summary
Implementation of 'NAVIGATE' in Ontario aims to help youth and emerging adults suffering from a first episode of psychosis. Although Ontario already has early psychosis intervention programs, the team's recent work has identified major challenges of delivering coordinated care, particularly those elements of care that enhance recovery. These challenges also exist nationally and internationally. By building on the already existing early psychosis intervention community of practice through the Early Psychosis Intervention Ontario Network, the investigators will implement NAVIGATE with the help of CAMH's Provincial System Support Program facilitators. The use of tele-videoconferencing through ECHO Mental Health Ontario and ECHO processes and protocols provide us with an opportunity to ensure sustainability. Using health administrative data held at the Institute for Clinical Evaluative Sciences (ICES), the investigators can examine system-level outcomes, including hospitalizations, emergency department visits, and outpatient physician visits of youth and emerging adults suffering from a first episode psychosis who are treated with NAVIGATE compared with those treated in early psychosis intervention programs without NAVIGATE and those who are not treated in early psychosis intervention programs. In addition, the investigators can also evaluate health care costs. Prior to initiating this project, the investigators obtained the input of youth and emerging adults with a first episode psychosis and family members. The investigators will also continue to measure engagement across the study.
Hypotheses:
-
Following the implementation of NAVIGATE, program fidelity (i.e. adaptability) to the Ontario early psychosis intervention standard will improve.
-
Compared to patients not receiving NAVIGATE, those who receive NAVIGATE through this implementation study will have fewer days in hospital, fewer emergency department visits, fewer suicide attempts, lower mortality, and lower healthcare costs.
-
Improvements in functioning and symptoms will be comparable to the RAISE study (an earlier study assessing NAVIGATE); improvement may be influenced by demographic, socio-economic, geographic, and clinical factors.
-
The project's engagement approach will demonstrate that the investigators used the full range of patient engagement based on objectively assessed engagement metrics.
| Condition or Disease | Intervention/Treatment | Phase |
|---|---|---|
|
Detailed Description
The emergence of early psychosis intervention (EPI) programs has provided tremendous hope for individuals experiencing psychosis by providing early intervention. Work by the team shows that youth who access an EPI program in Ontario, Canada receive faster psychiatric follow-up, better coordination of care between inpatient and outpatient services, reduced burden on the emergency department, and reduced all-cause mortality. The team's system-level data support EPI programs as a critical life-saving intervention. Internationally, EPI has demonstrated significant benefits compared to treatment as usual with respect to engagement, service utilization, and suicide.
Despite the initial life-saving benefits of EPI services, provincial, national and international data show that consistent delivery of high-quality, evidence-based care in EPI programs is a major challenge. Recovery rates in EPI programs remain low, and associated disability has not improved under routine clinical care. One explanation for these disappointing facts is that a low proportion of patients receive recovery-based services, namely, case management including individualized psychosocial interventions, family education and intervention, and supported education and employment. Even in clinical service delivery trials, recovery-based service is received by 15-56% of patients, with only 18% receiving comprehensive EPI services. While EPI standards across jurisdictions recommend coordinated and comprehensive recovery-based care, effective implementation and sustainability of such care in real-world settings remains poor.
The team's work in Ontario has uncovered a startling gap between the evidence-based standard of care, and real-world delivery of care, even in a jurisdiction (Ontario) that has prioritized EPI services. The results of the team's Ontario survey were mirrored in a national survey of 11 Canadian EPI programs. This study is designed to improve the delivery of recovery-oriented evidence-based EPI care in Ontario, and improve person-, system-, and economic-level outcomes for youth and emerging adults (YEA) suffering from a first episode psychosis (FEP), offering a potential roadmap for the rest of Canada.
In 2004, the province of Ontario allocated funding for EPI programs based on early evidence of efficacy. Between 2005 and 2007, this funding led to a major EPI program expansion in Ontario, but the absence of provincial standards led to program development in an ad-hoc manner, drawing upon general service delivery experience and advice from established programmes. Concomitantly, the Early Psychosis Intervention Ontario Network (EPION) was established, which has now grown to 52 EPI sites. The establishment of EPI program standards in Ontario in 2011, informed by existing international and national standards, but also by input from clinicians, patients, family-members, and policy makers was a crucial first step toward standardizing care. However, the release of standards alone is not sufficient to ensure effective implementation, practice change, and ongoing quality practice.
Key findings from the first survey by the Ontario Standards Implementation Steering Committee (SISC) of 52 EPI program sites in Ontario (92% participation) demonstrated important challenges in delivering evidence-based EPI care. A follow-up survey also identified opportunities for addressing these challenges. Building on the two surveys, the investigators conducted a study to measure fidelity to current EPI standards using the First Episode Psychosis Service - Fidelity Scale. The most notable finding was lack of delivery of consistent recovery-oriented care, with no structured or manualized process for these elements of care. Through site visits, and in-person interviews with nine Ontario EPI programs, the investigators obtained a richer and clearer picture of the current state, creating an opportunity to implement solutions that can address the major challenges identified in the fidelity study.
The investigators want to address these challenges in Ontario EPI settings by implementing NAVIGATE, a coordinated and comprehensive multidisciplinary treatment program for FEP that is deliverable in community mental health settings. NAVIGATE is manualized and measurement-based. To maximize affordability, adaptability, spread, and sustainability, the investigators will utilize the already-established EPION community of practice, the implementation science expertise of PSSP (Provincial System Support Program), and the sustainability and capacity building approach of The Extension of Community Health Outcomes (ECHO) Ontario Mental Health at CAMH and the University of Toronto (ECHO-ONMH).
NAVIGATE was developed in consultation with clinical and research experts, biostatisticians, health economists, consumers, family members, advocacy groups, and government officials. It is a form of coordinated specialty care for FEP consisting of 4 key intervention components: (i) individualized medication management using a decision support tool, (ii) a package of psychoeducation and a blend of evidence-based psychotherapies called "individual resiliency training" (IRT); (iii) supported employment and education (SEE); and (iv) a family education program. NAVIGATE was evaluated from 2009-2014 in a cluster randomized controlled trial involving 404 individuals with an FEP in 34 community mental health centers across the United States. Notably, it was delivered by re-allocating existing community mental health resources with no new funding for clinical care. Compared to usual care, NAVIGATE treatment provided greater improvement in symptoms, but more importantly, as prioritized by patients, significantly greater improvement in real-world functioning, including social functioning and engagement in educational and vocational training. NAVIGATE was also readily implementable across a broad range of community-based mental health settings, with enhanced engagement and delivery of multidisciplinary care to YEA presenting with psychosis and their families compared to standard care, and with longer and more consistent receipt of mental health services. A recent economic analysis revealed that NAVIGATE treatment was more cost-effective compared to standard community care, driven by the anticipated enhanced health benefits and improvements in quality of life. NAVIGATE implementation addresses the major challenges identified in the previously mentioned SISC fidelity study, as follows:
Challenge 1: NAVIGATE operationalizes current EPI standards using manualized protocols, ensuring consistency and reducing variability in care. The four NAVIGATE components (individualized medication management, IRT, SEE, and family education program) are systematically applied in collaboration with the patient. There is an overarching emphasis on the coordinated delivery of these elements of care. Every patient is offered these elements of care, and modules are completed in a systematic time-oriented fashion that reduces variability in care among sites and team-members within a site. At each patient visit, a contact/progress note is completed, including the modules delivered, that the team reviews to assess patient progress, fidelity, and determine need for adjustments.
Challenge 2. Ontario EPI sites have learned about NAVIGATE through didactic sessions and conferences offered through EPION. Because translation and implementation of evidence-based practice remains a challenge, training, implementation expertise, and other resources are required. The mandate of the CAMH PSSP is to support system change in Ontario by providing implementation, evaluation, knowledge exchange, engagement and information management expertise to organizations and networks across the province. PSSP operates regional offices throughout Ontario and their implementation teams work closely with local communities and key partners to implement and sustain system improvements. For this project, regional PSSP teams will support implementation and work closely with NAVIGATE content experts from the Slaight Centre (CAMH's EPI program), and community site leads, who are members of EPION.
Challenge 3. EPION has identified use of technology to transfer knowledge, and equitable application of the provincial standards as key priorities. ECHO consists of specialist hubs that connect with multiple learner teams in remote areas through televideo-conferencing technology, and thus can bridge the geographic gap required to bolster the Ontario EPI community of practice. The goal of ECHO clinics is to extend the reach of best practices in academic settings to the chronic and complex illnesses seen in local settings, thereby reducing variation, increasing access to specialist mentoring and services, and monitoring and improving patient outcomes. Initial evaluation data in Ontario has shown high provider satisfaction and engagement with ECHO, and increased knowledge and self-efficacy in managing mental health and addictions in rural and remote settings.
How is this Study Further Enhanced?
-
This project includes Patient and Family Engagement. In the direct planning and design of the present study, one of the team's Principal Investigators with lived experience was present at each team meeting, and was an equal partner in decisions regarding all of the outcome measures. A co-applicant family member (her son experienced an FEP) also joined the team, and she provided input, further refining the research objectives. The Advisory Committees will grow to include patient/family representation from each site, and will provide ongoing guidance to the research team in implementation, evaluation, analyses, and dissemination. The investigators will also continue qualitative work with patients and families to further evaluate NAVIGATE at the participating sites.
-
Using data from ICES, which captures all physician and hospital-provided insured services in Ontario, the investigators will compare population-based outcomes (hospitalizations, emergency department visits, suicide attempts and mortality) with two comparison groups: 1) all EPI program FEP patients who have not been part of the NAVIGATE trial; and 2) all FEP patients who are not attached to EPI programs. All EPI programs in Ontario report their service utilization data through the Ontario Common Assessment of Need (OCAN) dataset (OCAN is linked to ICES data). The investigators can accurately ascertain the initiation date of all EPI patients by sex within Ontario through ICES linkage. The ability to link NAVIGATE's primary clinical and implementation data with ICES data, and the capacity to identify all FEP patients in Ontario (whether treated in EPI programs or not) allows for a meaningful, valid, and highly generalizable comparison of outcomes. The use of propensity score methods will address confounding associated with observational studies, and mimics some of the characteristics of a randomized controlled trial.
Primary Study Hypotheses:
-
Following the implementation of NAVIGATE, program fidelity to the Ontario EPI standard will improve.
-
Compared to patients not receiving NAVIGATE, those who receive NAVIGATE through this implementation study will have fewer days in hospital, fewer ED visits, fewer suicide attempts, lower mortality, and lower costs.
-
Improvements in functioning and symptoms will be comparable to the RAISE study; improvement may be influenced by demographic, socio-economic, geographic, and clinical factors.
-
The project's engagement approach will demonstrate that the investigators used the full range of patient engagement based on objectively assessed engagement metrics.
Study Design
Arms and Interventions
| Arm | Intervention/Treatment |
|---|---|
| NAVIGATE EPI This group of first episode psychosis patients is receiving NAVIGATE early psychosis intervention (EPI) as their regular clinical standard of care. The project team is implementing NAVIGATE at several early psychosis intervention (EPI) programs in different geographic regions of Ontario. The team will recruit consecutive referrals to these programs in order to determine longitudinal change in functioning and symptoms (hypothesis #3). Additionally, the primary data collected for these patients will be linked deterministically to data sources held at the Institute for Clinical Evaluative Sciences (ICES) via their unique health card number. Routine system-level outcome measurements will be compared among NAVIGATE subjects and two control populations (hypothesis #2 - see other groups/cohorts). |
Behavioral: NAVIGATE
NAVIGATE was developed in consultation with clinical and research experts, biostatisticians, health economists, consumers, family members, advocacy groups, and government officials. It is a form of coordinated specialty care for first episode psychosis consisting of 4 key intervention components:
individualized medication management using a decision support tool;
a package of psychoeducation and a blend of evidence-based psychotherapies called "individual resiliency training" (IRT);
supported employment and education (SEE);
a family education program
|
| Non-NAVIGATE EPI This group of first episode psychosis patients received early psychosis intervention other than NAVIGATE as their regular clinical standard of care. The data already collected for these patients (not as a part of study/recruitment) is held at the Institute for Clinical Evaluative Sciences (ICES). Routine system-level outcome measurements will be compared among NAVIGATE subjects and two control populations (hypothesis #2 - see other groups/cohorts). |
|
| Non-EPI This group of first episode psychosis patients did not receive early psychosis intervention as their regular clinical standard of care. The data already collected for these patients (not as a part of study/recruitment) is held at the Institute for Clinical Evaluative Sciences (ICES). Routine system-level outcome measurements will be compared among NAVIGATE subjects and two control populations (hypothesis #2 - see other groups/cohorts). |
Outcome Measures
Primary Outcome Measures
- Hypothesis 1: Fidelity/Adaptability [Exploration stage of implementation [approx. year 1], following implementation [approx. year 2] and end of study [approx. year 4]]
First Episode Psychosis Service-Fidelity Scale (FEPS-FS) will be used to assess fidelity of service delivery to the current standard of early psychosis intervention in relation to 32 program-specific items (individual and team practices) on a 5-point scale from "1=not implemented" to "5=fully implemented". Ratings for each site will be made through a remote assessment process that includes a review of site administrative data, data abstracted from client health records, and phone interviews with site staff. Site will be the unit of analysis. Descriptive statistics (percentages, means, medians, range) will be reported for the total scale score and for subscale scores that align with NAVIGATE components. Fidelity scores will be calculated per site in relation to the four core NAVIGATE interventions using measurements of service utilization. Total and subscale scores will be an average of item scores, reported out of 5.
- Hypothesis 1: Penetration/Scalability [Month 48]
To determine whether implementation of NAVIGATE is associated with improvement in fidelity to the EPI standard, program staff will document delivery of core modules for each of the four interventions. We will calculate the percentage of core modules completed per intervention per patient to assess penetration and to identify variations in delivery both within sites and across sites. Additionally, we will calculate frequency of team activities (e.g., weekly meetings, direct supervision) and assess staff perceived competence in delivery of NAVIGATE using the Readiness Monitoring Tool and the competency assessment questionnaires completed at the conclusion of each ECHO cycle.
- Hypothesis 1: Contextual Factors [End of study [approx. year 4]]
The Consolidated Framework for Implementation Research (CFIR) will be used to systematically assess contextual factors that are associated with effective implementation. The CFIR constructs are organized within five major domains: intervention characteristics (e.g., complexity, relative advantage); outer setting (e.g., external policy, patient needs); inner setting (e.g., resources, fit, leadership); staff characteristics (e.g., knowledge, beliefs); and implementation process (e.g., facilitation, planning, coaching). The CFIR will be used to develop a semi-structured interview to guide data collection. Interviews will be conducted with stakeholders at each site (EPI staff, organization leaders) at the end of the study, recorded, and transcribed.
- Hypothesis 1: Sustainability [Ongoing throughout study [years 2-4 inclusive]]
Staff attendance during ECHO sessions will indicate ECHO engagement and retention.
- Hypothesis 1: Sustainability [Prior to ECHO participation [approx. year 2] and end of study [approx. year 4]]
A competency assessment questionnaire will be administered to staff to assess changes in attitudes, knowledge and self-efficacy (self-reported competence) in delivering NAVIGATE components. Question domains will include Likert scales assessing self-reported knowledge and skill domains specific to the NAVIGATE model. Self-efficacy questions will focus on key competency domains of NAVIGATE and assess confidence in delivering these components in their local settings.
- Hypothesis 1: Affordability [End of study [approx. year 4]]
The investigators will employ a costing algorithm developed in SAS®, and available at ICES, to estimate all direct patient-level health care costs incurred by the public third-party payer (Ontario Ministry of Health and Long-Term Care) across the three comparison groups. Included will be costs of hospitalizations (both non-psychiatric and psychiatric); ED visits; physician services (i.e. primary care, psychiatry and other care) and diagnostics tests; outpatient prescription drugs for individuals covered under the provincial public drug insurance plan only (for individuals under 65 who receive social assistance and for individuals under age 25 who lack private insurance coverage); home care; and other care (this includes other ambulatory care, such as same-day surgery/procedures, cancer and dialysis clinic visits, and other hospital-based care, such as rehabilitation and complex continuing care).
- Hypothesis 2: System-Level: Number of psychiatric hospitalization days in the year following NAVIGATE admission [End of study [approx. year 4]]
Propensity scores will be used to compare NAVIGATE participants with 2 groups: 1) FEP patients attached to EPI programs who are not participating in the NAVIGATE trial; and 2) FEP patients with no EPI program attachment. The primary outcome is days in hospital (psychiatric hospitalizations) in the year following NAVIGATE admission.
- Hypothesis 2: System-Level: Time to first psychiatric hospitalization [End of study [approx. year 4]]
Propensity scores will be used to compare NAVIGATE participants with 2 comparison groups: 1) FEP patients attached to EPI programs who are not participating in the NAVIGATE trial; and 2) FEP patients with no EPI program attachment. Hospitalization-based outcomes assessed include time to first psychiatric hospitalization.
- Hypothesis 3: Assessment for diagnosis [Admission to clinic [month 0]]
Psychiatric diagnosis/diagnoses will be confirmed using the Structured Clinical Interview for DSM-5 (SCID-5). Information from the SCID-5 will be supplemented by information from family informants, any previous psychiatrist, and medical records.
- Hypothesis 3: Assessment of clinical psychopathology [BPRS] [Admission to clinic [months 0, 6, 12, 18, 24]]
The Brief Psychiatric Rating Scale (BPRS) (24 item) will be used to assess the severity of positive symptoms, psychosis, negative symptoms, and general psychopathology. Scores range from 2-7 for each item (7 indicates worse symptoms). Total score is a sum ranging from 48-168.
- Hypothesis 3: Assessment of clinical psychopathology [PHQ-9] [Admission to clinic [month 0, 6, 12, 18, 24]]
The self-report Patient Health Questionnaire - 9 (PHQ-9) will be used to characterize the presence and severity of depressive symptoms. Higher score = worse symptoms (range 0-27).
- Hypothesis 3: Assessment of clinical psychopathology [QLS] [Admission to clinic [month 0, 6, 12, 18, 24]]
The Intrinsic Motivation Factor of the Quality of Life Scale (QLS) will serve as a specific measure of motivation to augment the above psychopathology measures. Semi-structured interview (subscale item) score range 0-6 (higher score = higher intrinsic motivation).
- Hypothesis 3: Assessment of illness severity and improvement [Admission to clinic [month 0, 6, 12, 18, 24]]
The Clinical Global Impressions Scale (CGI) will be administered to characterize overall illness severity. It rates both illness severity and improvement. It takes into account all available information, including knowledge of the patient's history, psychological circumstances, symptoms, behaviour, and the impact of the symptoms on the patient's ability to function. Structured interview, illness and improvement score range 0-7 (higher = worse severity/least improvement).
- Hypothesis 3: Assessment of functioning [WHODAS 2.0] [Admission to clinic [month 0, 6, 12, 18, 24]]
WHO Disability Assessment Schedule 2.0: assessment of functioning to provide an assessment of health and disability. Scoring will be done using item-response theory. It takes the coding for each item response as "none", "mild", "moderate", "severe" and "extreme" separately, and then uses an algorithm to determine the summary score by differentially weighting the items and the levels of severity. Domain and total scores will be produced (total score range 0 to 100, where 0 = no disability; 100 = full disability).
- Hypothesis 3: Assessment of parental socio-economic status [Admission to clinic [month 0]]
Parental and participant education will be used as indicator of SES.
- Hypothesis 3: Measurement of Service Utilization [Month 6, 12, 18, 24]
The Service Use and Resource Form (SURF) will be used to measure utilization of mental health and other medical services across residential, inpatient, and outpatient treatment settings. This will be administered by research staff either over the phone or via email survey, based on participant preference.
- Hypothesis 4: Engagement [End of implementation [approx. year 2], Study end [approx. year 4]]
The investigators will build on experiential knowledge to further evaluate NAVIGATE from the patient and family perspective. This particular approach will engage patients and family members to build on qualitative work regarding acceptability, feasibility, and preference for NAVIGATE, i.e. coordinated, manualized EPI care. Semi-structured interviews will be conducted with patients, families, and NAVIGATE staff members.
- Hypothesis 4: Engagement [PPEET/PCORI] [After the first engagement [approx. year 1]. end of implementation [approx. year 2]]
The investigators will build on experiential knowledge to further evaluate NAVIGATE from the patient and family perspective. This particular approach will engage patients and family members to build on qualitative work regarding acceptability, feasibility, and preference for NAVIGATE, i.e. coordinated, manualized EPI care. A modified (simplified/combined) PPEET (Public and Patient Engagement Evaluation Tool) & PCORI engagement activity inventory will be administered to Youth and Family Advisory Members.
- Hypothesis 4: Engagement [Interview] [After first engagement [approx. year 1], End of the implementation [approx. year 2], study end [approx. year 4]]
The investigators will build on experiential knowledge to further evaluate NAVIGATE from the patient and family perspective. This particular approach will engage patients and family members to build on qualitative work regarding acceptability, feasibility, and preference for NAVIGATE, i.e. coordinated, manualized EPI care. A semi-structured interview will be conducted with Youth and Family Advisory Members.
Secondary Outcome Measures
- Hypothesis 2: System-Level: Emergency Department Visits and Suicide Attempts [End of study [approx. year 4]]
Each NAVIGATE subject will be matched to two types of subjects: 1) FEP patients attached to EPI programs who are not participating in the NAVIGATE trial; and 2) FEP patients with no EPI program attachment. Investigators will measure psychiatric emergency department visits with a year of admission as well as ED visits for suicide attempts. Visits to psychiatrists and primary care physicians will also be counted. Visits to primary care physicians will be stratified as mental health related versus non-mental health related based on a previously validated algorithm.
- Hypothesis 3: Demographics [Admission to clinic [month 0]]
A demographic questionnaire will be administered. Example items include participant sex, gender, education, employment, financial support, ethnicity, race, language, legal system involvement, housing situation, and sibling and parent information.
Eligibility Criteria
Criteria
Early psychosis intervention (EPI) programs from specific geographic regions of Ontario will be included. Investigators will recruit consecutive referrals to the EPI programs participating in the study. All of the EPI sites follow people experiencing a first episode psychosis.
Individual inclusion criteria:
-
Age range of 14-35 years;
-
any DSM-diagnosis that can manifest as early psychosis (schizophrenia, schizoaffective disorder, schizophreniform disorder, bipolar I disorder, major depressive disorder with psychotic features, substance induced psychotic disorder, or unspecified psychotic disorder);
Exclusion Criteria:
- Absence of psychosis
Contacts and Locations
Locations
| Site | City | State | Country | Postal Code | |
|---|---|---|---|---|---|
| 1 | North Bay Regional Health Centre | North Bay | Ontario | Canada | P1B 0A4 |
| 2 | Durham Amaze- Lakeridge Health | Oshawa | Ontario | Canada | L1G 2B9 |
| 3 | Health Sciences North | Sudbury | Ontario | Canada | P3E 5J1 |
| 4 | Niagara Region Public Health | Thorold | Ontario | Canada | L2V 4Y6 |
| 5 | First Place Clinic and Regional Resource Centre | Thunder Bay | Ontario | Canada | P7A4K9 |
| 6 | CMHA Waterloo Wellington | Waterloo | Ontario | Canada | N2J 4M1 |
Sponsors and Collaborators
- Centre for Addiction and Mental Health
- Canadian Institutes of Health Research (CIHR)
- Institute for Clinical Evaluative Sciences
- North Bay Regional Health Centre
- Health Sciences North
- Niagara Region Public Health
- Lakeridge Health Corporation
- Canadian Mental Health Association - Waterloo Wellington
- Canadian Mental Health Association - Thunder Bay
Investigators
- Principal Investigator: Aristotle Voineskos, MD, PhD, Centre for Addiction & Mental Health
- Principal Investigator: George Foussias, MD, PhD, Centre for Addiction & Mental Health
- Principal Investigator: Paul Kurdyak, MD, PhD, Centre for Addiction & Mental Health
- Principal Investigator: Janet Durbin, PhD, MSc, Centre for Addiction & Mental Health
- Principal Investigator: Sanjeev Sockalingam, MD, Centre for Addiction & Mental Health
Study Documents (Full-Text)
More Information
Additional Information:
- World Economic Forum and the Harvard School of Public Health. (2011) Methodological Appendix: The Global Economic Burden of Non-Communicable Diseases.
- Ministry of Health and Long-Term Care (2011). Early Psychosis Intervention Program Standards
- Fixsen DL, Blasé KA, Naoom SF, Wallace F. Core Implementation Components. Research on Social Work Practice. 2009 Sep;19(5):531-540.
- Implementation of Early Psychosis Intervention Program Standards in Ontario: Results from a Provincial Survey: Centre for Addiction and Mental Health and the Ontario Working Group for Early Psychosis
- Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, National Implementation Research Network; 2005.
- United States Congress. Public Law 114-270-Dec. 14, 2016.
- Measuring health and disability: Manual for WHO Disability Assessment Schedule WHODAS 2.0. Geneva: World Health Organization.
- Children's Participation: From tokenism to citizenship, Innocenti Essay no. 4, International Child Development Centre, Florence (1992).
- Patient Engagement and Canada's SPOR Initiative. McMaster University (2015).
- CIHR's Citizen Engagement Handbook (2012).
- Briefing notes for researchers: involving the public in NHS, public health and social care research.
- PCORI Engagement Rubric (2014).
- Strategy for Patient-Oriented Research: Patient Engagement Framework (2014).
- Marshall MN. Sampling for Qualitative Research. Family Practice.1996 Dec;13(6):522-525.
- Patient and Public Engagement Evaluation Toolkit. Centre of Excellence on Partnership Patients and the Public.
Publications
- Abelson J, Li K, Wilson G, Shields K, Schneider C, Boesveld S. Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool. Health Expect. 2016 Aug;19(4):817-27. doi: 10.1111/hex.12378. Epub 2015 Jun 25.
- Addington D, Anderson E, Kelly M, Lesage A, Summerville C. Canadian Practice Guidelines for Comprehensive Community Treatment for Schizophrenia and Schizophrenia Spectrum Disorders. Can J Psychiatry. 2017 Sep;62(9):662-672. doi: 10.1177/0706743717719900. Review.
- Addington DE, Norman R, Bond GR, Sale T, Melton R, McKenzie E, Wang J. Development and Testing of the First-Episode Psychosis Services Fidelity Scale. Psychiatr Serv. 2016 Sep 1;67(9):1023-5. doi: 10.1176/appi.ps.201500398. Epub 2016 Apr 1.
- Anderson KK, Flora N, Ferrari M, Tuck A, Archie S, Kidd S, Tang T, Kirmayer LJ, McKenzie K; ACE Project Team. Pathways to First-Episode Care for Psychosis in African-, Caribbean-, and European-Origin Groups in Ontario. Can J Psychiatry. 2015 May;60(5):223-31.
- Anderson KK, McKenzie KJ, Kurdyak P. Examining the impact of migrant status on ethnic differences in mental health service use preceding a first diagnosis of schizophrenia. Soc Psychiatry Psychiatr Epidemiol. 2017 Aug;52(8):949-961. doi: 10.1007/s00127-017-1403-z. Epub 2017 Jun 10.
- Anderson KK, Norman R, MacDougall A, Edwards J, Palaniyappan L, Lau C, Kurdyak P. Effectiveness of Early Psychosis Intervention: Comparison of Service Users and Nonusers in Population-Based Health Administrative Data. Am J Psychiatry. 2018 May 1;175(5):443-452. doi: 10.1176/appi.ajp.2017.17050480. Epub 2018 Mar 2.
- Arnett JJ. Emerging adulthood. A theory of development from the late teens through the twenties. Am Psychol. 2000 May;55(5):469-80.
- Arora S, Kalishman S, Dion D, Som D, Thornton K, Bankhurst A, Boyle J, Harkins M, Moseley K, Murata G, Komaramy M, Katzman J, Colleran K, Deming P, Yutzy S. Partnering urban academic medical centers and rural primary care clinicians to provide complex chronic disease care. Health Aff (Millwood). 2011 Jun;30(6):1176-84. doi: 10.1377/hlthaff.2011.0278. Epub 2011 May 19.
- Arora S, Thornton K, Murata G, Deming P, Kalishman S, Dion D, Parish B, Burke T, Pak W, Dunkelberg J, Kistin M, Brown J, Jenkusky S, Komaromy M, Qualls C. Outcomes of treatment for hepatitis C virus infection by primary care providers. N Engl J Med. 2011 Jun 9;364(23):2199-207. doi: 10.1056/NEJMoa1009370. Epub 2011 Jun 1.
- Austin PC. An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. Multivariate Behav Res. 2011 May;46(3):399-424. Epub 2011 Jun 8.
- Birchwood M, Todd P, Jackson C. Early intervention in psychosis. The critical period hypothesis. Br J Psychiatry Suppl. 1998;172(33):53-9. Review.
- Bolton SL, Sareen J. Sexual orientation and its relation to mental disorders and suicide attempts: findings from a nationally representative sample. Can J Psychiatry. 2011 Jan;56(1):35-43.
- Bond GR, Drake RE, Luciano A. Employment and educational outcomes in early intervention programmes for early psychosis: a systematic review. Epidemiol Psychiatr Sci. 2015 Oct;24(5):446-57. doi: 10.1017/S2045796014000419. Epub 2014 Jul 14.
- Catts SV, Evans RW, O'Toole BI, Carr VJ, Lewin T, Neil AL, Harris MG, Frost AD, Crissman BR, Eadie K. Is a national framework for implementing early psychosis services necessary? Results of a survey of Australian mental health service directors. Early Interv Psychiatry. 2010 Feb;4(1):25-30. doi: 10.1111/j.1751-7893.2009.00157.x.
- Cheng C, Dewa CS, Goering P. Matryoshka Project: lessons learned about early intervention in psychosis programme development. Early Interv Psychiatry. 2011 Feb;5(1):64-9. doi: 10.1111/j.1751-7893.2010.00255.x.
- Connell M, Schweitzer R, King R. Recovery from first-episode psychosis and recovering self: A qualitative study. Psychiatr Rehabil J. 2015 Dec;38(4):359-64. doi: 10.1037/prj0000077. Epub 2014 Jun 9.
- Correll CU, Solmi M, Veronese N, Bortolato B, Rosson S, Santonastaso P, Thapa-Chhetri N, Fornaro M, Gallicchio D, Collantoni E, Pigato G, Favaro A, Monaco F, Kohler C, Vancampfort D, Ward PB, Gaughran F, Carvalho AF, Stubbs B. Prevalence, incidence and mortality from cardiovascular disease in patients with pooled and specific severe mental illness: a large-scale meta-analysis of 3,211,768 patients and 113,383,368 controls. World Psychiatry. 2017 Jun;16(2):163-180. doi: 10.1002/wps.20420. Erratum in: World Psychiatry. 2018 Feb;17 (1):120.
- Craig TK, Garety P, Power P, Rahaman N, Colbert S, Fornells-Ambrojo M, Dunn G. The Lambeth Early Onset (LEO) Team: randomised controlled trial of the effectiveness of specialised care for early psychosis. BMJ. 2004 Nov 6;329(7474):1067. Epub 2004 Oct 14.
- Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009 Aug 7;4:50. doi: 10.1186/1748-5908-4-50.
- Damschroder LJ, Lowery JC. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement Sci. 2013 May 10;8:51. doi: 10.1186/1748-5908-8-51.
- Damschroder LJ, Moin T, Datta SK, Reardon CM, Steinle N, Weinreb J, Billington CJ, Maciejewski ML, Yancy WS Jr, Hughes M, Makki F, Richardson CR. Implementation and evaluation of the VA DPP clinical demonstration: protocol for a multi-site non-randomized hybrid effectiveness-implementation type III trial. Implement Sci. 2015 May 12;10:68. doi: 10.1186/s13012-015-0250-0.
- de Oliveira C, Cheng J, Rehm J, Kurdyak P. The Economic Burden of Chronic Psychotic Disorders in Ontario. J Ment Health Policy Econ. 2016 Dec 1;19(4):181-192.
- Doyle R, Turner N, Fanning F, Brennan D, Renwick L, Lawlor E, Clarke M. First-episode psychosis and disengagement from treatment: a systematic review. Psychiatr Serv. 2014 May 1;65(5):603-11. doi: 10.1176/appi.ps.201200570. Review.
- Durbin J, Selick A, Hierlihy D, Moss S, Cheng C. A first step in system improvement: a survey of Early Psychosis Intervention Programmes in Ontario. Early Interv Psychiatry. 2016 Dec;10(6):485-493. doi: 10.1111/eip.12201. Epub 2014 Nov 4.
- Durlak JA, DuPre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008 Jun;41(3-4):327-50. doi: 10.1007/s10464-008-9165-0. Review.
- Ferrari M, Flora N, Anderson KK, Tuck A, Archie S, Kidd S, McKenzie K; ACE Project Team. The African, Caribbean and European (ACE) Pathways to Care study: a qualitative exploration of similarities and differences between African-origin, Caribbean-origin and European-origin groups in pathways to care for psychosis. BMJ Open. 2015 Jan 14;5(1):e006562. doi: 10.1136/bmjopen-2014-006562.
- Fixsen DL, Blase KA, Van Dyke MK. Mobilizing communities for implementing evidence-based youth violence prevention programming: a commentary. Am J Community Psychol. 2011 Sep;48(1-2):133-7. doi: 10.1007/s10464-010-9410-1.
- Forsythe LP, Ellis LE, Edmundson L, Sabharwal R, Rein A, Konopka K, Frank L. Patient and Stakeholder Engagement in the PCORI Pilot Projects: Description and Lessons Learned. J Gen Intern Med. 2016 Jan;31(1):13-21. doi: 10.1007/s11606-015-3450-z. Epub 2015 Jul 10.
- Fortney J, Enderle M, McDougall S, Clothier J, Otero J, Altman L, Curran G. Implementation outcomes of evidence-based quality improvement for depression in VA community based outpatient clinics. Implement Sci. 2012 Apr 11;7:30. doi: 10.1186/1748-5908-7-30.
- Fusar-Poli P, McGorry PD, Kane JM. Improving outcomes of first-episode psychosis: an overview. World Psychiatry. 2017 Oct;16(3):251-265. doi: 10.1002/wps.20446.
- Gatov E, Kurdyak P, Sinyor M, Holder L, Schaffer A. Comparison of Vital Statistics Definitions of Suicide against a Coroner Reference Standard: A Population-Based Linkage Study. Can J Psychiatry. 2018 Mar;63(3):152-160. doi: 10.1177/0706743717737033. Epub 2017 Oct 23.
- Gatov E, Rosella L, Chiu M, Kurdyak PA. Trends in standardized mortality among individuals with schizophrenia, 1993-2012: a population-based, repeated cross-sectional study. CMAJ. 2017 Sep 18;189(37):E1177-E1187. doi: 10.1503/cmaj.161351.
- Global Burden of Disease Study 2013 Collaborators. Global, regional, and national incidence, prevalence, and years lived with disability for 301 acute and chronic diseases and injuries in 188 countries, 1990-2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2015 Aug 22;386(9995):743-800. doi: 10.1016/S0140-6736(15)60692-4. Epub 2015 Jun 7. Review.
- Grawe RW, Falloon IR, Widen JH, Skogvoll E. Two years of continued early treatment for recent-onset schizophrenia: a randomised controlled study. Acta Psychiatr Scand. 2006 Nov;114(5):328-36.
- Heinrichs DW, Hanlon TE, Carpenter WT Jr. The Quality of Life Scale: an instrument for rating the schizophrenic deficit syndrome. Schizophr Bull. 1984;10(3):388-98.
- Henderson J, Brownlie E, Rosenkranz S, Chaim G, Beitchman J. Integrated Knowledge Translation and Grant Development: Addressing the Research Practice Gap through Stakeholder-informed Research. J Can Acad Child Adolesc Psychiatry. 2013 Nov;22(4):268-74.
- Henderson JL, Cheung A, Cleverley K, Chaim G, Moretti ME, de Oliveira C, Hawke LD, Willan AR, O'Brien D, Heffernan O, Herzog T, Courey L, McDonald H, Grant E, Szatmari P. Integrated collaborative care teams to enhance service delivery to youth with mental health and substance use challenges: protocol for a pragmatic randomised controlled trial. BMJ Open. 2017 Feb 6;7(2):e014080. doi: 10.1136/bmjopen-2016-014080.
- Herman DB, Susser ES, Jandorf L, Lavelle J, Bromet EJ. Homelessness among individuals with psychotic disorders hospitalized for the first time: findings from the Suffolk County Mental Health Project. Am J Psychiatry. 1998 Jan;155(1):109-13.
- Hjorthøj C, Stürup AE, McGrath JJ, Nordentoft M. Years of potential life lost and life expectancy in schizophrenia: a systematic review and meta-analysis. Lancet Psychiatry. 2017 Apr;4(4):295-301. doi: 10.1016/S2215-0366(17)30078-0. Epub 2017 Feb 22. Erratum in: Lancet Psychiatry. 2017 Sep;4(9):e19.
- Hui CLM, Lo MCL, Chan EHC, Chen ESM, Ko RWT, Lee EHM, Chang WC, Chan SKW, Chen EYH. Perception towards relapse and its predictors in psychosis patients: A qualitative study. Early Interv Psychiatry. 2018 Oct;12(5):856-862. doi: 10.1111/eip.12378. Epub 2016 Aug 29.
- Jääskeläinen E, Juola P, Hirvonen N, McGrath JJ, Saha S, Isohanni M, Veijola J, Miettunen J. A systematic review and meta-analysis of recovery in schizophrenia. Schizophr Bull. 2013 Nov;39(6):1296-306. doi: 10.1093/schbul/sbs130. Epub 2012 Nov 20. Review.
- Kane JM, Robinson DG, Schooler NR, Mueser KT, Penn DL, Rosenheck RA, Addington J, Brunette MF, Correll CU, Estroff SE, Marcy P, Robinson J, Meyer-Kalos PS, Gottlieb JD, Glynn SM, Lynde DW, Pipes R, Kurian BT, Miller AL, Azrin ST, Goldstein AB, Severe JB, Lin H, Sint KJ, John M, Heinssen RK. Comprehensive Versus Usual Community Care for First-Episode Psychosis: 2-Year Outcomes From the NIMH RAISE Early Treatment Program. Am J Psychiatry. 2016 Apr 1;173(4):362-72. doi: 10.1176/appi.ajp.2015.15050632. Epub 2015 Oct 20.
- Kane JM, Schooler NR, Marcy P, Correll CU, Brunette MF, Mueser KT, Rosenheck RA, Addington J, Estroff SE, Robinson J, Penn DL, Robinson DG. The RAISE early treatment program for first-episode psychosis: background, rationale, and study design. J Clin Psychiatry. 2015 Mar;76(3):240-6. doi: 10.4088/JCP.14m09289.
- Kessler RS, Purcell EP, Glasgow RE, Klesges LM, Benkeser RM, Peek CJ. What does it mean to "employ" the RE-AIM model? Eval Health Prof. 2013 Mar;36(1):44-66. doi: 10.1177/0163278712446066. Epub 2012 May 21.
- Kuipers E, Holloway F, Rabe-Hesketh S, Tennakoon L; Croydon Outreach and Assertive Support Team (COAST). An RCT of early intervention in psychosis: Croydon Outreach and Assertive Support Team (COAST). Soc Psychiatry Psychiatr Epidemiol. 2004 May;39(5):358-63.
- Kurdyak P, Lin E, Green D, Vigod S. Validation of a Population-Based Algorithm to Detect Chronic Psychotic Illness. Can J Psychiatry. 2015 Aug;60(8):362-8.
- Leifker FR, Patterson TL, Heaton RK, Harvey PD. Validating measures of real-world outcome: the results of the VALERO expert survey and RAND panel. Schizophr Bull. 2011 Mar;37(2):334-43. doi: 10.1093/schbul/sbp044. Epub 2009 Jun 12.
- Lester H, Khan N, Jones P, Marshall M, Fowler D, Amos T, Birchwood M. Service users' views of moving on from early intervention services for psychosis: a longitudinal qualitative study in primary care. Br J Gen Pract. 2012 Mar;62(596):e183-90. doi: 10.3399/bjgp12X630070.
- Malla A, Schmitz N, Norman R, Archie S, Windell D, Roy P, Zipursky RB. A multisite Canadian study of outcome of first-episode psychosis treated in publicly funded early intervention services. Can J Psychiatry. 2007 Sep;52(9):563-71.
- McGlashan TH. Early detection and intervention of schizophrenia: rationale and research. Br J Psychiatry Suppl. 1998;172(33):3-6. Review.
- Meyers DC, Durlak JA, Wandersman A. The quality implementation framework: a synthesis of critical steps in the implementation process. Am J Community Psychol. 2012 Dec;50(3-4):462-80. doi: 10.1007/s10464-012-9522-x. Review.
- Moore DE Jr, Green JS, Gallis HA. Achieving desired results and improved outcomes: integrating planning and assessment throughout learning activities. J Contin Educ Health Prof. 2009 Winter;29(1):1-15. doi: 10.1002/chp.20001.
- Mueser KT, Penn DL, Addington J, Brunette MF, Gingerich S, Glynn SM, Lynde DW, Gottlieb JD, Meyer-Kalos P, McGurk SR, Cather C, Saade S, Robinson DG, Schooler NR, Rosenheck RA, Kane JM. The NAVIGATE Program for First-Episode Psychosis: Rationale, Overview, and Description of Psychosocial Components. Psychiatr Serv. 2015 Jul;66(7):680-90. doi: 10.1176/appi.ps.201400413. Epub 2015 Mar 16.
- Nagendra A, Schooler NR, Kane JM, Robinson DG, Mueser KT, Estroff SE, Addington J, Marcy P, Penn DL. Demographic, psychosocial, clinical, and neurocognitive baseline characteristics of Black Americans in the RAISE-ETP study. Schizophr Res. 2018 Mar;193:64-68. doi: 10.1016/j.schres.2017.06.038. Epub 2017 Jul 11.
- Nilsen L, Norheim I, Frich JC, Friis S, Røssberg JI. Challenges for group leaders working with families dealing with early psychosis: a qualitative study. BMC Psychiatry. 2015 Jul 2;15:141. doi: 10.1186/s12888-015-0540-8.
- Nolin M, Malla A, Tibbo P, Norman R, Abdel-Baki A. Early Intervention for Psychosis in Canada: What Is the State of Affairs? Can J Psychiatry. 2016 Mar;61(3):186-94. doi: 10.1177/0706743716632516.
- Petersen L, Jeppesen P, Thorup A, Abel MB, Øhlenschlaeger J, Christensen TØ, Krarup G, Jørgensen P, Nordentoft M. A randomised multicentre trial of integrated versus standard treatment for patients with a first episode of psychotic illness. BMJ. 2005 Sep 17;331(7517):602. Epub 2005 Sep 2. Erratum in: BMJ. 2005 Nov 5;331(7524):1065.
- Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009 Jan;36(1):24-34. doi: 10.1007/s10488-008-0197-4. Epub 2008 Dec 23.
- Rosenheck R, Leslie D, Sint K, Lin H, Robinson DG, Schooler NR, Mueser KT, Penn DL, Addington J, Brunette MF, Correll CU, Estroff SE, Marcy P, Robinson J, Severe J, Rupp A, Schoenbaum M, Kane JM. Cost-Effectiveness of Comprehensive, Integrated Care for First Episode Psychosis in the NIMH RAISE Early Treatment Program. Schizophr Bull. 2016 Jul;42(4):896-906. doi: 10.1093/schbul/sbv224. Epub 2016 Jan 31.
- Rosenheck R, Mueser KT, Sint K, Lin H, Lynde DW, Glynn SM, Robinson DG, Schooler NR, Marcy P, Mohamed S, Kane JM. Supported employment and education in comprehensive, integrated care for first episode psychosis: Effects on work, school, and disability income. Schizophr Res. 2017 Apr;182:120-128. doi: 10.1016/j.schres.2016.09.024. Epub 2016 Sep 23.
- Saunders NR, Lebenbaum M, Stukel TA, Lu H, Urquia ML, Kurdyak P, Guttmann A. Suicide and self-harm trends in recent immigrant youth in Ontario, 1996-2012: a population-based longitudinal cohort study. BMJ Open. 2017 Sep 1;7(9):e014863. doi: 10.1136/bmjopen-2016-014863.
- Schoenbaum M, Sutherland JM, Chappel A, Azrin S, Goldstein AB, Rupp A, Heinssen RK. Twelve-Month Health Care Use and Mortality in Commercially Insured Young People With Incident Psychosis in the United States. Schizophr Bull. 2017 Oct 21;43(6):1262-1272. doi: 10.1093/schbul/sbx009.
- Scott VC, Kenworthy T, Godly-Reynolds E, Bastien G, Scaccia J, McMickens C, Rachel S, Cooper S, Wrenn G, Wandersman A. The Readiness for Integrated Care Questionnaire (RICQ): An instrument to assess readiness to integrate behavioral health and primary care. Am J Orthopsychiatry. 2017;87(5):520-530. doi: 10.1037/ort0000270. Epub 2017 Apr 10.
- Sockalingam S, Arena A, Serhal E, Mohri L, Alloo J, Crawford A. Building Provincial Mental Health Capacity in Primary Care: An Evaluation of a Project ECHO Mental Health Program. Acad Psychiatry. 2018 Aug;42(4):451-457. doi: 10.1007/s40596-017-0735-z. Epub 2017 Jun 7.
- Soleimani A, Rosychuk RJ, Newton AS. Predicting time to emergency department re-visits and inpatient hospitalization among adolescents who visited an emergency department for psychotic symptoms: a retrospective cohort study. BMC Psychiatry. 2016 Nov 9;16(1):385.
- Steele LS, Glazier RH, Lin E, Evans M. Using administrative data to measure ambulatory mental health service provision in primary care. Med Care. 2004 Oct;42(10):960-5.
- Tambuyzer E, Pieters G, Van Audenhove C. Patient involvement in mental health care: one size does not fit all. Health Expect. 2014 Feb;17(1):138-50. doi: 10.1111/j.1369-7625.2011.00743.x. Epub 2011 Nov 10.
- Whitbeck LB, Sittner Hartshorn KJ, Crawford DM, Walls ML, Gentzler KC, Hoyt DR. Mental and substance use disorders from early adolescence to young adulthood among indigenous young people: final diagnostic results from an 8-year panel study. Soc Psychiatry Psychiatr Epidemiol. 2014 Jun;49(6):961-73. doi: 10.1007/s00127-014-0825-0. Epub 2014 Feb 2.
- Wodchis WP, Austin PC, Henry DA. A 3-year study of high-cost users of health care. CMAJ. 2016 Feb 16;188(3):182-188. doi: 10.1503/cmaj.150064. Epub 2016 Jan 11.
- Zarate CA Jr, Weinstock L, Cukor P, Morabito C, Leahy L, Burns C, Baer L. Applicability of telemedicine for assessing patients with schizophrenia: acceptance and reliability. J Clin Psychiatry. 1997 Jan;58(1):22-5.
- Zeldin S, Christens BD, Powers JL. The psychology and practice of youth-adult partnership: bridging generations for youth development and community change. Am J Community Psychol. 2013 Jun;51(3-4):385-97. doi: 10.1007/s10464-012-9558-y.
- Zhou C, Crawford A, Serhal E, Kurdyak P, Sockalingam S. The Impact of Project ECHO on Participant and Patient Outcomes: A Systematic Review. Acad Med. 2016 Oct;91(10):1439-1461.
- MY9-158586