Patient Portal Reminder/Recall for Influenza Vaccination in a Health System- RCT 2

• Sponsor: University of California, Los Angeles (Other)
• Overall Status: Completed
• CT.gov ID: NCT04110314
• Collaborator: (none)
• Enrollment: 430,636
• Location: 1
• Arms: 6
• Actual Duration (Months): 5.5
• Patients Per Site Per Month: 78960.7

Study Details

Study Description

Brief Summary

This trial is taking place in Los Angeles, CA among patients from primary care practices within the UCLA Health System.

Despite the Advisory Committee on Immunization Practices (ACIP) recommendation in 2010 that all people above 6 months of age should receive an annual flu vaccine, vaccination rates remain low. The investigators will assess the effectiveness of message-framing (gain-framed, loss-framed messages, no messages), as well as the effectiveness of a pre-commitment prompt (pre-commitment prompt, no prompt) asking about a patient's intention to get the influenza vaccination, using a 3 x 2 factorial design.

Condition or Disease	Intervention/Treatment	Phase
Influenza; Respiratory Tract Infections	Behavioral: Portal Reminders for Influenza Vaccination: Gain-framed; Behavioral: Portal Reminders for Influenza Vaccination: Loss-framed; Behavioral: Pre-commitment prompt	N/A

Detailed Description

Sub-optimal vaccination rates are a significant problem in the U.S., despite their effectiveness in preventing morbidity and mortality from vaccine-preventable illness. For influenza specifically, annual epidemics of influenza cause substantial morbidity in the U.S. with up to 40,00-80,000 deaths/year and many hospitalizations, emergency and outpatient visits, and significant costs.

Reminder/recall (R/R), sent by phone, mail or other modality, can improve child and adult influenza vaccination rates. However, the majority of pediatric or adult primary care practices to not conduct R/R. Barriers are lack of finances, personnel, and algorithms to identify eligible patients.

A technological breakthrough that might overcome these barriers involves patient portals-- secure, web-based communication systems, embedded within electronic health records (EHRs), for patients and providers to communicate with each other via email and the internet. Portals are used by about half of Americans and half of UCLA patients.

There has been considerable research on how best to communicate about vaccines with families. A number of studies have examined the impact of gain-framed (messages emphasizing the benefits of vaccination) versus loss-framed (messages emphasizing the risks of not getting vaccinated); however, the findings have been inconsistent across studies and by sub-population.

In addition to framing, another potential influence on the decision to vaccinate is pre-commitment. Previous studies suggest that, upon being prompted, indicating an intention to do something increases the likelihood of following through. Therefore, a pre-commitment prompt is being asked of half the participants in each arm to evaluate its impact on influenza vaccination.

The purpose of this randomized controlled trial is to evaluate the impact of gain-framed and loss-framed reminder-recall messages sent via the patient portal, pre-commitment prompts sent via the patient portal, and the interactions between the two type of messages and pre-commitment on influenza vaccination among UCLA Health System's primary care patients aged 6 months and older.

The proposed design of this randomized-controlled trial is a 3 x 2 factorial design.

Standard of care control (no reminder messages, no pre-commitment prompt) No portal messages, pre-commitment prompt Gain-framed portal messages, pre-commitment prompt Gain-framed portal messages, no pre-commitment prompt Loss-framed portal messages, pre-commitment prompt Loss-framed portal messages, no pre-commitment prompt

Hyp 1. Vaccination rates will differ by receipt of no reminders, gain-framed reminders and loss-framed reminders.

1a. Vaccination rates will be higher among patients receiving gain-framed reminders than among patients receiving no reminders.

1b. Vaccination rates will be higher among patients receiving loss-framed reminders than among patients receiving no reminders.

1c. Vaccination rates will be higher among patients receiving gain-framed reminders than among patients receiving loss-framed reminders.

Hyp 2. Vaccination rates will be higher among patients receiving a pre-commitment prompt than among patients not receiving any prompt.

For relevant study arms, the first R/R messages will be sent in October 2019.

Study Design

• Study Type: Interventional
• Actual Enrollment: 430636 participants
• Allocation: Randomized
• Intervention Model: Factorial Assignment
• Masking: Single (Participant)
• Primary Purpose: Health Services Research
• Official Title: Improving Influenza Vaccination Delivery Across a Health System by the Electronic Health Records Patient Portal- RCT #2
• Actual Study Start Date: Oct 18, 2019
• Actual Primary Completion Date: Apr 1, 2020
• Actual Study Completion Date: Apr 1, 2020

Arms and Interventions

Arm	Intervention/Treatment
Active Comparator: Gain-framed portal reminders + Pre-commitment Prompts Participants receive gain-framed reminder/recall messages regarding influenza vaccination via the patient portal and a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season	Behavioral: Portal Reminders for Influenza Vaccination: Gain-framed; Patients receive gain-framed reminder/recall messages via the patient portal to get an influenza vaccination.; Behavioral: Pre-commitment prompt; Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination
Active Comparator: Gain-framed portal reminders + No pre-commitment prompt Participants receive gain-framed reminder/recall messages regarding influenza vaccination via the patient portal and no pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season	Behavioral: Portal Reminders for Influenza Vaccination: Gain-framed; Patients receive gain-framed reminder/recall messages via the patient portal to get an influenza vaccination.
Active Comparator: Loss-framed portal reminders + Pre-commitment prompt Participants receive loss-framed reminder/recall messages regarding influenza vaccination via the patient portal and a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season	Behavioral: Portal Reminders for Influenza Vaccination: Loss-framed; Patients receive loss-framed reminder/recall messages via the patient portal to get an influenza vaccination.; Behavioral: Pre-commitment prompt; Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination
Active Comparator: Loss-framed portal reminders + No pre-commitment prompt Participants receive loss-framed reminder/recall messages regarding influenza vaccination via the patient portal and a no pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season	Behavioral: Portal Reminders for Influenza Vaccination: Loss-framed; Patients receive loss-framed reminder/recall messages via the patient portal to get an influenza vaccination.
Active Comparator: No portal reminder + Pre-commitment prompt Participants do not receive any reminder/recall messages regarding influenza vaccination via the patient portal but do receive a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season	Behavioral: Pre-commitment prompt; Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination
No Intervention: No portal reminders + No pre-commitment prompt Participants do not receive any reminder/recall messages regarding influenza vaccination via the patient portal and do not receive a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season

Outcome Measures

Primary Outcome Measures

1. Receipt of the Annual Influenza Vaccine Among Index Patients. [6 months]

   Receipt of the annual influenza vaccination (between 10/2/19 - 4/1/20) among index patients. Outcomes will be assessed via vaccine data extraction from the electronic health record and external claims and pharmacy data. Index patients will be randomly selected individuals within an included household, eligible for the following groups (<18, 18-64 non-diabetic, 65+ non-diabetic, and 18+ diabetic). One index patient per group may be selected from each household. The index patients must also be an active UCLA Health MyChart user (>= 1 login over the last 12 months from 8/1/19, excluding activity on the user's initial profile activation date). Individuals not affiliated with any primary care practice will be excluded from the primary analysis.

Eligibility Criteria

Criteria

• Ages Eligible for Study: 4 Months and Older
• Sexes Eligible for Study: All
• Accepts Healthy Volunteers: No

Inclusion Criteria:

• A patient within the UCLA Health System identified as a primary care patient per an internal algorithm,

Exclusion Criteria:

• A patient within the UCLA Health System not identified as a primary care patient per an internal algorithm

Contacts and Locations

Locations

	Site	City	State	Country	Postal Code
1	University of California LA	Los Angeles	California	United States	99095

Sponsors and Collaborators

• University of California, Los Angeles

Investigators

• Principal Investigator: Peter G Szilagyi, MPH, MD, University of California, Los Angeles

Study Documents (Full-Text)

• Study Protocol and Statistical Analysis Plan - Oct 9, 2019

More Information

Publications

None provided.

• Responsible Party: Peter G Szilagyi, MD MPH, Principal Investigator, University of California, Los Angeles
• ClinicalTrials.gov Identifier: NCT04110314
• Other Study ID Numbers: R01AI135029
• First Posted: Oct 1, 2019
• Last Update Posted: May 5, 2022
• Last Verified: Apr 1, 2022

• Studies a U.S. FDA-regulated Drug Product: No
• Studies a U.S. FDA-regulated Device Product: No
• Keywords provided by Peter G Szilagyi, MD MPH, Principal Investigator, University of California, Los Angeles: Vaccination; Reminder recall; Patient Portal
• Additional relevant MeSH terms: Influenza, Human; Respiratory Tract Infections

Study Results

Participant Flow

Arm/Group Title	Gain-framed Portal Reminders + Pre-commitment Prompts	Gain-framed Portal Reminders + No Pre-commitment Prompt	Loss-framed Portal Reminders + Pre-commitment Prompt	Loss-framed Portal Reminders + No Pre-commitment Prompt	No Portal Reminder + Pre-commitment Prompt	No Portal Reminders + No Pre-commitment Prompt
Arm/Group Description	Participants receive gain-framed reminder/recall messages regarding influenza vaccination via the patient portal and a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Gain-framed: Patients receive gain-framed reminder/recall messages via the patient portal to get an influenza vaccination. Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination	Participants receive gain-framed reminder/recall messages regarding influenza vaccination via the patient portal and no pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Gain-framed: Patients receive gain-framed reminder/recall messages via the patient portal to get an influenza vaccination.	Participants receive loss-framed reminder/recall messages regarding influenza vaccination via the patient portal and a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Loss-framed: Patients receive loss-framed reminder/recall messages via the patient portal to get an influenza vaccination. Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination	Participants receive loss-framed reminder/recall messages regarding influenza vaccination via the patient portal and a no pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Loss-framed: Patients receive loss-framed reminder/recall messages via the patient portal to get an influenza vaccination.	Participants do not receive any reminder/recall messages regarding influenza vaccination via the patient portal but do receive a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination	Participants do not receive any reminder/recall messages regarding influenza vaccination via the patient portal and do not receive a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season
Period Title: Overall Study
STARTED	71781	71492	72177	71512	71536	72138
Patients Who Met Inclusion	36526	36388	36541	36377	36289	36411
COMPLETED	36526	36388	36541	36377	36289	36411
NOT COMPLETED	35255	35104	35636	35135	35247	35727

Baseline Characteristics

Arm/Group Title	Gain-framed Portal Reminders + Pre-commitment Prompts	Gain-framed Portal Reminders + No Pre-commitment Prompt	Loss-framed Portal Reminders + Pre-commitment Prompt	Loss-framed Portal Reminders + No Pre-commitment Prompt	No Portal Reminder + Pre-commitment Prompt	No Portal Reminders + No Pre-commitment Prompt	Total
Arm/Group Description	Participants receive gain-framed reminder/recall messages regarding influenza vaccination via the patient portal and a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Gain-framed: Patients receive gain-framed reminder/recall messages via the patient portal to get an influenza vaccination. Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination	Participants receive gain-framed reminder/recall messages regarding influenza vaccination via the patient portal and no pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Gain-framed: Patients receive gain-framed reminder/recall messages via the patient portal to get an influenza vaccination.	Participants receive loss-framed reminder/recall messages regarding influenza vaccination via the patient portal and a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Loss-framed: Patients receive loss-framed reminder/recall messages via the patient portal to get an influenza vaccination. Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination	Participants receive loss-framed reminder/recall messages regarding influenza vaccination via the patient portal and a no pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Loss-framed: Patients receive loss-framed reminder/recall messages via the patient portal to get an influenza vaccination.	Participants do not receive any reminder/recall messages regarding influenza vaccination via the patient portal but do receive a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination	Participants do not receive any reminder/recall messages regarding influenza vaccination via the patient portal and do not receive a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season	Total of all reporting groups
Overall Participants	36526	36388	36541	36377	36289	36411	218532
Age (Count of Participants)
<=18 years	3686 10.1%	3632 10%	3727 10.2%	3610 9.9%	3655 10.1%	3736 10.3%	22046 10.1%
Between 18 and 65 years	26288 72%	26191 72%	26265 71.9%	26170 71.9%	26180 72.1%	26077 71.6%	157171 71.9%
>=65 years	6552 17.9%	6565 18%	6549 17.9%	6597 18.1%	6454 17.8%	6598 18.1%	39315 18%
Sex/Gender, Customized (Count of Participants)
Female Female	21088 57.7%	20881 57.4%	21058 57.6%	21057 57.9%	20947 57.7%	20835 57.2%	125866 57.6%
Male	15437 42.3%	15506 42.6%	15482 42.4%	15319 42.1%	15342 42.3%	15575 42.8%	92661 42.4%
Other	0 0%	0 0%	1 0%	0 0%	0 0%	0 0%	1 0%
Unknown	1 0%	1 0%	0 0%	1 0%	0 0%	1 0%	4 0%
Race/Ethnicity, Customized (Count of Participants)
White	20096 55%	20133 55.3%	20115 55%	20056 55.1%	20401 56.2%	20401 56%	121202 55.5%
Black/African American	1674 4.6%	1666 4.6%	1657 4.5%	1649 4.5%	1727 4.8%	1533 4.2%	9906 4.5%
Asian	3867 10.6%	3789 10.4%	3886 10.6%	3900 10.7%	3715 10.2%	3828 10.5%	22985 10.5%
Other/Unknown	10889 29.8%	10800 29.7%	10883 29.8%	10772 29.6%	10815 29.8%	10649 29.2%	64808 29.7%
Region of Enrollment (participants) [Number]
United States	36526 100%	36388 100%	36541 100%	36377 100%	36289 100%	36411 100%	218532 100%
Insurance Type (Count of Participants)
Private	31307 85.7%	31164 85.6%	31302 85.7%	31129 85.6%	31148 85.8%	31167 85.6%	187217 85.7%
Public	4642 12.7%	4646 12.8%	4671 12.8%	4690 12.9%	4584 12.6%	4675 12.8%	27908 12.8%
Other/UNK	577 1.6%	578 1.6%	568 1.6%	558 1.5%	557 1.5%	569 1.6%	3407 1.6%
Adults with diabetes (Count of Participants)
Count of Participants [Participants]	3660 10%	3644 10%	3663 10%	3485 9.6%	3534 9.7%	3539 9.7%	21525 9.8%

Outcome Measures

1. Primary Outcome

Title	Receipt of the Annual Influenza Vaccine Among Index Patients.
Description	Receipt of the annual influenza vaccination (between 10/2/19 - 4/1/20) among index patients. Outcomes will be assessed via vaccine data extraction from the electronic health record and external claims and pharmacy data. Index patients will be randomly selected individuals within an included household, eligible for the following groups (<18, 18-64 non-diabetic, 65+ non-diabetic, and 18+ diabetic). One index patient per group may be selected from each household. The index patients must also be an active UCLA Health MyChart user (>= 1 login over the last 12 months from 8/1/19, excluding activity on the user's initial profile activation date). Individuals not affiliated with any primary care practice will be excluded from the primary analysis.
Time Frame	6 months

Outcome Measure Data

Analysis Population Description

Each row represents a combination of intervention conditions in our factorial design. Each column represents patient subpopulations which received different versions of the intervention. The number analyzed in each cell is the number from that column's subpopulation assigned to that row's intervention combination.

Arm/Group Title	Young Adults 18-64yr Without Diabetes	Older Adults Greater Than or Equal to 65 yr Without Diabetes	Adults 18years or Older With Diabetes	Children <18y
Arm/Group Description	Participants ages 18-64 Without Diabetes	Older adults with diabetes greater than or equal to 65 years without diabetes.	Adults 18 years or older with Diabetes	Participants age <18
Measure Participants	145166	29795	21525	22046
No reminders and no pre-commitment	8804 24.1%	2795 7.7%	2163 5.9%	2123 5.8%
Loss-framed reminders and no pre-commitment	8880 24.3%	2752 7.6%	2077 5.7%	2056 5.7%
Gain-frame reminders and no pre-commitment	8796 24.1%	2772 7.6%	2183 6%	2076 5.7%
No reminders and pre-commitment	8937 24.5%	2757 7.6%	2144 5.9%	2084 5.7%
Loss-frame reminders and pre-commitment	9055 24.8%	2753 7.6%	2220 6.1%	2203 6.1%
Gain-frame reminders and pre-commitment	8883 24.3%	2741 7.5%	2249 6.2%	2168 6%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Young Adults 18-64yr Without Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	1.01
	Confidence Interval	(2-Sided) 95%; 1.00 to 1.03
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving a pre-commitment prompt to the arm receiving no pre-commitment prompt.

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Older Adults Greater Than or Equal to 65 yr Without Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	1.00
	Confidence Interval	(2-Sided) 95%; 0.99 to 1.01
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving a pre-commitment prompt to the arm receiving no pre-commitment prompt.

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Adults 18years or Older With Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	1.01
	Confidence Interval	(2-Sided) 95%; 0.99 to 1.03
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving a pre-commitment prompt to the arm receiving no pre-commitment prompt.

Statistical Analysis 4

Statistical Analysis Overview	Comparison Group Selection	Children <18y
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	1.02
	Confidence Interval	(2-Sided) 95%; 1.00 to 1.04
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving a pre-commitment prompt to the arm receiving no pre-commitment prompt.

Statistical Analysis 5

Statistical Analysis Overview	Comparison Group Selection	Young Adults 18-64yr Without Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	0.99
	Confidence Interval	(2-Sided) 95%; 0.98 to 1.01
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving gain-framed reminders to the arm receiving no reminders.

Statistical Analysis 6

Statistical Analysis Overview	Comparison Group Selection	Older Adults Greater Than or Equal to 65 yr Without Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	0.99
	Confidence Interval	(2-Sided) 0.95%; 0.97 to 1.02
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving gain-framed reminders to the arm receiving no reminders.

Statistical Analysis 7

Statistical Analysis Overview	Comparison Group Selection	Adults 18years or Older With Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	0.99
	Confidence Interval	(2-Sided) 95%; 0.97 to 1.01
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving gain-framed reminders to the arm receiving no reminders.

Statistical Analysis 8

Statistical Analysis Overview	Comparison Group Selection	Children <18y
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	1.01
	Confidence Interval	(2-Sided) 95%; 0.98 to 1.05
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving gain-framed reminders to the arm receiving no reminders.

Statistical Analysis 9

Statistical Analysis Overview	Comparison Group Selection	Young Adults 18-64yr Without Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	1.01
	Confidence Interval	(2-Sided) 95%; 0.99 to 1.02
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving loss-framed reminders to the arm receiving no reminders.

Statistical Analysis 10

Statistical Analysis Overview	Comparison Group Selection	Older Adults Greater Than or Equal to 65 yr Without Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	1.00
	Confidence Interval	(2-Sided) 95%; 0.98 to 1.02
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving loss-framed reminders to the arm receiving no reminders.

Statistical Analysis 11

Statistical Analysis Overview	Comparison Group Selection	Adults 18years or Older With Diabetes
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	0.99
	Confidence Interval	(2-Sided) 95%; 0.97 to 1.02
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving loss-framed reminders to the arm receiving no reminders.

Statistical Analysis 12

Statistical Analysis Overview	Comparison Group Selection	Children <18y
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value
	Comments
	Method
	Comments
Method of Estimation	Estimation Parameter	Risk Ratio (RR)
	Estimated Value	1.02
	Confidence Interval	(2-Sided) 95%; 1.00 to 1.04
	Parameter Dispersion	Type: Value:
	Estimation Comments	These results compare the arm receiving loss-framed reminders to the arm receiving no reminders.

Adverse Events

Time Frame	We did not collect information regarding adverse events.
Adverse Event Reporting Description	We did not collect adverse event information. We monitored whether reminder recall through the portal was effective at increased influenza vaccination rates. Because influenza vaccination is recommended by the Advisory Committee on Immunization Practices and is part of the U.S. recommended vaccine schedule, we did not collect adverse event data.
Arm/Group Title	Gain-framed Portal Reminders + Pre-commitment Prompts		Gain-framed Portal Reminders + No Pre-commitment Prompt		Loss-framed Portal Reminders + Pre-commitment Prompt		Loss-framed Portal Reminders + No Pre-commitment Prompt		No Portal Reminder + Pre-commitment Prompt		No Portal Reminders + No Pre-commitment Prompt
Arm/Group Description	Participants receive gain-framed reminder/recall messages regarding influenza vaccination via the patient portal and a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Gain-framed: Patients receive gain-framed reminder/recall messages via the patient portal to get an influenza vaccination. Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination		Participants receive gain-framed reminder/recall messages regarding influenza vaccination via the patient portal and no pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Gain-framed: Patients receive gain-framed reminder/recall messages via the patient portal to get an influenza vaccination.		Participants receive loss-framed reminder/recall messages regarding influenza vaccination via the patient portal and a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Loss-framed: Patients receive loss-framed reminder/recall messages via the patient portal to get an influenza vaccination. Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination		Participants receive loss-framed reminder/recall messages regarding influenza vaccination via the patient portal and a no pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Portal Reminders for Influenza Vaccination: Loss-framed: Patients receive loss-framed reminder/recall messages via the patient portal to get an influenza vaccination.		Participants do not receive any reminder/recall messages regarding influenza vaccination via the patient portal but do receive a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season Pre-commitment prompt: Patients receive a pre-commitment prompt asking about their intention to get an influenza vaccination		Participants do not receive any reminder/recall messages regarding influenza vaccination via the patient portal and do not receive a pre-commitment prompt asking if they plan to receive the influenza vaccine in the upcoming season
All Cause Mortality
	Gain-framed Portal Reminders + Pre-commitment Prompts		Gain-framed Portal Reminders + No Pre-commitment Prompt		Loss-framed Portal Reminders + Pre-commitment Prompt		Loss-framed Portal Reminders + No Pre-commitment Prompt		No Portal Reminder + Pre-commitment Prompt		No Portal Reminders + No Pre-commitment Prompt
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)
Serious Adverse Events
	Gain-framed Portal Reminders + Pre-commitment Prompts		Gain-framed Portal Reminders + No Pre-commitment Prompt		Loss-framed Portal Reminders + Pre-commitment Prompt		Loss-framed Portal Reminders + No Pre-commitment Prompt		No Portal Reminder + Pre-commitment Prompt		No Portal Reminders + No Pre-commitment Prompt
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)
Other (Not Including Serious) Adverse Events
	Gain-framed Portal Reminders + Pre-commitment Prompts		Gain-framed Portal Reminders + No Pre-commitment Prompt		Loss-framed Portal Reminders + Pre-commitment Prompt		Loss-framed Portal Reminders + No Pre-commitment Prompt		No Portal Reminder + Pre-commitment Prompt		No Portal Reminders + No Pre-commitment Prompt
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)

Limitations/Caveats

[Not Specified]

More Information

Certain Agreements

Principal Investigators are NOT employed by the organization sponsoring the study.

There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.