Email Campaign to Increase Uptake of myGeisinger

• Sponsor: Geisinger Clinic (Other)
• Overall Status: Completed
• CT.gov ID: NCT04332614
• Collaborator: (none)
• Enrollment: 14,099
• Location: 1
• Actual Duration (Months): 1.1
• Patients Per Site Per Month: 13410.6

Study Details

Study Description

Brief Summary

The purpose of the current study is to test different email messages to determine the most effect way of promoting enrollment in Geisinger's patient portal, called myGeisinger.

Condition or Disease	Intervention/Treatment	Phase
Telehealth; Health Behavior	Behavioral: Less-is-better; Behavioral: Social proof; Behavioral: Endowment / decision staging; Behavioral: Standard email

Detailed Description

Online patient portals are convenient tools that improve patient access to healthcare services while often reducing burden to both patients and providers. However, fewer than half of patients have enrolled in Geisinger's patient portal, myGeisinger.

The purpose of the current study is to assess what kind of messaging improves the effectiveness of an email campaign targeted at enrolling patients into Geisinger's online patient portal, called myGeisinger. Currently, Geisinger sends an email every month to patients who have started but not completed the enrollment process (they have had an activation code generated but have not yet used that code to enroll). This study will A/B test 5 email messages to assess if they perform better than the current standard email message.

Performance will be assessed based on whether patients open the email, click on the myGeisinger enrollment link, and enroll in myGeisinger. Statistical analyses will employ generalized linear models with a binary distribution and log-link function.

Study Design

• Study Type: Observational
• Actual Enrollment: 14099 participants
• Observational Model: Case-Control
• Time Perspective: Prospective
• Official Title: Email Campaign to Encourage Patients to Complete Sign-up Process for Geisinger's Patient Portal
• Actual Study Start Date: Apr 6, 2020
• Actual Primary Completion Date: Apr 13, 2020
• Actual Study Completion Date: May 8, 2020

Arms and Interventions

Arm	Intervention/Treatment
Control A standard marketing message encouraging activation and describing the benefits of myGeisinger	Behavioral: Standard email; Email
Focused on provider communication A simple message focused on one benefit of myGeisinger: communicating easily with providers	Behavioral: Less-is-better; Email
Focused on scheduling A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online	Behavioral: Less-is-better; Email
Focused on medical information access A simple message focused on one benefit of myGeisinger: accessing medical information, like test results	Behavioral: Less-is-better; Email
Social proof A message similar to control, but including information about how many other patients are using myGeisinger	Behavioral: Social proof; Email
Endowment / decision staging A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete	Behavioral: Endowment / decision staging; Email

Outcome Measures

Primary Outcome Measures

1. Email Opened: Control vs. Less-is-better vs. Social Proof vs. Endowment [8-10 days post-intervention]

   Whether email was opened (yes / no)

2. Link Clicked: Control vs. Less-is-better vs. Social Proof vs. Endowment [8-10 days post-intervention]

   Whether the link / button to start the activation process was clicked (yes / no)

3. Enrollment: Control vs. Less-is-better vs. Social Proof vs. Endowment [8-10 days post-intervention]

   Whether the patient enrolled in myGeisinger (yes / no)

Secondary Outcome Measures

1. Email Opened: Control vs. Less-is-better vs. Social Proof vs. Endowment [32-35 days post-intervention]

   Whether email was opened (yes / no)

2. Link Clicked: Control vs. Less-is-better vs. Social Proof vs. Endowment [32-35 days post-intervention]

   Whether the link / button to start the activation process was clicked (yes / no)

3. Enrollment: Control vs. Less-is-better vs. Social Proof vs. Endowment [32-35 days post-intervention]

   Whether the patient enrolled in myGeisinger (yes / no)

4. Email Opened: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access) [8-10 days post-intervention]

   Whether email was opened (yes / no)

5. Link Clicked: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access) [8-10 days post-intervention]

   Whether the link / button to start the activation process was clicked (yes / no)

6. Enrollment: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access) [8-10 days post-intervention]

   Whether the patient enrolled in myGeisinger (yes / no)

7. Email Opened: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access) [32-35 days post-intervention]

   Whether email was opened (yes / no)

8. Link Clicked: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access) [32-35 days post-intervention]

   Whether the link / button to start the activation process was clicked (yes / no)

9. Enrollment: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access) [32-35 days post-intervention]

   Whether the patient enrolled in myGeisinger (yes / no)

Eligibility Criteria

Criteria

• Ages Eligible for Study: 18 Years to 88 Years
• Sexes Eligible for Study: All
• Accepts Healthy Volunteers: No

Inclusion Criteria:

• Geisinger patients whose myGeisinger status is "Pending", meaning an activation code has been generated for them, but they have not completed activation

Exclusion Criteria:

• Patients without email addresses on file

• Patients who received emails from this monthly myGeisinger marketing campaign within the last 2 months

Contacts and Locations

Locations

	Site	City	State	Country	Postal Code
1	Geisinger	Danville	Pennsylvania	United States	17822

Sponsors and Collaborators

• Geisinger Clinic

Investigators

• Principal Investigator: Amir Goren, PhD, Geisinger Clinic

Study Documents (Full-Text)

• Study Protocol and Statistical Analysis Plan - Apr 5, 2021

More Information

Publications

None provided.

• Responsible Party: Amir Goren, Program Director, Behavioral Insights Team, Geisinger Clinic
• ClinicalTrials.gov Identifier: NCT04332614
• Other Study ID Numbers: 2020-0315
• First Posted: Apr 2, 2020
• Last Update Posted: May 3, 2021
• Last Verified: Apr 1, 2021

• Individual Participant Data (IPD) Sharing Statement: Yes
• Plan to Share IPD: Yes
• Studies a U.S. FDA-regulated Drug Product: No
• Studies a U.S. FDA-regulated Device Product: No
• Keywords provided by Amir Goren, Program Director, Behavioral Insights Team, Geisinger Clinic: Behavioral Economics; Patient Portal; Social Proof; Endowment Effect; Less-is-better

Study Results

Participant Flow

Arm/Group Title	Control	Focused on Provider Communication	Focused on Scheduling	Focused on Medical Information Access	Social Proof	Endowment / Decision Staging
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger Standard email: Email	A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email	A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email	A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email	A message similar to control, but including information about how many other patients are using myGeisinger Social proof: Email	A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete Endowment / decision staging: Email
Period Title: ~1 Week After Study Launch
STARTED	2350	2350	2350	2350	2350	2349
Data in EHR	2245	2273	2251	2254	2251	2249
COMPLETED	2117	2129	2124	2131	2127	2115
NOT COMPLETED	233	221	226	219	223	234
Period Title: ~1 Week After Study Launch
STARTED	2117	2129	2124	2131	2127	2115
COMPLETED	2095	2107	2094	2107	2107	2096
NOT COMPLETED	22	22	30	24	20	19

Baseline Characteristics

Arm/Group Title	Control	Focused on Provider Communication	Focused on Scheduling	Focused on Medical Information Access	Social Proof	Endowment / Decision Staging	Total
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger Standard email: Email	A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email	A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email	A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email	A message similar to control, but including information about how many other patients are using myGeisinger Social proof: Email	A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete Endowment / decision staging: Email	Total of all reporting groups
Overall Participants	2350	2350	2350	2350	2350	2349	14099
Age (Count of Participants)
<=18 years	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN
Between 18 and 65 years	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN
>=65 years	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN
Sex: Female, Male (Count of Participants)
Female	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN
Male	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN	NA NaN
Race and Ethnicity Not Collected (Count of Participants)
Count of Participants [Participants]	0 0%
Region of Enrollment (participants) [Number]
United States	2350 100%	2350 100%	2350 100%	2350 100%	2350 100%	2349 100%	14099 100%

Outcome Measures

1. Primary Outcome

Title	Email Opened: Control vs. Less-is-better vs. Social Proof vs. Endowment
Description	Whether email was opened (yes / no)
Time Frame	8-10 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who were delivered an email (i.e. email did not bounce back) and for whom there was a record in the EHR 10 days post-intervention.

Arm/Group Title	Control	Less-is-better	Social Proof	Endowment / Decision Staging
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger	A simple message focused on one benefit of myGeisinger.	A message similar to control, but including information about how many other patients are using myGeisinger	A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete
Measure Participants	2117	6384	2127	2115
Count of Participants [Participants]	558 23.7%	1528 65%	375 16%	488 20.8%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0248
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8791371
	Confidence Interval	(2-Sided) 95%; 0.7856042 to 0.9838058
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	<0.0001
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.5980109
	Confidence Interval	(2-Sided) 95%; 0.5159511 to 0.6931219
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Control, Endowment / Decision Staging
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0133
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8380003
	Confidence Interval	(2-Sided) 95%; 0.7285712 to 0.9638653
	Parameter Dispersion	Type: Value:
	Estimation Comments

2. Primary Outcome

Title	Link Clicked: Control vs. Less-is-better vs. Social Proof vs. Endowment
Description	Whether the link / button to start the activation process was clicked (yes / no)
Time Frame	8-10 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who opened the intervention email and for whom there was a record in the EHR 10 days post-intervention.

Arm/Group Title	Control	Less-is-better	Social Proof	Endowment / Decision Staging
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger	A simple message focused on one benefit of myGeisinger.	A message similar to control, but including information about how many other patients are using myGeisinger	A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete
Measure Participants	558	1528	375	488
Count of Participants [Participants]	134 5.7%	160 6.8%	31 1.3%	73 3.1%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	<0.0001
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.3700794
	Confidence Interval	(2-Sided) 95%; 0.2870513 to 0.4771231
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	<0.0001
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.2851440
	Confidence Interval	(2-Sided) 95%; 0.1881589 to 0.4321194
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Control, Endowment / Decision Staging
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0003
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.5565905
	Confidence Interval	(2-Sided) 95%; 0.4059477 to 0.7631354
	Parameter Dispersion	Type: Value:
	Estimation Comments

3. Primary Outcome

Title	Enrollment: Control vs. Less-is-better vs. Social Proof vs. Endowment
Description	Whether the patient enrolled in myGeisinger (yes / no)
Time Frame	8-10 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who opened the intervention email and for whom there was a record in the EHR 10 days post-intervention.

Arm/Group Title	Control	Less-is-better	Social Proof	Endowment / Decision Staging
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger	A simple message focused on one benefit of myGeisinger.	A message similar to control, but including information about how many other patients are using myGeisinger	A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete
Measure Participants	558	1528	375	488
Count of Participants [Participants]	41 1.7%	83 3.5%	18 0.8%	38 1.6%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.103
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.72429741
	Confidence Interval	(2-Sided) 95%; 0.49167734 to 1.0669736
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.120
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.63578603
	Confidence Interval	(2-Sided) 95%; 0.35943749 to 1.1246013
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Control, Endowment / Decision Staging
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.789
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.06482385
	Confidence Interval	(2-Sided) 95%; 0.67282661 to 1.6852036
	Parameter Dispersion	Type: Value:
	Estimation Comments

4. Secondary Outcome

Title	Email Opened: Control vs. Less-is-better vs. Social Proof vs. Endowment
Description	Whether email was opened (yes / no)
Time Frame	32-35 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who were delivered an email (i.e. email did not bounce back) and for whom there was a record in the EHR 32 days post-intervention.

Arm/Group Title	Control	Less-is-better	Social Proof	Endowment / Decision Staging
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger	A simple message focused on one benefit of myGeisinger.	A message similar to control, but including information about how many other patients are using myGeisinger	A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete
Measure Participants	2095	6308	2107	2096
Count of Participants [Participants]	591 25.1%	1626 69.2%	402 17.1%	512 21.8%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0286
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8837908
	Confidence Interval	(2-Sided) 95%; 0.7912382 to 0.9871694
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	<0.0001
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.6000149
	Confidence Interval	(2-Sided) 95%; 0.5193168 to 0.6932528
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Control, Endowment / Decision Staging
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0055
	Comments
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8225743
	Confidence Interval	(2-Sided) 95%; 0.7167043 to 0.9440832
	Parameter Dispersion	Type: Value:
	Estimation Comments

5. Secondary Outcome

Title	Link Clicked: Control vs. Less-is-better vs. Social Proof vs. Endowment
Description	Whether the link / button to start the activation process was clicked (yes / no)
Time Frame	32-35 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who opened the intervention email and for whom there was a record in the EHR 32 days post-intervention.

Arm/Group Title	Control	Less-is-better	Social Proof	Endowment / Decision Staging
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger	A simple message focused on one benefit of myGeisinger.	A message similar to control, but including information about how many other patients are using myGeisinger	A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete
Measure Participants	591	1626	402	512
Count of Participants [Participants]	147 6.3%	188 8%	34 1.4%	86 3.7%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	<0.0001
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.3948795
	Confidence Interval	(2-Sided) 95%; 0.3104370 to 0.5022914
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	<0.0001
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.2790595
	Confidence Interval	(2-Sided) 95%; 0.1875017 to 0.4153254
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Control, Endowment / Decision Staging
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0011
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.6097538
	Confidence Interval	(2-Sided) 95%; 0.4528751 to 0.8209762
	Parameter Dispersion	Type: Value:
	Estimation Comments

6. Secondary Outcome

Title	Enrollment: Control vs. Less-is-better vs. Social Proof vs. Endowment
Description	Whether the patient enrolled in myGeisinger (yes / no)
Time Frame	32-35 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who opened the intervention email and for whom there was a record in the EHR 32 days post-intervention.

Arm/Group Title	Control	Less-is-better	Social Proof	Endowment / Decision Staging
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger	A simple message focused on one benefit of myGeisinger.	A message similar to control, but including information about how many other patients are using myGeisinger	A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete
Measure Participants	591	1626	402	512
Count of Participants [Participants]	55 2.3%	152 6.5%	29 1.2%	55 2.3%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.976
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.0049587
	Confidence Interval	(2-Sided) 95%; 0.72693253 to 1.3893201
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.246
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.7576895
	Confidence Interval	(2-Sided) 95%; 0.47412590 to 1.2108458
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Control, Endowment / Decision Staging
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.428
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.1728665
	Confidence Interval	(2-Sided) 95%; 0.79090107 to 1.7393021
	Parameter Dispersion	Type: Value:
	Estimation Comments

7. Secondary Outcome

Title	Email Opened: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access)
Description	Whether email was opened (yes / no)
Time Frame	8-10 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who were delivered an email (i.e. email did not bounce back) and for whom there was a record in the EHR 10 days post-intervention.

Arm/Group Title	Focused on Provider Communication	Focused on Scheduling	Focused on Medical Information Access
Arm/Group Description	A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email	A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email	A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email
Measure Participants	2129	2124	2131
Count of Participants [Participants]	524 22.3%	522 22.2%	482 20.5%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	1
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.9980487
	Confidence Interval	(2-Sided) 95%; 0.8680298 to 1.147543
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.276
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8953032
	Confidence Interval	(2-Sided) 95%; 0.7771682 to 1.031395
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Less-is-better, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.289
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8970536
	Confidence Interval	(2-Sided) 95%; 0.7785981 to 1.033531
	Parameter Dispersion	Type: Value:
	Estimation Comments

8. Secondary Outcome

Title	Link Clicked: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access)
Description	Whether the link / button to start the activation process was clicked (yes / no)
Time Frame	8-10 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who opened the intervention email and for whom there was a record in the EHR 10 days post-intervention.

Arm/Group Title	Focused on Provider Communication	Focused on Scheduling	Focused on Medical Information Access
Arm/Group Description	A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email	A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email	A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email
Measure Participants	524	522	482
Count of Participants [Participants]	45 1.9%	42 1.8%	73 3.1%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.94591
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.9313889
	Confidence Interval	(2-Sided) 95%; 0.6003591 to 1.444944
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0041
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.8998642
	Confidence Interval	(2-Sided) 95%; 1.2809137 to 2.817898
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Less-is-better, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0015
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	2.0398184
	Confidence Interval	(2-Sided) 95%; 1.3648075 to 3.048678
	Parameter Dispersion	Type: Value:
	Estimation Comments

9. Secondary Outcome

Title	Enrollment: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access)
Description	Whether the patient enrolled in myGeisinger (yes / no)
Time Frame	8-10 days post-intervention

Outcome Measure Data

Analysis Population Description

[Not Specified]

Arm/Group Title	Focused on Provider Communication	Focused on Scheduling	Focused on Medical Information Access
Arm/Group Description	A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email	A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email	A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email
Measure Participants	524	522	482
Count of Participants [Participants]	24 1%	28 1.2%	31 1.3%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.829
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.180837
	Confidence Interval	(2-Sided) 95%; 0.6750639 to 2.065546
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.404
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.432003
	Confidence Interval	(2-Sided) 95%; 0.8279689 to 2.476702
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Less-is-better, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.753
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.212702
	Confidence Interval	(2-Sided) 95%; 0.7161724 to 2.053480
	Parameter Dispersion	Type: Value:
	Estimation Comments

10. Secondary Outcome

Title	Email Opened: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access)
Description	Whether email was opened (yes / no)
Time Frame	32-35 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who were delivered an email (i.e. email did not bounce back) and for whom there was a record in the EHR 32 days post-intervention.

Arm/Group Title	Focused on Provider Communication	Focused on Scheduling	Focused on Medical Information Access
Arm/Group Description	A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email	A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email	A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email
Measure Participants	2107	2094	2107
Count of Participants [Participants]	568 24.2%	554 23.6%	504 21.4%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.9284
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.9747188
	Confidence Interval	(2-Sided) 95%; 0.8501783 to 1.1175028
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0612
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8518974
	Confidence Interval	(2-Sided) 95%; 0.7414610 to 0.9787827
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Less-is-better, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.1407
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8739930
	Confidence Interval	(2-Sided) 95%; 0.7602296 to 1.0047804
	Parameter Dispersion	Type: Value:
	Estimation Comments

11. Secondary Outcome

Title	Link Clicked: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access)
Description	Whether the link / button to start the activation process was clicked (yes / no)
Time Frame	32-35 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who opened the intervention email and for whom there was a record in the EHR 32 days post-intervention.

Arm/Group Title	Focused on Provider Communication	Focused on Scheduling	Focused on Medical Information Access
Arm/Group Description	A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email	A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email	A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email
Measure Participants	568	554	504
Count of Participants [Participants]	57 2.4%	51 2.2%	80 3.4%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.8848
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.9089673
	Confidence Interval	(2-Sided) 95%; 0.6109603 to 1.352333
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0127
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.6914929
	Confidence Interval	(2-Sided) 95%; 1.1762411 to 2.432450
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Less-is-better, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.0033
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.8608951
	Confidence Interval	(2-Sided) 95%; 1.2800275 to 2.705356
	Parameter Dispersion	Type: Value:
	Estimation Comments

12. Secondary Outcome

Title	Enrollment: Within Less-is-better (Provider Communication vs. Managing Appointments vs. Medical Information Access)
Description	Whether the patient enrolled in myGeisinger (yes / no)
Time Frame	32-35 days post-intervention

Outcome Measure Data

Analysis Population Description

Patients who opened the intervention email and for whom there was a record in the EHR 32 days post-intervention.

Arm/Group Title	Focused on Provider Communication	Focused on Scheduling	Focused on Medical Information Access
Arm/Group Description	A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email	A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email	A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email
Measure Participants	568	554	504
Count of Participants [Participants]	54 2.3%	46 2%	52 2.2%

Statistical Analysis 1

Statistical Analysis Overview	Comparison Group Selection	Control, Less-is-better
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.759
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	0.8619131
	Confidence Interval	(2-Sided) 95%; 0.5709078 to 1.301251
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 2

Statistical Analysis Overview	Comparison Group Selection	Control, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.897
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.0950508
	Confidence Interval	(2-Sided) 95%; 0.7331329 to 1.635633
	Parameter Dispersion	Type: Value:
	Estimation Comments

Statistical Analysis 3

Statistical Analysis Overview	Comparison Group Selection	Less-is-better, Social Proof
	Comments
	Type of Statistical Test	Superiority
	Comments
Statistical Test of Hypothesis	p-Value	0.498
	Comments	We used an a priori threshold of p < .05. All 3 comparisons run within same logistic regression model, correcting for the 3 pairwise comparisons using Tukey's Contrasts.
	Method	Regression, Logistic
	Comments
Method of Estimation	Estimation Parameter	Odds Ratio (OR)
	Estimated Value	1.2704886
	Confidence Interval	(2-Sided) 95%; 0.8377249 to 1.926816
	Parameter Dispersion	Type: Value:
	Estimation Comments

Adverse Events

Time Frame	No adverse events were evaluated.
Adverse Event Reporting Description	We only received information about email engagement and portal enrollment. We did not collect or receive any information regarding adverse events.
Arm/Group Title	Control		Focused on Provider Communication		Focused on Scheduling		Focused on Medical Information Access		Social Proof		Endowment / Decision Staging
Arm/Group Description	A standard marketing message encouraging activation and describing the benefits of myGeisinger Standard email: Email		A simple message focused on one benefit of myGeisinger: communicating easily with providers Less-is-better: Email		A simple message focused on one benefit of myGeisinger: scheduling and managing appointments online Less-is-better: Email		A simple message focused on one benefit of myGeisinger: accessing medical information, like test results Less-is-better: Email		A message similar to control, but including information about how many other patients are using myGeisinger Social proof: Email		A message similar to control, but framing myGeisinger as something that patients already have, and just need to take one more step to activate. Endowment: Give patients the impression they already have the account, so they value it more. Decision staging: Break down the process into multiple stages - "having" an account and "activating" that account- and giving the impression that they are almost there because the first stage is complete Endowment / decision staging: Email
All Cause Mortality
	Control		Focused on Provider Communication		Focused on Scheduling		Focused on Medical Information Access		Social Proof		Endowment / Decision Staging
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)
Serious Adverse Events
	Control		Focused on Provider Communication		Focused on Scheduling		Focused on Medical Information Access		Social Proof		Endowment / Decision Staging
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)
Other (Not Including Serious) Adverse Events
	Control		Focused on Provider Communication		Focused on Scheduling		Focused on Medical Information Access		Social Proof		Endowment / Decision Staging
	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events	Affected / at Risk (%)	# Events
Total	0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)		0/0 (NaN)

Limitations/Caveats

For this study, we only received data about engagement with the email and enrollment in the patient portal. Demographic information such as age, sex, or gender were not collected. Note that for some patients, enrollment data could not be extracted from Geisinger's electronic health records; these patients were excluded from analyses. Also note that "1 week" and "1 month" outcome data were extracted within a time range that was a few days longer than 7 and 30 days post-intervention, respectively.

More Information

Certain Agreements

All Principal Investigators ARE employed by the organization sponsoring the study.

There is NOT an agreement between Principal Investigators and the Sponsor (or its agents) that restricts the PI's rights to discuss or publish trial results after the trial is completed.