In total, 240 patients presenting to the Massachusetts General Hospital for inpatient orthopedic surgery were prospectively enrolled in this randomized controlled study. The trial used a cluster randomization method. The patients were recruited between February 2018 and February 2019 and followed for 3 months.

Members of the research team screened and approached all eligible patients to ask for consent. All patients aged ≥18 years admitted to the hospital for an orthopedic condition with the need for outpatient follow-up were eligible for the study. Patients were excluded if they were unable to consent for themselves, could not communicate in English, or did not possess a smartphone.

Institutional review board approval (IRB 2017P001594) was obtained prior to the initiation of the study, and all patients were given a fact sheet if they consented.

Eligible patients were cluster-randomized by week into 2 groups. The control group received information on how to enroll into and use the EPP in their discharge paperwork, whereas the intervention group was actively enrolled and taught how to use the EPP.

In the period between hospital discharge and follow-up, patients from both groups who were registered in the EPP were requested to fill out a survey on their personal device and received a notification of their upcoming clinic appointment.

For all enrolled patients, their age, gender, race (coded as White vs non-White), zip code, and admission diagnosis or service were recorded. Division of race into White and non-White was done to improve the robustness of the statistical analysis. The median income for each patient was abstracted using the zip code of the patient’s residence based on US census data, and the percentage of patients with an income less than the median state income was calculated [20].

Patients were followed for 3 months to ascertain if they completed their follow-up orthopedic clinic appointment and if they used the EPP to read or send a message with their providers, view a result, or answer a survey during the time period from their discharge to their follow-up.

A total of 229 patients were included (116 patients randomized to the control group and 113 patients randomized to the intervention group). The average patient age was 53.5 (SD 16.4) years. Of the 229 patients, 49.8% (n=114) were male and 16.2% (n=37) were non-White. In total, 31% (n=71) of the patients were admitted for the management of an acute traumatic injury, whereas 9.6% (n=22) were admitted for the treatment of an acute musculoskeletal infection. Demographic characteristics were balanced between the intervention and control groups, suggesting successful randomization (Table 1).

Patient enrollment is displayed with a flow diagram (Figure 1).

Descriptive statistics were used for the demographic data. Differences between groups were assessed using the chi-square or Fisher exact test for categorical variables and the 2-tailed t test or ANOVA for continuous variables. Demographic or treatment factors associated with improved follow-up or EPP use were assessed using forward stepwise logistic regression modeling to avoid overfitting. We also performed a subgroup analysis assessing the effects of the patient’s race and average median income. A robustness analysis exploring the likelihood of enrolling in an EPP or completing follow-up in all patients was also performed. Significance was set at P<.05. Stata statistical software (version 14; StataCorp) was used for all analyses.

An a priori power analysis was completed to determine the sample size. We assumed an existing follow-up rate of 70%, and to detect an approximate 10% difference in follow-up with an α of .05, we calculated an approximate sample size of 200 patients distributed equally between both groups.

Among the 229 patients, 28.8% (n=66) of White patients enrolled in the EPP, whereas only 6.1% (n=14) of non-White patients enrolled (P=.07 by Fisher exact test). This difference was driven by enrollment disparity in the control group. For non-White patients, only 1 out of 17 in the control group enrolled in the EPP (but did not use it). In contrast, of the 20 non-White patients in the intervention group, 13 (65%) registered and used the EPP (P<.001 by Fisher exact test comparing intervention vs control for both groups). Once enrolled, use of the EPP was not statistically different between White and non-White patients (P=.81 by Fisher exact test). No statistical differences in EPP registration (P>.05), use (P>.05), or clinical follow-up (P>.05) were observed for median income.

To compare our results to prior studies on the likelihood of enrolling in an EPP, we performed a backward stepwise logistic regression using measured demographic factors for all patients. We found that older age (OR 0.97, 95% CI 0.95-0.99; P=.03) and non-White race (OR 0.13, 95% CI 0.02-1.09; P=.06) were associated with decreased odds of EPP enrollment.

Tracking patient outcomes following orthopedic surgery is often difficult due to variable and poor follow-up. Electronic apps such as EPPs may be able to bridge this gap by engaging patients following hospital discharge [1,9-12]. In this randomized controlled study, we found that guided inpatient enrollment of patients who underwent orthopedic surgery into an EPP increased EPP use, but this did not independently result in enhanced follow-up. Patients who were enrolled as inpatients and subsequently used the portal had the highest likelihood of 3-month follow-up. In addition, we found that guided inpatient enrollment was associated with increased registration and use of the EPP in non-White patients.

In 2 recent studies of patients who underwent orthopedic surgery, EPP use was associated with lower no-show rates and increased patient satisfaction [18,19]. Both studies also found significant disparities in EPP enrollment based on demographic and treatment factors, but neither assessed strategies to mitigate these disparities [18,19]. Our results suggest that a method to improve the registration and use of EPPs, especially by disadvantaged groups, is to enroll patients while they are still inpatients following surgery. Although this may not independently result in improved follow-up rates, it is a standardized method to improve EPP registration and use for all patients, especially since EPP use is known to improve patient care. For example, a recent systematic review by Schwebel and Larimer [21] demonstrated that SMS text messaging improved patient compliance to appointments, whereas Bigby et al [22] had comparable results through phone calls or manual letters in an outpatient primary care setting.

Multiple retrospective studies have demonstrated that EPP use improves the likelihood of attending follow-up visits [18,19,23]. Using a prospective framework, we also found that EPP enrollment and use was associated with improved follow-up, but simple enrollment in an EPP was not independently associated with improved follow-up. This result suggests that a possible explanation for results in prior retrospective studies between EPP registration and use and enhanced follow-up may be due to patient confounding. Patients who are motivated to register and enroll in an EPP are also more likely to present for clinical follow-up. As in other social interactions frameworks, our findings suggest that patient portal apps may improve follow-up rates and survey completion if some preconditions are met: (1) patients need to be widely exposed and aware of the patient portal and (2) patients need to incorporate the use of the app into their daily routines with relevant content and context (ie, “stickiness” and appropriate context). To reinforce the importance of using these portals to patients, clinicians may need to implement a few changes in their practice. First, someone from the clinical team should enroll patients in the app either while they are still an inpatient or in the outpatient clinics to ensure successful enrollment and an understanding of the app. Next, to make the notifications from the app more readily accessible to patients, there needs to be an update to the app that includes notifications in forms more immediate than email reminders such as SMS text messaging or app push notifications. Finally, surgeons should also encourage communication through the patient portal, so patients feel more motivated to check and use the app.

Finally, in our supplemental analysis, we observed that non-White race was associated with decreased odds of EPP enrollment. For non-White patients, only 1 out of 17 patients in the control group signed up for the EPP, and that patient never used it. In contrast, of the 20 non-White patients in the intervention group, 65% used the EPP. This analysis suggests that standardized enrollment only partially alleviates the barriers to benefits from EPP use. Future studies should further assess the effects of guided enrollment in disadvantaged groups [18].

There were several important limitations to this study that may have impacted the results. First, we specifically approached English-speaking patients with active email addresses and smartphones. If we learned that they did not have either upon interview, we would exclude them from the study. This exclusion criteria may have decreased enrollment from the older patient population as well as patients from lower socioeconomic backgrounds who were less likely to be technologically active, although we attempted to mitigate this in our analysis by including median income by zip code. Future studies are needed to assess the effects of guided inpatient enrollment specifically in disadvantaged groups based on existing literature and our study [18,19]. Based on the post hoc power analysis, our results lacked the statistical power (power=25.6%) to detect no differences in clinical follow-up rates. We may have similarly been limited by the sample size for our subgroup analysis of non-White race, although our sample estimates are proportional to state population statistics [20]. We also referred to the non-White subgroup as disadvantaged not due to race alone but other socioeconomic features measured in our data set. Therefore, although this can be generalized in aggregate, it may not be true for any single patient. With a larger sample size, it may be that guided enrollment, especially for some patient populations, would have statistically and clinically relevant differences in follow-up rates.

The inpatient enrollment of patients who underwent orthopedic surgery into an EPP increased EPP use, but this did not independently result in enhanced follow-up. Patients who were enrolled as inpatients and subsequently used the portal had the highest likelihood of 3-month follow-up. Future studies targeted toward disadvantaged groups are critically needed.