Published on in Vol 3, No 2 (2020): Jul-Dec

Preprints (earlier versions) of this paper are available at, first published .
Usability of Mobile Health Apps for Postoperative Care: Systematic Review

Usability of Mobile Health Apps for Postoperative Care: Systematic Review

Usability of Mobile Health Apps for Postoperative Care: Systematic Review

Authors of this article:

Ben Patel1 Author Orcid Image ;   Arron Thind2 Author Orcid Image


1Guy's and St Thomas' Hospital National Health Service Foundation Trust, London, United Kingdom

2East Surrey Hospital, Redhill, United Kingdom

Corresponding Author:

Ben Patel, BA, BMBCh

Guy's and St Thomas' Hospital National Health Service Foundation Trust

Westminster Bridge Road

London, SE1 7EH

United Kingdom

Phone: 44 7720994804


Background: Mobile health (mHealth) apps are increasingly used postoperatively to monitor, educate, and rehabilitate. The usability of mHealth apps is critical to their implementation.

Objective: This systematic review evaluates the (1) methodology of usability analyses, (2) domains of usability being assessed, and (3) results of usability analyses.

Methods: The A Measurement Tool to Assess Systematic Reviews checklist was consulted. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline was adhered to. Screening was undertaken by 2 independent reviewers. All included studies were assessed for risk of bias. Domains of usability were compared with the gold-standard mHealth App Usability Questionnaire (MAUQ).

Results: A total of 33 of 720 identified studies were included for data extraction. Of the 5 included randomized controlled trials (RCTs), usability was never the primary end point. Methodology of usability analyses included interview (10/33), self-created questionnaire (18/33), and validated questionnaire (9/33). Of the 3 domains of usability proposed in the MAUQ, satisfaction was assessed in 28 of the 33 studies, system information arrangement was assessed in 11 of the 33 studies, and usefulness was assessed in 18 of the 33 studies. Usability of mHealth apps was above industry average, with median System Usability Scale scores ranging from 76 to 95 out of 100.

Conclusions: Current analyses of mHealth app usability are substandard. RCTs are rare, and validated questionnaires are infrequently consulted. Of the 3 domains of usability, only satisfaction is regularly assessed. There is significant bias throughout the literature, particularly with regards to conflicts of interest. Future studies should adhere to the MAUQ to assess usability and improve the utility of mHealth apps.

JMIR Perioper Med 2020;3(2):e19099



Industry experts have forecasted significant growth in mobile app users [1]. Given this projected surge, mobile health (mHealth) apps offer a unique and readily accessible platform to the patient, surgeon, and innovator. mHealth apps are now being integrated into various sectors of health care, with over 318,000 [2] apps currently helping to track, educate, and diagnose [3].

One area of particular growth is the use of mHealth apps as a means of monitoring patients in the important postoperative period. Well-designed apps have the potential to encourage earlier discharge, reduce in-person follow-ups [4,5], rehabilitate [6], aid clinicians in picking up surgical complications [7], and improve communication between patient and health care professional [8]. In addition to the economic and medical benefit of early discharge, postoperative monitoring apps have the potential to empower patients, giving them autonomy over their own health, which in turn might improve patient satisfaction and motivation for recovery [9].

The usability of mHealth apps is important [10,11] because those with poor usability will be less commonly used [12,13]. This is particularly significant in the postoperative period, given the focus of mHealth apps on rehabilitation, for which patient engagement is critical. One study revealed that around half of all mHealth app users stop engaging for various reasons, including loss of interest [14]. Despite this, little empirical research is undertaken to analyze the usability of mHealth apps before they are launched [15].

Several definitions and domains of usability have been previously defined without clear unification [11,16,17], but with several recurring themes. For example, the International Organization for Standardization (ISO) 3-pronged definition includes effectiveness (ie, whether users can use the product to complete their goals), efficiency (ie, the extent to which individuals expend resource in achieving their goals), and satisfaction [18]. Another definition [19] has been designed specifically for mHealth apps and includes factors such as mobility, connectivity, and additional cognitive load.

Different methods have been proposed for assessing domains of usability, such as the Post-Study System Usability Questionnaire [20] and the System Usability Scale (SUS) [21]. However, these tools were not originally created to evaluate mHealth apps. The Mobile App Rating Scale [22] was recently created for researchers and clinicians to assess the quality of mHealth apps, with the simpler user version of the Mobile App Rating Scale (uMARS) [23] being proposed shortly after. While quality of an mHealth app shares several components with usability, there are important differences.

Given the heterogeneity in definitions and methods used for assessing the usability of mHealth apps, one group has recently developed and validated the 21-item mHealth App Usability Questionnaire (MAUQ) [24]. This tool explores 3 domains of usability, which are in line with the ISO definition: (1) ease of use and satisfaction, akin to ISO satisfaction; (2) system information arrangement, akin to ISO efficiency; and (3) usefulness, akin to ISO effectiveness. This systematic literature review aims to determine whether the usability of postoperative mHealth apps is being rigorously assessed, using the validated MAUQ as the gold-standard reference. We consider which empirical methods are being used and analyze whether postoperative mHealth apps are indeed usable.

Database Search

The A Measurement Tool to Assess Systematic Reviews checklist [25] was analyzed before conducting this review, with all methodology being established prior to the review being conducted. A university librarian experienced in the field of systematic literature review methodology was consulted. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [26] reporting guideline was adhered to for this review. Rayyan (Qatar Computing Research Institute) [27] software was used for the search.

Textbox 1 shows the questions that were defined.

The Medline, Embase, and Association for Computing Machinery Digital Library databases were searched. The search string was generated and aimed to provide maximum coverage while maintaining manageability. We defined 4 broad themes for our search. Terms within a theme were combined using Boolean operator OR, as seen in Table 1. Themes were then combined using Boolean operator AND.

Search questions.

1. Which dimensions of usability are dealt with most often?

2. Which empirical methods are used to evaluate usability?

3. In which surgical specialties are mobile health apps’ usability being evaluated?

4. What types of operating systems have been used?

5. What are the results obtained by the usability evaluation of the apps?

Textbox 1. Search questions.
Table 1. Search strings for the 4 themes.
Mobile contextSmartphone OR smart phone OR mobile phone OR mobile device OR mHealtha OR tablet
SoftwareApp OR application OR operating system OR OSb OR ios OR android OR windows OR google play
PostoperativePostoperative OR post-operative OR surgery OR surgical OR operation OR perioperative OR peri operative
UsabilityUsab* OR understandab* OR learnab* OR operab* OR attractive* OR user experience OR engag* OR satisf* OR adher* OR willing* OR accepta* OR effectiv* OR aesthetic OR intuitive*

amHealth: mobile health.

bOS: operating system.

Screening of Papers for Inclusion and Exclusion

Each study recruited from the initial search was evaluated to determine whether it should be admitted for analysis. The inclusion and exclusion criteria are shown in Textbox 2.

Screening of article titles and abstracts was performed by 2 authors independently. In situations where eligibility of a study could not be determined based on abstract alone, the full-text article was retrieved. We executed a full-text review of the remaining studies after title and abstract screening to further analyze appropriateness for inclusion. We analyzed all review articles to identify any other appropriate studies. We also reviewed the reference list of included papers.

Inclusion and exclusion criteria.

Inclusion Criteria

  • The paper uses a mobile health app, defined as an application (rather than a web-based tool) on a portable device (including smartphones and tablets). We include apps designed both for the patient and for the health care professional. We include all types of apps, including monitoring, educational, and rehabilitation apps
  • The paper analyzes the postoperative period, defined as the point at which the patient leaves the operating theater, having undergone a surgical procedure
  • The paper studies usability of the mobile health app. Any level of assessment is included, from structured questionnaire to analysis of engagement or time spent on the app
  • The paper must be a full paper (not an abstract)

Exclusion Criteria

  • The paper is not written in English
  • The paper was published before 2000, in keeping with the launch of the first smartphone, the Ericsson R380 (Ericsson Mobile Communications)
  • The paper only uses web-based, text-based, or email-based technologies (no mobile health app). We want to concentrate on mobile health apps, given that they are the subject of such traction in the market
  • The app is not targeted to the postoperative period. For example, surgical apps monitoring patients following trauma or burns are excluded if no operative intervention is used. Furthermore, nonsurgical papers (eg, monitoring patients with chronic pain) are excluded. In addition, apps only used for education of surgeons are excluded
  • Inappropriate study types, including reviews, case reports, and feasibility/pilot studies without any real-life postoperative analysis
  • App is not designed for humans
Textbox 2. Inclusion and exclusion criteria.

Database Search Results

The initial search and reference list screening identified 721 studies. After title and abstract screening, 660 were excluded, leaving 61 full-text studies to be assessed. Of these, 28 were excluded, leaving 33 studies included for data extraction. The PRISMA summary of the database search is presented in Figure 1.

Figure 1. PRISMA summary of the literature search and exclusion process. mHealth: mobile health. PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
View this figure

Study Characteristics

A total of 33 studies were included. Of the 33 studies, 21 were from North America (14 from the United States and 6 from Canada), 9 were from Europe, 2 were from Asia, and 1 was from South America. Most studies specified the type of mobile device used by participants. Smartphones were used in 22 studies, tablets were used in 9, smartwatches were used in 1, iPod touch (Apple Inc) devices were used in 2, and 3 studies did not specify. Regarding the operating system, 11 studies used iOS (Apple Inc), 5 used Android, 1 used Windows (Microsoft Corp), and 17 did not specify.

Among the included studies, mHealth apps were used within a wide range of surgical subspecialties, including orthopedics (8 studies), general surgery (6 studies), head and neck (4 studies), transplant (3 studies), pediatrics (2 studies), breast (1 study), vascular (1 study), neurosurgery (1 study), and others/multiple (7 studies).

Functionality was divided into 5 clear categories; 26 studies included monitoring of symptoms or wounds, 8 included educational content, 5 provided a communication platform, 5 included physiotherapy and rehabilitation, and 2 enabled medication management. App details are presented in Table 2.

Study characteristics are presented in Table 3. With regards to study design, 5 studies were randomized controlled trials (RCTs), 25 were prospective noncontrolled studies, and 3 were retrospective reviews. Sample sizes ranged from 4 to 494, with a median of 39 patients and a mean of 81 patients. Follow-up ranged from 30 minutes postoperation to 12 months postdischarge. The follow-up period was less than 7 days in 4 studies, between 1 week and 1 month in 15 studies, greater than 1 month in 9 studies, and not declared in 5 studies.

Table 2. App details, including the study, country of origin, type of mobile device used, app name, surgical subspecialty, and app function.
StudyCountryPrimary mobile device (operating system)App nameSurgical subspecialtyFunction
Timmers et al [28]NetherlandsSmartphone and tablet (—a)Patient Journey App (Interactive Studios)Orthopedics (elective total knee replacement)Personalized educational information regarding pain; Physiotherapy; Wound monitoring; Self-care
Yadav et al [29]IndiaSmartphone (—)WhatsApp (Facebook Inc)Endocrine surgeryTele–follow-up including wound check and communication
Ramkumar et al [30]United StatesSmartphone (iOS)TKR (Focus Ventures)Orthopedics (elective total knee replacement)Monitoring of mobility and range of movement using wearable sleeve; PROMsb; Analgesia need; Home exercise program compliance
Argent et al [31]IrelandTablet (Android)Orthopedics (elective total knee replacement)Rehabilitation using an inertial measurement unitc, consisting of wearable sleeve; PROMs monitoring, including pain and perceived exercise difficulty
Brunner et al [32]United StatesTablet (iOS)Proloquo2Go (AssistiveWare)Head and neck surgeryAugmentative and alternative communication in patients who are unable to speak postoperatively
van der Meij et al [33]NetherlandsSmartphone (—)Abdominal surgery (laparoscopic cholecystectomy, inguinal hernia surgery, laparoscopic adnexal surgery)Information about surgical procedure; Insight into convalescence plan; Recovery monitor
Felbaum et al [34]United StatesSmartphone (—)TrackMyRecoveryNeurosurgeryPostoperative instructions; Pain reporting; Wound monitoring
Goz et al [35]United StatesSmartphone (—)Spine surgeryPostoperative communication through messaging app
Gunter et al [36]United StatesSmartphone (iOS)WoundCheckVascular surgeryWound monitoring using photographs and questionnaire
Gustavell et al [37]SwedenSmartphone and tablet (—)Interaktor (Health Navigator)Pancreatic surgerySymptom monitoring; Education links to evidence-based care advice
Harder et al [38]United KingdomSmartphone (iOS)bWellBreast surgeryRehabilitation (arm exercises); Symptom monitoring
Higgins et al [39]CanadaSmartphone (—)QoC Health (QoC Health Inc)Orthopedics (ACLd reconstruction)Symptom monitoring; QoR-9e questionnaire
Highland et al [40]United StatesSmartphone (—)mCareSurgery using peripheral nerve blockSymptom control using DVPRSf
Khanwalkar et al [41]United StatesSmartphone (—)HealthLoopENTg (septoplasty and FESSh)PROMs, including VASi pain score, PROMISj, and SNOT-22k
Mata et al [42]CanadaTablet (iOS)SeamlessMD (Seamless Mobile Health Inc)Colorectal surgeryMilestones checklist; Symptom-monitoring questionnaires; Educational content
Nilsson et al [43]SwedenSmartphone (—)Recovery Assessment by Phone PointsDay surgerySwQoRl questionnaire
Pecorelli et al [44]CanadaSmartphone (—)SeamlessMDColorectal surgeryMilestones checklist; Symptom-monitoring questionnaires; Educational content
Sousa and Turrini [45]BrazilSmartphone (iOS)OrtogAppOrthognathic surgeryEducational content; Communication platform
Sun et al [46]CanadaiPod touch (iOS)Panda (Balsamiq Solutions)Pediatric surgeryPostoperative pain monitoring; Medication management
Tsapepas et al [47]United StatesTablet (—)Medication Regimen EducationKidney transplantEducational content
Scott et al [48]United StatesSmartphone (—)SeamlessMDColorectal surgerySymptom tracker; Photograph of wound; Temperature recording
Warren-Stomberg et al [49]SwedenSmartphone (iOS and Android)Medipal (Novatelligence AB)Day orthopedic surgerySymptom questionnaire
Debono et al [50]FranceSmartphone and tablet (—)Lumbar discectomySymptom monitoring
Gunter et al [51]United States— (iOS)WoundCheckVascular and general surgerySymptom monitoring; Photograph of wound
Ponce et al [52]United States— (iOS)HelpLightningOrthopedics and neurosurgeryVirtual examination
Jiang et al [53]United StatesSmartphone (Windows)PocketPATHLung transplantData entry of health indicators; Self-monitoring
Chai et al [54]South KoreaTablet (iOS)Self-Reporting ApplicationThyroid surgerySelf-reporting of symptoms
Shellmer et al [55]United States— (Android)Teen Pocket PATHSolid organ transplantMonitoring of medications
Sun et al [56]CanadaSmartphone (—)PandaPediatric surgeryPostoperative pain monitoring using electronic versions of FPS-Rm and CASn
Jaensson et al [57]SwedenSmartphone (—)Recovery Assessment by Phone PointsDay surgerySwQoR questionnaire
Symer et al [58]United StatesSmartphone (iOS and Android) with paired smartwatchoColorectal surgeryPain monitoring; Symptom monitoring; Patient reminders/alerts; Photograph of wound
Semple et al [59]CanadaSmartphone or tablet (Android)QoC HealthBreast and orthopedic surgeryMobile version of the QoR-9 questionnaire
Bini and Mahajan [60]United StatesiPod touch (iOS)CaptureProofOrthopedic surgeryPhysiotherapy videos

aNot available.

bPROMs: patient-reported outcome measures.

cShimmer3; Shimmer.

dACL: anterior cruciate ligament.

eQOR-9: quality of recovery 9.

fDVPRS: Defense and Veterans Pain Rating Scale.

gENT: ear, nose, and throat.

hFESS: functional endoscopic sinus surgery.

iVAS: visual analog scale.

jPROMIS: Patient-Reported Outcomes Measurement Information System.

kSNOT-22: Sino-Nasal Outcome Test 22.

lSwQoR: Swedish Web Version of Quality of Life.

mFPS-R: Faces Pain Scale – Revised.

nCAS: color analog scale.

oFitbit; Fitbit Inc.

Table 3. Study characteristics, including study design, number of patients included, duration of follow-up, method of usability analysis, usability domain, and selected usability results.
StudyStudy designNumber of patientsDuration follow-upMethod of analysis of usability /outcome measureAspects of usability measuredSelected quantitative measure of usability
Timmers et al [28]Multicenter RCTa2134 weeksMeasurement of patient usage; Interview of small group of patients (n=6)UsefulnessApp used 26 times/patient; Videos watched 36 times/patient; Qualitative reporting of usefulness
Yadav et al [29]Prospective study (no control)1076 monthsSelf-created questionnaireSatisfaction; Usefulness1% unsatisfied across the questionnaire; 53% very satisfied with effectiveness; 78% very satisfied with app overall; Comfortable: 78% very satisfied; Convenience: 86%-91% very satisfied
Ramkumar et al [30]Prospective study (no control)223 monthsSemi-structured interviewSatisfaction; UsefulnessA1: average score 2.6/10 (1=easiest to use; 10=most difficult)
Argent et al [31]Mixed methods, including prospective study152 weeksQuestionnaires (SUSb and uMARSc); Semi-structured interviewSatisfaction; System information arrangement; UsefulnessuMARS average score 4.1/5 (SD 0.39); SUS average score 90.8 (SD 7.8)
Brunner et al [32]Prospective preintervention and postintervention study384 daysSelf-created questionnaires; Measurement of usageSatisfaction; Usefulness66% used the app; 60% satisfied with the app; 85% felt it was helpful
van der Meij et al [33]RCT3443 monthsMeasurement of usage; Self-created questionnaire; Semistructured interviewsSatisfaction49.6% had used the app; Mean score for app 7.6/10
Felbaum et al [34]Prospective study (no control)56dSelf-created questionnaireUsefulnessUsefulness ranged from 8.39-9.0 out of 10 (Likert scale)
Goz et al [35]Prospective study (no control)212 weeksMeasurement of usage/engagement; Self-created questionnaireSatisfaction; Usefulness82% satisfied (would recommend to others); 75% found useful (felt the app made it less likely for them to call the clinic); Engagement: 3.38 messages/person over 2 weeks
Gunter et al [36]Prospective study (no control)402 weeksSUS (questionnaire); Measurement of usageSatisfaction; System information arrangementSUS average score of 87.2
Gustavell et al [37]Prospective study (no control)64 weeksMeasurement of usage; Semistructured interviewsSatisfaction; System information arrangement; UsefulnessAdherence to reporting daily was 84%; Other measurements qualitative
Harder et al [38]Prospective study (no control)48 weeksMeasurement of usage; Self-created questionnaireSatisfaction; System information arrangement; UsefulnessOverall rating (Likert scale) 4.6/5; All used the app almost daily or several times/day
Higgins et al [39]Retrospective case series326 weeksInterview; Self-created questionnaireSatisfactionOverall satisfaction was reported as excellent (43%), good (40%), fair (10%), poor (7%); 94% would use the app again
Highland et al [40]RCT24 (only 12 assessed usability)10 daysSUS questionnaire; Additional questionnaireSatisfaction; System information arrangement; UsefulnessSUS average score 76.26/100; No difference in convenience between intervention and standard of care (telephone follow-up)
Khanwalkar et al [41]Prospective study (no control)2493 monthsMeasurement of usageNone77.4% response rate (usage)
Mata et al [42]RCT504 weeks; Satisfaction measured at dischargeMeasurement of usage; Self-created questionnaire using 4 items from S-CAHPSeSatisfactionUsage: postoperative day 0=94%, day 1=82%, day 2=72%, day 3=48%; 4/5 satisfaction across all 4 questions
Nilsson et al [43]Prospective study (no control)49414 daysMeasurement of usage (response rate)NoneUsage: day 1=86.8%, day 7=69%, day 14=57.5%
Pecorelli et al [44]Prospective study (no control)454 weeksSUS questionnaireSatisfaction; System information arrangementSUS average score 87/100
Sousa and Turrini [45]Prospective study (no control)30SUS questionnaire; Satisfaction measured according to experience sampling method technique; UsageSatisfaction; System information arrangementSUS average score 79.8/100, 73.3% >68 (cutoff), 100% >50 (acceptable); Satisfaction 82.9%; Usage: 100% used at least once, 40% used 2-3 times, 10% used 5 times, 20% used >5 times
Sun et al [46]Prospective study (no control)29CSUQf Unstructured interviewsSatisfaction; System information arrangement; UsefulnessMedian CSUQ score 2 (IQRg 1-3); 93% found app easy to use; 59% would use the app at home
Tsapepas et al [47]Retrospective study282Self-created questionnaireSatisfactionSatisfaction rated 4 or 5 in 92%
Scott et al [48]Prospective study (no control)2014 daysSUS questionnaire; Semi-structured interview; Measurement of usageSatisfaction; System information arrangement; UsefulnessMedian SUS 95/100; Usage: 30% did not use after discharge
Warren-Stomberg et al [49]Prospective study (no control)1011 weekMeasurement of usageNone55/101 used the app; Of those that used the app, 53% used >13 times out of possible 15
Debono et al [50]Prospective study (no control)6015 daysTelephone interviewSatisfaction; Usefulness1 (poor) to 4 (excellent) scale: Overall satisfaction 3.4 Usability 3.5 Usefulness at home 3.2 Facilitating return at home 3.1; 91.6% would use the device again
Gunter et al [51]Prospective study (no control)9SUS questionnaireSatisfaction; System information arrangementAverage SUS score 83.3/100; 55.6% were able to complete the tasks independently
Ponce et al [52]Prospective3124 days15-point questionnaireSatisfaction; UsefulnessReassurance 4.6-4.8/5; Useful 4.5-4.8/5; Satisfaction 4.2-4.6/5
Jiang et al [53]Secondary retrospective analysis of previous RCT data9612 monthsTechnology acceptance subscales used to measure: intention to use (1 item); perceived usefulness (4 items); and perceived ease of use (4 items)Satisfaction; Usefulness85% strongly agree with intention to use item; 80% gave high rating of perceived usefulness (>24/28); 82% gave high rating of perceived ease of use (>24/28)
Chai et al [54]Prospective comparison study (nonrandomized)5414 daysSelf-created questionnaireSatisfaction; UsefulnessSatisfaction was >7.2/10 across all 4 items on questionnaire
Shellmer et al [55]Prospective study76 weeks8/16 questions from PSSUQh surveySatisfaction; System information arrangement; UsefulnessSatisfaction 1/7 (1=strongly agree); Ease of use 1/7; Felt comfortable using application 1/7; “I could clearly tell when I missed my medication” 1/7; Liked tracking medications 3/7; Helpful to track medications 2/7
Sun et al [56]Prospective study6630 minutes postoperationSingle question asked regarding preference of monitoring (app vs paper version of questionnaire)Satisfaction76%-81% preferred the app over the paper version
Jaensson et al [57]Prospective study10Self-created questionnaire on system layout and technical issues, satisfaction, and usefulnessSatisfaction; System information arrangement; Usefulness
Symer et al [58]Prospective study3130 daysMeasurement of usage; Self-created questionnaireSatisfaction; System information arrangement; Usefulness83.9% used the app 70% of the time; 89.3%: easy to navigate; 88.9%: easy to use; 85.2%: survey questions relevant for identifying problems related to readmission; 66.7% found reminders useful; 92.9% would recommend to others
Semple et al [59]Prospective study6530 daysSelf-created survey; Interview; UsageSatisfactionSatisfaction 3.7-3.9/4; 100% wiling to use in future; 100% surgeons found platform intuitive and easy to use; Usage: mean number of logins 19.3-23.9/30 days; Mean number of photographs uploaded 38-63/30 days
Bini and Mahajan [60]RCT2924 weeksSelf-created survey; Free-form feedback; UsageSatisfactionEase of use: 3.9-4.4/5; Satisfaction 4.2/5

aRCT: randomized controlled trial.

bSUS: System Usability Scale.

cuMARS: user version of the Mobile App Rating Scale.

dNot available.

eS-CAHPS: Surgical Care Consumer Assessment of Healthcare Providers and Systems.

fCSUQ: Computer System Usability Questionnaire.

gIQR: interquartile range.

hPSSUQ: Post-Study System Usability Questionnaire.

Usability Analysis

Regarding the method of usability analysis, usage (ie, monitoring of user engagement with the app) was used in 15 studies and was the only usability analysis employed in 4 studies. Interviews were used in 10 studies. Self-created questionnaires were used in 18 studies. Validated questionnaires were used in 9 studies. Of these, 7 used the SUS questionnaire, 1 used the uMARS questionnaire, 1 used the technology acceptance subscale, and 1 used the Computer System Usability Questionnaire (CSUQ).

We have categorized the domains of usability according to the MAUQ. A total of 28 studies covered ease of use and satisfaction, 11 studies covered system information arrangement, and 18 studies covered usefulness.

Average SUS scores ranged from 76 to 95 out of 100, with a median score of 87. The uMARS score was 4.1 out of 5. The CSUQ score was 2 out of 7 (whereby a score of 1 would indicate greatest usability).


There is significant potential for bias in studies evaluating the usability of mHealth apps. Hidden agenda bias and secondary gains bias were common and seemingly underreported in the literature. Of the 33 included studies, 8 officially reported authors’ conflicts of interest, stating that they held shares in the app. Furthermore, several of the study groups were provided with the apps free of charge [28], which has clear implications on the usability domain of satisfaction; users who have paid for an app might be expected to have higher expectations than those who have been given an app for free. Perhaps more worryingly, a number of groups [38] declared no conflict of interest, despite seemingly being founders of their app.

Nonresponse bias is a further concern. Some studies, such as Pecorelli et al [44], had high response rates (96%) to usability analyses. However, others, such as Nilsson et al [43], had much lower rates (57.5% on day 14), and some [51] did not disclose the proportion of responders. Nonresponders to usability analyses are more likely to have reported poor usability. Therefore, studies with high rates of nonresponders are likely to have inflated usability results.

Population bias is a further issue. Younger audiences are likely to be more adept at using mobile technologies. Therefore, studies that include a younger demographic are likely to demonstrate inflated usability results. Additionally, the generalizability of results from studies [44] that included patients that were not used to mobile technologies may be limited and may change in the future, when greater numbers of older patients are used to mobile technologies.

Principal Findings

To our knowledge, this is the first comprehensive systematic review to assess usability of mHealth apps in postoperative management. This review identified 33 studies evaluating the usability of mHealth apps in the postoperative period across a broad range of surgical subspecialties, demonstrating the growing interest in this area. Most of the included studies were derived from the United States and Europe, which appear to be hubs of innovation in the field. Unsurprisingly, smartphones were the most commonly used devices. However, we suspect that wearable devices such as smartwatches, which have additional monitoring capabilities such as electrocardiogram monitors, will play an increasingly important role in the future [61].

With respect to study designs, 25 of 33 studies were prospective noncontrolled trials. There were 5 RCTs, but usability was never a primary end point in these studies. We feel RCTs comparing mHealth apps to normal practice (eg, in-person follow-up, telephone follow-up, or no follow-up) would be particularly beneficial in assessing the domains of satisfaction and usefulness. It has also been suggested that mHealth app interventions are associated with a falsely heightened level of user satisfaction due to patients’ affinities for their digital devices [62]. This could be minimized by comparing postoperative mHealth apps to a sham app. However, we also acknowledge that RCTs have previously been described as an impractical evaluation methodology for mHealth apps, due to their prolonged duration from recruitment to results and their high costs [63].

The methodology for assessing usability was generally poor. The majority of analyses used simplistic self-created questionnaires that asked rudimentary questions focusing on the domain of satisfaction (28/33 studies) rather than other domains of usability. Indeed, only 11 of the 33 usability analyses assessed the domain of system information arrangement. We would argue that formal usability analyses should cover all 3 common domains of (1) satisfaction, (2) usefulness, and (3) system arrangement, according to the ISO definition of usability [18]. Validated questionnaires are helpful in assessing these areas reliably. Only 9 of the 33 included studies used validated questionnaires, most of which used the SUS. The SUS is a Likert scale made up of 10 questions. The average SUS score is 68 out of 100, meaning that all 7 studies that used the SUS scored above average in terms of usability. Although the SUS is a quick and cheap means of assessing usability, it was created in 1986, before the first smartphone or the concept of an app was realized. The SUS has not been validated for assessing mHealth apps. In comparison, the MAUQ was recently proposed and validated for use in mHealth apps in a population of English-speaking adults [64]. This is the gold-standard reference for analysis of mHealth app usability. While scores on the MAUQ have previously been shown to correlate with the SUS, this is not a strong correlation (r=0.643), thereby highlighting the inadequacy of studies that have only used the SUS.

A major concern in these studies is the risk of bias. A number of the studies’ authors have a financial interest in the usability of their apps, with high user satisfaction making adoption by hospitals and investors more likely. Furthermore, devices were sometimes provided free of charge, which could influence the feedback from users.


mHealth apps have significant potential during the postoperative period for encouraging earlier discharge, improving patient engagement, and offering a safety net for early identification of complications. Thorough analysis of usability is critical to the adoption of these novel technologies in the postoperative period; those with poor usability will have little impact in health care. According to this review, usability analyses to date have been substandard. They have focused on satisfaction, a narrow dimension of usability, with simplistic self-created questionnaires. Furthermore, there is a significant risk of bias, given the common conflicts of interest among authors of published studies. We hope this review changes future practice, with researchers undertaking more robust assessments of usability by employing validated questionnaires, such as the MAUQ, in blinded RCTs.

Authors' Contributions

BP contributed the study conception and design. BP and AT performed the acquisition of data and analysis of data. BP drafted the manuscript. All work was self-funded.

Conflicts of Interest

None declared.

  1. Ericsson Mobility Report. Ericsson website.   URL: [accessed 2018-05-29]
  2. The Growing Value of Digital Health. IQVIA Institute. 2017 Nov 07.   URL: [accessed 2018-05-29]
  3. Byambasuren O, Sanders S, Beller E, Glasziou P. Prescribable mHealth apps identified from an overview of systematic reviews. NPJ Digit Med 2018;1:12 [FREE Full text] [CrossRef] [Medline]
  4. Armstrong KA, Coyte PC, Brown M, Beber B, Semple JL. Effect of Home Monitoring via Mobile App on the Number of In-Person Visits Following Ambulatory Surgery: A Randomized Clinical Trial. JAMA Surg 2017 Jul 01;152(7):622-627 [FREE Full text] [CrossRef] [Medline]
  5. Dahlberg K, Philipsson A, Hagberg L, Jaensson M, Hälleberg-Nyman M, Nilsson U. Cost-effectiveness of a systematic e-assessed follow-up of postoperative recovery after day surgery: a multicentre randomized trial. Br J Anaesth 2017 Nov 01;119(5):1039-1046 [FREE Full text] [CrossRef] [Medline]
  6. Belarmino A, Walsh R, Alshak M, Patel N, Wu R, Hu JC. Feasibility of a Mobile Health Application To Monitor Recovery and Patient-reported Outcomes after Robot-assisted Radical Prostatectomy. Eur Urol Oncol 2019 Jul;2(4):425-428. [CrossRef] [Medline]
  7. Hwang JH, Hee Hwang J, Mun GH. An evolution of communication in postoperative free flap monitoring: using a smartphone and mobile messenger application. Plast Reconstr Surg 2012 Jul;130(1):125-129. [CrossRef] [Medline]
  8. Chang P, Lin L, Zhang H, Zhao Y, Xie J, Yu Y, et al. Effect of smartphone application assisted medical service on follow-up adherence improvement in pediatric cataract patients. Graefes Arch Clin Exp Ophthalmol 2018 Oct;256(10):1923-1931. [CrossRef] [Medline]
  9. deBronkart D. From patient centred to people powered: autonomy on the rise. BMJ 2015 Feb 10;350:h148. [CrossRef] [Medline]
  10. Constantinos CK, Kim DJ. A Meta-Analytical Review of Empirical Mobile Usability Studies. J Usability Stud 2011;6(3):117-171 [FREE Full text] [CrossRef]
  11. Zapata BC, Fernández-Alemán JL, Idri A, Toval A. Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst 2015 Feb;39(2):1. [CrossRef] [Medline]
  12. Schnall R, Mosley JP, Iribarren SJ, Bakken S, Carballo-Diéguez A, Brown IW. Comparison of a User-Centered Design, Self-Management App to Existing mHealth Apps for Persons Living With HIV. JMIR Mhealth Uhealth 2015;3(3):e91 [FREE Full text] [CrossRef] [Medline]
  13. McCurdie T, Taneva S, Casselman M, Yeung M, McDaniel C, Ho W, et al. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol 2012 Sep;Suppl:49-56 [FREE Full text] [CrossRef] [Medline]
  14. Krebs P, Duncan DT. Health App Use Among US Mobile Phone Owners: A National Survey. JMIR Mhealth Uhealth 2015 Nov;3(4):e101 [FREE Full text] [CrossRef] [Medline]
  15. Boulos MNK, Brewer AC, Karimkhani C, Buller DB, Dellavalle RP. Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J Public Health Inform 2014;5(3):229 [FREE Full text] [CrossRef] [Medline]
  16. Nassar V. Common criteria for usability review. Work 2012;41 Suppl 1:1053-1057. [CrossRef] [Medline]
  17. Baharuddin R, Singh D, Razali R. Usability Dimensions for Mobile Applications-A Review. RJASET 2013 Feb 21;11(9):2225-2231. [CrossRef]
  18. ISO 9241. International Organisation for Standardization.   URL: [accessed 2019-11-08]
  19. Harrison R, Flood D, Duce D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 2013;1(1):1. [CrossRef]
  20. Lewis JR. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 1995 Jan;7(1):57-78. [CrossRef]
  21. Brooke J. SUS - A quick and dirty usability scale. Usability Evaluation In Industry 1996 Jun 11:189-194 [FREE Full text]
  22. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015 Mar;3(1):e27 [FREE Full text] [CrossRef] [Medline]
  23. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016 Jun 10;4(2):e72 [FREE Full text] [CrossRef] [Medline]
  24. Zhao B, Tai-Seale M, Longhurst C, Clay B. Utilization of Hospital Room Hospitality Features on Patient-Controlled Tablet Computers: Cohort Study. JMIR Mhealth Uhealth 2019 Jun 20;7(6):e13964 [FREE Full text] [CrossRef] [Medline]
  25. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017 Sep 21;358:j4008 [FREE Full text] [CrossRef] [Medline]
  26. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009 Jul 21;339:b2535 [FREE Full text] [CrossRef] [Medline]
  27. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016 Dec 05;5(1):210 [FREE Full text] [CrossRef] [Medline]
  28. Timmers T, Janssen L, van der Weegen W, Das D, Marijnissen W, Hannink G, et al. The Effect of an App for Day-to-Day Postoperative Care Education on Patients With Total Knee Replacement: Randomized Controlled Trial. JMIR Mhealth Uhealth 2019 Oct 21;7(10):e15323 [FREE Full text] [CrossRef] [Medline]
  29. Yadav SK, Jha CK, Mishra SK, Mishra A. Smartphone-Based Application for Tele-follow-up of Patients with Endocrine Disorders in Context of a LMIC: A Compliance, Satisfaction, Clinical Safety and Outcome Assessment. World J Surg 2020 Feb;44(2):612-616. [CrossRef] [Medline]
  30. Ramkumar PN, Haeberle HS, Ramanathan D, Cantrell WA, Navarro SM, Mont MA, et al. Remote Patient Monitoring Using Mobile Health for Total Knee Arthroplasty: Validation of a Wearable and Machine Learning-Based Surveillance Platform. J Arthroplasty 2019 Oct;34(10):2253-2259. [CrossRef] [Medline]
  31. Argent R, Slevin P, Bevilacqua A, Neligan M, Daly A, Caulfield B. Wearable Sensor-Based Exercise Biofeedback for Orthopaedic Rehabilitation: A Mixed Methods User Evaluation of a Prototype System. Sensors (Basel) 2019 Jan 21;19(2) [FREE Full text] [CrossRef] [Medline]
  32. Brunner TH, DiFortuna K, LeTang M, Murphy J, Stemplewicz K, Kovacs M, et al. Feasibility of an iPad to Facilitate Communication in Postoperative Patients With Head and Neck Cancer. J Perianesth Nurs 2018 Aug;33(4):399-406 [FREE Full text] [CrossRef] [Medline]
  33. van der Meij E, Huirne JA, Ten Cate AD, Stockmann HB, Scholten PC, Davids PH, et al. A Perioperative eHealth Program to Enhance Postoperative Recovery After Abdominal Surgery: Process Evaluation of a Randomized Controlled Trial. J Med Internet Res 2018 Jan 02;20(1):e1 [FREE Full text] [CrossRef] [Medline]
  34. Felbaum DR, Stewart J, Anaizi A, Sandhu FA, Nair MN, Voyadzis JM. Implementation and Evaluation of a Smartphone Application for the Perioperative Care of Neurosurgery Patients at an Academic Medical Center: Implications for Patient Satisfaction, Surgery Cancelations, and Readmissions. Oper Neurosurg (Hagerstown) 2018 Mar 01;14(3):303-311. [CrossRef] [Medline]
  35. Goz V, Anthony C, Pugely A, Lawrence B, Spina N, Brodke D, et al. Software-Based Postoperative Communication With Patients Undergoing Spine Surgery. Global Spine J 2019 Feb;9(1):14-17 [FREE Full text] [CrossRef] [Medline]
  36. Gunter RL, Fernandes-Taylor S, Rahman S, Awoyinka L, Bennett KM, Weber SM, et al. Feasibility of an Image-Based Mobile Health Protocol for Postoperative Wound Monitoring. J Am Coll Surg 2018 Mar;226(3):277-286 [FREE Full text] [CrossRef] [Medline]
  37. Gustavell T, Langius-Eklöf A, Wengström Y, Segersvärd R, Sundberg K. Development and Feasibility of an Interactive Smartphone App for Early Assessment and Management of Symptoms Following Pancreaticoduodenectomy. Cancer Nurs 2019;42(3):E1-E10. [CrossRef] [Medline]
  38. Harder H, Holroyd P, Burkinshaw L, Watten P, Zammit C, Harris PR, et al. A user-centred approach to developing bWell, a mobile app for arm and shoulder exercises after breast cancer treatment. J Cancer Surviv 2017 Dec;11(6):732-742 [FREE Full text] [CrossRef] [Medline]
  39. Higgins J, Semple J, Murnaghan L, Sharpe S, Theodoropoulos J. Mobile Web-Based Follow-up for Postoperative ACL Reconstruction: A Single-Center Experience. Orthop J Sports Med 2017 Dec;5(12):2325967117745278 [FREE Full text] [CrossRef] [Medline]
  40. Highland KB, Tran J, Edwards H, Bedocs P, Suen J, Buckenmaier CC. Feasibility of App-Based Postsurgical Assessment of Pain, Pain Impact, and Regional Anesthesia Effects: A Pilot Randomized Controlled Trial. Pain Med 2019 Aug 01;20(8):1592-1599. [CrossRef] [Medline]
  41. Khanwalkar AR, Shen J, Kern RC, Welch KC, Smith SS, Tan BK, et al. Utilization of a novel interactive mobile health platform to evaluate functional outcomes and pain following septoplasty and functional endoscopic sinus surgery. Int Forum Allergy Rhinol 2019 Apr;9(4):345-351. [CrossRef] [Medline]
  42. Mata J, Pecorelli N, Kaneva P, Moldoveanu D, Gosselin-Tardiff A, Alhashemi M, et al. A mobile device application (app) to improve adherence to an enhanced recovery program for colorectal surgery: a randomized controlled trial. Surg Endosc 2020 Feb;34(2):742-751. [CrossRef] [Medline]
  43. Nilsson U, Dahlberg K, Jaensson M. The Swedish Web Version of the Quality of Recovery Scale Adapted for Use in a Mobile App: Prospective Psychometric Evaluation Study. JMIR Mhealth Uhealth 2017 Dec 03;5(12):e188 [FREE Full text] [CrossRef] [Medline]
  44. Pecorelli N, Fiore JF, Kaneva P, Somasundram A, Charlebois P, Liberman AS, et al. An app for patient education and self-audit within an enhanced recovery program for bowel surgery: a pilot study assessing validity and usability. Surg Endosc 2018 May;32(5):2263-2273. [CrossRef] [Medline]
  45. Sousa CS, Turrini R. Development of an educational mobile application for patients submitted to orthognathic surgery. Rev Lat Am Enfermagem 2019 Jul 18;27:e3143 [FREE Full text] [CrossRef] [Medline]
  46. Sun T, Dunsmuir D, Miao I, Devoy GM, West NC, Görges M, et al. In-hospital usability and feasibility evaluation of Panda, an app for the management of pain in children at home. Paediatr Anaesth 2018 Oct;28(10):897-905. [CrossRef] [Medline]
  47. Tsapepas DS, Salerno D, Jandovitz N, Hammad S, Jordan P, Mohan S, et al. Using technology to enhance medication regimen education after solid organ transplantation. Am J Health Syst Pharm 2018 Dec 01;75(23):1930-1937. [CrossRef] [Medline]
  48. Scott AR, Alore EA, Naik AD, Berger DH, Suliburk JW. Mixed-Methods Analysis of Factors Impacting Use of a Postoperative mHealth App. JMIR Mhealth Uhealth 2017 Feb 08;5(2):e11 [FREE Full text] [CrossRef] [Medline]
  49. Warren-Stomberg M, Jacobsson J, Brattwall M, Jildenstål P. At-home monitoring after surgery/anaesthesia - a challenge. J Eval Clin Pract 2016 Dec;22(6):882-886. [CrossRef] [Medline]
  50. Debono B, Bousquet P, Sabatier P, Plas J, Lescure J, Hamel O. Postoperative monitoring with a mobile application after ambulatory lumbar discectomy: an effective tool for spine surgeons. Eur Spine J 2016 Nov;25(11):3536-3542. [CrossRef] [Medline]
  51. Gunter R, Fernandes-Taylor S, Mahnke A, Awoyinka L, Schroeder C, Wiseman J, et al. Evaluating Patient Usability of an Image-Based Mobile Health Platform for Postoperative Wound Monitoring. JMIR Mhealth Uhealth 2016 Sep 28;4(3):e113 [FREE Full text] [CrossRef] [Medline]
  52. Ponce BA, Brabston E, Zu S. Telemedicine with mobile devices and augmented reality for early postoperative care. 2016 Presented at: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); Aug 16-20, 2016; Orlando, FL. [CrossRef]
  53. Jiang Y, Sereika S, Dabbs AD, Handler S, Schlenk E. Acceptance and Use of Mobile Technology for Health Self-Monitoring in Lung Transplant Recipients during the First Year Post-Transplantation. Appl Clin Inform 2016;7(2):430-445 [FREE Full text] [CrossRef] [Medline]
  54. Chai YJ, Song J, Kang J, Woo J, Song R, Kwon H, et al. A comparative study of postoperative pain for open thyroidectomy versus bilateral axillo-breast approach robotic thyroidectomy using a self-reporting application for iPad. Ann Surg Treat Res 2016 May;90(5):239-245 [FREE Full text] [CrossRef] [Medline]
  55. Shellmer DA, Dew MA, Mazariegos G, DeVito Dabbs A. Development and field testing of Teen Pocket PATH(®), a mobile health application to improve medication adherence in adolescent solid organ recipients. Pediatr Transplant 2016 Feb;20(1):130-140 [FREE Full text] [CrossRef] [Medline]
  56. Sun T, West N, Ansermino JM, Montgomery CJ, Myers D, Dunsmuir D, et al. A smartphone version of the Faces Pain Scale-Revised and the Color Analog Scale for postoperative pain assessment in children. Paediatr Anaesth 2015 Dec;25(12):1264-1273. [CrossRef] [Medline]
  57. Jaensson M, Dahlberg K, Eriksson M, Grönlund Å, Nilsson U. The Development of the Recovery Assessments by Phone Points (RAPP): A Mobile Phone App for Postoperative Recovery Monitoring and Assessment. JMIR Mhealth Uhealth 2015 Sep 11;3(3):e86 [FREE Full text] [CrossRef] [Medline]
  58. Symer MM, Abelson JS, Milsom J, McClure B, Yeo HL. A Mobile Health Application to Track Patients After Gastrointestinal Surgery: Results from a Pilot Study. J Gastrointest Surg 2017 Sep;21(9):1500-1505. [CrossRef] [Medline]
  59. Semple JL, Sharpe S, Murnaghan ML, Theodoropoulos J, Metcalfe KA. Using a mobile app for monitoring post-operative quality of recovery of patients at home: a feasibility study. JMIR Mhealth Uhealth 2015 Feb 12;3(1):e18 [FREE Full text] [CrossRef] [Medline]
  60. Bini S, Mahajan J. Clinical outcomes of remote asynchronous telerehabilitation are equivalent to traditional therapy following total knee arthroplasty: A randomized control study. J Telemed Telecare 2017 Feb;23(2):239-247. [CrossRef] [Medline]
  61. Smartwatch Market - Growth, Trends, Forecasts. Mordor Intelligence.   URL: [accessed 2020-01-28]
  62. Torous J, Firth J. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry 2016 Feb;3(2):100-102. [CrossRef] [Medline]
  63. Pham Q, Wiljer D, Cafazzo JA. Beyond the Randomized Controlled Trial: A Review of Alternatives in mHealth Clinical Trial Methods. JMIR Mhealth Uhealth 2016 Sep 09;4(3):e107 [FREE Full text] [CrossRef] [Medline]
  64. Zhou L, Bao J, Setiawan IMA, Saptono A, Parmanto B. The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study. JMIR Mhealth Uhealth 2019 Apr 11;7(4):e11500 [FREE Full text] [CrossRef] [Medline]

CSUQ: Computer System Usability Questionnaire
ISO: Organization for Standardization
MAUQ: mHealth App Usability Questionnaire
mHealth: mobile health
PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
SUS: System Usability Scale
uMARS: user version of the Mobile App Rating Scale

Edited by R Lee; submitted 03.04.20; peer-reviewed by M Tai-Seale, J Hitt, Y Chu; comments to author 07.05.20; revised version received 21.05.20; accepted 26.05.20; published 20.07.20


©Ben Patel, Arron Thind. Originally published in JMIR Perioperative Medicine (, 20.07.2020.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Perioperative Medicine, is properly cited. The complete bibliographic information, a link to the original publication on, as well as this copyright and license information must be included.