Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Advertisement

Citing this Article

Right click to copy or hit: ctrl+c (cmd+c on mac)

Published on 20.07.20 in Vol 3, No 2 (2020): Jul-Dec

Preprints (earlier versions) of this paper are available at http://preprints.jmir.org/preprint/19099, first published Apr 03, 2020.

This paper is in the following e-collection/theme issue:

    Review

    Usability of Mobile Health Apps for Postoperative Care: Systematic Review

    1Guy's and St Thomas' Hospital National Health Service Foundation Trust, London, United Kingdom

    2East Surrey Hospital, Redhill, United Kingdom

    Corresponding Author:

    Ben Patel, BA, BMBCh

    Guy's and St Thomas' Hospital National Health Service Foundation Trust

    Westminster Bridge Road

    London, SE1 7EH

    United Kingdom

    Phone: 44 7720994804

    Email: benjpatel@gmail.com


    ABSTRACT

    Background: Mobile health (mHealth) apps are increasingly used postoperatively to monitor, educate, and rehabilitate. The usability of mHealth apps is critical to their implementation.

    Objective: This systematic review evaluates the (1) methodology of usability analyses, (2) domains of usability being assessed, and (3) results of usability analyses.

    Methods: The A Measurement Tool to Assess Systematic Reviews checklist was consulted. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guideline was adhered to. Screening was undertaken by 2 independent reviewers. All included studies were assessed for risk of bias. Domains of usability were compared with the gold-standard mHealth App Usability Questionnaire (MAUQ).

    Results: A total of 33 of 720 identified studies were included for data extraction. Of the 5 included randomized controlled trials (RCTs), usability was never the primary end point. Methodology of usability analyses included interview (10/33), self-created questionnaire (18/33), and validated questionnaire (9/33). Of the 3 domains of usability proposed in the MAUQ, satisfaction was assessed in 28 of the 33 studies, system information arrangement was assessed in 11 of the 33 studies, and usefulness was assessed in 18 of the 33 studies. Usability of mHealth apps was above industry average, with median System Usability Scale scores ranging from 76 to 95 out of 100.

    Conclusions: Current analyses of mHealth app usability are substandard. RCTs are rare, and validated questionnaires are infrequently consulted. Of the 3 domains of usability, only satisfaction is regularly assessed. There is significant bias throughout the literature, particularly with regards to conflicts of interest. Future studies should adhere to the MAUQ to assess usability and improve the utility of mHealth apps.

    JMIR Perioper Med 2020;3(2):e19099

    doi:10.2196/19099

    KEYWORDS



    Introduction

    Industry experts have forecasted significant growth in mobile app users [1]. Given this projected surge, mobile health (mHealth) apps offer a unique and readily accessible platform to the patient, surgeon, and innovator. mHealth apps are now being integrated into various sectors of health care, with over 318,000 [2] apps currently helping to track, educate, and diagnose [3].

    One area of particular growth is the use of mHealth apps as a means of monitoring patients in the important postoperative period. Well-designed apps have the potential to encourage earlier discharge, reduce in-person follow-ups [4,5], rehabilitate [6], aid clinicians in picking up surgical complications [7], and improve communication between patient and health care professional [8]. In addition to the economic and medical benefit of early discharge, postoperative monitoring apps have the potential to empower patients, giving them autonomy over their own health, which in turn might improve patient satisfaction and motivation for recovery [9].

    The usability of mHealth apps is important [10,11] because those with poor usability will be less commonly used [12,13]. This is particularly significant in the postoperative period, given the focus of mHealth apps on rehabilitation, for which patient engagement is critical. One study revealed that around half of all mHealth app users stop engaging for various reasons, including loss of interest [14]. Despite this, little empirical research is undertaken to analyze the usability of mHealth apps before they are launched [15].

    Several definitions and domains of usability have been previously defined without clear unification [11,16,17], but with several recurring themes. For example, the International Organization for Standardization (ISO) 3-pronged definition includes effectiveness (ie, whether users can use the product to complete their goals), efficiency (ie, the extent to which individuals expend resource in achieving their goals), and satisfaction [18]. Another definition [19] has been designed specifically for mHealth apps and includes factors such as mobility, connectivity, and additional cognitive load.

    Different methods have been proposed for assessing domains of usability, such as the Post-Study System Usability Questionnaire [20] and the System Usability Scale (SUS) [21]. However, these tools were not originally created to evaluate mHealth apps. The Mobile App Rating Scale [22] was recently created for researchers and clinicians to assess the quality of mHealth apps, with the simpler user version of the Mobile App Rating Scale (uMARS) [23] being proposed shortly after. While quality of an mHealth app shares several components with usability, there are important differences.

    Given the heterogeneity in definitions and methods used for assessing the usability of mHealth apps, one group has recently developed and validated the 21-item mHealth App Usability Questionnaire (MAUQ) [24]. This tool explores 3 domains of usability, which are in line with the ISO definition: (1) ease of use and satisfaction, akin to ISO satisfaction; (2) system information arrangement, akin to ISO efficiency; and (3) usefulness, akin to ISO effectiveness. This systematic literature review aims to determine whether the usability of postoperative mHealth apps is being rigorously assessed, using the validated MAUQ as the gold-standard reference. We consider which empirical methods are being used and analyze whether postoperative mHealth apps are indeed usable.


    Methods

    Database Search

    The A Measurement Tool to Assess Systematic Reviews checklist [25] was analyzed before conducting this review, with all methodology being established prior to the review being conducted. A university librarian experienced in the field of systematic literature review methodology was consulted. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [26] reporting guideline was adhered to for this review. Rayyan (Qatar Computing Research Institute) [27] software was used for the search.

    Textbox 1 shows the questions that were defined.

    The Medline, Embase, and Association for Computing Machinery Digital Library databases were searched. The search string was generated and aimed to provide maximum coverage while maintaining manageability. We defined 4 broad themes for our search. Terms within a theme were combined using Boolean operator OR, as seen in Table 1. Themes were then combined using Boolean operator AND.


    Textbox 1. Search questions.
    View this box
    Table 1. Search strings for the 4 themes.
    View this table

    Screening of Papers for Inclusion and Exclusion

    Each study recruited from the initial search was evaluated to determine whether it should be admitted for analysis. The inclusion and exclusion criteria are shown in Textbox 2.

    Screening of article titles and abstracts was performed by 2 authors independently. In situations where eligibility of a study could not be determined based on abstract alone, the full-text article was retrieved. We executed a full-text review of the remaining studies after title and abstract screening to further analyze appropriateness for inclusion. We analyzed all review articles to identify any other appropriate studies. We also reviewed the reference list of included papers.


    Textbox 2. Inclusion and exclusion criteria.
    View this box

    Results

    Database Search Results

    The initial search and reference list screening identified 721 studies. After title and abstract screening, 660 were excluded, leaving 61 full-text studies to be assessed. Of these, 28 were excluded, leaving 33 studies included for data extraction. The PRISMA summary of the database search is presented in Figure 1.

    Figure 1. PRISMA summary of the literature search and exclusion process. mHealth: mobile health. PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
    View this figure

    Study Characteristics

    A total of 33 studies were included. Of the 33 studies, 21 were from North America (14 from the United States and 6 from Canada), 9 were from Europe, 2 were from Asia, and 1 was from South America. Most studies specified the type of mobile device used by participants. Smartphones were used in 22 studies, tablets were used in 9, smartwatches were used in 1, iPod touch (Apple Inc) devices were used in 2, and 3 studies did not specify. Regarding the operating system, 11 studies used iOS (Apple Inc), 5 used Android, 1 used Windows (Microsoft Corp), and 17 did not specify.

    Among the included studies, mHealth apps were used within a wide range of surgical subspecialties, including orthopedics (8 studies), general surgery (6 studies), head and neck (4 studies), transplant (3 studies), pediatrics (2 studies), breast (1 study), vascular (1 study), neurosurgery (1 study), and others/multiple (7 studies).

    Functionality was divided into 5 clear categories; 26 studies included monitoring of symptoms or wounds, 8 included educational content, 5 provided a communication platform, 5 included physiotherapy and rehabilitation, and 2 enabled medication management. App details are presented in Table 2.

    Study characteristics are presented in Table 3. With regards to study design, 5 studies were randomized controlled trials (RCTs), 25 were prospective noncontrolled studies, and 3 were retrospective reviews. Sample sizes ranged from 4 to 494, with a median of 39 patients and a mean of 81 patients. Follow-up ranged from 30 minutes postoperation to 12 months postdischarge. The follow-up period was less than 7 days in 4 studies, between 1 week and 1 month in 15 studies, greater than 1 month in 9 studies, and not declared in 5 studies.

    Table 2. App details, including the study, country of origin, type of mobile device used, app name, surgical subspecialty, and app function.
    View this table
    Table 3. Study characteristics, including study design, number of patients included, duration of follow-up, method of usability analysis, usability domain, and selected usability results.
    View this table

    Usability Analysis

    Regarding the method of usability analysis, usage (ie, monitoring of user engagement with the app) was used in 15 studies and was the only usability analysis employed in 4 studies. Interviews were used in 10 studies. Self-created questionnaires were used in 18 studies. Validated questionnaires were used in 9 studies. Of these, 7 used the SUS questionnaire, 1 used the uMARS questionnaire, 1 used the technology acceptance subscale, and 1 used the Computer System Usability Questionnaire (CSUQ).

    We have categorized the domains of usability according to the MAUQ. A total of 28 studies covered ease of use and satisfaction, 11 studies covered system information arrangement, and 18 studies covered usefulness.

    Average SUS scores ranged from 76 to 95 out of 100, with a median score of 87. The uMARS score was 4.1 out of 5. The CSUQ score was 2 out of 7 (whereby a score of 1 would indicate greatest usability).

    Bias

    There is significant potential for bias in studies evaluating the usability of mHealth apps. Hidden agenda bias and secondary gains bias were common and seemingly underreported in the literature. Of the 33 included studies, 8 officially reported authors’ conflicts of interest, stating that they held shares in the app. Furthermore, several of the study groups were provided with the apps free of charge [28], which has clear implications on the usability domain of satisfaction; users who have paid for an app might be expected to have higher expectations than those who have been given an app for free. Perhaps more worryingly, a number of groups [38] declared no conflict of interest, despite seemingly being founders of their app.

    Nonresponse bias is a further concern. Some studies, such as Pecorelli et al [44], had high response rates (96%) to usability analyses. However, others, such as Nilsson et al [43], had much lower rates (57.5% on day 14), and some [51] did not disclose the proportion of responders. Nonresponders to usability analyses are more likely to have reported poor usability. Therefore, studies with high rates of nonresponders are likely to have inflated usability results.

    Population bias is a further issue. Younger audiences are likely to be more adept at using mobile technologies. Therefore, studies that include a younger demographic are likely to demonstrate inflated usability results. Additionally, the generalizability of results from studies [44] that included patients that were not used to mobile technologies may be limited and may change in the future, when greater numbers of older patients are used to mobile technologies.


    Discussion

    Principal Findings

    To our knowledge, this is the first comprehensive systematic review to assess usability of mHealth apps in postoperative management. This review identified 33 studies evaluating the usability of mHealth apps in the postoperative period across a broad range of surgical subspecialties, demonstrating the growing interest in this area. Most of the included studies were derived from the United States and Europe, which appear to be hubs of innovation in the field. Unsurprisingly, smartphones were the most commonly used devices. However, we suspect that wearable devices such as smartwatches, which have additional monitoring capabilities such as electrocardiogram monitors, will play an increasingly important role in the future [61].

    With respect to study designs, 25 of 33 studies were prospective noncontrolled trials. There were 5 RCTs, but usability was never a primary end point in these studies. We feel RCTs comparing mHealth apps to normal practice (eg, in-person follow-up, telephone follow-up, or no follow-up) would be particularly beneficial in assessing the domains of satisfaction and usefulness. It has also been suggested that mHealth app interventions are associated with a falsely heightened level of user satisfaction due to patients’ affinities for their digital devices [62]. This could be minimized by comparing postoperative mHealth apps to a sham app. However, we also acknowledge that RCTs have previously been described as an impractical evaluation methodology for mHealth apps, due to their prolonged duration from recruitment to results and their high costs [63].

    The methodology for assessing usability was generally poor. The majority of analyses used simplistic self-created questionnaires that asked rudimentary questions focusing on the domain of satisfaction (28/33 studies) rather than other domains of usability. Indeed, only 11 of the 33 usability analyses assessed the domain of system information arrangement. We would argue that formal usability analyses should cover all 3 common domains of (1) satisfaction, (2) usefulness, and (3) system arrangement, according to the ISO definition of usability [18]. Validated questionnaires are helpful in assessing these areas reliably. Only 9 of the 33 included studies used validated questionnaires, most of which used the SUS. The SUS is a Likert scale made up of 10 questions. The average SUS score is 68 out of 100, meaning that all 7 studies that used the SUS scored above average in terms of usability. Although the SUS is a quick and cheap means of assessing usability, it was created in 1986, before the first smartphone or the concept of an app was realized. The SUS has not been validated for assessing mHealth apps. In comparison, the MAUQ was recently proposed and validated for use in mHealth apps in a population of English-speaking adults [64]. This is the gold-standard reference for analysis of mHealth app usability. While scores on the MAUQ have previously been shown to correlate with the SUS, this is not a strong correlation (r=0.643), thereby highlighting the inadequacy of studies that have only used the SUS.

    A major concern in these studies is the risk of bias. A number of the studies’ authors have a financial interest in the usability of their apps, with high user satisfaction making adoption by hospitals and investors more likely. Furthermore, devices were sometimes provided free of charge, which could influence the feedback from users.

    Conclusions

    mHealth apps have significant potential during the postoperative period for encouraging earlier discharge, improving patient engagement, and offering a safety net for early identification of complications. Thorough analysis of usability is critical to the adoption of these novel technologies in the postoperative period; those with poor usability will have little impact in health care. According to this review, usability analyses to date have been substandard. They have focused on satisfaction, a narrow dimension of usability, with simplistic self-created questionnaires. Furthermore, there is a significant risk of bias, given the common conflicts of interest among authors of published studies. We hope this review changes future practice, with researchers undertaking more robust assessments of usability by employing validated questionnaires, such as the MAUQ, in blinded RCTs.

    Authors' Contributions

    BP contributed the study conception and design. BP and AT performed the acquisition of data and analysis of data. BP drafted the manuscript. All work was self-funded.

    Conflicts of Interest

    None declared.

    References

    1. Ericsson Mobility Report. Ericsson website.   URL: https://www.ericsson.com/en/mobility-report [accessed 2018-05-29]
    2. The Growing Value of Digital Health. IQVIA Institute. 2017 Nov 07.   URL: https://www.iqvia.com/institute/reports/the-growing-value-of-digital-health [accessed 2018-05-29]
    3. Byambasuren O, Sanders S, Beller E, Glasziou P. Prescribable mHealth apps identified from an overview of systematic reviews. NPJ Digit Med 2018;1:12 [FREE Full text] [CrossRef] [Medline]
    4. Armstrong KA, Coyte PC, Brown M, Beber B, Semple JL. Effect of Home Monitoring via Mobile App on the Number of In-Person Visits Following Ambulatory Surgery: A Randomized Clinical Trial. JAMA Surg 2017 Jul 01;152(7):622-627 [FREE Full text] [CrossRef] [Medline]
    5. Dahlberg K, Philipsson A, Hagberg L, Jaensson M, Hälleberg-Nyman M, Nilsson U. Cost-effectiveness of a systematic e-assessed follow-up of postoperative recovery after day surgery: a multicentre randomized trial. Br J Anaesth 2017 Nov 01;119(5):1039-1046 [FREE Full text] [CrossRef] [Medline]
    6. Belarmino A, Walsh R, Alshak M, Patel N, Wu R, Hu JC. Feasibility of a Mobile Health Application To Monitor Recovery and Patient-reported Outcomes after Robot-assisted Radical Prostatectomy. Eur Urol Oncol 2019 Jul;2(4):425-428. [CrossRef] [Medline]
    7. Hwang JH, Hee Hwang J, Mun GH. An evolution of communication in postoperative free flap monitoring: using a smartphone and mobile messenger application. Plast Reconstr Surg 2012 Jul;130(1):125-129. [CrossRef] [Medline]
    8. Chang P, Lin L, Zhang H, Zhao Y, Xie J, Yu Y, et al. Effect of smartphone application assisted medical service on follow-up adherence improvement in pediatric cataract patients. Graefes Arch Clin Exp Ophthalmol 2018 Oct;256(10):1923-1931. [CrossRef] [Medline]
    9. deBronkart D. From patient centred to people powered: autonomy on the rise. BMJ 2015 Feb 10;350:h148. [CrossRef] [Medline]
    10. Constantinos CK, Kim DJ. A Meta-Analytical Review of Empirical Mobile Usability Studies. J Usability Stud 2011;6(3):117-171 [FREE Full text] [CrossRef]
    11. Zapata BC, Fernández-Alemán JL, Idri A, Toval A. Empirical studies on usability of mHealth apps: a systematic literature review. J Med Syst 2015 Feb;39(2):1. [CrossRef] [Medline]
    12. Schnall R, Mosley JP, Iribarren SJ, Bakken S, Carballo-Diéguez A, Brown IW. Comparison of a User-Centered Design, Self-Management App to Existing mHealth Apps for Persons Living With HIV. JMIR Mhealth Uhealth 2015;3(3):e91 [FREE Full text] [CrossRef] [Medline]
    13. McCurdie T, Taneva S, Casselman M, Yeung M, McDaniel C, Ho W, et al. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol 2012 Sep;Suppl:49-56 [FREE Full text] [CrossRef] [Medline]
    14. Krebs P, Duncan DT. Health App Use Among US Mobile Phone Owners: A National Survey. JMIR Mhealth Uhealth 2015 Nov;3(4):e101 [FREE Full text] [CrossRef] [Medline]
    15. Boulos MNK, Brewer AC, Karimkhani C, Buller DB, Dellavalle RP. Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J Public Health Inform 2014;5(3):229 [FREE Full text] [CrossRef] [Medline]
    16. Nassar V. Common criteria for usability review. Work 2012;41 Suppl 1:1053-1057. [CrossRef] [Medline]
    17. Baharuddin R, Singh D, Razali R. Usability Dimensions for Mobile Applications-A Review. RJASET 2013 Feb 21;11(9):2225-2231. [CrossRef]
    18. ISO 9241. International Organisation for Standardization.   URL: https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en [accessed 2019-11-08]
    19. Harrison R, Flood D, Duce D. Usability of mobile applications: literature review and rationale for a new usability model. J Interact Sci 2013;1(1):1. [CrossRef]
    20. Lewis JR. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. International Journal of Human-Computer Interaction 1995 Jan;7(1):57-78. [CrossRef]
    21. Brooke J. SUS - A quick and dirty usability scale. Usability Evaluation In Industry 1996 Jun 11:189-194 [FREE Full text]
    22. Stoyanov SR, Hides L, Kavanagh DJ, Zelenko O, Tjondronegoro D, Mani M. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR Mhealth Uhealth 2015 Mar;3(1):e27 [FREE Full text] [CrossRef] [Medline]
    23. Stoyanov SR, Hides L, Kavanagh DJ, Wilson H. Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS). JMIR Mhealth Uhealth 2016 Jun 10;4(2):e72 [FREE Full text] [CrossRef] [Medline]
    24. Zhao B, Tai-Seale M, Longhurst C, Clay B. Utilization of Hospital Room Hospitality Features on Patient-Controlled Tablet Computers: Cohort Study. JMIR Mhealth Uhealth 2019 Jun 20;7(6):e13964 [FREE Full text] [CrossRef] [Medline]
    25. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017 Sep 21;358:j4008 [FREE Full text] [CrossRef] [Medline]
    26. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ 2009 Jul 21;339:b2535 [FREE Full text] [CrossRef] [Medline]
    27. Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev 2016 Dec 05;5(1):210 [FREE Full text] [CrossRef] [Medline]
    28. Timmers T, Janssen L, van der Weegen W, Das D, Marijnissen W, Hannink G, et al. The Effect of an App for Day-to-Day Postoperative Care Education on Patients With Total Knee Replacement: Randomized Controlled Trial. JMIR Mhealth Uhealth 2019 Oct 21;7(10):e15323 [FREE Full text] [CrossRef] [Medline]
    29. Yadav SK, Jha CK, Mishra SK, Mishra A. Smartphone-Based Application for Tele-follow-up of Patients with Endocrine Disorders in Context of a LMIC: A Compliance, Satisfaction, Clinical Safety and Outcome Assessment. World J Surg 2020 Feb;44(2):612-616. [CrossRef] [Medline]
    30. Ramkumar PN, Haeberle HS, Ramanathan D, Cantrell WA, Navarro SM, Mont MA, et al. Remote Patient Monitoring Using Mobile Health for Total Knee Arthroplasty: Validation of a Wearable and Machine Learning-Based Surveillance Platform. J Arthroplasty 2019 Oct;34(10):2253-2259. [CrossRef] [Medline]
    31. Argent R, Slevin P, Bevilacqua A, Neligan M, Daly A, Caulfield B. Wearable Sensor-Based Exercise Biofeedback for Orthopaedic Rehabilitation: A Mixed Methods User Evaluation of a Prototype System. Sensors (Basel) 2019 Jan 21;19(2) [FREE Full text] [CrossRef] [Medline]
    32. Brunner TH, DiFortuna K, LeTang M, Murphy J, Stemplewicz K, Kovacs M, et al. Feasibility of an iPad to Facilitate Communication in Postoperative Patients With Head and Neck Cancer. J Perianesth Nurs 2018 Aug;33(4):399-406 [FREE Full text] [CrossRef] [Medline]
    33. van der Meij E, Huirne JA, Ten Cate AD, Stockmann HB, Scholten PC, Davids PH, et al. A Perioperative eHealth Program to Enhance Postoperative Recovery After Abdominal Surgery: Process Evaluation of a Randomized Controlled Trial. J Med Internet Res 2018 Jan 02;20(1):e1 [FREE Full text] [CrossRef] [Medline]
    34. Felbaum DR, Stewart J, Anaizi A, Sandhu FA, Nair MN, Voyadzis JM. Implementation and Evaluation of a Smartphone Application for the Perioperative Care of Neurosurgery Patients at an Academic Medical Center: Implications for Patient Satisfaction, Surgery Cancelations, and Readmissions. Oper Neurosurg (Hagerstown) 2018 Mar 01;14(3):303-311. [CrossRef] [Medline]
    35. Goz V, Anthony C, Pugely A, Lawrence B, Spina N, Brodke D, et al. Software-Based Postoperative Communication With Patients Undergoing Spine Surgery. Global Spine J 2019 Feb;9(1):14-17 [FREE Full text] [CrossRef] [Medline]
    36. Gunter RL, Fernandes-Taylor S, Rahman S, Awoyinka L, Bennett KM, Weber SM, et al. Feasibility of an Image-Based Mobile Health Protocol for Postoperative Wound Monitoring. J Am Coll Surg 2018 Mar;226(3):277-286 [FREE Full text] [CrossRef] [Medline]
    37. Gustavell T, Langius-Eklöf A, Wengström Y, Segersvärd R, Sundberg K. Development and Feasibility of an Interactive Smartphone App for Early Assessment and Management of Symptoms Following Pancreaticoduodenectomy. Cancer Nurs 2019;42(3):E1-E10. [CrossRef] [Medline]
    38. Harder H, Holroyd P, Burkinshaw L, Watten P, Zammit C, Harris PR, et al. A user-centred approach to developing bWell, a mobile app for arm and shoulder exercises after breast cancer treatment. J Cancer Surviv 2017 Dec;11(6):732-742 [FREE Full text] [CrossRef] [Medline]
    39. Higgins J, Semple J, Murnaghan L, Sharpe S, Theodoropoulos J. Mobile Web-Based Follow-up for Postoperative ACL Reconstruction: A Single-Center Experience. Orthop J Sports Med 2017 Dec;5(12):2325967117745278 [FREE Full text] [CrossRef] [Medline]
    40. Highland KB, Tran J, Edwards H, Bedocs P, Suen J, Buckenmaier CC. Feasibility of App-Based Postsurgical Assessment of Pain, Pain Impact, and Regional Anesthesia Effects: A Pilot Randomized Controlled Trial. Pain Med 2019 Aug 01;20(8):1592-1599. [CrossRef] [Medline]
    41. Khanwalkar AR, Shen J, Kern RC, Welch KC, Smith SS, Tan BK, et al. Utilization of a novel interactive mobile health platform to evaluate functional outcomes and pain following septoplasty and functional endoscopic sinus surgery. Int Forum Allergy Rhinol 2019 Apr;9(4):345-351. [CrossRef] [Medline]
    42. Mata J, Pecorelli N, Kaneva P, Moldoveanu D, Gosselin-Tardiff A, Alhashemi M, et al. A mobile device application (app) to improve adherence to an enhanced recovery program for colorectal surgery: a randomized controlled trial. Surg Endosc 2020 Feb;34(2):742-751. [CrossRef] [Medline]
    43. Nilsson U, Dahlberg K, Jaensson M. The Swedish Web Version of the Quality of Recovery Scale Adapted for Use in a Mobile App: Prospective Psychometric Evaluation Study. JMIR Mhealth Uhealth 2017 Dec 03;5(12):e188 [FREE Full text] [CrossRef] [Medline]
    44. Pecorelli N, Fiore JF, Kaneva P, Somasundram A, Charlebois P, Liberman AS, et al. An app for patient education and self-audit within an enhanced recovery program for bowel surgery: a pilot study assessing validity and usability. Surg Endosc 2018 May;32(5):2263-2273. [CrossRef] [Medline]
    45. Sousa CS, Turrini R. Development of an educational mobile application for patients submitted to orthognathic surgery. Rev Lat Am Enfermagem 2019 Jul 18;27:e3143 [FREE Full text] [CrossRef] [Medline]
    46. Sun T, Dunsmuir D, Miao I, Devoy GM, West NC, Görges M, et al. In-hospital usability and feasibility evaluation of Panda, an app for the management of pain in children at home. Paediatr Anaesth 2018 Oct;28(10):897-905. [CrossRef] [Medline]
    47. Tsapepas DS, Salerno D, Jandovitz N, Hammad S, Jordan P, Mohan S, et al. Using technology to enhance medication regimen education after solid organ transplantation. Am J Health Syst Pharm 2018 Dec 01;75(23):1930-1937. [CrossRef] [Medline]
    48. Scott AR, Alore EA, Naik AD, Berger DH, Suliburk JW. Mixed-Methods Analysis of Factors Impacting Use of a Postoperative mHealth App. JMIR Mhealth Uhealth 2017 Feb 08;5(2):e11 [FREE Full text] [CrossRef] [Medline]
    49. Warren-Stomberg M, Jacobsson J, Brattwall M, Jildenstål P. At-home monitoring after surgery/anaesthesia - a challenge. J Eval Clin Pract 2016 Dec;22(6):882-886. [CrossRef] [Medline]
    50. Debono B, Bousquet P, Sabatier P, Plas J, Lescure J, Hamel O. Postoperative monitoring with a mobile application after ambulatory lumbar discectomy: an effective tool for spine surgeons. Eur Spine J 2016 Nov;25(11):3536-3542. [CrossRef] [Medline]
    51. Gunter R, Fernandes-Taylor S, Mahnke A, Awoyinka L, Schroeder C, Wiseman J, et al. Evaluating Patient Usability of an Image-Based Mobile Health Platform for Postoperative Wound Monitoring. JMIR Mhealth Uhealth 2016 Sep 28;4(3):e113 [FREE Full text] [CrossRef] [Medline]
    52. Ponce BA, Brabston E, Zu S. Telemedicine with mobile devices and augmented reality for early postoperative care. 2016 Presented at: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); Aug 16-20, 2016; Orlando, FL. [CrossRef]
    53. Jiang Y, Sereika S, Dabbs AD, Handler S, Schlenk E. Acceptance and Use of Mobile Technology for Health Self-Monitoring in Lung Transplant Recipients during the First Year Post-Transplantation. Appl Clin Inform 2016;7(2):430-445 [FREE Full text] [CrossRef] [Medline]
    54. Chai YJ, Song J, Kang J, Woo J, Song R, Kwon H, et al. A comparative study of postoperative pain for open thyroidectomy versus bilateral axillo-breast approach robotic thyroidectomy using a self-reporting application for iPad. Ann Surg Treat Res 2016 May;90(5):239-245 [FREE Full text] [CrossRef] [Medline]
    55. Shellmer DA, Dew MA, Mazariegos G, DeVito Dabbs A. Development and field testing of Teen Pocket PATH(®), a mobile health application to improve medication adherence in adolescent solid organ recipients. Pediatr Transplant 2016 Feb;20(1):130-140 [FREE Full text] [CrossRef] [Medline]
    56. Sun T, West N, Ansermino JM, Montgomery CJ, Myers D, Dunsmuir D, et al. A smartphone version of the Faces Pain Scale-Revised and the Color Analog Scale for postoperative pain assessment in children. Paediatr Anaesth 2015 Dec;25(12):1264-1273. [CrossRef] [Medline]
    57. Jaensson M, Dahlberg K, Eriksson M, Grönlund Å, Nilsson U. The Development of the Recovery Assessments by Phone Points (RAPP): A Mobile Phone App for Postoperative Recovery Monitoring and Assessment. JMIR Mhealth Uhealth 2015 Sep 11;3(3):e86 [FREE Full text] [CrossRef] [Medline]
    58. Symer MM, Abelson JS, Milsom J, McClure B, Yeo HL. A Mobile Health Application to Track Patients After Gastrointestinal Surgery: Results from a Pilot Study. J Gastrointest Surg 2017 Sep;21(9):1500-1505. [CrossRef] [Medline]
    59. Semple JL, Sharpe S, Murnaghan ML, Theodoropoulos J, Metcalfe KA. Using a mobile app for monitoring post-operative quality of recovery of patients at home: a feasibility study. JMIR Mhealth Uhealth 2015 Feb 12;3(1):e18 [FREE Full text] [CrossRef] [Medline]
    60. Bini S, Mahajan J. Clinical outcomes of remote asynchronous telerehabilitation are equivalent to traditional therapy following total knee arthroplasty: A randomized control study. J Telemed Telecare 2017 Feb;23(2):239-247. [CrossRef] [Medline]
    61. Smartwatch Market - Growth, Trends, Forecasts. Mordor Intelligence.   URL: https://www.mordorintelligence.com/industry-reports/smartwatch-market [accessed 2020-01-28]
    62. Torous J, Firth J. The digital placebo effect: mobile mental health meets clinical psychiatry. Lancet Psychiatry 2016 Feb;3(2):100-102. [CrossRef] [Medline]
    63. Pham Q, Wiljer D, Cafazzo JA. Beyond the Randomized Controlled Trial: A Review of Alternatives in mHealth Clinical Trial Methods. JMIR Mhealth Uhealth 2016 Sep 09;4(3):e107 [FREE Full text] [CrossRef] [Medline]
    64. Zhou L, Bao J, Setiawan IMA, Saptono A, Parmanto B. The mHealth App Usability Questionnaire (MAUQ): Development and Validation Study. JMIR Mhealth Uhealth 2019 Apr 11;7(4):e11500 [FREE Full text] [CrossRef] [Medline]


    Abbreviations

    CSUQ: Computer System Usability Questionnaire
    ISO: Organization for Standardization
    MAUQ: mHealth App Usability Questionnaire
    mHealth: mobile health
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    SUS: System Usability Scale
    uMARS: user version of the Mobile App Rating Scale


    Edited by R Lee; submitted 03.04.20; peer-reviewed by M Tai-Seale, J Hitt, Y Chu; comments to author 07.05.20; revised version received 21.05.20; accepted 26.05.20; published 20.07.20

    ©Ben Patel, Arron Thind. Originally published in JMIR Perioperative Medicine (http://periop.jmir.org), 20.07.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Perioperative Medicine, is properly cited. The complete bibliographic information, a link to the original publication on http://periop.jmir.org, as well as this copyright and license information must be included.