This study involved a psychometric evaluation of data originating from a multicenter, 2-group, parallel, single-blind, randomized controlled trial conducted from October 2015 to July 2016 at 4 day surgery departments in Sweden. The primary aim was to estimate the cost-effectiveness of using, vs not using, RAPP for follow-up on recovery after day surgery [19]. This study involves only those participants who were randomized into the intervention group and who underwent local or peripheral block anesthesia. This study followed the ethical standards of the Helsinki Declaration (6th revision) and was approved by the Uppsala Regional Ethics Committee (2015/262).

The data collection procedure was as follows. Information on the planned surgery was provided to the patients together with written information on the study. Upon their arrival at the day surgery department, a research nurse provided patients with oral information about the study and invited them to enroll. The inclusion criteria were age ≥18 years, undergoing day surgery, able to understand written and spoken Swedish, and access to a smartphone. Exclusion criteria were memory and visual impairment, undergoing surgical abortion, and ongoing substance abuse.

The SwQoR-LA includes 16 of the 24 postoperative symptoms included in the SwQoR. The symptoms are scored on an 11-point numeric visual analogue scale from 0 (“none of the time”) to 10 (“all of the time”). Each question appears separately on the screen, and a dot on the visual analogue scale has to be moved to indicate an answer. The symptoms disappear from the screen immediately after a response is given, and each question on a symptom must be answered to submit the daily assessment [20].

Preoperatively, the research nurse assisted with the installation of RAPP, including SwQoR, onto each participant’s smartphone for both participants who underwent general and local or peripheral block anesthesia. The participants were encouraged to do a test run of the app by putting in fake responses. The research nurse also explained other functionalities of the RAPP, such as how to move between the items and how to use the navigation keys.

The participants were instructed to complete the SwQoR in the RAPP every day until postoperative day 14. A daily reminder helped the participants to remember to send in their daily report on their recovery. The health care professionals at the day surgery department had access to all patient data via a web administrator interface.

This study includes data for the 16 symptoms (ie, SwQoR-LA) on postoperative days 1-7 from the participants that underwent local or peripheral block anesthesia. Based on the opinion of both patients and clinicians using RAPP in clinical practice, 7 days of assessment was considered appropriate, as a short recovery period after minor surgery with local or peripheral block anesthesia is expected. In addition to the SwQoR, other collected variables were age, gender, American Society of Anesthesiologists physical status, and type of anesthesia.

The psychometric evaluation was guided by the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) [21] and a previous psychometric evaluation of the SwQoR [18]. Acceptability, which measures the clinical user friendliness, was assessed in terms of the successful response rate on postoperative days 1-7. Floor and ceiling effects (ie, the number of respondents who achieved the lowest or highest possible scores) were measured on days 1-7; it was considered a problem if more than 15% of the study population achieved the lowest or highest possible score [22]. Construct validity is the extent to which questionnaire scores are consistent with hypotheses, assuming that the questionnaire validly measures the construct being addressed. A correlation coefficient >0.4 was considered to be evidence of construct validity (ie, moderate to strong correlation). To analyze construct validity, a priori hypothesis testing was conducted, under the hypothesis that the SwQoR-LA, just as with the SwQoR [18], on day 1 would correlate positively with the duration of surgery, duration of stay at the postanesthesia care unit (PACU), and patient age. In addition, lower quality of recovery (ie, higher degree of postoperative symptoms) was not expected in women versus men, just as with the SwQoR [18]. Discriminant validity was tested on day 7, and it was expected that patients who underwent local anesthesia would have significantly lower scores on the symptoms related to general anesthesia that are not included in the SwQoR-LA: voice not sounding the same as usual, sore throat, sore mouth, having trouble breathing, muscle pain, trouble urinating, diarrhea, and feeling constipated. For example, sore throat and sore mouth are symptoms related to the endotracheal tub or laryngeal mask used under general anesthesia.

Reliability was assessed with (1) internal consistency, by measuring the average correlation between the SwQoR items on days 1-7, indicated by Cronbach α, and (2) split-half reliability, by measuring the correlation between randomly split segments of the SwQoR on days 1-7. Responsiveness, which was used to evaluate the SwQoR-LA’s sensitivity and ability to detect clinically important changes, was measured with (1) Cohen d effect size, calculated as average changes in scores from days 1 to 7, divided by the pooled SD of all measurements (where 0.2-0.5 indicates a small effect, 0.5-0.8 a moderate effect, and 0.8-1.2 a large effect) [23], and (2) mean changes over time and percent changes from baseline on days 1-7.

The sample size was calculated for the original randomized controlled trial [19]; therefore, no sample size was calculated for the SwQoR-LA. Descriptive statistics are presented as means, SDs, numbers and percentages, ranges or minimum-maximum, or 95% CI for the sake of clarity. In this study, when analyzing the overall level of recovery after local anesthesia, we used the global score of the SwQoR-LA, with a minimum value of 0 and a maximum value of 160.

To investigate differences between symptoms and gender, the Mann-Whitney U-test was performed. Associations were measured with Spearman rank coefficients (rho). Cronbach α and split-half reliability with the Spearman-Brown coefficient were used to assess the internal consistency. SPSS version 24 (IBM Corp, Armonk, NY) for Windows was used for the statistical analyses. The null hypothesis was rejected at a two-tailed P<.05.

Of the 513 patients, 19 were excluded due to cancelled operations (n=15), refusal to participate (n=3), or technical issues (n=1), leaving 494 patients. Of the remaining patients, 107 underwent local or peripheral nerve block anesthesia, 362 underwent general anesthesia, and 25 had missing information about the type of anesthesia and were thereby excluded from the analysis. The results of this study only include the patients who underwent local anesthesia (n=107), except for the discriminant validity analysis. Patients’ demographic variables and perioperative factors are presented in Table 1.

The response rate was 88.8% (95/107) on postoperative day 1 and 72.9% (78/107) on day 7. The global SwQoR-LA score decreased from 35.7 (SD 24.4) on day 1 to 15.5 (SD 15.5) on day 7 (Table 2).

Because the patients had to respond to each item in order to move on to the next item, there were no missing answers. Pain in the surgical wound was the symptom that occurred most frequently, starting with a value of 4.6 on day 1 and ending at 1.8 on day 7 (Figure 1).

The distributions of the SwQoR-LA scores on days 1-7 were skewed to the left and ranged between 0 and 101. No patient gave the maximum score (ie, there was no ceiling effect). No floor effects were present either (Table 3).

Construct validity analysis indicated low correlations between the SwQoR-LA on day 1 and PACU stay (rho=0.21, P=.05), duration of surgery (rho=0.28, P<.001), and patient age (rho=0.18, P=.11). There were no significant differences in global SwQoR-LA between the genders on day 1: women, 38.7 (SD 24.9) versus men, 29.8 (22.6).

Discriminant validity was determined by comparing 8 symptoms related to general anesthesia on postoperative day 1 between patients who had undergone general anesthesia and patients who had undergone local anesthesia. All symptoms except for “Diarrhea” were significantly lower in the patients who had undergone local anesthesia (Table 4).

Regarding internal consistency, the Cronbach α for the sum score of the SwQoR-LA ranged between .87 and .89, while the split-half coefficient ranged between 0.82 and 0.90 (Table 3).

Cohen d effect size between days 1 and 7 was 0.98. The mean change in the global SwQoR-LA score from day 1 to day 7 was –19.7 (SD 19.4) with a 95% CI of 15.2-24.2, P<.001. The percentage of change from baseline was 43.4%.

There are some limitations in our study. First, the sample size is relatively small, but considered sufficient for examining psychometric properties with 16 items. However, there is no consensus about the number of participants for each type of psychometric analysis. To analyze construct validity, responsiveness, and floor and ceiling effects, a sample size of at least 50 participants is recommended [22]. Second, no test-retest reliability was conducted. This feature could be improved in future studies by involving a larger pool of patients undergoing a wider range of peripheral nerve block. Third, the duration for data entry was not measured.

To our knowledge, this study is the first to evaluate a real-time recovery questionnaire, the SwQoR-LA, in patients undergoing local or peripheral nerve block anesthesia. The SwQoR-LA is valid, reliable, responsive, and clinically feasible for the real-time assessment of patient recovery in order to detect those patients who might benefit from timely follow-up and intervention.