This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Perioperative Medicine, is properly cited. The complete bibliographic information, a link to the original publication on http://periop.jmir.org, as well as this copyright and license information must be included.
Pelvic organ prolapse (POP) refers to symptomatic descent of the vaginal wall. To reduce surgical failure rates, surgical correction can be augmented with the insertion of polypropylene mesh. This benefit is offset by the risk of mesh complication, predominantly mesh exposure through the vaginal wall. If mesh placement is under consideration as part of prolapse repair, patient selection and counseling would benefit from the prediction of mesh exposure; yet, no such reliable preoperative method currently exists. Past studies indicate that inflammation and associated cytokine release is correlated with mesh complication. While some degree of mesh-induced cytokine response accompanies implantation, excessive or persistent cytokine responses may elicit inflammation and implant rejection.
Here, we explore the levels of biomaterial-induced blood cytokines from patients who have undergone POP repair surgery to (1) identify correlations among cytokine expression and (2) predict postsurgical mesh exposure through the vaginal wall.
Blood samples from 20 female patients who previously underwent surgical intervention with transvaginal placement of polypropylene mesh to correct POP were collected for the study. These included 10 who experienced postsurgical mesh exposure through the vaginal wall and 10 who did not. Blood samples incubated with inflammatory agent lipopolysaccharide, with sterile polypropylene mesh, or alone were analyzed for plasma levels of 13 proinflammatory and anti-inflammatory cytokines using multiplex assay. Data were analyzed by principal component analysis (PCA) to uncover associations among cytokines and identify cytokine patterns that correlate with postsurgical mesh exposure through the vaginal wall. Supervised machine learning models were created to predict the presence or absence of mesh exposure and probe the number of cytokine measurements required for effective predictions.
PCA revealed that proinflammatory cytokines interferon gamma, interleukin 12p70, and interleukin 2 are the largest contributors to the variance explained in PC 1, while anti-inflammatory cytokines interleukins 10, 4, and 6 are the largest contributors to the variance explained in PC 2. Additionally, PCA distinguished cytokine correlations that implicate prospective therapies to improve postsurgical outcomes. Among machine learning models trained with all 13 cytokines, the artificial neural network, the highest performing model, predicted POP surgical outcomes with 83% (15/18) accuracy; the same model predicted POP surgical outcomes with 78% (14/18) accuracy when trained with just 7 cytokines, demonstrating retention of predictive capability using a smaller cytokine group.
This preliminary study, incorporating a sample size of just 20 participants, identified correlations among cytokines and demonstrated the potential of this novel approach to predict mesh exposure through the vaginal wall following transvaginal POP repair surgery. Further study with a larger sample size will be pursued to confirm these results. If corroborated, this method could provide a personalized medicine approach to assist surgeons in their recommendation of POP repair surgeries with minimal potential for adverse outcomes.
Pelvic organ prolapse (POP), defined as symptomatic descent of the vagina and surrounding pelvic organs, affects approximately 50% of parous women and 6% of nonparous women between ages 20 and 59 years [
Inflammatory responses are associated with mesh exposure due to asymptomatic mesh infection that inhibits the mesh from integrating with the surrounding environment [
Leveraging a patient-specific, multifaceted immune response for the prediction of postsurgical complications is an ideal problem for the application of principal component analysis (PCA) and supervised machine learning models. In fact, this approach has been used to predict complications following other surgical procedures as well as progressive disease outcomes. In a liver transplant study, Raji and Vinod Chandra [
PCA and supervised machine learning have also been applied to biological measurements for predicting medical outcomes. Tseng et al [
The application of PCA and machine learning to predict postsurgical complications in women after POP surgery has also shown promising results. In the study of Jelovsek et al [
In this preliminary study, we explored the levels of baseline and stimulus-induced cytokines in blood isolated from patients who had undergone POP repair surgery with a polypropylene mesh. Proinflammatory and anti-inflammatory cytokine levels from these data were analyzed using PCA to establish the principal components (PCs) and to identify associative or opposing trends among cytokines. In addition, supervised machine learning models were applied to demonstrate predictive capabilities when models were trained with either all 13 cytokines or a smaller group of 7 cytokines determined most effective by a random forest method. The results demonstrate that leveraging PCA and supervised machine learning models to predict outcomes of vaginal mesh implantation has the potential to benefit future patients when they are faced with this surgical decision, which carries a relatively high risk of unsuccessful surgical outcome.
In total, 20 healthy, nonpregnant female participants aged 56-89 years at Prisma Health Greenville Memorial Hospital with a history of surgical intervention to correct POP via a procedure that used polypropylene mesh were selected for the study. The participants, who were not matched, included 10 who experienced postsurgical mesh complication in which the implanted mesh protruded through the vaginal wall (also referred to as mesh exposure) and 10 participants who did not experience this complication post surgery. This sample size was estimated as an effective cohort for the pilot study using a 1-tailed
The study protocol was approved by the institutional review board (IRB) of Prisma Health (Pro00067964). Informed consent from all study participants was obtained using an IRB-approved informed consent form. All samples collected and data analyzed were deidentified and followed IRB protocol.
Blood samples were obtained from the 20 selected participants. Approximately 12 mL of blood was drawn from the upper extremity of each participant into 3 BD Vacutainer EDTA-coated tubes. Deidentified blood samples were then transferred on ice to a laboratory facility at the University of South Carolina School of Medicine Greenville for immediate processing. Each participant’s blood sample was divided into equal aliquots for 24-hour incubation at 37 °C under 3 distinct conditions: (1) incubation with inflammatory agent lipopolysaccharide (LPS) at 20 ng/mL (positive control), (2) incubation with sterile polypropylene mesh area of 2 cm × 2 cm (experimental), and (3) incubation alone (negative control). After incubation, the plasma layer was collected following centrifugation (1500 × g, 10 min, 4 °C) and immediately stored at –80 °C.
Cytokine levels in each blood sample were quantified using the bead-based MILLIPLEX Human Cytokine/Chemokine/Growth Factor Panel A—Immunology Multiplex Assay (EMD Millipore Corp), which is composed of analytes for target cytokines interleukin 1α (IL-1α), IL-1β, IL‑2, IL-4, IL-6, IL-8, IL-10, IL-12p40, IL-12p70, IL-17A, interferon-gamma (IFN-γ, tumor necrosis factor-alpha (TNF-α), and granulocyte-macrophage colony-stimulating factor (GM-CSF). Frozen plasma samples were thawed at room temperature and analyzed following Milliplex protocol guidelines. Cytokine concentrations were measured using a Bio-Plex 200 (Bio-Rad) and Bio-Plex Manager software (Bio-Rad). Sample volume was doubled to ensure measurable levels of cytokines, and assay output data were adjusted to reflect concentrations in plasma samples. Each multiplex assay was performed in duplicate, and cytokine levels were evaluated in 3 independent measurements.
Cytokine data gathered from the multiplex immunoassay were analyzed using data mining and predictive analytical methods. PCA was used to identify important cytokines by studying their contributions to each PC as well as to discern associations between cytokines. Supervised machine learning models were created to determine whether cytokine levels can accurately predict which patients are more likely to experience mesh exposure post surgery.
The statistical programming language R (version 4.1.2; R Foundation) was used to analyze raw cytokine data values generated from the multiplex immunoassay. The imported data structure contained 60 observations (20 participants × 3 independent measurements) and 40 total variable fields (13 cytokines × 3 blood treatments + 1 target variable). The target variable was the participant’s outcome, which indicated a postsurgical complication that participants might have experienced following POP surgery. Observations marked “presence” represent participants who experienced mesh exposure through the vaginal wall. Observations marked “absence” represent participants who did not experience any mesh exposure through the vaginal wall. Univariate and multivariate methods were used to explore the data set, including identifying missing values, analyzing outliers, and visualizing frequency distributions.
PCA was performed using the FactoMineR package (version 2.4; R Foundation) [
Supervised machine learning models were created using the caret package (version 6.0-90; R Foundation) [
Additionally, this process was replicated to study the effects of reducing the number of cytokines needed to predict a postsurgical mesh exposure. A random forest algorithm was used to select important cytokines for this study. These models were trained and tested for accuracy, sensitivity, and specificity as detailed above. The results are compared to models trained with all 13 cytokines.
To identify significant associations among the cytokines, PCA was used to examine a total of 60 blood samples (20 participants × 3 blood treatments). Among the 20 participants, 10 experienced postsurgical mesh exposure through the vaginal wall and 10 did not.
PCA was performed using cytokine levels in blood samples of postsurgical POP subjects; the analysis included 60 blood samples (20 subjects × 3 blood treatments), wherein each sample was evaluated in 3 independent measurements performed in duplicate. Biplots illustrating individual cytokines were constructed for blood samples incubated in the presence of LPS (A), incubated in the presence of polypropylene mesh (B), or incubated alone (C). Arrow direction indicates the cytokine correlation; arrow length indicates the magnitude of the variation. IL-1α: interleukin-1 alpha; IL-1β: interleukin-1 beta; IL-2: interleukin-2; IL-4: interleukin-4; IL-6: interleukin-6; IL-8: interleukin-8; IL-10: interleukin-10; IL-12p40: interleukin-12p40; IL-12 p70: interleukin-12p70; IL-17A: interleukin-17A; IFN-γ: interferon gamma; TNF-α: tumor necrosis factor-alpha; GM-CSF: granulocyte-macrophage colony-stimulating factor; PC: principal component.
When PCA was used to examine only blood samples incubated with polypropylene mesh, PC 1 and PC 2 explained 60.1% and 13.1% of the total data variance, respectively (
In order to visualize associations between the participants presenting the absence or presence of postsurgical mesh exposure through the vaginal wall, a biplot illustrating individual participants was created (
PCA was performed using cytokine levels in patient blood samples incubated with polypropylene mesh; the analysis included 20 blood samples (20 subjects × 1 blood treatment), wherein each sample was evaluated in 3 independent measurements performed in duplicate. Each cytokine’s contribution to PC 1 (A) and PC 2 (B) was determined. The dashed line at 7% corresponds to the expected value if the contribution were uniform. IL-1α: interleukin-1 alpha; IL-1β: interleukin-1 beta; IL-2: interleukin-2; IL-4: interleukin-4; IL-6: interleukin-6; IL-8: interleukin-8; IL-10: interleukin-10; IL-12p40: interleukin-12p40; IL-12p70: interleukin-12p70; IL-17A: interleukin-17A; IFN-γ: interferon gamma; TNF-α: tumor necrosis factor-alpha; GM-CSF: granulocyte-macrophage colony-stimulating factor; PC: principal component.
PCA was performed using cytokine levels in patient blood samples incubated with polypropylene mesh; the analysis included 20 blood samples (20 subjects × 1 blood treatment), wherein each sample is represented by the average of 3 independent measurements performed in duplicate. A biplot was constructed illustrating individual patient averages (indicated by numbers) exhibiting the presence (red triangle) or absence (green circle) of mesh exposure through the vaginal wall. Concentration ellipses draw focus to the distribution of a group with the presence (red) or absence (green) of mesh exposure. Centroids of the concentration ellipses (large symbols) indicate the mean of each group. PC: principal component.
Four supervised machine learning models incorporating all 13 cytokines were trained using 70% (42/60) of the available 60 observations (20 participants × 3 independent measurements); the remaining 30% (18/60) was used to test the models’ accuracy when predicting the presence of mesh exposure through the vaginal wall. All 4 machine learning machines achieved at least 62% (26/42) training accuracy (
Summary of supervised learning model statistics. All 13 cytokines were used to predict the presence or absence of postsurgical mesh exposure through the vaginal wall; 70% (42/60) of observations were used for training, and 30% (18/60) of observations were used for testing.
Model | Training accuracy, n (%) | Prediction accuracy, n (%) | 95% CI | Sensitivity, n (%) | Specificity, n (%) | Prediction, κ |
Artificial neural network | 33 (79) | 15 (83) | 0.586-0.964 | 14 (78) | 16 (89) | 0.667 |
Decision tree | 27 (64) | 11 (61) | 0.57-0.827 | 6 (33) | 16 (89) | 0.222 |
Naïve Bayes | 26 (62) | 11 (61) | 0.357-0.827 | 6 (33) | 16 (89) | 0.222 |
Logistic regression | 31 (73) | 9 (50) | 0.260-0.740 | 10 (56) | 8 (44) | 0.000 |
Additional models and predictive analyses explored whether a smaller set of cytokines could achieve similar predictive results. Feature selection using a random forest method identified a group of 7 cytokines capable of yielding effective predictive analysis: IL-1β, IL-8, IL-12p40, IL-12p70, TNF-α, IL-17A, and IL-6.
A random forest algorithm was used to identify a group of 7 cytokines capable of yielding effective predictive analysis. The importance of each cytokine is evident in individual supervised learning models: ANN (A), DT (B), NB (C), and LR (D). IL-1α: interleukin-1 alpha; IL-1β: interleukin-1 beta; IL-2: interleukin-2; IL-4: interleukin-4; IL-6: interleukin-6; IL-8: interleukin-8; IL-10: interleukin-10; IL-12p40: interleukin-12p40; IL-12p70: interleukin-12p70; IL-17A: interleukin-17A; IFN-γ: interferon gamma; TNF-α: tumor necrosis factor-alpha; GM-CSF: granulocyte-macrophage colony-stimulating factor; PC: principal component; ANN: artificial neural network; DT: decision tree; NB: Naïve Bayes; LR: logistic regression.
Summary of supervised learning model statistics. Feature selection via random forest was used to identify a group of 7 cytokines capable of yielding effective predictive analysis. The subset of cytokines was used to predict the presence or absence of postsurgical mesh exposure through the vaginal wall; 70% (42/60) of observations were used for training, and 30% (18/60) of observations were used for testing.
Model | Training accuracy, n (%) | Prediction accuracy, n (%) | 95% CI | Sensitivity, n (%) | Specificity, n (%) | Prediction, κ |
Artificial neural network | 34 (81) | 14 (78) | 0.524-0.936 | 12 (67) | 16 (89) | 0.556 |
Decision tree | 27 (64) | 11 (61) | 0.356-0.827 | 6 (33) | 16 (89) | 0.222 |
Naïve Bayes | 30 (72) | 10 (56) | 0.308-0.785 | 6 (33) | 14 (78) | 0.111 |
Logistic regression | 34 (81) | 13 (72) | 0.465-0.903 | 12 (67) | 14 (78) | 0.444 |
Among patients with POP who undergo mesh implantation surgery, 17% of them experience mesh exposure through the vaginal wall [
The PCA analysis illustrated in
The cytokines that contribute most to each PC segregate into proinflammatory and anti-inflammatory cytokines (
When juxtaposing the biplot of polypropylene-stimulated cytokine observations (
When creating models trained with a subgroup of 7 cytokines (
This study implemented rigorous research methods to identify physiological relationships among cytokine markers and developed robust machine learning models to predict mesh exposure; yet, some limitations should be noted. First, because this is a pilot study, the sample size is limited to 20 participants within a single hospital system. Nevertheless, this limited sample size predicted 83% (15/18) accuracy, a level that compares favorably with another predictive model study by Chu et al [
While this preliminary study is limited to a sample size of just 20 participants, this novel approach to using cytokine response to predict POP surgical outcomes has successfully distinguished important cytokines and their correlations. Moreover, these relationships point toward prospective therapies that could promote better surgical outcomes. Supervised learning models also demonstrate a high level of accuracy, specificity, and sensitivity, even when a smaller group of cytokine data is used. This result suggests that blood cytokine analysis might be feasibly used in a clinical setting to predict POP surgical outcomes. Further study with a larger patient population will be needed to confirm the utility of this method. If successful at a larger scale, this approach has the potential to change perspectives in which surgeons would recommend and proceed with POP repair surgeries and to prevent undesired outcomes of mesh-related surgeries in patients.
acute kidney injury
granulocyte-macrophage colony-stimulating factor
interferon
interleukin
institutional review board
lipopolysaccharide
principal component
principal component analysis
pelvic organ prolapse
tumor necrosis factor
This work was supported by a Transformative Research Grant from Greenville Health System Health Sciences Center to MM, TW, RH, and RG.
Data are openly available in the GitHub public repository [
None declared.