Prediction of Visual Outcomes After Diabetic Vitrectomy Using Clinical Factors From Common Data Warehouse

Abstract

Purpose: We sought to analyze the visual outcome and systemic prognostic factors for diabetic vitrectomy and predicted outcomes using these factors.

Methods: This was a multicenter electronic medical records (EMRs) review study of 1504 eyes with type 2 diabetes that underwent vitrectomy for proliferative diabetic retinopathy at 6 university hospitals. Demographics, laboratory results, intra-operative findings, and visual acuity (VA) values were analyzed and correlated with visual outcomes at 1 year after the vitrectomy. Prediction models for visual outcomes were obtained using machine learning.

Results: At 1 year, VA was 1.0 logarithm of minimal angle resolution (logMAR) or greater (poor visual outcome group) in 456 eyes (30%). Baseline visual acuity, duration of diabetes treatment, tractional membrane, silicone oil tamponade, smoking, and vitreous hemorrhage correlated with logMAR VA at 1 year (r = 0.450, −0.159, 0.221, 0.280, 0.067, and −0.105; all P ≤ 0.036). An ensemble decision tree model trained using all variables generated accuracy, specificity, F1 score (the harmonic means of which precision and sensitivity), and receiver-operating characteristic curve area under curve values of 0.77, 0.66, 0.85, and 0.84 for the prediction of poor visual outcomes at 1 year after vitrectomy.

Conclusions: Visual outcome after diabetic vitrectomy is associated with pre- and intra-operative findings and systemic factors. Poor visual outcome after diabetic vitrectomy was predictable using clinical factors. Intensive care in patients who are predicted to result in poor vision may limit vision loss resulting from type 2 diabetes.

Translational Relevance: This study demonstrates that a real world EMR big data could predict outcome after diabetic vitrectomy using clinical factors.

Introduction

Diabetic retinopathy (DR) is the leading cause of blindness among adults aged 20 to 74 years.^1,2 As DR progresses, the development of new vessels leads to vitreous hemorrhage and tractional membrane formation, which can result in a devastating deterioration of vision in patients with diabetes.^1,3 To minimize vision loss, surgical intervention is often required. To date, vitrectomy has been the mainstay surgical treatment for blinding complications of advanced DR, including vitreous hemorrhage and tractional retinal detachment.⁴ 

Factors associated with visual outcome after diabetic vitrectomy have previously been analyzed.^5,6 Systemic conditions, such as the duration of diabetes, comorbid hypertension, and coronary vascular disease, as well as pre-operative ocular findings, including vision in the operated and fellow eyes, macular detachment, and long-acting intraocular tamponade, are known to be prognostic factors.^7,8 However, available results are limited by small sample study population numbers and various follow-up durations. 

Recent advances in machine learning and deep learning in the field of medicine have shown promising performance in the prediction of diseases based on a larger-sized database.^9–11 The application of artificial intelligence in DR has mostly focused on the diagnosis and prognosis prediction of DR stage using retinal images.^12,13 In this study, we analyzed 1-year visual outcomes in a large number of patients who underwent vitrectomy for proliferative DR (PDR) at 6 university hospitals and assessed correlated systemic prognostic factors. Using clinical factors, models for predicting visual outcome were trained and validated. 

Materials and Methods

This study was approved by the institutional review board of the Catholic University Medical Center as well as each of the following involved hospitals: Bucheon St. Mary's Hospital (in Gyeonggi-do, Korea), Incheon St. Mary's Hospital (in Incheon, Korea), Yeoeuido St. Mary's Hospital (in Seoul, Korea), Euijeongbu St. Mary's Hospital (in Gyeonggi-do, Korea), Eunpyeong St. Mary's Hospital (in Seoul, Korea), and St. Vincent's Hospital (in Gyeonggi-do, Korea). The need for written informed consent was waived due to this study's retrospective design, and the investigation was conducted in accordance with the tenets of the Declaration of Helsinki (institutional review board [IRB] number: XC20WIDI0127). 

Data Preparation

From 6 referral hospitals that share the same electronic medical records system, the medical records of patients diagnosed with type 2 diabetes mellitus (T2DM) by internists who underwent vitrectomy for DR and were followed up with for at least 1 year between January 2009 and July 2020 were obtained. The diagnosis of type T2DM was made based on a fasting plasma glucose level of at least 126 mg/dL or 2-hour post-glucose level of at least 200 mg/dL after a 75-g oral glucose tolerance test.¹⁴ Patients who underwent vitrectomy for PDR were identified by operation title and diagnosis for operation. Included diagnoses were vitreous hemorrhage, proliferative membrane, and/or tractional retinal detachment. 

Clinical data—including age at operation; duration of T2DM treatment in the referral hospital; sex, height, and weight; smoking status; systolic and diastolic blood pressure values; and the use of insulin, aspirin, and clopidogrel—were collected. Body mass index (BMI) and mean arterial pressure (MAP) were calculated. Co-existing hypertension, chronic kidney disease (CKD), cardiovascular disease, and cerebrovascular disease were assessed. From laboratory tests, serum levels of glucose at random, glycated hemoglobin (HbA1c), alanine aminotransferase (AST), aspartate aminotransferase (ALT), blood urea nitrogen (BUN), and creatinine within 1 month prior to surgery were collected. From ophthalmologic records, visual acuity (VA) values at baseline and 1, 3, 6, and 12 months after surgery; intra-operative findings (e.g. vitreous hemorrhage, tractional membrane, macular edema, and neovascular glaucoma), use of pre-, intra-, or postoperative bevacizumab; and concomitant procedures (e.g. phacoemulsification, scleral encircling, and silicone oil tamponade) were collected. 

Training and Evaluation of the Prediction Models

All collected variables were included for developing a prediction model for poor visual outcomes (i.e. VA 1.0 logarithm of minimal angle resolution [logMAR] or greater) after diabetic vitrectomy at 1 year. The data were randomly divided into training and validation (80%), and test sets (20%) using “cvpartition” function in MATLAB. Training and validation were performed using 15-fold cross validation. Prediction models were trained using support vector machine (SVM), naïve Bayes, decision tree, ensemble decision tree, and neural network approaches. Fifteen-fold cross-validation was used to validate these models. Naïve Bayes and ensemble decision tree models were obtained using the optimization process. Each trained model was tested on a test set. All experiments were performed using MATLAB 2020a (MathWorks, Inc., Natick, MA, USA). 

Statistics

Statistical analysis was performed using MATLAB 2020a. VA values were converted to logMAR values for statistical purposes. A t-test was used to compare continuous variables between groups, whereas the Mann–Whitney U test was used when normal distribution was not confirmed. The chi-squared test was used for categorical variables. Repeated measures analysis of variance (RM-ANOVA) was used to compare VA values at each time point. Pearson's correlation was used to assess the relationship between final VA and continuous clinical variables. The performance of models was evaluated using accuracy, specificity, F1 score, and area under the receiver operating characteristic (ROC) curve (AUC). The F1 score was calculated as 2 × (precision) × (sensitivity) / [(precision) + (sensitivity)]. Continuous variables are presented as mean ± standard deviation values. 

Results

Baseline Characteristics

A total of 1504 eyes from 1175 patients with a mean age of 54.5 ± 11.4 years (range = 19.2–90.1 years), 54% of whom were male patients, were included. All study participants were Korean. The mean duration of diabetes mellitus (DM) treatment was 3.4 ± 3.9 years (range = 0–18.9 years). Forty-five percent of participants had CKD, 64% had hypertension, 25% had cerebrovascular disease, and 25% had cardiovascular disease. Fifty-three percent of the included eyes were right eyes. 

Among the total study group, 456 eyes (30.3%) had final vision at 1 year of 1.0 logMAR or greater (Snellen equivalent 20/200 or less; poor vision group) and 939 eyes had final vision at 1 year of less than 1.0 logMAR (good vision group). The duration of DM treatment, presence of vitreous hemorrhage, tractional membrane, and concurrent scleral encircling and silicone oil tamponade significantly differed between the 2 groups (P < 0.001, < 0.001, < 0.001, 0.024, and < 0.001). The baseline patient characteristics are summarized in Table 1. 

Table 1.

 

Baseline Characteristics of Enrolled Subjects

 

VA Change

At 1 year after vitrectomy, vision was 20/40 or better in 586 eyes (39.0%). During the year after surgery, 1188 eyes (78.9%) experienced improved or consistent vision. The mean VA improved from 1.15 ± 0.82 logMAR to 0.85 ± 0.79 logMAR (P < 0.001, RM-ANOVA) in the operated eye. The mean VA of the fellow eye also improved—albeit to a lesser extent than that in the operated eye—from 0.72 ± 0.77 logMAR to 0.64 ± 0.72 logMAR (P < 0.001, RM-ANOVA). When divided into bilateral and unilateral cases, the VA improvement in the fellow eye was observed in the bilateral cases only (P = 0.054 in the unilateral group and P < 0.001 in the bilateral group, RM-ANOVA; Supplementary Fig. S1). Visual improvement was greatest from postoperative months 1 to 3 in both eyes (both P < 0.001; Fig. 1A). 

Figure 1.

 

VA changes during the 1-year follow-up after diabetic vitrectomy. (A) VA improved in the operated eyes and in the fellow eyes (both P < 0.001, RM-ANOVA). (B) The good visual outcome group showed significant improvement in vision (P < 0.001, RM-ANOVA), whereas the poor visual outcome group experienced deterioration in vision (P < 0.001, RM-ANOVA). P values: paired t-test with the value of the previous follow-up period. *Statistically significant P value.

When participants were divided into poor and good vision groups, the good vision group experienced a significant improvement in vision (from 0.95 ± 0.80 logMAR to 0.41 ± 0.37 logMAR; P < 0.001, RM-ANOVA), whereas the poor vision group experienced a deterioration in vision (from 1.57 ± 0.71 logMAR to 1.77 ± 0.59 logMAR; P < 0.001, RM-ANOVA). In the poor vision group, vision did not improve at any time during the follow-up period relative to at baseline (Fig. 1B). 

Risk Factor Analysis

Baseline VA positively correlated with the VA at 1 year after vitrectomy (r = 0.450 and P < 0.001) whereas the duration of T2DM treatment showed a negative correlation (r = −0.159 and P < 0.001; Table 2) on Pearson's correlation analysis. 

Table 2.

 

Correlation Between Visual Acuity at 1 Year After Vitrectomy and Clinical Variables

 

Forward conditional binary logistic regression for poor visual outcome revealed sex; diabetes treatment duration, tractional membrane; silicone oil tamponade; and baseline VA values of the operated eye fellow to be significant associated factors (B = 1.479, 1.060, 0.405, 0.403, 0.278, and 0.726; P = 0.018, = 0.008, < 0.001, < 0.001, < 0.001, and = 0.004; Table 3). 

Table 3.

 

Multivariable Binary Logistic Regression for Poor Visual Outcome

 

Prediction Models

Machine learning models for the prediction of a poor visual outcome were trained using all the variables in Tables 1 and 2. Prediction models trained using logistic regression, SVM, naïve Bayes, decision trees, ensemble decision trees, and neural networks yielded AUC values of 0.74, 0.83, 0.74, 0.75, 0.84, and 0.77, respectively, and F1 scores of 0.81, 0.85, 0.81, 0.84, 0.85, and 0.84 points, respectively, for the test set (Table 4). Predictor importance analysis of the ensemble decision tree revealed baseline VA of the study eye, age at vitrectomy, duration of DM, glucose, ALT, HbA1c, BMI, BUN, creatinine, smoking, AST, MAP, VA of the fellow eye, systolic blood pressure, tractional membrane, and silicone oil tamponade as important predictors for poor visual outcome after diabetic vitrectomy (Fig. 2). 

Table 4.

 

Performance of Machine Learning Classifiers in the Prediction of Poor Visual Outcome After Diabetic Vitrectomy

 

Figure 2.

 

Important predictors for poor visual outcome after diabetic vitrectomy. A histogram of the importance of variables obtained from an ensemble decision tree prediction model for predicting poor visual outcomes after diabetic vitrectomy.

Discussion

Vitrectomy for DR significantly improved the visual outcome, thereby enhancing the quality of life of patients with PDR. Since its introduction, several decades have passed and vitrectomy has since achieved remarkable advancements using modern operating systems. Nonetheless, some patients do not experience improvements in vision and may persist at the level of legal blindness even after surgery. Knowing risk factors for poor visual outcome and developing a prediction model for patient stratification may be of great help in reducing blindness caused by complications of T2DM. 

In this study, we analyzed systemic and intra-operative risk factors of poor visual outcome after diabetic vitrectomy, developed prediction models for visual outcomes using these factors, and assessed the performance of these prediction models. The results revealed baseline VA, duration of diabetes treatment at the referral hospital, tractional membrane, silicone oil tamponade, smoking, and vitreous hemorrhage to be relevant factors. Machine learning models trained using these factors could predict poor visual outcomes at 1 year after vitrectomy with an accuracy of up to 0.77. 

Final vision at 1 year was 20/40 or better in about 39% of the treated eyes, which is comparable to findings of other recent studies.^7,15,16 The Diabetic Retinopathy in Various Ethnic Groups (DRIVE-UK) study reported that visual outcomes were improved significantly in eyes with complications attributed to DR relative to those previously reported in the Diabetic Retinopathy Vitrectomy Study.^7,17 The proportion of eyes achieving vision of 20/40 or better improved from 11% to 20% to 38% in the last 3 decades. A large proportion of patients with end-stage DR can retain their vision with vitrectomy. Furthermore, as the tendency for the VA to stabilize by 1 year after vitrectomy performed for DR had been reported, this outcome may suggest the eventual or long-term visual outcome.¹⁸ 

Tractional membrane, silicone oil tamponade, smoking, and baseline VA were correlated with the final VA and also associated with poor vision after diabetic vitrectomy. These factors were reported also to be predictors for poor vision in previous studies.^7,19,20 Conversely, vitreous hemorrhage was reported to be a protective factor, and this was made evident again in the current study. The duration of T2DM treatment at the referral hospital was also a protective factor in this study. This may imply that the visual outcome can be enhanced with rigorous DM control for longer periods prior to the operation. Additionally, BUN and creatinine were important predictors in the machine learning model in this study. The association between kidney function and DR progression is well studied.^21–23 Regarding diabetic vitrectomy, kidney function was also reported to be a factor affecting postoperative vision and recurrent hemorrhage.^20,24 However, this result requires further validation as estimated glomerular filtration rate, an important parameter for kidney function, did not correlate with visual outcome after diabetic vitrectomy.²⁵ Other factors identified as important predictors in the machine learning model, such as liver function and smoking, should similarly be studied in depth in subsequent studies. 

There is currently no available model for the prediction of the outcome after diabetic vitrectomy. Although factors to help predict the outcome have been studied, participant numbers in previous studies were relatively inadequate for developing a proper prediction model.^19,20,26 In contrast, we were able to train and test prediction models for discerning poor visual outcomes after diabetic vitrectomy by including a large number of patients from multiple referral hospitals. The models trained using machine learning demonstrated relatively fair performance in prediction. In particular, the ensemble decision tree and SVM models showed the best performance with AUC, F1 score, and accuracy values of 0.84, 0.85, and 0.77 and 0.83, 0.85, and 0.76, respectively. Additional pre-operative imaging results, such as those from fundus photography, optical coherence tomography, or B-scan ultrasonography, may enhance the performance of prediction models in the future. 

This study has several limitations inherent to its nature of being retrospective and a medical records review study. The duration of diabetes was not included in the study because the exact necessary information could not be acquired. In addition, factors other than those evaluated in this study, such as duration of surgery, cholesterol level, and serum albumin concentration, may have affected the results.^7,27 To minimize this disadvantage, we tried to include as many available relevant factors as possible. Additionally, both 23-gauge and 25-gauge systems for vitrectomy were included in the analysis, but the effect of this is likely negligible according to Ding et al.²⁴ Difference in skill level of the surgeon and duration of surgery might have affected the visual outcome. Furthermore, medical and ophthalmologic diagnoses were assessed based on the disease code entered by clinicians and were not evaluated in detail. A more detailed evaluation of patients’ medical conditions would have been ideal. In addition, lack of generalizability needs to be considered because data were from same network of clinics. Nonetheless, the diagnostic codes were entered by experienced clinicians according to their expert judgments. 

To summarize, the visual outcome after diabetic vitrectomy was associated with pre- and intra-operative findings and systemic factors, which included baseline VA, tractional membrane, and silicone oil tamponade. Prediction models trained using these factors via machine learning could identify eyes that may demonstrate poor vision after diabetic vitrectomy. Intensive care in these patients may reduce vision loss caused by diabetes. 

Acknowledgments

The authors thank the Catholic Medical Center Research Foundation for financial support made in the program year 2022. 

Disclosure: S.-S. Lee, None; D.J. Chang, None; J.W. Kwon, None; J.W. Min, None; K. Jo, None; Y.-S. Yoo, None; B. Lyu, None; J. Baek, None