Reducing Ophthalmic Health Disparities Through Transfer Learning: A Novel Application to Overcome Data Inequality

Abstract

Purpose: Race disparities in the healthcare system and the resulting inequality in clinical data among different races hinder the ability to generate equitable prediction results. This study aims to reduce healthcare disparities arising from data imbalance by leveraging advanced transfer learning (TL) methods.

Method: We examined the ophthalmic healthcare disparities at a population level using electronic medical records data from a study cohort (N = 785) receiving care at an academic institute. Regression-based TL models were usesd, transferring valuable information from the dominant racial group (White) to improve visual field mean deviation (MD) rate of change prediction particularly for data-disadvantaged African American (AA) and Asian racial groups. Prediction results of TL models were compared with two conventional approaches.

Results: Disparities in socioeconomic status and baseline disease severity were observed among the AA and Asian racial groups. The TL approach achieved marked to comparable improvement in prediction accuracy compared to the two conventional approaches as evident by smaller mean absolute errors or mean square errors. TL identified distinct key features of visual field MD rate of change for each racial group.

Conclusions: The study introduces a novel application of TL that improved reliability of the analysis in comparison with conventional methods, especially in small sample size groups. This can improve assessment of healthcare disparity and subsequent remedy approach.

Translational Relevance: TL offers an equitable and efficient approach to mitigate healthcare disparities analysis by enhancing prediction performance for data-disadvantaged group.

Introduction

Healthcare disparities prevail across various medical fields and clinical care systems, including ophthalmology and vision health. These disparities are observed based on demographic factors such as race/ethnicity, age, and gender, as well as socioeconomic factors including family income and education.^1–5 In ophthalmology, racial/ethnic minority groups face a higher risk of vision loss because of eye diseases such as glaucoma, cataracts, and diabetic retinopathy.⁶ These disparities can be attributed to challenges in accessing healthcare services, undersourced medical facilities in low-income neighborhoods, language barriers, and more.⁵ Adherence to treatment is critical for slowing vision loss in various ophthalmic diseases, but socioeconomically disadvantaged individuals may encounter difficulties in prioritizing treatment plans, such as clinic visits or purchasing eye drops.^7,8 Furthermore, despite the rapid growth of telemedicine as a viable care delivery method, especially during the COVID-19 pandemic, disparities persist in the assessment and utilization of ophthalmic telehealthcare, with minority groups being significantly less likely to complete video telehealth encounters.⁹ Therefore understanding causes of visual healthcare disparities through population-level ophthalmic studies is crucial.^10,11 

Artificial intelligence (AI) has shown great potential in aiding physicians and clinicians with eye disease diagnosis, prognosis, and management. AI-based predictive analysts leveraging large datasets, including the electronic medical record (EMR) databases, eye imaging databases, and genomic data consortia, have been developed for prediction/classification tasks related to clinical decision-making and biomedical research.^12,13 However, studies have revealed that data-driven AI methods can generate results that favor the dominant, data-advantaged groups, with lower prediction performance for the data-disadvantaged groups. Racial minority groups are often under-represented in the EMR database because of long-standing healthcare disparities in the clinical care system. When AI methods are trained on such imbalanced datasets and applied in clinical care, they can inadvertently perpetuate additional healthcare disparities among the data-disadvantaged racial groups.¹⁴ Several methods have been developed to address the challenge of data imbalance, such as the low-shot deep learning (DL) technique, which enables the use of smaller training datasets compared to traditional DL. Generative DL can produce synthetic data to augment the training samples from under-represented groups.¹⁵ These two methods have been applied to rare retinal diseases and age-related macular degeneration with image data.^16,17 However, it is widely recognized that DL models exhibit limited transparency and flexibility, which can cause some challenges in the context of clinical decision-making.¹⁸ 

Transfer learning has been used recently to overcome data inequality and distribution discrepancy between different datasets or groups (termed domains hereafter) in prediction tasks.^14,19–26 Transfer learning involves transferring knowledge from pretrained models trained on source domains to the target domain of interest, which often comprises a smaller dataset. This approach significantly reduces the data requirement to train new models and is increasingly used in healthcare disparity studies^14,26 and medical image analysis, such as radiography, computed tomography, magnetic resonance tomography, positron emission tomography, and ultrasonography.^19,21–25 Although various transfer learning approaches are used in the deep learning model scheme, concerns about the “black-box” issue and interpretability of these models in clinical research and application remain.^27,28 This motivates us to explore regression-based transfer learning, which enhances the prediction accuracy by borrowing information from other source data and enables the study of associations between parameters of interest and clinical outcomes. 

In this study, we examine ophthalmic healthcare disparities in glaucoma progression at the population level by leveraging EMR data. The transfer learning method is applied to a study cohort of 785 individuals who received care at the Department of Ophthalmology in NYU Langone Health from 2018 to 2021. We apply the regression-based transfer learning method proposed by Tian and Feng²⁹ to this dataset, aiming to reduce healthcare disparities by improving the prediction performance of disease progression, defined as the rate of change per year in visual field (VF) mean deviation (MD), particularly among data-disadvantaged African American and Asian groups. We evaluate the prediction performance of transfer learning approach compared to linear regression models applied to the specific racial groups only and linear regression models trained on pooled racial data. 

Methods

Study Design and EMR Data Collection

The study involved a retrospective chart review of all healthy individuals and subjects with glaucoma, aged 18 years or older, who received care at the Department of Ophthalmology in NYU Langone Health, between January 2018 and December 2021. To ensure compliance with Health Insurance Portability and Accountability Act regulations, all information was deidentified before data analysis. The study obtained prior approval from the Institutional Review Board of NYU School of Medicine. 

The study cohort consisted of individuals who underwent comprehensive ophthalmic examinations and had a minimum of three visual field tests. Exclusion criteria included individuals with a history of ocular trauma or intraocular surgery, other than uncomplicated glaucoma, within the six months before the start date of study. 

Demographic information including self-reported race, socioeconomic factors, glaucoma diagnosis, and optical coherence tomography and visual field data were extracted from the electronic medical records for eligible subjects (N = 1417). After excluding subjects with missing data, including subjects with race reported as Other, a total of 785 subjects were qualified for analysis (Fig. 1). This included 484 (61.7%) White individuals, 219 (27.9%) African Americans, and 82 (10.4%) Asian. 

Figure 1.

 

Flow chart of study-eligible population and final analysis population.

Data Preprocessing and Features Included in the Predictive Models

The primary outcome in the prediction task was to predict the rate of change per year in visual field MD because it is a continuous variable that is commonly used to asses visual function. To estimate the rate of change of MD, a linear regression model was employed, fitting MD measurements obtained for each individual against the visual field testing time from baseline. The area deprivation index³⁰ (ADI) was used to capture the socioeconomic context of each study subject based on the five-digit zip code of their home address. ADI quantifies and ranks the socioeconomic disadvantage based on the level of access to public resources, such as educational institutions, health systems, not-for-profit organizations, and government agencies. A lower ADI indicates that the individual resides in a neighborhood with easier access to public resources. The predicted models included the following baseline features: demographic and socioeconomic variables (age, gender, glaucoma diagnosis, ADI), baseline VF MD, and optical coherence tomography measurements of retinal nerve fiber layer (RNFL) thickness, ganglion cell inner plexiform layer (GCIPL) thickness, rim area, disc area, cup-to-disc ratio (CDR), vertical CDR (VCDR), and cup volume. 

Transfer Learning for Visual Field MD Rate of Change Prediction

Previous studies have highlighted the advantages of using transfer learning in scenarios when there is data inequality and distribution discrepancy among different study groups.¹⁴ The general framework of transfer learning involves leveraging and transferring information between one or multiple source domains and a target domain, as depicted in Figure 2A. The source domains correspond to datasets that share similarities with the target population of interest, whereas the target domain represents the dataset collected from the specific population under study. These source domains are used to train a machine learning model, and the parameters are adjusted to improve the performance in the target domain through a process known as debiasing. 

Figure 2.

 

(A) The general framework of transfer learning. (B) The algorithm design for transfer learning models.

In this study, we used a two-step transfer learning algorithm on high-dimensional generalized linear models proposed by Tian and Feng²⁹ to enhance the accuracy of MD rate of change prediction. This approach relaxes the assumption that the underlying distribution of features from different domains are the same and is capable to identify important features associated with the study outcome. Moreover, the method enables transferring learning from multiple sources and assessing transferability of each source to prevent negative transfer. The first step involves transferring the information from source domain(s) by fitting a generalized linear model with l₁-penalty on the combined dataset, which includes both the source domain(s) and a portion of the target domain. Initial estimates of model parameters are obtained from Step 1. It should be noted that local access to both source domain(s) and target domain data is necessary for the development of the model in the first step. However, the estimator is generally biased because of differences in data distribution between groups or datasets. The second step involves correcting the bias by fitting the contrast solely on the target domain, using another l₁-regularization penalty. This step helps mitigate the effects of data distribution discrepancies and refine the estimators, thereby enhancing the prediction accuracy. 

In this study, the dominant race group, namely the White group with 484 subjects, serves as the source domain. Two target domains were considered separately (i.e., the African American [AA] group and Asian group). To assess the model performance, each target domain was divided into a 70% training set and a 30% testing set. Taking the AA group as an example, the source domain dataset from the White group and the 70% training set from the AA group were combined to obtain the transfer learning parameter estimates, denoted as \({\hat{\beta }_{TL}}\), which were subsequently used for the MD rate of change prediction on the 30% testing set from the AA group (Fig. 2B). 

Conventional Methods

The performance of the proposed transfer learning method was compared with two conventional approaches. The first approach involved fitting a linear regression model solely on the data from the specific racial group. The model incorporated the same covariates as the transfer learning models, and the results model parameters were denoted as \({\hat{\beta }_T}\) shown in Figure 2B. The second approach involved pooling data from the White, AA, and Asian racial groups to create a larger dataset, and train a linear regression model using the pooled dataset, using the same features as transfer learning models along with race as an additional predictor. This approach assumed an additive effect of race on the primary outcome, while assuming all other features have consistent effect sizes across different race groups. The resulted model parameters were denoted as \({\hat{\beta }_P}\) (not shown in Fig. 2B). 

Statistical Analysis

Continuous variables were summarized as mean ± standard deviation or as median with interquartile range. Categorical variables were presented as numbers and frequencies. Multiple group comparisons were performed using one-way analysis of variance (or Kruskal-Wallis test for non-normally distributed variables), followed by Tukey's post-hoc test to assess the difference in group means (medians). The frequencies of categorical variables between groups were compared using Pearson's χ² test. 

To evaluate the prediction performance of the assessed prediction models, mean absolute errors (MAE) and mean square errors (MSE) of the training and testing set were used to evaluate the accuracy of each method. A lower MAE or MSE value signifies better model accuracy. We further quantify the prediction accuracy improvement/loss using the formula \(( {1 - \frac{{{\rm{accuracy\ of\ proposed\ method}}}}{{{\rm{accuracy\ of\ competing\ method}}}}} ) \times 100\%\). Regression coefficients, along with their corresponding 95% confidence intervals, were obtained from the predicted models. The null hypothesis of no association between the covariate of interest and the primary outcome was examined using the Wald test. P < 0.05 was considered statistically significant. All statistical analyses were conducted using R version 4.2.2 (R Core Team, 2022),³¹ and the glmtrans³² (v2.0.0) package was used for the analyses. 

Results

Demographic and Socioeconomic Characteristics

The baseline characteristics of the study cohort and stratified groups by race are summarized in Table 1. Subjects from the White group were significantly older (65.29 ± 13.10 years) than the AA (60.19 ± 12.57 years) and Asian (57.01 ± 12.79 years) groups. Evaluating the age distribution, it was apparent that the Asian group included a subgroup of subjects around 25 years of age, which was less prominent in the other groups (Fig. 3A). AA subjects had significantly higher ADI (3.61 ± 1.94) than other groups with bimodal distribution with peaks around 5 and 2 (Fig. 3B), whereas the ADI distribution for the White group exhibits a skewed distribution toward 1. The Asian group also had more subjects with lower ADI, but to a lesser extent than the White group. No significant difference was detected in the number of VF tests and follow-up duration per subject among the three groups. The AA group had significantly lower visual field MD (−3.40 [−8.43, −1.13]) than the White group whereas other comparisons were not significant (Fig. 3C). These findings indicate that the AA subjects had the lowest socioeconomic status and the worst disease condition at baseline. The Asian group had the highest baseline RNFL thickness (81.96 ± 15.64 µm), whereas the other two racial groups had comparable thickness. There were no significant differences among the groups in ganglion cell inner plexiform layer thickness and rim area. Disc area, CDR, VCDR, and cup volume showed significant difference among all groups, with the AA subjects having the highest average values for these parameters. The differences among the racial groups summarized above emphasize the need for careful modeling when considering these racial groups. 

Table 1.

 

Baseline Characteristics of the Study Population

 

Figure 3.

 

Shown are the Violin plots demonstrating the distribution of baseline age, area deprivation index, and MD for each racial group. Tukey's post-hoc pairwise comparisons were performed to compare each pair of racial groups. NS, P > 0.05; **P ≤ 0.01; ***P ≤ 0.001.

Prediction Performance of the Transfer Learning Model

Linear models trained solely on individual racial group exhibited the highest MAE and MSE values, indicating poorest prediction performance in both the training set and testing set for both the AA group and the Asian group (Table 2). In contrast, the MAE and MSE of the transfer learning predictive model in the training set lie between that of the linear model trained solely on the minority group and the pooled linear regression that used the data of three racial groups. 

Table 2.

 

The Training and Testing MAE and MSE of all Models

 

For the MD rate of change of the AA group in the testing set, the transfer learning model achieved substantially smaller MAE and MSE values (4% and 1% improvement, respectively) compared to the linear model. However, there was a slight accuracy loss of 0.6% (measured by MAE) and 4% (measured by MSE) observed when compared to the pooled linear model. Notably, the transfer learning approach showed the most substantial benefits in predicting MD rate of change in the Asian group, with the smallest MAE and MSE among the different methods. Specifically, the MAE and MSE was reduced by 20% and 35%, respectively, compared to linear model and 4% and 10%, respectively, compared to pooled linear model. Similar results were also noted when TL was compared to pooled linear regression model using AA and Asian data only. 

Varied Key Features Identified With Assessed Models

We further investigated the association patterns (or termed key features) detected with three assessed machine learning models (Table 3). The results of the pooled linear model reveal that baseline MD (P < 0.01), and rim area (P = 0.01) exhibit significant associations with rate of change of MD for the full study cohort, indicating their highest feature importance among the assessed features. 

Table 3.

 

The Estimated Coefficients and Their 95% CI of Each Parameter From Transfer Learning, and Linear Model of the Specific Racial Group, and Pooled Linear Model With Combined Dataset

 

When considering the association patterns for each racial group, the results of transfer learning model in the African American group indicate significant associations with Baseline MD (P = 0.008) and RNFL (P = 0.005), whereas baseline age shows marginal significance (P = 0.085). Comparatively, in the Asian group, the transfer learning model identifies baseline age (P = 0.015) and rim area (P = 0.053) as significantly and marginally significantly associated with rate of change of MD, respectively. The results of the linear model using solely the racial group data demonstrate different association patterns. Baseline MD and VCDR and baseline MD and rim area are significantly or marginally significantly associated with rate of change of MD in the AA and Asian groups, respectively. Therefore the results from three models demonstrate that transfer learning enables fine-tuning of association parameters of the linear model, allowing different association patterns between different racial groups. 

Discussion

This study proposes the application of regression-based transfer learning methods to reduce healthcare disparities research arising from imbalanced ophthalmic clinical data. Specifically, we used the dominant racial group, White, as the source domain, whereas the minority racial groups, AA and Asian, served as the target domains. We used the regression-based transfer learning to transfer knowledges from the source task to construct the target task models. To evaluate its performance, we compared it using two conventional approaches: linear models and pooled linear models. The transfer learning approach for prediction of visual field damage progression in the Asian group demonstrated a marked improvement in prediction accuracy, with a 35% and 10% reduction in MSE compared to the pooled linear model and the linear model trained solely on the Asian group, respectively. In contrast, the performance of the transfer learning for the prediction in the AA group was comparable to that of the pooled linear model. These findings suggest that the prediction accuracy of the Asian group benefits more from the transfer learning approach than the AA group. A potential contributing factor to this difference is the considerably smaller sample size of the Asian group compared to the AA group. Another plausible explanation might lie in the significant difference in baseline VF MD, and rate of change in VF MD between the AA and White groups (Table 1). The difference in feature distribution may lower the transferability of the White source task to the AA target domain, in contrast to the Asian target domain. However, quantifying the transferability of one source task to another is currently beyond the scope of this study and remains for future research. 

Our study did not find a significant association between MD rate of change and the socioeconomic factor, ADI. This lack of significance may be attributed to the low resolution of the data, because we only obtained five-digit zip codes instead of more detailed nine-digit codes. Another limitation of our data is the recruitment of subjects from a single site, which may introduce selection bias and homogenize certain demographic characteristics, such as place of residence. However, we choose this design to represent the common reality where most clinical studies are based on data obtained from a single site. To overcome these limitations, future research including higher resolution data from multiple sites and involving patient surveys to gather additional socioeconomic variables are warranted. Additionally, we aggregated longitudinal outcome data into rate of change per year in visual field MD. This approach may have limitations in capturing the full dynamics of a chronic disease like glaucoma. Therefore there is a need to adapt the current regression-based transfer learning method to longitudinal studies, enabling a more granular analysis of important features associated with prediction of chronic diseases. 

Another intriguing avenue for future research involves relaxing the requirement of local access to all source domain(s) and target domain, similar to federated learning^33,34 and some other transfer learning approaches that accommodate pretrained models from source domains.^35,36 This collaborative approach could facilitate knowledge transfer from multiple sources without the need to exchange training data. 

In conclusion, transfer learning is a valuable strategy for developing efficient and equitable prediction models, especially when dealing with limited sample sizes. Although transfer learning has been applied to small medical datasets for some time, it is important to note that many of these approaches rely on deep learning models with limited interpretability. In contrast, our novel application in ophthalmology leverages regression-based transfer learning, which overcomes the “black-box” concern in the deep learning scheme. Moreover, the application of regression-based transfer learning allows the identification of specific features that contribute to improving healthcare racial equity. Specifically, our results demonstrated that baseline MD and RNFL thickness were significantly associated with MD rate of change in the AA group, but not in the Asian group. In contrast, baseline age was significant, and baseline rim area was marginally significant for the Asian group, but not for the AA group. By using this approach, we can gain deeper insights into the unique factors influencing different racial groups' healthcare outcomes, thus promoting more equitable healthcare practices. 

Acknowledgments

Supported by NIH R01-EY013178, U54-MD000538, unrestricted grant from Research to Prevent Blindness. 

Disclosure: T. Lee, None; G. Wollstein, None; C.T. Madu, None; A. Wronka, None; L. Zheng, None; R. Zambrano, None; J.S. Schuman, Zeiss (F); J. Hu None