Automatic Screening and Identifying Myopic Maculopathy on Optical Coherence Tomography Images Using Deep Learning

Abstract

Purpose: The purpose of this study was to engineer deep learning (DL) models that can identify myopic maculopathy in patients with high myopia based on optical coherence tomography (OCT) images.

Methods: An artificial intelligence (AI) system was developed using 2342 qualified OCT macular images from 1041 patients with pathologic myopia admitted to the Affiliated Eye Hospital of Wenzhou Medical University (WMU). We adopted an ResNeSt101 architecture to train five independent models to identify the following five myopic maculopathies: macular choroidal thinning, macular Bruch membrane (BM) defects, subretinal hyper-reflective material (SHRM), myopic traction maculopathy (MTM), and dome-shaped macula (DSM). We tested the models with an independent test dataset that included 450 images obtained from 297 patients with high myopia. Focal loss was used to address class imbalance, and optimal operating thresholds were determined according to the Youden Index. The performance was quantified using the area under the receiver operating characteristic (AUC), sensitivity, specificity, and confusion matrix.

Results: For the identification of myopic maculopathy, the AUCs of receiver operating characteristic (ROC) curves were 0.927 to 0.974 for 5 myopic maculopathies. Our AI system achieved sensitivities equal to or even better than those of junior retinal specialists (56.16–99.73%). The diagnosis of it is also interpretable that we provide visual explanations clearly via heatmaps.

Conclusions: We developed a convolutional neural network (CNN)-based DL AI system for detection and classification of myopic maculopathy in patients with high myopia using OCT macular images. Our AI system achieved sensitivities equal to or even better than those of junior retinal specialists.

Translational Relevance: This AI system can be widely applied in sophisticated situations in large-scale high myopia screening.

Introduction

As medical imaging expands around the world at an unprecedented rate, more and more data require human expertise to interpret and classify. In many clinical specialties, there is a relative shortage of specialists to provide a timely diagnosis. For example, the extensive application of optical coherence tomography (OCT) has not been matched by the available ophthalmologist to make interpretation of the scans and refer patients to appropriate treatment centers. Artificial intelligence (AI), especially deep learning (DL), shows great potential for identifying, localizing, and even quantifying pathological features¹ in image-based ophthalmic research.² As reported by Gu et al.,³ Lin et al.,⁴ and Phene et al.,⁵ a series of applications of AI systems have achieved outstanding performances in keratopathy, glaucoma, and cataract on the basis of images. Sogawa et al.⁶ and Li et al.⁷ have demonstrated that AI has a great value in identifying high myopia-related lesions in OCT images. For example, retinoschisis and myopic choroidal neovascularization (mCNV) are highly dependent on OCT for accurately diagnosing. However, retinal disorders of pathologic myopia (PM) are still challenging to be diagnosed by OCT images, and AI has not reached its full potential for screening PM-related retinal damage. 

Myopia and high myopia are estimated to affect around 49.8% (5 billion) people and 10.0% (1 billion) of the world population by 2050, respectively.⁸ With the progression of high myopia, a proportion of patients suffered from vision-threatening retinal damage, such as posterior staphyloma and maculopathy, diagnosed as PM.⁹ According to the Meta-Analyses of Pathologic Myopia (META-PM) classification, a photographic grading system based on the fundus photographs,¹⁰ PM may be more precisely defined as diffuse choroidal atrophy and other more severe conditions or the presence of mCNV or lacquer cracks.^11,12 The limitations of the META-PM classification were obvious as the system did not include myopic traction maculopathy (MTM) and dome-shaped macula (DSM), which were common PM lesions and considered necessary components for the myopic macular classification. Given this, we adopted and modified the OCT-based classification of myopic maculopathy introduced by Fang et al.¹¹ in this study. There are five subclassifications corresponding to different morphologies of OCT images: macular choroidal thinning, macular Bruch membrane (BM) defects, subretinal hyper-reflective material (SHRM), MTM, and DSM. (1) Macular choroidal thinning is defined as choroidal thickness <62 µm at the subfovea¹¹; (2) BM defects are characterized by a lack of the outer retinal layers, choriocapillaris, and the middle-sized choroidal vessels layer in the OCT images¹³; (3) SHRM is characterized by the deposition of hyper-reflective material in the subretinal space on OCT images and is commonly observed in fibrosis, hemorrhage, choroidal neovascularization (CNV), exudate, and vitelliform material¹⁴; (4) MTM is an umbrella term to define a group of retinal disorders caused by traction force, including maculoschisis, lamellar or full-thickness macular hole, and foveal detachment¹⁵; (5) DSM is defined as an inward protrusion of the retinal pigment epithelium (RPE) with a maximal height >50 µm above a line connecting the RPE on both sides outside of the DSM.¹⁶ Of these, the first three cover all the categories of the META-PM, and the latter two compensate for the above deficiency of the system.¹¹ These five types of myopic maculopathy are essential to be identified promptly. Additionally, regular follow-up is necessary for providing dependable guidance in early treating and judging the prognosis.¹⁷ However, the uneven distribution of medical resources and relatively insufficient number of fundus doctors lead to a dilemma of screening the patients at high risk. 

In this paper, we developed an AI system to identify the five myopic maculopathy changes (macular choroidal thinning, macular BM defects, SHRM, MTM, and DSM) in patients with high myopia. After that, we test the models on an independent test dataset that was collected from clinical work. The test performance of the AI system was compared with that of human ophthalmologists in the affiliated eye hospital of Wenzhou Medical University (WMU, Wenzhou, China). 

Methods

In this study, OCT images from patients with PM were retrospectively collected in two affiliated eye hospitals of WMU (Wenzhou and Hangzhou) from 2016 to 2021. High myopia was defined as a refractive error (RE) <−8.0 diopters (D) or an axial length (AxL) ≥ 26.5 mm according to the Ministry of Health and Welfare in Japan.¹⁸ Eyes with ocular disorders other than PM were excluded, such as posterior uveitis, diabetic retinopathy, retinitis pigmentosa, congenital stationary night blindness, and retinal vein occlusion (detailed standards seen in Supplementary Material). The study was conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics committee of the Affiliated Eye Hospital of WMU. Signed informed consent documentation was obtained from all participants. 

Data Collection for Algorithm Generation

OCT was performed using the Spectralis HRA OCT (Heidelberg Engineering, Heidelberg, Germany) instrument. Horizontal and vertical slices through the fovea are important for detecting myopic maculopathy. The OCT scanning protocols included simultaneously horizontal and vertical slices through the fovea, and the scan length was 6 mm and 8 mm. Through repeated scanning, the fovea can be located by the fixation point of the fundus and the tomographic features of the fovea. OCT images of patients with multiple follow-ups were not duplicated. Both horizontal and vertical lines were included for AI development in most cases, whereas sometimes only one line was chosen in a few cases due to the poor image quality or failure of crossing the fovea. 

We initially screened 1594 eyes of 1090 patients with PM who were examined at the Affiliated Eye Hospital of WMU in Hangzhou from May 2016 to May 2020. Among these eyes, 55 eyes were excluded for other eye diseases. In the end, 2342 qualified OCT images obtained in 1041 patients were selected for AI development. These images were split into training and validation images in a 4:1 ratio. To test the AI system in a real clinical setting, 450 images obtained from 297 patients with high myopia recruited in the Affiliated Eye Hospital of WMU in Wenzhou from June 2020 to January 2021 were selected as an independent dataset according to the same criteria. The previous training and validation dataset did not use the test dataset. 

For each patient, we obtained the diagnoses from four clinical experts, two of whom were retina specialists and two attending ophthalmologists. For each OCT scan, the two retina specialists, respectively, provided five separate decisions for the five classifications according to the recognized diagnostic criteria (see Supplementary Materials). Then, a standard diagnosis was defined for each image if the two retina specialists assigned it the same disease. When there was disagreement, the results were confirmed by a discussion between the two retina specialists and another senior specialist. The diagnoses specified by the attending ophthalmologists were only used to compare the performance of the AI system. 

Deep Learning Methods Development

Our framework uses an ensemble of five classification model instances (see Fig. 1) to identify macular choroidal thinning, macular BM defects, SHRM, MTM, and DSM. Model development was done on a PyTorch platform. We used a convolutional neural network (CNN) architecture named ResNeSt101 to construct five independent binary classifiers. The output of each myopic macular lesion classifier ranged from 0 to 1, representing the probability of the existence of each classification. Transfer learning can improve the performance of AI system based on limited biomedical images. 

Figure 1.

 

Workflow of the AI system. Workflow of our AI system. Vertical or horizontal macular OCT images from a high myopia eye were independently subjected to the AI system as the input. Images were processed by five rounds of categorization. The positive diagnoses with corresponding heat maps and a highlighted output label for the case of macular choroidal thinning were given as the output. AI, artificial intelligence.

Due to the class imbalance between positive and negative classes, we adopted the Focal Loss strategy as a loss function during training for BM defects, SHRM, MTM, and DSM detection. Focal Loss is put initially forward to ease both the imbalance between positive and negative samples and the imbalance of hard and easy samples. In the training process, we adapted it to focus on a sparse set of hard examples and the classes with fewer images than the other. The two factors of Focal Loss were manually set according to the distribution of our training set. 

The raw OCT images were preprocessed before being used in the training time. First, we cropped the regions of interest from the original images and resized them to the same size. Then, the images were all normalized according to the mean and variance of the pretraining dataset. During each training epoch, the same number of new variations of original images were produced and took as the inputs of the network. This online random augmentation helped our system generalize better while the number of training images was limited and made our system more robust. The set of transformations included horizontal flipping, rotation, cropping, contrast and brightness variation, and contrast limited adaptive histogram equalization. These transformations were adopted with certain probabilities at each time. The batch size was set to 32 while training the network. The Adam optimizer was used with an initial learning rate of 0.001, and set the sizes of all the input images to 342 × 342. We optimized the hyperparameters on the validation dataset and selected the models with the best area under the receiver operating characteristics (AUCs) in each category as the final models. 

ResNet101 has been proven to perform better than most mainstream CNN architectures in the ImageNet classification challenge. ResNeSt is a novel variant of ResNet with split attention networks, which enables attention across different feature maps. Specifically, a squeeze-and-excitation operation was performed on each cardinal group, and the implementation of the overall network architecture was optimized to be computationally efficient. 

Gradient-weighted Class Activation Mapping (Grad-CAM) was used to visualize the activation regions on a single input image. This approach is achieved by computing the gradients and use them as the weights of feature maps. 

Evaluation of the AI System

The overall process of our AI system is shown in Figure 1. Each input image was first preprocessed and then taken into five independent models. The system provided the interpretation results like a real human expert does. The performance of the AI system was evaluated by sensitivity, specificity, and AUCs. Heatmaps were also shown to help us intuitively visualize how the AI system worked. This was done by gradient-based localization on the feature maps produced in the process of network inference. 

Statistical Analysis

Receiver operating characteristic (ROC) curves were analyzed and plotted with the Python packages of matplotlib 2.2.3 and scikit-learn 0.19.2. The AUCs of ROC curves, sensitivity, and specificity were used to assess the performance of the AI models, and the 95% confidence intervals (CIs) represented the Wilson Score intervals for sensitivity, specificity, and Delong intervals for the AUCs, which were calculated with the R packages of Hmisc_4.2–0 and pRoc_1.15.3. 

Results

Demographics and Baseline Characteristics of the Datasets

The 2342 images obtained from 1041 patients (1539 eyes) for AI system development consisted of 1903 (81.25%) images showing macular choroidal thinning, 437 (18.66 %) images showing macular BM defects, 154 (6.58%) images showing SHRM, 481 (20.54 %) images showing MTM, and 121 (5.17%) images showing DSM. This dataset was divided into training datasets (80%) for each model training and validation datasets (20%) for parameter adjustment in each model. In the independent test dataset, 450 images were obtained from 395 examinations in 390 eyes of 297 patients. Table 1 and Table 2 reveal the details of the labels in the training, validation and test datasets, and other basic information. 

Table 1.

 

Demographics and Baseline Characteristics of the Datasets

 

Table 2.

 

Number of Images Included in the Training, Validation, and Test Sets

 

The Performance of the AI Models

We developed our architecture in the challenging context of OCT imaging for ophthalmology. Table 3 reveals that our models trained with 2342 available training images achieved an AUC of 92.7% to 97.4% for all myopic maculopathy detection. The AUCs of ROC curves were 0.927 for macular choroidal thinning, 0.938 for BM defect, 0.927 for SHRM, 0.974 for MTM, and 0.955 for DSM. Focal Loss was used as the final loss function due to a massive imbalance between the positive and negative samples in each category. Some examples that were extremely hard for the models to learn also gained more attention by adapting the weighting factor of Focal Loss during the training (index for Focal Loss was seen in Supplementary Materials Table S3). The hyperparameters of models were optimized according to their performance on the validation dataset. A huge number of experiments were done during the process, and we only selected the models with the best performance on the mentioned dataset. Table 4 reveals that ResNeSt101 architecture (applying split attention networks) outperformed ResNet101 (without split attention networks) in accuracy on image classification. The results show that our system had good generalization on the test data set, which was completely independent of the training and test set, and achieved good sensitivity and specificity, which was comparable to that of junior retinal specialized ophthalmologists. 

Table 3.

 

The Performance of AI Models

 

Table 4.

 

Compare the Performance of AI Models With and Without Split Attention Networks

 

Comparison of the AI Models With Human Retinal Specialists

We defined a gold standard to evaluate our framework to evaluate the performance of the AI system. The final diagnosis was determined by examining the patient's clinical record based on subsequently acquired information, using unavailable information at the first patient visit and OCT scan. For each patient, we obtained the diagnoses from four clinical experts, two of whom were senior retina specialists (2 attending ophthalmologists) and two junior retina specialists (1 fellow and 1 resident). We compared each of their performances against the gold standard. The performance of the AI system was then compared with the four reviewers using ROC curve plots (see Fig. 2). The performance of the AI system is further evaluated through the ROC curves plotted in Figure 2 for each myopic maculopathy condition, respectively (see Fig. 2). Our AI system achieved sensitivities equal to or even better than junior retinal specialists (56.16–99.73%) in BM defect, MTM, and DSM detecting (see Fig. 3 and Fig. 4). When detecting BM defects, SHRM, MTM, and macular choroidal thinning in cases with comorbidities, the AI system did better than retinal specialists (see Supplementary Fig. S2). 

Figure 2.

 

ROC curves of the AI system. (A) The performance of the AI system and ophthalmologists for macular choroidal thinning. (B) The performance of the AI system and ophthalmologists for macular Bruch membrane (BM) defects. (C) The performance of the AI system and ophthalmologists for subretinal hyperreflective material (SHRM). (D) The performance of the AI system and ophthalmologists for myopic traction maculopathy (MTM). (E) The performance of the AI system and ophthalmologists for dome-shaped macula (DSM). AI, artificial intelligence; AUC, area under the curve; ROC, receiver operating characteristic.

Figure 3.

 

Comparison of the AI models with human retinal specialists. (A) Macular choroidal thinning: AI system (sensitivity = 90.50% and specificity = 88.70%), junior specialists (94.27% and 89.32%), and senior specialists (99.73% and 92.65%). (B) Bruch membrane defect: AI system (88.90% and 84.80%), junior specialists (68.84% and 95.23%), and senior specialists (98.28% and 95.23%). (C) Subretinal hyperreflective material: AI system (73.90% and 91.30%), junior specialists (88.52% and 93.48%), and senior specialists (96.62% and 97.34%). (D) Myopic traction maculopathy: AI system (92.80% and 90.50%), junior specialists (89.37% and 90.98%), and senior specialists (97.71% and 96.93%). (E) Dome-shaped macula: AI system (74.50% and 94.00%), junior specialists (73.29% and 95.25%), and senior specialists (94.79% and 99.25%). Junior specialists: 3 years of ophthalmic clinical experience; and senior specialists: 9 years of ophthalmic clinical experience.

Figure 4.

 

Confusion matrix of (A) macular choroidal thinning, (B) macular Bruch membrane (BM) defects, (C) subretinal hyper-reflective material (SHRM), (D) myopic traction maculopathy (MTM), and (E) dome-shaped macula (DSM).

Heatmaps

The AI system provided activation maps that demonstrate that our system made the diagnosis based on the correct lesion for diagnosis (seen in Fig. 5). Regions highlighted with warmer colors represented those areas more critical for the final class determination. The myopic maculopathy region of interest (ROI) was captured precisely, and results were compatible with the judgment of the retinal specialist. As for the output of choroidal thinning, according to the output labels of the algorithm (i.e. choroidal thinning/no choroidal thinning), once the choroidal thinning is detected, a highlighted sentence will appear at the bottom of the OCT image. Choroid with a normal thickness or indetectable thinness will not be recognized (seen in Fig. 6). 

Figure 5.

 

Heatmaps for macular BM defects, myopic macular retinoschisis, and dome-shape macular. (A) An example of a macular BM defects lesion detected by our AI system. (B) An example of an SHRM lesion detected by our AI system. (C) An example of MTM lesion detected by our AI system. (D) An example of a DSM lesion detected by our AI system. AI, artificial intelligence; BM, Bruch membrane; SHRM, subretinal hyperreflective material; MTM, myopic traction maculopathy; DSM, dome-shape macula.

Figure 6.

 

Typical case in which AI correctly identified choroidal thinning. Subfoveal choroidal thickness was measured less than 62 µm manually.

Discussion

AI systems based on CNNs have been widely used in the field of medical image diagnoses, such as diabetic retinopathy,^19–21 age-related macular degeneration (AMD),^22,23 and glaucoma.²⁴ In this study, we implemented a CNN-based DL systems model that outperformed junior retinal specialty ophthalmologists in myopic maculopathy recognition. The heatmaps’ output from our AI system can accurately show the location and characteristics of the lesions, which is helpful with targeted diagnosis and may also reduce the burden of ophthalmologists by providing them with a reference. Our application cannot only help speed up the process and reduce the cost of myopic maculopathy diagnosis but also promote long-term follow-up of patients with high myopia, which may reduce avoidable vision loss. It is particularly useful in areas where a retinal specialist is not available for a variety of reasons, such as economic problems or allocation of medical resources. Further and timely referral to a retinal specialist can be assigned to those abnormalities that have been detected by the AI system. 

In highly myopic eyes, progressive thinning of the choroid contributes to the progression from the normal fundus to diffuse atrophy.¹¹ Macular diffuse choroidal atrophy (MDCA) is significantly associated with increased severity of high myopia and the enlargement of chorioretinal atrophy.¹¹ Various manual methods have been used to achieve accurate quantitative measurement, but the work is time-consuming and cannot be used to promote screening. BM defects are considered as hallmarks of myopic CNV-related macular atrophy and patchy atrophy.¹³ As photoreceptors are absent, BM defects represent an absolute scotoma psychophysically and indicate a significantly lower visual acuity.¹¹ Thus, early detection of BM defect lesions is essential and regular follow-up should be carried out. Myopic CNV is a typical SHRM-associated disease, myopic CNV usually leads to severe vision impairment and needs timely antivascular endothelial growth factor (VEGF) treatment.¹⁴ Prompt treatment is not only for the short-term improvement of visual acuity, but it also contributes to a decrease in the prevalence of BM defects.¹³ As OCT becomes routine, MTM could be recognized clearly in daily clinical practice. Several studies have revealed that the natural course of maculoschisis can be stable generally, and patients have no subjective symptoms.^25,26 Whereas maculoschisis has a risk for progression to foveal detachment and full-thickness of the macular hole, leading to substantial visual impairment.^26,27 For these patients, timely surgical intervention is crucial for the improvement of visual function.^25,26 DSM occurs in 18.5% of the patients with high myopia²⁸ and correlates positively with the severity of myopic maculopathy.²⁹ Although BCVA seems to remain stable over time, DSM increases the likelihood of developing multiple complications, including subretinal fluid, myopic CNV, retinoschisis, etc.^30,31 When the bulge height of DSM is more than 400 µm, the occurrence of subfoveal serous detachment will be more common.³¹ Thus, detecting DSM is also vital for avoiding vision-threatening conditions. 

Our AI system was used to identify choroidal atrophy, and the sensitivity was close to the human ophthalmologists. The derived models may play an important role in the diagnosis of MDCA in the future. Moreover, the AI system showed better sensitivities for BM defect and MTM than junior retinal specialized ophthalmologists (see Fig. 2). Our AI system achieved an AUC of 0.927 for SHRM detecting, and around half of the patients with SHRM were diagnosed as mCNV by examining the patient's clinical record, whereas others are identified as macular atrophy or hemorrhage. Therefore, our AI system is helpful in the referral of patients with SHRM for further diagnosis and treatment. Our AI system achieved an AUC of 0.974 for MTM detecting and outperformed human experts in some cases, which may be a promising tool in the follow-up of patients with MTM. The heatmaps showed that our AI model could accurately identify the features of myopic maculopathy lesions. There were several cases in which the AI system had better performance than human experts (see Supplementary Fig. S2). 

In this study, the proposed ResNeSt101 architecture achieved better accuracy than ResNet101 architecture. ResNeSt architecture applies channel-wise attention on different network branches to leverage their success in capturing cross-feature interactions and learning diverse representations.³² The Focal Loss was reported to have been designed to prevent the vast number of easy negatives from overwhelming the detector during training in which an extreme imbalance exists between foreground and background classes.³³ The application of the Focal Loss in our study also provided an effective solution for cases in which imbalanced data were processed during medical AI training, especially for the macular choroidal thinning dataset. 

However, our study still has some limitations. There are still a few images that the AI system cannot recognize well. The following may contribute to miscalculation: poor image quality, microlesions, and lesions in the forms that are less frequently appeared in the training set (see Supplementary Fig. S3). Histogram equalization and denoising were adopted to optimize the models. First, our model was trained to focus on myopic maculopathy but not to recognize other retinal diseases with similar symptoms. Second, the image quality also affected the accuracy of diagnosis. Due to the opacity of the refractive medium, the quality of the OCT image of high myopia sometimes was low. For example, the detection of SHRM seems to be less effective. The misdiagnosed images and corresponding heat maps were carefully reviewed to determine possible reasons for misdiagnosis. The potential factors of false-negative diagnosis include (1) low image quality; (2) micro-lesions, and (3) lesions under the RPE layer. For example, false-negative diagnosis may occur when the image quality is relatively low (e.g. Supplementary Fig. S4A), when atypical micro-lesions are encountered (e.g. see Supplementary Fig. S4B), or when the lesions are located under the RPE layer (e.g. see Supplementary Fig. S4C). Image quality was the main contributor to false-positive diagnoses (see Supplementary Fig. S4D). In addition, this study was conducted in two medical centers, and multicenter research is the next step to improve the reliability of our AI systems in the future. 

Conclusion

We showed that various myopic maculopathy lesions (macular choroidal thinning, macular BM defects, SHRM, MTM, and DSM) could be screened and identified using our models developed with DL based on OCT images. While assisted by our AI system, as it was developed with high sensitivities and specificities, the workload of human experts can be greatly reduced in the large-scale high myopia screening. 

Acknowledgments

The authors thank the Wenzhou Scientific Research Project (Y20190627). 

Ethics approval and consent to participate: The study was conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics committee of the Affiliated Eye Hospital of Wenzhou Medical University. Informed consent to participate in the study was obtained from all of the participants. 

Informed consent for publication: Informed consent to publish was obtained from all of participants before their inclusion in the study. 

Availability of data and materials: The datasets used and analyzed during the current study are available from the corresponding author on reasonable request. 

Authors Contributions: Lijun Shen had full access to all the data in the study and will take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Xin Ye and Jun Wang. Acquisition, analysis, or interpretation of data: Xin Ye, Jun Wang, Yiqi Chen, Zhe Lv, Shucheng He, Jianbo Mao, and Jiahao Xu. Drafting of the manuscript: Xin Ye and Jun Wang. Critical revision of the manuscript for important intellectual content: Xin Ye. Study supervision: Lijun Shen. 

Disclosure: X. Ye, None; J. Wang, None; Y. Chen, None; Z. Lv, None; S. He, None; J. Mao, None; J. Xu, None; L. Shen, None