Application of Deep Learning for Automated Detection of Polypoidal Choroidal Vasculopathy in Spectral Domain Optical Coherence Tomography

Abstract

Objective: To develop an automated polypoidal choroidal vasculopathy (PCV) screening model to distinguish PCV from wet age-related macular degeneration (wet AMD).

Methods: A retrospective review of spectral domain optical coherence tomography (SD-OCT) images was undertaken. The included SD-OCT images were classified into two distinct categories (PCV or wet AMD) prior to the development of the PCV screening model. The automated detection of PCV using the developed model was compared with the results of gold-standard fundus fluorescein angiography and indocyanine green (FFA + ICG) angiography. A framework of SHapley Additive exPlanations was used to interpret the results from the model.

Results: A total of 2334 SD-OCT images were enrolled for training purposes, and an additional 1171 SD-OCT images were used for external validation. The ResNet attention model yielded superior performance with average area under the curve values of 0.8 and 0.81 for the training and external validation data sets, respectively. The sensitivity/specificity calculated at a patient level was 100%/60% and 85%/71% for the training and external validation data sets, respectively.

Conclusions: A conventional FFA + ICG investigation to differentiate PCV from wet AMD requires intense health care resources and adversely affects patients. A deep learning algorithm is proposed to automatically distinguish PCV from wet AMD. The developed algorithm exhibited promising performance for further development into an alternative PCV screening tool. Enhancement of the model's performance with additional data is needed prior to implementation of this diagnostic tool in real-world clinical practice. The invisibility of disease signs within SD-OCT images is the main limitation of the proposed model.

Translational Relevance: Basic research of deep learning algorithms was applied to differentiate PCV from wet AMD based on OCT images, benefiting a diagnosis process and minimizing a risk of ICG angiogram.

Introduction

Age-related macular degeneration (AMD) is a leading cause of vision impairment and loss among older adults. It is classified into dry or wet AMD. Wet AMD is often taken to be synonymous with exudative AMD, although not entirely so. Currently, neovascular AMD (nvAMD) can either be exudative or nonexudative. It is characterized by hemorrhage or exudate from leakage of abnormal vessels in the macular area (serosanguinous maculopathy). Polypoidal choroidal vasculopathy (PCV) presents with clinical features similar to those found in wet AMD. It is increasingly considered a subtype of wet AMD. Both PCV and wet AMD conditions are associated with frequent relapse and require long-term follow-up treatment with intravitreous injections of anti–vascular endothelial growth factor (anti-VEGF).^1,2 A significantly higher prevalence of PCV is found in many East Asian countries compared to Caucasian populations.^3,4 

Ophthalmic imaging technologies are widely used for diagnosis of macular diseases, including spectral domain optical coherence tomography (SD-OCT) and simultaneous fundus fluorescein and indocyanine green (FFA + ICG) angiogram. SD-OCT images are commonly used to diagnose AMD. The three main findings of SD-OCT in macular diseases are pigment epithelial detachment (PED), subretinal fluid (SRF), and intraretinal fluid, which is typically referred to as cystoid macular edema (CME) in the macular area (diabetic macular edema, a common term for macular edema caused by diabetic retinopathy, is closely related to CME) (Fig. 1). All three of these features are observed on SD-OCT images in both wet AMD and PCV. 

Figure 1.

 

Characteristics of serosanguinous maculopathy observed by SD-OCT.

In clinical practice, PCV and wet AMD conditions are closely related and share common characteristics of serosanguinous maculopathy. Simultaneous FFA + ICG is considered the gold-standard investigation for diagnosis of both conditions. The presence of a hypercyanescence polypoidal lesion in the early phase of ICG is required for a diagnosis of PCV. Although the mainstay treatment for both diseases is intravitreal anti-VEGF injections, it is important to differentiate between wet AMD and PCV because laser photodynamic therapy is an additional treatment for PCV. In addition to FFA + ICG being an invasive and time-consuming investigation, patients are also at significant risk of anaphylactic shock or rental shutdown with a relatively less risk of dye-related allergy or kidney injury. In contrast, SD-OCT is noninvasive, fast, less expensive, and more practical for use in routine clinical practice. Wet AMD and PCV share very similar characteristics on SD-OCT images, but PCV also shows sharp variable notching or peak PED, double-hump PED, and branching vascular networks (BVNs). A thumb-like polyp containing hyperreflective rings with or without internal hyporeflective lumen is also observed.⁵ Clinical interpretation of SD-OCT for PCV screening could reach a sensitivity of 89% to 95% and a specificity of 85% to 93%.^6,7 

Artificial intelligence (AI) or specifically deep learning (DL) models are a promising tool for helping ophthalmologists with screening, diagnosis, and recommending proper treatment. Development and use of an automated system to distinguish wet AMD from PCV using SD-OCT images is less expensive, safer, and less time-consuming than using FFA + ICG. It would also help to compensate for the scarcity of retinal specialists in a rural area and lessen the need for referrals to secondary or tertiary care hospitals. Using such a system, cases could be properly diagnosed and treated early, and patients who need to be referred could be referred based on evidence of a strong diagnostic suspicion. 

In this study, we proposed automated detection of PCV based solely on SD-OCT segment photographs using deep learning algorithms. Several deep convolutional neural networks with advanced techniques were thoroughly investigated to enhance the model's performance. We relied on standard network architectures with a transfer learning technique and a simplified version of standard networks with an attention technique. A cross-validation technique was used when training these models in order to guarantee performance stability. Finally, a framework of SHapley Additive exPlanations (SHAP)⁸ was implemented for interpreting results from the proposed models. 

Model experiments with data application, an evaluation process, and the SHAP explainable framework distinguish our work from others. Our main contribution is applying several deep learning models with advanced techniques to differentiate PCV from wet AMD using SD-OCT images. In addition, we introduced evaluation performance at the patient level by taking multiple cross-sectional images from the same person into account. To the best of our knowledge, no previous study has trained similar deep learning algorithms using Thai population data. Finally, we adopted the SHAP framework to better understand the model's functionality. 

Related Work

In recent years, AI has been applied to various domains, including health care (medicine), and specifically in ophthalmology. Several novel algorithms have been applied to various tasks such as disease diagnosis in ophthalmology, as described in several previous studies.^9–20 Most of these algorithms were introduced to automatically detect and differentiate macular diseases, such as AMD.²¹ A large number of medical images combined with automated image analysis yielded a satisfactory result from the developed models that was near equal to the results from human evaluation. Fundus photographs and SD-OCT are among the most common imaging tests to diagnose retinal diseases. 

A fundus photograph is normally used to document and sometimes diagnose certain eye conditions, including macular diseases such as AMD and PCV. An automated prediagnosis of AMD using typical machine learning models to detect an appearance of a drusen from fundus images was proposed.²² Even though machine learning methods perform relatively well, a lot of effort is needed to sufficiently train them to identify important features. To address this, deep learning models, especially convolutional neural networks (CNNs) and their variations, have gained popularity over the past few years. A custom-designed CNN was employed to automatically and accurately diagnose AMD at an early stage.²³ Another deep convolutional neural network–based model was introduced to predict exudative AMD based on fundus images.²⁴ In addition, DeepSeeNet²⁵ and its extension²⁶ were proposed to detect individual AMD risk factors, which were further used to classify different disease severity stages. Similar works by Burlina et al.,²⁷ Grassmann et al.,²⁸ and Liu et al.²⁹ used CNN-based models to perform this task. Another pool of research work relied on a transfer learning concept that applied knowledge learned from one task to another task. The networks fully trained on a standard large data set were fine-tuned with a retina image data set, as proposed in previous studies.^30–34 

An ophthalmologist typically uses SD-OCT for a certain diagnosis by observing the macula's distinctive layers to map the abnormal characteristic and measure the central retinal thickness. In an early era of advanced AI, the traditional machine learning method, coupled with specific techniques, was implemented to detect different stages of disease based on SD-OCT images.^35–38 Other research set forth to identify and segment specific macular fluid from SD-OCT images using a CNN-based autoencoder.^39,40 In addition, a large pool of work relied on deep neural networks to classify input SD-OCT images into different abnormalities, such as AMD versus normal. Some of these works trained CNNs from a completely blank network,^41–44 while others relied on the transfer learning method.^45–50 Additional complicated techniques were considered when training deep neural networks, such as segmenting retina components^51-53 and incorporating a novel residual unit subsuming atrous separable convolution.⁵⁴ In addition, an attention technique⁵⁵ was adopted for automated retinal image localization and recognition. For example, Fang et al.⁵⁶ introduced a novel lesion-aware convolutional neural network (LACNN) using a soft attention map to identify lesion location within SD-OCT images. Mishra et al.⁵⁷ developed multilevel CNNs with dual-attention, while Wu et al.⁵⁸ focused on specific parts of an image in the model. A novel joint-attention network consisting of a supervised encoding network and an unsupervised attention network was introduced.⁵⁹ 

Although various models have been proposed to distinguish types of AMD, very few studies have focused on the detection of PCV. Most PCV-related works aimed to automate segmentation of PED or to quantify measurement of PED volume. Optical coherence tomography angiography (OCTA) and multiple image systems were used to evaluate the three-dimensional characteristics of polypoidal structures, BVNs, and PCV.⁶⁰ Another work by Xu et al.⁶¹ introduced dual-stage deep neural networks for PED segmentation in PCV. Recent work has been proposed to directly diagnose PCV disease. Yang et al.⁶² distinguish normal nvAMD from PCV using ICG angiography images. Xu et al.⁶³ used a bimodality convolutional neural network to differentiate AMD from PCV using fundus and SD-OCT images. 

Methodology

This study developed and validated an automated detection system to categorize a given SD-OCT image as either PCV or wet AMD. Multiple deep CNNs were implemented to compare with the findings of retinal specialists, which was considered the gold-standard evaluation method. The end-to-end flow process chart of our proposed system is illustrated in Figure 2. This retrospective cross-sectional study included SD-OCT imaging of patients who attended the outpatient department of the Department of Ophthalmology of the Faculty of Medicine Siriraj Hospital, Mahidol University, Bangkok, Thailand, from August 2019 to April 2021. The protocol for this study was approved by the Siriraj Institutional Review Board (approval no. 176/2563(IRB1)), and the requirement to obtain written informed consent was waived due to the retrospective nature of the study. 

Figure 2.

 

End-to-end flow process chart.

Data Collection Process and Data Preparation

A retrospective review of chart records was performed, and SD-OCT images were collected and preprocessed. Macular SD-OCT images were routinely processed via a cross-sectional scan by SPECTRALIS (Heidelberg Engineering, Heidelberg, Germany). Twenty-five horizontal raster scans covered the center of the fovea, 20 × 20 degrees horizontally and vertically. Automatic real time averaging was employed with an enhanced depth imaging feature turned off. These images were extracted and saved as JPEG files with no patient name or identification. An example of a cross-sectional scan of an SD-OCT image is shown in Figure 1. 

These cross-sectional SD-OCT images were manually classified and labeled as either PCV or wet AMD by retinal subspecialists. Several images of the SD-OCT scan that did not have abnormal lesions were excluded from the study. Poor-quality images were also eliminated prior to training of the models. All SD-OCT scans in the PCV training group were from confirmed cases of PCV by FFA + ICG. Diagnosis of PCV was based on EVEREST diagnostic criteria: the presence of focal subretinal hyperfluorescence on ICG within the first 6 minutes plus one of the following criteria. These criteria included nodular appearance of polyp(s) on stereoscopic fundus examination, hypofluorescent halo around nodule(s), presence of a branching vascular network, pulsation of polyp(s) on dynamic ICG, orange subretinal nodules on color fundus photography, or massive submacular hemorrhage (4-disc areas in size).⁶⁴ 

The first set of collected data (set 1) was used to develop CNNs whose performance was internally verified. It was then divided into three groups to create training, validation, and test data sets with approximately 80%, 10%, and 10% for each set, respectively. The second data set (set 2) was collected for external validation. The selected model with fine-tuned parameters obtained from the model development process was evaluated using the external validation set. The categorization of images into different groups is summarized in Table 1. 

Table 1.

 

Number of SD-OCT Images and Corresponding Subjects (Patient Eyes) in the Model Development and External Validation Data Sets

 

Model Development and Verification

CNNs generally yield superior performance in computer vision and image-processing tasks.⁶⁵ Several experiments among various choices of networks and techniques were performed to identify a suitable model architecture and relevant parameters. The accuracy of the developed systems was repeatedly evaluated until the desired level of performance was achieved. A cross-validation technique was employed to enhance model performance and verify model stability. 

Regarding the transfer learning technique, we retrieved the standard VGG16 and ResNet50 source models with pretrained weights based on the ImageNet data set. The pretrained models were fine-tuned for our specific task to identify PCV and wet AMD characteristics. Specifically, only the fully connected layers or the head of the networks was fine-tuned while the remaining layers were frozen. Using transfer learning, full standard models were developed using much less computational power and computation time. 

Simplified standard models were also used to fully train the networks. With smaller and simpler networks, weights in the whole model were able to be trained from scratch. In this work, a joint-attention ResNet-50 was implemented with the use of the simplified ResNet-50. We also trained the Optic-Net⁵⁴ consisting of a residual unit subsuming atrous separable convolution, as well as a novel building block based on our data, and further improved it with attention blocks. The joint-attention Optic-Net was also implemented in this work. In summary, we considered four groups of simplified models, which included ResNet,⁶⁶ ResNet with attention,⁵⁵ Optic-Net,⁵⁴ and joint-attention Optic-Net.⁵⁹ 

From our experiments, the best network was selected based on the average performance between the validation and test sets. The selected network was then further evaluated using the external validation data set. The probability threshold, defined as threshold_i to differentiate between two classes (i.e., PCV and wet AMD), was fine-tuned to yield the highest performance at an image level. Then, multiple evaluation metrics were computed based on the findings of the models on external validation. A confusion matrix was constructed, whereas sensitivity (recall), specificity, overall accuracy, and F1 were also calculated, with PCV diagnosis considered a positive class. Receiver operating characteristic curves and area under the receiver operating characteristic (AUC) curve were also generated and calculated. AUC was mainly used when comparing among different models with various parameters, whereas F1 score was employed to fine-tune the probability threshold. 

Several cross-sectional SD-OCT images of each patient's scan were generated as input data for the networks. In addition to image-based evaluation metrics, patient-based metrics were also computed by considering all cross-sectional images extracted from the same eye of the same patient. The patient-level threshold, defined as threshold_p, to classify two distinct classes was also fine-tuned. Finally, the selected model with optimized thresholds was subjected to the external validation process. 

In order to better understand how the proposed models made predictions, we adopted a unified framework named SHAP for interpreting the results. The SHAP value was inspired mainly from the well-known Shapley value in game theory to identify a feature importance value. Applying this technique to an image classification task like ours, each pixel was assigned an estimated SHAP value. Pixels with positive SHAP values represent the increasing probability of the class of interest, while negative SHAP values suggest a reduced probability. 

Results

Multiple models were extensively tested and compared to obtain the desired performance, as well as verify their generalizability. Considering all data folds, the average AUC of the validation and test sets in set 1 data was computed to enhance the reliability of the reported results. As shown in Table 2, the ResNet with attention model yielded the highest average AUC. Hence, this model was selected for external validation, the result of which showed 0.81 AUC (Fig. 3). 

Table 2.

 

AUC Performance and the Average AUC of Experimented Models on Set 1 Data

 

Figure 3.

 

The AUC from external validation of the best-performing model.

Using the selected model, we fine-tuned the probability threshold based on the test set in set 1 data to achieve the highest F1 score. After varying the threshold_i, the 0.75 threshold was found to yield the highest F1 score of 0.66. This fine-tuned probability threshold was then used to verify the model's performance during external validation (Table 3). We also evaluated the results from a patient perspective by considering all SD-OCT images generated from the same patient. The mean prediction score between predicted PCV and wet AMD was computed for each patient eye. As a result, setting threshold_p at 0.32 yielded the best performance with the highest F1 score. In other words, if the mean prediction score considering all images was greater than 0.32, we diagnosed this patient eye as having PCV. Finally, we tested the preferred model with the fine-tuned thresholds using the external validation set, as shown in Table 3. 

Table 3.

 

Diagnostic Performance of the ResNet Attention Model With Fine-Tuned Parameters in the Test Set of the Model Development Part (Set 1) and in the External Validation Part (Set 2)

 

In addition, we computed the SHAP value by using the ResNet with an attention model, which provides superior performance based on our experiments. According to multiple images predicted to be PCV, we observed pixels with large positive SHAP values (red-labeled pixels) clustered at high peak PED (Fig. 4A) with a probability of 1.0 and pixels with some positive SHAP values scattered at notching PED (Fig. 4B) with a probability of 0.48. A nonspecific SD-OCT image of SRF without PED, which could be observed in both wet AMD and PCV, showed a low number of pixels with positive SHAP values (Fig. 4C) with a probability of 0.45. In contrast to the PCV images, a far lower number of pixels with positive SHAP values (probability 0.06) could be observed on wet AMD-classified images (Fig. 4D). 

Figure 4.

 

SHAP values on SD-OCT images when applying the ResNet attention model.

Discussion

In this study, CNNs to differentiate PCV from wet AMD using cross-sectional SD-OCT images were proposed. Four simplified networks with more advanced techniques, including ResNet, ResNet with attention, Optic-Net, and a joint-attention Optic-Net, could enhance model performance beyond the traditional transfer learning technique. The attention mechanism was adopted in order to specify the area of interest that was essentially related to our task. These models achieved reasonably desirable AUC values ranging from 0.76 to 0.80 in training data. 

Compared to previous work, we adopted various advanced techniques in terms of both network architecture and the validation mechanism in order to differentiate PCV from wet AMD. In our study, the attention technique enhanced the model's performance compared to traditional supervised learning CNNs. However, Optic-Net did not outperform the residual attention networks. Complicated networks typically require large data sets in order to learn hidden insights and perform well. The observed inferior performance of the more advanced Optic-Net architecture in our study can likely be explained by the small size of our data set. 

To our knowledge, no prior work has focused on PCV identification in the Thai population. Since data collection in a medical domain related to human subjects is always challenging, a relatively small data set was included and analyzed in this study. A study from Yang et al.⁶² used AI models trained on a publicly available AI platform to diagnose PCV based on ICG images. AI performance was comparable to retinal specialists for diagnosing PCV from ICG images with an accuracy of 0.83. To the best of our knowledge, only two studies have used OCT images in training the AI model to detect PCV. Xu et al.⁶³ proposed the model, which takes simultaneously the fundus image and the OCT image as a bimodal input and categorizes it as a four-way output: wet AMD (excluding PCV), dry AMD (atrophic AMD and early stage AMD), PCV, and normal. The OCT-based model alone reached an accuracy of 83.2%, and the bimodal DL model combining fundus and OCT images had an improvement in accuracy of 87.4%. Another study from Chou et al.⁶⁷ showed a comparable result. They proposed a bimodal model with an ensemble stacking technique, combining color fundus photographs (CFPs) and clinical features of OCT biomarkers to distinguish between PCV and nvAMD with an accuracy of 83.67%, sensitivity of 80.76%, and specificity of 84.72%. In our proposed system, we evaluated the model at a patient level, which was relatively more suitable for a clinical application. We relied solely on OCT images without taking fundus images into consideration. Since only a small data set was available, a cross-validation technique was implemented to enhance model stability. We selected the best model based on the average AUC value between validation and test sets in order to strengthen the reliability of the model. The AUC from external validation is similar to the average AUC corresponding to the model development data. This implies a desirable level of generalizability of the model. 

According to the observed SHAP values, polyp lesions in PCV are observed underneath the retinal pigment epithelium (RPE) layer, so RPE bulging and peak, steep slope PED, or notching PED at the lesion site are observed on SD-OCT images. Pixels with large positive SHAP values clustered at the steep slope of PED (Fig. 4A) indicated specific features with a high possibility of PCV diagnosis, while pixels with extremely low positive SHAP values (Fig. 4D) indicated a high possibility of a wet AMD diagnosis. However, inferior performance of the algorithm was still observed in some SD-OCT images of PCV with prominent notching PED (Fig. 4B). 

Main limitations in differentiating SD-OCT images of serosanguinous macula lesions were that only a few cuts of SD-OCT scans that passed through the polyp lesion sites and showed a high positive SHAP value result were retrieved. From many other cuts in the same patient, polypoidal lesions were not obviously seen on SD-OCT images. These cuts may have exhibited intraretinal fluid, subretinal fluid, or nonspecific PED, which gave unreliable SHAP values and decreased the performance of automated algorithms (Fig. 4C). To enhance the performance of the algorithm, fine-turning different thresholds at a patient level with additional data could better differentiate those with polyp lesions detected. 

In medical practice, multiple cross-sectional SD-OCT images are generally obtained. All of these scans should be collectively considered in order to make a clear diagnosis. We provided the final prediction at a patient level in clinical practice at a threshold_p cutoff of 0.32. With the small patient-level threshold, there is a lower chance that wet AMD will be diagnosed. Our proposed method provided a desirable patient-level performance at a sensitivity of 0.85 and a specificity of 0.71 based on the external validation. As expected, higher sensitivity for detecting PCV was observed when applying the patient-level threshold compared to the image-level threshold. 

Combined multimodal retinal imaging techniques and noninvasive retinal investigations (e.g., OCT angiogram) were employed to identify polypoidal lesions to aid PCV diagnosis without the need for invasive FFA + ICG.^68–71 However, variable sensitivity was observed in detecting polypoidal lesions in OCTA due to an obscured flow signal by a pigment epithelial layer in the retina or by dense hemorrhage overlying the polypoidal lesions. The main advantage of our proposed model is that deep learning algorithms are used to differentiate PCV from wet AMD using cross-sectional SD-OCT images only. Our algorithm yielded promising performance of deep learning based on the use of only a small sample size. The main limitations include limited data and data quality, which are common problems in the medical field, especially in relatively new practical applications like PCV identification. When a model is implemented in clinical practice, the new data can be used to continuously improve the model. 

Conclusion

An automated detection system to differentiate PCV from wet AMD using SD-OCT images was proposed in this study. The typical transfer learning techniques with standard networks were compared with more advanced techniques, as well as with novel network architectures. The residual attention networks provided superior performance at an image level based on a limited amount of locally collected data. Further analysis at a patient level performed better, which portends the possible benefit of this technology in real-world clinical practice. The SHAP value technique was also implemented for interpretation purposes. Using the available data, a promising performance of the proposed model and an explanation regarding how it interprets data were both achieved. The collection of additional data to further train the model may enhance model performance. 

Acknowledgments

The authors thank Jirat Boonphun for contributing the results of standard networks using the transfer learning technique as the benchmark. 

Supported by a grant from the Kasetsart University Research and Development Institute. However, any opinions, findings, and conclusions or recommendations in this document are those of the authors and do not necessarily reflect views of the sponsor. 

Disclosure: P. Wongchaisuwat, None; R. Thamphithak, None; P. Jitpukdee, None; N. Wongchaisuwat, None