Classification of Pachychoroid on Optical Coherence Tomographic En Face Images Using Deep Convolutional Neural Networks

Abstract

Purpose: To study the efficacy of deep convolutional neural networks (DCNNs) to differentiate pachychoroid from nonpachychoroid on en face optical coherence tomography (OCT) images at the large choroidal vessel.

Methods: En face OCT images were collected from eyes with neovascular age-related macular degeneration, polypoidal choroidal vasculopathy, and central serous chorioretinopathy. All images were prelabeled pachychoroid or nonpachychoroid based on quantitative and qualitative criteria for choroidal morphology on multimodal imaging by two retina specialists. In total, 1188 nonpachychoroid and 884 pachychoroid images were used for training (80%) and validation (20%). Accuracy for identification of pachychoroid by DCNN models was analyzed. Trained models were tested on a test set containing 79 nonpachychoroid and 93 pachychoroid images.

Results: The accuracy on the validation set was 94.1%, 93.2%, 94.7%, and 94.4% in DenseNet, GoogLeNet, ResNet50, and Inception-v3, respectively. On a test set, each model demonstrated accuracy of 80.2%, 83.1%, 89.5%, and 90.1% and an F1 score of 0.782, 0.824, 0.904, and 0.901, respectively.

Conclusions: DCNN models could classify pachychoroid and nonpachychoroid with good performance on OCT en face images. Automated classification of pachychoroid will be useful for tailored treatment of individual patients with exudative maculopathy.

Translational Relevance: En face OCT images can be used by DCNN for classification of pachychoroid.

Introduction

Pachychoroid is a condition characterized by pathologically dilated Haller vessels (i.e., pachyvessel), choriocapillaris attenuation, and choroidal vascular hyperpermeability.^1–3 These features are well observed on B-scans and en face images of optical coherence tomography (OCT) and indocyanine green angiography.^4,5 Recognizing pachychoroid is important in exudative maculopathies because some have distinctive features from typical age-related macular degeneration (AMD). These exudative maculopathies were often considered AMD but can now be classified into the pachychoroid category. The pattern of pachyvessel in pachychoroid diseases can best be visualized with OCT en face using long wavelengths such as swept-source or enhanced depth spectral-domain images.^4,6 

The spectrum of pachychoroid disease encompasses central serous chorioretinopathy (CSC), pachychoroid neovasculopathy (PNV), and thick-choroid polypoidal choroidal vasculopathy (PCV).^2,3,7 Previous studies have shown that the pattern of vessels at the choroidal large vessel layer differs between pachychoroid and nonpachychoroid eyes.^1,4,6,8–12 Vascular densities are significantly increased in diffusely distributed vessels with large diameters in pachychoroid eyes, while only focal or no pachyvessels are observed in nonpachychoroid eyes. These findings suggest that choroidal large vessel morphology can be used to differentiate pachychoroid and nonpachychoroid eyes. 

Pachychoroid and typical AMD seem to differ in clinical features, genetics, natural course, and responsiveness to treatment.^13–16 Patients with pachychoroid are usually younger.^7,14,16,17 In addition, a lower injection rate with a longer injection-free period was reported in PCV eyes with pachychoroid features.^13,18 Similar results have been reported in PNV compared to typical AMD.¹⁹ Therefore, pachychoroid identification would be helpful in disease prognosis of a patient and in planning tailored treatment options and follow-up intervals. 

As diagnosis of retinal diseases depends largely on imaging, artificial intelligence (AI), especially its subclassification deep learning, has been actively applied in many areas of retina-associated disease, including diabetic retinopathy and AMD.^20–22 Deep convolutional neural networks (DCNN) have been applied to achieve automated grading of AMD using color fundus images²⁰ to detect specific lesions and subtypes of AMD, as well as in decision making for treatment.^23–25 In the current study, we investigated the performance of DCNN models for detection of pachychoroid using en face OCT images at the choroidal large vessel layer level to investigate the potential of automatic classification of pachychoroid. As this is the first study to train DCNN models using en face OCT images, the results of the current study will also provide details of the DCNN models that could be useful in future studies using en face OCT images. 

Materials and Methods

The study was approved by the Institutional Review Board of Seoul and Bucheon St. Mary's Hospital (HC19RESI0086 and HC21RASI0007), which waived the need for written informed consent because of the study's retrospective design. The study was conducted in accordance with the tenets of the Declaration of Helsinki. 

Subjects

En face OCT images of patients with exudative macular diseases, including typical AMD, PCV, PNV, and CSC, were collected from consecutive patients who visited Seoul St. Mary's Hospital (The Catholic University of Korea, Republic of Korea) between April 2016 and July 2020. All patients underwent high-definition OCT (DRI OCT Triton; Topcon Corporation, Tokyo, Japan) with enhanced depth imaging, fluorescent angiography (Optos California P200DTx; Optos, Dunfermline, United Kingdom), and indocyanine green angiography (ICGA; Optos California P200DTx) at baseline to confirm the diagnosis. 

All images were prelabeled as pachychoroid or nonpachychoroid. Confirmation of pachychoroid strictly followed the quantitative and qualitative definition described below by two retina specialists to avoid controversies. The criteria were based on thorough review of previous publications and comprised⁷: (1) subfoveal choroidal thickness (SFCT) ≥ 300 µm and (2) presence of attenuation of the inner choroid with dilated choroidal vessels (pachyvessel) under the diseased area on OCT B-scans. (3) Presence of pachydrusen on fundus photography and OCT and choroidal vascular hyperpermeability on ICGA were the optional features used to confirm pachychoroid. We specified the quantitative value of SFCT and utilized multimodal images in the definition of pachychoroid to minimize the ambiguity of distinguishing pachychoroid from nonpachychoroid. 

The exclusion criteria for this study were as follows: (1) choroidal neovascularization other than AMD (e.g., punctate inner choroidopathy, vitelliform dystrophy); (2) history of previous treatment or scar at the retina (e.g., photodynamic therapy, laser photocoagulation, intraocular injections, periocular injections, and systemic injections); (3) high myopia (> −6.00 diopters or axial length > 26 mm); (4) poor image quality (signal strength index < 40); and (5) large subretinal/preretinal hemorrhage or pigment epithelial detachment that could obscure the choroidal vascular image. 

Image Preparation, Model Training, and Performance Metrics

Assessment of choroidal morphology was largely based on our previous studies.^4,13 Volume data of the posterior pole were acquired over a 12 × 9 mm² area containing 512 × 256 A-scans using OCT. En face image acquisition was subjected to automated segmentation and topology normalization using IMAGEnet6 software (version 1.19.11030; Topcon Corporation, Tokyo, Japan), with the Bruch’s membrane used as a reference plane for “flattening” operation. En face images of choroidal large vessel layer used in this study were obtained at 50% of the total choroidal thickness for analysis of the large choroidal vessel layer. All images were saved as JPEG files. 

Training and validation of DCNN models were performed using MATLAB R2020a (MathWorks, Inc., Natick, MA, USA). Among various pretrained DCNN architectures, this study included DenseNet, GoogLeNet, ResNet50, and Inception-v3 for transfer learning. DCNN is a neural network that can process three-dimentional data of color images and composed of convolution layer and pooling layer. A brief description for each DCNN architecture used in this study follows: (1) DenseNet utilizes dense connections between layers, through Dense Blocks, all layers with matching feature-map sizes are connected directly with each other; (2) GoogLeNet is a 22-layer CNN which is a variant of the Inception, in which stack modules or blocks within which are convolutional layers; (3) ResNet50 also stacks blocks and consists of five stages each with a convolution and Identity block; (4) Inception-v3 is an architecture from the Inception family with several improvements (Fig. 1). Selection of DCNN architectures was based on the top-5 error rate reported previously in image classification and application in ophthalmologic images.^26,27 

Figure 1.

 

Training, internal validation, and external validation of deep convolutional neural network (DCNN) models. En face optical coherence tomographic (OCT) images were obtained at the level of total choroidal thickness for analysis of the large choroidal vessel layer. The images were labeled either “pachychoroid” or “nonpachychoroid” by two independent graders. Each en face OCT image was scaled to 224 × 224 × 3 pixels for DCNN training and trained using DenseNet, GoogLeNet, ResNet50, and Inception-v3 architectures. From the dataset, 80% of the images were used for training, and the remaining 20% was used for validation. Trained DCNN models were externally validated using a test set.

During the training process, each en face OCT image was scaled to 224 × 224 × 3 pixels. The input images were labeled as either “pachychoroid” or “nonpachychoroid” by two independent graders experienced in retinal imaging. In total, 1267 nonpachychoroid and 977 pachychoroid images were collected. Randomly selected 79 nonpachychoroid and 93 pachychoroid images were used as a test set. Rule of thumb was not applied in distribution of test set to use more data on model training and validation. The rest of the images were trained with 80% of the dataset, which was randomly selected: 950 nonpachychoroid and 707 pachychoroid images. The remaining 20% was used as the validation set: 238 nonpachychoroid and 177 pachychoroid images. The performance of trained models was tested on a test set. 

All experiments were conducted on a computer equipped with NVIDIA RTX 2060 and Intel i7 CPUs. Each model was trained for 30 epochs with a maximum of 1320 iterations. The performance of each model was evaluated for accuracy, precision, recall (sensitivity), specificity, F1 score, and kappa score. 

Statistics

Statistical analysis was performed using a commercial program (Statistical Package for the Social Sciences version 22.0.1 for Windows; IBM Corp., Armonk, NY, USA). Independent t-test was used to compare demographics between groups, and the chi-square test was used to compare categorical variables. Accuracy, precision, recall, specificity, and F1 score were calculated for each model. The kappa score mean and standard deviation (SD) were used to determine the agreement between truth and each model. 

Results

Characteristics of the Subjects

In total, 370 images of CSC, 237 images of PNV, 370 images of PCV, and 912 images of typical AMD were used for training/validation and test. For the dataset used in training and validation, the mean age in the nonpachychoroid and pachychoroid group was 73.78 ± 8.83 and 61.59 ± 12.24, respectively (P < 0.001). Proportion of male was 50.19% and 70.49%, respectively (P < 0.001). For the test set, the mean age was 73.90 ± 10.44 and 59.62 ± 12.5 (P < 0.001) and the male percentage was 41.94% and 70.76% (P = 0.001) for nonpachychoroid and pachychoroid, respectively. No statistical difference was observed in mean age, sex distribution, or subfoveal choroidal thickness between the dataset and test set (all P ≥ 0.144). Clinical characteristics of the subjects are summarized in Table 1. 

Table 1.

 

Characteristics of Each Dataset

 

Performance of the DCNN Models

The final accuracy on the validation set was 94.1%, 93.2%, 94.7%, and 94.4% in DenseNet, GoogLeNet, ResNet50, and Inception-v3, respectively (Fig. 2). The time spent in model training was 171, 9, 21, and 37 minutes, respectively. 

Figure 2.

 

Internal validation performances of the models. The final accuracy of the 20% validation set was 94.1%, 93.2%, 94.7%, and 94.4% in DenseNet, GoogLeNet, ResNet50, and Inception-v3, respectively.

In a test set, the models demonstrated 80.2%, 83.1%, 89.5%, and 90.1% accuracy, respectively. The precision of the models was 0.9683, 0.9444, 0.8947, and 0.9872 and recall was 0.6559, 0.7312, 0.9140, and 0.8280, respectively. The F1 score of each model was 0.7821, 0.8242, 0.9043, and 0.9006, respectively. The accuracy, precision, recall, specificity, F1 score, and kappa values of each model are summarized in Table 2. 

Table 2.

 

Classification Performance of the Deep Convolutional Neural Network Models

 

Analysis of Prediction Errors of DCNN Models

In the test set, one nonpachychoroid image was misclassified as pachychoroid, and 11 pachychoroid images were misclassified as nonpachychoroid by all four models. The images of the nonpachychoroid eye that was misclassified as pachychoroid were from a 64-year-old female with PCV (Fig. 3). Although the image was labeled as nonpachychoroid due to a subfoveal choroidal thickness of 219 µm, OCT B-scans of the eye revealed dilated vessels under the lesion, and en face OCT revealed increased vascular density with diffuse distribution of dilated large vessels. For pachychoroid eyes that were misclassified as nonpachychoroid, all eyes were diagnosed with PCV, and the mean subfoveal choroidal thickness was 303.82 ± 5.46 (range: 300–315 µm). Figure 4 shows a representative case of a misclassified pachychoroid image. 

Figure 3.

 

A representative case of a nonpachychoroid eye misclassified as pachychoroid. A. Optical coherence tomography (OCT) B-scan demonstrates a subfoveal choroidal thickness of 219 µm, but dilated Haller’s vessels are seen under the lesion (asterisks). B & C. Indocyanine green angiography and fluorescein angiography reveal polypoidal lesion and leakage, confirming the diagnosis of polypoidal choroidal vasculopathy. D. En face OCT shows increased vascular density with diffuse distribution of Haller’s vessels, which might have caused misclassification of this eye as pachychoroid.

Figure 4.

 

A representative case of a pachychoroid eye misclassified as nonpachychoroid. A. Optical coherence tomography (OCT) B-scan demonstrates subfoveal choroidal thickness of 300 µm, diffusely dilated Haller's vessels (asterisks), and attenuation of choriocapillaris + Sattler's layer under the lesion (arrow). B & C. Indocyanine green angiography and fluorescein angiography reveal polypoidal lesion and leakage, confirming the diagnosis of polypoidal choroidal vasculopathy. D. Although en face OCT shows diffuse distribution of Haller's vessels, the diameter of the vessels does not appear increased, which might have caused misclassification of this eye as nonpachychoroid.

Discussion

Pachychoroid should be distinguished from other causes of exudative maculopathy, especially typical AMD, because these two conditions differ in clinical features, genetics, natural course, and responsiveness to treatment.^13–16 In the current study, we evaluated the performance of several DCNN models trained to classify nonpachychoroid and pachychoroid using en face OCT images. The results demonstrated that the DCNN model could identify pachychoroid eyes with good performance, and certain DCNN architectures showed better performance than others. Analysis of errors in classification revealed that most occurred in PCV eyes with equivocal features. 

Description of vascular patterns at the choroidal large vessel level was initiated by Dansingani et al.¹ They demonstrated that diseases of pachychoroid entities share morphologic findings in the choroid, including increased thickness and dilated outer choroidal vessels seen on OCT en face. Numerous studies have confirmed the peculiar morphology of the choroidal vasculature of pachychoroid eyes, especially based on en face OCT images.^{1,4,6,28–30} Ng et al.⁶ revealed that different exudative maculopathies show distinctive choroidal vasculature morphology and can be classified accordingly. Vascular density had a greater diffuse pattern of dilated vessels in pachychoroid eyes compared to nonpachychoroid AMD eyes in quantitative analysis.^4,30 These morphologic differences can serve as features that could distinguish pachychoroid from nonpachychoroid eyes for DCNNs. 

In this study, DCNN architectures of DenseNet, GoogLeNet, ResNet50, and Inception-v3 were trained using en face OCT images that represent features of the choroidal large vessel level in each disease. The results revealed an accuracy of 80.2% to 90.1% with the independent test set. In our previous study, we demonstrated the feasibility of an auto-AI platform trained with ultrawidefield ICGA in identification of pachychoroid.¹¹ The precision and recall of the model was up to 89.19%. DCNN models trained with en face OCT images in the current study showed comparable results in terms of precision compared to the auto-AI platform models trained using ultrawidefield ICGA images. The model trained with ResNet50 in this study revealed better performance in both precision and recall. 

There was some difference in performance of models trained with different DCNN architectures. The F1 scores of ResNet50 and Inception-v3 were higher than those of DenseNet and GoogLeNet. This may be due to their inherent capacity for image classification, as demonstrated with lower error rates in previous ImageNet Large Scale Visual Recognition Challenges²⁷, or the feasibility of the architecture for en face OCT images. The performances of ResNet50 and Inception-v3 were not only better, but the time consumed in training was fairly short relative to that of DenseNet. This result may assist in selection of DCNN architecture in training en face OCT images for future studies. 

For all DCNN models, the final accuracies for validation were higher compared to the accuracies yielded from the test set. This suggests the possibility of overfitting the models. To maximize the size of the training data, we included images from both eyes if the disease was bilateral, and this might have caused differences in internal validation of datasets and external validation using test sets. Nonetheless, the DCNN models demonstrated good performance, as demonstrated by high precision, recall, specificity, F1 score, and kappa score on external validation, especially with ResNet50 and Inception-v3. 

Analysis of the prediction errors revealed that misclassifications are likely to occur in equivocal cases, and a large proportion of these equivocal cases may contain PCV. Subfoveal choroidal thickness shows bimodal distribution with two peaks around 170 µm and 360 µm, and this suggests inhomogeneity of the disease in terms of choroidal morphology.¹⁷ Further studies are needed to support this hypothesis by showing different characteristics of PCV eyes with thin and thick choroid.^4,6,13,15 Most of the misclassification occurred in eyes with a subfoveal choroidal thickness between 300 and 315 µm. This can be interpreted as decreased capacity of the model to differentiate eyes in an overlapping range between nonpachychoroid and pachychoroid. Alternatively, it may also be explained by the possible existence of PCV entities with overlapping characteristics of thin and thick choroid, a hypothesis that requires further studies for support. 

Several limitations of this study should be considered. First, there might be a selection bias caused by omitting images with poor quality or other blocking materials. Eyes with large pigment epithelial detachment or hemorrhage should be treated carefully, but DCNN models trained in this study cannot distinguish such eyes. Second, the definition of pachychoroid may be controversial. Borrelli et al.³¹ reported that15.4% of neovascular AMD subjects were re-evaluated as pachychoroid. We minimized the controversy by specifying the quantitative value of SFCT and exploring multimodal images by two graders in setting the ground truth. Nonetheless, this can be a clear limitation of the current study since there can be overlap of SFCT between pachychoroid and nonpachychoroid eyes.³² Third, there was a significant difference in age and sex distribution between pachychoroid and nonpachychoroid eyes both in the dataset and test set. However, we believe this will not significantly affect clinical application of the models since younger age and male prominence are well-established features of pachychoroid and also should be observed in real life.^6,17 Further research involving en face images from many OCT machines and more retinal specialists are warranted to validate the results of the current investigation. 

In conclusion, DCNN models could distinguish pachychoroid from nonpachychoroid with good performance on OCT en face images. To the best of our knowledge, this is the first AI article that has dealt with en face OCT images and implies that OCT images of not only B-scans, but also en face scans can be used in automatizing disease classification. Furthermore, automated classification of pachychoroid will improve treatment of patients with exudative maculopathies by assisting in tailored treatment of each diseased eye. 

Acknowledgments

Supported by the Institute of Clinical Medicine Research of Bucheon St. Mary's Hospital, Research Fund, 2020 and a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute, funded by the Ministry of Health and Welfare, Republic of Korea (grant no: HI17C2012030018). 

Contributions were as follows: H.R., preparation of data and data analysis; N.Y.K., preparation of data; J.H.L., collection of data and data analysis; K.L., data analysis; J.B., conception and design of the study, writing manuscript text, preparing figures, collection and assembly of data, data analysis and interpretation, and supervision; W.K.L: supervision. All authors reviewed the manuscript. 

Disclosure: K. Lee, None; H. Ra, None; J.H. Lee, None; J. Baek, None; W.K. Lee, None