Deep Learning for Diagnosing and Segmenting Choroidal Neovascularization in OCT Angiography in a Large Real-World Data Set

Abstract

Purpose: To diagnose and segment choroidal neovascularization (CNV) in a real-world multicenter clinical OCT angiography (OCTA) data set using deep learning.

Methods: A total of 105,66 OCTA scans from 3135 eyes, including 4701 with CNV and 5865 without, were collected in five eye clinics. Both 3 × 3-mm and 6 × 6-mm scans of the central and temporal macula were included. Scans with CNV were collected from multiple diseases, and scans without CNV were collected from both healthy controls and those with multiple diseases. No scans were removed during training or testing due to poor quality. The trained hybrid multitask convolutional neural network outputs a CNV diagnosis and membrane segmentation, respectively.

Results: The model demonstrated a highly accurate CNV diagnosis (area under receiver operating characteristic curve = 0.97), achieving a sensitivity of 95% at 95% specificity. The model also correctly segmented CNV lesions (F1 score = 0.78 ± 0.19). Additionally, model performance was comparable on both high-definition 3 × 3-mm scans and low-definition 6 × 6-mm scans. The model did not suffer large performance variations under different diseases. We also show that a subclinical lesion in a patient with neovascular age-related macular degeneration can be monitored over a multiyear time frame using our approach.

Conclusions: The proposed method can accurately diagnose and segment CNV in a large real-world clinical data set.

Translational Relevance: The algorithm could enable automated CNV screening and quantification in the clinic, which will help improve CNV diagnosis and treatment evaluation.

Introduction

Choroidal neovascularization (CNV) is pathologic angiogenesis in which abnormal vessels connected to the choroid develop in the outer retina.^1–5 It can lead to vision loss and blindness, with symptoms including hemorrhage, lipid exudation, fluid exudation, or formation of fibrotic scars.⁶ Early diagnosis is key to preventing these outcomes.⁷ Reported methods for CNV classification/segmentation rely on indocyanine green/fluorescein angiography (ICGA/FA), optical coherence tomography (OCT), and/or OCT angiography (OCTA).^8–22 Limitations for CNV analysis using FA and ICGA include^9–12 (1) a restriction to two-dimensional (2D) imaging that projects three-dimensional (3D) features onto a 2D plane; (2) detection reliant on the appearance of hyperfluorescence caused by CNV vessel leakage, which obscures CNV vessels at the capillary level; and (3) a limited ability to diagnose and characterize nonexudative lesions. A final concern is that as invasive techniques, ICGA and FA are ill-suited for routine examination and screening. Structural OCT also has important limitations for CNV diagnosis and quantification.^13–15 While it can display retinal pigment epithelium (RPE) alterations caused by CNV, pathologic features caused by nonexudative CNV appear similar to drusen in structural OCT.²³ 

Recently, OCTA has introduced dyeless, depth-resolved in vivo imaging of retinal circulatory networks down to capillary scale resolution.²⁴ In contrast to dye injection modalities, OCTA procedures are noninvasive, rapid, and inexpensive—each exceptional qualities for a screening technology. Furthermore, OCTA is capable of detecting subclinical CNV,²⁵ and OCTA measurements of CNV lesion complexity and size are predictive of treatment response.^22,26 Nonetheless, CNV diagnosis using OCTA faces several challenges. Choroidal neovascularization identification may require clinicians and graders to review cross-sectional scans.²³ Because a single OCTA data volume typically contains hundreds of cross-sectional scans, manual identification may be time-consuming. This issue is exacerbated by the presence of artifacts in OCTA data.^27,28 In particular, projection artifacts can cause specious flow signal that mimics the morphology of superficial vasculature to appear in the outer retina. Such artifacts can be confused for true CNV or obscure its signal. 

Fortunately, these concerns can be alleviated by well-designed software solutions. The reported CNV diagnosis/segmentation methods can be classified into two categories: rules-based^{15–18,20,21} and deep learning–based methods.^13,14,19 The relative efficacy of these different approaches remains to be assessed and will depend on the nature of the algorithms being compared. However, in our previous study, a rules-based method could fail on challenging scans with strong residual artifacts or large CNV lesions and could not differentiate CNV signal from residual artifacts.^16,22 On a similar data set and using similar inputs, we reported a convolutional neural network (CNN) that avoided these drawbacks. This is not surprising because deep learning offers greater context sensitivity and can learn complex features that are not easily captured through codified rules, and it should be further noted that deep learning approaches have consistently outperformed rules-based approaches in image analysis tasks.²⁹ However, the existing deep learning–based method was limited to a small data set consisting exclusively of 3 × 3-mm macular scans of patients with age-related macular degeneration (AMD).¹⁹ But 3 × 3-mm scans typically have higher image quality than the larger scan sizes that are also frequently used in the clinic, and CNV is not an AMD-specific pathology. 

In this study, we developed and validated a hybrid multitask CNN model for CNV diagnosis and membrane segmentation. We evaluated performance using a real-world clinical data set of 10,566 OCTA scans from 3135 eyes collected at five eye clinics. This data set included not just eyes diagnosed with neovascular AMD but also several other diseases in which CNV is implicated. Likewise, controls without CNV consisted of not just healthy eyes but also eyes diagnosed with other common and rare retinal diseases. 

Methods

Data Set

The data set used in this work was collected from multiple eye institutes, including the Casey Eye Institute, Oregon Health Science University, Portland, OR, USA; Shanxi Eye Hospital, Taiyuan, Shanxi, PR China; Zhongshan Ophthalmic Center, Sun Yat-Sen University, Guangzhou, Guangdong, PR China; Tongren Hopital, Beijing, PR China; and the Department of Ophthalmology, Aichi Medical University, Nagakute, Japan. OCTA scans with CNV were identified by retina experts (please see “Ground Truth Label Generation” section) in multiple retinal diseases, including neovascular AMD, pathologic myopia, and polypoidal choroidal vasculopathy. Remaining cases were associated with rare diseases, including macular telangiectasia, choroideremia, pattern dystrophies, acute multifocal placoid pigment epitheliopathy, vitelliform macular dystrophy, Doyne honeycomb retinal dystrophy, chorioretinitis, angioid streaks, and familial dominant drusen. Control scans without CNV were collected and identified from healthy eyes and eyes with diabetic retinopathy, nonneovascular AMD, branch retinal vein occlusion, central retinal vein occlusion, branch retinal artery occlusion, central retinal artery occlusion, retinitis pigmentosa, central serous chorioretinopathy, or birdshot chorioretinopathy. 

This study was conducted in compliance with the Declaration of Helsinki. Participants were scanned using a 70-kHz commercial spectral-domain OCTA system (RTVue-XR; Optovue, Fremont, CA, USA) with a central wavelength of 840 nm. Both 3 × 3-mm and 6 × 6-mm scans of the central and temporal macula were included. The temporal macular data was included to improve the data set diversity, as CNV can occur at this location as well. Two sets of B-frames were acquired at the same position, and the split-spectrum amplitude-decorrelation angiography algorithm was utilized to generate OCTA signal.²⁴ One X-Fast and one Y-Fast volume were acquired and registered to suppress motion artifacts.³⁰ In 3 × 3-mm scans, each OCT/OCTA volume included 304 B-frames, and each B-frame included 304 A-lines, providing ∼10 µm/line sampling density. Sampling density was reduced in half (∼20 µm/line) in 6 × 6-mm scans. Repeat and follow-up scans centered on the same area were included in the data set. The follow-up scans were treated as unique samples since the CNV patterns noticeably changed. No single sample was included in both training or testing. No scans were excluded due to low image quality. 

Preprocessing

Projection artifact removal is an essential preprocessing step for generating anatomically accurate angiograms of plexuses beneath the superficial vascular complex (Fig. 1A). In this work, the projection-resolved (PR) OCTA algorithm³¹ removed projection artifacts volumetrically while preserving the real flow signal, enabling a clear presentation of the CNV vasculature. To produce structural or angiographic en face images or volumes of specific anatomic slabs, retinal layer segmentation is also required in preprocessing. Our algorithm requires segmenting the inner limiting membrane (ILM), the outer border of the outer plexiform layer (OPL), and Bruch's membrane (BM). This was done using a graph search-based algorithm implemented in our COOL-ART custom-designed OCTA reading software.³² Segmentation errors on the challenging scans were manually corrected using the same software. In this work, we distinguish between the inner retina, defined as the slab between the boundaries of the ILM and outer border of the OPL, and the outer retina, which is the slab between the boundaries of the outer border of the OPL and BM. In healthy eyes, the inner retina is vascularized, while the outer retina is avascular. 

Figure 1.

 

Proposed CNV diagnosis and segmentation method. (A) OCTA data was preprocessed by removing projection artifacts and segmenting retinal layers; OCTA scans were further graded as CNV/non-CNV scans, and CNV membrane area was outlined in a projection-resolved outer retinal angiogram. (B) Model inputs consist of an en face OCTA image set, thickness maps, an en face structural OCT image set, and outer retinal OCT and OCTA volumes. The OCTA image set consists of uncorrected inner and outer retinal angiograms, a slab-subtracted outer retinal angiogram, and a projection-resolved outer retinal angiogram, all produced using maximum projection. The structural OCT image set includes en face mean projections of 10 equal partitions of the entire data volume as well as a mean projection covering just the inner retina and a mean projection of the entire volume. (C) The proposed hybrid multitask CNN model combines 3D and 2D convolution; the feature extraction block is shared between the CNV diagnosis and membrane segmentation modules. F1 to F8 are tensor connection tags.

Ground Truth Label Generation

The ground truth CNV diagnosis and membrane segmentation (Fig. 1A) were generated from projection-resolved outer retinal OCT angiograms graded by two experts. To exclude residual artifacts, the graders also referred to the inner retinal angiogram. The first grader (KT) labeled the CNV scans and drew the CNV membrane area. The second grader (TTH) reviewed and refined the first grader's labels. Questionable scans were reviewed in collaboration. B-scans were also referenced to confirm the position of CNV in unclear or low-quality scans. 

Deep Learning Model for CNV Diagnosis and Segmentation

The proposed hybrid multitask CNN model achieves two tasks: (1) CNV diagnosis and (2) CNV membrane segmentation (Fig. 1C). The architecture includes several subnets: one each for 3D and 2D feature extraction, bridge subnets to integrate the 2D and 3D features, and a subnet for CNV diagnosis and membrane segmentation. In the 3D to 2D feature compression bridge subnet, mean and maximum projection operations are applied to extract important and general features from the volume; these features are input into the 2D feature extraction subnet. This allows the proposed model to learn to identify volumetric features useful for projection onto 2D images. Since the data set included two different fields of view (3 × 3 and 6 × 6 mm), we incorporated feature extraction at two different scales (3 × 3 and 5 × 5 convolution kernels with depth 4 and 6, respectively). This improved the network's ability to learn relevant features from both input field of view sizes. In the last step, the extracted features were fed into one Res-Net³³ subnet for CNV diagnosis and a U-Net³⁴ subnet for CNV membrane segmentation. Both the segmentation and diagnosis errors were evaluated by the categorical cross-entropy, and training loss was the average of the segmentation and diagnostic error. 

An important part of our algorithm's design is incorporating a large but relevant input library consisting of both OCT/OCTA volumes and their 2D projections. For the 2D input arm, the OCTA inputs to our algorithm consist of four en face angiograms: (1) the inner and (2) outer retinal uncorrected OCTA angiograms, (3) the slab-subtracted outer retinal angiogram,^18,20,35 and (4) the projection-resolved outer retinal angiogram³¹ (Fig. 1B). Both slab subtraction and PR-OCTA are processing techniques used to remove projection artifacts. Including both methods, as well as uncorrected angiograms, provides a set of images that can help unambiguously distinguish CNV. The uncorrected inner and outer retinal angiograms can be used to identify projection artifacts, as the artifacts in the uncorrected outer retinal angiogram are duplicated in the vascular pattern in the inner retinal angiogram. The uncorrected outer retinal angiogram may also include true flow suppressed by the slab-subtracted or projection-resolved solutions. However, in uncorrected angiograms, the true CNV signal is often obscured by overlying projection artifacts. In slab subtraction processing, the (scaled) inner retinal flow signal is subtracted from the outer retinal angiogram; this preserves the most significant CNV features, but less prominent features may be removed. The projection-resolved outer retinal angiogram can provide a clear image of the CNV vascular pattern, but residual projection artifacts could still be confused with the real CNV signal. 

The network also takes 2D structural OCT projections and thickness information as input for the 2D subnet (Fig. 1B). Pathologic changes in reflectivity are particularly associated with exudative CNV, so structural images can also help identify CNV lesions. To preserve some depth information for the network to learn, we split the structural volume into 10 equal partitions, using mean projection to obtain the en face images. The mean projection images of the entire structural volume and inner retina were also included. The width of different anatomic layers is also often modulated by the presence of a CNV membrane. Thickness maps for both the inner and outer retina can, therefore, also help identify CNV lesions, and these maps were included in the model input. Additionally, thickness maps can help the network recognize projection artifacts since they often coincide with retinal pigment epithelium elevations.¹⁹ 

Finally, the 3D input arm accepts the outer retinal structural OCT, and PR-OCTA volumes with A-lines were resampled to identical sizes (Fig. 1B). These volumes enabled the network to learn the complete spatial information latent in the data. 

Implementation

The loss of both the CNV diagnosis and segmentation was calculated by the cross-entropy function, and the training loss was defined as the average of CNV diagnosis and segmentation losses. During the training, the Adam optimizer was utilized to accelerate convergence. Batch normalization was also applied to help prevent overfitting and improve generalization capability. A learning rate decay of 10⁻¹ was used to achieve optimal parameters, and the minimum learning rate was 10⁻⁶. 

The designed CNN was implemented using TensorFlow with an intel i9-10980XE@3.0GHZ, DDR4 256 GB RAM, and Nvidia Quadro RTX 8000 graphics card (4608 CUDA cores, 48 GB GDDR6 memory). 

Results

Study Population

A total of 10,566 OCTA scans from 3135 eyes were collected, including 4701 scans with CNV and 5865 without (Tables 1 and 2). Repeat and follow-up centered on the same area were included in the data set. The follow-up scans were treated as unique samples since these scans showed large, apparent changes in the CNV vascular patterns (Fig. 2). The enrolled OCTA scans have a wide range of signal strength indice (SSI; 34 ≤ SSI ≤ 95). 

Table 1.

 

Data Set Enrolled in the Study

 

Table 2.

 

CNV Data Enrollment in Retinal Diseases

 

Figure 2.

 

A representative case in which follow-ups with large, apparent changes were considered independent samples.

Overall Performance Evaluation of CNV Diagnosis and Membrane Segmentation

Fivefold cross-validation was applied to test the performance of the proposed model on the entire data set. Individual samples were included in either only the training or only the testing data set. Diagnostic sensitivity was 95%, with 95% specificity, and the area under receiver operating characteristic curve (AROC) was 0.97 (Table 3). CNV membrane segmentation performance was evaluated by measuring the intersection over union (IOU) and F1 score defined by  

\begin{eqnarray} \left[\begin{array}{@{}*{2}{c}@{}} {\rm{IOU}} = \frac{{{\rm{GT}} \cap {\rm{Output}}}}{{{\rm{GT}}\cup {\rm{Output}}}} &{{\rm{Precision}} = \frac{{{\rm{TP}}}}{{{\rm{TP}} + {\rm{FP}}}}}\\ {{\rm{Recall}} = \frac{{{\rm{TP}}}}{{{\rm{TP}} + {\rm{FN}}}}}&{{\rm{F}}1{\rm{\;Score}} = 2\frac{{{\rm{Precision\;}} \times {\rm{\;Recall}}}}{{{\rm{precision}} + {\rm{Recall}}}}} \end{array} \right] \end{eqnarray}

(1)

 

where GT is the ground truth membrane segmentation and Output is the segmentation predicted by the proposed model, TP is a true positive, FP is a false positive, and FN is a false negative. The IOU measures the overlap between the ground truth and model predictions, precision measures the rate of correctly segmented CNV area in the output, recall measures the rate of correctly segmented CNV area within the ground truth CNV membranes, and F1 score measures the segmentation accuracy by taking into account both precision and recall. Generally, there was an agreement between the model predictions and the ground truth. The average IOU was 0.66, and the F1 score was 0.78 (Table 3). The F1 score was 0.79 ± 0.23 on both 3 × 3-mm and 6 × 6-mm scans with SSI ≥ 50. However, even if we consider low-quality scans (SSI < 50), the F1 score was 0.73 ± 0.22, which indicates that the CNV membrane prediction was usually essentially correct. We evaluated the methodologic contributions of the proposed model using ablations by excluding the 3D and 5 × 5 feature extraction subnets, separately. Both the 3D subnet and 5 × 5 feature extraction subnets contribute to both CNV diagnosis (Delong's method,³⁶ P < 0.01) and membrane segmentation (t-test, P < 0.01). 

Table 3.

 

Overall Performance of CNV Diagnosis and Membrane Segmentation

 

A majority of scans were accurately segmented and diagnosed with high confidence (Fig. 3, row 1). Undersegmentation mostly occurred when CNV vessels with weak signal co-occurred with strong residual projection artifacts (Fig. 3, row 2). Oversegmentation usually resulted from some confusion with non-CNV pathology (e.g., drusen being included in the segmentation result) or in scans with only indistinct CNV due to either weak signal or strong residual artifacts (Fig. 3, row 3). Finally, the major cause for large errors was low contrast between CNV vessels and the background (Fig. 3, row 4). The cases in which the network performed poorly often also required a time-consuming review of cross-sectional scans to achieve accurate ground truth segmentations, indicating that these scans were also difficult for human graders to analyze. 

Figure 3.

 

Representative CNV membrane segmentation results. (A) Inner retinal angiogram. (B) Projection-resolved outer retinal angiogram with CNV membrane ground truth (white outline). (C) Output CNV membrane probability map. (D) Structural OCT cross section at the location of the white line in A, with flow signal overlaid (violet: inner retinal; yellow: pathologic outer retinal; red: choroidal). Row 1: example in which the network achieved an accurate segmentation. Row 2: example of undersegmentation. The lesion in this scan extended beyond the OCTA scan's field of view, which may be difficult for the network to analyze properly due to the truncated nature of the information available at the edge of the image. Row 3: an example of oversegmentation and low prediction confidence in the CNV membrane area. Note that the pathologic flow signal in the outer retina was of similar magnitude to the background in this scan, making analysis difficult. Row 4: an example of a segmentation error in an extremely low-quality scan. This eye exhibits extreme defocus and a large number of residual artifacts. Note that the network did make a low-confidence prediction of a CNV membrane in a region that overlapped with the ground truth.

Performance of CNV Diagnosis and Membrane Segmentation on Different Scan Sizes

We evaluated performance separately on both 3 × 3-mm and 6 × 6-mm scans that were acquired at the same visit. The CNV diagnostic sensitivities were 99% and 90% in 3 × 3-mm and 6 × 6-mm scans with 95% specificity, respectively. Both the CNV diagnosis and membrane segmentation accuracy were higher in 3 × 3-mm scans (Table 4, P < 0.01); this could indicate that better image resolution with a more detailed CNV flow signal helped to improve CNV diagnostic and segmentation accuracy. 

Table 4.

 

Performance of CNV Diagnosis and Membrane Segmentation on Different Scan Sizes

 

Both scan sizes face challenges for CNV diagnosis and segmentation. Small scans may fail to capture the entire extent of a CNV lesion (Fig. 4B1, case 1). On the other hand, 6 × 6-mm scans in commercial systems are typically obtained with undersampling, leading to reduced signal-to-noise ratios and an increase in the quantity and severity of artifacts. While our model does suffer some performance decline on larger scans, the segmentation remains accurate on average. The model can therefore be used to effectively evaluate CNV progression even as lesions grow to cover regions larger than typical 3 × 3-mm scan patterns (Figs. 4C1, C2). 

Figure. 4.

 

CNV membrane segmentation on both 3 × 3-mm and 6 × 6-mm scans. (A) Inner retinal angiogram. (B) Projection-resolved outer retinal angiogram with CNV membrane ground truth (white outline). (C) CNV probability map. (D) Structural OCT cross section at the location of the white line in (A), with flow signal overlaid (violet: inner retinal; yellow: pathologic outer retinal; red: choroidal). Case 1 shows a large CNV lesion that cannot be fully imaged using a small scanning window. Case 2 shows a small CNV lesion in a defocused scan with strong residual artifacts. Both cases at both resolutions resulted in correct detections.

Performance of CNV Diagnosis and Membrane Segmentation on Different Diseases

We also investigated model performance in different diseases and with different pathologies (Fig. 5). The model achieved similar, high performance among the different retinal diseases, including both leading causes of blindness (AMD),^37–40 a pathology that is difficult to image with OCTA (polypoidal choroidal vasculopathy),^41–44 and rare diseases. The outputs of the CNV diagnostic probabilities were similar in different diseases (Table 5, P > 0.10). The CNV membrane segmentation accuracy of rare diseases was better than the other three diseases (P < 0.05), which can be explained by the best image quality of rare disease scans (P < 0.01) that helped to increase the CNV segmentation accuracy. 

Figure 5.

 

CNV membrane segmentation results in several retinal diseases. (A) Inner retinal angiogram. (B) Projection-resolved outer retinal angiogram with CNV membrane ground truth (white outline). (C) CNV probability map. (D) Structural OCT cross section at the location of the white line in (A), with flow signal overlaid (violet: inner retinal; yellow: pathologic outer retinal; red: choroidal). Row 1: example segmentation in neovascular AMD. Row 2: example segmentation in angiomatous proliferation. The identity of the lesion is best appreciated in the cross-sectional image (D2). Despite the small lesion size, this scan resulted in a correct prediction. Row 3: example segmentation in polypoidal choroidal vasculopathy (PCV). The flow signal in PCV is often very weak, which makes it difficult to distinguish from the background (B3). Row 4: example segmentation in pathologic myopia. Eyes with pathologic myopia often have extreme defocus in OCTA imaging (A4, B4). The residual artifacts in these scans probably account for the relatively low performance (C4).

Table 5.

 

Performance of CNV Diagnosis and Membrane Segmentation on Different Diseases

 

Clinical Application

A growing number of CNV vascular pixels predict exudation.²⁵ Using the proposed method, we segmented CNV membrane areas on 3 × 3-mm OCTA scans acquired from a patient with neovascular AMD who had 46 months of follow-up visits and also generated CNV vessel masks using the Otsu algorithm⁴⁵ within the membrane areas (Fig. 6). The CNV membrane and vascular areas were very small when they were first identified but grew rapidly. Conversion to exudative CNV was observed at a 6-month follow-up visit. The patient was subsequently treated with anti-vascular endothelial growth factor (VGEF) therapy. Following treatments, the CNV pattern changed dramatically, with the CNV vessel area growing more slowly than the membrane area. 

Figure 6.

 

CNV growth dynamics illuminated by the proposed detection algorithm. CNV vessels were detected in the segmented CNV membrane area (white outline) using Otsu's algorithm. CNV vascular patterns changed dramatically after treatment. The scatterplot of the CNV vessel area and membrane area shows the CNV dynamics following anti-vascular endothelial growth factor (VEGF) treatments (red arrows) in longitudinal follow-ups.

Discussion and Conclusion

In this study, we developed a hybrid multitask CNN model for CNV diagnosis and membrane segmentation. The proposed model was tested on a realistic clinical data set and showed reliable CNV diagnosis and membrane segmentation. The overall AROC of segmentation was 0.97, and the primary CNV membrane area was detected with a 0.78 F1 score. 

Several novel features contributed to the model's accurate CNV diagnosis and membrane segmentation. 

First, the proposed model was trained with a real-world, diverse data set containing over 10,000 scans. These scans were collected from multiple eye clinics, included multiple eye diseases, and spanned a large range of image qualities. Since no scans were excluded due to quality, many kinds of artifacts that we might see in the clinic (such as shadows, defocusing, and motion) were apparent in the data set. Scans also included many pathologies, such as geographic atrophy, drusen, macular edema, and hard exudates. While this data set was consequently more challenging, by using it, the model was able to learn more contextual information, which could prevent underperformance on artifact-rich scans. 

Second, our model contains a large number of carefully chosen inputs. By including two projection artifact correction techniques, the network had access to information that enabled it to identify the projection artifacts that often plague OCTA studies. This capability is paramount for any automated approaches operating in the outer retina, which is near the strongly reflecting RPE and, therefore, prone to disruption by projection artifacts. Additionally, the en face OCT images and thickness maps can identify pathologic regions since exudation can cause changes in retinal thickness and reflectivity. In tandem, these structural OCT and OCTA inputs provide separate information that can be correlated to identify CNV more readily. Finally, the OCT and OCTA volumes include spatial information that can help identify real CNV flow since CNV occurs at specific anatomic depths. 

Third, a novel CNN architecture was designed specifically for the tasks in this work. The architecture contains separate pipelines for 3D and 2D information to ensure that the structure of each data format can be fully exploited. By separating the diagnosis and segmentation tasks, a Res-Net³³ and a U-Net³⁴ structure could be employed. These network architectures are considered state of the art in medical image processing and have previously shown excellent performance in OCTA imaging in particular.²⁹ The proposed method also relied purely on deep learning to determine outputs, removing arbitrary feature size thresholding used in some OCT feature segmentation architectures.^19,46 

Choroidal neovascularization is a significant development in several diseases.^{2,41–44,47,48} However, as indicated by several of the example images in this study, it is not always prominently visible in realistic data sets (which will include low-quality scans). Performance on pathologic myopia exhibited the most variation in IOU, recall, and F1 scores. In general, scans from eyes with pathologic myopia were of low quality due to difficulty removing defocus (Fig. 5, row 4). Unsurprisingly, scans with severe artifacts like those present in pathologic myopia were more difficult for the model to analyze. Rare diseases are also challenging since this category contains a variety of pathologies that may not share similar features. Evaluating the performance on each specific pathology could illuminate this variation in performance, but the data set used in this work is not large enough to comment on this in a statistically satisfactory manner. 

The proposed algorithm does have some limitations. 

The reliance on retinal layer segmentation—even though the proposed work only requires knowledge of the location of the inner limiting membrane, outer plexiform layer, and Bruch's membrane—is one such limitation. While automated retinal layer segmentation algorithms exist,³² manual correction is frequently required to produce accurate layer boundaries on some scans with a large amount of pathology. Future work should include improved automation of retinal layer segmentation or forego reliance on layer boundaries altogether. 

We also did not include a vessel segmentation module in this study. Vessel segmentation has been included in other CNV segmentation platforms¹⁹; however, in the low-definition scans (6 × 6 mm) included in this study, CNV vessel width can be as small as a single pixel. And with small, dim vessels, it is difficult to say which pixels exactly correspond to true flow, even for a human grader. Under these conditions, it is difficult to generate a ground truth to train a CNN for CNV vessel segmentation, and it is also difficult to gauge performance since metrics like the IOU are sensitive to anatomically irrelevant pixel displacements. However, this limitation should be addressed if vessel metrics such as tortuosity prove useful for analyzing CNV progression. 

Due to the unbalanced data set used in this work, we were unable to investigate model performance based on patient identity characteristics. We are collaborating with different clinics to collect balanced sub–data sets to explore model performance in different ethnic groups. Similarly, we would like to characterize model performance in specific clinics, but we also lack balanced data sets to make this comparison. This could also be addressed in future work. 

Comparisons with other algorithms could also be considered, but these are best performed on public data sets. Unfortunately, the OCTA-500⁴⁹ data set was normalized and compressed from 32 to 8 bits. The ground truth CNV segmentation in our work was generated by reviewing PR-OCTA in high data precision volumes. This allowed us to include suspicious small and weak CNV flows that are one of the main targets in our work since these represent early manifestations of a pathology implicated in a high risk of vision loss. Due to the lower bit depth in the OCTA-500 data set, we are unable to similarly identify such early, small lesions, meaning that one of the main advantages of the model cannot be reasonably assessed on this data set. 

Last, while it is generally possible to improve network design to enhance performance, errors in model output in this work were usually on scans that were difficult to assess by eye. Clinicians might also miss the CNV in these cases, and the ground truth membrane area is ambiguous anyway (Fig. 2B4). For these reasons, software or hardware enabling improved image quality may be a more fruitful approach to improving OCTA analysis of CNV. 

Despite these limitations, the proposed model achieved two important tasks: CNV diagnosis and membrane segmentation. These tasks were achieved on a clinically realistic data set that included over 10,000 scans of several diseased and healthy eyes from several clinics. Our results indicate the plausibility of automating CNV analysis in the clinic. 

Acknowledgments

Supported by the National Institutes of Health (R01 EY024544, R01 EY027833, P30 EY010572, T32 EY023211, UL1TR002369), an unrestricted departmental funding grant and H. James and Carole Free Catalyst Award from Research to Prevent Blindness (New York). 

Disclosure: J. Wang, Optovue/Visionix (P, R); T.T. Hormel, None; K. Tsuboi, None; X. Wang, None; X. Ding, None; X. Peng, None; D. Huang, Optovue/Visionix, Inc. (F, P, R), Boeringer Ingelheim (C); S.T. Bailey, None; Y. Jia, Optovue/Visionix, Inc. (P, R), Optos (P)