Macular Ischemia Quantification Using Deep-Learning Denoised Optical Coherence Tomography Angiography in Branch Retinal Vein Occlusion

Abstract

Purpose: To examine whether deep-learning denoised optical coherence tomography angiography (OCTA) images could enhance automated macular ischemia quantification in branch retinal vein occlusion (BRVO).

Methods: This retrospective, single-center, cross-sectional study enrolled 74 patients with BRVO and 46 age-matched healthy subjects. The severity of macular ischemia was graded as mild, moderate, or severe. Denoised OCTA images were produced using a neural network model. Quantitative parameters derived from denoised images, including vessel density and nonperfusion area, were compared with those derived from the OCTA machine. The main outcome measures were correlations between quantitative parameters, and areas under receiver operating characteristic curves (AUCs) in classifying the severity of the macular ischemia.

Results: The vessel density and nonperfusion area from denoised images were correlated strongly with the corresponding parameters from machine-derived images in control eyes and BRVO eyes with mild or moderate macular ischemia (all P < 0.001). However, no such correlation was found in eyes with severe macular ischemia. The vessel density and nonperfusion area from denoised images had significantly larger area under receiver operating characteristic curve than those derived from the original images in classifying moderate versus severe macular ischemia (0.927 vs 0.802 [P = 0.042] and 0.946 vs 0.797, [P = 0.022], respectively). There were no significant differences in the areas under receiver operating characteristic curve between the denoised images and the machine-derived parameters in classifying control versus BRVO, and mild versus moderate macular ischemia.

Conclusions: A neural network model is useful for removing speckle noise on OCTA images and facilitating the automated grading of macular ischemia in eyes with BRVO.

Translational Relevance: Deep-learning denoised optical coherence tomography angiography images could enhance automated macular ischemia quantification.

Introduction

Macular ischemia is one of the most important causes of visual loss in common retinal vascular diseases such as diabetic retinopathy (DR) and retinal vein occlusion (RVO).¹ It could cause disorganization of the retinal inner layers and compromise the photoreceptors in the outer retina.^2,3 The severity of macular ischemia not only impacts anatomic and functional outcomes, but is also associated with the clinical course and responsiveness to treatment for macular edema in retinal vascular diseases.^4–7 It is important to have a reliable quantitative macular ischemia classification scheme. 

In recent years, optical coherence tomography angiography (OCTA) provides a rapid, noninvasive, and volume-rendering imaging tool for evaluating retinal microvascular changes. The parafoveal vessel density (VD) and nonperfusion area (NPA) on OCTA images are useful quantitative biomarkers for determining the severity of macular ischemia.^4,8–10 However, diffuse speckle noise on the OCTA images of eyes with a large NPA is a common cumbersome problem (Fig. 1A).¹¹ It may lead to a substantial increase in false-positive vascular signals and interfere with VD calculation (Fig. 1B). More important, these false-positive signals may impede automated NPA calculations (Fig. 1C). Although several automated algorithms have been developed for parafoveal NPA calculations in DR,^9,12–18 none had been validated in RVO. One possible reason is that, although the NPA is usually small and disseminated in DR,⁶ it could be large ^4,7 and accompanied by more prominent speckle noise in RVO. Most traditional local or global thresholding methods have difficulty removing prominent speckle noise (Fig. 1D–E). Such noise could occur consistently in sequential OCTA images; hence, even multiple image averaging methods may still be unable to remove them completely. 

Figure 1.

 

Effect of different denoising methods in an eye with severe macular ischemia. (A) Original OCTA image. (B) VD map provided by machine software. Numbers in grids show machine-derived VDs, which were significantly overestimated in grids with diffuse speckle noise. (C) The whitish area denotes the parafoveal NPA calculated from the dark blue regions in (B). (D) Global thresholding by Otsu method. (E) Local adaptive thresholding by the Phansalkar method. (F) Red boundaries demarcate superior and inferior half-ring subfields used for macular ischemia grading. (G) NN denoised OCTA image. (H) Perfusion density map calculated from denoised OCTA image. (I) The whitish area denotes the NPA calculated using the denoised OCTA image.

It is unknown how speckle noise may affect macular ischemia quantification, and whether deep learning denoising technique could improve those quantitative measurements, especially in eyes with a large NPA. This study used a novel neural network (NN) model (Fig. 2) for speckle noise removal and binarization of OCTA images in eyes with branch RVO (BRVO). The aim of this study was to evaluate whether the denoising process may enhance the automated VD and NPA calculation, which in turn will facilitate the grading of macular ischemia in eyes with BRVO. 

Figure 2.

 

Proposed architecture of encoder (a) and decoder (b) NN in this study.

Methods

This retrospective, single-center, cross-sectional study reviewed OCTA images obtained between November 2017 and October 2019 at Chang Gung Memorial Hospital, Keelung, Taiwan. The study was approved by the Chang Gung Memorial Hospital Institutional Review Board (IRB no. 201901669B0) and followed the tenets of the Declaration of Helsinki. The BRVO group comprised 74 eyes from 74 patients. The control group consisted of 46 normal eyes from age-matched healthy subjects. The exclusion criteria were (1) macular ischemia caused by other retinal diseases such as DR, central RVO, or ocular ischemic syndrome, (2) the presence of significant media opacity such as dense cataract or vitreous hemorrhage, and (3) inadequate quality of OCTA image (scan quality score of <6/10). 

Macular Ischemia Grading in BRVO Eyes

The severity of macular ischemia of each OCTA image in the BRVO group was classified using a method inspired by a prior fluorescein angiography grading scheme that considered specific anatomic circumstances in BRVO.¹⁹ The annular parafoveal area was divided into two half-ring–shaped subfields (i.e., superior and inferior) (Fig. 1F). Macular ischemia on OCTA images was graded as mild if the ischemic area was one-third or less of the involved subfield, moderate if one-third or more but two-thirds or less of the involved subfield, and severe more than two-thirds of the involved subfield. Two trained graders independently determined the grading for each eye with BRVO before further quantitative analyses. In case of any disagreement, an experienced retinal specialist (L.Y.) adjudicated the final grading. Agreement between graders was evaluated using the intraclass correlation coefficient. 

OCTA Parameters

OCTA images of a 3 × 3-mm scan, centered on the fovea, were acquired in all eyes using the AngioVue (Optovue RTVue XR Avanti; Optovue Inc., Fremont, CA) machine. The machine's AngioVue software (version: A2017,1,0,151) was used for automatic segmentation. Images of the superficial vascular plexus, which includes the vasculature between the internal limiting membrane and 10 µm above the inner plexiform layer, were used in this study. The VD was defined as the percentage of area occupied by all the vascular components in a particular region. The vascular components are those pixels classified as vessel during the binarization of the images. The same definition was used in both machine software and our denoised images. Parafoveal area was the annulus area between the 1-mm and 3-mm diameter circles within the Early Treatment Diabetic Retinopathy map. The VD used in this study was the parafoveal area VD calculated by ([area of vascular components within parafoveal region/total area of parafoveal region] × 100)%. 

Brief Introduction of the Novel NN Model

The NN model was created by Python 3 (Python Software Foundation; https://www.python.org/) and Pytorch 1.6 (Pytorch Software Foundation: https://pytorch.org/) on GTX 1080ti graphics card. The present architecture (Fig. 2) was inspired by the original U-net.²⁰ With such encoder and decoder architecture adopted, features were extracted from an OCTA image to generate the segmentation image. There were two major improvements in the NN model used in this study. First, this network used a residual block mechanism²¹ to avoid the gradient vanishing and degradation problem in a vast/deep feedforward network model. It could deliver low-level features to high-level layers when directly transmitting information in backpropagation. Second, the present decoder structure adopts a parallel network for complex tasks. One of the decoders is the unique Gaussian generator layer for producing a Gaussian feature map. It could simulate the shape of a blood vessel to enhance the ability of the decoding procedure. 

The use of the Gaussian generator layer was inspired by observing a blood vessel's distribution, as shown in Supplementary Figure S1. Blood vessels comprising several two-dimensional Gaussian hills could be observed using the Matlab meshgrid tool. Therefore, this study used the shrinking block to generate the cubic features, namely, B × H × W × C, representing the batch size, height, width, and channel, respectively. The number of channels in the cubic feature is 6, denoted as confidence,  µ_x, µ_y, σ_x,σ_y and ρ to simulate the bivariate Gaussian distribution as follows (1).  

\begin{eqnarray*}\!f\left( {x,y} \right) &=& \frac{{confidence}}{{2\pi {\sigma _x}{\sigma _y}\sqrt {1 - {\rho ^2}} }}\exp\nonumber\\ && \left( {\frac{1}{{2\left( {1 - {\rho ^2}} \right)}}\left[ {\frac{{{{\left( {x - {{\rm{\mu }}_x}} \right)}^2}}}{{{\sigma _x}^2}} + \frac{{{{\left( {y - {{\rm{\mu }}_y}} \right)}^2}}}{{{\sigma _y}^2}} - \frac{{2\rho \left( {x - {{\rm{\mu }}_x}} \right)\left( {y - {{\rm{\mu }}_y}} \right)}}{{{\sigma _x}{\sigma _y}}}} \right]} \right)\quad\end{eqnarray*}

(1)

where x, y are coordinates of a Gaussian feature map, σ_x, σ_y > 0 and ρ > 0. The Adam optimizer was used, and the loss function L was defined as cross entropy loss (CE) between ground truth \(\hat y\) and output y, which is shown as follows:  

\begin{eqnarray*}{\rm{L}} = {\rm{CE}}({y,\hat y})\end{eqnarray*}

 

We compiled a database containing 60 OCTA images (304 × 304 pixels) of the superficial vascular plexus and corresponding ground truths with manually pixel-wise segmented vascular components (Fig. 3). The manual segmentations were first done by two trained graders. Each annotated image would then be checked and confirmed by an experienced retinal specialist. Six OCTA images (10%) were segmented by both graders for cross-validation. The cross-validation metrics are in Supplementary Table S1. A root mean square error that is less than 0.025 and a structural similarity index metric that is greater than 0.750 suggest that our manual annotation work has an acceptable consistency. The OCTA database is composed of 11 normal eyes and 49 eyes with various retinal vascular diseases. The database was separated into 30 training images and 30 images for testing. The performance metrics of testing images across scan qualities as well as that of all testing images were summarized in Table 1. More technical details of our model and how it compares with other well-established algorithms are available elsewhere.²² 

Figure 3.

 

Representative OCTA images used for the training and testing of our NN model. (A–C) Normal eye. (D–F) An eye with proliferative DR. (A, D) Original OCTA images. (B, F) Ground truth images with manually segmented vascular components. (C, F) Superimposing the ground truth images on the respective original OCTA images.

Table 1.

 

The Performance of the NN Model Across Different Scan Qualities of Testing Images

 

Parameters Calculated From Denoised OCTA Images

A binarized denoised OCTA image was obtained for each eye using the NN model as described elsewhere in this article(Fig. 1G). Each pixel in the image had been given a probability of being vessel versus nonvessel. A threshold of 0.5 was used to generate the binarized output image. The VD over the parafoveal area was measured using the ImageJ software (National Institutes of Health, Bethesda, MD; http://rsb.info.nih.gov/ij). To calculate the NPA, a perfusion map was created using a previously described method.²³ In brief, various morphologic image processing operations, including skeletonization and Gaussian blur filter in OpenCV (Open Source Computer Vision Library: http://opencv.org), were implemented to generate the perfused area. With the perfusion map generated (Fig. 1H), low pixel value regions were identified as the nonperfused regions and the contour detection in OpenCV was performed to calculate the areas of all parafoveal nonperfused regions (Fig. 1I). 

Parameters Derived From Images Produced by the OCTA Machine

The machine-derived VD in this study refers to the parafoveal VD produced by the AngioVue software. Although the OCTA machine does not automatically calculate the NPA, it does provide a VD map (Fig. 1B) with color keys. Therefore, we were able to estimate the sizes of NPA by calculating the dark blue areas within the parafoveal area in this VD map by using OpenCV (Fig. 1C). 

Statistical Analysis

Demographic data between BRVO eyes and controls were compared using independent samples t test for continuous variables and the χ² test for categorical variables. The Pearson correlation coefficient was used to determine the correlation between machine-derived quantitative parameters and the corresponding parameters calculated from denoised OCTA images. The classification ability was illustrated by the area under receiver operating characteristic curve (AUC) for control versus BRVO, mild versus moderate, and moderate versus severe. The comparison of AUCs was performed using DeLong's test. A two-tailed P value of less than 0.05 was considered statistically significant. All data were analyzed using IBM SPSS Statistics Version 26.0 (IBM Corp., Armonk, NY). 

Results

In total, 74 eyes from 74 patients (28 female and 46 male) with BRVO and 46 eyes from 46 healthy controls (19 female and 27 male) were enrolled in this study. The mean age was 62.4 ± 10.6 years and 61.6 ± 9.2 years in the BRVO and control groups, respectively. There was no difference in age (P = 0.416) or gender (P = 0.705) between the two groups. In eyes with BRVO, macular ischemia was classified as mild in 30 eyes, moderate in 35 eyes, and severe in 9 eyes. The intraclass correlation coefficient between the two masked graders was 0.94. Figure 4 illustrates the representative cases in control eyes and BRVO eyes with macular ischemia of different severity. 

Figure 4.

 

Representative cases of macular ischemia of different severity. Speckle noise on OCTA image is not obvious in control eye (A) and eye of mild macular ischemia (E). However, speckle noise becomes visible in moderate macular ischemia (I) and is substantial in severe macular ischemia (M). Possible significant false-positive vascular signals (N).

Correlations between machine-derived parameters and denoised OCTA parameters are shown in Table 2. A progressive decrease in the VD and enlargement in the NPA were found, with an increased severity of macular ischemia in eyes with BRVO. Parameters calculated from denoised OCTA images correlated strongly with machine-derived parameters (all P < 0.001) in control eyes and BRVO eyes with mild or moderate macular ischemia. However, no such correlation was found in BRVO eyes with severe macular ischemia. 

Table 3 shows the AUCs for classifying macular ischemia of different severity. The VD and NPA calculated from denoised OCTA images had a better performance than the corresponding parameters calculated from machine-derived images did in classifying moderate versus severe macular ischemia (0.927 vs 0.802 [P = 0.042] and 0.946 vs 0.797 [P = 0.022], respectively). There were no significant difference in AUCs between denoised images and machine-derived parameters in classifying control versus BRVO, and mild versus moderate macular ischemia. 

Table 2.

 

Pearson Correlation Between Machine-Derived Image Parameters and Denoised OCTA Image Parameters

 

Table 3.

 

AUC Based on Different Quantitative Parameters

 

Discussion

BRVO is a common retinal vascular disease with prevalence of 4.42 per 1000 in the adult population.²⁴ Macular perfusion status is one of the most important factors associated with its visual prognosis and anti-vascular endothelial growth factor treatment response.^4,10,25–27 Many earlier studies used fluorescein angiography for macular ischemia grading,^2,19,28 but the irregular configuration and dynamic changes of fluorescein leakage may lead to high inter-rater variability.¹⁹ Quantitative measurements on OCTA images may determine the macular perfusion status with better reproducibility.²⁹ Perfusion density or VD, which could usually be obtained using OCTA machine software, has been shown predictive in functional outcome and risk of recurrent macular edema in eyes with BRVO.^5,8,10,27 

Nonetheless, diffuse speckle noise on OCTA images could be frequently seen in eyes with severe macular ischemia. During image acquisition, a variation of the signals may occur randomly or nonrandomly, which in aggregate are called noise¹¹ The noise component is usually small when the vascular signal is strong, whereas it could predominate in areas with weak signals.¹¹ Therefore, speckle noise could present as substantial false-positive vascular signal in eyes with a large area of nonperfusion. 

Much effort had been made to improve quantitative measurements on OCTA images. Binarization, which classifies each pixel into vessel versus nonvessel, is an important process before further quantitative analyses could be done. Some OCTA machines use their own proprietary software, for instance, Cirrus (Carl Zeiss Meditec Inc., Dublin, CA), AngioVue (Optovue Inc., Fremont, CA), and RS-3000 Advance (Nidek, Gamagori, Japan). Other platforms may require exporting images for post-processing with different thresholding methods.³⁰ Global mean-based, ImageJ default, global/local Otsu, local Phansalkar, and other thresholding or filtering algorithms have been used widely in research.^30,31 Although easy to perform, the major trade-off of these thresholding methods was the loss of details of fine vessels during a vigorous denoising procedure (Fig. 1D). In contrast, preserving more details of fine vessels usually retains more noise in the image (Fig. 1E). 

The multiple image averaging approach has been shown to be a promising method for decreasing background noise and improving vessel continuity on OCTA images.^32,33 However, capturing multiple images over the same area could significantly increase image acquisition time.³⁴ The loss of information in the microcirculation may occur in the multiple image averaging process,³⁴ and erroneously enlarged vessel caliber owing to subtle misalignments between registered frames are also possible.³⁵ 

Advancement in artificial intelligence opens a new window for coping with this complicated noise problem on OCTA images.^14,36 For example, OCT HS-100 (Canon Inc., Tokyo, Japan) has built-in propriety deep learning algorithms to provide denoised image with higher contrast-to-noise ratio and peak signal-to-noise ratio.³⁶ However, capillary over–dropout artifacts, capillary overgeneration, and motion artifacts may still exist in some denoised images.³⁶ Further large-scale studies could be helpful in validating its correlation with functional and anatomic outcomes. 

This study not only adopted a useful encoder and decoder segmentation NN structure, but also proposed an attention mechanism, called the Gaussian generator mechanism, and incorporated it into the decoder to simulate the signal's Gaussian distribution in a cross-section of the retinal vessel. These methods can facilitate a black box deep learning model in understanding human vision and get good performance. The present results demonstrated that this NN model can be effective in removing speckle noise while preserving the majority of fine vessels in eyes with macular ischemia of different severities. 

Although denoising and binarization were usually performed in separate steps via different methods in other models,^14,30–39 our NN model removes the speckle noise as it performs binarization. Our advantage is convenience. The output images can be used directly for quantitative measurements. Our drawback is that some delicate details in the grey scaled image could be lost during processing. For example, the intensity of signal, which might provide information on the flow velocity of capillary, could be lost. 

Our results showed a strong correlation between machine-derived quantitative parameters and corresponding parameters derived from denoised OCTA images in control eyes and BRVO eyes with mild or moderate macular ischemia, but not in eyes with severe macular ischemia. This finding could be explained by the mechanism explained elsewhere in this article, namely, that the noise may be more predominant in areas with weak signals,¹¹ resulting in a more profound change during the denoising process in eyes with severe macular ischemia. Denoised OCTA images had a significantly better performance in classifying moderate versus severe macular ischemia. In other words, denoising could be an important process in quantitative measurements and classifying macular ischemia in BRVO eyes, especially in those with large NPA. 

Parafoveal NPA on OCTA images is an important prognostic factor in retinal vascular diseases. An increased size of the parafoveal NPA has shown a strong association with impaired visual acuity and decreased macular sensitivity in RVO.^4,7 Parafoveal NPA is also useful for the diagnosis, staging, and predicting of the risk of progression in DR.^6,12,13 Manual demarcation of the NPA could be time consuming and may cause inter-rater variability. Several algorithms have been developed for automatically measuring the size of parafoveal NPA in eyes with DR using the vessel–distance map,¹⁸ multilevel imaging processing,¹³ or convolutional NN^6,12 with good reliability. However, in contrast with eyes with DR, in which the parafoveal NPA were usually small and disseminated, the parafoveal NPA in RVO could sometimes be large and accompanied by prominent speckle noise. Therefore, it remains unclear how those automated algorithms may perform in eyes with RVO. 

Diffuse speckle noise may impede the automated NPA calculation by presenting multiple patches of false-positive vascular signals over the large ischemic area (Fig. 1B). Clearing up these noises makes the automated parafoveal NPA measurement much easier to perform. This study created a perfusion map from a denoised, skeletonized, and Gaussian-smoothed OCTA image. The NPA could be easily determined by filtering out the areas of low perfusion density. The mean NPAs in this study were 0.65 mm², 1.89 mm², and 2.94 mm² in mild, moderate, and severe macular ischemia, respectively. With the total area of a half ring–shaped subfield being [(1.5²π–0.5²π)/2] ≈ 3.14 mm², the area percentage of NPA in involved subfields were 21%, 60%, and 94%, respectively. These values were compatible with the present macular ischemia grading scheme that had cut points at one-third and two-thirds of the involved subfield. The high AUCs of NPA suggested that it is a powerful parameter in grading of macular ischemia in BRVO, and might even be better than VD. 

There are some limitations in this study. First, images with poor scan quality score (<6/10) were excluded. Therefore, the obtained results cannot be generalized to all BRVO images. In view of the rapid advancement in scanning technology of OCTA, the quality problem might eventually be alleviated. Second, our NN model had not been trained to handle shadow artifacts and motion artifacts. However, there were many other methodologies developed for this purpose. For examples, our OCTA machine has both a built-in eye tracking (DualTrac) function and a motion correction technology that are useful for decreasing motion artifacts.^37,38 Automated software has also been developed for removing shadow artifacts.³⁹ We believe that all these technologies, as well as our NN model, could combine to provide a more meaningful and reliable quantitative analysis of OCTA images. Third, the sample size in this study was small. Correlation between quantitative parameters derived from denoised images and functional or anatomic outcomes merits further large-scale investigations. However, macular ischemia is frequently seen not only in BRVO, but also in other retinal vascular diseases. Hence, the present NN model plus denoising method has potential applications in other retinal vascular diseases such as DR and central RVO. 

Conclusions

This study showed that the NN model is useful in removing speckle noise on OCTA images, providing better quantitative parameters, including VD and NPA, which in turn facilitates automated quantitative grading of macular ischemia in eyes with BRVO. The denoise process is particularly important in eyes with large NPA. In fact, any of the aforementioned denoising algorithms, including multilevel thresholding, multiple image averaging, and deep learning models, might also have this potential facilitating effect if speckle noise could be delicately removed. 

Acknowledgments

Supported by grants from Chang Gung Memorial Hospital (CMRPG2J0212) and the Ministry of Science and Technology of Taiwan (MOST 109-2314-B-182A-025-). 

Disclosure: L. Yeung, None; Y.-C. Lee, None; Y.-T. Lin, None; T.-W. Lee, None; C.-C. Lai, None