An Open-Source Deep Learning Network for Reconstruction of High-Resolution OCT Angiograms of Retinal Intermediate and Deep Capillary Plexuses

Abstract

Purpose: We propose a deep learning–based image reconstruction algorithm to produce high-resolution optical coherence tomographic angiograms (OCTA) of the intermediate capillary plexus (ICP) and deep capillary plexus (DCP).

Methods: In this study, 6-mm × 6-mm macular scans with a 400 × 400 A-line sampling density and 3-mm × 3-mm scans with a 304 × 304 A-line sampling density were acquired on one or both eyes of 180 participants (including 230 eyes with diabetic retinopathy and 44 healthy controls) using a 70-kHz commercial OCT system (RTVue-XR; Optovue, Inc., Fremont, California, USA). Projection-resolved OCTA algorithm removed projection artifacts in voxel. ICP and DCP angiograms were generated by maximum projection of the OCTA signal within the respective plexus. We proposed a deep learning–based method, which receives inputs from registered 3-mm × 3-mm ICP and DCP angiograms with proper sampling density as the ground truth reference to reconstruct 6-mm × 6-mm high-resolution ICP and DCP en face OCTA. We applied the same network on 3-mm × 3-mm angiograms to enhance these images further. We evaluated the reconstructed 3-mm × 3-mm and 6-mm × 6-mm angiograms based on vascular connectivity, Weber contrast, false flow signal (flow signal erroneously generated from background), and the noise intensity in the foveal avascular zone.

Results: Compared to the originals, the Deep Capillary Angiogram Reconstruction Network (DCARnet)–enhanced 6-mm × 6-mm angiograms had significantly reduced noise intensity (ICP, 7.38 ± 25.22, P < 0.001; DCP, 11.20 ± 22.52, P < 0.001), improved vascular connectivity (ICP, 0.95 ± 0.01, P < 0.001; DCP, 0.96 ± 0.01, P < 0.001), and enhanced Weber contrast (ICP, 4.25 ± 0.10, P < 0.001; DCP, 3.84 ± 0.84, P < 0.001), without generating false flow signal when noise intensity lower than 650. The DCARnet-enhanced 3-mm × 3-mm angiograms also reduced noise, improved connectivity, and enhanced Weber contrast in 3-mm × 3-mm ICP and DCP angiograms from 101 eyes. In addition, DCARnet preserved the appearance of the dilated vessels in the reconstructed angiograms in diabetic eyes.

Conclusions: DCARnet can enhance 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiogram image quality without introducing artifacts.

Translational Relevance: The enhanced 6-mm × 6-mm angiograms may be easier for clinicians to interpret qualitatively.

Introduction

Optical coherence tomographic angiography (OCTA) is a noninvasive imaging technique that can provide angiograms at specific depths within the retina and visualize microvasculature in vivo.^1,2 Numerous studies have demonstrated its utility in the diagnosis of retinal pathology.^3–5 However, current OCTA technology has some significant limitations. One of these limitations is the presence of artifacts. While many artifacts, such as projection artifacts and bulk motion artifacts, can be removed by postprocessing algorithms,^6–8 images may still have strong background noise, and vessels may become fragmented due to image processing and display strategies. Another important limitation in current OCTA technology is a relatively small field of view. Contemporary research is moving toward wider fields of view.^9,10 At a fixed device speed, capturing a larger field of view without increasing the duration of the imaging procedure requires reducing sampling density, which both lowers image resolution and exacerbates the effects of artifacts.¹¹ We previously reported on a deep learning–based method that effectively enhances en face images of the superficial vascular complex (SVC).¹² Plexus-specific pathology of the intermediate capillary plexus (ICP) and deep capillary plexus (DCP), however, can provide novel diagnostic and prognostic indicators.⁵ For example, dilated capillaries in the ICP and DCP are associated with the more severe diabetic retinopathy (DR)¹³ and may be associated with increased treatment requirements in branch retinal vein occlusion.¹⁴ However, optimal imaging of the ICP and DCP is more difficult due to their posterior location and the increased prevalence of artifacts. 

Several groups have proposed algorithms to improve OCTA image quality. However, most previous work has focused on traditional image-processing algorithms that must be handcrafted for specific contexts. This is in contrast to cutting-edge deep learning–based approaches, which usually generalize well.^15–20 State-of-the-art deep learning–based methods are currently applied to several different kinds of clinical image-processing tasks,²¹ such as image classification,^22,23 semantic segmentation,^24–31 and enhancement.^32,33 In this study, we propose and evaluate a Deep Capillary Angiogram Reconstruction Network (DCARnet), a deep learning–based method to enhance 6-mm × 6-mm and 3-mm × 3-mm ICP and DCP en face OCTA images. We have released the Python source code at https://github.com/octangio/DCARnet. 

Methods

Data Acquisition

The study was approved by the Institutional Review Board/Ethics Committee of Oregon Health & Science University, and informed consent was collected from all participants, in compliance with the Declaration of Helsinki. In total, 274 eyes from 180 participants were scanned. We acquired 3-mm × 3-mm and 6-mm × 6-mm OCTA macular scans from the same eye using a 70-kHz commercial OCTA system (RTVue-XR version 2017.1.0.151; Optovue, Inc., Fremont, California, USA). Each raster position takes two repeated B-scans to produce motion contrast and construct an OCTA data volume using the efficient split-spectrum amplitude-decorrelation algorithm.¹ Each 3-mm × 3-mm B-scan consists of 304 A-lines, or 101 A-lines/mm. There were 400 A-lines in each 6-mm × 6-mm B-scan, which yields a lower sampling density of 67 A-lines/mm. Thirty-eight 6-mm × 6-mm OCTA macular scans from a commercial 120-kHz spectral domain OCT system (Solix; Optovue, Inc.) with 15-µm lateral resolution and 5-µm axial resolution were acquired. 

All en face OCTA images require retinal layer segmentation in order to determine the regions over which flow signal should be projected and to isolate separate plexuses. We used a guided bidirectional graph search to perform this segmentation³⁴ (Figs. 1A, 1D). A projection-resolved (PR) OCTA algorithm was used to suppress the projection artifacts in retinal capillary plexuses.⁷ The 3-mm × 3-mm and 6-mm × 6-mm angiograms of the ICP (Figs. 1B, 1E) were generated by maximum projection of the OCTA signal in a slab including the inner plexiform layer (IPL) and inner nuclear layer (INL).³⁵ The 3-mm × 3-mm and 6-mm × 6-mm angiograms of the DCP (Figs. 1C, 1F) were projected in a slab including the INL and outer plexiform layer. 

Figure 1.

 

Examples of data acquisition in a healthy eye. (A) ICP (outer 20% of the ganglion cell complex and inner 50% of the INL) slab and DCP (outer 50% of the INL and outer plexiform layer) segmentation boundaries on a 3-mm × 3-mm scan. (B, C) The 3-mm × 3-mm angiograms of the ICP and DCP generated by maximum projection of the OCTA signal in the slabs delineated in (A). (D–F) Equivalent images for 6-mm × 6-mm angiograms from the same eye are of lower quality than 3-mm × 3-mm angiograms.

Convolutional Neural Network Architecture

DCARnet takes individual en face angiograms as input (Fig. 2A). DCARnet comprises shallow feature extraction layers, deep feature extraction layers, and a reconstruction layer. The image features at different scales are extracted by three bypasses. The first bypass includes 21 convolutional layers. The first convolutional layer with 128 channels operates on the input image without changing the resolution of the image to extract shallow features, and the remaining 20 convolutional layers with 64 feature maps are used to extract deeper features. The second bypass includes 16 convolutional layers and a deconvolutional layer. The stride of the first convolutional layer with 128 channels is 2 in order to down-sample the image and extract low-level features. The remaining 15 convolutional layers with 64 channels generate deeper features, and a deconvolutional layer with 128 channels is used to up-sample the image to achieve the same scale as the input image. The strides of the deconvolutional layer and remaining convolutional layers are 2 and 1, respectively. The third bypass includes 12 convolutional layers and 2 deconvolutional layers. The strides of the first and second convolutional layer with 128 channels are set to 2 to reduce image resolution and extract shallow features. The remaining 10 convolutional layers with 64 feature maps are applied to generate high-level features. The stride of both deconvolutional layers is set to 2 to achieve the same scale as the input image. Then, all of the deep features extracted by the three bypasses are fused by concatenation. The fused hierarchical features are sent to a reconstruction layer that has one feature map to produce the final reconstructed image (Fig. 2B). In all layers, the kernel size is 3-mm × 3-mm pixels. The feature extraction bypasses used in DCARnet can extract features at different scales. In the first bypass, all feature maps were kept at the same resolution as the original input. The other two bypasses have down-sampling layers that reduced the size of feature map by factors of 2 and 4, respectively. These two bypasses reduced the number of parameters to accelerate the training process and enhanced the tolerance of DCARnet to disturbances like noise and flow signal strength changes. The number of convolutional layers in the three passes decreases successively, making the network pay more attention to the feature extraction from the original input image. 

Figure 2.

 

DCARnet structure. Input to the network consists of a single OCTA en face image (A). The network is composed of a shallow feature extraction module and a deep feature extraction module, as well as a reconstruction module. The different-colored cubes represent convolutional layers or deconvolutional layers with different parameters. The kernel size in all layers is 3 × 3. The width and height of the input image are represented by w and h, respectively. The stride is represented by s. C represents concatenation operation. The output reconstructed angiogram shows higher contrast between vessels and background (B).

Training

Data Preprocessing

To acquire the input and the ground truth for training DCARnet, we up-sampled the 6-mm × 6-mm SVC (Fig. 3B), ICP, and DCP angiograms (Fig. 3(D)) to achieve the same scale as a 3-mm × 3-mm scan (Figs. 3A, 3C). We then registered the original 3-mm × 3-mm scan to the up-sampled 6-mm × 6-mm angiograms using the relatively noise-free and vasculature-rich SVC slabs¹²; the same transformations were applied to the deeper ICP and DCP slabs. Next, we cropped the maximum-area inscribed rectangle from registered 3-mm × 3-mm and 6-mm × 6-mm angiograms (Fig. 3E). The cropped 3-mm × 3-mm angiograms serve as the ground truth (Fig. 3F), and the cropped central 6-mm × 6-mm angiograms [Fig. 3(G)] are the inputs used for training DCARnet. 

Figure 3.

 

Data preprocessing flowchart. (A) The original 3-mm × 3-mm SVC angiogram. (B) Up-sampled 6-mm × 6-mm SVC angiogram. (C) The original 3-mm × 3-mm DCP angiogram. (D) Up-sampled 6-mm × 6-mm DCP angiogram. (E) Registered DCP angiograms. The yellow box is the largest inscribed rectangle. (F) Cropped original 3-mm × 3-mm DCP angiogram. (G) Cropped central 3-mm × 3-mm section from the 6-mm × 6-mm DCP angiogram. Preprocessing for ICP angiograms follows the same pattern.

Loss Function

The loss function in this work consisted of the mean square error³⁶ (MSE):  

\begin{equation}{\rm{MSE}} = \frac{1}{{{\rm{w\;}} \times {\rm{\;h}}}}\mathop \sum \nolimits_{{\rm{i}} = 1}^{\rm{w}} \mathop \sum \nolimits_{{\rm{j}} = 1}^{\rm{h}} {({{\rm{D}}_{\rm{T}}}\left( {{\rm{i}},{\rm{j}}} \right) - {{\rm{D}}_{\rm{P}}}\left( {{\rm{i}},{\rm{j}}} \right))^2},\end{equation}

(1)

the structural similarity index (SSIM),³⁷ 

\begin{equation}{\rm{SSIM}} = \frac{{2{{\rm{\mu }}_{{{\rm{D}}_{\rm{T}}}}}{{\rm{\mu }}_{{{\rm{D}}_{\rm{P}}}}} + {{\rm{C}}_1}}}{{{\rm{\mu }}_{{{\rm{D}}_{\rm{T}}}}^2 + {\rm{\mu }}_{{{\rm{D}}_{\rm{P}}}}^2 + {{\rm{C}}_1}}} \cdot \frac{{2{{\rm{\sigma }}_{{{\rm{D}}_{\rm{T}}}{{\rm{D}}_{\rm{P}}}}} + {{\rm{C}}_2}}}{{{{\rm{D}}_{\rm{T}}} + {\rm{\sigma }}_{{{\rm{D}}_{\rm{P}}}}^2 + {{\rm{C}}_2}}},\end{equation}

(2)

and root mean square (RMS) contrast loss index³⁸ (C_loss)  

\begin{eqnarray}{{\rm{C}}_{{\rm{loss}}}} &=& \Bigg| \sqrt {\frac{1}{{{\rm{w\;}} \times {\rm{\;h}}}}\mathop \sum \nolimits_{{\rm{i}} = 1}^{\rm{w}} \mathop \sum \nolimits_{{\rm{j}} = 1}^{\rm{h}} {{({{\rm{D}}_{\rm{T}}}\left( {{\rm{i}},{\rm{j}}} \right) - {{\rm{\mu }}_{{{\rm{D}}_{\rm{T}}}}})}^2}} \nonumber\\ &&- \sqrt {\frac{1}{{{\rm{w\;}} \times {\rm{\;h}}}} \times \mathop \sum \nolimits_{{\rm{i}} = 1}^{\rm{w}} \mathop \sum \nolimits_{{\rm{j}} = 1}^{\rm{h}} {{({{\rm{D}}_{\rm{P}}}\left( {{\rm{i}},{\rm{j}}} \right) - {{\rm{\mu }}_{{{\rm{D}}_{\rm{P}}}}})}^2}} \Bigg|,\nonumber\\\end{eqnarray}

(3)

where w, h  refer to the width and height of the image, respectively; D_T (i, j) and D_P(i, j) are the pixel value at position (i, j); \({{\rm{\mu }}_{{{\rm{D}}_{\rm{T}}}}}\) and \({{\rm{\mu }}_{{{\rm{D}}_{\rm{P}}}}}\) are the mean pixel values; \({{\rm{\sigma }}_{{{\rm{D}}_{\rm{T}}}}}\) and \({{\rm{\sigma }}_{{{\rm{D}}_{\rm{P}}}}}\) are the standard deviations; and \({{\rm{\sigma }}_{{{\rm{D}}_{\rm{T}}}{{\rm{D}}_{\rm{P}}}}}\) is the covariance of the ground truth image D_T and predicted image D_P, respectively. The values of the constants C₁ = 0.01 and C₂ = 0.03 were taken from the literature.³⁷ The full loss function is then a linear combination of these components:  

\begin{equation}{\rm{Loss}} = {\rm{MSE}} + \left( {1 - {\rm{SSIM}}} \right) + {{\rm{C}}_{{\rm{loss}}}}.\end{equation}

(4)

 

Together, each loss term provides helpful information during training. The MSE is used to measure the pixel-wise difference between ground truth and output image; SSIM is based on three comparison measurements: reflectance amplitude, contrast, and structure, while C_loss is used to further enhance the contrast of the image. Although MSE can suppress large errors during training, it does not consider the relationship between adjacent pixels and often produces excessively smooth textures. Just using MSE, it is almost impossible to capture high-frequency texture details to generate satisfactory perceptual results.³⁹ SSIM considers the similarity in three prominent perceptual categories in human vision: brightness, contrast, and shape. Even though SSIM considers the contrast of the overall structure, C_loss is used to further enhance the contrast so that the produced image will not be overly smooth—results using a contrast loss calculation preserve more details. The combination of these loss functions makes up for the shortcomings of a single loss. 

Participants

In this study, the whole data set consists of 274 eyes scanned from 180 participants. Each eye was scanned once using a 3-mm × 3-mm and once using a 6-mm × 6-mm scan pattern. The training data set includes 140 paired 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiograms. The validation data set includes 33 paired 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP scans. The remaining 101 paired 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP scans were reserved for testing. The training data includes eyes with DR (n = 125) and healthy eyes (n = 15). The validation data set consists of 30 eyes with DR and 3 healthy eyes. The performance of this network on test data was separately evaluated on eyes with DR (n = 75) and healthy controls (n = 26). Fifteen healthy eyes from the test set were used to verify whether our algorithm generates false flow signal. In addition, to compare the performance of eight repeated en face averaging with our algorithm, we used four healthy eyes from the test set. 

Training Parameters

The training data set was expanded by several data augmentation methods, including horizontal flipping, vertical flipping, transposition, and 90-degree rotation. To reduce computation cost, 76 × 76-pixel patches were used for training. Thus, after data augmentation, the training data set included 1400 images that were further divided into 34,955 patches extracted by cropping the ICP and DCP angiograms with a stride of 38. The validation data set of 330 images was also decomposed into 9900 patches by randomly cropping an image into 30 patches. As DCARnet is a fully convolutional neural network, images with arbitrary sizes can be input into the DCARnet during testing. Thus, we input the entire image into the model for testing. We used an Adam optimizer with a learning rate 0.01 to train DCARnet. The training batch size was 128. We performed 10 epochs of training to get an optimal model. DCARnet was implemented in Python 3.6 with Keras (Tensorflow-backend) on a PC with a 32G RAM and Intel i9 CPU, as well as two NVIDIA, Santa Clara, California, USA. GeForce GTX1080Ti graphics cards. 

Evaluation of Noise Intensity, Vascular Connectivity, and Contrast

We evaluated DCARnet's performance by measuring noise intensity:  

\begin{equation}{{\rm{I}}_{{\rm{Noise}}}} = \frac{1}{{\rm{R}}} \times \mathop \sum \limits_{\left( {{\rm{i}},{\rm{j}}} \right) \in {\rm{R}}} {\rm{D}}{\left( {{\rm{i}},{\rm{j}}} \right)^2},\end{equation}

(5)

where D(i, j) is the pixel value at position (i, j) of the angiograms and R is a manually delineated circle inscribing the foveal avascular zone (FAZ). We also examined vascular connectivity.¹² The angiograms were binarized using a global adaptive threshold method, then skeletonized. Connected flow pixels were defined as any contiguous flow region with a length of at least 5 (including diagonal connections), and the vascular connectivity was defined as the ratio of the number of connected flow pixels to the total number of pixels on a vascular skeleton map. We also evaluated reconstructed angiograms using Weber contrast, a metric used to evaluate a small fraction of features (capillaries) within a uniform background (speckled background). The foreground and background pixel positions were obtained by binarized angiograms. Symbolically, Weber contrast is  

\begin{equation}{{\rm{I}}_{\rm{C}}} = \frac{{{{\rm{D}}_{\rm{f}}} - {{\rm{D}}_{\rm{b}}}}}{{{{\rm{D}}_{\rm{b}}}}},\end{equation}

(6)

where D_f is the averaged luminance of foreground image and D_b is the averaged luminance of background image. 

Results

Assessment of False Flow Signal

Due to their depth within the retina, ICP and DCP angiograms include relatively strong noise that may mislead image enhancement algorithms into generating a false flow signal. To determine whether this is an issue with DCARnet, we examined 3-mm × 3-mm high-quality ICP and DCP angiograms from 15 healthy eyes. These angiograms were denoised using simple Gabor and median filters (Fig. 4A1). We then added simulated Gaussian noise with different parameters (μ, σ) to these denoised angiograms, where μ and σ are the mean value and the standard deviation of the Gassian distribution, respectively. In all, we obtained 6000 ICP and DCP angiograms that were 3 × 3 mm (Figs. 4B1–D1) with different simulated noise intensities (0–2100; equation (5)) by changing μ and σ separately in increments of 0.005 from 0.001 to 1 and from 0.001 to 0.05. Finally, the denoised angiograms and angiograms with simulated noise were input to DCARnet (Figs. 4A2–D2). To determine if the reconstructed angiograms produced false flow, we calculated  

\begin{equation}{{\rm{I}}_{{\rm{False\;}}}} = \frac{1}{{\rm{R}}} \times \mathop \sum \nolimits_{\left( {{\rm{i}},{\rm{j}}} \right) \in {\rm{R}}} {\rm{D}}{\left( {{\rm{i}},{\rm{j}}} \right)^2}.\end{equation}

(7)

 

Figure 4.

 

False flow signal characterized with simulated noise. (A1–D1) DCP angiograms with different simulated noise intensities. (A2–D2) DCARnet output for angiograms (A1–D1). Only at the highest simulated noise intensity is false flow generated.

Because the FAZ is avascular, anatomically the value of I_False should vanish. The results show that DCARnet did not produce false flow signal when noise intensity was under 650, which is above the noise intensity measured in the original angiograms (3 × 3 mm, 427.35 ± 246.62; 6 × 6 mm, 401.63 ± 304.49; Fig. 5). 

Figure 5.

 

False flow signal and noise intensity. (A) The relationship between false flow signal and noise intensity. The blue points indicate six thousand 3-mm × 3-mm ICP and DCP angiograms with different simulated noise intensities. The red line represents a cutoff value (I_Noise = 650) under which no false flow signal was generated. (B) Boxplots of the noise intensity measured in original 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiograms from all data sets. The average noise intensities of original 3-mm × 3-mm and 6-mm × 6-mm angiograms are below 650.

Performance on DR Angiograms

Studies have shown the OCTA can detect pathology related to DR in the deep capillary plexus.⁴⁰ Microaneurysms and dilated capillaries are closely associated with DR progression.^13,41,42 Thus, a basic requirement for image reconstruction is that pathologies such as these can be preserved. Visual inspection of DCARnet's output from eyes with severe DR reveals these pathologic vascular abnormalities in both ICP (Fig. 6) and DCP (Fig. 7) angiograms. We also compared the noise intensity, connectivity, and Weber contrast of angiograms from eyes with DR and healthy eyes. As with healthy eyes, DCARnet was able to significantly reduce noise intensity in eyes with DR (Fig. 8A), the connectivity of the reconstructed angiograms of eyes with DR was similarly greatly improved (Fig. 8B), and the contrast of the reconstructed angiograms of eyes with DR also was significantly enhanced (Fig. 8C). 

Figure 6.

 

Qualitative performance on representative ICP scans. Shown are (A) a healthy eye, (B) an eye diagnosed with mild nonproliferative diabetic retinopathy (NPDR), (C) an eye diagnosed with moderate NPDR, and (D) an eye diagnosed with severe proliferative diabetic retinopathy (PDR). Compared to original, low-resolution 6-mm × 6-mm ICP angiograms (row 1), clearer capillary details and suppressed background can be perceived in DCARnet output (row 2). Dilated capillaries (green arrows) are visible in the reconstructed and original angiograms, demonstrating that DCARnet preserves this vascular pathology.

Figure 7.

 

Qualitative performance on representative DCP scans. Shown are (A) a healthy eye, (B) an eye diagnosed with mild nonproliferative diabetic retinopathy (NPDR), (C) an eye diagnosed with moderate NPDR, and (D) an eye diagnosed with severe proliferative diabetic retinopathy (PDR). Compared to original, low-resolution 6-mm × 6-mm ICP angiograms (row 1), clearer capillary details and suppressed background can be perceived in DCARnet output (row 2). Dilated capillaries (green arrows) are visible in the reconstructed and original angiograms, demonstrating that DCARnet preserves this vascular pathology.

Figure 8.

 

Quantitative evaluation in healthy eyes and eyes with DR. (A) Noise intensity, (B) connectivity, and (C) contrast in the original scans and DCARnet output. Significant improvement from the original images was achieved by all metrics, as evaluated by paired t-test with Holm–Bonferroni correction.

Comparison to Other Methods

The quality of OCT angiograms can be improved by many image-filtering methods, including Frangi¹⁹ and Gabor²⁰ filters. We applied these methods to the original 3-mm × 3-mm (Fig. 9) and 6-mm × 6-mm (Fig. 10) ICP and DCP angiograms. All these methods are based on different filters and require parameter tuning. We selected these parameters by searching for the values that yielded the best results for the performance metrics (noise intensity and connectivity) discussed above. Compared to the original angiograms, the Gabor filter significantly reduced noise intensity (Fig. 11(A) and improved connectivity (Fig. 11B) but greatly reduced the contrast in both 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiograms (Fig. 11C). The Frangi filter greatly improved connectivity (Fig. 11B) and enhanced contrast (Fig. 11C) but did not significantly decrease noise intensity in 6-mm × 6-mm ICP and DCP angiograms (Fig. 11A). The Frangi filter may also produce false flow signal. Compared to the original angiograms, DCARnet significantly reduced noise intensity, improved connectivity, and enhanced contrast in both 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiograms. Compared to other methods, the angiograms reconstructed by DCARnet showed the lowest noise intensity and best connectivity in both 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiograms. 

Figure 9.

 

The 3-mm × 3-mmmm angiogram enhancement of the ICP (top row, A1–D1) and DCP (bottom row, A2–D2) from an eye with DR using various methods. The original images (A), Gabor filtered images (B), Frangi filtered images (C), and DCARnet reconstructed images (D) are demonstrated. Compared to the original and Gabor filtered images, DCARnet produces images with visibly lower background levels. While the Frangi filtered images also achieve high contrast, note the artifactual signal generated in the FAZ, which is absent in DCARnet's output.

Figure 10.

 

The 6-mm × 6-mm angiogram enhancement of the ICP (top row, A1–D1) and DCP (bottom row, A2–D2) from an eye with DR using various methods. The original images (A), Gabor filtered images (B), Frangi filtered images (C), and DCARnet reconstructed images (D) are demonstrated. Vascular detail is visibly most clear in the DCARnet output.

Figure 11.

 

Quantitative evaluation of different methods. (A) Noise intensity, (B) connectivity, and (C) contrast compared between original scans and scans processed by different methods. A significant difference between the original and reconstructed images was achieved by all metrics, evaluated by paired t-test with Holm–Bonferroni correction. DCARnet achieves superior performance in noise intensity and connectivity.

Comparison to Multiple En Face Image Averaging

Multiple B-scan averaging is widely used to enhance the image quality of OCT angiograms. This technique also can reduce noise and improve the continuity of blood vessel segments or fragments.^17,43 In addition to the filters discussed above, we also compared the performance of the averaged en face images with that of the original images reconstructed by DCARnet. 

We evaluated the performance of 8-scan averaged en face images and the images reconstructed by DCARnet. SVC angiograms (Figs. 12A1–A8) were first registered, and the registration information of SVC was transferred to ICP or DCP angiograms (Figs. 12B1–B8). These registered angiograms were then averaged to generate the 8-scan averaged en face images (Fig. 12B). As a comparison, each of the original angiograms was reconstructed by DCARnet (Figs. 12C1–C8). We found 8-scan averaging was able to reduce noise intensity and increase connectivity but reduced contrast, whereas the results from our algorithm show low noise intensity, good connectivity, and strong contrast. More important, DCARnet just needs a single image to reconstruct a high-definition image, which dramatically reduces the time constraint on data collection. 

Figure 12.

 

Comparison of 8-scan averaged en face image and angiograms reconstructed by DCARnet. (A1–A8) The 3-mm × 3-mm angiogram of the SVC, (B1–B8) DCP, and (C1–C8) reconstructed DCP from a healthy eye. Scans were registered using the SVC to produce 8-scan averaged composite images. (A) The 8-scan averaged SVC en face images. (B) The 8-scan averaged DCP en face images. DCARnet's output does not include the blurring effect generated by the averaging approach.

Performance on Scans from Different Devices

We also tested the robustness and generalization of our algorithm on 38 independent scans from a different device. We input 6-mm × 6-mm ICP and DCP angiograms from a commercial 120-kHz spectral domain OCT system (Solix; Optovue, Inc.) with a 15-µm lateral resolution and a 5-µm axial resolution to DCARnet (Fig. 13). The reconstructed scans from the Solix device by DCARnet also showed low noise intensity, good connectivity, and strong Weber contrast. 

Figure 13.

 

The output of DCARnet on 6-mm × 6-mm ICP and DCP angiograms from a Solix device. (A) Original 6-mm × 6-mm ICP angiogram. (B) Reconstructed 6-mm × 6-mm ICP angiogram. (C, D) Equivalent image from a 6-mm × 6-mm DCP angiogram. Vascular detail is more apparent in images reconstructed from data from the Solix instrument.

Discussion

Assessment of plexus-specific pathology in the ICP and DCP using projection-resolved OCTA is helpful in assessing DR^44,45 and other retinal vascular diseases.⁴ However, OCTA data from the ICP and DCP are especially susceptible to image artifacts due to the attenuated flow signal.^46–48 Strong background noise and vessel fragments are common in ICP and DCP angiograms, affecting the quantitative and qualitative assessment of OCTA. Lower sampling densities used in wider-field-of-view OCTA further exacerbate these problems. Researchers have proposed many methods to enhance image quality. Traditional approaches with filters are less effective for wide-field-of-view OCTA with lower sampling density,⁴⁹ and averaging multiple en face images requires long acquisition time, increasing the probability of image artifacts caused by eye movements. However, the lack of accessible algorithms to enhance ICP and DCP OCTA en face images renders any possible advantage to be gained from high-resolution OCT angiograms inaccessible; therefore, we provide an open-source platform (https://github.com/octangio/DCARnet) that we hope will be of use to other researchers that may find such enhancement. This study shows that a deep learning–based image enhancement approach, whether from eyes with DR or healthy eyes, achieves lower noise intensity, better connectivity, and stronger contrast than these alternative approaches without extending scan acquisition time. 

Researchers have used very deep and complicated network structures to improve performance in natural image super-resolution reconstruction.^39,50,51 However, with increased depth, networks can encounter more problems during training, such as gradient explosion/vanishing and overfitting.^52,53 Furthermore, as the number of network layers increases, the features in the input image may be lost. Our proposed solution, DCARnet, is a simple, efficient, and easy-to-train network that aims to learn an end-to-end mapping function between the undersampled 6-mm × 6-mm angiograms and 3-mm × 3-mmmm angiograms with proper sampling density. Feature maps at the original resolution extracted more detailed information, such as vascular morphology. Down-sampled feature maps extracted enhanced the network's denoising ability. DCARnet fused multiscale features to enhance representational ability, preserve details, and improve tolerance to artifacts, low signal quality, and other disturbances. We used the validation data set to select the best model trained by the training data set. Then a test data set, which is completely independent of the training and validation data sets, is used to evaluate our algorithm. Even though DCARnet is not a very deep and complicated network, DCARnet produced good performance. 

A significant concern in image reconstruction is the introduction of structure produced from noise that may mimic a true vascular signal. This is called a false signal and should be considered in image reconstruction projects. As noted, this is an issue for handcrafted algorithms such as a Frangi filter.¹⁹ The angiograms reconstructed by DCARnet did not produce false flow signal in angiograms with normal noise intensities. However, for those angiograms with noise intensity higher than 650, which is above the average noise intensity of this data set, DCARnet may produce some artifacts. As the performance of OCT systems improves, the signal-to-noise ratios in OCTA images will grow. This issue will therefore be less of a concern in future OCTA devices. 

DCARnet was designed to remove background noise and reconstruct high-resolution capillary details. Our algorithm therefore does not suppress other artifacts due to projection or motion. For this reason, DCARnet is best used in conjunction with other artifact removal algorithms such as the reflectance-based algorithm for projection-resolved OCTA⁷ and the regression-based algorithm for bulk motion subtraction in OCTA.⁸ DCARnet also does not compensate for shadow artifacts to recover capillaries in the regions severely affected by shadows. If no capillary signal is detected by OCTA, DCARnet has no way to recover it. We also tested and demonstrated the strong generalization of this network on independent scans with similar sampling density acquired by a different device than the one used to acquire our training data. 

We previously reported on High-resolution Angiogram Reconstruction Network (HARNet), which enhances SVC image resolution, and we now propose DCARnet to enhance ICP and DCP angiograms. Although a single, unified network theoretically could provide denoising simultaneously in the SVC, ICP, and DCP, anatomic differences between the layers make this difficult. Compared to SVC angiograms, ICP and DCP angiograms have denser capillaries and stronger background noise. If SVC, ICP, and DCP angiograms are trained together, the features specific to ICP and DCP angiograms may be introduced into SVC angiograms and vice versa. In addition, smaller blood vessels in the SVC may be lost due to excessive denoising. Conversely, noise will be misjudged as capillaries in ICP and DCP angiograms, because the noise intensity in the ICP and DCP is much higher than that in the SVC. 

Conclusion

In summary, we developed a high-resolution reconstruction network to enhance ICP and DCP angiograms. DCARnet significantly reduced noise intensity and improved connectivity in 3-mm × 3-mm and 6-mm × 6-mm ICP and DCP angiograms without producing false flow signal. The enhanced 3-mm × 3-mm or 6-mm × 6-mm angiograms may improve qualitative and quantitative analysis of these angiograms. 

Acknowledgments

Supported by grants from the National Institutes of Health (R01 EY027833, R01 EY024544, R01 EY031394, P30 EY010572, T32 EY023211), an unrestricted departmental funding grant, and the William & Mary Greve Special Scholar Award from the Research to Prevent Blindness (New York, NY) and the Bright Focus Foundation (G2020168). 

Disclosure: M. Gao, None; T.T. Hormel, None; J. Wang, None; Y. Guo, None; S.T. Bailey, None; T. S. Hwang, None; Y. Jia, Optovue (F, P), Optos (P)