Automatic Segmentation in Multiple OCT Layers For Stargardt Disease Characterization Via Deep Learning

Abstract

Purpose: This study sought to perform automated segmentation of 11 retinal layers and Stargardt-associated features on spectral-domain optical coherence tomography (SD-OCT) images and to analyze differences between normal eyes and eyes diagnosed with Stargardt disease.

Methods: Automated segmentation was accomplished through application of the deep learning–shortest path (DL-SP) framework, a shortest path segmentation approach that is enhanced by a deep learning fully convolutional neural network. To compare normal eyes and eyes diagnosed with Stargardt disease, various retinal layer thickness and intensity feature maps associated with the outer retinal layers were generated.

Results: The automated DL-SP approach achieved a mean difference within a subpixel accuracy range for all layers when compared to manually traced layers by expert graders. The algorithm achieved mean and absolute mean differences in border positions for Stargardt features of −0.11 ± 4.17 pixels and 1.92 ± 3.71 pixels, respectively. In several of the feature maps generated, the characteristic Stargardt features of flecks and atrophic-appearing lesions were readily visualized.

Conclusions: To the best of our knowledge, this is the first automated algorithm for 11 retinal layer segmentation on OCT in eyes with Stargardt disease, and, furthermore, the feature differences found between eyes diagnosed with Stargardt disease and normal eyes may inform new insights and the better understanding of retinal characteristic morphologic changes caused by Stargardt disease.

Translational Relevance: The automated algorithm's performance and the feature differences found using the algorithm's segmentation support the future applications of SD-OCT for the quantitative monitoring of Stargardt disease.

Introduction

Stargardt disease is an inherited disorder of the retina and the most prevalent form of juvenile-onset macular dystrophy, causing progressive damage or degeneration of the macula.^1–8 Fundus autofluorescence is a broadly accessible imaging modality that can lend to the diagnosis and monitoring of Stargardt disease. Fundus autofluorescence imaging provides an in vivo assay of the lipofuscin content within the retinal pigment epithelium (RPE) cells but is only an indirect measure of the anatomical status of the photoreceptors themselves.⁹ In contrast, spectral-domain optical coherence tomography (SD-OCT) can produce a three-dimensional visualization of the microstructure of the retina, allowing for the direct and individual assessment of the status of the individual retinal layers, including the photoreceptors and RPE.^10,11 

Stargardt disease on SD-OCT imaging may show various levels of disruption of the photoreceptor inner and outer segments, RPE, as well as other outer retinal layers. The quantitative measurements associated with the OCT retinal layer changes or damage due to the Stargardt disease may facilitate a better understanding of its pathogenesis and provide endpoints for novel clinical trials for the early prevention and treatment of this disease. At a Retinal Disease Endpoints meeting with the Food and Drug Administration in November of 2016, the integrity of the ellipsoid zone (EZ) was proposed as a reliable measure of the anatomic status of the photoreceptors and a suitable regulatory endpoint for therapeutic intervention clinical trials.¹² 

However, manual segmentation of the features and affected layers of Stargardt disease in volumetric SD-OCT images is an extremely laborious and impractical process for studies with large datasets. Approaches to automated segmentation in SD-OCT have included the use of shortest path frameworks,^13,14 active contour modeling,¹⁵ graph search frameworks,^16–19 and deep learning methods.^20–24 Among these approaches, graph-based methods have generally outperformed the others. However, graph-search frameworks tend to be slow, and shortest-path frameworks lack interaction constraints. The algorithm discussed in this article combines the shortest path framework and deep learning methods in the form of a fully convolutional neural network (i.e., the U-Net^20–22) to enhance performance. In a recent study, we developed a graph-based algorithm for the automated segmentation of 11 retinal layers on OCT in eyes with age-related macular degeneration (AMD).²⁵ Here, we have sought to automatically segment the atrophic-appearing lesions and fleck features of Stargardt disease along with 11 retinal surfaces in SD-OCT images through the refinement of the deep learning–shortest path framework used previously. Previous studies into the automated segmentation of eyes diagnosed with Stargardt disease have segmented only the inner limiting membrane and retinal pigment epithelium layers.²⁶ To the best of our knowledge, this is the first automated algorithm for 11 retinal layer segmentation on OCT in eyes with Stargardt disease. Using this segmentation, we further generate and analyze various retinal layer thickness and intensity feature maps associated with the outer retinal layers and compare them between normal eyes and eyes diagnosed with Stargardt disease. 

Methods

Imaging Dataset

One hundred ninety-seven eyes diagnosed with Stargardt disease were identified from the ProgStar study. Twenty healthy eyes were identified from patients at the Doheny-University of California Los Angeles Eye Centers. Ninety-six of the eyes diagnosed with Stargardt disease were used to train neural networks to identify relevant layers to Stargardt disease, whereas 101 eyes were used for testing. This corresponded to 2000 and 4949 images (B-scans) for training and testing, respectively. All of the healthy eyes, 740 images (B-scans), were used to train neural networks to identify the remaining layers to be segmented. The image scan dimensions are 496 (depth) × 1024 (A scans) × 49 (B-scans) pixels or 496 (depth) × 512 (A scans) × 49 (B-scans) pixels for eyes diagnosed with Stargardt disease and 496 (depth) × 1024 (A scans) × 37 (B-scans) pixels for healthy eyes. All images were resized to a standard width of 1024 and standard height of 496, and all the right (OD) eye scans were flipped horizontally to provide consistency in the analysis. All data used were deidentified according to the Health and Insurance Portability and Accountability Act Safe Harbor, and informed consent was obtained from subjects prior to analysis. 

Deep Learning–Shortest Path

Retinal segmentation of the eyes diagnosed with Stargardt disease was conducted using the deep learning–shortest path (DL-SP) algorithm described in our group's previous work.²⁵ The DL-SP algorithm is a shortest path segmentation approach that is guided by the image gradient in the z-direction and probability maps generated by a deep learning fully convolutional neural network based on the U-Net architecture. The algorithm is trained and performed on two-dimensional OCT B-scan images rather than on three-dimensional volumes, allowing for higher computational efficiency. The DL-SP algorithm can be applied to segment both retinal layers and characteristic features of diseases, such as the lipofuscin deposits of Stargardt disease. 

For this study, the algorithm was modified to have two separate branches depending on the severity of degeneration, measured by thickness between the inner-outer segment (IS-OS) junction (i.e., the EZ) and outer retinal pigment epithelium (RPE). A rough estimate of this thickness was obtained by binarizing the probability maps for the layers of interest and averaging across the columns of the B-scan. A cutoff point of 10 pixels was chosen between mild and severe degeneration. For mild degeneration, the layer segmentation was performed in the following sequence: internal limiting membrane (ILM), IS-OS, outer RPE, nerve fiber-ganglion cell (NF-GC) junction, inner RPE, Stargardt features, outer plexiform-outer nuclear (OP-ON) junction, inner plexiform-inner nuclear (IP-IN) junction, inner nuclear-outer plexiform (IN-OP) junction, external limiting membrane (ELM), ganglion cell-inner plexiform (GC-IP) junction, and choroid-sclera (C-S) junction. For severe degeneration, the layer segmentation was performed in the following sequence: ILM, IS-OS junction, outer RPE, NF-GC junction, inner RPE, Stargardt features, IP-IN junction, IN-OP junction, OP-ON junction, ELM, GC-IP junction, and C-S junction. Additionally, for severe degeneration, the outer RPE, NF-GC junction, IP-IN junction and GC-IP junction were segmented using only the gradient, as the probability maps were found to be error-prone when degeneration was severe. Figure 1 shows a graphical summary of the method. 

Figure 1.

 

Summary of the modified Deep Learning-Shortest Path approach for retinal layer segmentation of eyes diagnosed with Stargardt disease. The orange box represents input, blue have been excluded from this overview; however, there was no overlap between training and testing data.

The retinal layer segmentation permits the visualization and comparison of the layer features in healthy eyes and eyes diagnosed with Stargardt disease. As a preliminary investigation in this article, we first look at various feature maps of pixels lying in between the IS-OS junction and inner RPE of each B-scan (the EZ band). For each column of each B-scan, the minimum intensity, median intensity, maximum intensity, mean intensity, standard deviation, skewness, excess kurtosis, gray level entropy, and thickness were evaluated for the pixels of interest. 

More specifically, the skewness reflects if the intensity distribution of a column is symmetric and is defined for a sample as  

\begin{equation}\begin{array}{@{}l@{}} \begin{array}{@{}*{1}{l}@{}} {Sample\;{\rm{ }}Skewness = \displaystyle\frac{{\sqrt {n\left( {n - 1} \right)} }}{{n - 2}}\frac{{{m_3}}}{{{{\sqrt {{m_2}} }^3}}}}\\ {{\rm{where}}}\\ \displaystyle{{m_3} = \frac{{{{\sum {\left( {x - \overline x } \right)} }^3}}}{n}\;{\rm{and}}\;{m_2} = \frac{{{{\sum {\left( {x - \overline x } \right)} }^2}}}{n}} \end{array} \end{array}\end{equation}

(1)

where n is the number of pixels in a column and \(\bar x\) is the mean.²⁷ 

The excess kurtosis reflects the intensity distribution pattern (e.g., normal distribution) of a column and is defined for a sample as  

\begin{eqnarray} &&Sample\;{\rm{ }}Excess{\rm{ }}\;Kurtosis = \frac{{n - 1}}{{\left( {n - 2} \right)\left( {n - 3} \right)}}\nonumber\\ &&\quad \times\left[ {\left( {n + 1} \right)\left( {\frac{{{m_4}}}{{{m_2}^2}} - 3} \right) + 6} \right]\nonumber\\ &&{{\rm{where}}}\\ &&{m_4} = \frac{{{{\sum {\left( {x - \overline x } \right)} }^4}}} {n}\;{\rm{and}}\; {m_2} = \frac{{{{\sum {\left( {x - \overline x } \right)} }^2}}} {n}\nonumber \end{eqnarray}

(2)

n is the number of pixels in a column, and \(\bar x\) is the mean.²⁷ 

The gray level entropy is a measure of the amount of the intensity levels of a column and is defined as  

\begin{equation}Entropy = - \sum\limits_{k = 0}^{M - 1} {{p_k}{{\log }_2}\left( {{p_k}} \right)} \end{equation}

(3)

where M is the number of gray levels in a column and p_k is the probability associated with gray level k.^28–30 

This resulted in feature maps of size 37 × 1024 for normal eyes and of size 49 × 1024 for eyes diagnosed with Stargardt disease. Feature maps were resized to common size of 512 × 512 for ease of viewing and comparison. 

Results

The automated layer segmentation was evaluated through comparison against manually traced layers completed by certified OCT graders from the Doheny Image Reading Center who were masked to the automated segmentation outcome. The reading center medical director reviewed and confirmed the manual tracing for each case. The automated layer segmentations were not refined before comparison with the manually traced layers. The mean and absolute mean differences in border position between the automated layer segmentation and the manually traced layers were calculated to evaluate the automated segmentation accuracy. To achieve a high-level of precision, even single pixel differences were treated as discrepancies. For the mean differences in border position, a negative value indicates that the segmented layer was positioned above the corresponding manually traced layer, and a positive value indicates that the segmented layer was positioned below. 

The Table shows the mean and absolute mean border position differences of six layers: ILM, IN-OP, ELM, IS-OS, inner RPE, and outer RPE in OCT with Stargardt disease from the automated segmentation. It also shows the mean and absolute mean border position differences for Stargardt features. Because we do not have ground truth for the remaining five layers (NF-GC, GC-IP, IP-IN, OP-ON, and C-S) in our OCT data with Stargardt disease, their border position differences against manual ground truth cannot be evaluated. However, we have achieved segmentation to subvoxel accuracy for these five layers by our group previously with AMD eyes with similar layer damage features. We expect the DL-SP algorithm will produce segmentation with comparable accuracy. In Figure 2, an example of segmentation of a B-scan of a normal eye is shown compared to examples of segmentation of B-scans of eyes diagnosed with Stargardt disease. The segmented 11 layers and segmented Stargardt features are shown overlaid on SD-OCT images. 

Table.

 

Mean and Absolute Mean Border Position Differences of the Automated and Manual Segmentations

 

Figure 2.

 

Segmentation of normal eyes and eyes diagnosed with Stargardt disease overlaid SD-OCT B-scans. (a) and (b) are the original B-scan and segmentation of a normal eye. (c) and (d) are the original B-scan and segmentation of an eye diagnosed with Stargardt disease showcasing mild degeneration and deposits corresponding to the characteristic flecks of Stargardt disease. (e) and (f) are the original B-scan and segmentation of an eye diagnosed with Stargardt disease showcasing severe degeneration and an atrophic-appearing lesion. The layers in order from top to bottom are the ILM, NF-GC, GC-IP, IP-IN, IN-OP, OP-ON, ELM, IS-OS, Stargardt features, inner RPE, outer RPE, and C-S.

In Figure 3, feature maps and a representative B-scan for a normal eye are shown. In Figure 4, feature maps and a representative B-scan for an eye diagnosed with Stargardt disease at an intermediate stage of degeneration are shown. In Figure 5, feature maps and a representative B-scan for an eye diagnosed with Stargardt disease at a later stage are shown. The feature maps included are minimum intensity, median intensity, maximum intensity, mean intensity, standard deviation, skewness, kurtosis, gray level entropy, and thickness (i.e., the number of pixels between the IS-OS junction and inner RPE). 

Figure 3.

 

Feature maps of the outer retinal layers for a normal eye. In order from left to right, top to bottom, the feature maps are minimum intensity, median intensity, maximum intensity, mean intensity, standard deviation, thickness, skewness, excess kurtosis, and gray level entropy. Below the feature maps is a representative B-scan corresponding to the location of the gray lines drawn on the feature maps. Segmentation for the IS-OS junction (red) and inner RPE (green) are shown to delineate the layers analyzed in the feature maps.

Figure 4.

 

Feature maps of the outer retinal layers for an eye diagnosed with Stargardt disease at an intermediate stage of degeneration. In order from left to right, top to bottom, the feature maps are minimum intensity, median intensity, maximum intensity, mean intensity, standard deviation, thickness, skewness, excess kurtosis, and gray level entropy. Below the feature maps is a representative B-scan corresponding to the location of the gray lines drawn on the feature maps. Segmentation for the IS-OS junction (red) and inner RPE (green) are shown to delineate the layers analyzed in the feature maps. The hyperreflective deposits due to Stargardt are clearly reflected in the minimum intensity, median intensity, maximum intensity, mean intensity, standard deviation, thickness, and gray level entropy.

Figure 5.

 

Feature maps of the outer retinal layers for an eye diagnosed with Stargardt disease at a later stage of degeneration. In order from left to right, top to bottom, the feature maps are minimum intensity, median intensity, maximum intensity, mean intensity, standard deviation, thickness, skewness, excess kurtosis, and gray level entropy. Below the feature maps is a representative B-scan corresponding to the location of the gray lines drawn on the feature maps. Segmentation for the IS-OS junction (red) and inner RPE (green) are shown to delineate the layers analyzed in the feature maps. The atrophic region due to Stargardt are clearly reflected in the median intensity, maximum intensity, mean intensity, standard deviation, thickness, skewness, excess kurtosis, and gray level entropy.

Discussion

The automated DL-SP approach achieved a mean difference within a subpixel accuracy range for all layers when compared to manually traced layers by expert graders. In particular, the algorithm achieved mean and absolute mean differences in border positions for Stargardt features of –0.11 ± 4.17 pixels and 1.92 ± 3.71 pixels, respectively. In Figure 2, the successful segmentation of both atrophic-appearing lesions and of the flecks is shown, where the flecks appear as hyperreflective deposits along the RPE. 

When comparing the feature maps generated from the automated segmentation to those of normal eyes (Fig. 3), there are several distinguishing features that can be observed in eyes diagnosed with Stargardt disease. In the eyes with intermediate Stargardt stage, in seven (i.e., minimum intensity, median intensity, maximum intensity, mean intensity, standard deviation, thickness, and gray level entropy) of the nine feature maps, distributed dotted regions with differing values can usually be observed. Figure 4 is an illustration. Such dotted regions in the feature maps correspond to the hyperreflective deposits as reflected between the OCT IS-OS and RPE layer due to Stargardt resulted RPE disorder. With the progression of Stargardt disease to advanced stage, the RPE cell may die and retinal atrophy may appear. In many of the feature maps with atrophy, a circular region with differing values can be observed. Figure 5 presents a Stargardt atrophy case. Among the eight (median intensity, maximum intensity, mean intensity, standard deviation, thickness, skewness, excess kurtosis, and gray level entropy) of the nine feature maps, the circular regions are clearly seen. The circular regions correspond to where the IS-OS junction has been completely disrupted and disappeared in the SD-OCT scans. For example, when looking at maximum intensity projection of the outer retinal layers, a clear circular region of increased maximum intensity is observed near the fovea. This pattern can also be observed when looking at the median intensity and mean intensity feature maps. In comparison, the feature maps for normal eyes show no clear pattern or even a decrease in intensity toward the fovea center. When inspecting the minimum intensity feature maps, there is commonly a lower minimum intensity at the fovea center for normal eyes, although there is normally no such pattern for eyes diagnosed with Stargardt disease where the fovea is disrupted. There is a region of increased standard deviation around the fovea center compared to the normal eyes that is more clearly distinguished in the later stages of Stargardt disease. The skewness feature map shows a matching region of reduced skewness, and the same pattern is seen for excess kurtosis, whereas in normal eyes there is no clear pattern. 

In the remaining features, a cluster or ring pattern appears for eyes diagnosed with Stargardt disease. For the ellipsoid zone thickness, there is a cluster of small areas with increased thickness at the intermediate stage of degeneration. At the later stage of degeneration, there is a ring of increased thickness surrounding a region of somewhat lower values. These areas of increased thickness correspond to the characteristic flecks, or lipofuscin deposits, present in eyes with Stargardt disease. The interior of the ring patterns corresponds to atrophic-appearing lesions in the SD-OCT scans. A similar pattern can be seen in the gray level entropy feature map. In contrast, there is no such cluster or ring pattern present in the feature maps for normal eyes. Such feature differences may provide new insights for the understanding of the retinal characteristic morphological changes due to Stargardt disease. 

Our study is not without limitations. First, although the mean values of the border position differences are under sub-pixel for all the evaluated layers, the absolute values for some layers are not ideal. For instance, the absolute border position differences for IN-OP is 4.73 ± 6.00 pixels, which indicates our segmentation is substantially fluctuates around the manual ground truth and thus our algorithm needs further improvement. Second, because of the relative paucity of Stargardt disease data, it is difficult to capture the full diversity of manifestations of the disease to train a deep learning model, and even human graders struggle in making grading determinations related to the correct segmentations. Hence, the manual ground truth may even need to be improved with the further understanding of the retinal layer features associated with Stargardt disease. Despite these limitations, our study has many strengths as noted above, demonstrating for the first time, an automated algorithm for 11 OCT retinal layer segmentation algorithm for eyes with Stargardt disease, as well as characterization of other Stargardt-associated features. 

Conclusion

In this study, an automated approach was successfully applied to segment retinal layers on SD-OCT in eyes with Stargardt disease. The characteristic Stargardt features of atrophic-appearing lesions and flecks were segmented alongside 11 retinal layers. This method achieves a mean difference that is within the subpixel accuracy range for Stargardt features alongside associated retinal layers. For Stargardt features, a mean and absolute mean difference in border position of −0.11 ± 4.17 pixels and 1.92 ± 3.71 pixels were achieved, respectively. To the best of our knowledge, this is the first automated algorithm for 11 retinal layer segmentation on OCT in eyes with Stargardt disease, extending beyond previous work that only examined segmentation of the ILM and RPE layers.²⁶ The performance of this automated segmentation lends to the possibility of using SD-OCT in monitoring and quantifying the progression of Stargardt disease. Clinically, this method would circumvent the laborious manual segmentation process of the OCT retinal layers, allowing for efficient retinal layer segmentation in reduced effort and time. Furthermore, from this segmentation, various thickness and intensity feature maps characterizing the outer retinal layers were generated, allowing for visualization of the feature differences between eyes diagnosed Stargardt disease and normal eyes. Both the characteristic flecks and lesions were readily visualized across several of the feature maps considered, showcasing clear patterns or differences not present in feature maps of normal eyes. Such feature differences may provide new insights for the understanding of the characteristic morphological changes of the retina due to Stargardt disease. The automated segmentation of the retinal layers and the derived feature maps further enables predictive modeling of the changes in retinal layers over time because of Stargardt disease that can aid in prognosis. 

Acknowledgments

Supported by the National Eye Institute of the National Institutes of Health under award number R21EY029839. 

Disclosure: Z. Mishra, None; Z. Wang, None; S.R. Sadda, None; Z. Hu, None