March 2016
Volume 5, Issue 2
Open Access
Articles  |   April 2016
Automated Segmentability Index for Layer Segmentation of Macular SD-OCT Images
Author Affiliations & Notes
  • Kyungmoo Lee
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, USA
    Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA, USA
  • Gabriëlle H.S. Buitendijk
    Department of Ophthalmology, Erasmus Medical Center, Rotterdam, the Netherlands
    Department of Epidemiology, Erasmus Medical Center, Rotterdam, the Netherlands
  • Hrvoje Bogunovic
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, USA
    Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA, USA
  • Henriët Springelkamp
    Department of Ophthalmology, Erasmus Medical Center, Rotterdam, the Netherlands
    Department of Epidemiology, Erasmus Medical Center, Rotterdam, the Netherlands
  • Albert Hofman
    Department of Epidemiology, Erasmus Medical Center, Rotterdam, the Netherlands
    Netherlands Consortium for Healthy Aging, Netherlands Genomics Initiative, the Hague, the Netherlands
  • Andreas Wahle
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, USA
    Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA, USA
  • Milan Sonka
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, USA
    Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA, USA
    Department of Ophthalmology and Visual Sciences, University of Iowa Hospitals and Clinics, Iowa City, IA, USA
  • Johannes R. Vingerling
    Department of Ophthalmology, Erasmus Medical Center, Rotterdam, the Netherlands
    Department of Epidemiology, Erasmus Medical Center, Rotterdam, the Netherlands
  • Caroline C.W. Klaver
    Department of Ophthalmology, Erasmus Medical Center, Rotterdam, the Netherlands
    Department of Epidemiology, Erasmus Medical Center, Rotterdam, the Netherlands
  • Michael D. Abràmoff
    Department of Electrical and Computer Engineering, University of Iowa, Iowa City, IA, USA
    Iowa Institute for Biomedical Imaging, University of Iowa, Iowa City, IA, USA
    Department of Ophthalmology and Visual Sciences, University of Iowa Hospitals and Clinics, Iowa City, IA, USA
    Department of Biomedical Engineering, University of Iowa, Iowa City, IA, USA
    Stephen A. Wynn Institute for Vision Research, University of Iowa, Iowa City, IA, USA
    Department of Veterans Affairs, Iowa City VA Medical Center, Iowa City, IA, USA
  • Correspondence: Michael D. Abràmoff, 11205 PFP, Department of Ophthalmology and Visual Sciences, University of Iowa Hospitals and Clinics, 200 Hawkins Drive, Iowa City, IA 52242, USA; e-mail: michael-abramoff@uiowa.edu 
Translational Vision Science & Technology April 2016, Vol.5, 14. doi:10.1167/tvst.5.2.14
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to Subscribers Only
      Sign In or Create an Account ×
    • Get Citation

      Kyungmoo Lee, Gabriëlle H.S. Buitendijk, Hrvoje Bogunovic, Henriët Springelkamp, Albert Hofman, Andreas Wahle, Milan Sonka, Johannes R. Vingerling, Caroline C.W. Klaver, Michael D. Abràmoff; Automated Segmentability Index for Layer Segmentation of Macular SD-OCT Images. Trans. Vis. Sci. Tech. 2016;5(2):14. doi: 10.1167/tvst.5.2.14.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose: To automatically identify which spectral-domain optical coherence tomography (SD-OCT) scans will provide reliable automated layer segmentations for more accurate layer thickness analyses in population studies.

Methods: Six hundred ninety macular SD-OCT image volumes (6.0 × 6.0 × 2.3 mm3) were obtained from one eyes of 690 subjects (74.6 ± 9.7 [mean ± SD] years, 37.8% of males) randomly selected from the population-based Rotterdam Study. The dataset consisted of 420 OCT volumes with successful automated retinal nerve fiber layer (RNFL) segmentations obtained from our previously reported graph-based segmentation method and 270 volumes with failed segmentations. To evaluate the reliability of the layer segmentations, we have developed a new metric, segmentability index SI, which is obtained from a random forest regressor based on 12 features using OCT voxel intensities, edge-based costs, and on-surface costs. The SI was compared with well-known quality indices, quality index (QI), and maximum tissue contrast index (mTCI), using receiver operating characteristic (ROC) analysis.

Results: The 95% confidence interval (CI) and the area under the curve (AUC) for the QI are 0.621 to 0.805 with AUC 0.713, for the mTCI 0.673 to 0.838 with AUC 0.756, and for the SI 0.784 to 0.920 with AUC 0.852. The SI AUC is significantly larger than either the QI or mTCI AUC (P < 0.01).

Conclusions: The segmentability index SI is well suited to identify SD-OCT scans for which successful automated intraretinal layer segmentations can be expected.

Translational Relevance: Interpreting the quantification of SD-OCT images requires the underlying segmentation to be reliable, but standard SD-OCT quality metrics do not predict which segmentations are reliable and which are not. The segmentability index SI presented in this study does allow reliable segmentations to be identified, which is important for more accurate layer thickness analyses in research and population studies.

Introduction
Optical coherence tomography (OCT) is a noninvasive, noncontact imaging modality designed to provide cross-sectional images of the retinal structure with high axial resolution.1 Spectral-domain OCT (SD-OCT) with faster scanning speed and higher spatial resolution has become essential clinical tools to diagnose and monitor retinal diseases.2,3 Reliable quantification of the retinal structures such as retinal nerve fiber layer (RNFL), ganglion cell layer (GCL), outer segment, and choroidal thickness is crucial.49 
OCT scans with insufficient image quality, either because of media opacities or aberrations, or because of improper patient alignment, negatively affect intraretinal layer segmentation and cause unreliable measurements.10,11 A metric that quantifies this quality would ideally be a continuous variable because different uses of the retinal layer segmentations may require different levels of reliability; thus, our preference for a continuous variable that can be thresholded. For example, population studies where the segmentation results may be used without human review of the segmentation quality may require a different threshold than segmentation in a clinical context where the segmentation is also reviewed by the clinician. Figure 1 shows examples of successful and failed intraretinal layer segmentations in macular SD-OCT volumes with good, moderate, and bad image quality. Figure 2 shows that including all layer analyses (420 successful RNFL segmentations as well as 270 failed segmentations) obtained from a population study into average RNFL thickness yields suboptimal results. The standard deviation (SD) RNFL thickness map has large regional variations because of the failed segmentations caused by the image quality issues (Figs. 1C, 1F). 
Figure 1
 
Automated intraretinal layer segmentations of macular SD-OCT volumes with good (A), moderate (B), and bad (C) image quality and corresponding segmentability index SI, which is explained in Methods. (D–F) B-scans in (A–C) overlaid with automatically detected layers (red line: ILM, orange line: boundary between RNFL and GCL, green line: outer boundary of retinal pigment epithelium [RPE]).
Figure 1
 
Automated intraretinal layer segmentations of macular SD-OCT volumes with good (A), moderate (B), and bad (C) image quality and corresponding segmentability index SI, which is explained in Methods. (D–F) B-scans in (A–C) overlaid with automatically detected layers (red line: ILM, orange line: boundary between RNFL and GCL, green line: outer boundary of retinal pigment epithelium [RPE]).
Figure 2
 
Mean and SD RNFL thickness maps of 690 macular OCT volumes including 420 volumes with successful RNFL segmentations and 270 volumes with failed segmentations. The number at the bottom of the thickness map is mean ± SD thickness of the RNFL. The thickness maps were created after horizontally flipping left eye OCT volumes and aligning the OCT volumes based on the fovea.
Figure 2
 
Mean and SD RNFL thickness maps of 690 macular OCT volumes including 420 volumes with successful RNFL segmentations and 270 volumes with failed segmentations. The number at the bottom of the thickness map is mean ± SD thickness of the RNFL. The thickness maps were created after horizontally flipping left eye OCT volumes and aligning the OCT volumes based on the fovea.
Several OCT manufacturers supply signal quality metrics, and at least two researchers have proposed alternative signal metrics.12,13 Stein et al.12 have introduced the quality index (QI), based on the image histogram information, which is expressed as  where Low, Noise, Saturation are the pixel intensity values corresponding to 1%, 75%, 99% of all pixels, and Middle is the mean value of Noise and Saturation pixel intensities.12 Higher QI values represent better OCT image quality. In this paper, an entire OCT volume consisting of multiple B-scans was treated as a single image to obtain a QI value.  
Huang et al.13 has proposed the maximum tissue contrast index (mTCI), based on the intensity histogram decomposition model, which is calculated as  where N1, N2, and N3 denote the voxel intensity values of the mode point (the highest peak in the voxel intensity histogram), the separation point (99% of the accumulative density of the background voxels) between the vitreous having low reflectance and the foreground corresponding to various retinal tissues having higher reflectance, and the saturation point (99.9% of all voxels), respectively. Higher mTCI values represent better image quality, and the lowest value is 1.  
While some prior studies showed a positive relationship between OCT manufacturers' image quality indices and RNFL thickness measurements,1416 other studies failed to show any such relationship.17,18 Because the QI, mTCI, and similar quality metrics correspond to visual qualitative assessment of image quality, they necessarily identify those OCT scans that are expected to lead to correct intraretinal layer segmentations. Therefore, we have developed a new segmentability index SI that quantifies the expected RNFL segmentability of the OCT scans. 
The purpose of the present study is to introduce the segmentability index and verify that it can be used to differentiate SD-OCT scans that will result in reliable RNFL segmentations. 
Methods
Human Subjects and Data Acquisition
The Rotterdam Study is a prospective population-based cohort study organized in the city of Rotterdam in the Netherlands and ongoing since 1990.19 The Rotterdam Study has been approved by the Medical Ethics Committee of the Erasmus MC and by the Ministry of Health, Welfare, and Sport of the Netherlands, implementing the “Wet Bevolkingsonderzoek: ERGO (Population Studies Act: Rotterdam Study).” All participants provided written informed consent prior to participation in the study and gave permission to obtain information from their treating physicians. 
Macular SD-OCT image volumes were obtained from one eyes of all subjects using Topcon OCT devices (3D OCT-1000, software version: 3.44; Topcon Europe, Capelle aan den Ijssel, the Netherlands). The Topcon software is a custom version for the Rotterdam Study. It provides an image quality score. Each volume is composed of 512 × 128 × 650 voxels, corresponding to physical dimensions of 6.0 × 6.0 × 2.3 mm3, and the voxel image intensity depth is 2 bytes. 
Six hundred ninety macular SD-OCT image volumes were obtained from one eye (672 right eyes, 18 left eyes) of 690 subjects (74.6 ± 9.7 [mean ± SD] years, 37.8% of males) randomly selected from the population-based Rotterdam Study. The dataset consisted of 420 OCT volumes with successful automated RNFL segmentations and 270 volumes with failed segmentations. The dataset was divided into a training set (80%) including 336 OCT volumes with successful RNFL segmentations, 216 volumes with failed segmentations, and a testing set (20%) that included 84 volumes with successful segmentations, as well as 54 volumes with failed segmentations. 
To validate our automated identification method of the OCT volumes providing successful automated RNFL segmentations, an expert created ground truth by manually categorizing the 690 macular OCT volumes into two groups: 420 OCT volumes with successful RNFL segmentations and 270 with globally/locally failed segmentations. The OCT volumes having small RNFL segmentation errors in small regions were regarded as having failed segmentations. For the ground truth determination, only RNFL boundary accuracy was taken into account, and any other information like scan centering was not considered. 
Automated Intraretinal Layer Segmentation
Ten intraretinal layers (11 surfaces) from the macular SD-OCT volumes were automatically detected by applying our previously reported automated graph-based segmentation method, the Iowa Reference Algorithms (available in the public domain from https://www.iibi.uiowa.edu/content/shared-software-download/).2022 Out of the 11 surfaces, the internal limiting membrane (ILM), the boundary between RNFL and GCL, and the outer boundary of RPE were used for determining our segmentability index SI. The ILM and the boundary between RNFL and GCL define the RNFL, and the outer boundary of RPE was used because segmentation of the boundary between RNFL and GCL was constrained by that of the ILM and the outer boundary of RPE. The layer segmentation method used three-dimensional (3D) edge-based cost functions that are inverted, Gaussian-smoothed gradient magnitudes of the OCT voxel intensities: the dark-to-bright transition from top to bottom of the OCT volume for segmentation of the ILM and the bright-to-dark transition for segmentation of the boundary between RNFL and GCL, and the outer boundary of RPE. Using these cost functions, optimal surfaces are detected as having minimal aggregate costs from all feasible surfaces. Figure 3 shows automated intraretinal layer segmentation results of a macular OCT volume. 
Figure 3
 
Automated intraretinal layer segmentation results for a macular SD-OCT volume. (A) B-scan of the OCT volume. (B) Cost function image for segmentation of the ILM. (C) Cost function image for segmentation of the boundary between RNFL and GCL, and the outer boundary of RPE. (D) B-scan in (A) overlaid with three detected surfaces. (E) 3D rendering of the three detected surfaces.
Figure 3
 
Automated intraretinal layer segmentation results for a macular SD-OCT volume. (A) B-scan of the OCT volume. (B) Cost function image for segmentation of the ILM. (C) Cost function image for segmentation of the boundary between RNFL and GCL, and the outer boundary of RPE. (D) B-scan in (A) overlaid with three detected surfaces. (E) 3D rendering of the three detected surfaces.
Segmentability Index
Our new metric, the segmentability index SI, expresses the expected reliability of automated layer segmentations, especially, RNFL segmentations for this study. The SI can be obtained from a random forest regressor23 based on the 12 following features: 
  •  
    Features 1 and 2: mean and SD of whole OCT voxel intensities (Fig. 4A).
  •  
    Features 3 and 4: mean and SD of whole edge-based costs of the dark-to-bright transition from top to bottom of the OCT volume that are inverted, Gaussian-smoothed gradient magnitudes of the OCT voxel intensities (Fig. 4B).
  •  
    Features 5 and 6: mean and SD of whole edge-based costs of the bright-to-dark transition that are inverted, Gaussian-smoothed gradient magnitudes of the OCT voxel intensities (Fig. 4C).
  •  
    Features 7 and 8: mean and SD of the on-surface costs consisting of the edge-based costs of the dark-to-bright transition along the ILM (Fig. 4D).
  •  
    Features 9 and 10: mean and SD of the on-surface costs consisting of the edge-based costs of the bright-to-dark transition along the boundary between RNFL and GCL (Fig. 4E).
  •  
    Features 11 and 12: Mean and SD of the on-surface costs consisting of the edge-based costs of the bright-to-dark transition along the outer boundary of RPE (Fig. 4F).
Figure 4
 
Feature images for calculation of segmentability index SI. (A) The mean and SD of OCT voxel intensities are calculated from the entire 3D OCT volume. (B) The mean and SD of edge-based costs of the top-to-bottom dark-to-bright transitions are calculated from the entire 3D OCT volume. (C) The mean and SD of edge-based costs of the bright-to-dark transitions are calculated from the entire 3D OCT volume. (D–F) The means and SDs of on-surface costs of three detected surfaces are calculated along the respective surfaces detected in 3D.
Figure 4
 
Feature images for calculation of segmentability index SI. (A) The mean and SD of OCT voxel intensities are calculated from the entire 3D OCT volume. (B) The mean and SD of edge-based costs of the top-to-bottom dark-to-bright transitions are calculated from the entire 3D OCT volume. (C) The mean and SD of edge-based costs of the bright-to-dark transitions are calculated from the entire 3D OCT volume. (D–F) The means and SDs of on-surface costs of three detected surfaces are calculated along the respective surfaces detected in 3D.
Features 1 through 6 were included because automated segmentation of the RNFL is globally affected by OCT voxel intensities and edge-based costs. Features 7 through 10 are directly related to RNFL segmentation. If the RNFL is successfully detected, its aggregate on-surface costs are small. Otherwise, they are large. Features 11 and 12 were also used because segmentation of the outer boundary of RPE affects the boundary between RNFL and GCL. Important features were identified with respect to increase in mean square error using a training set. A feature selection step was performed by 3-fold cross-validation using the training set in terms of area under the curve (AUC). The SI ranges from 0 to 1. Larger SI values correspond to an OCT volume allowing a more reliable RNFL segmentation. Statistical computing software (R version 2.15.2, R Development Core Team; provided in the public domain by R Foundation for Statistical Computing, Vienna, Austria, available at http://www.r-project.org/, ‘randomForest' package) was used for the random forest regression. Appendix shows the R source code for random forest regression using a four training dataset and a two testing dataset. 
Statistical Analysis
The SI approach was compared with the QI and mTCI methods using receiver operating characteristic (ROC) analysis. The accuracy is the ratio of the number of RNFL segmentations correctly identified by, respectively, the QI, mTCI, or SI to that of all RNFL segmentations. The true positive rate (sensitivity) is the ratio of the number of successful RNFL segmentations passed by respectively the QI, mTCI, or SI and the ground truth to that of all successful RNFL segmentations, and the false positive rate (1 – specificity) is the fraction of the number of successful RNFL segmentations passed by respectively the QI, mTCI, or SI and rejected by the ground truth to that of all unsuccessful RNFL segmentations. The AUC was used to measure aggregated regression performance. The AUC difference between the QI or mTCI and SI ROC curves was statistically assessed using the 95% confidence interval (CI) by the method of DeLong et al.,24 and the R statistical computing software package was used. The operating point of the QI, mTCI, or SI curves were chosen so as to result in a set with the same number of OCT volumes with RNFL segmentations expected to be successful as the ground truth, though obviously the specific OCT volumes in the sets may be different. The threshold, accuracy, sensitivity, and specificity for each ROC curve were measured at the operating point. An unpaired t-test (95% CI) was used to compare the average of RNFL thicknesses in good quality as determined by either the ground truth or the QI, mTCI, or SI, respectively, using Excel (Microsoft Corp., Redmond, WA). 
Results
The relative importance of all 12 features was tested with respect to increase in mean square error using the training set (Fig. 5). Feature 12 (SD of the on-surface costs consisting of the edge-based costs of the bright-to-dark transition along the outer boundary of RPE) is the most important feature, and Feature 6 (SD of whole edge-based costs of the bright-to-dark transition) is the least important feature of the employed feature set. Based on the importance, a feature selection step was performed on the training set in terms of AUC, and the AUC obtained by the SI approach using all 12 features represented the highest value (0.868). 
Figure 5
 
Table representing a list of the importance of 12 features in the training set including 336 OCT volumes with successful RNFL segmentations and 216 volumes with failed segmentations.
Figure 5
 
Table representing a list of the importance of 12 features in the training set including 336 OCT volumes with successful RNFL segmentations and 216 volumes with failed segmentations.
The ROC graph in Figure 6 shows the performance of the QI, mTCI approaches and the SI methods using Features 1 through 6, Features 1 through 12 for automated identification of the OCT volumes expected to provide reliable automated RNFL segmentations. The 95% CIs and AUCs of the QI, mTCI, SI (Features 1–6), and SI (Features 1–12) ROC curves are summarized in the Table. The AUC of the SI (Features 1–12) ROC curve is significantly larger than that of the QI, mTCI, or SI (Features 1–6) curve (P < 0.039). In addition, it shows the range, threshold, accuracy, sensitivity, and specificity at the operating point for the four approaches. 
Figure 6
 
ROC graph of the QI, mTCI approaches and the SI methods using Features 1–6 and Features 1–12. The markers show the operating points which are chosen so as to result in a set with the same number (84) of OCT volumes with RNFL segmentations expected to be successful as the ground truth.
Figure 6
 
ROC graph of the QI, mTCI approaches and the SI methods using Features 1–6 and Features 1–12. The markers show the operating points which are chosen so as to result in a set with the same number (84) of OCT volumes with RNFL segmentations expected to be successful as the ground truth.
Table
 
Range, 95% CI, AUC, Threshold, Accuracy, Sensitivity, and Specificity at the Operating Point for QI, mTCI, SI (Features 1–6), and SI (Features 1–12) ROC Curves
Table
 
Range, 95% CI, AUC, Threshold, Accuracy, Sensitivity, and Specificity at the Operating Point for QI, mTCI, SI (Features 1–6), and SI (Features 1–12) ROC Curves
To better understand the OCT scans flagged as insufficient by the different indices, Figure 7 shows the mean and SD RNFL thickness maps for 138 macular OCT volumes with both successful and unsuccessful segmentations, as well as a subset of 84 volumes from the 138 that were deemed successful by either the independent standard or by the QI, mTCI, SI (Features 1–6), and SI (Features 1–12) approaches. The mean RNFL thickness of the 84 volumes with successful segmentations determined from the independent standard is 38.12 ± 5.62 μm, and that with success identified by the QI, mTCI, SI (Features 1–6), or SI (Features 1–12) approach is 39.46 ± 6.32, 39.16 ± 6.10, 38.98 ± 6.57, or 38.16 ± 5.49 μm, respectively. Though Figure 7 shows regional differences, the RNFL thickness determined from the independent standard is not significantly different from that identified by the QI, mTCI, SI (Features 1–6), or SI (Features 1–12) approaches (P > 0.148). 
Figure 7
 
Mean (first, third rows) and SD (second, fourth rows) RNFL thickness maps of 138 testing macular OCT volumes with all segmentations and 84 testing volumes with successful segmentations determined from independent standard and with segmentation success identified by the QI, mTCI approaches, and the SI methods using Features 1 through 6 and Features 1 through 12. The number at the top of the mean thickness map is a mean ± SD QI, mTCI, or SI value of the OCT volumes, and that at the bottom of the thickness map is mean ± SD thickness of the RNFL. The thickness maps were created after horizontally flipping left eye OCT volumes and aligning the OCT volumes based on the fovea.
Figure 7
 
Mean (first, third rows) and SD (second, fourth rows) RNFL thickness maps of 138 testing macular OCT volumes with all segmentations and 84 testing volumes with successful segmentations determined from independent standard and with segmentation success identified by the QI, mTCI approaches, and the SI methods using Features 1 through 6 and Features 1 through 12. The number at the top of the mean thickness map is a mean ± SD QI, mTCI, or SI value of the OCT volumes, and that at the bottom of the thickness map is mean ± SD thickness of the RNFL. The thickness maps were created after horizontally flipping left eye OCT volumes and aligning the OCT volumes based on the fovea.
Discussion
For the accurate thickness analyses of population studies, it is necessary to automatically identify the macular OCT scans that have resulted in reliable automated intraretinal layer segmentations. Figure 7 shows the importance of identifying the OCT scans with accurate automated RNFL segmentations. The mean RNFL thickness map created using 84 good quality OCT scans as determined from the independent standard shows clearer arcuate nerve fiber bundle regions than that derived from 138 OCT scans that include both good quality and bad quality scans. The SD RNFL thickness map obtained from the 138 OCT scans has large variations in any region due to 54 mis-segmented RNFLs. 
To automatically identify those OCT scans that can be expected to provide reliable automated RNFL segmentations, we have introduced a segmentability index SI obtained from a random forest regressor trained by the features using OCT voxel intensities, edge-based costs, and on-surface costs. The SD of the on-surface costs consisting of the edge-based costs of the bright-to-dark transition along the outer boundary of RPE (Feature 12) is the most important feature for the identification (Fig. 5), which means that the appearance of the outer boundary of RPE has a large effect on the expected reliability on automated segmentation. The second most important feature is the mean of the on-surface costs consisting of the edge-based costs of the bright-to-dark transition along the boundary between RNFL and GCL (Feature 9), which is directly related to the RNFL segmentation. The feature selection step did not show any performance improvement in terms of AUC, so all 12 Features were used for the SI approach. 
Based on the ROC analysis, the SI using all 12 Features showed significantly better performance than the QI or mTCI in terms of AUC (P < 0.01; Fig. 6). Additionally, it showed significantly better performance than the SI using Features 1 through 6 (P = 0.038), which means that the means and SDs of the on-surface costs (Features 7–12) are useful for the identification. 
The mean and SD RNFL thickness maps of the macular OCT scans with segmentation success identified by the SI using all 12 features are more similar to those of the OCT scans with successful segmentations determined from the independent standard than those of the OCT scans with segmentation success identified by the QI, the mTCI, or the SI using Features 1 through 6 (Fig. 7). The QI and mTCI thickness maps may be thicker on average because OCT scans with thick RNFLs, and thus are bright in the OCT image, may correlate with high QI and mTCI values, as further discussed below. 
Figure 8 shows an example of a macular OCT volume in which the SI using all 12 features (successfully) identifies a successful RNFL segmentation but the QI and mTCI fail, based on the thresholds mentioned in Table. The means and SDs of the on-surface costs (Features 7–12) calculated using three reliable surface segmentations obtained from the method helped the SI approach to identify the OCT scans with successful RNFL segmentations compared to the QI or mTCI approaches. 
Figure 8
 
A macular OCT volume in which the SI approach using all 12 features succeeds to identify a successful RNFL segmentation but the QI and mTCI methods fail, based on the thresholds in the Table. (A) B-scan of the OCT volume (QI = 12.7, mTCI = 2.08, SI [Features 1–6] = 0.605, and SI [Features 1–12] = 0.765). (B) B-scan in (A) overlaid with the successful RNFL segmentation.
Figure 8
 
A macular OCT volume in which the SI approach using all 12 features succeeds to identify a successful RNFL segmentation but the QI and mTCI methods fail, based on the thresholds in the Table. (A) B-scan of the OCT volume (QI = 12.7, mTCI = 2.08, SI [Features 1–6] = 0.605, and SI [Features 1–12] = 0.765). (B) B-scan in (A) overlaid with the successful RNFL segmentation.
Previous studies reported the positive relationship between OCT manufacturer's image quality index and RNFL thickness. Darma et al.14 assessed the effect of media opacities on intraretinal layer thickness measurements of macular Topcon OCT scans (3D OCT-1000) using a set of filters with known optical density, and the layer thickness measurements were influenced by image degradation caused by optical density filters. Cheung et al.15 examined the relationship between signal strength and RNFL measured by TD-OCT (Stratus; Carl Zeiss Meditec, Oberkochen, Germany), and the RNFL thickness in general increased with the signal strength of the image. Wu et al.16 identified factors associated with variability in RNFL thickness measurements obtained by repeat TD-OCT (Stratus), and signal strength variability was associated with variability in RNFL thickness measurements. Other studies did not show any relationship between them. Rao et al.17 evaluated the effects of signal strength, age, sex, optic disc size, and axial length on the normal ONH, RNFL, macular measurements with SD-OCT (RTVue; Optovue Inc., Fremont, CA), and any identifier including the signal strength did not influence the RNFL measurements. Samarawickrama et al.18 examined the influence of different signal strengths on measurements of macular, ONH, and RNFL parameters using TD-OCT (Stratus), and significant differences in RNFL parameters with increasing signal strength were not observed. 
There are some limitations to the proposed identification approach. First, in the present study, the method was tested using a subset of the population-based dataset, most of which are normal OCT scans. The results need to be replicated with OCT scans showing pathology for more general use. Second, the method was biased toward the opinion of the expert who created the reference standard. For more objective validation of the method, a reference standard created by multiple experts is required. 
The method used the RNFL segmentations as ground truth. For the ground truth, other intraretinal layer segmentations such as GCL or total retina, or the combination of all layer segmentations can be used. Incorporation of layer-specific textural features may further improve the accuracy of the method. This identification method can be easily extended for validation of general automated image segmentation. 
In conclusion, the segmentability index SI is well suited to identify the OCT scans expected to provide reliable automated intraretinal layer segmentations, which allows more accurate and more reliable automated analyses of intraretinal layers in population studies. 
Acknowledgments
Supported by grants from The National Institutes of Health grants R01-EY017066, R01-EY018853, R01-EB004640; the Department of Veterans Affairs; the Netherlands Organization for Scientific Research, the Hague; Swart van Essen, Rotterdam; Bevordering van Volkskracht, Rotterdam; Rotterdamse Blindenbelangen Association, Rotterdam; Algemene Nederlandse Vereniging ter Voorkoming van Blindheid, Doorn, The Netherlands; Oogfonds Nederland, Utrecht; MDFonds, Utrecht; Vereniging Trustfonds Erasmus Universiteit Rotterdam, Rotterdam, The Netherlands; and Lijf en Leven, Krimpen aan de IJssel, The Netherlands. An unrestricted grant was obtained from Topcon Europe BV, Capelle aan den IJssel, The Netherlands. 
Disclosure: K. Lee, None; G.H.S. Buitendijk, None; H. Bogunovic, None; H. Springelkamp, None; A. Hofman, None; A. Wahle, None; M. Sonka, P; J.R. Vingerling, None; C.C.W. Klaver, None; M.D. Abràmoff,
References
Huang D, Swanson EA,, Lin CP, et al. Optical coherence tomography. Science. 1991; 254: 1178–1181.
Fujimoto J,, Bouma B, Tearney G,, et al. New technology for high speed and high resolution optical coherence tomography. Ann N Y Acad Sci. 1998; 838: 95–107.
van Velthoven ME, Faber DJ,, Verbraak FD, et al. Recent developments in optical coherence tomography for imaging the retina. Prog Retin Eye Res. 2007; 26: 57–77.
Mwanza JC,, Budenz DL, Godfrey DG,, et al. Diagnostic performance of optical coherence tomography ganglion cell-inner plexiform layer thickness measurements in early glaucoma. Ophthalmology. 2014; 121: 849–854.
Sari ES, Ermis SS,, Yazici A, et al. The effect of intracameral anesthesia on macular thickness and ganglion cell-inner plexiform layer thickness after uneventful phacoemulsification surgery: prospective and randomized controlled trial. Graefes Arch Clin Exp Ophthalmol. 2014; 252: 433–439.
Chen X,, Zhang L, Sohn EH,, et al. Quantification of external limiting membrane disruption caused by diabetic macular edema from SD-OCT. Invest Ophthalmol Vis Sci. 2012; 53: 8042–8048.
Zhang L, Lee K,, Niemeijer M, et al. Automated segmentation of the choroid from clinical SD-OCT. Invest Ophthalmol Vis Sci. 2012; 53: 7510–7519.
Kay CN,, Abramoff MD, Mullins RF,, et al. Three-dimensional distribution of the vitelliform lesion, photoreceptors, and retinal pigment epithelium in the macula of patients with best vitelliform macular dystrophy. Arch Ophthalmol. 2012; 130: 357–364.
van Dijk HW, Verbraak FD,, Kok PH, et al. Decreased retinal ganglion cell layer thickness in patients with type 1 diabetes. Invest Ophthalmol Vis Sci. 2010; 51: 3660–3665.
Ho J,, Castro DP, Castro LC,, et al. Clinical assessment of mirror artifacts in spectral-domain optical coherence tomography. Invest Ophthalmol Vis Sci. 2010; 51: 3714–3720.
Folio LS, Wollstein G,, Ishikawa H, et al. Variation in optical coherence tomography signal quality as an indicator of retinal nerve fibre layer segmentation error. Br J Ophthalmol. 2012; 96: 514–518.
Stein DM,, Ishikawa H, Hariprasad R,, et al. A new quality assessment parameter for optical coherence tomography. Br J Ophthalmol. 2006; 90: 186–190.
Huang Y, Gangaputra S,, Lee KE, et al. Signal quality assessment of retinal optical coherence tomography images. Invest Ophthalmol Vis Sci. 2012; 53: 2133–2141.
Darma S,, Kok PH, van den Berg TJ,, et al. Optical density filters modeling media opacities cause decreased SD-OCT retinal layer thickness measurements with inter- and intra-individual variation. Acta Ophthalmol . In press.
Cheung CY, Leung CK,, Lin D, et al. Relationship between retinal nerve fiber layer measurement and signal strength in optical coherence tomography. Ophthalmology. 2008; 115: 1347–1351.
Wu Z,, Vazeen M, Varma R,, et al. Factors associated with variability in retinal nerve fiber layer thickness measurements obtained by optical coherence tomography. Ophthalmology. 2007; 114: 1505–1512.
Rao HL, Kumar AU,, Babu JG, et al. Predictors of normal optic nerve head, retinal nerve fiber layer, and macular parameters measured by spectral domain optical coherence tomography. Invest Ophthalmol Vis Sci. 2011; 52: 1103–1110.
Samarawickrama C,, Pai A, Huynh SC,, et al. Influence of OCT signal strength on macular, optic nerve head, and retinal nerve fiber layer parameters. Invest Ophthalmol Vis Sci. 2010; 51: 4471–4475.
Hofman A, Darwish Murad S,, van Duijn CM, et al. The Rotterdam Study: 2014 objectives and design update. Eur J Epidemiol. 2013; 28: 889–926.
Abràmoff MD,, Garvin MK, Sonka M. Retinal imaging and image analysis. IEEE Trans Med Imaging. 2010; 3: 169–208.
Garvin MK, Abràmoff, , Wu X, et al. Automated 3-D intraretinal layer segmentation of macular spectral-domain optical coherence tomography images. IEEE Trans Med Imaging. 2009; 28: 1436–47.
Lee K,, Niemeijer M, Garvin MK,, et al. Segmentation of the optic disc in 3-D OCT scans of the optic nerve head. IEEE Trans Med Imaging. 2010; 29: 159–68.
Breiman L. Random forests. Machine Learning. 2001; 45: 5–32.
DeLong ER, DeLong DM,, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988; 44: 837–845.
Appendix
1. R source code for random forest regression using four training dataset and two testing dataset: 
library(randomForest) 
input = read.csv(“C:/Input.csv”) 
train = input[c(1:4),] 
test = input[c(5:6),] 
r = randomForest(Truth ∼., data = train, importance = T) 
test = predict(r, test) 
output = write.table(test, file = “C:/Output.csv”, append = T, sep = “,”, col.names = F) 
2. Input: 
In ‘Input.csv', 
Table
Table
3. Output: 
Table
Table
Figure 1
 
Automated intraretinal layer segmentations of macular SD-OCT volumes with good (A), moderate (B), and bad (C) image quality and corresponding segmentability index SI, which is explained in Methods. (D–F) B-scans in (A–C) overlaid with automatically detected layers (red line: ILM, orange line: boundary between RNFL and GCL, green line: outer boundary of retinal pigment epithelium [RPE]).
Figure 1
 
Automated intraretinal layer segmentations of macular SD-OCT volumes with good (A), moderate (B), and bad (C) image quality and corresponding segmentability index SI, which is explained in Methods. (D–F) B-scans in (A–C) overlaid with automatically detected layers (red line: ILM, orange line: boundary between RNFL and GCL, green line: outer boundary of retinal pigment epithelium [RPE]).
Figure 2
 
Mean and SD RNFL thickness maps of 690 macular OCT volumes including 420 volumes with successful RNFL segmentations and 270 volumes with failed segmentations. The number at the bottom of the thickness map is mean ± SD thickness of the RNFL. The thickness maps were created after horizontally flipping left eye OCT volumes and aligning the OCT volumes based on the fovea.
Figure 2
 
Mean and SD RNFL thickness maps of 690 macular OCT volumes including 420 volumes with successful RNFL segmentations and 270 volumes with failed segmentations. The number at the bottom of the thickness map is mean ± SD thickness of the RNFL. The thickness maps were created after horizontally flipping left eye OCT volumes and aligning the OCT volumes based on the fovea.
Figure 3
 
Automated intraretinal layer segmentation results for a macular SD-OCT volume. (A) B-scan of the OCT volume. (B) Cost function image for segmentation of the ILM. (C) Cost function image for segmentation of the boundary between RNFL and GCL, and the outer boundary of RPE. (D) B-scan in (A) overlaid with three detected surfaces. (E) 3D rendering of the three detected surfaces.
Figure 3
 
Automated intraretinal layer segmentation results for a macular SD-OCT volume. (A) B-scan of the OCT volume. (B) Cost function image for segmentation of the ILM. (C) Cost function image for segmentation of the boundary between RNFL and GCL, and the outer boundary of RPE. (D) B-scan in (A) overlaid with three detected surfaces. (E) 3D rendering of the three detected surfaces.
Figure 4
 
Feature images for calculation of segmentability index SI. (A) The mean and SD of OCT voxel intensities are calculated from the entire 3D OCT volume. (B) The mean and SD of edge-based costs of the top-to-bottom dark-to-bright transitions are calculated from the entire 3D OCT volume. (C) The mean and SD of edge-based costs of the bright-to-dark transitions are calculated from the entire 3D OCT volume. (D–F) The means and SDs of on-surface costs of three detected surfaces are calculated along the respective surfaces detected in 3D.
Figure 4
 
Feature images for calculation of segmentability index SI. (A) The mean and SD of OCT voxel intensities are calculated from the entire 3D OCT volume. (B) The mean and SD of edge-based costs of the top-to-bottom dark-to-bright transitions are calculated from the entire 3D OCT volume. (C) The mean and SD of edge-based costs of the bright-to-dark transitions are calculated from the entire 3D OCT volume. (D–F) The means and SDs of on-surface costs of three detected surfaces are calculated along the respective surfaces detected in 3D.
Figure 5
 
Table representing a list of the importance of 12 features in the training set including 336 OCT volumes with successful RNFL segmentations and 216 volumes with failed segmentations.
Figure 5
 
Table representing a list of the importance of 12 features in the training set including 336 OCT volumes with successful RNFL segmentations and 216 volumes with failed segmentations.
Figure 6
 
ROC graph of the QI, mTCI approaches and the SI methods using Features 1–6 and Features 1–12. The markers show the operating points which are chosen so as to result in a set with the same number (84) of OCT volumes with RNFL segmentations expected to be successful as the ground truth.
Figure 6
 
ROC graph of the QI, mTCI approaches and the SI methods using Features 1–6 and Features 1–12. The markers show the operating points which are chosen so as to result in a set with the same number (84) of OCT volumes with RNFL segmentations expected to be successful as the ground truth.
Figure 7
 
Mean (first, third rows) and SD (second, fourth rows) RNFL thickness maps of 138 testing macular OCT volumes with all segmentations and 84 testing volumes with successful segmentations determined from independent standard and with segmentation success identified by the QI, mTCI approaches, and the SI methods using Features 1 through 6 and Features 1 through 12. The number at the top of the mean thickness map is a mean ± SD QI, mTCI, or SI value of the OCT volumes, and that at the bottom of the thickness map is mean ± SD thickness of the RNFL. The thickness maps were created after horizontally flipping left eye OCT volumes and aligning the OCT volumes based on the fovea.
Figure 7
 
Mean (first, third rows) and SD (second, fourth rows) RNFL thickness maps of 138 testing macular OCT volumes with all segmentations and 84 testing volumes with successful segmentations determined from independent standard and with segmentation success identified by the QI, mTCI approaches, and the SI methods using Features 1 through 6 and Features 1 through 12. The number at the top of the mean thickness map is a mean ± SD QI, mTCI, or SI value of the OCT volumes, and that at the bottom of the thickness map is mean ± SD thickness of the RNFL. The thickness maps were created after horizontally flipping left eye OCT volumes and aligning the OCT volumes based on the fovea.
Figure 8
 
A macular OCT volume in which the SI approach using all 12 features succeeds to identify a successful RNFL segmentation but the QI and mTCI methods fail, based on the thresholds in the Table. (A) B-scan of the OCT volume (QI = 12.7, mTCI = 2.08, SI [Features 1–6] = 0.605, and SI [Features 1–12] = 0.765). (B) B-scan in (A) overlaid with the successful RNFL segmentation.
Figure 8
 
A macular OCT volume in which the SI approach using all 12 features succeeds to identify a successful RNFL segmentation but the QI and mTCI methods fail, based on the thresholds in the Table. (A) B-scan of the OCT volume (QI = 12.7, mTCI = 2.08, SI [Features 1–6] = 0.605, and SI [Features 1–12] = 0.765). (B) B-scan in (A) overlaid with the successful RNFL segmentation.
Table
 
Range, 95% CI, AUC, Threshold, Accuracy, Sensitivity, and Specificity at the Operating Point for QI, mTCI, SI (Features 1–6), and SI (Features 1–12) ROC Curves
Table
 
Range, 95% CI, AUC, Threshold, Accuracy, Sensitivity, and Specificity at the Operating Point for QI, mTCI, SI (Features 1–6), and SI (Features 1–12) ROC Curves
Table
Table
Table
Table
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×