Platform-Independent Cirrus and Spectralis Thickness Measurements in Eyes with Diabetic Macular Edema Using Fully Automated Software

Abstract

Purpose: We determine whether the automated segmentation software, Duke Optical Coherence Tomography Retinal Analysis Program (DOCTRAP), can measure, in a platform-independent manner, retinal thickness on Cirrus and Spectralis spectral domain optical coherence tomography (SD-OCT) images in eyes with diabetic macular edema (DME) under treatment in a clinical trial.

Methods: Automatic segmentation software was used to segment the internal limiting membrane (ILM), inner retinal pigment epithelium (RPE), and Bruch's membrane (BM) in SD-OCT images acquired by Cirrus and Spectralis commercial systems, from the same eye, on the same day during a clinical interventional DME trial. Mean retinal thickness differences were compared across commercial and DOCTRAP platforms using intraclass correlation (ICC) and Bland-Altman plots.

Results: The mean 1 mm central subfield thickness difference (standard error [SE]) comparing segmentation of Spectralis images with DOCTRAP versus HEYEX was 0.7 (0.3) μm (0.2 pixels). The corresponding values comparing segmentation of Cirrus images with DOCTRAP versus Cirrus software was 2.2 (0.7) μm. The mean 1 mm central subfield thickness difference (SE) comparing segmentation of Cirrus and Spectralis scan pairs with DOCTRAP using BM as the outer retinal boundary was −2.3 (0.9) μm compared to 2.8 (0.9) μm with inner RPE as the outer boundary.

Conclusions: DOCTRAP segmentation of Cirrus and Spectralis images produces validated thickness measurements that are very similar to each other, and very similar to the values generated by the corresponding commercial software in eyes with treated DME.

Translational Relevance: This software enables automatic total retinal thickness measurements across two OCT platforms, a process that is impractical to perform manually.

Introduction

Commercially manufactured spectral domain optical coherence tomography (SD-OCT) imaging systems, such as the Cirrus HD-OCT (Carl Zeiss Meditec, Dublin, CA) and Spectralis (Heidelberg Engineering, Heidelberg, Germany), have proprietary software to segment inner and outer retinal boundaries, from which thickness measurements can be calculated. However, each system's segmentation software identifies the outer retinal boundary at slightly different locations, making it hard to compare thickness measurements between platforms.^1,2 There is an unmet need to develop software that can identify boundaries at common reference points across multiple OCT platforms, so that retinal thickness can be compared across platforms. 

In a previous study, we found that the Duke Optical Coherence Tomography Retinal Analysis Program (DOCTRAP) consistently segments the inner limiting membrane (ILM), retinal pigment epithelium (RPE) boundary, and Bruch's membrane (BM) in eyes with diabetic macular edema (DME).³ Previous studies have been limited by nonstandardized imaging protocols, datasets without same-day same-eye scan pairs implementing two OCT systems, or only studying eyes with minimal pathology.^4–7 

In our present study, datasets were acquired from a clinical trial that tested the comparative efficacy of monthly injection of bevacizumab and ranibizumab for DME.⁸ In that trial, study eyes were scanned systematically with two different OCT platforms during the same visit. After randomly selecting same-day, same-eye pairings and automatically segmenting them with commercial and DOCTRAP algorithms, we compared sector thickness results in the standard nine-field Early Treatment of Diabetic Retinopathy Study (ETDRS) grid. We then stratified our analysis according to whether the outer retinal boundary was assigned to BM, as done by Spectralis software, or to the inner RPE, as done by Cirrus software, and to whether commercial or DOCTRAP algorithms were used. 

Methods

Study Dataset

This study was approved by the Duke University Institutional Review Board and adhered to the tenets of the Declaration of Helsinki. The clinical trial was conducted at two sites, the National Eye Institute (Bethesda, MD) and University Hospitals Bristol National Health Service Foundation Trust (Bristol, UK). Institutional review board/independent ethics committee approval was obtained at both sites and all participants gave written informed consent. 

The clinical trial consisted of a randomized, double-masked, 36-week crossover study with a parallel group extension phase in which participants enrolled one or both eyes and underwent monthly evaluation, including OCT scanning. Treatment with either bevacizumab or ranibizumab according to an assigned treatment schedule was given monthly during the 36-week crossover phase, and then as-needed (as often as every month) at investigator discretion thereafter. The study protocol specified standardized OCT scanning using a Cirrus imaging system for all study eyes at all study visits. In 145 study visits at one site (University Hospitals Bristol National Health Service Foundation Trust), Cirrus scanning was supplemented by standardized Spectralis imaging. Over 52 weeks, 41 participants at that study site received Cirrus and Spectralis imaging of the same eye, on the same day, at one or more study visits. In total, 145 scan pairs for which Cirrus and Spectralis data were obtained for a given eye on the same day were generated accordingly, some of these at visits during the 36-week crossover phase (during which anti-vascular endothelial growth factor [VEGF] treatment was given monthly), and others at visits during the extension phase through 52 weeks (during which anti-VEGF treatment was given as needed at investigator discretion). The details of the patient population and study methodology have been published recently.⁸ 

The Spectralis SD-OCT scans were obtained according to the following protocol: 49 cross-sectional B-scan images (each scan was 512 × 496 pixels) were acquired in high-speed mode, and covered an area of 20° × 20°. The Cirrus SD-OCT scans were obtained according to the following protocol: 128 cross-sectional B-scan images, each comprising 128 A-scans, were acquired at high-speed (each A-scan was 512 × 1024 pixels), and covered an area of 6 × 6 mm. Retinal layer segmentation and thickness calculations were determined on Heidelberg Engineering commercial software (Heidelberg Eye Explorer [HEYEX]), version 5.6.3.0, and Cirrus software, version 6.0.2.81 (Fig. 1). 

Figure 1

 

Example of DOCTRAP segmentation of ILM (white), inner RPE (pink), and BM (blue) boundaries in study eye with diabetic macular edema.

From the 145 available Spectralis–Cirrus scan pairs, an initial random sample of 40 paired datasets (40 Cirrus and 40 Spectralis) comprising 27 patients was selected for further analysis. Each dataset included all 49 Spectralis B-scans and all 128 Cirrus B-scans. A Duke Reading Center Senior Reader evaluated image quality according to previously reported criteria.⁹ Ten paired datasets were excluded from the analysis due to poor resolution, eye movement artifact, or poor foveal centration in which the ETDRS grid fell outside of the acquisition area, in at least one of the two volume scans that comprised each pair, leaving 30 paired datasets for analysis. The recommended minimum signal strength sufficient to produce satisfactory image quality for Cirrus is 6 on a 1 to 10 scale and for Spectralis is 15 on a 1 to 40 scale.^10,11 In our study, the Cirrus signal strength averaged 8.1 and the Spectralis signal strength averaged 15, both well above the recommended minimums. Accordingly, high quality images from both systems were available for comparison.¹² When the sample size is 30, a two-sided 95.0% confidence interval (CI) computed using the large sample normal approximation for an intraclass correlation based on two measurements will extend approximately 0.007 from the observed intraclass correlation when the expected intraclass correlation is 0.990.^3,13 Before this project, some of the OCT images obtained in this clinical trial were used to develop the DOCTRAP segmentation algorithms. None of the images from that training dataset was included in the 30 scan-pairs considered in the present analysis. 

Automated Segmentation by Commercial Software

For each dataset, the Duke Reading Center reader first centered the standard nine-field ETDRS grid at the fovea, as needed, by systematically viewing all B-scans to locate foveal landmarks, which included the point at which the inner retinal layers were thinnest, and the foveal depression and/or hyperreflective dot that corresponded to reflected light at the foveal center, if present (Fig. 2). The foveas then were marked and registered to each other. Cirrus and HEYEX software then was used to segment, in an automated manner, each B-scan image (Fig. 3, left column). The Cirrus segmentation software algorithm defines retinal thickness as the distance between the ILM to the inner one-third of the RPE, whereas the HEYEX segmentation software defines retinal thickness as the distance between the ILM and the outer RPE boundary at BM. Both software algorithms produced thickness values for all 9 ETDRS sectors that included the central 1 mm subfield, and the four inner and four outer retinal subfields (Table 1, Fig. 2), while taking into consideration grid orientation (right eye vs. left eye). HEYEX software additionally reported each sector volume and the total volume encompassed by the ETDRS grid, but did not produce the volume in the 20° × 20° scan area. The Cirrus software determined the volume in the 6 × 6 mm scan area, but did not determine the volume of individual sectors, or the volume encompassed by the ETDRS grid (Table 1). No manual corrections were applied to the segmentation lines. The ETDRS sector thickness values then were entered into an Excel spreadsheet for further analysis. 

Figure 2

 

ETDRS sector map of OD macula (mirror image for OS).

Figure 3

 

Upper left: Spectralis software segmentation; automatic segmentation lines are placed on the ILM (top red line) and BM (bottom red line). Upper right: DOCTRAP software segmentation using Spectralis dataset; automatic segmentation lines are placed on the ILM (white line), inner one-third of RPE (pink line), and BM (blue line). Lower Left: Cirrus software segmentation; automatic segmentation are lines placed on the ILM (white line) and the inner one-third of the RPE (black line). Lower Right: DOCTRAP software segmentation using Cirrus dataset; automatic segmentation lines are placed on the ILM (white line), inner 1/3 of RPE (pink line), and BM (blue line).

Table 1

 

Comparison of DOCTRAP and Commercial Software Segmentation

 

Automated Segmentation by DOCTRAP Software

Spectralis and Cirrus datasets were exported from commercial software as VOL and IMG files, respectively, and converted to bitmap files using MATLAB (MathWorks, Natick, MA). For all images, DOCTRAP segmented in an automated manner the ILM, inner RPE, and BM (Fig. 3, right column). To match the positions of these boundaries with commercial software segmentations, constant offsets of −0.5, 1.5, and 1.0 pixels were applied to the three DOCTRAP-segmented boundaries, respectively, for all Cirrus images. These offsets apply to any Cirrus machine, as all Cirrus machines have an axial pixel spacing of 1.9531 μ per pixel; Cirrus is a Food and Drug Administration (FDA)–approved commercial OCT imaging system, and FDA approval requires verification of resolution consistency during the manufacturing process. Furthermore, we previously verified the pixel pitch for two Cirrus machines in our previous publication.¹⁴ 

Similarly, offsets of 0.5, 0.5, and 0 pixels were applied to DOCTRAP segmentations on Spectralis images. With these offsets, it was possible to segment with DOCTRAP using HEYEX layer boundaries (ILM and BM) on a Cirrus-acquired image and using Cirrus boundaries (ILM and inner one-third of the RPE) on a Spectralis image. We then could compare directly the thickness measurements determined on the two different machines. To compare DOCTRAP outputs to commercial software outputs, which have unique segmentation boundaries, offsets were applied in DOCTRAP to create segmentation boundaries that were equivalent to those produced by commercial software. 

The DOCTRAP segmentation algorithm used in this manuscript was developed based on graph theory and dynamic programming (GTDP) and customized for eyes with DME. A detailed description of this three-layer boundary DME segmentation algorithm has been reported previously.^3,9,15 Additional modifications, specific to this study, were made to the algorithm so that lower quality images and those with more severe pathology could be consistently segmented. The retina was flattened based on estimates of the ILM or RPE, whichever had the lowest residual norm after fitting to a second order polynomial. Modified weights and search regions were applied to the initial inner RPE and BM segmentations to generate refined segmentations. To further improve the accuracy of the second segmentation iteration, this process then was applied to the refined second segmentation to generate a final optimized segmentation. 

Manual Segmentation Line Correction

After automated thickness measurements were recorded for all datasets, the reader manually corrected all segmentation errors observed across all datasets to determine the effect of software segmentation errors on the calculated retinal thickness. Corrections were performed on images segmented by commercial Cirrus software, HEYEX software, and DOCTRAP software (Fig. 4). The reader performed corrections using each system's default manual correction software. A thickness change of >1 μm was used as the threshold cutoff to calculate the percentage of B-scans manually corrected. This threshold was used to isolate user-performed corrections because the majority of Spectralis datasets after correction had <1 μm variations in their boundary positions. Additionally, correction rate was computed using thickness change instead of boundary change because Cirrus only exports thickness maps, not boundary positions. Therefore, after using thickness change and the 1 μm threshold, DOCTRAP manual correction rates were 7% and 9% for Spectralis and Cirrus datasets, respectively, while commercial Spectralis and Cirrus correction rates were 13% and 4%, respectively. 

Figure 4

 

Above: Cirrus software segmentation showing an uncorrected segmentation error in the inner one-third of the RPE (traced in red). Below: The segmentation of the inner 1/3 of the RPE (traced in red) after manual correction.

Statistical Analysis

Retinal thickness measurements were summarized using mean differences (standard error [SE]) and absolute mean differences (SE), controlling for sector and eye for the following paired thickness measurements: (1) DOCTRAP-determined measurements from Spectralis datasets and DOCTRAP-determined measurements from Cirrus datasets using both pairs of segmentation boundaries, (2) HEYEX- determined measurements from Spectralis datasets and DOCTRAP-determined measurements from Spectralis datasets, (3) HEYEX-determined measurements from Spectralis datasets and DOCTRAP-determined measurements from Cirrus datasets using HEYEX segmentation boundaries, (4) Cirrus software-determined measurements from Cirrus datasets and DOCTRAP-determined measurements from Cirrus datasets, and (5) Cirrus software–determined measurements from Cirrus datasets and DOCTRAP-determined measurements from Spectralis datasets using Cirrus segmentation boundaries. Means and standard errors were computed for each of the nine sectors. The intraclass correlation (ICC) was used to determine agreement between retinal thickness measurements obtained by DOCTRAP and HEYEX software, and between DOCTRAP and Cirrus software. The variance components within the ICC computation are determined by a 1-way random effects analysis of variance (ANOVA) model with an exact 95% CI computed using the method defined by McGraw and Wong 1996.¹⁶ Bland-Altman plots were constructed to compare differences between DOCTRAP and commercial segmentation software. Statistical analyses were performed using SAS 9.3 (SAS Institute, Inc., Cary, NC) with a significance level of 0.05. 

Results

DOCTRAP Versus Commercial Software

The thickness values determined by DOCTRAP agreed well with that determined by the commercial software. When the DOCTRAP segmentation algorithm was applied to Spectralis datasets using ILM and BM as the inner and outer retinal boundaries (the boundaries used by HEYEX software) and compared to HEYEX, the mean thicknesses (SE) of the 1 mm central subfield, sector 1, before manual correction was nearly identical 314.2 (11.4) and 314.9 (11.5) μm, respectively (Table 2). The mean 1 mm central subfield thickness difference (SE) was 0.7 (0.3) μm (0.2 pixels) before manual segmentation correction and 0.6 (0.3) μm (0.2 pixels) after manual correction (Table 3). The 95% limits of agreement were −2.4 and 3.9 μm (Fig. 5). 

Table 2

 

Retina Thickness Comparisons

 

Table 2

 

Extended

 

Table 3

 

Mean Paired Difference in Retinal Thickness as Measured from ILM to BM

 

Table 3

 

Extended

 

Figure 5

 

Bland-Altman plots of the differences versus the mean retinal thickness. Solid reference line indicates line of best fit (N = 30).

When the DOCTRAP segmentation algorithm was applied to Cirrus datasets using the ILM and inner one-third RPE boundary as the inner and outer retinal boundaries (the boundaries used by Cirrus software) the mean thicknesses (SE) of the 1-mm central subfield, sector 1, determined before manual correction was nearly identical 301.4 (11.3) and 303.7 (11.4) μm, by DOCTRAP and Cirrus software, respectively (Table 2). The mean 1 mm central subfield thickness difference (SE) was 2.2 (0.7) μm (1.1 pixels) before manual segmentation correction and was 1.6 (0.3) μm (0.8 pixels) after manual correction (Table 4). The 95% limits of agreement were −8.2 and 18.2 μm (Fig. 5). 

Table 4

 

Mean Paired Difference in Retinal Thickness as Measured from ILM to Inner RPE Layer

 

Table 4

 

Extended

 

Dependence of DOCTRAP Thickness Measurements on Segmentation Boundaries

The retinal thickness was readily determined by DOCTRAP on either Cirrus or Spectralis images with either ILM/BM boundary pairs or ILM/inner RPE boundary pairs. However, the difference in DOCTRAP-calculated retinal thickness measurements determined on Cirrus images compared to those determined on Spectralis images was slightly smaller when the ILM to BM segmentation boundary pair (used by HEYEX) was used; the mean uncorrected 1 mm central subfield thickness difference (SE) was −2.3 (0.9) μm and improved to −2.0 (0.9) μm after correction (Table 3). When DOCTRAP was programmed to use the ILM and inner RPE boundaries (used by Cirrus software), the mean 1 mm central subfield thickness difference before correction was 2.8 (0.9) and 2.6 (0.9) μm after correction (Table 4). 

Discussion

In this study, we validated the ability of a fully automated segmentation software system, DOCTRAP, to produce OCT platform-independent fully automated thickness measurements across 9 separate ETDRS grid sectors, in a clinical trial setting, from eyes of patients with DME. We found that when DOCTRAP was programmed to use similar inner and outer retinal boundaries as that used by the native Cirrus and HEYEX commercial software, thickness measurements were similar to that obtained on the commercial software. The interclass correlations between DOCTRAP and commercial machine software central subfield thickness values were very high. Furthermore, when the retinal boundaries were set to a common reference point, the thickness measurements derived by DOCTRAP were similar on images obtained on Cirrus and Spectralis systems from the same patient on the same day, with nearly identical central subfield thicknesses. 

With DOCTRAP, it was possible to generate retinal thickness measurements whether the outer retinal boundary was the inner one-third of the RPE layer or BM, the boundaries used by the Cirrus and HEYEX commercial software segmentation algorithms, respectively. While DOCTRAP produced consistent measurements, regardless of the outer retinal boundary used, the measurements obtained with BM as the outer retinal boundary were slightly more consistent than those obtained using inner RPE as the outer retinal boundary. We hypothesized that a lesser effect of macular edema–associated artifacts, such as shadowing on the BM–RPE interface when compared to the inner RPE boundary, may account for these differences. We believe more precise segmentation, resulting in smaller standard errors, occurs using BM due to the sharper, more defined contrast between the hyper- and hyporeflective pixels as compared to the more graded pixel intensity at the OS/RPE junction. As BM is only a few pixels thick with greater contrast between hyper- and hyporeflective pixels, it is more accurately segmented. Accordingly, when DOCTRAP is used in a clinical or clinical trial setting, it would be preferable to use BM as a common reference point to compare thickness measurements across OCT platforms. 

In eyes with DME, regional and global retinal thickness and volume measurements often are used to plan laser photocoagulation or pharmacologic treatment strategies, and changes in these parameters are used frequently to assess the effect of treatment over time.^17–20 HEYEX software computes volume over the area encompassed by the total ETDRS grid and for individual sectors 1 to 9, while Cirrus software computes only the volume over the 6 × 6 mm scanned area (Table 1). With DOCTRAP, we expect volume to correlate well in each of the 9 ETRDS subfields, across the entire ETDRS grid, and across the entire area encompassed by the volume scan, regardless of the OCT platform, given the precision of the thickness measurements. Frequently there are regional retinal volume increases in eyes with DME, and regional changes in thickness in response to therapy, that may or may not be reflected by the total macular volume, across the ETDRS subfields, or the area encompassed by the entire OCT volume scan. Accordingly, the ability of DOCTRAP to precisely monitor these regional changes, in which similar regions can be compared to one another in an OCT-platform independent manner, will be advantageous in a clinical trial setting when more than one OCT system is used across different study sites, or in the clinic, when a different OCT platform is used to scan an individual patient from one exam to the next. While it would be of benefit for future DOCTRAP versions to implement B- and C-scans to more accurately segment volume scans or fluid, the current B-scan only version is advantageous in that it can analyze more effectively less dense scan patterns, such as line or radial scans, and volumetric scans with relatively widely spaced B-scans.^21,22 

In clinical trials, when more than one OCT system has been used to determine retinal thickness, the percentage change from baseline for an individual patient often has been used with the commercially available software segmentation algorithms as a method to compare thicknesses across OCT platforms.^23–26 However, this method still leads to a different percent change for the individual patient, for each respective OCT system, and the magnitude depends on the baseline retinal thickness. For example, in one study, the average measured thickness was 21 μm greater when the Spectralis algorithm was used, compared to the Cirrus algorithm.²⁷ Similarly, in our study, we calculated an average thickness difference of 15.9 μm greater when the Spectralis algorithm was used, compared to the Cirrus algorithm. If the baseline thickness as measured by Spectralis was 300 μm, and increased to 400 μm, then the percentage increase would be 33% ([400 − 300]/300 × 100). Had a Cirrus system been used, the corresponding percentage change would be 36% ([379 – 279]/279 × 100). While these differences might not be meaningful for clinical management, they might be important in a clinical trial designed to determine efficacy of a particular drug. These differences would be minimized with software, such as DOCTRAP, that can segment in an automated manner OCT images using common reference boundaries across platforms. 

This study has limitations. The scan protocols for Cirrus and Spectralis differed per the clinical study protocol. For Spectralis, 49 cross-sectional high-speed B-scan images (512 × 496 pixels) were acquired, while for Cirrus, 128 cross-sectional high-speed B-scan images (512 × 1024 pixels) were acquired. Accordingly, the distance between scans was greater for the Spectralis-acquired images. Additionally, each system internally samples the data differently, which causes a difference in the μm/pixel ratio of images between systems. For example, Spectralis images were 3.87 μm/pixel axially as compared to Cirrus images with an axial resolution of 1.95 μm/pixel. The Heidelberg and Zeiss companies produce this number, which is a constant across their systems. Despite these differences, the DOCTRAP algorithm produced very similar thickness measurements from the two OCT systems, when used to scan the same eye on the same day. These data further highlighted the potential use of DOCTRAP to produce consistent thickness measurements across platforms in the clinic and in clinical trials. 

There are accumulating data that retinal microstructural abnormalities, in addition to retinal thickness and volume, correlate with visual function. For example, disruption of the ellipsoid zone, external limiting membrane, and inner retinal layers is associated with worse visual acuity.^28–31 Accordingly, it would be very advantageous to segment these layers in an automated manner, to calculate their thickness and volume, a process that is impractical to perform manually. In the present study, we validated the ability of DOCTRAP software algorithms to identify inner and outer retinal boundaries. We previously have shown in eyes without DME, that DOCTRAP can be used to simultaneously segment seven retinal layers, including those described above.¹⁵ We currently are testing the ability of DOCTRAP to measure in an automated manner these boundaries and layers in eyes with DME, and other retinal diseases, to further advance the structure–function correlations. 

Acknowledgments

Supported by the National Eye Institute, NIH, Department of Health and Human Services under Contract No. HHSN263201200001C and NIH R01 EY022691; by the National Institute for Health Research's Clinical Research Network West of England; and Moorfields Biomedical Research Center as part of the Universities and National Institutes Transatlantic Eye (UNITE) consortium. 

Disclosure: A. Willoughby, None; S.J. Chiu, None; P.R. Silverman, None; S. Sina Farsiu, None; P.C. Bailey, None; H.E. Wiley, None; F.L. Ferris III, None; G. Jaffe, Heidelberg Engineering (C), Alcon (C), Neurotech (C), Roche (C)