Conversion of Central Subfield Thickness Measurements of Diabetic Macular Edema Across Cirrus and Spectralis Optical Coherence Tomography Instruments

Abstract

Purpose: Develop equations to convert Cirrus central subfield thickness (CST) to Spectralis CST equivalents and vice versa in eyes with diabetic macular edema (DME).

Methods: The DRCR Retina Network Protocol O data were split randomly to train (70% sample) and validate (30% sample) conversion equations. Data from an independent study (CADME) also validated the equations. Bland–Altman 95% limits of agreement between predicted and observed values evaluated the equations.

Results: Protocol O included 374 CST scan pairs from 187 eyes (107 participants). The CADME study included 150 scan pairs of 37 eyes (37 participants). Proposed conversion equations are Spectralis = 40.78 + 0.95 × Cirrus and Cirrus = 1.82 + 0.94 × Spectralis regardless of age, sex, or CST. Predicted values were within 10% of observed values in 101 (90%) of Spectralis and 99 (88%) of Cirrus scans in the validation data; and in 136 (91%) of the Spectralis and 148 (99%) of the Cirrus scans in the CADME data. Adjusting for within-eye correlations, 95% of conversions are estimated to be within 17% (95% confidence interval, 14%–21%) of CST on Spectralis and within 22% (95% confidence interval, 18%–28%) of CST on Cirrus.

Conclusions: Conversion equations developed in this study allow the harmonization of CST measurements for eyes with DME using a mix of current Cirrus and Spectralis device images.

Translational Relevance: The CSTs measured on Cirrus and Spectralis devices are not directly comparable owing to outer boundary segmentation differences. Converting CST values across spectral domain optical coherence tomography instruments should benefit both clinical research and standard care efforts.

Introduction

For clinical and research purposes, central retina imaging with spectral domain optical coherence tomography (SDOCT) remains the primary method to assess anatomic disease severity and to determine management recommendations in eyes with diabetic macular edema (DME). In the DRCR Retina Network^1–4 and other studies, SDOCT measurements of central subfield thickness (CST) and changes in these metrics over time have been widely used as outcome measures and criteria for study eligibility and treatment algorithms.^1–4 

Because the algorithms to determine CST are instrument dependent, the DRCR Retina Network Protocol O developed equations to convert CST measurements from SDOCT instruments to equivalent values on time domain OCT, the primary method used at the time.⁵ However, subsequent use of SDOCT throughout the network and across clinical practices globally has been rapidly adopted owing to its higher resolution, reduced artifacts, and greater efficiency. Consequently, SDOCT systems, including Cirrus (Carl Zeiss Meditec, Jena, Germany) and Spectralis (Heidelberg Engineering, Heidelberg, Germany), are commonly used in retina practices managing DME and are currently the only accepted SDOCT systems at network sites. Because the boundaries for CST are from the internal limiting membrane to the inner third of the retinal pigment epithelium on Cirrus, to Bruchs membrane on Spectralis, and to the ellipsoid zone on Stratus,^5–7 automated CST measurements generated by each system will differ from one another when performed on the same eye on the same day for both healthy eyes and eyes with diabetic macular pathology.^8–10 Redrawing of the boundary lines provides highly similar values for retinal thickness from the images obtained from these instruments,^6,11,12 but requires a specialized reading center.⁷ 

The development of equations to convert between Cirrus and Spectralis CST measurements would allow a direct comparison of measurements from these instruments, thereby benefiting clinical care for patients whose measurements were obtained on a mix of the two instruments over time and clinical research studies by allowing either instrument for participation when using a specialized reading center is not feasible. 

This analysis develops and validates equations to convert Cirrus to Spectralis measurements and vice versa. Images from the DRCR Retina Network Protocol O participants who underwent imaging on both Cirrus and Spectralis were used for initial training and validation; images from the CADME, a prospective, multicenter, randomized clinical trial of bevacizumab versus ranibizumab for the treatment of DME, were used to further validate the models.¹³ 

Methods

Protocol O, a cross-sectional observational study, adhered to the tenets of the Declaration of Helsinki. The ethics board associated with each site provided approval. Study participants provided written informed consent. We enrolled 692 adults (1298 eyes) with type 1 or type 2 diabetes at 31 sites in the United States between August 2009 and May 2011.⁵ 

A subset of the full Protocol O cohort, consisting of eyes seen at seven sites that had access to and acquired test and retest images of each study eye on the same day on both Cirrus and Spectralis, was used for this analysis (n = 187 eyes from 107 participants). Images were acquired from August 2009 to April 2010. The order of acquisition of Cirrus and Spectralis images was based on logistic feasibility at the clinic. Only eyes judged by the investigator to have ocular abnormalities that could have affected the images were excluded (n = 8 eyes from 7 participants). The participants included in this analysis all had at least one eye with DME, but not all eyes had DME as the analysis cohort included 55 participants with only one eye with DME. 

The CADME prospective, multicenter, randomized, three-period, two-treatment, crossover trial compared monthly intravitreal injections of bevacizumab with ranibizumab for the treatment of DME.¹³ The investigators enrolled 56 adults (62 eyes) with type 1 or 2 diabetes, at least one eye with DME involving the center of the macula, Electronic-Early Treatment Diabetic Retinopathy Study visual acuity letter score of 78 to 24 (20/32–20/400), and a CST of 330 µm or greater on Cirrus OCT.¹⁴ Following a breakdown of the Cirrus machine at one clinical site, necessitating a temporary switch to Spectralis, both Cirrus and Spectralis scan pairs (n = 150 from 37 eyes of 37 participants), were captured at all protocol visits at that site from January 2014 to August 2014; these scan pairs constituted the CADME analysis dataset. All eyes were receiving monthly anti-vascular endothelial growth factor treatment during this period. 

In both studies, Cirrus macular cube 512 × 128 scans, consisting of 128 horizontal B-scans and 512 A-scans covering an area of 6 × 6 mm centered on the point of fixation, were obtained and analyzed using Cirrus software version 3 or higher (Protocol O) or version 6.0.2.81 (CADME). For Spectralis in both studies, 49 horizontal high-speed B scans covering an area of 20° × 20° centered on the point of fixation with an automatic real-time mean of 16 frames were obtained and analyzed using Heidelberg Eye Explorer version 5.6.3.0 (CADME; data not available, Protocol O). In Protocol O, the second (retest) Spectralis scan was taken using real-time image registration with the first scan as the reference. 

Images in both studies were assessed by a reading center for decentration or segmentation issues. Autograded CST measurements were used for analysis if the image was judged acceptable. If an issue was identified, the reading center corrected the CST value, and the corrected value was used in the analyses. 

Scans from the subset of participants who underwent imaging on both Cirrus and Spectralis instruments in Protocol O were used in this study to train and validate an SDOCT conversion equation. In addition, Spectralis/Cirrus image pairs from the CADME study were used to perform an independent validation of the conversion equations. 

Statistical Methods

Differences between test–retest measurements within each instrument and between instruments were analyzed using Bland–Altman methods.¹⁵ A conversion equation between Cirrus and Spectralis (and vice versa) for the 1-mm diameter central subfield was developed using data from a random sample of 70% of the Protocol O participants (training dataset), and data from the other 30% of the participants (validation dataset) were used to evaluate the performance of the equations. Equations were assumed to have a linear functional form because nonlinear models have not been found to be superior to the linear approach in previous OCT investigations. 

The proposed equations were validated by analyzing the agreement between predicted and observed values in the Protocol O validation dataset and separately in the CADME study data using Bland–Altman methods.¹⁵ In the CADME data, there were no test–retest data or participants with two study eyes, but eyes were imaged with both Cirrus and Spectralis at multiple visits over time. The Bland–Altman 95% limits of agreement (LOA) and within-eye intraclass correlations were estimated using linear mixed models with the differences between measurements as the dependent variable and eye-level random intercepts. Confidence intervals (CIs) for LOAs were estimated using the method of variance estimates recovery.^16,17 Results are presented in the original micron scale as well as in the relative (%) difference scale. Relative differences between test–retest scans are reported relative to the test-retest mean and relative differences between predicted and observed values are reported relative to the observed values. 

Sensitivity analyses were conducted to compare the conversion equations derived from the training sample with equations derived from the full sample, equations adding a participant-level random effect, and equations adding sex and sex–instrument interaction terms. All analyses were conducted in SAS (version 9.4; SAS Institute, Cary, NC). 

Results

From Protocol O, 374 CST measurement pairs from 187 eyes (107 participants) were eligible for analysis. Automatically generated CST values were used for analysis for all except 27 scans (7%) on Cirrus, for which manually graded CST values from the Reading Center were used. The mean ± standard deviation (SD) CST test–retest value was 386 ± 111 µm on Spectralis and 365 ± 113 µm on Cirrus (Table 1). The center involved (CI)-DME sex and machine-based thresholds^1,18 were met or exceeded by 72% of the scans on Spectralis and 71% on Cirrus. 

Table 1.

 

Participant and Study Eye Characteristics Overall and by Training and Test Data Subsets

 

The median (interquartile range [IQR]) absolute differences between CST test–retest values for Spectralis was 4 µm (2–12 µm) and 3 µm (1–6 µm) for Cirrus (Table 2). Ninety-five percent of the test–retest differences were within 34 µm, or 10%, of the mean test–retest value on Spectralis and 28 µm, or 8% on Cirrus.¹⁵ 

Table 2.

 

Agreement Between Test–Retest Scans and Between Observed and Predicted (Converted) Values

 

The CADME study data included 150 Spectralis/Cirrus measurement pairs from 37 eyes (37 participants) obtained from a median (IQR) of 3 visits (2–5 visits) per participant (max = 10). The mean ± SD CST was 313 ± 81 µm on Spectralis and 304 ± 80 µm on Cirrus. 

Comparison of Cirrus and Spectralis Images

In Protocol O, CST on Spectralis was greater than CST on Cirrus by a mean ± SD of 21 ± 30 µm with a median (IQR) difference of 17 µm (11–28). The 95% LOA for Spectralis–Cirrus differences were −37 to 80 (95% confidence interval, [−44 to −31 µm] to [74 to 86 µm]), based on Spectralis/Cirrus test pairs and Spectralis/Cirrus retest pairs, after accounting for eye-level correlation (Fig. 1). In CADME, the CST on Spectralis was greater than the CST on Cirrus by a mean ± SD of 9 ± 8 µm with a median (IQR) difference of 8 µm (6–13 µm) and 95% LOA for Spectralis/Cirrus CST differences of −8 to 25 µm (Fig. 1). The within-eye intraclass correlations for differences between Spectralis and Cirrus measurement pairs was 0.67 in Protocol O and 0.22 in CADME. The Spectralis/Cirrus differences did not appear to be associated with thickness in Protocol O data (β = 0.01 for regression of difference on Spectralis test-retest mean, P = 0.55) or CADME (β = 0.01 for regression of difference on Spectralis, P = 0.30); however, the absolute difference increased as the measurement increased in Protocol O data (β = 0.04 for regression of absolute difference on Spectralis test–retest mean, P = 0.01) and CADME data (β = 0.02 for regression of absolute difference on Spectralis, P = 0.01), suggesting greater variability in the differences between the instruments in eyes with a thicker CST. 

Figure 1.

 

Relationship between observed Spectralis versus observed Cirrus over the identity line in Protocol O (A) and in CADME (B) and Bland–Altman plots of machine differences in Protocol O (C) and CADME (D).

Conversion Equation Derivation and Sensitivity Analyses

Protocol O training data used 262 measurement pairs from 131 eyes (75 participants). The conversion equations developed are:  

\begin{eqnarray*}Spectralis = 40.78 + \;0.95\; \times Cirrus\end{eqnarray*}

 

\begin{eqnarray*}Cirrus = 1.82 + 0.94\; \times Spectralis\end{eqnarray*}

 

The equations are not the inverse of each other; each equation was estimated to minimize the error of the converted values (dependent variable). Switching the dependent and independent variables in simple linear regression typically does not result in inverse equations, except when all data points fall exactly on a line. The within-eye intraclass correlations was 0.60 for the conversion of Cirrus to Spectralis values and 0.61 for the conversion of Spectralis to Cirrus values. 

Conversion Equation Validation

The Protocol O validation dataset included 112 measurement pairs from 56 eyes (32 participants). On Spectralis, the observed mean ± SD was 394 ± 135 µm and the predicted mean ± SD was 392 ± 132 µm. On Cirrus, the observed mean ± SD was 372 ± 140 µm and the predicted mean ± SD was 372 ± 126 µm. Predicted values were within 10% of the observed values for 101 (90%) of the Cirrus converted to Spectralis pairs and 99 (88%) of the Spectralis converted to Cirrus pairs. Adjusting for within-eye correlations, 95% of CST on Cirrus converted to Spectralis are expected to be within 69 µm or 17% (95% confidence interval, 14%–21%) of the CST directly observed on Spectralis; and 95% of CST on Spectralis converted to Cirrus are expected to be within 73 µm or 22% (95% confidence interval, 18%–28%) of the CST directly observed on Cirrus (Table 2 and Fig. 2). Sex- and machine-specific CI-DME cut off points on Cirrus and Spectralis are shown in Figure 2 to lie close to values estimated by the conversion equations. 

Figure 2.

 

Protocol O validation data evaluation of the Cirrus to Spectralis conversion equation (A) and Spectralis to Cirrus conversion equation (B) and Bland–Altman plots of differences between predicted and observed CST on Spectralis (C) and between predicted and observed CST on Cirrus (D). CI-DME is defined as follows by CST according to OCT machine and sex: Heidelberg Spectralis 305 µm or greater in women and 320 µm or greater in men; Zeiss Cirrus 290 µm or greater in women and 305 µm or greater in men.

In the CADME data, on Spectralis, the observed mean ± SD was 313 ± 81 µm and the predicted mean ± SD was 329 ± 76 µm. On Cirrus, the observed mean ± SD was 304 ± 80 µm and the predicted mean ± SD was 296 ± 76 µm. Predicted values were within 10% of the observed values in 136 (91%) of Cirrus converted to Spectralis pairs and in 148 (99%) of Spectralis converted to Cirrus pairs. The 95% LOA for differences between predicted and observed values were −3 to 33 µm (relative difference, −2%–12%) on Spectralis and −28 to 10 µm (relative difference, −8% to 3%) on Cirrus (Table 2 and Fig. 3). Although the conversion equations tended to overestimate Spectralis and underestimate Cirrus in the CADME data, the LOAs between predicted and observed values calculated with the CADME data were within the LOAs estimated using the Protocol O validation data. Overall, when comparing CST scans from the Protocol O validation and CADME datasets together, 90% of the Cirrus converted to Spectralis values were within 10% of the paired Spectralis scans and 94% of the Spectralis converted to Cirrus values were within 10% of the paired Cirrus scan. 

Figure 3.

 

CADME data evaluation of the Cirrus to Spectralis conversion equation (A) and Spectralis to Cirrus conversion equation (B) and Bland–Altman plots of differences between predicted and observed CST on Spectralis (C) and between predicted and observed CST on Cirrus (D).

Sensitivity Analyses

Sensitivity analyses demonstrated that the recommended conversion equations derived from the training data closely resembled conversion equations derived from the full Protocol O sample. When CST varied from 200 to 800 µm, the mean absolute difference in converted CST predictions based on the equations derived from full data versus the training data was less than 3 µm, with a maximum difference of less than 8 µm for conversions in either direction. In the training data, allowing data from participants with two study eyes to be correlated had little influence on the conversion equation estimates and sex did not seem to modify the converting relationship in either direction (Supplementary Table S1 and Supplementary Figs. S1 and S2). Last, different random 70/30 splits of the Protocol O data resulted in similar estimates of the conversion equation parameters and LOAs between predicted and observed values (Supplementary Table S2 and Figs. S3 and S4). 

Discussion

Cirrus CST measurements are thinner on average owing to differences in outer boundaries (inner third of the retinal pigment epithelium vs. the Bruch's membrane), making it unsuitable to directly substitute one value for the other. This study provides equations to convert Cirrus CST measurements to equivalent Spectralis values and vice versa that perform well in validation data sets from DRCR Protocol O and CADME, with approximately 90% of the predicted conversions within 10% of the observed values overall. 

The widespread use of SDOCT to assess central retinal thickness for clinical and research efforts suggests that these equations, which allow interoperability from one instrument to another, may be widely applicable. In 2019, 64% of DRCR sites had a Spectralis SDOCT instrument and 41% had a Cirrus instrument, with 53% and 47% of scans acquired on Spectralis and Cirrus, respectively. These instruments command a major market share in ophthalmic imaging for clinical practices worldwide. 

The agreement between measurements observed on different instruments is limited by the repeatability (degree of difference expected from measurement variability) within each instrument. The 95% LOAs for relative differences between replicate scans (repeatability) were 10% with Spectralis and 8% with Cirrus. Although the conversion models for CST improved the agreement between the Spectralis and Cirrus instruments, the conversion models also introduced additional variability because the model predictions are not perfect. The 95% LOAs for relative differences among conversions (17% with Cirrus converted to Spectralis; 22% with Spectralis converted to Cirrus) are acceptable given the thresholds for within-instrument measurement errors alone were approximately 10%. Findings from this study support the use of these conversion equations to transform CST values on individual and group levels in research. Just as there can occasionally be large disparities in measurement of CST by Cirrus versus Spectralis, there can be substantial differences between predicted and observed values for individual scans. Although the median differences between converted and measured values in the Protocol O validation set were only −2 and 4 µm, the absolute differences between these values were greater than 80 µm in approximately 6% of scans. For group means, conversion errors on individual scans will tend to balance out as the sample size increases. Converted values are not recommended for CI-DME classification. Observed CST and the corresponding gender and machine-specific thresholds should determine CI-DME status.^1,18 

The CADME data provide a useful comparison set in which to test these equations. In contrast with the Protocol O cohort, the CADME participants were followed longitudinally and Cirrus and Spectralis images were obtained after 2 or more months of treatment with intravitreal anti-vascular endothelial growth factor. Equations to convert Cirrus to Spectralis values and vice versa that were trained from the CADME study provide similar results to those proposed in the current study. However, the equations trained on Protocol O data may be more generalizable to a wider range of eyes with DME because Protocol O contained more than 3.5-fold the number of eyes available for training than the CADME study. These conversion equations reliably predicted the observed CADME data, which suggests they are applicable to repeated OCT scans from the same eye over time. 

A strength of this study is that these equations were trained and tested in prospectively collected scans from a large cohort of persons with DME from a diverse population of patients enrolled at clinical sites across the United States and Great Britain. Additionally, OCT scans were obtained by certified imagers and manual segmentation correction was performed by a Reading Center as needed to obtain the most accurate CST values. This study has several limitations. First, the application of these equations depends on the manual correction of segmentation when necessary. Second, conversion equations are less precise than specialized software used at reading centers, where retinal boundaries are redrawn at a common reference point.⁷ Third, it is unknown whether similar results would be obtained in a different cohort or using different scan protocols from the ones used by the DRCR Retina Network and CADME. Finally, these equations do not address the conversion of OCT variables other than CST, such as total retinal volume or thickness of noncentral macular subfields, or CST conversion in eyes with conditions other than DME. 

This study provides validated conversion equations to transform Spectralis to Cirrus CST values and vice versa; these equations are suitable for the transformation of values from individual scans as well as averaged values from cohorts of eyes with DME. Which conversion equation to use may be based on preference or the prevalence of devices, where the chosen equation converts values to the equivalents of the instrument with the most observations. Although previous work enabled the conversion of Spectralis and Cirrus measurements to time domain OCT equivalents, this method has proven suboptimal for reporting current study results, because time domain OCT is largely obsolete. Thus, these equations will be used to report future CST results in eyes with DME from the DRCR Retina Network in SDOCT equivalents. The ability to convert CST values from one SDOCT instrument to another will enable better interpretation and comparison of individual patient OCT images and facilitate future observational studies and clinical trials that use these different instruments for evaluation of eyes with DME. 

Acknowledgments

Supported by the National Eye Institute and the National Institute of Diabetes and Digestive and Kidney Diseases of the National Institutes of Health under Award Number UG1EY014231. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. 

Disclosure: J.K. Sun, None; K. Josic, None; M. Melia, None; A.R. Glassman, None; C. Bailey, None; K.V. Chalam, None; E.Y. Chew, None; C. Cukras, None; S. Grover, None; G.J. Jaffe, None; R. Lee, None; J.S. Nielsen, Clinical/Epi Research for Digital Diagnostics; D.J.S. Thompson, None; H.E. Wiley, None; F.L. Ferris, None