Intradevice Repeatability and Interdevice Agreement of Ocular Biometric Measurements: A Comparison of Two Swept-Source Anterior Segment OCT Devices

Abstract

Purpose: To assess the repeatability and agreement of ocular biometric parameters measured using the Tomey CASIA SS-1000 and Heidelberg ANTERION anterior segment optical coherence tomography (AS-OCT) devices.

Methods: Both eyes of subjects 18 years of age or older were scanned three times with the CASIA and ANTERION under standardized dark lighting. One AS-OCT image along the horizontal (temporal-nasal) meridian was analyzed per eye and per scan. Pupillary diameter (PD) was within 15% for all pairwise comparisons. Anterior chamber depth, lens vault, anterior chamber width, angle opening distance, trabecular iris space area, and scleral spur angle (SSA500) were measured using manufacturer-provided image analysis software. Intraclass correlation (ICC), Wilcoxon signed-rank, and Bland-Altman analyses were performed to assess intradevice repeatability and interdevice agreement of measurements.

Results: Thirty-two eyes of 21 subjects were analyzed. There was excellent agreement (ICC >0.98) and no significant difference (P > 0.05) in PD across all comparisons. Intradevice measurement repeatability was excellent for both the CASIA (ICC range 0.93–0.99) and ANTERION (ICC range 0.97–0.99). Interdevice measurement agreement was also excellent (ICC range 0.85–0.96). Measurements within and between devices were similar (P > 0.06) for all parameters except SSA500 (P = 0.03). Linear regression and Bland-Altman plots showed the relationship was consistent across the entire range of measurements.

Conclusions: Intradevice measurement repeatability is excellent for the CASIA and ANTERION. Interdevice measurement agreement between the two devices exceeds metrics reported by previous comparison studies.

Translational Relevance: Modern swept-source AS-OCT devices produce highly repeatable measurements of ocular biometric parameters that are nearly interchangeable across devices.

Introduction

Anterior segment optical coherence tomography (AS-OCT), a form of in vivo ocular imaging with broad applications for clinical care and scientific research, has steadily advanced over the past two decades.^1–3 Early time-domain AS-OCT devices, such as the Zeiss Visante (Carl Zeiss Meditec, Dublin, CA), were slow and produced low-quality images compared to modern Fourier-domain optical coherence tomography (OCT) devices. While spectral-domain OCT devices, such as the Heidelberg Spectralis (Heidelberg Engineering, Heidelberg, Germany), provide faster imaging speeds and higher spatial resolution, their shorter wavelengths also result in reduced penetration of ocular tissues and inability to visualize the entire anterior segment in one image. The Tomey CASIA SS-1000 (Tomey Corporation, Nagoya, Japan) was the first swept-source OCT device to use a longer 1310-nm wavelength laser, which provided fast imaging speed and enhanced range of imaging depth and penetration. However, while the CASIA SS-1000 has been used extensively for scientific research, it never received US Food and Drug Administration (FDA) approval for clinical use.^1–3 The latest swept-source AS-OCT device is the Heidelberg ANTERION (Heidelberg Engineering). While the ANTERION also uses a 1300-nm wavelength laser, it supports image averaging and greater range of imaging depth than the CASIA SS-1000. 

Recent advances in AS-OCT technology have led to renewed interest in its clinical applications, especially in the field of primary angle closure disease (PACD).^4,5 When anatomical landmarks, such as the scleral spur, are marked, quantitative measurements of biometric parameters describing the position or configuration of anatomic structures can be obtained, including the width of the anterior chamber angle.^1–3 The importance of these measurements is threefold. First, biometric parameters of anatomical structures, such as the lens and iris, are risk factors for gonioscopic angle closure and PACD.^6–11 Second, these measurements could be used to track longitudinal progression of angle narrowing or response to treatments for angle closure, such as laser peripheral iridotomy (LPI).^12–15 Finally, the recent landmark Zhongshan Angle Closure Prevention trial demonstrated that the current definition of primary angle closure suspect, which comprises the vast majority for PACD, may be too broad to guide treatment with LPI.¹⁶ Therefore, AS-OCT measurements, which reflect anatomic and physiologic changes related to angle closure, could complement gonioscopy to refine the management of early stage PACD.^17,18 

In this study, we assess the intradevice repeatability and interdevice agreement of measurements by the Tomey CASIA SS-1000, a device on which many key studies on angle closure were conducted, and the newly commercialized and Conformité Européene (CE) marked Heidelberg ANTERION. Comparing measurements within and between these two devices is crucial since previous AS-OCT devices have demonstrated variable interdevice measurement agreement, ranging from poor to excellent.^19–22 We also assess the effect of image averaging, a feature that is unique to the ANTERION, on intradevice measurement repeatability. 

Methods

Ethics committee approval was obtained from the University of Southern California (USC) Medical Center Institutional Review Board. All study procedures adhered to the recommendations of the Declaration of Helsinki. All study participants provided informed consent at the time of enrollment. 

Clinical Data Collection

Consecutive patients 18 years of age and older undergoing routine eye examinations were recruited from the USC Roski Eye Institute. Exclusion criteria included history of eye procedures, such as LPI and cataract surgery, and corneal opacities that precluded gonioscopy and AS-OCT imaging. 

Each subject received a complete eye examination by a fellowship-trained glaucoma specialist (B.Y.X.), including gonioscopy and AS-OCT imaging. Both gonioscopy and AS-OCT imaging were performed prior to pharmacologic dilation in the seated position under stable dark ambient lighting at 0.1 cd/m² (Dr. Meter Digital Illuminance Meter, model-LX1330B; CA, USA). Gonioscopy was performed with a Volk four-mirror lens (model VG4HAN2; Volk Optical, Inc., Mentor, OH) and 1-mm light beam. Care was taken to avoid light falling on the pupil and inadvertent indentation of the globe. The gonioscopy lens could be tilted up to 10 degrees. The angle was graded in each quadrant according to the modified Shaffer classification system. Angle closure was defined as inability to visualize the pigmented trabecular meshwork (grades 0 and 1). 

AS-OCT imaging was performed after 3 minutes of dark adaptation. Three consecutive scans, first from the left eye and then from the right eye, were acquired using the CASIA SS-1000 or ANTERION selected at random. The process was then repeated using the other AS-OCT device. Subjects were instructed to maintain fixation with the imaged eye on the central fixation target built into each device. 

Radial scans were obtained using each device. For the CASIA, each scan produced 128 images spaced 1.4 degrees apart, each based on a single B-scan (Fig. 1). For the ANTERION, each scan in the Metrics Application image acquisition mode produced six images spaced 30 degrees apart, each an average of eight B-scans (Fig. 1). The ANTERION automatically registers the location of each B-scan based on the corneal light reflex, which allows the device to acquire and average multiple B-scans from the same location. The number of B-scans averaged to form a single image cannot be adjusted within the Metrics Application, which is the only image acquisition mode that supports measurements of biometric parameters. Therefore, unaveraged images based on single B-scans could not be measured on the ANTERION. 

Figure 1.

 

Representative AS-OCT images from the CASIA SS-1000 (top) and ANTERION (bottom).

Measurement of Biometric Parameters

One AS-OCT image per eye oriented along the horizontal (temporal-nasal) meridian was analyzed using the manufacturer-provided Tomey SS OCT Viewer (version 3.0; Tomey Corporation) or Heidelberg Eye Explorer (HEYEX, version 2.0; Heidelberg Engineering) software. Both programs automatically segmented anterior segment structures, including anterior and posterior corneal surfaces and anterior iris and lens surfaces (Fig. 1). One trained expert observer (A.A.P.) masked to the identities and examination results of the subjects confirmed the segmentation and marked the scleral spurs in each image. The scleral spur was defined as the inward protrusion of the sclera where a change in curvature of the corneoscleral junction was observed.²³ 

After both scleral spurs were marked in each image, the manufacturer-provided software automatically produced measurements of seven biometric parameters: pupillary diameter (PD), anterior chamber width (ACW), lens vault (LV), anterior chamber depth (ACD), angle open distance 500 µm from the scleral spur (AOD500), trabecular iris space 500 µm from the scleral spur (TISA500), and scleral spur angle 500 µm from the scleral spur (SSA500). PD was defined as the iris tip-to-tip distance. ACW was defined as the distance between scleral spurs. LV was defined as the perpendicular distance from the apex of the anterior lens surface to a line between scleral spurs. ACD was defined as the distance from the apex of the anterior lens surface to the apex of the corneal endothelium. AOD500, TISA500, and SSA500 have previously been defined.¹⁶ The ANTERION did not produce measurements of anterior chamber area (ACA), iris area (IA), and iris curvature (IC) at the time of this study. 

Intra- and interdevice comparisons in eyes with PD differing by more than 15% between the two scans were excluded from analysis to minimize the effects of pupil size on measurement values.²⁴ For intradevice measurement comparisons, two of three scans with the smallest difference in PDs from the same eye and device were chosen for analysis. For the interdevice measurement comparison, two scans, one from each device, with the smallest difference in PDs from the same eye were chosen for analysis. For the intraobserver measurement comparison, the grader reanalyzed a set of 32 images from each device 3 months after they were first graded. 

Statistical Analysis

Distributions of PD were assessed for normality using the Kolmogorv-Smirnov (KS) test and compared using the Wilcoxon signed-rank test to confirm that other measurements would not be affected by differences in PD.²⁴ KS and Wilcoxon signed-rank tests were then performed on the remainder of the measurements. The mean and standard deviation of all measurements were calculated for both sets of scans. The difference between the mean measurements was calculated as a percentage of the average of the mean measurements: mean 1 – mean 2 / [(mean 1 + mean 2) / 2]. Intradevice repeatability and interdevice agreement of measurements were assessed in the form of intraclass correlation coefficients (ICC) and Bland-Altman plots with limits of agreement (LoA). Intraobserver measurement repeatability was assessed in the form of ICCs. ICC values were classified based on the following convention: poor = less than 0.4, fair = 0.40 to 0.59, good = 0.60 to 0.74, and excellent = 0.75 to 1.00.²⁵ Linear regression models were used to establish the interdevice relationship between measurements. The analyses were first performed on all measurements from all eyes. The analyses were then repeated on one measurement per parameter and one eye per subject to control for intra- and intereye correlations. All data analysis was performed using MATLAB (MathWorks, Natick, MA). 

Results

Forty-four consecutive patients were recruited to participate as study subjects. Eleven subjects were excluded due to inability to tolerate or complete AS-OCT imaging. Twelve subjects were excluded due to PDs differing by more than 15% in the intra- and/or interdevice comparison(s). Thirty-two eyes of 21 subjects (10 female, 11 male) were included in the final analysis. The mean age of these subjects was 59.0 ± 19.9 years. There were 2 quadrants with grade 0 (3.1%), 9 quadrants with grade 1 (14.1%), 5 quadrants with grade 2 (7.8%), 24 quadrants with grade 3 (37.5%), and 24 quadrants with grade 4 (37.5%) on gonioscopy. Angle closure was detected in 11 of the 64 quadrants (17.2%). Both scleral spurs were detectable in all images included in the final analysis. No images were excluded due to poor image quality or segmentation errors. Each set of 32 images (one per eye) yielded 32 measurements of PD, ACD, ACW, and LV and 64 measurements of AOD500, TISA500, and SSA500. 

Intraobserver ICC values for A.A.P. reflected excellent measurement repeatability for all parameters across both devices. The ICC values for the CASIA were as follows: PD = 0.99, ACW = 0.96, ACD = 0.98, LV = 0.97, AOD500 = 0.94, TISA500 = 0.93, SSA500 = 0.96. The ICC values for the ANTERION were as follows: PD = 0.99, ACW = 0.96, ACD = 0.99, LV = 0.99, AOD500 = 0.98, TISA500 = 0.98, SSA500 = 0.98. 

Intradevice Repeatability of Measurements for the CASIA and ANTERION

For the CASIA, no set of measurements was normally distributed (P < 0.001) for any of the parameters. There was no significant difference in PD (P = 0.97) between the two sets of images (Table 1). The mean PD for was 4.74 ± 1.70 mm for set 1 and 4.74 ± 1.68 mm for set 2. There were no significant measurement differences (P > 0.06) between the two sets of images for any of the other parameters. The measurement difference as a percentage of the averaged mean ranged between 0.03% (ACD) and 4.83% (TISA500). Intradevice measurement repeatability was excellent for all parameters; ICC values ranged between 0.93 (ACW) and 0.99 (PD) with narrow 95% confidence intervals (CIs). Bland-Altman analysis revealed narrow LoAs. 

Table 1.

 

Intra-device Measurement Repeatability of the CASIA SS-1000.

 

For the ANTERION, no set of measurements was normally distributed (P < 0.0001) for any of the parameters. There was no significant difference (P = 0.39) in PD between the two sets of images (Table 2). The mean PD for was 4.87 ± 1.72 mm set 1 and 4.85 ± 1.72 mm for set 2 (Table 2). There were no significant measurement differences (P > 0.26) between the two sets of images for any of the other parameters. The measurement difference as a percentage of the averaged mean ranged between 0.01% (ACW) and 2.00% (TISA500). Intradevice measurement repeatability was excellent for all parameters; ICC values ranged between 0.96 (ACW, TISA500) and 0.99 (PD, ACD, LV) with narrow 95% CIs. Bland-Altman analysis revealed narrow LoAs. 

Table 2.

 

Intra-device Measurement Repeatability of the ANTERION.

 

Interdevice Agreement of Measurements Between the CASIA and ANTERION

There was no significant difference (P = 0.15) in PD between the two sets of images from the CASIA and ANTERION (Table 3). The mean PD was 4.76 ± 1.70 mm for the CASIA and 4.84 ± 1.75 mm for the ANTERION. There were no significant measurement differences (P > 0.06) between the two sets of images for any of the other parameters except SSA500 (P = 0.03). The ICC for PD was excellent (ICC = 0.98). Interdevice agreement of other measurements was also excellent; ICC values ranged between 0.85 (TISA500) and 0.96 (LV) with small 95% CIs. The measurement difference as a percentage of the averaged mean ranged between 0.03% (SSA500) and 7.91% (AOD500). Bland-Altman plots revealed a consistent difference between the two sets of measurements with narrow LoAs (Fig. 2). The range of slopes from the linear regression models ranged from 0.84 (ACW) to 1.09 (LV). 

Table 3.

 

Inter-device Measurement Agreement Between the CASIA SS-1000 and ANTERION.

 

Figure 2.

 

Interdevice measurement agreement between the CASIA SS-1000 and ANTERION. Each subfigure includes linear regression (left) and Bland-Altman (right) plots. Dotted line in linear regression plot represents the equivalence line and solid line represents the regression line. Bland-Altman plot includes mean difference (solid line) and limits of agreement (dotted lines). Plots are shown for PD, ACW, LV, AOD500, TISA500, and SSA500.

Results were similar when analyses were repeated using one measurement per parameter and one eye per subject (Supplementary Table S1) to control for intra- and intereye correlations. There were no significant measurement differences (P > 0.10) between the two scans for any of the parameters except SSA500 (P = 0.04). ICC values ranged between 0.86 (ACD) and 0.99 (PD) with small 95% CIs. The measurement difference as a percentage of the averaged mean ranged between 0.17% (ACW) and 3.40% (LV). Bland-Altman analysis revealed narrow LoAs. 

Discussion

In this study, we assessed the intradevice repeatability and interdevice agreement of ocular biometric measurements obtained using two swept-source AS-OCT devices: the Tomey CASIA SS-1000, a swept-source AS-OCT device that has been used in scientific research for the past decade, and the Heidelberg ANTERION, a newly commercialized and CE marked swept-source AS-OCT device. Both devices demonstrated excellent intradevice measurement repeatability despite differences in image quality and averaging. Interdevice measurement agreement was also excellent for all biometric parameters, and measurements were similar for all parameters except SSA500. We believe this study highlights important advances and limitations in AS-OCT technology for clinical care and scientific research. 

The fact that the CASIA and ANTERION both demonstrated excellent intradevice measurement repeatability is unsurprising given that this metric has effectively plateaued since the popularization of Fourier-domain OCT technology.^19–22 What is surprising, however, is that interdevice agreement of measurements between the CASIA SS-1000 and ANTERION was equal to or higher than metrics reported by previous studies comparing other AS-OCT devices. Comparisons of measurements from the swept-source Tomey CASIA2 AS-OCT device with the time-domain Zeiss Visante (Carl Zeiss Meditec) or Heidelberg Spectralis reported mean ICC values of 0.78 and 0.78, respectively, for AOD500, and 0.81 and 0.78, respectively, for TISA500.^21,22 These lower ICC values may be related to the fact that the Visante is a time-domain device with relatively poor image resolution and the Spectralis uses a shorter wavelength laser that is primarily intended for posterior segment imaging. The improvement in interdevice agreement translates into measurements that are nearly interchangeable between the ANTERION and CASIA for all parameters except SSA500. 

The generalizability and interchangeability of measurements by AS-OCT devices will become increasingly important as the physiologic significance of specific measurement values is elucidated. For example, change points in the relationship between AS-OCT measurements of angle width and intraocular pressure were identified based on data from a population-based study of Chinese Americans.¹⁷ These change points could have important implications for predicting anatomic and physiologic changes and refining current definitions of PACD. However, it is not feasible to repeat these studies for every AS-OCT device due to the time and resources required to obtain and analyze the data. If this type of measurement-specific finding was generalizable across AS-OCT devices, it could greatly enhance and expedite the clinical utility of AS-OCT imaging, especially when clinical management may depend on specific measurements, as is the case in PACD. 

Interdevice differences among measurements also highlight an important limitation related to the lack of standardized methods for applying refractive correction to AS-OCT images. Optical principles, such as Fermat's principle and Snell's law, predict the path that light takes as it travels through the cornea or is reflected by intraocular structures. These principles are applied to scale and dewarp AS-OCT images so that biometric parameters can be measured. The fact that interdevice measurement agreement was excellent suggests that corrective algorithms applied by Tomey and Heidelberg are similar. However, due to the proprietary nature of these algorithms, it is unclear what assumptions are made by device manufacturers to apply refractive correction to AS-OCT images. Researchers have also applied their own independent corrective algorithms to uncorrected AS-OCT images.²⁶ Unfortunately, there is no gold standard by which the accuracy of these algorithms and measurements can be assessed. However, the field would benefit from some consensus about how refractive correction should be applied to AS-OCT images so that the benefit of specific findings, such as the aforementioned change points in angle closure eyes, can be maximized. 

The ANTERION introduces a new feature to AS-OCT devices, image averaging, that can be used to improve image signal-to-noise ratio. Image averaging alters the quality of AS-OCT images at the cost of extended imaging time. We could not perform a direct comparison of measurements from unaveraged and averaged ANTERION images since the HEYEX Metrics Application cannot be used to obtain unaveraged images. However, intraobserver and intradevice measurement repeatability were similar for unaveraged CASIA images and averaged ANTERION images. This result indirectly suggests that image averaging has limited benefit for repeatability of scleral spur detection and measurement of the biometric parameters assessed in this study, at least when anatomic landmarks are identified by a highly experienced observer. Therefore, we recommend that AS-OCT device manufacturers optimize single B-scan acquisition modes to support anatomic studies of dynamic intraocular structures, such as the iris and lens. We also recommend that AS-OCT devices provide automated detection and registration of pupil size, as relatively small changes in pupil size can have dramatic effects on other biometric measurements.²⁴ In our study, both eyes of 12 out of 33 subjects were excluded due to PD differences of more than 15% despite carefully controlled lighting conditions. This finding suggests that physiologic processes apart from the pupillary light reflex can have pronounced effects on pupil size and consequently other biometric measurements. 

Our study has a few limitations. First, images were acquired in a relatively short time span. Differences in repeatability or agreement of measurements may be more evident if a longer time elapses between scans. Second, the study sample size was relatively small. However, the excellent correlations between intra- and interdevice measurements and their narrow confidence intervals support the adequacy of the sample size. Our study cohort also comprised subjects with a wide range of biometric measurements and gonioscopy grades. Third, we were unable to compare biometric parameters, such as ACA, IA, and IC, that are important risk factors for angle, since these are currently not available on the ANTERION. We hope that future versions of the HEYEX software will make these parameters available for analysis. Fourth, we analyzed images only from the temporal and nasal quadrants. Therefore, it is possible that repeatability and agreement of measurements differ between quadrants, especially since the superior and inferior quadrants are more challenging to image. Finally, images were analyzed by only one expert trained grader with experience grading over 25,000 images. It is feasible that features such as image averaging may be more beneficial for less experienced graders. However, given that our primary aim was to assess the intra- and interdevice repeatability of these two AS-OCT devices, adding a second grader fell outside the scope of this study. 

In summary, the Tomey CASIA SS-1000 and Heidelberg ANTERION demonstrate excellent intradevice repeatability and interdevice agreement, matching or exceeding that of previous-generation devices. While our results suggest that AS-OCT devices are moving in a direction where their measurements could be used or applied interchangeably, this practice should be exercised with caution pending the results of additional generalizability studies. In addition, neither device is FDA approved. Therefore, significant efforts are still required before swept-source AS-OCT devices and their biometric measurements become useful for routine clinical practice, especially in the management of PACD. 

Acknowledgments

Supported by Grants K23 EY029763 and P30 EY029220 from the National Eye Institute, National Institute of Health, Bethesda, MD, and an unrestricted grant to the USC Department of Ophthalmology from Research to Prevent Blindness, New York, NY. 

Disclosure: A.A. Pardeshi, None; A.E. Song, None; N. Lazkani, None; X. Xie, None; A. Huang, Heidelberg Engineering (F); B.Y. Xu, Heidelberg Engineering (F)