Comparison of Visual Acuity Results Between ATS-HOTV and E-ETDRS Testing Methods in Children With Optic Pathway Gliomas

Abstract

Purpose: To determine if visual acuity (VA) outcomes are comparable using the amblyopia treatment study HOTV protocol (ATS-HOTV) and electronic Early Treatment of Diabetic Retinopathy Study (E-ETDRS) protocol in children with optic pathway gliomas (OPGs).

Methods: Children enrolled in a prospective study of OPGs were eligible if they completed both the ATS-HOTV and E-ETDRS during the same visit. The contribution of age, testing order, having neurofibromatosis type 1, visual field loss, and circumpapillary retinal nerve fiber layer thickness to VA difference were assessed using generalized estimating equations to account for the intereye correlation.

Results: Forty-eight children (median age, 10.3 years; range, 5.2–17.1 years; 49% female) met inclusion criteria and contributed 93 study eyes at their initial visit. Eleven patients (22 eyes) had more than one study visit, permitting longitudinal evaluation. ATS-HOTV measures of VA were higher than E-ETDRS at the initial (0.13 ± 0.36 vs. 0.23 ± 0.39 logarithm of the minimum angle of resolution [logMAR], P < 0.001) and all visits (0.13 ± 0.34 vs. 0.21 ± 0.36 logMAR, P < 0.001). VA remained significantly higher with ATS-HOTV regardless of test order, but the mean difference between tests was most profound when tested with ATS-HOTV first compared to E-ETDRS first (P < 0.001).

Conclusions: VA results differ significantly between the ATS-HOTV and E-ETDRS testing methods in children with OPGs. Given the wide range of ages and testing ability of children, one VA testing method should be used throughout longitudinal OPG clinical trials.

Translational Relevance: It is imperative that age-appropriate VA testing methods are standardized across all pediatric OPG clinical trials.

Introduction

Visual acuity (VA), measured using standardized testing protocols, serves as the most common primary outcome measure in ophthalmology clinical trials. The computer-based version of the Early Treatment of Diabetic Retinopathy Study (E-ETDRS) protocol has long been accepted as the gold standard for VA measurement in adults and older children,^1,2 while the amblyopia treatment study (ATS) HOTV visual acuity protocol has become the gold-standard measure for a younger pediatric population.³ Unfortunately, children with optic pathway gliomas (OPGs)—low-grade tumors that can occur anywhere along the optic nerves, chiasm, or optic tracts—enter clinical trials across a wide age range (e.g., 1 to 10 years), making it important to select one uniform VA testing method that all patients can complete.^4–7 Using more than one recognition VA testing method to determine the same primary outcome in a clinical trial can be problematic given the inherent differences between tests. Furthermore, children with OPGs frequently have developmental delay and other medical comorbidities that may further exacerbate known differences between E-ETDRS and ATS-HOTV results that have been previously described in large groups of otherwise healthy children with and without amblyopia.^1,8 

The importance of standardized VA testing protocols has become increasingly relevant to children with OPGs as new therapies are being tested in clinical trials with an emphasis on including VA as a primary or secondary outcome measure.^6,7,9,10 Ultimately, for clinicians to make use of these data when making treatment decisions for patients outside of a multicenter clinical trial, they must be able to compare the clinical trial VA results to the standard-of-care VA result. To address these issues, we sought to compare VA outcomes assessed using the ATS-HOTV protocol and E-ETDRS protocol in children with OPGs. 

Methods

This study was approved by the Children's Hospital of Philadelphia Institutional Board of Review. Protocols followed the tenets of the Declaration of Helsinki. All data was collected and stored according to Health Insurance Portability and Accountability Act guidelines. Children enrolled in a prospective OPG study conducted by the Neuro-Ophthalmology service at the Children's Hospital of Philadelphia were eligible for inclusion. Patients were included if they had a diagnosis of an OPG (either sporadic or associated with neurofibromatosis type 1 [NF1]) and had completed both ATS-HOTV and E-ETDRS testing, as well as formal perimetry and optical coherence tomography (OCT), all during the same study visit. Formal perimetry and OCT were included in the analysis in case visual field loss or the severity of optic neuropathy would affect the ability to complete the different VA tests. Patients were excluded if they had amblyopia or a history of elevated intracranial pressure with or without subsequent optic atrophy. Clinical and demographic data were extracted from the patient’s clinical chart. 

Visual Acuity

VA testing was performed at the beginning of the visit using the same computerized testing system (M&S Technologies, Inc., Niles, IL, USA) in which the computer monitor was calibrated to the standardized testing distance. The examination chair was raised to ensure the patient’s eyes were in plane to the monitor and the room lights were dimmed. The operator entered the patient’s response using a remote control. The ATS-HOTV and E-ETDRS protocols were conducted during the same visit as previously described.^1,12 One relevant difference between methods is that the ATS-HOTV assessment can done by naming or facilitated by using a matching card, whereas E-ETDRS is done exclusively by naming. For both protocols, patients were encouraged to provide an answer even when they were not sure. Patients completed testing using their current glasses, if needed. Testing order was not predetermined. All VA results were reported as logarithm of the minimum angle of resolution (logMAR). 

Visual Field Testing

Patients performed either a Humphrey visual field 24-2 SITA fast protocol or Goldmann kinetic perimetry depending on their ability to reliably fixate or VA. Visual fields were evaluated per eye and classified as abnormal if a defect occurred in at least one quadrant. Goldmann abnormalities were defined as a restriction greater than 10 degrees along three contiguous meridians while Humphrey abnormalities required at least six stimuli points below 0.5%. 

Optical Coherence Tomography

All participants underwent OCT imaging using the Spectralis (Heidelberg Engineering, Heidelberg, Germany). The circumpapillary retinal nerve fiber layer (cpRNFL) was measured using the standard 3.4-mm circle centered over the optic nerve head with an automatic real time of 16. The accuracy of all OCT scans was verified and corrected by manual segmentation, when necessary, by the same operator (AG). An abnormal cpRNFL was analyzed as a binary variable (normal versus abnormal defined as thickness <80 microns¹¹) as well as severity levels of axonal loss (normal, >80 microns; mild atrophy, 60–80 microns; and moderate to severe atrophy, <60 microns). 

Statistical Analysis

The agreement of VA between HOTV and ETDRS was assessed by calculating mean difference (HOTV – ETDRS), 95% limits of agreement, and P value for the VA difference. Generalized linear models were used to determine factors associated with VA difference, including age, testing order, NF1, visual field loss, and cpRNFL thickness. In these generalized linear models, the intereye correlation and longitudinal correlation were accounted for using generalized estimating equations. The generalized linear models provided both the within-group P value for determining whether there is a significant difference between HOTV and ETDRS within a specific group and between-group P value for determining whether the VA difference between HOTV and ETDRS was significant across specific groups of a factor. Age was analyzed as a continuous variable as well as a categorical variable: 6 to 8 years old, 8.1 to 12 years old, or 12.1 to 17 years old. We performed these analyses using data from the initial visit (91 eyes) and data from all visits (145 eye visits). In addition, we performed the longitudinal analysis of VA change among 22 eyes of 11 patients who had at least one follow-up visit. All statistical analyses were performed in R statistical package, and two-sided P < 0.05 was considered statistically significant. 

Results

Forty-eight children (median age, 10.3 years; range, 5.6–16.1 years; 49% female) met inclusion criteria and contributed 91 study eyes at their initial visit. Eleven patients (22 eyes) had more than one study visit (38 total visits after the initial visit; mean, 3.45 visits; range, 2–9), permitting longitudinal analysis. Thirty-seven (77%) of the children carried a diagnosis of NF1. 

In analyzing VA of 91 study eyes at their initial visit, higher VA results occurred when testing with ATS-HOTV (0.13 ± 0.36 logMAR) than E-ETDRS (0.23 ± 0.39 logMAR), with a mean difference of −0.10 logMAR (95% limits of agreement, −0.32–0.13; P < 0.001, Table). Figure 1 illustrates the comparison of VA between E-ETDRS and HOTV testing. Sixteen patients (33.3%) demonstrated a ≥0.2 logMAR difference between VA testing methods: 11 in one eye and 5 in both eyes. Nine of these 16 patients that demonstrated a ≥0.2 logMAR difference between VA testing methods had normal baseline VA (e.g., baseline VA better than 0.2 logMAR), whereas 7 had abnormal baseline VA (e.g., baseline VA worse than 0.2 logMAR). Baseline VA and the presence/absence of a visual field deficit were not associated with a ≥0.2 logMAR difference between VA testing methods in both univariable and multivariable analysis (P = 0.38 and P = 0.41, respectively). Of the 11 patients who demonstrated a ≥0.2 logMAR this difference in VA in only one eye, 10 had OPGs that impacted both sides of their visual pathway, such as both optic nerves or the optic chiasm. One patient had a unilateral left optic nerve glioma, but the 0.2 logMAR difference between methods occurred in his unaffected right eye. 

Table.

 

Mean Visual Acuity Discrepancy Between HOTV and E-ETDRS Testing

 

Figure 1.

 

Comparing visual acuity differences in logMAR between ATS-HOTV and E-ETDRS (n = 91 eyes).

Univariable analysis did not find any significant associations between VA differences (ATS-HOTV versus E-ETDRS) and NF1 diagnosis (present versus absent), visual field deficits (present versus absent), OCT cpRNFL as a binary variable (<80 or ≥80 microns), OCT cpRNFL severity of axonal loss (normal, mild, or moderate to severe atrophy), and testing order. However, the mean VA from ATS-HOTV was significantly higher than that from E-ETDRS within each subgroup defined by each of these variables (Table). Age modeled as either a continuous variable or as a categorical variable (e.g., 6–8 years, 8.1–12 years, and 12.1–17 years) was not associated with VA differences between testing methods (P = 0.40 and P = 0.16, respectively; Figs. 2a, 2b). 

Figure 2.

 

Differences in visual acuity between ATS-HOTV and E-ETDRS (HOTV minus ETDRS) versus age (a) and age group (b). (a) The LOWESS (locally weighted scatterplot smoothing) smoothed line (blue) and 95% confidence limits (shaded area) are shown. (b) The boxplot for differences in visual acuity is shown for each age group. The three horizontal lines for the box represent the values for the first quartile, median, and the third quartile. The minimum and maximum (excluding outliers) values are shown by lower and upper whiskers.

Eleven patients (23%) had multiple visits using both VA testing methods, so the above univariable analysis was repeated with 145 total eye visits. Similar to the single-visit analysis, higher VA results occurred when testing with ATS-HOTV (0.13 ± 0.34 logMAR) than E-ETDRS (0.21 ± 0.36 logMAR), with a mean difference of −0.08 logMAR (95% limits of agreement, −0.31 to 0.15; P < 0.001). Again, in univariable analysis, the following factors did not significantly affect differences between VA testing methods: age, NF1 diagnosis, visual field deficits, OCT cpRNFL as a binary variable, and OCT cpRNFL severity of axonal loss (normal, mild, or moderate to severe atrophy; P > 0.05, all comparisons), however. VA testing method order did reach significance (P = 0.002) as those tested first with ATS-HOTV had a greater mean difference between methods (−0.12 ± 0.11) compared to those tested first with E-ETDRS (−0.06 ± 0.12), and this difference remained statistically significant in multivariable analysis (P = 0.01) after adjusting by age, NF1 diagnosis, visual field deficits, and OCT cpRNFL severity of axonal loss. 

Among 22 eyes with more than one visit (Fig. 3), we calculated the longitudinal VA change between the first visit and last visit for each eye from each VA testing method and compared their longitudinal VA change between ATS-HOTV and E-ETDRS. Within the median interval of 617 days between the first and last visit, the mean ± SD of VA logMAR change was 0.03 ± 0.22 for ATS-HOTV and −0.03 ± 0.24 for E-ETDRS, and their difference was not significantly significant (P = 0.17). 

Figure 3.

 

Within-patient changes in VA (logMAR) over time for ATS-HOTV and E-ETDRS testing protocols.

Discussion

The goal of this study was to compare results from the two most common VA testing methods in children with OPGs. Our results indicate VA is significantly higher when tested with the ATS-HOTV protocol compared to the E-ETDRS protocol regardless of age, NF1 diagnosis, visual field deficits, or OCT measures of the cpRNFL thickness. One-third of patients demonstrated a ≥0.2 logMAR difference between VA testing methods. 

The higher VA with the ATS-HOTV protocol compared to E-ETDRS has been reported in children with amblyopia. In a study of amblyopic children aged 5 to 11 years, Birch et al.¹ reported that the ATS-HOTV tended to have higher VA by 0.068 logMAR, or the equivalent of 3.4 letters. In 20% of their treated amblyopic eyes, ATS-HOTV tested ≥0.2 logMAR lower than E-ETDRS, with a trend related to the density of amblyopia, whereas only 4% of the fellow nonamblyopic eyes demonstrated a similar overestimation. Likewise, the study by Rice et al.¹² of children 5 to 12 years old with amblyopia yielded similar results, reporting that the ATS-HOTV protocol had a −0.08 logMAR difference, which is the equivalent of three letters on a standard logMAR chart. In comparison with Birch et al.¹ and Rice et al.,¹² our results found that there was a −0.10 logMAR difference with ATS-HOTV in children with OPGs. Furthermore, 33.3% of children tested ≥0.2 logMAR lower with ATS-HOTV in our study, suggesting a higher degree of variability in children with OPGs compared to those otherwise healthy children with amblyopia.^1,2 

The above differences between our study and others are relevant when considering that OPG treatment decisions are frequently made when a change of ≥0.2 logMAR is detected—making the differences in results between VA methods unacceptable. As an example, if during a clinical trial a patient transitioned from ATS-HOTV to E-ETDRS and the VA worsened by 0.2 logMAR, the trial would consider this a treatment failure and the patient would be taken off study or switched to another chemotherapy, whereas in reality, the change in VA was due to differences in VA testing methods. Thus, it seems prudent that patients are tested using the same VA method throughout the trial or when being monitored outside of a clinical trial. 

We speculate that there may be several contributing factors that account for the ATS-HOTV method providing higher VA results compared to E-ETDRS. First, the HOTV protocol uses four optotypes for the patient to select on a matching handheld card. Each of the four optotypes are quite dissimilar in shape. In contrast, the E-ETDRS has the option of 10 different Sloan letters, and no matching card is provided. Furthermore, the young patient may well believe that there are 26 potential letters, thereby further decreasing the probability of a correct guess by random chance. Both VA testing protocols used a forced-choice paradigm in which the child is asked to guess if they were uncertain. Therefore, the probability of choosing the correct answer is much higher with the ATS-HOTV protocol than the E-ETDRS. Another possibility is that more letter presentations are required in the E-ETDRS protocol compared to ATS-HOTV, which could contribute to fatigue and inattention. Additional support for the potential fatigue effect is our finding that the VA difference was larger when ATS-HOTV was tested first. The impact of fatigue on VA measurements is particularly important in children with OPGs, as many of them are at risk for cognitive/behavioral deficits.¹³ Last, it is conceivable that baseline VA, the presence/absence of a visual field defect, or even tumor location could be associated with greater variability between methods. Interestingly, our post hoc analysis found no significant association between these factors and the variability between VA testing methods. 

Since children with OPGs frequently manifest an optic neuropathy and visual field defects, it is conceivable that these conditions could affect VA results. We evaluated these factors in our univariable and multivariable models. Neither the presence or absence of visual field loss nor the presence/magnitude of cpRNFL thickness were shown to affect the VA difference between the testing protocols. 

Several limitations should be considered when interpreting our results. While the patients were enrolled in a prospective study of children with OPGs, testing order was not randomized and done at the discretion of the examiner. However, a near-equal number of patients were tested first with ATS-HOTV compared to E-ETDRS (48.3% vs. 51.7%, respectively), arguing against any particular testing bias. Next, our median patient age was just over 10 years, which is much older than patients undergoing initial treatment for the OPG,^14–16 whereas it is a similar age to patients being treated for recurrent or refractory OPGs.^17,18 Finally, our modest sample size may not have provided enough statistical power to detect factors associated with VA difference between VA testing methods. 

Determining comparability of VA with the ATS-HOTV and E-ETDRS procedures in children with OPGs is vital as differences can complicate short-term and longitudinal assessment of VA. Our results indicate that transitioning from one testing protocol to another throughout the course of a patient’s treatment, whether during clinical care or when participating in a clinical trial, can be problematic. Clinicians whose patients transition from ATS-HOTV to E-ETDRS need to be aware that there may be a performance decline when moving from the former to the latter. As a result, we recommend that clinicians use ATS-HOTV as a standardized VA assessment protocol across all ages to provide a more accurate comparison, since transitioning between testing methods would confound results. Although ETDRS is considered the gold standard for VA testing in older children and ATS-HOTV testing is intended for younger children,^13,19 our study did not directly compare the feasibility between these methods across age groups. 

In conclusion, the ATS-HOTV protocol leads to a slightly higher VA when compared to E-ETDRS. Given the wide range of ages and testing ability of children enrolled in OPG clinical trials, along with differences in VA testing methods, one consistent method should be used throughout the entire trial as combining methods would be inaccurate. 

Acknowledgments

Supported by National Institutes of Health grants EY022673 (RAA), EY029687 (RAA), and P30 EY001583 (G-SY). 

Disclosure: R.A. Avery, None; C. Go, None; M.J. Fisher, None; G.T. Liu, None; A. Garcia, None; M. Richter, None; B. McGeehan, None; G.E. Quinn, None; G.-S. Ying, None