Reducing Variability of Perimetric Global Indices from Eyes with Progressive Glaucoma by Censoring Unreliable Sensitivity Data

Abstract

Purpose: Recent evidence suggests that increasing perimetric contrast all the way to 0 dB may not be clinically useful. This study examines whether raising the floor for point-wise sensitivities affects the ability of global indices to detect change.

Methods: Longitudinal data from eyes with progressive glaucoma were used. Point-wise sensitivities were censored at various cutoffs (12–19 dB). At each cutoff, mean deviations (MD) were recalculated using censored sensitivities, called censored mean deviation (CMD). Both MD and CMD were fitted using a linear model. MD and CMD rate of changes (signal) and the standard deviations (SD) of the residuals (noise) were obtained from the fitted models. The linear signal to noise ratio (LSNR) for MD (LSNR_MD) and CMD (LSNR_CMD) were compared. Additionally, at each cutoff, the ratios of LSNR_CMD to LSNR_MD were calculated and tested.

Results: CMD provided significantly (P <0.05) better LSNR than MD when using any point-wise sensitivity cutoff between 15–19 dB for progressing eyes. Moreover, the ratios of LSNR_CMD to LSNR_MD were significantly (P <0.05) greater than 1 at all cutoffs from 15–19 dB.

Conclusion: This study demonstrates that censoring is an effective tool to reduce variability at low sensitivities for progressing eyes.

Translational Relevance: This study suggests that 15–19 dB could be a more suitable endpoint for perimetric testing algorithms.

Introduction

Standard automated white-on-white perimetry (SAP) has been a benchmark test for assessing visual fields (VF) in glaucoma. Unfortunately, SAP measurements are known to become considerably more variable when VF damage is present.^1–6 The intrinsic variability associated with this test, especially in areas manifesting damage, masks true glaucomatous VF progression, making it difficult for clinicians to assess true progression rates. Such variability limits the usefulness of functional testing in glaucoma. 

Given this limitation, reducing the variability of SAP testing is an important objective. Various approaches have been implemented to address this issue. For example, variability in SAP data can be reduced through improved data acquisition methods.⁷ Alternatively, post-processing techniques, such as spatial filtering of currently available data^8–11 appear to reduce variability or choosing a more valid statistical model,¹² explains variability in the SAP data to some degree. Furthermore, a recent study¹³ recommended using Goldmann size V stimuli instead of size III to produce more reliable VF data later into the glaucomatous disease process. 

In regions damaged by glaucoma, the 95% test–retest confidence intervals around VF sensitivities are wide.¹ A recent paper by Gardiner et al.¹⁴ examined the reliability of low sensitivities in glaucomatous eyes. They concluded that clinical VF testing appeared to be unreliable when locations had estimated sensitivities below approximately 15–19 dB, and recommended restricting analyses to locations with sensitivities Display Formula\( \ge \)19 dB. However, their results focused on point-wise sensitivities. No study to date has examined whether imposing such a cutoff reduces the variability observed in global indices, such as mean deviation (MD), and whether this would result in a loss of essential clinical information. The overall goal of this study is to identify the optimal floor for point-wise VF sensitivity measurements for determining global VF progression rates and significance when extending the previous point-wise results to the global index MD. 

Methods

Data

We used data from the ongoing Portland Progression Project conducted at Devers Eye Institute in Portland, OR (see Pathak et al.¹² for a detailed description). Participants were tested every 6 months with SAP using the Humphrey Field Analyzer II (Carl Zeiss Meditec Inc., Dublin, CA), 24–2 test pattern and standard testing protocols, using the SITA standard algorithm. Only reliable VF tests (≤15% false-positives, ≤33% false-negatives, and fixation losses) were included. The study adhered to the tenets of the Declaration of Helsinki, and local institutional review boards approved the protocol. 

To address objectives of this study, we censored point-wise VF sensitivities at various cutoff levels from 12–19 dB. These cutoffs were chosen in order to cover the range of cutoffs that Gardiner et al.¹⁴ suggested in their paper. The censoring was achieved by replacing VF sensitivities below the specified cutoff by the given cutoff value. For example, to censor sensitivities at 19 dB, all sensitivities below 19 dB were replaced by 19 dB. After censoring the point-wise sensitivities at the given cutoff, total deviations were calculated at each test location. The censored MDs (CMDs) were derived by taking the mean of all total deviations. We then used longitudinal series of MD and CMD to calculate a linear signal–to–noise ratio (LSNR). The LSNR was defined using the same approach described by Gardiner et al¹⁴; signal is defined as the rate of change of MD (or CMD) from linear regression and noise is defined as the standard deviation (SD) of residuals from this fitted model. A more negative LSNR indicates that it is easier to distinguish true deterioration from measurement variability. Hence, a more negative LSNR of CMD data compared with MD would provide evidence that censoring reduces the variability and improves the ability to monitor change. 

Statistical Analysis

All statistical analyses were performed using freely available statistical software R (R Foundation, Vienna, Austria). Initially at each cutoff, only eyes with eight or more VFs in their longitudinal sequence and having at least one point-wise sensitivity below the given cutoff at any of the 52 nonblindspot 24–2 test locations were included in the analyses. Both MD and CMD data from individual eyes were then fitted using the following linear model implementing generalized estimation equation with autoregressive AR(1) type within eye error structure.¹²  

\begin{equation}\eqalign{&M{D_i}\left( {or{\rm{\ C}}M{D_i}} \right) = \alpha + \beta \times t + {\varepsilon _t}{\rm{\ }} \cr&i = 1,2,...,N;\ {\rm{t}} = 1,2,...,{T_i};\ {\varepsilon _t}\sim N(0,{\Sigma _{{T_i} \times {T_i}}}) \cr} \end{equation}

where Display Formula\(i\)represents number of eyes included in the analyses, and Display Formula\(t\) is an indicator representing the number of test time points per eye. The model parameters Display Formula\(\alpha {\rm{,\ \beta }}\) are the intercept and slope respectively, and Display Formula\({{\rm{\varepsilon }}_t}\) are the errors associated with the fitted line. The model's errors are assumed to be temporally correlated according to a continuous autoregressive (CAR1) model, wherein the correlation between two residuals derived from the same eye decreases with the amount of time between them.  

The MD and CMD rates of change (signal), the corresponding SDs of the residuals (noise) and the associated P value of the signal were obtained for each eye from the fitted model. In the final analysis, we selected only those eyes that displayed significant deterioration over time that is only eyes with a significant (Display Formula\(P\)<0.05) negative MD and CMD rate of change were selected for further analyses. Furthermore, at each cutoff, the ratio of LSNR_CMD to LSNR_MD was also calculated. If censoring VF data was effective at reducing the variability then: Display Formula\(\left( i \right)\) LSNR_CMD would be significantly lower than LSNR_MD at the given cutoff and Display Formula\(\left( {ii} \right)\) the LSNR_CMD to LSNR_MD ratio would be significantly greater than 1. We tested the above two hypotheses using the Wilcoxon matched pairs test. 

Results

Data from 270 eyes with series of at least eight visits were available. The number of eyes included in analyses, however, varied greatly depending on the cutoff used, because eyes were required to have a significant worsening of both MD and CMD when using that cutoff. At cutoff 19 dB, a total of 133 eyes with progressive glaucoma (defined as a significant rate of worsening of MD and CMD) were available. Table 1 presents the characteristics of the study population when the cutoff was set at 19 dB, restricted to the eligible series. The mean age was 69.1 (±10.97) years. The mean follow-up duration was 11.31 years (±2.15). 

Table 1

 

Characteristics of the Study Population Where MD and CMD Represent Uncensored Mean Deviations and Censored Mean Deviations, Respectively

 

Figure 1 shows histograms of signal (rate of change) for MD (left) and CMD (right) respectively. The mean MD rate of change was −0.28 dB/year (Display Formula\( \pm 0.15\)). Likewise, the mean CMD rate of change was −0.26 dB/year (Display Formula\( \pm 0.09\)). This indicates that the mean rate of progression appeared to be faster for uncensored data (MD) than censored data (CMD). 

Figure 1

 

Histograms of signal for MD (left) and CMD (right) at cutoff 19 dB, respectively, where signal is defined as the rate of VF decay in decibel per year.

Figure 2 shows histograms of noise for MD (left) and CMD (right) data, respectively. The mean noise for MD rate of change was 0.73 dB (Display Formula\( \pm 1.17\)). Likewise, the mean noise for CMD rate of change was 0.55 dB (Display Formula\( \pm 0.42\)). The mean noise for CMD was smaller than the mean noise for MD. 

Figure 2

 

Histogram of noise for MD (left) and CMD (right) at cutoff 19 dB, respectively, where noise is defined as the SD of residuals from the fitted model.

To identify the ideal cutoff for reducing variability, and hence improving LSNR, we compared LSNR_CMD with LSNR_MD at various cutoffs. Table 2 presents the means of LSNR_MD and LSNR_CMD and the ratios of LSNR_CMD to LSNR_MD at various cutoffs, 12 to 19 dB. Censoring appeared to be effective for any cutoff between 15 and 19 dB. For example, at cutoff 19 dB, the LSNR_CMD was −0.68 (Display Formula\( \pm 0.62),\), which was significantly (Display Formula\(P\) <0.001) lower (better) than the corresponding LSNR_MD, −0.57 (Display Formula\( \pm 0.52\)). Furthermore, the ratios LSNR_CMD/LSNR_MD was significantly greater than 1 at the 19 dB (Display Formula\(P\)<0.001). At all cutoffs below 15 dB, LSNR_CMDs and LSNR_MDs were statistically equivalent (Display Formula\(P \gt 0.05)\). At the 12 dB cutoff, the ratio LSNR_CMD/LSNR_MD was not significantly greater than 1 (Display Formula\(P = 0.062\)). 

Table 2

 

Comparisons of the Longitudinal Signal to Noise Ratios for Mean Deviations (LSNR_MD) and Censored Mean Deviations (LSNR_MD)

 

Discussion

Several studies have reported that test–retest variability is substantially increased in SAP when glaucomatous VF damage exists.^{1–6,15–20} Various efforts have been made to control perimetric variability, either using better data acquisition methods or by applying filtering or other post-processing techniques to existing data. Our study applied data censoring as an alternative means of reducing variability, extracting a more reliable signal, and attempted to do so without losing the ability to detect and monitor change. 

This study demonstrates that censoring leads to a significant increase in the longitudinal signal–to–noise ratio in progressive eyes. For example, LSNR_CMD was significantly better than LSNR_MD when using cutoffs ranging from 15–19 dB. It may be more efficient to stop estimating perimetric sensitivity when it falls below 15 dB. Notably, this is within the 15–19 dB range for the lower limit of reliable sensitivities that was suggested by Gardiner et al.,¹⁵ using a completely different method that compared the correlation between perimetric sensitivities and those measured using frequency-of-seeing curves. Intuitively, censoring would be expected to reduce variability at locations below the chosen cutoff. However, if threshold values in this region were reliable and documenting change then the signal would also likely be reduced. Our results show that the LSNR improves; that is, that the reduction in variability outweighs any reduction in signal. 

If low sensitivities are not providing useful information about glaucomatous VF progression, then there is no benefit in trying to measure sensitivities that fall this low on the decibel scale. Instead, testing algorithms could be adjusted such that they stop testing below 15 dB, instead of continuing testing down to 0 dB. This would shorten test durations, reducing fatigue for patient, and hence potentially improve the reliability of results from other locations in the VF. Furthermore, it would allow test duration to be more consistent across patients; currently testing takes substantially longer in eyes with severe defects than in relatively healthy eyes. Even though it is useful to know whether sensitivity at a VF location is below 15 dB, using current automated perimetry to determine whether that sensitivity is in fact 5 or 12 dB does not appear to be a worthwhile use of time, because thresholds this low are not sufficiently reliable to be useful. 

In this study, only eyes progressing significantly by MD (and CMD) were included, so we may not have included data from eyes in which only one or two locations were deteriorating. This may make our results less applicable in eyes with very early and/or suspected glaucomatous VF damage that have nonsignificant rates of MD and CMD change, or eyes with only a few locations that are changing over time. However, other analyses have shown that censoring does not harm the ability to distinguish point-wise change from variability.¹⁸ Moreover, even though the potential correlation between repeated measurements was accounted for, linear regression was used to estimate MD and CMD rate of change, which may not be optimal,¹² especially when sensitivities reach the imposed cut off. This happens more for CMD than MD, so our approach may actually underestimate the potential benefits of censoring. 

In summary, this study demonstrated that in eyes with progressive glaucomatous VFs changes, censoring point-wise VF sensitivities below 15 dB does not reduce the ability to detect and monitor change using global indices such as MD. Threshold values below 15 dB should be treated cautiously for clinical use and in research studies. Furthermore, this study suggested that 15–19 dB could be a more suitable endpoint for perimetric testing algorithms than continuing testing down to 0 dB. 

Acknowledgments

Disclosure: M. Pathak, None; S. Demirel, None; S.K. Gardiner, None