Development of a Visual Field Simulation Model of Longitudinal Point-Wise Sensitivity Changes From a Clinical Glaucoma Cohort

Abstract

Purpose: To develop a new visual field simulation model that can recreate real-world longitudinal results at a point-wise level from a clinical glaucoma cohort.

Methods: A cohort of 367 glaucoma eyes from 265 participants seen over 10.1 ± 2.5 years were included to obtain estimates of “true” longitudinal visual field point-wise sensitivity and estimates of measurement variability. These two components were then combined to reconstruct visual field results in a manner that accounted for correlated measurement error. To determine how accurately the simulated results reflected the clinical cohort, longitudinal variability estimates of mean deviation (MD) were determined by calculating the SD of the residuals from linear regression models fitted to the MD values over time for each eye in the simulated and clinical cohorts. The new model was compared to a previous model that does not account for spatially correlated errors.

Results: The SD of all the residuals for the clinical and simulated cohorts was 1.1 dB (95% confidence interval [CI]: 1.1–1.2 dB) and 1.1 dB (95% CI: 1.1–1.1 dB), respectively, whereas it was 0.4 dB (95% CI: 0.4–0.4 dB) using the previous simulation model that did not account for correlated errors.

Conclusions: A new simulation model accounting for correlated measurement errors between visual field locations performed better than a previous model in estimating visual field variability in glaucoma.

Translational Relevance: This model can provide a powerful framework to better understand use of visual field testing in clinical practice and trials and to evaluate new methods for detecting progression.

Introduction

Visual field testing is an invaluable tool in the clinical management of glaucoma by allowing both the nature and extent of vision loss to be characterized and monitored over time. The information gained from this test can provide crucial insights into the current level and future risk of functional disability experienced by an individual with this condition.^1,2 It also remains an important outcome measure for clinical trials in glaucoma, as it represents a clinically relevant endpoint.^3–5 However, detecting progressive visual field loss remains a challenging task due to the extent and complexity of its measurement variability⁶ and nature of change over time⁷ in eyes with visual field damage. 

There have been several attempts to develop improved methods for the detection of visual field progression. However, studies investigating these methods are limited by the lack of an independent reference standard to which the performance of a new method can be compared. For example, in order to evaluate the specificity of a newly proposed algorithm, one needs to test it against a population of patients with stable disease. The accurate detection of truly stable eyes is crucial since clinical management decisions—such as the intensification or initiation of glaucoma treatment or the mere diagnosis of the condition—can have negative consequences for individuals.^8,9 However, how does one define stability in studies testing a new visual field algorithm? Recent studies have used short-term test-retest data to evaluate specificity since progression can be assumed to be absent over such a time frame.^10–16 However, this approach may insufficiently capture the full extent of measurement variability present in real-world visual field tests over a long-term period, making it hard to accurately determine the true potential performance of these new methods in clinical practice. 

As such, the ability to characterize and accurately recreate real-world visual field results over time can provide a powerful framework for the development of improved methods to detect visual field progression. It can also provide substantial insights into both clinical practice and trials, such as the impact of different clinical testing frequencies¹⁷ or paradigms¹⁸ on the ability to detect progression or sample size requirements in a clinical trial using a visual field endpoint.¹⁹ Previous studies have simulated visual field results (primarily to evaluate different thresholding algorithms) using models of an individual's responses to different stimulus intensities (i.e., psychometric functions),^20–31 often with estimates obtained from an experimental, rather than a clinical, setting.³² 

Instead, Russell and colleagues³³ recently simulated visual field results using parameters obtained from a large longitudinal cohort of glaucoma patients under routine clinical care. Such an approach is highly advantageous because allows visual field results to be reconstructed in a manner that closely reflects those expected clinically. However, two important methodological refinements are required to ensure that such simulations better represent real-world results. First, the correlations between the estimates of measurement variability should be accounted for, since real-world visual fields contain such correlations at the individual level, evident from a previous study demonstrating how accounting for such correlations provided a better fit of longitudinal visual field data.³⁴ Second, the assumption of linearity held for changes in point-wise visual field sensitivity over time is unlikely to remain valid over a long follow-up duration, with nonlinear models better capturing such changes.⁷ 

This study, therefore, developed a new visual field simulation model after including such refinements, demonstrated how it better reflected real-world results, and provided an example of an application of this model for improving our understanding of visual field testing in clinical practice. 

Methods

Participants

This study included participants who were enrolled in a prospective longitudinal observational study evaluating structural and functional damage in glaucoma. The study received institutional review board approval and was conducted in adherence with the Declaration of Helsinki and the Health Insurance Portability and Accountability Act. All participants in this study provided written informed consent after the test procedures were explained. 

Participants in this study underwent a comprehensive ophthalmologic evaluation that included a review of their medical history, visual acuity measurements, visual field testing, slit-lamp biomicroscopy, ophthalmoscopic examination, gonioscopy, intraocular pressure measurement, and stereoscopic optic disc photography. This study included only eyes considered to have glaucoma, based on the masked evaluation of the optic nerve on stereophotographs.³⁵ This study also included only glaucoma eyes with ≥10 abnormal visual field tests (defined as having a pattern standard deviation [PSD] value with P < 0.05 or Glaucoma Hemifield Test outside normal limits) over at least 5 years. Participants were also required to have open angles on gonioscopy and a best-corrected visual acuity of 20/40 or better, and they were excluded if they had any other ocular or systemic disease that could affect the optic nerve or the visual field. 

Visual Field Testing

All visual field tests were performed on the Humphrey Field Analyzer II-i (Carl Zeiss Meditec, Inc., Dublin, CA) using the Swedish Interactive Thresholding Algorithm Standard 24-2 strategy, with the results being considered unreliable and excluded from the analyses if they had >33% fixation losses or false-negative errors (with an exception for false-negative errors when the visual field mean deviation [MD] was less than −12 dB) or >15% false-positive errors. The visual field tests were reviewed for the presence of artifacts, including fatigue or learning effects, inattention, inappropriate fixation, eyelid or rim artifacts, and evidence that visual field results were influenced by a disease other than glaucoma (such as a homonymous hemianopia); tests with such artifacts were not included in the analyses.³⁶ Visual fields were then repeated if found to be unreliable or contained artifacts. 

Development of Visual Field Computer Simulations

To allow reconstruction of a series of real-world visual field tests from glaucoma eyes, two components were required in a computer simulation: (1) longitudinal estimates of true visual field sensitivity at each location (or estimates of the true pattern of visual field change over time) and (2) estimates of measurement variability (or a noise component). These two components could then be combined in such a way to generate series of visual field results from different individuals (each with a unique pattern of variability and performance) for a given pattern of visual field change over time. The details of how each component was obtained and then combined are outlined below. 

To obtain longitudinal estimates of the “true” point-wise visual field sensitivity for each eye included in this study, a sigmoid regression model was fitted to the measured threshold sensitivities at each location over time using a method described previously.⁷ The sigmoid model assumes a nonlinear rate of visual field loss, with natural asymptotes occurring at normal levels of sensitivity and the perimetric floor. The model can be expressed as follows: s = γ / (1 + e^{α + βx}), where s denotes the measured sensitivity in decibels, γ indicates the estimate of the initial sensitivity, α indicates how soon the sigmoid function begins a steep decline, β indicates the steepness of this decline, and x indicates the time. This regression model was fitted using an iterative feasible generalized nonlinear least squares method (being equivalent to maximum likelihood estimation), except for locations where at least two out of the three initial tests had a measurement of 0 dB, which were fitted with a value of 0 dB throughout the duration of the follow-up. An example illustrating four locations that were fitted with this sigmoid regression model over the entire perimetric range is shown in Figure 1. The parameters of the sigmoid model could then be used to estimate true sensitivities at each location for an eye at any given time point; these derived sensitivity estimates were termed the “sensitivity template.” 

Figure 1

 

An example illustrating how visual field sensitivity in a glaucoma eye changes across the entire dynamic range. This example highlights how four locations with different levels of sensitivity at baseline, all ≥20 dB (indicated by numbers 1 to 4 on the pattern deviation maps, bottom left), that eventually become scotomatous over a 16-year follow-up exhibit different patterns of change over time, as shown on the plots labeled with their corresponding numbers (right). A sigmoid regression was fitted to the raw sensitivities at each location (black lines), and the residuals were obtained by subtracting the measured value (circular markers) from the fitted value.

To obtain estimates of measurement noise, residuals were derived by subtracting the measured values from those fitted by the sigmoid regression model and binned according to these fitted values (rounded to the nearest 1 dB). Residuals were pooled across all locations and eyes to generate residual distributions for each fitted sensitivity bin, which were termed the “empirical probability distribution functions” (PDFs). The residuals at each location for each test of each eye were then converted into probabilities based on the empirical PDFs of its fitted sensitivity because the distribution of the residuals were not expected to be the same for different fitted sensitivities. For instance, a residual of −3 dB would occur more frequently in regions of visual field damage than in healthy regions within a single visual field test independent of the performance of an individual during a test. These probabilities provide a standardized estimate of the deviation of the individual's response from the fitted sensitivity and thus collectively provide a template of patient performance during a test (in a way similar to joint probabilities) that accounts for the correlation between the measured values at each location during a visit (i.e., a global visit effect, such as from varying levels of attention between visits). This was termed a “noise template.” A noise template could then be combined with a sensitivity template to simulate real-world visual field results through a process explained further below. The noise templates from all participants were then combined to create a database of patient-level variability that was used in the visual field simulations through a process also explained below. As a minimum of 10 visits were included for each eye in this study, we included only the noise templates from the first 10 visits from one eye of each participant (randomly selecting one eye if both eyes were available) so that each participant contributed equally to these estimates of variability. To minimize the likelihood of selecting the same noise template during the simulations, we increased the number of noise templates available from each patient through randomizing the probabilities by location within the same eccentricity for each noise template by 100 times. As a result, a database of patient-level variability was generated that contained 1000 different sets of noise templates (10 tests × 100 randomized sets) for each participant, and we refer to each of these participants as a “model participant.” 

Sequences of real-world visual field results can then be simulated for each eye by combining the longitudinal estimates of true point-wise sensitivity and estimates of measurement variability. For each sequence, sensitivity templates were derived at each time point from the sigmoid regression model, and a model participant was selected at random from the database of patient-level variability. A noise template was subsequently chosen at random for each test in this sequence from the 1000 available noise templates from the model participant. Real-world visual field results were then recreated by using the probabilities at each location to determine the magnitude of the residual (or the noise component) to be added to the sensitivity template (representing the true pattern of damage) by sampling the residual from the empirical PDF corresponding to the true fitted level of sensitivity. An example in Figure 2 illustrates how a sensitivity template was combined with a noise template to create a simulated real-world visual field test result. 

Figure 2

 

Illustration of the process of simulating a single visual field test result (right column), where a sensitivity template (representing an estimate of the true visual field sensitivity at each location in decibels, left column) is combined with a noise template (middle column), consisting of probabilities at each location that were then converted into estimates of measurement error based on the empirical probability distribution function of the corresponding sensitivity bin from each location.

Comparison With a Previous Visual Field Simulation Model

The visual field simulation model developed in this study was compared with another model described previously.³³ Our methods are in essence similar to those used previously, but the key difference is that the previous method does not account for correlations between test locations for the estimates of measurement variability, accounted for by the noise templates with our model. In other words, visual fields were simulated by taking a sensitivity template and adding estimates of measurement variability by simply sampling residuals from the empirical PDF corresponding to the true fitted level of sensitivity at each location at random. This difference in methodology can be conceptualized as using an individual-based pattern of measurement variability with our method and a random selection of measurement variability for the previous method. To ensure that the primary comparison was performed between models that did or did not account for such correlated measurement errors, the sigmoid regression model was also used in this model (as opposed to a linear regression model used previously³³) to determine the impact of accounting for such correlations. 

Comparison of the Simulation Models and the Glaucoma Clinical Cohort

We investigated and compared the longitudinal variability of visual field MD as derived from the simulation models described above to those obtained from the clinical glaucoma cohort followed over time. As MD is a global index frequently used for trend-based assessment of progression and estimation of rates of visual field loss, it is important to determine whether a proposed simulation model accurately reflects the variability of this index over time. For this purpose, we obtained the residuals from an ordinary least squares linear regression model fitted to MD values over time for each eye in the clinical cohort, calculated by subtracting the measured and fitted MD values. As the linearity of MD values can diminish as the duration of follow-up increases, only MD values within the 2- to 4-year period with the greatest number of tests were included for this evaluation. 

For the simulation models, we simulated 100 sequences of visual field tests for each eye in the glaucoma clinical cohort at the same visits as those included within the 2- to 4-year period described above. Residuals from ordinary least squares linear regression models fitted to MD values over time for each sequence of tests were then obtained. 

We also compared the cumulative proportion of eyes detected as having progressed using a global trend–based analysis of MD for eyes in the longitudinal clinical cohort and simulated results from the models described above. Visual field progression was considered to have occurred when a statistically significant negative MD slope (P < 0.05) was present at two consecutive visits. For the simulation models, we also simulated 100 sequences of visual field tests for each eye in the glaucoma clinical cohort at the same visits at which they were seen clinically. 

Statistical Analysis

The SD of the residuals was calculated by using a random intercepts model—a type of linear mixed model—to account for the hierarchical nature of the data (e.g., when multiple tests from one eye were evaluated). For the clinical glaucoma cohort, the random intercept model was used with tests nested within eyes and nested within participants. For the simulated visual field results where 100 sequences of visual field tests were generated for each eye, the random intercept model was used, with tests nested within sequences, within eyes, and within participants. All computer simulations and analyses were performed using statistical software (Stata Version 14; StataCorp, College Station, TX). 

Results

Participant Characteristics

A total of 367 eyes from 265 participants with glaucoma were included in this study, and they were on average 62.7 ± 11.3 years old at the first visit (range, 25–88 years old) and were seen at 15.6 ± 5.0 visits (range, 10–39 visits) over 10.1 ± 2.5 years (range, 5–17 years). At the first visit, the median (interquartile range) MD and PSD of these eyes was −4.04 dB (−7.86 to −2.17 dB) and 4.38 (2.53–8.59 dB), respectively, and was −6.37 dB (−11.88 to −3.37 dB) and 6.55 dB (3.40–10.03 dB), respectively, at the last visit. 

Comparison of Longitudinal Variability of MD

The longitudinal variability of the visual field MD values was evaluated by examining the SD of the residuals, the difference between the actual measured or simulated MD values and the fitted values from a linear regression model. These longitudinal residuals were obtained from a follow-up period between 2 and 4 years in duration that had the greatest number of visits, and 6.9 ± 2.7 visits (range, 5 to 17 visits) were included over 2.6 ± 0.6-year period. 

The SD of the residuals for MD for the longitudinal clinical cohort was 1.1 dB (95% confidence interval [CI] = 1.1–1.2 dB) and was 1.1 dB (95% CI = 1.1–1.1 dB) using the simulated visual field results from the methods developed in this study. In contrast, the SD of the residuals was 0.4 dB (95% CI = 0.4–0.4 dB) for the visual fields simulated using the method that does not account for correlations between test locations within each test. Histograms for the distribution of these residuals are shown in Figure 3. 

Figure 3

 

Histograms showing the longitudinal residuals of visual field MD representing the difference between measured or simulated MD values from fitted values obtained from a linear regression model. The histograms are shown for the measured visual field results from the clinical glaucoma cohort (left), simulated visual field results using the methods from this study that account for correlations between test locations during each visit (middle) and simulated visual field results that did not account for such correlations (right).

Comparison of the Cumulative Proportion of Eyes That Progressed

The cumulative proportion of eyes showing visual field progression over time was evaluated using trend-based MD, with progression defined as the presence of a statistically significant negative slope at two consecutive visits. At the final visit, 45.8% of eyes were detected as having progressed in the longitudinal clinical cohort, whereas an average of 43.4% (95% CI = 40.3%–46.4%) of the eyes progressed using the simulated visual field results from the methods developed in this study. In contrast, an average of 58.3% (95% CI = 56.1%–60.5%) of the eyes progressed by the final visit when using the simulated visual field results that do not account for correlations between test locations. 

Examples of Simulated Visual Fields

In Figure 4, four series of simulated visual field results over time are shown in comparison to the actual measured visual field results from an eye in this study, demonstrating how the simulated visual field results resemble the real-world findings. 

Figure 4

 

An example showing the measured visual field results of an eye over time (top row) and four series of simulated visual field results (bottom rows).

Application in Understanding Variability of Global Metrics

The visual field simulations were applied to provide insights into the variability characteristics of MD and PSD values across a range of different true levels of damage and patterns of loss. This was done by simulating 100 different sequences of visual field tests for each eye included in the clinical cohort and then calculating the SD of the difference between the simulated MD and PSD measurements and their true values for 100 simulated tests for each sequence. In other words, this is in effect similar to simulating 100 different patients with the same pattern of visual field loss who underwent 100 visual field tests each. The median of the SDs for each eye was then taken to provide a population-averaged estimate of variability for that pattern of visual field loss. 

Figure 5 shows a plot of the SD of the difference between the simulated values from the true value against the true value of MD or PSD itself. These results can be interpreted as the extent to which a measured value will differ from the true underlying value. The results demonstrated that the variability of visual field MD increases with a worsening of MD and PSD, reaching a peak at an MD of approximately −18 dB and PSD of approximately 10 dB, where the SD of the difference between the simulated MD measurements from the true MD was 1.4 and 1.3 dB, respectively. For the variability of PSD values, a peak was reached at approximately 5 dB, where the SD of the difference between the simulated PSD measurements and the true PSD values was 0.8 dB. When the true MD and PSD were −3 and 3 dB, respectively (representing an early level of visual field damage), the SD of the simulated measurements from their true values was 0.9 and 0.7 dB, respectively. 

Figure 5

 

Illustration of the application of the visual field simulations developed in this study to understand the variability of MD against (A) the true MD and (B) the true PSD value and (C) the variability PSD against its true value. For each graph, variability is indicated by the SD of difference of the values of the simulated visual fields from the true value (circular markers). A locally weighted polynomial regression fit of the data is also shown (black line).

Discussion

This study demonstrated that real-world visual field results could be reconstructed using the new simulation model developed in this study, with the magnitude of longitudinal MD variability and cumulative proportion of eyes detected as having progressed, closely matching those obtained in a clinical glaucoma cohort. This was contrasted with a previous model that did not account for such correlated errors, which underestimated longitudinal MD variability. To provide an example of its potential application, this model was used to examine the variability characteristics of MD and PSD values across different levels and patterns of visual field damage. This simulation model could provide a powerful framework for future studies to gain important insights into various aspects of visual field testing in clinical practice and clinical trials, and we will discuss such translational relevance below. 

The visual field simulation model developed in this study was based on the approach first developed by Russell and colleagues³³ but incorporated two important refinements to ensure that the results accurately represent findings expected in routine clinical practice. The most important refinement involves accounting for the correlated measurement errors between locations in a given test. This resulted in longitudinal MD variability estimates that were very similar between the simulated and clinical visual fields. Without accounting for such correlations, the estimate of longitudinal variability was reduced to almost a third of those observed clinically, having a SD of 0.4 dB for all MD residuals, which falls within a similar range of 0.2 to 0.7 dB reported by Russell and colleagues.³³ The proportion of eyes detected as having progressed when such correlations were not accounted for was over a third higher than when these correlations were accounted for. When the correlations were accounted for in our proposed simulation model, the proportion of eyes detected as progressing was very similar to that found in the longitudinal clinical cohort. The value of accounting for such correlations has also recently been observed in a different context, where accounting for a global visit effect (i.e., correlated measurement error) resulted in a better fit of longitudinal visual field data.³⁴ The other refinement involves using a sigmoid regression model to capture the nonlinear behavior of visual sensitivity changes at a point-wise level over the entire perimetric range, which a recent study demonstrated to provide the best fit to such data in a large clinical cohort of glaucoma eyes seen over time.⁷ Indeed, we observed that the sigmoid regression model had a lower root mean square error (RMSE) when compared to a linear model in 88% of the 19,084 locations evaluated in this study, having a mean (SD) RMSE of 2.6 (1.9) and 2.8 (2.0), respectively. 

The application of the visual field simulation model for understanding the variability of visual field MD against various true MD and PSD values (as shown in Figure 5) showed the same characteristics as reported by Russell and colleagues.³³ Note that the application of the simulation model in this situation helps clinicians understand their expected variability across various disease severities when seeking to identify true change. We observed that the variability of MD values peaked at approximately −18 and 10 dB for true MD and PSD, similar to the previous report of peaks at −20 and 8 dB, respectively, although the magnitude of the SD at these peaks were substantially different (see above).³³ The magnitude of the residual variability observed using our simulations is also similar to those from real-world clinical findings in a previous study where the variability estimates were obtained with a weighted moving average regression analysis of longitudinal visual field data.³⁷ For example, they report that 95% of the residual differences fell within a 4.5-dB range at an estimated true MD of −10 dB (and thus having a SD of approximately 1.1 dB), similar to an SD of approximately 1.3 dB from our findings. The variation in MD variability with different levels of visual field damage has also been reported in previous studies using short-term test-retest data.^11,38 However, the magnitude of the variability presented in those studies appears somewhat smaller than that observed with our longitudinal data, although a direct comparison is difficult due to the limited sample size of those previous studies. Our simulations also provided insights into the variability of the PSD measure, demonstrating how the range of variability can be especially wide for eyes where PSD values are between 4 and 8 dB. 

The ability to simulate visual field results that closely resemble real-world findings can also allow numerous other aspects of visual field testing to be investigated, which has direct translational relevance. For instance, it provides a framework to evaluate new or compare current methods for detecting visual field progression for implementation in clinical practice. One of the challenges of evaluating new algorithms for detecting glaucomatous visual field progression is the difficulty in determining its true specificity due to the lack of a perfect independent reference standard that represents true stability. Several approaches have been used for this, such as collecting test-retest visual field data over short periods of time^10–16 (to ensure that no true progression is seen), although this approach is unlikely to truly represent long-term test-retest variability. Instead, the simulation model presented in our study provides a powerful method to estimate the specificity of new algorithms for detecting visual field progression, as results for eyes that are truly stable can be simulated to examine this. The simulated results would exhibit variability characteristics more typical of real-world patients seen in clinical practice, therefore providing a better evaluation of the actual potential real-world performance of a method. The simulations could also be used to examine the impact of testing frequencies¹⁷ or paradigms¹⁸ on the ability to detect visual field progression in clinical practice or sample size requirements in glaucoma clinical trials¹⁹ using point-wise methods of analyses. 

Some limitations of the study to consider include the relatively smaller sample size used to create these visual field simulations when compared to the nearly 10-fold sample size evaluated by Russell and colleagues.³³ However, there are future opportunities to perform such analyses in even larger longitudinal datasets collated from multiple clinical centers (using the concept of “big data”), which can include up to tens of thousands of patients.^39,40 Nonetheless, we believe the study's sample size was sufficient and that including a large sample would only strengthen the conclusions reached. Another limitation to consider when interpreting the variability estimates shown in this study is that they were obtained from participants seen at approximately biannual intervals, and such estimates may be slightly higher for patients seen at annual intervals. 

In conclusion, this study presents a new, refined method for simulating longitudinal visual field results, closely resembling real-world findings. The findings underscored the importance of accounting for correlated measurement errors within a test and demonstrated the application of this visual field simulation model in understanding the variability characteristics of visual field MD and PSD measurements. 

Acknowledgments

We are grateful to David P. Crabb for providing the R code for generating the simulated visual field results in Figures 2 and 4, which was developed by Nicholas D. Smith and Richard A. Russell. 

Supported in part by a National Institutes of Health/National Eye Institute Grant EY021818 (FAM) and National Health and Medical Research Council Early Career Fellowship 1104985 (ZW). 

Disclosure: Z. Wu, None; F.A. Medeiros, Alcon Laboratories (F), Allergan (F, C), Bausch & Lomb (F), Carl Zeiss Meditec (F, C), Heidelberg Engineering (F), Merck (F), National Eye Institute (F), Reichert (F), Topcon (F), Novartis (C)