Normative Percentiles of Retinal Nerve Fiber Layer Thickness and Glaucomatous Visual Field Loss

Abstract

Purpose: Circumpapillary retinal nerve fiber layer thickness (RNFLT) measurement aids in the clinical diagnosis of glaucoma. Spectral domain optical coherence tomography (SD-OCT) machines measure RNFLT and provide normative color-coded plots. In this retrospective study, we investigate whether normative percentiles of RNFLT (pRNFLT) from Spectralis SD-OCT improve prediction of glaucomatous visual field loss over raw RNFLT.

Methods: A longitudinal database containing OCT scans and visual fields from Massachusetts Eye & Ear glaucoma clinic patients was generated. Reliable OCT-visual field pairs were selected. Spectralis OCT normative distributions were extracted from machine printouts. Supervised machine learning models compared predictive performance between pRNFLT and raw RNFLT inputs. Regional structure-function associations were assessed with univariate regression to predict mean deviation (MD). Multivariable classification predicted MD, pattern standard deviation, MD change per year, and glaucoma hemifield test.

Results: There were 3016 OCT-visual field pairs that met the reliability criteria. Spectralis norms were found to be independent of age, sex, and ocular magnification. Regional analysis showed significant decrease in R² from pRNFLT models compared to raw RNFLT models in inferotemporal sectors, across multiple regressors. In multivariable classification, there were no significant improvements in area under the curve of receiver operating characteristic curve (ROC-AUC) score with pRNFLT models compared to raw RNFLT models.

Conclusions: Our results challenge the assumption that normative percentiles from OCT machines improve prediction of glaucomatous visual field loss. Raw RNFLT alone shows strong prediction, with no models presenting improvement by the manufacturer norms. This may result from insufficient patient stratification in tested norms.

Translational Relevance: Understanding correlation of normative databases to visual function may improve clinical interpretation of OCT data.

Introduction

Glaucoma is a progressive optic neuropathy that is a leading cause of irreversible blindness worldwide.^1,2 Early diagnosis and therapeutic intervention are key to preservation of visual function, as it may present as an asymptomatic disease until late stages.³ Glaucoma progression is associated with loss of retinal ganglion cells (RGCs), structural changes to the optic nerve head, and thinning of the retinal nerve fiber layer.³ To aid early diagnosis, circumpapillary retinal nerve fiber layer thickness (RNFLT) measurements from spectral-domain optical coherence tomography (SD-OCT) scans is increasingly incorporated into clinical practice.⁴ RNFLT was shown to provide earlier correlation with visual field loss than optic nerve head (ONH) photographs and presented greater sensitivity for diagnosis of early glaucoma than visual field testing.^5,6 

However, the clinical utility of RNFLT in diagnosis and monitoring of glaucoma has been complicated by disagreement of structure and function. Precedence of structural or functional loss appears to vary based on ocular comorbidities, patient-specific variation, protective features of ocular anatomy, and dysfunction of RGCs prior to cell death.³ Furthermore, monitoring of progression is challenging due to disagreement between annual rates of change of RNFLT and mean deviation (MD).^6,7 

OCT manufacturers attempt to ease the interpretation of structural information by providing population normative datasets, as RNFLT is known to be associated with age, sex, eye-side, and ocular magnification.^8–13 Machine printouts provide a graphical representation of an individual patient's percentiles of RNFLT compared to a healthy control population (Fig. 1). We refer to these percentiles of RNFLT as pRNFLT in the remainder of our work. Patient RNFLT is then depicted over manufacturer normative data showing thickness limits separated into normal (green), borderline (yellow), and abnormal (red) color-coding defined by percentile thresholds suggesting whether a patient has structural degradation outside of normal limits.¹⁴ However, false-positive results and false-negative results, described as “red disease” and “green disease,” may complicate interpretation of these plots.¹⁴ Normative databases developed by institutions for specific clinical populations have been shown to improve detection of glaucoma.^15,16 OCT manufacturer-provided plots based on diverse reference populations therefore have the potential to improve clinical detection or monitoring of glaucoma. However, there has been little external validation of manufacturer-provided databases with clinical populations. Real-world populations may have differing demographic makeup compared to normative reference populations. Furthermore, correlation of abnormal RNFLT, as defined by the machine, to functional disease outcomes has yet to be explored. In the absence of such validation, usage of normative plots remains an individualized choice for clinicians. Manufacturer norms with low clinical relevance may still bias readers due to the presence of prominent color coding on an OCT report.^17,18 

Figure 1.

 

Spectralis (Heidelberg Engineering) report from circumpapillary SD-OCT scan. Top left and right plots display RNFLT of the individual eye (black line) superimposed on the “European Descent 2009” norms (color shaded curves). Corresponding RNFLTs for borders of shaded regions were determined across 768 evenly spaced points.

One approach to validation of manufacturer pRNFLT data may be through development of predictive models correlating structural and functional data. Artificial intelligence (AI) models have been developed to predict visual field changes from SD-OCT input data, as they can improve modeling of complex, nonlinear relationships over traditional statistical methods.^19–22 Nonlinear machine learning methods may be preferred for prediction of global visual field parameters like MD and pattern standard deviation (PSD), as they are measured on a logarithmic dB scale.^23,24 Machine learning methods additionally provide insight into the discriminative power of their input variables. Comparison of raw RNFLT- and pRNFLT-based predictive models may show whether manufacturers’ normative percentiles contain features with improved clinical significance, as the performance of each model may serve as a proxy for strength of agreement between structure and function. In this study, we investigate whether pRNFLT from the Spectralis SD-OCT improves prediction of glaucomatous visual field loss over raw RNFLT values. 

Methods

This retrospective study used all available de-identified data from a database of Massachusetts Eye & Ear (Boston, MA, USA) patients accessed on January 25, 2021. The institutional review board approved this study (No.: 2019P000936) and a waiver of informed consent was granted. All methods adhered to the tenets of the Declaration of Helsinki for research involving human participants, and the study was conducted in accordance with regulations of the Health Insurance Portability and Accountability Act. 

A longitudinal database of patients seen at Massachusetts Eye & Ear with a diagnosis or suspected diagnosis of glaucoma was extracted. Each patient completed at least one Humphrey 24-2 visual field and one circumpapillary SD-OCT scan acquired with the Spectralis platform (Heidelberg Engineering, GmbH, Heidelberg, Germany), between 2009 and 2020. This SD-OCT provides retinal nerve fiber layer thickness measurements at 768 evenly spaced points projected in a circle of approximately 3.45 mm diameter, centered on the optic disc.⁸ 

From this source of 30,452 SD-OCT scans and 158,484 Humphrey 24-2 visual fields, OCT-visual field pairs acquired within 30 days of each other were selected. OCT retinal nerve fiber layer B-scans with a quality of less than 20 dB were excluded, as per manufacturer recommendations. Unreliable visual fields with greater than 20% fixation loss rate and 15% false positive or false negative rates were excluded. Repeated SD-OCT measurements from the same day that met criteria were averaged into one RNFLT reading. 

Norms Extraction

Reports were generated from the Spectralis SD-OCT machine, using the built-in “European Descent 2009,” reference database.^25,26 The tested normative database has been the Spectralis default since 2009, regardless of software version. Premium modules, such as the Glaucoma Module Premium Edition, which need to be licensed separately, may provide different sets of norms and analysis. Our institution did not license any non-standard modules and provides the machine in its default mode on software version 6.16.11.0, which was used for testing. These SD-OCT reports were analyzed in the following way to evaluate the underlying normative database (see Fig. 1). Reports from subjects with different age, sex, and axial length were generated. Within the 5th to 95th percentile area, a line described as the mean in the manual (denoted by dark green in the following plots) was traced to determine mean RNFLT for each of the 768 points on the scan circle.²⁵ Similarly, the 95th percentile (upper border of green shaded area), the 5th percentile (upper border of yellow shaded area), and the first percentile (upper border of red shaded area) were traced.²⁵ Slopes of 1st, 5th, mean, and 95th percentile RNFLT were calculated against age, sex, and axial length to determine how the Spectralis norms compensate for these patient-specific parameters. 

Machine Learning Models

In order to reduce dimensionality of multivariable models, pointwise RNFLT was binned from 768-point scan circle into 6, 16, 32, or 64 sectors. The six-sector binning corresponded to the standard printout from the Spectralis machine (Fig. 2). For each multivariable model, inputs from 6, 16, 32, and 64-sector binning were compared to determine the level of structural granularity that provides the best prediction of visual field loss. 

Figure 2.

 

Visualization of how 768-point RNFLT scan was binned into 6, 16, 32, or 64 sectors.

Univariate regression was first performed to determine regional structural degradation associations with visual field loss. Raw RNFLT and pRNFLT values were binned to average values over 16 sectors, as shown in Figure 2. The input variable was the mean of raw RNFLT or pRNFLT across each sector. Output variable was mean deviation. The regression was repeated with each of the 16 sectors encompassing the scan circle, to identify scanning regions most predictive of visual field loss. The following regressors types were used: linear, polynomial (third degree), k-nearest neighbors, and support vector regression, implemented with Python library scikit-learn 1.0.2. 

We additionally performed multivariable classification using multi-sector (6, 16, 32, and 64-sector inputs) mean of raw RNFLT or pRNFLT, age, and scan diameter as input variables. Spectralis SD-OCT estimates scan diameter from the focus settings applied by the device operator. Our prior work demonstrates that this may be used as a proxy for axial length and refractive error, which are known to affect measured RNFLT.⁸ Outputs included MD, PSD, and Glaucoma Hemifield test (GHT) results (Fig. 3). For a subset of patients who had at least 1 year of longitudinal visual field information available, we performed an additional classification to assess prediction of future degradation of visual field loss. For this, input variables were multi-sector mean of raw RNFLT or RNFLT, age, scan diameter, and baseline MD at first visit. Output was MD change per year (dB/year; see Fig. 3). All outputs were binarized for compatibility with logistic regression and calculation of area under the curve of receiver operating characteristic curve (ROC-AUC) score. 

Figure 3.

 

Output variables with corresponding categories for multivariable classification models.

The following classifiers were used: logistic regression, k-nearest neighbors, decision tree, random forest, gradient-boosted trees, and support vector machine implemented with Python libraries scikit-learn version 1.0.2 and XGBoost version 1.3.3. As k-nearest neighbors and support vector machine are sensitive to feature size, minimum-maximum normalization was applied to scale non-percentile variables to a range from 0 to 1. 

Statistical Analysis

For all analyses, a train-test split of 80% to 20% was used with group shuffle split based on deidentified patient ID. The 10-fold cross validation was performed with the training set to assess model stability with R² scores in univariate models and accuracy scores in multivariable models. Univariate model performance on the testing set was evaluated with R² values. Multivariable analysis was validated using ROC-AUC scores on the testing set. Statistical significance between raw RNFLT and pRNFLT models was evaluated using two-sample t-tests against cross-validation R² scores for univariate models, and Delong's test against test set ROC-AUC scores for multivariable models.²⁷ 

Results

Norms Extraction

On the extracted Spectralis norms, RNFLT was found to be independent of the patient-specific parameters age, sex, and ocular magnification. We found that the dark green line, described as the mean RNFLT in the manual, is equidistant from the 5th and 95th percentile lines, and therefore a symmetric distribution where mean and median coincide (see Fig. 1). Furthermore, the extracted percentiles could be fitted by normal distributions centered at the mean at each of the 768 points around the scan circle. 

Population Description

There were 3016 OCT-visual field pairs from 1427 patients that met our reliability criteria. A subset of 1117 OCT-visual field pairs from 790 patients contained enough (see Fig. 3) longitudinal visual field information to calculate mean deviation change per year, with an average of 4.00 visual fields acquired per patient. Patients with mild glaucoma and moderate/severe glaucoma, defined by mean deviation, show significant differences in age, PSD, global RNFLT, and scan diameter (see the Table). Boxplots showing mean RNFLT from Massachusetts Eye & Ear patients show decreased means relative to Spectralis norms from a healthy population (Fig. 4). 

Table.

 

Summary of Baseline Characteristics.

 

Figure 4.

 

Boxplots representing RNFLT distributions of Massachusetts Eye & Ear patients over 32 sectors, plotted on Spectralis normative plot.

Regional Analysis

R² plotted across 16 sectors shows decrease with pRNFLT input compared to raw RNFLT, in inferotemporal (270 degrees to 315 degrees) sectors, with support vector, polynomial, and linear regressions (t-tests, P = 0.00–0.02; Fig. 5). Support vector regression (SVR) also showed a significant decrease in a superotemporal sector (45 degrees to 67.5 degrees, t-test, P = 0.04). The nearest neighbor regressor showed no significant difference in performance in any sector. 

Figure 5.

 

R² scores for univariate regression models predicting MD. Letters in background represent locations. T = temporal; S = superior; N = nasal; and I = inferior. (A) Tested regressors. (B) Linear regression. Polynomial regression (third degree). (C) Nearest neighbor regression. (D) Support vector regression.

Global Analysis

In multivariable classification, 16-sector inputs provided higher accuracy and ROC-AUC performance compared to 6-, 32-, or 64-sector inputs. The 16-sector inputs were chosen for remainder of analyses. For prediction of MD, the highest performing raw RNFLT model was logistic regression (AUC = 0.781), and highest performing pRNFLT model was support vector classification (SVC; AUC = 0.773; Fig. 6A). For prediction of MD change per year, highest performing raw RNFLT model was gradient-boosted trees (AUC = 0.576), and highest performing pRNFLT model was decision tree (AUC = 0.579). For prediction of PSD, highest performing raw RNFLT model was SVC (AUC = 0.808), and highest performing pRNFLT model was random forest (AUC = 0.822). Last, for prediction of GHT, highest performing model was random forest for both raw RNFLT (AUC = 0.818) and pRNFLT (AUC = 0.820). DeLong test comparison of AUC scores did not show significant improvement in prediction of any variable from pRNFLT inputs. 

Figure 6.

 

Testing set ROC-AUC comparison for multivariable models with raw RNFLT and pRNFLT inputs. (A) Prediction of mean deviation. (B) Prediction of pattern standard deviation. (C) MD change per year. (D) Glaucoma hemifield test.

Discussion

Retinal nerve fiber layer thickness is a relevant parameter for the detection of glaucomatous loss. Thus, to discriminate between normal and glaucomatous eyes, OCT machine printouts provide color-coded normative plots to aid in the clinical diagnosis of glaucoma. There is existing literature on evaluating relevance of normative database source populations to real-world clinical populations,^28,29 as well as on development of correction models.^8,30,31 Our study, however, presents the first attempt to evaluate the clinical significance of a manufacturer database, by assessing discriminative ability for visual function. We do this by providing a context of structure-function relationship in a large clinical population, and show that the tested manufacturer norms provide no significant improvement in prediction of function in univariate or multivariable models. 

In our extraction of Spectralis SD-OCT, “European Descent 2009,” norms, we find that they are independent of the patient-specific parameters age, sex, and ocular magnification.²⁶ Manufacturer documentation describes the source database as having contained 201 Caucasian subjects with healthy eyes, with a mean age of 48 years, and refractive error between +5 and −7 diopters (D).^25,26 This may limit generalizability to more diverse and older Massachusetts Eye & Ear patient cohorts. Furthermore, the discriminative power of the manufacturer norms may be lower than published norms that do adjust for these factors, those from future Spectralis software versions (or add-on modules), and those from other manufacturers.^8,12 

Regional analysis demonstrates strongest structure-function associations in inferotemporal (IT) and superotemporal (ST) regions of the scan circle. This is consistent with the locations of the macular vulnerability zone (MVZ) and superior vulnerability zone (SVZ), respectively, which are known to be prone to early glaucomatous damage and associated with arcuate visual defects.^32,33 These regions are anatomically thicker due to the presence of major nerve fiber bundles. The 24-2 visual field is additionally centered temporally to the disc.³⁴ Within these key regions, we did not find any improvement in prediction of MD from pRNFLT inputs. With certain regressors, we observed decreases in mean cross-validation R² scores with pRNFLT inputs compared to raw RNFLT inputs. Whereas pRNFLT curves follow established anatomic trends with improved relationships in the MVZ and SVZ regions, they ultimately show decreased correlations compared to raw RNFLT curves (see Fig. 5). Inspection of regional regression plots suggest that raw RNFLT inputs preserve a trend with mean deviation across the thickness range, whereas percentile inputs only show a consistent relationship below the 20th percentile (Supplemantary Material). The percentiles might lose information that is still contained in the original thickness data. Above the 20th percentile, the relationship is flat, including a broad range of patients with moderate to severe visual field loss. The Spectralis percentiles may maintain high specificity but show decreased sensitivity for moderate to severe glaucoma. 

Global analysis shows that percentile inputs do not significantly improve the performance of any of our tested models in prediction of MD, MD change per year, PSD, or GHT. We attempt to mitigate the lack of adjustment for age and axial length in the Spectralis, “European Descent 2009,” norms, by including age and refraction (using scan diameter as a proxy) in our models.⁸ The addition of these covariates improves the performance of pRNFLT models, relative to the deficits seen in regional analysis (see Fig. 5). However, adjustment for these factors in linear and nonlinear models remain insufficient to improve upon raw RNFLT. In prediction of PSD and GHT, we observe decreases in performance with pRNFLT models with certain classifiers (e.g. k-nearest neighbors and SVC). This may be an artifact of conversion to percentiles skewing relative thinning effects versus minimum-maximum normalization of raw thickness values, thus impacting prediction of regional visual field loss with those classifiers. Prediction of future visual field degradation is poor across all tested models, indicating that prediction of glaucoma progression from baseline structural factors is a difficult task, especially with the lack of treatment information, longitudinal visual fields, or detailed ocular anatomic parameters.^35,36 Perhaps decomposition of baseline RNFLT into structural loss patterns may better correlate with rates of visual field progression than sectoral RNFLT.³⁷ 

Our results are counter-intuitive, as clinical instinct may suggest that any normalization of RNFLT to a reference database should improve discrimination of visual field loss over no normalization at all. Normative data should include factors that influence RNFLT, such as age, sex, and refraction.^8,11–13,31 The found absence of improvement may be caused by the lack of adjustment for such influence factors in the tested reference database. One example of norms that do accommodate these factors is available in a cloud-based visualizer (https://apps.health-atlas.de/rnflt-visualizer/), developed by Peschel et al.¹² Our results demonstrate the need to provide external validation of machine normative outputs to clinical populations. Furthermore, evaluation of relevance of normative outputs to visual function and glaucoma progression may better support clinical interpretation. Future software packages, add-on modules, and those from other manufacturers may adjust for some of the aforementioned factors. We encourage manufacturers to provide detailed information about demographics of reference populations for normative databases, and additionally offer data on the clinical relevance of RNFLT categorized as outside of normal limits within population subgroups. Inclusion of this information may support the development of more robust norms and reduction of clinical instances of “red disease” (false positives) and “green disease” (false negatives). 

In the broader context of structure-function relationship, our regional analysis seems to suggest superiority of nonlinear machine learning algorithms over linear regression. Existing concepts in glaucoma structure-function relationship generally describe linear relationships between visual field loss and RNFLT, ganglion cell number, or ganglion cell receptive field density.³⁸ The model by Hood et al. suggests that receptive field sensitivity must be mapped on a linear scale, rather than a nonlinear dB, to observe this relationship.³⁹ Furthermore, they report that there is a floor effect of RNFL thickness beyond which further thinning does not occur in severe disease. In our regional models, we predict MD in dB, with greater performance using nonlinear third degree polynomial and support vector regression. In addition, we show a floor effect of thinning around 30 to 50 µm thickness, consistent with previously reported findings.⁴⁰ In our global analysis, logistic regression performs similarly to nonlinear methods in prediction of MD. However, support vector machine and ensemble methods like random forest improve performance over logistic regression in prediction of PSD, GHT, and MD change per year. Prior studies have also shown strong discriminative ability between healthy and glaucomatous eyes with these methods.^22,41 

One limitation of our study is that quality control of individual SD-OCT scans was restricted to machine-reported parameters rather than individual inspection of B-scans, largely due to the high number of cases included. Perhaps additional assurance of quality by checking layer segmentation would further improve model performance. An additional limitation is the evaluation of normative percentiles from the Spectralis SD-OCT machine alone, on the default software configuration, along with a singular set of norms. This may limit generalizability to configurations in other clinical settings. Comparison of percentile performance between OCT platforms and with different normative databases could provide greater reflection on their clinical utility. 

Conclusions

Our results challenge the assumption that normative percentiles from OCT machines improve prediction of visual field loss. Raw RNFLT shows strong prediction of current visual field loss metrics like MD, PSD, and GHT, with no models showing significant improvement from the manufacturer norms. Lack of specific adjustment for demographic factors and ocular anatomy, may result in loss of information during conversion to percentiles, information that is retained in original thickness data. OCT manufacturers may improve their normative databases with greater patient stratification based on these factors. Until greater clinical validation of norms is established, interpretation of normative plots must be approached with caution. 

Acknowledgments

Supported by National Institutes of Health (NIH; R01 EY030575 and P30 EY003790), BrightFocus Foundation. 

Disclosure: R. Singh, None; F.G. Rauscher, None; Y. Li, None; M. Eslami, Genentech (F); S. Kazeminasab, None; N. Zebardast, None; M. Wang, Alcon (F), Genentech (F); T. Elze, Genentech (F)