An Automated Method for Assessing Topographical Structure–Function Agreement in Abnormal Glaucomatous Regions

Abstract

Purpose: To develop an automated/objective method for topographically comparing abnormal regions on optical coherence tomography (OCT) and visual field (VF) tests of eyes with early glaucoma.

Methods: A custom R program was developed that allows for both visualization and automatic assessment of the topographical agreement between functional (24-2 and/or 10-2 VF) and structural (widefield OCT retinal nerve fiber layer and/or retinal ganglion cell layer) deviation/probability maps. It was optimized using information from 98 eyes: 53 diagnosed as “definitely glaucoma” (DG) and 45 recruited as healthy (H) controls. Different pairs of abnormal VF (P <1%, <2%, <5%) and abnormal OCT (P <5%, <10%, <15%) criteria were evaluated. The percentages of abnormal structure–abnormal function (aS-aF) agreement found in DG eyes and nonagreement found in H eyes were used to define the optimal criteria and number of aS-aF locations for the detection of aS-aF agreement.

Results: A criterion of two aS-aF locations with “OCT <10% and VF <5%” on VF pattern deviation (PD) probability and OCT deviation/probability maps yielded high overall agreement (92%) with high aS-aF agreement for the DG eyes (89%) and high aS-aF nonagreement for the H eyes (95%). Total deviation probability maps achieved slightly lower performance than PD maps.

Conclusions: The method described here can automatically and objectively evaluate aS-aF agreement with a direct comparison of abnormal regions of function and structure.

Translational Relevance: As glaucoma diagnosis often involves assessing structure–function agreement, this technique can overcome subjectivity in this assessment.

Introduction

There is no single (litmus) test for diagnosing primary open-angle glaucoma (POAG), the most prevalent chronic optic neuropathy. POAG is characterized by progressive retinal ganglion cell (RGC) degeneration that results in structural changes to the optic nerve head and the retinal nerve fiber layer (RNFL), as well as in the loss of visual function.¹ Thus, most clinicians consider structural (e.g., fundus photos and/or optical coherence tomography [OCT]) and functional (e.g., visual field [VF] testing) information in diagnosing and monitoring glaucoma. While there is general agreement about the need to compare structural and functional agreement, there is no consensus on the best method to use. 

Hood and De Moraes² have recently suggested, through the results of a pilot study, that a topographic comparison of structural and functional defects shows good agreement, which significantly increases if both 24-2 and 10-2 VF tests were performed and if structural (i.e., OCT) information from both the RGC and RNFL layers was included. However, their report carried some limitations, which included the qualitative, and therefore subjective, aspect of their analysis, as well as the retrospective nature of their study and its related biases (e.g., selection bias). 

In any case, the combination of structural and functional information, as well as the subsequent topographic comparison, provides useful information for the clinician. For instance, this approach can use the strengths of each test (i.e., structural and functional) and compensate for their weaknesses. For example, structural arcuate-like defects can be present in OCT reports of healthy eyes if the locations of the major superior and inferior blood vessels differ from the average location of healthy controls.^3,4 Such artifacts could trigger an incorrect diagnosis of glaucoma, if misinterpreted. Perimetry, on the other hand, rarely shows defects of an arcuate pattern in healthy eyes. Therefore, the two tests (i.e., structural and functional) combined could potentially reduce the number of false positives. 

Here we describe an automated technique that determines the topographic agreement between regions of abnormal structure–abnormal function (aS-aF) and effectively provides an objective and novel way in the assessment of aS-aF. 

Methods

A custom program was developed in R,⁵ which allowed for both visualization and automatic assessment of the agreement between aS and aF. The program takes as input the functional (24-2 and/or 10-2 VFs) and structural (OCT – RNFL and RGC) probability data described below. Before describing the approach,we first describe the data set to be used in analysis here. 

Participants

Structural (OCT) and functional (24-2 and 10-2 VFs) data were obtained from Columbia University's prospective Macular Damage in Early Glaucoma and Progression (ClinicalTrials.gov Identifier: NCT02547740). Patients’ eyes (n = 53, one eye per patient; mean 66.93 ± 9.54 years of age) were deemed glaucomatous (i.e., definitely glaucoma [DG]) based upon the referring glaucoma expert's interpretation of functional (24-2 and 10-2 VFs) and structural (fundus photos, OCT) information, as well as intraocular pressure (IOP) and clinical history. (Note: these experts were not involved in the development of the aS-aF algorithm.) Upon recruitment, the mean deviation (MD) of all study eyes was better than –6 dB on a SITA-Standard 24-2 VF test. In addition, 45 healthy (H) controls (mean 35.7 ± 15.2 years of age) were enrolled from hospital staff, after confirming normal fundus examination, normal VFs, and IOP ≤22 mm Hg. Despite the significant age difference between the two groups (i.e., DG and H), an age effect is minimized for the purpose of this study due to the employment of age-corrected VF and OCT deviation/probability maps (see details below). 

For the purposes of this study, we used the baseline test of these DG and H eyes (total n = 98 eyes/patients). The definition of normal/abnormal function or structure did not follow a specific set of rules to mitigate the impact of such rules in the classification systems. Instead, definitions were left up to glaucoma experts’ discretion and the final diagnosis given to patients. All participants had best-corrected visual acuity better than 20/40 and open angles. 

Approximately 80% of these 98 eyes (n = 78) completed testing with OCT and 24-2 and 10-2 VFs on the same day. All eyes had OCT and both VF tests completed within 4 weeks, except for one eye whose difference between the 10-2 VF test and the rest of structural and functional testing was 40 days. 

This prospective study was approved by the Institutional Review Board of Columbia University Medical Center and adhered to the tenets of the Declaration of Helsinki and Health Insurance Portability and Accountability Act. Written informed consent was obtained from all participants involved in this study. 

Functional (VF) Data

All 98 eyes had reliable 24-2 and 10-2 VF tests, as defined by fixation losses under 33%, false positives under 15%, and false negatives under 33%. For all eyes, we extracted total deviation (TD) and pattern deviation (PD) age-corrected probability values from both 24-2 and 10-2 VFs. Figure 1A shows an example of the 24-2 and 10-2 conventional Humphrey Field Analyzer (HFA; Carl Zeiss Meditec, Inc., Dublin, CA) reports from one of the study eyes. The black frames highlight the TD and PD probability maps. Since the probabilities in these maps are not continuous, we recorded them as observations of ordinal variables; these values were ≥5%, <5%, <2%, and <1%. 

Figure 1.

 

An example of the functional (24-2 and 10-2) and structural (one-page wide-field OCT^3,6) reports used in this study. Black frames highlight the total deviation and pattern deviation probability maps that provided the functional information. Purple frames highlight the RNFL and RGC+ probability maps that provided the structural information.

Structural (OCT) Data

Wide-field (12 × 9 mm) swept-source OCT volume scans (Atlantis; Topcon, Inc., Tokyo, Japan) were obtained for each eye. Thickness values of the RNFL were extracted. In addition, a 6-mm × 6-mm macular scan was derived from each wide-field scan in order to retrieve thickness values of the retinal ganglion cell plus inner plexiform layer (RGC+). Using a reference database from the OCT device manufacturers (data provided by Topcon, Inc.), we generated age-corrected deviation/probability maps, as previously described and employed in an established OCT wide-field report.⁶ The reference database included eyes with (1) normal 24-2 VFs (abnormal defined as Glaucoma Hemifield Test “outside normal limits” and/or Pattern Standard Deviation (PSD) <5%), (2) no eye pathology, and (3) intraocular pressure lower than 21 mm Hg. The purple frames in Figure 1B highlight the two probability maps (upper: RNFL; lower: RGC+) on the OCT wide-field report. The scale in these maps is continuous from dark red (<0.1%) through red (1%) to yellow (5%) to green (>10%). Both probability maps (RNFL and RGC+) are rotated around the horizontal meridian, so the upper region corresponds to superior VF and inferior retina; thus, they are in field view. 

The wide-field OCT scans were acquired without taking the individual's fovea-to-disc (FD) angle (i.e., vector) into consideration; we call these unrotated scans. In order to correct for head-eye torsion, as well as some FD angle-related anatomical differences (see Fig. 16 in Hood et al.⁷), we rotated each scan such that the individual's FD vector aligned with the mean FD vector calculated from the reference database⁸; we call these “rotated scans.” RNFL and RGC+ thickness probability values, along with their corresponding coordinates, were extracted from both unrotated and “rotated” scans. Since the “rotated” scans showed slightly, but not significantly, better performance in assessing aS-aF agreement (see Supplementary Table S1 for more information), we used the results of “rotated” scans. 

Combining Structural and Functional Data

The newly developed R program takes as input the functional and structural probability data described above. An example of the program's output is given in Figure 2 with the RGC+ map on the left (Fig. 2A) and the RNFL map on the right (Fig. 2B). In each case, the VF locations are shown for the 10-2 (small circles) and 24-2 (large circles). Note that the VF locations within the central 10 degrees were morphed to adjust for RGC displacement near the fovea as previously described.^9,10 The circles denoting VF locations were either filled black circles to represent abnormal values below a given threshold or remained nonfilled to indicate within normal limits (WNL). Given the nature of the OCT probability data (i.e., probabilities are given on a continuous scale), the program presents the OCT probability maps from green (>10%), through yellow (<5%) and red (<1%), to dark red (<0.1%). 

Figure 2.

 

An example of the output of the R program. (A, B) The RGC+ and RNFL probability plots, respectively, with the 24-2 and 10-2 locations superimposed as large (24-2) and small (10-2) circles. A filled (black) circle indicates an abnormal VF location. The probability scale for the OCT map varies continuously from green (P > 10%) to dark red (P < 0.1%). The locations enclosed in the large symbols indicate aS-aF agreement for the 24-2 (diamonds) and 10-2 (squares) locations.

The locations where abnormal VF (aF) locations (filled symbols) correspond with abnormal structure (aS) on either RNFL or RGC+ maps (i.e., aS-aF agreement) are indicated with a large square or diamond to differentiate aS-aF agreement on the 10-2 or 24-2, respectively. To determine the aS-aF agreement, the program first constructs an OCT superpixel map. For each VF location, the structural probabilities are averaged for a superpixel with a radius of 0.5 degrees centered on that location. The 0.5-degree radius was chosen in order to construct a structural superpixel that would be slightly bigger than the VF Goldmann stimulus size III. Each averaged superpixel is compared with the criterion defined by the user to derive whether that location is WNL or abnormal. For the purposes of this study, we used three different structural criteria (P <15%, <10%, and <5%) and three functional criteria (P <5%, <2%, and <1%). The large diamonds and squares indicate agreement for a particular pair of aS-aF criteria. In the case of Figure 2, the criteria were <5% for VF and <10% for OCT. These criteria resulted in a total number of 23 aS-aF locations for this eye. 

Identifying Optimal aS and aF Criteria to Categorize Individual Eyes

We evaluated all nine possible pairs of VF criteria (1%, 2%, and 5%) and OCT criteria (5%, 10%, and 15%) for both PD and TD probability maps. The number of locations showing aS-aF agreement was determined automatically for each eye for the nine sets of criteria. 

For the purposes of our analysis, we assume that the optimal criteria set will maximize the number of DG eyes showing aS-aF agreement and the number of H eyes not showing aS-aF agreement. Note that we are not suggesting that aS-aF agreement should be used to define glaucoma, although we do suggest a potential translational use in the Discussion. 

Validating the Optimal Criteria on a Different Set of Eyes

To assess whether our method would present similar frequencies of aS-aF agreement on a different data set, we used 101 eyes with confirmed or suspected POAG from a sample that has previously been reported.⁶ These eyes also had 24-2 MD greater than or equal to –6 dB, similar to the eyes involved during development. As previously described,⁶ two glaucoma experts/clinicians reviewed VFs (24-2 and 10-2), fundus photos, patient chart information, and a single-page wide-field OCT report from each eye and labeled each as healthy, probably healthy, with presence of optic neuropathy (ON), or with probable ON. From the 101 eyes, 57 (mean age: 52.66 ± 10.51) were classified as glaucomatous based on the judgment of the two experts (i.e., ON or probable ON), while 44 eyes (mean age: 61.71 ± 15.69) were identified as suspects if the two clinicians labeled them as healthy or probably healthy. More details on the nature and characteristics of these eyes can be found in Hood et al.⁶ 

We applied the set of cirteria (i.e., OCT-VF) that showed the best results in identifying aS-aF agreement in the DG eyes and lack of agreement in the H eyes of the previous data set. In a similar manner as before, we hypothesized that most of the 57 glaucomatous eyes should show aS-aF agreement, while in general, the 44 suspect eyes should not. 

Results

To illustrate our approach, Figures 3 and 4 illustrate typical results with a criterion for an aS location of <10% and a criterion for an aF location of <5%. For the H eye in Figure 3, there are a few “abnormal” VF (aF) locations with VF values <5%, but none of these locations show aS-aF agreement, and consequently, the eye does not meet any of our criteria for aS-aF agreement. On the other hand, Figure 4 shows the results for one of the DG eyes. The RGC+ plot (left panel) and RNFL (right panel) show five and eight locations with aS-aF agreement, respectively. Thus, both of these maps in this example would meet the criteria of aS-aF if the cutoff threshold was, for example, “≥5” abnormal aS-aF points. 

Figure 3.

 

The maps as in Figure 2 for a healthy/control (H) eye with a few abnormal VF locations. None of these locations show an abnormal structural region, and thus there are no aS-aF locations.

Figure 4.

 

The maps as in Figure 2 for an eye (DG) with early glaucoma that shows aS-aF agreement in both the RGC+ (A) and the RNFL (B) probability plots. If a criterion of “OCT <10% and VF <5%” is applied, this eye shows five aS-aF locations in the RGC+ and eight in the RNFL.

Identifying Optimal aS and aF Criteria to Categorize Individual Eyes

The aS-aF agreement was evaluated with the nine sets of OCT-VF criteria for both PD (Table 1) and TD (Table 2) VF probability maps. The two tables provide three percentages of frequencies at each criterion for each cutoff threshold level: (1) the percentage of DG eyes that show aS-aF agreement, (2) the percentage of H eyes showing no aS-aF agreement, and (3) the total percentage of (1) and (2) combined, in brackets. 

Table 1.

 

Percentage of Eyes Showing aS-aF for Each OCT-VF (Pattern Deviation) Criterion Pair and Each Threshold Number of Locations

 

Table 2.

 

Percentage of Eyes Showing aS-aF for Each OCT-VF (Total Deviation) Criterion Pair and Each Threshold Number of Locations

 

A VF criterion of <5% was consistent with our hypothesis more frequently than the other two criteria (<2% or <1%). Hence, we focus here on the pairs of criteria with a VF criterion of <5%. 

Pattern Deviations

For the PD VF probability maps, the “OCT <15% (aS) – VF <5% (aF)” criterion with a threshold of two aS-aF locations identified aS-aF agreement in DG eyes, as well as no aS-aF agreement in H eyes, in 91 of 98 (92.9%) eyes, the highest among the different criteria sets and their thresholds; see double underlined percentages in Table 1. Of the remaining seven eyes, two were glaucomatous (i.e., DG without aS-aF agreement) and five were healthy controls (i.e., H with aS-aF agreement). 

Total Deviations

Similar to the PD probability values, the two criteria that most frequently identified aS-aF agreement in DG eyes, as well as no aS-aF agreement in H eyes, were the “OCT <15% – VF <5%” and “OCT <10% – VF<5%”. The latter criterion was consistent with our hypothesis in 86 of 98 (87.8%) eyes, the highest among other criteria applied on TD probability maps but five eyes less than the highest in the PD probability maps (see Table 2). 

Validation of Optimal OCT-VF Criteria

We tested the “OCT <10%, VF <5% – PD” criterion with a cutoff threshold point of two locations on the independent and previously described sample of eyes (see Methods). Of the 57 glaucomatous eyes, 54 (94.7%) showed aS-aF agreement. On the other hand, only 3 (6.8%) of the 44 suspect eyes showed aS-aF agreement, even though these eyes were not healthy controls but rather glaucoma suspects. 

Discussion

Glaucoma diagnosis often involves the assessment of structure-function agreement. Clinicians regularly seek a correlation between the appearance of structural defects seen on OCT scans and the appearance of functional defects on 24-2 or 10-2 VF tests. To overcome subjectivity in this assessment, we developed an automated method that combines structural and functional data and topographically compares abnormal regions on wide-field OCT and VF probability maps. We tested our technique on a sample that purposely included early glaucomatous eyes (24-2 MD better than –6 dB), as good aS-aF agreement is expected in eyes with moderate and severe glaucoma using a variety of criteria. In addition, we selected glaucomatous eyes with a “definitely glaucoma” classification from the referring physician to avoid diagnostic regions of uncertainty. Table 1 can be used to choose the best criterion according to a particular research/clinical purpose. If one seeks, for example, a criterion that allows 5% of the H eyes to show aS-aF agreement, then the “OCT <10% – VF <5%” with two aS-aF locations would be the most appropriate. On our sample, this criterion would identify only two H eyes with aS-aF agreement (i.e., 4.4% of the H group) while 47 DG eyes would show aS-aF agreement (i.e., 88.7% of the DG group); see bold percentages in Table 1. If, on the other hand, say for screening purposes, one wanted to maximize the number of DGs showing aS-aF agreement, then a different criterion could be chosen. 

We developed this method to provide an automated technique that objectively determines the topographic agreement between regions of aS-aF. We are not suggesting our method should be used as a reference standard or as a definition of glaucoma, or for the purpose of screening for glaucoma. We are suggesting that this method will be better than other objective methods for assessing the extent to which, and the conditions under which, aS and aF agree. For example, in a recent study, we present evidence that our automated method performs better than more traditional methods based upon summary statistics such as PSD and global RNFL thickness.¹¹ 

We evaluated the performance of our method for both the TD and PD probability maps from the 24-2 and 10-2 VF tests. The results showed a nonsignificant superiority of the PD probability map over the TD one. This is not particularly surprising as the TD map can be affected by cataract and response criteria. Although we did not control for pseudophakic eyes or cataract presence (or absence or stage), our sample eyes did not have signs of significant cataract upon enrollment in their study. Presumably had we included eyes with significant cataracts, the advantage of the PD maps would have been greater. 

Despite the high performance in detecting aS-aF agreement reported here, there is still room for further improvement. For example, information about the pattern of RNFL defects could be incorporated, as illustrated in Figures 6 to 8 and 18 in Hood.¹² Consider the glaucomatous eye in Figure 5 of this study. This eye did not show aS-aF agreement for our topographic method with any of our criteria. However, the VF shows two typical arcuate defects (one in the superior and one in the inferior hemifield) on the nasal side of the field, while in general agreement, the wide-field OCT shows both superior and inferior RNFL defects of an arcuate form. While direct comparison of aS-aF regions may not indicate agreement in this eye, we would argue that there is a clear aS-aF agreement in this case. Two hypothetical defects, one superior and one inferior, are shown based upon the RNFL bundle trajectories from Jansonius et al.¹³ as previously described.¹² Note that this defect pattern makes the agreement clear. Future incorporation of such information should improve the detection of aS-aF agreement. 

Figure 5.

 

The maps as in Figure 2 for an eye with early glaucoma with arcuate structural and functional defects in both the superior and inferior regions. However, only one point shows aS-aF, and thus the eye does not show aS-aF agreement by our method. Two hypothetical defects, one superior (red) and one inferior (black), are marked based upon a model of retinal nerve fiber bundle trajectories¹³ to make the defect patterns and therefore the aS-aF agreement in this case clear.

The present study has limitations/concerns worth addressing. First, the sample size of 53 glaucomatous and 45 healthy control eyes, involved in the development and evaluation of the new method, is relatively small. Nonetheless, the results of this study agree with a previous qualitative analysis of a different data set of a similar nature,² and the results from the validation group of 101 eyes showed excellent agreement. The age difference between the two groups (i.e., DG and H eyes) is also not desirable, mainly in diagnostic studies, although ours is not. However, the fact that we are using age-corrected probability information, both structural and functional, for assessing topographical agreement should minimize the effects of the age difference. Nonetheless, this does not completely eliminate concerns about possible effects due to age, and age-similar control-patient groups should ideally be used in attempts to replicate our findings. Second, the functional information provided in this study is limited to the recording of VF probabilities in ordinal variables, as those are provided from the HFA report. While a criterion of <5% was the best-performing VF criterion, there was no capability of investigating higher probability thresholds (e.g., <10% or <15%) that might yield even better aS-aF agreement. Third, wide-field OCT scans were selected as input of structural information (i.e., RNFL and RGC+ probabilities). The scanned region of 12 × 9 mm is larger than that covered by the more commonly used two 6-mm × 6-mm volume scans centered on the fovea and disc. However, it is still smaller than the retinal area tested by the wider 24-2 VF test. As a result, evaluation of aS-aF agreement is limited to only a few locations of the 24-2 (in most cases approximately half of them) but not to the 10-2 where all VF locations are included in the scanned wide-field area. Fourth, the radius size of the structural superpixel was set at 0.5 degrees, a size that is between Goldmann sizes IV and V. Future investigations could test the differences in the detection of aS-aF agreement based on the various Goldmann sizes or other techniques such as M-scaling. Finally, the diagnosis of DG relied solely on the judgment of the referring glaucoma specialist who had both functional (i.e., VF) and structural (i.e., fundus photos, OCT, etc.) information available. While this may not be an ideal reference standard, and indeed there is no generally accepted “structure–function” definition of glaucoma, we chose these eyes as we were reasonably confident about their status (i.e., glaucomatous or healthy controls). The results shown in DG and H eyes of the first data set are further confirmed by those of the validation group, a group whose diagnosis depended upon two specialists who were not involved in the care of these patients. 

In conclusion, the method described here can automatically and objectively evaluate aS-aF agreement with a direct comparison of abnormal regions of function and structure. 

Acknowledgments

Supported by National Institutes of Health grants EY-02115 (DCH) and EY-025253 (CGDM). 

Disclosure: E. Tsamis, None; N.K. Bommakanti, None; A. Sun, None; K.A. Thakoor, None; C.G. De Moraes, Carl Zeiss Meditec, Inc. (R), Topcon, Inc. (R), Heidelberg Engineering (R), Novartis, Inc. (C), Galimedix, Inc. (C), Lin Biosciences, Inc. (C), Reichert, Inc. (C); D.C. Hood, Topcon, Inc. (F, C), Heidelberg Engineering (F,C), Novartis, Inc. (C)