Evaluation of a One-Page Report to Aid in Detecting Glaucomatous Damage

Abstract

Purpose: : We assessed the use of a customized, one-page structure + function report for aiding in detection of glaucomatous damage.

Methods: : Two individuals (report specialists), experienced in analyzing optical coherent tomography (OCT) and visual field (VF) results, examined a customized one-page report for 50 eyes from 50 patients who either had glaucoma or were glaucoma suspects. The report contained key features of OCT scans with VF information. All patients had 24-2 VFs with a mean deviation (MD) better than −6 dB. The report specialists classified each hemifield and eye as either glaucomatous or nonglaucomatous based upon only the customized report, either without (phase 1) or with (phase 2) 24-2 VF information included on the report. Their results were compared to the classifications made by 3 ophthalmologists (glaucoma specialists) based upon traditional measures, namely stereo photographs, 24-2 VFs, and a commercially available, OCT disc scan report.

Results: : The two report specialists agreed on all but one eye and four hemifields in phase 1, and on all eyes and all but one hemifield in phase 2. In phase 2, they judged 31 eyes abnormal. Of these 31 eyes, 30 were judged abnormal by all three glaucoma specialists and the 31st by two of the three. Without the VF information (phase 1), one report specialist classified 1, and the other 2, of these 31 “abnormal” eyes as normal.

Conclusions: : When using the one-page report, the experienced readers showed excellent inter-rater repeatability and diagnostic ability relative to glaucoma specialists.

Translational Relevance: : This condensed report may help the clinician assess glaucomatous damage.

Introduction

Glaucoma is diagnosed using a combination of a functional test, commonly the visual field (VF) obtained with static automated perimetry, and a structural test, traditionally optic disc fundus stereo photographs. Glaucoma specialists have increasingly relied upon optical coherence tomography (OCT) to confirm structural damage. 

Numerous studies have shown that OCT summary measures of retinal nerve fiber layer (RNFL) thickness around the optic disc have good sensitivity and specificity for detecting glaucomatous damage (see Refs. 1 and 2 for reviews). However, Hood and Raza³ recently argued that the current effectiveness of OCT could be improved by visually examining Fourier-domain OCT (fdOCT) peripapillary images, by examining the local thinning of the RNFL and retinal ganglion cell (RGC) layer, and by comparing this local thinning to abnormal regions seen on the VF. They proposed a one-page report that summarizes the information from OCT macular and disc cube scans, and allows a direct comparison to the results from VF tests. Although they warned against summarizing the rich and complex information available from OCT scans in a single page report, they suggested that a qualitative analysis of this report would typically be sufficient to make a diagnosis of glaucomatous damage and, in any case, it would be an improvement over how OCT typically is used by clinicians. 

To further our understanding of the use of this report, two individuals without formal ophthalmological training, but who developed the report, determined whether eyes of patients were normal or abnormal based only on the one-page report with OCT and VF information. Their results were compared to those of glaucoma experts using stereoscopic photographs, VFs, and a commercial RNFL OCT report. To focus on early damage, which typically is difficult to detect with any single test, only patients with mild glaucoma or who were glaucoma suspects were selected. 

Methods

Patients

We evaluated 50 eyes of 50 patients meeting the inclusion criteria. Based upon the stereo photograph appearance of the disc, all eyes were considered abnormal or suspicious by the referring glaucoma specialist. For inclusion, the mean deviation (MD) of the 24-2 VF had to be better than −6 dB; the VFs did not have to be abnormal. For inclusion, an eye required gonioscopically open angles, disc stereo photographs, 24-2 VF tests, and fdOCT scans within 6 months. Eyes were excluded if the refractive error was >6D; if their cataract scores, as defined by slit-lamp examination, were equal to or worse than N02, NC02, C2, and P2.24 on the Lens Opacities Classification System III (LOCS III); or if the eyes had other conditions likely to affect the VF results (e.g., corneal opacity, neurophthalmologic or retinal diseases). 

Patients had a mean age of 57.5 ± 15.2 years (range, 18.6–77.4 years). The mean MD ± SD on the 24-2 VF test was −2.08 ± 1.94 dB (range, 1.81 to −5.84) and the best-corrected visual acuity ranged from 20/20 to 20/30. 

VF, Stereo Photography, and OCT Tests

The 24-2 VFs were obtained using SITA-Standard Automated Perimetry (Humphrey Field Analyzer; Carl Zeiss Meditec, Inc., Dublin, CA). The VF examination closest in date to the OCT test was used. Simultaneous stereo photographs of the optic disc were obtained with a Nidek 3-Dx mydriatic fundus camera (Nidek, Inc., Gamagori, Japan). All stereo photographs were analyzed on a computer screen with the aid of a stereoviewer. 

Cube scans of the macula and disc (6 × 6 mm, 128 horizontal B-scans with 512 A-scans each), as well as a peripapillary circle scan (1.7-mm radius, 1024 A-scans with at least 16 overlapping averages) were obtained (3D-OCT 2000; Topcon Corp. Paramus, NJ). The RNFL and RGC + inner plexiform (RGC+) layer were each segmented by the machine's algorithm without any operator correction. 

The Custom OCT Report

A one-page report was created for each of the 50 eyes using a custom program in MATLAB (MathWorks, Inc., Natick, MA). This report (Fig. 1) has been described previously.³ Briefly, the upper left two panels are the peripapillary circle scan (upper) and peripapillary RNFL thickness profile (lower) plots. The curves in the latter include the RNFL thickness derived directly from the circle scan (dashed) as well as from the RNFL thickness extracted from the disc cube (solid) scans. The derived (solid) curve typically is the more reliable measure of RNFL thickness as the circle is aligned with the disc center after the scan is obtained. The direct circle scan (dashed) has better spatial detail and is examined to assess the segmentation and to see local RNFL details. 

Figure 1.

 

The one-page report used by the report specialists in phase 2. See text and the study of Hood and Raza³ for a description.

These two panels are oriented as if the scan started and ended in the nasal quadrant (at 3 o'clock for a right eye and 9 o'clock for the left). With this NSTIN (nasal-superior-temporal-inferior-nasal) plot, the central portion of the scan corresponds to the central portion of the VF, thus making it easier to compare the RNFL thickness plots to VF results.³ In particular, “0 degrees” in Figure 1 (upper left two panels) corresponds to the center of the temporal quadrant of the disc (9 o'clock for a right eye, 3 o'clock for the left), which receives input from the papillomacular RNFs. While some commercial software has an NSTIN plot option, the typical (default) is a TSNIT plot in which the plot/scan starts in the center of the temporal quadrant. In the TSNIT plot, the region receiving input from the central portion of the VF is split and appears at the ends of the plot/scan. 

The three lower pseudo-color maps on the left are the peripapillary RNFL, macular RNFL, and macular RGC+ thickness presented in retinal view. To compare the changes in thickness to VFs, these maps are converted into probability maps by comparing the thickness at each point to the thickness of a group of healthy controls.³ In particular, the pseudo-color maps in the right panels are the probability maps for the RNFL (upper) and RGC+ (lower) thickness. They are presented in field view by flipping them along the horizontal meridian so that the inferior retina is above the superior retina. This allows a topographical comparison to the VF data. In the upper right panel, the information from the 24-2 VF test is superimposed upon the RNFL probability map. The small black dots are test locations falling above a probability of 0.10, while the colored points are associated with probabilities less than 0.10 as indicated by the scale to the right. This scale applies to the VF and OCT data. 

Classification Based upon the Custom One-Page Report

Two report specialists, who had developed the report and who had extensive experience examining VFs and OCT scans of patients with glaucoma, independently evaluated the report for each eye. For each eye, they only had the one-page report; they were masked to all other clinical information, including the actual VF and commercial OCT reports produced by the commercial machines. They were asked to classify each eye, as well as each hemifield, as abnormal or normal. In phase 1, the task was first done without the 24-2 VF information on the report (Fig. 2). When this phase was completed, they were given the report with the 24-2 VF information (Fig. 1) and given the opportunity to change their classifications (i.e., they had access to their previous classification in phase 1). 

Figure 2.

 

The same report as in Figure 1, but without the 24-2 visual field information.

Classification by Glaucoma Specialists

To evaluate the results from the report specialists, 3 ophthalmologists (glaucoma specialists), masked to all nonstudy patient data, evaluated stereo photographs, 24-2 VFs, and commercial OCT reports. The referring physician was not one of the evaluators. In phase 1, the glaucoma specialists were given the standard 24-2 VF report and stereo photographs as shown in Figure 3 for the same eye as in Figures 1 and 2. In phase 2, the standard OCT report (Fig. 4) produced by the commercial OCT machine was added and the glaucoma specialists were given the opportunity to change their classifications (i.e., they had access to their previous classification in phase 1). 

Figure 3.

 

An example of the information available to the glaucoma specialists during phase 1. Upper panel: disc fundus stereo photographs. Lower panel: 24-2 visual field report.

Figure 4.

 

The commercial peripapillary circle scan added to the information the glaucoma specialist had in phase 2.

Results

Report Specialists and the Report without VF Information

In phase 1, the report specialists viewed the version of the one-page report that contained the RNFL and RGC+ data, but not the VF data. With this report, the specialists agreed in 49 of the 50 eyes; 29 of these 49 eyes were classified as abnormal, while 20 were classified normal. Thus, they disagreed in the case of only one eye (P46). In addition, one specialist noted that one eye (P6) that he classified as normal had a “hole” in the RNFL (see P6, Fig. 5) and, thus, based upon past work,⁴ may have very early glaucomatous damage. 

Figure 5.

 

The white arrow indicates the hypodense region (“hole”) seen on the OCT scan of patient P6.

Further, the report specialists agreed on all but 4 (4%) of the 100 hemifields. Thus, with only the OCT information, there was good agreement between them. 

Report Specialists and the Report with 24-2 VF Information

Adding the VF information to the report further increased the agreement between the report specialists. They agreed on the classification of all eyes and all but one hemifield. In particular, they classified 31 of the 50 eyes as abnormal and 19 as normal. The 31 eyes classified as abnormal included the 30 eyes previously classified as such in phase 1 (29 by one specialist and the same 29 plus one other by the second), as well as an additional eye (P3). Figure 6 shows the report for this eye. The OCT results by themselves were judged to be within the normal range in phase 1. However, when the borderline RNFL thinning in the inferior quadrant of the disc (red arrows in the upper panels) was compared to the location of the abnormal points on the 24-2 (black arrow, upper right) in phase 2, both specialists changed the classification of this eye from normal to abnormal. 

Figure 6.

 

The report for patient P3 showing subtle OCT thinning (red arrows) and abnormal visual field locations (black arrow).

After the 24-2 VF information was added, one report specialist changed the categorization of three hemifields, while the other changed six. In all nine hemifields, the VF led to a change in the classification of the hemifield from “normal” to abnormal. (Fig. 6 is an example.) Further, the report specialists disagreed in the case of only one hemifield. 

Comparison to Glaucoma Specialists

After phase 1 involving the stereo photographs and 24-2 VFs, all three glaucoma specialists agreed that 24 of the eyes were abnormal; they lacked unanimous consensus on 26 eyes. Adding the OCT commercial report in phase 2 increased the agreement, although there still was a lack of consensus on 19 eyes. (Note that even the two specialists showing the best agreement did not agree in the case of 18 eyes after phase 1 and 11 eyes after phase 2.) 

In any case, after phase 2, all three glaucoma specialists classified 30 of the 50 eyes as abnormal. All 30 of these eyes were judged abnormal by the report specialists. The 31st eye judged abnormal by the report specialists was judged abnormal by two of the three glaucoma specialists. An additional four eyes also were judged abnormal by two of the three glaucoma specialists. 

In general, the report specialists showed excellent agreement with the glaucoma specialists. However, in the case of four eyes, the two report specialists and one of the glaucoma specialists disagreed with the other two glaucoma specialists. Figure 7A shows the total deviation and pattern deviation probability plots for these four eyes. Except for P23, the VF abnormalities are subtle in these eyes. Note, for example, the glaucoma hemifield test (GHT) is within normal limits (WNL) for P42 and borderline in the cases of P13 and P19. In addition, the commercial OCT report for three of these eyes was normal; only one eye (P19) showed borderline RNFL thinning. Thus, with the possible exception of P23, it appears that two of the glaucoma specialists based their evaluation heavily on the stereo photographs, which two of them judged to be abnormal. (The stereo photographs for these eyes can be found in the Supplementary Material.) Conversely, the report specialists seemed to have been basing their judgments heavily on the OCT findings, discounting the VF abnormalities because they did not correspond to the normal-appearing RNFL. (The RNFL plots from the customized reports are shown in Figure 7B.) In the case of P23, the eye with the clear VF defect, they chose to assume it was an artifact rather than conclude the healthy-appearing RNFL on OCT (Fig. 7B, lower left panel) was a false negative. 

Figure 7.

 

(A) The 24-2 total deviation and pattern deviation probability plots for the four eyes judged normal by one glaucoma specialist and the two report specialists, but abnormal by two of the glaucoma specialists. (B) The peripapillary RNFL thickness plots from our report for the same four eyes as in (A).

Finally, after the study was completed, the referring physician (RR), who was not one of the three glaucoma specialists involved in classifying eyes, reviewed all the information currently available in the patient's chart, including the one-page report, as well as VFs from previous visits. He was “fairly sure” there was no damage in the case of P42. Although he was not sure in the cases of P13 and P19, his “best guess” was “no damage.” Finally for P23, he was “fairly sure there is damage.” However, he is considering getting a consult from a neuro-ophthalmologist because in his opinion the disc appearance did not match the VF defect. 

Discussion

Our purpose was to assess the potential use of a customized, one-page report that visually combined topographic peripapillary and macular RNFL thickness and probability plots, macular RGC+ thickness and probability plots, and a VF probability plot. In the hands of two individuals with extensive experience, the inter-rater agreement was excellent. After the report specialists viewed the report with the topographic OCT information and VF probability plots, they agreed on all eyes and disagreed in the case of only one hemifield. Interestingly, the two glaucoma specialists who showed the best agreement with each other disagreed in the case of 11 eyes after phase 2, in which they had traditional stereo photographs, VFs, and the OCT commercial reports. 

It is important to emphasize that this study was not designed to determine whether the report specialists would show better agreement than glaucoma specialists. The report specialists had worked together for approximately 8 years, while the glaucoma specialists worked independently. While others have reported that glaucoma specialists show only moderate agreement when asked to detect glaucomatous damage with stereo photographs,^5–7 we are aware of only one study assessing agreement on glaucomatous damage when disc photographs, VFs, and commercially available traditional OCT information were available. This recent study by Bae et al.⁸ found only slightly better than moderate agreement (κ of 0.63, n = 7). While there are a number of key differences between our studies (e.g., 25% of the eyes in their study had more extreme damage than any of the eyes in our study), our results were similar. Our two glaucoma specialists with the best agreement also showed moderate agreement (κ of 0.50). 

In assessing the presence of glaucomatous damage, the report specialists' results compared well to those of glaucoma specialists using a more traditional approach. Of the 31 eyes judged abnormal by the report specialists, 30 were judged abnormal by all three glaucoma specialists. In fact, these 30 were the only eyes judged abnormal by all three glaucoma specialists. The 31st eye judged abnormal by the report specialists was judged abnormal by two of the three glaucoma specialists. In addition, two of the three glaucoma specialists judged four additional eyes abnormal. Based upon all the evidence available, including information from previous visits in three cases, it was not possible to determine definitively whether these eyes have glaucomatous damage. However, faced with all the evidence in the patients' chart, including the one-page report, the referring physician was fairly sure one of these eyes has damage, and one does not. He was not sure in the case of the other two, but his best guess was “no damage.” A longer follow-up period may help further clarify the status of these eyes. 

Advantages of the One-Page Report

The success of the one-page report with OCT and VF information probably is due to several features. First, including information on a single page encourages the reader to look for topographical consistency among the various RGC, RNFL, and VF plots. Second, presenting the RNFL thickness plot in the NSTIN format, rather than the typical TSNIT format, makes it easier to relate thinning on the RNFL plot to central defects on the VF. Third, by directly comparing the topography of the VF and OCT abnormalities, subtle damage, which may be dismissed on the basis of each alone, can be confirmed. Figure 6 is an example from the present study. Fourth, as suggested by Hood and Raza,³ the large scan image in the upper left panel of the report can be examined for algorithm errors, as well as for structural abnormalities. An example of the latter is the “hole” seen in Figure 5. Based upon previous work, this eye, classified as normal by two of the three glaucoma specialists, likely has early glaucomatous damage.⁴ Finally, by including the RGC+ and RNFL thickness information from the macular cube scans, glaucomatous damage to the macula can be detected, including damage that may be missed on the peripapillary RNFL thickness plots.^3,9 

The Value of Topographic Data from OCT Scans

We were somewhat surprised by how well the report specialists performed without VF information, particularly in light of the early spectrum of disease. With only the OCT information in phase 1, one report specialist classified as abnormal 29 of the 30 eyes judged abnormal by all five specialists after phase 2, while the other identified 28. Thus, adding the VF information only added one (2%) or two (4%) additional abnormal eyes, and either three (3%) or six (6%) additional abnormal hemifields, depending upon the specialist. Of course, to assess the relative value of the VF versus the OCT information, the order of presenting OCT and VF information must be counterbalanced. For example, in the Bae et al.⁸ study, OCT information was added after fundus photographs and VFs, and they concluded that it added relatively little to rater agreement. They did, however, conclude that it was likely very useful in cases of suspicious discs and vague VF defects, conditions similar to those for many of the eyes in our study. Our results for the glaucoma specialists support this conclusion. Adding the OCT commercial reports in phase 2 increased the agreement among the glaucoma specialists. In the case of the two glaucoma specialists showing the best overall agreement, they disagreed in 18 of the eyes when only the stereo photographs and 24-2 VF were available and 11 eyes when OCT commercial reports were added. Thus, the OCT information improved the agreement in the case of 7 (14%) of the 50 eyes. 

Limitations/Caveats

This study had a number of limitations. First, the absence of a healthy control group precluded a true estimate of a false-positive rate. Second, as with most OCT studies, the results do not necessarily apply to those with high myopia, which were not included in the sample. Third, and most important, the experimental design does not allow us to determine how much of the success of the one-page report is due to its format and content, as opposed to the skills and experience of the two report specialists. Studies in which individuals are trained to read the one-page report are needed. A number of possible studies could be done. For example, the agreement among representative clinicians involved in diagnosing glaucoma could be assessed before and after replacing the commercial OCT report with the one-page report. Of course, a training procedure would be needed to help the clinician interpret the one-page report. 

Finally, the inclusion criteria were designed to focus on the detection of early glaucoma, as more advanced cases are less likely to be misdiagnosed. Thus, our sample included eyes that are among those most challenging for glaucoma specialists and did not include eyes with very clear glaucomatous damage. While this is a strength of the current study, it does mean that the eyes studied are not a representative sample of those seen in clinical practice. This should be taken into consideration when making comparisons to other studies. 

Conclusion and Future Work

This is a proof of concept study. If two specialists familiar with the one-page report had shown poor inter-rater agreement and/or clearly had performed more poorly than glaucoma specialists using a more traditional approach, then there would be little point in proceeding with additional studies. However, the two specialists showed excellent agreement with each other, as well as with the glaucoma specialists. However, we are not suggesting replacing VF reports, fundus photographs, clinical measures such as IOP, or the patient's history with a one-page report. We are suggesting that careful analysis of a one-page report by trained readers or clinicians may offer the opportunity to better classify glaucoma suspects as either normal or glaucomatous. We suggest that the one-page report with OCT and VF information should at least augment the commercial OCT reports. The extent to which it can replace commercial OCT reports, as well as other clinical information, remains to be tested. 

Acknowledgments

We thank those involved in the recruiting and testing of the patients in this study, including Monica Chen, Rithu Ramachandran, Ilana Traynis, Diane Wang, and Daiyan Xin, and especially Jeff Odel for his encouragement and advice during the development of the customized report. 

Supported by National Institute of Health Grant R01-EY-02115 (DCH) and a grant of equipment from Topcon, Inc. 

Disclosure: D.C. Hood, Topcon, Inc. (F, C); A.S. Raza, None; C.G. De Moraes, None; P.A. Alhadeff, None; J. Idiga, None; D.M. Blumberg, None; J.M. Liebmann, Topcon, Inc. (F, C); R. Ritch, None