Objective Grading of Tonography Tracings for the Measurement of Outflow Facility

Abstract

Purpose: Tonography is the standard method for non-invasive measurement of aqueous humor outflow facility. However, assessment of tonography tracing quality is currently subjective, and acceptance of poor-quality data or inappropriately discarding valid data can bias results. The purpose of this study was to develop an objective method for assessing the quality of tonography tracings.

Methods: Pneumatonography tracings were obtained from glaucoma and ocular hypertension patients as part of an ongoing multicenter study of aqueous humor dynamics. Intraocular pressure (IOP) was captured digitally at 40 Hz over 2 minutes. Root mean square error (RMSE) of a linear best-fit line was obtained for each tracing. Each tracing was also graded by seven experienced tonographers using a scale of 1 (worst) to 10 (best) for quality (Expert score). A Reference set of 35 tracings was used to determine the relationship between RMSE values and Expert scores using a logarithmic curve. This relationship was used to calculate a predicted score in a second Test set of 20 tracings. A logarithmic curve was used to account for the fixed range of Expert scores and unbounded upper range for RMSE values. The differences between the predicted scores and the Expert scores were evaluated using Bland–Altman analysis.

Results: There was a very strong correlation between predicted and Expert scores (R = 0.94). The mean difference between Expert and predicted scores was −1.01 ± 0.84, and the limits of agreement were between −2.65 and +0.63.

Conclusions: Objective assessment of pneumatonography tracings can be performed using RMSE of a fitted line and calculation of a predicted quality score that closely matches scores given by expert graders.

Translational Relevance: Tonography tracing quality can now be objectively assessed.

Introduction

Reduction of intraocular pressure (IOP) is currently the only demonstrated effective treatment for glaucoma.¹ Elevated IOP in patients with glaucoma is primarily due to a reduction of aqueous humor outflow facility.^2,3 Aqueous humor is formed by the ciliary processes; it flows through the pupil and anterior chamber and exits the eye at the anterior chamber angle. It returns to the venous system primarily through the trabecular or conventional pathway, with a secondary route through the uveoscleral or unconventional route.^4,5 The ease of aqueous humor outflow through the conventional path is expressed as outflow facility. Although aqueous humor outflow facility is not routinely measured to diagnose glaucoma in current clinical practice, it is used as a research tool to study physiological properties of the eye and the mechanisms of action of glaucoma medications and surgery.^4,6,7 Measurement of aqueous humor outflow facility is therefore a critical component of aqueous humor dynamics for research purposes. 

Outflow facility is most commonly measured noninvasively by tonography, a method based on the ratio of volume change to a known pressure change over a fixed period of time. Pressure is increased by applying a weighted tonometer to the cornea, such as the electronic Schiøtz tonometer originally described by Grant⁸ or the weighted pneumatonometer described by Langham et al.⁹ The force from the tonometer probe increases IOP, which results in an increased aqueous humor outflow rate and a slow decline in IOP toward baseline. The volume change accounting for the measured IOP change is determined from tables specific to the device being used.^9–11 The induced pressure and volume change during a 2- to 4-minute application of the tonometer is then used to estimate outflow facility. 

Although electronic Schiøtz tonography was the standard method for measuring outflow facility for many years, this instrument is no longer commercially available. The pneumatonometer is commercially available and can be adapted for tonography with the addition of a weight to the measurement tip. Previous studies have compared outflow facility measured by pneumatonography and Schiøtz tonography.^6,12,13 Although the measurements are not identical, they are strongly correlated and show similar variability.¹¹ Due to the shorter measurement time for pneumatonography compared with Schiøtz tonography, combined with the wide availability of commercial equipment, pneumatonography is the preferred method for most current studies. 

However, a limitation of tonography is that interpretation of the pressure decay curves remains challenging. In particular, accurate measurement is still highly dependent on a skilled operator and the acquisition of a clear pressure decay curve. Use of poor-quality pressure tracings or inappropriately discarding tracings can lead to bias of results or loss of data. An objective method for assessing pressure tracing quality has been lacking. In this study, we developed and tested an objective method for assessing the quality of tonography tracings based on a regression model to mimic evaluations from experienced users of tonography. 

Methods

Tonography Tracings

Fifty-five tonography tracings were collected sequentially from participants with either mild to moderate open-angle glaucoma or ocular hypertension who were enrolled in a multicenter study of aqueous humor dynamics (ClinicalTrials.gov NCT04412096). Participating centers were part of the Eye Dynamics and Engineering Network (EDEN) consortium and included Mayo Clinic (Rochester, MN), The Ohio State University (Columbus, OH), and University of Nebraska Medical Center (Omaha, NE). Pneumatonography was performed as part of a comprehensive evaluation of aqueous humor dynamics, including IOP, aqueous humor flow rate, outflow facility, episcleral venous pressure (EVP), and calculated uveoscleral flow. 

Each participant underwent a general health interview and comprehensive ophthalmologic examination, including visual acuity, IOP with pneumatonometry, gonioscopy, slit-lamp biomicroscopy, and fundoscopy. Each subject gave informed consent to participate after discussion of the nature and possible risks of the study. This study was approved by the centralized Institutional Review Board at The Ohio State University under the Common Rule, as well as at the two collaborating institutions (Mayo Clinic and University of Nebraska), and followed the tenets of the Declaration of Helsinki. 

Pneumatonography

Outflow facility was measured by using a pneumatonometer (Model 30; Reichert, Depew, NY) with a tonography option. Each 2-minute pressure tracing was digitally acquired at 40 Hz (40 IOP readings per second) using prototype software (Reichert) on a laptop computer connected to the pneumatonometer via the USB port. Subjects were placed in a supine position and asked to breathe normally and fixate on a target on the ceiling approximately 2 meters from their eyes during the procedure. The ocular surfaces were anesthetized by instilling proparacaine 0.5% topically. The eyelids were gently retracted by the examiner's fingers without applying additional pressure on the globe. The pneumatonometer probe (tip diameter of 5.3 mm) with an added 10-g weight was placed on the center of the cornea and held perpendicular to the corneal surface. IOP was continuously recorded for 2 minutes. The probe was then removed from the eye, artificial tears were placed on the eyes, and the subject was instructed to close their eyes. The right eye was always measured before the left eye. Tracings from both eyes were included in the analysis. In addition, some subjects had repeated measurements performed on different days or due to unsatisfactory tracing quality. These tracings were included in the analysis to provide a broader range of tracing scores. Outflow facility was calculated from the pressure decay curves using Langham's pressure–volume relationship tables and a polynomial fitted to the decay curve, as described previously.^9,11 

Quality Measures of IOP Tracings

A linear regression line was fitted to the pressure curve for each tracing using Excel 365 (Microsoft Corporation, Redmond, WA). Tonography tracings were then masked, and each tracing was graded for quality by seven experienced tonographers (AJS, CBT, VG, AK, SF, DR, and SEM) on a scale from 1 (worst) to 10 (best) (Fig. 1). Although it is subjective, expert grading is currently the only method for determining the quality of tracings. The mean grade for each tracing was then calculated as the Expert score and was considered the reference standard. 

Figure 1.

 

(A, B) Examples of poor-quality (A) and good-quality (B) tracings. Poor-quality tracings showed significant deviations from a fitted line and were scored low by a panel of experts. Good-quality tracings showed strong correlation with a fitted line and were scored high by a panel of experts.

Three measures of data fit quality were then calculated for each tracing: 

• 1. Pearson correlation coefficient between the linear regression line (R) and the IOP data collected at 40 Hz
• 2. Root mean square error (RMSE) of data points compared to the regression line, which was determined by calculating the square of the difference between each data point on the IOP tracing and the linear regression line and then calculating the square root of the mean of these squared differences
• 3. EDEN score, which represented a predicted Expert score

The relationship between RMSE values and Expert scores from the initial 35 tracings collected (Reference set) was first determined using a logarithmic regression line. A logarithmic curve was used to account for the fixed range of Expert scores and unbounded upper range for RMSE values. For any new tracing, an EDEN score could then be calculated based on the RMSE value and this regression line. 

Test Set and Statistical Analysis

Twenty tracings, collected after the Reference set, were used as a Test set to evaluate the effectiveness of the three data fit measures at predicting tracing quality, with Expert scores used as the gold standard. The relationship between the data fit measures (R, RMSE, EDEN score) and Expert scores were determined by linear regression analysis, and the strength of relationship was evaluated using the coefficient of determination (R²). Differences between EDEN scores and Expert scores in the Test set were evaluated using Bland–Altman analysis to determine the limits of agreement. Interrater agreement between the expert graders was evaluated using Kendall's coefficient of concordance (W).¹⁴ Potential relationships between Expert scores and outflow facility were explored using linear regression analysis. 

Results

Based on the tracings for the Reference set, there was a strong correlation between Expert scores and RMSE using a logarithmic curve (R² = 0.53) (Fig. 2). Predicted Expert scores (EDEN scores) were assigned based on the following:  

\begin{eqnarray*} {EDEN} \, {score} = 10 \quad {\rm{for}} \quad {\rm{RMSE}} \le 0.68\, {\rm{mm\;Hg}} \end{eqnarray*}

 

\begin{eqnarray*} {EDEN} \, {score} &=& -5.25 \, ln \left( {RMSE} \right) + 7.96\\ && {\rm{for}} \quad 0.68 < {\rm{RMSE}} < 3.77\, {\rm{mm\;Hg}} \end{eqnarray*}

 

\begin{eqnarray*} {EDEN} \, {score} = 1\quad {\rm{for}} \quad {RMSE} \ge 3.77\, {\rm{mm\;Hg}} \end{eqnarray*}

Limits were assigned such that values above 10 were assigned a score of 10, and values below 1 were assigned a score of 1 to correspond with the scale used by the expert graders. 

Figure 2.

 

Logarithmic relationship between Expert scores and RMSE values for linear best-fit curves from each tonography tracing in the Reference set. This relationship was used to calculate a predicted Expert score (the EDEN score) based on the RMSE for any tonography tracing. Error bars show standard deviations in Expert scores.

For the Test set of tracings, the three data fit measures all had different strengths of association with the Expert scores. The Pearson correlation coefficient of the linear regression line for the tracings only had a moderate strength of association with Expert scores (R² = 0.53). In contrast, RMSE had a good strength of association (R² = 0.84), whereas the EDEN score had the strongest association with Expert scores (R² = 0.87) (Fig. 3). In the ideal case, the EDEN and Expert scores should be identical, and our analysis showed that the regression line between the two was not significantly different from the identity line. 

Figure 3.

 

Relationship between Expert scores and EDEN scores with linear regression. There was a very strong correlation between calculated EDEN scores and Expert scores in the Test set of 20 tracings. Error bars show standard deviations in Expert scores.

Comparison of Expert scores and EDEN scores on the Test set using Bland–Altman analysis demonstrated a mean difference of −1.01 ± 0.84, indicating that the EDEN scores were typically slightly lower than Expert scores (Fig. 4). The 95% confidence interval (CI) limits of agreement were 0.63 upper and −2.65 lower. 

Figure 4.

 

Bland–Altman plot showing differences between EDEN scores and Expert scores and limits of agreement.

One tracing in the Reference set appeared as an outlier with a much larger RMSE than the other tracings and was removed from our primary analysis. Inclusion of the outlier resulted in a small decrease in the magnitude of the mean difference between EDEN scores and Expert scores (−0.70 ± 0.86) in the Test set, with no change in the strength of association (R² = 0.87). However, the Bland–Altman analysis demonstrated a negative trend in the differences between EDEN and Expert scores suggestive of proportional bias. This negative trend was not present in the primary analysis without the outlier value being removed. 

Agreement between the scores of the expert graders was excellent based on Kendall's coefficient of concordance (W = 0.87; 95% CI, 0.79–0.91; W = 0 is no concordance and W = 1 is perfect concordance).¹⁴ The mean outflow facility for all the tracings was 0.31 ± 0.20 µL/min/mm Hg. There was a moderate positive correlation between the Expert scores and outflow facility (R² = 0.15, P = 0.004) (Fig. 5). 

Figure 5.

 

Relationship between Expert scores and outflow facility. There was a moderate positive correlation between Expert scores and outflow facility with the Reference sets and Test sets included. Error bars show standard deviations in Expert scores.

Discussion

Tonography is the most common method for non-invasive measurement of aqueous humor outflow facility, and pneumatonography is the only method currently possible using commercially available equipment. However, the technique of tonography can be technically challenging. IOP fluctuates constantly, as shown by implantable pressure sensors in rabbits and non-human primates.^15,16 There are numerous causes for IOP fluctuations,¹⁷ and these all can cause noise in the tonography tracing. Although IOP fluctuations due to the cardiac cycle cannot be avoided, minimizing the other sources of error are critical to obtaining a clean, useful tonography tracing. 

One of the most common causes of IOP measurement noise during tonography is loss of fixation and eye movement. Subjects receiving tonography are asked to fixate on a target approximately 2 meters away with their contralateral eye. Despite this, eye movement to varying degrees is common during tonography, including unavoidable microsaccades, and movement of the eyes outside of the primary position can cause measurement error, as the tonometer tip may not maintain proper contact with the cornea. If proper contact and applanation of the cornea are maintained with the tonometer tip, IOP measurements appear to be relatively insensitive to the location on the cornea being measured.¹⁸ However, IOP elevations can be associated with different eye positions,¹⁹ which would also result in noise in the tonography curve despite accurately reflecting the IOP. If the fluctuations are infrequent and of short duration, they tend to have limited effect on the EDEN score and the validity of the pressure decay curve. 

Another important source of error is the effect of intrathoracic pressure on IOP. Numerous studies have documented the effect of breathing patterns and Valsalva maneuvers on IOP elevation.^20–22 The likely mechanism involves an increase in intrathoracic venous pressure, which leads to an increase in ocular venous pressure via jugular venous distension and subsequently an increase in choroidal volume.²³ Increased intrathoracic pressure may also increase EVP, but this would not be expected to cause an immediate change in IOP. A change in the backpressure of the conventional outflow system does not result in a sudden change of the intraocular volume because time is required for a buildup of additional aqueous humor in the eye and elevation of IOP. However, increases in EVP could potentially explain curves where the pressure increases instead of decays with time. Although study subjects can be coached to relax and breathe normally, the 2-minute tonography procedure can be difficult to tolerate for some individuals. It is not known if shorter duration tonography curves (e.g., 1 minute) can provide results similar to those for 2-minute pneumatonography. 

Weighted tonography (including pneumatonography) has important assumptions that may also be potential sources for measurement error, including a fixed volume change for a given change in IOP based on standard tables,¹¹ a fixed change in EVP during measurement, and steady state prior to measurement.²⁴ Alteration of parameters used in the calculation of outflow facility, including ocular rigidity coefficient, volume change, and EVP change, can result in alteration of the magnitude of outflow facility from tonography. However, these potential sources of error would not be expected to alter the shape or validity of the tonography curve. 

Other techniques for non-invasive measurement of outflow facility have been reported for living human subjects, but all have inherent assumptions and limitations. Suction cup tonography, where a suction cup is applied instead of a weight, induces a localized stretch in the corneal and limbal tissues which allows aqueous humor to accumulate within the globe.²⁵ After releasing the suction, IOP increases due to the additional fluid volume in the eye. The decay rate of IOP over time is then used to determine outflow facility. However, suction tonography has drawbacks, including long measurement times (15 minutes of vacuum, followed by 15 minutes of IOP decay) compared to weighted tonography (2 to 4 minutes). Additionally, there is a potential risk to glaucoma patients associated with the application of the vacuum, which increases IOP by an average of 28 mm Hg. Comparisons of weighted and suction cup tonography have reported that measurements between the two devices are highly correlated.,²⁶ but weighted tonography has been noted to be more practical.²⁷ 

The aqueous suppression technique is a fundamentally different method for determining outflow facility based on the change in IOP and aqueous humor outflow rate after administration of an aqueous humor–suppressant medication. The technique avoids contact with the eye but can be much more logistically challenging. Fluorescein eye drops are instilled the evening before the measurement to establish a uniform fluorescein depot in the cornea. On the morning of the measurements, baseline fluorophotometry is performed, with a time interval of at least an hour (and ideally 3–4 hours) to ensure reliable detection of fluorescence decay and calculation of aqueous humor flow rate, followed by baseline IOP measurements. Aqueous humor–suppressant medication (typically acetazolamide) is then administered followed by repeat fluorophotometry to determine the decrease in aqueous humor flow rate and IOP measurements. Important assumptions include stable EVP, steady state during the baseline and post-treatment phases, and stable outflow facility throughout the measurement period of hours.²⁴ No data exist to suggest that either tonography or fluorophotometry is superior to the other for measurement of outflow facility. 

One limitation of the approach used in this study is that calculation of the EDEN quality score assumed that discrete IOP data points were available. In this study, the data were captured digitally at 40 Hz using prototype software. It is anticipated that this will be commercially available from the manufacturer but may not be backward compatible with older pneumatonometers. The techniques used in this study can still be applied to printed IOP tracings that have been digitized.¹¹ Care would have to be taken to ensure that the digitized curve closely matches the printed curve. In addition, the maximum data extraction resolution should be selected to most closely match the 40-Hz data used in this study. However, the number of data points that can be extracted from the IOP curves will be limited by the quality of the printout, and it is not clear if the equations developed in this study will be similar. Our study also assumes a linear best fit for the data. In principle, a pressure decay curve would be expected to have an exponential pattern. However, in our experience, this is rarely the case and either a first or second order polynomial usually best represents the data. We have also found through hundreds of tracings that a linear fit is typically the most robust compared to other models, in that it is less likely to produce spurious results (e.g., negative outflow facility) due to data outliers. 

We also found that Expert scores had a moderate positive correlation with outflow facility values. Very noisy tonography tracings make it difficult to detect an IOP decay within the 2-minute period of the tonography measurement. This results in a tonography regression line with a very shallow slope and hence a very low outflow facility calculation. Removal of a few of the tracings with Expert scores below 3 eliminates any significant correlation with outflow facility. 

Finally, although our study provides a method for objectively grading tonography tracings, the cutoff values for what constitutes an acceptable data curve remain subjective. As noted above, there was a strong correlation between the Expert scores and the EDEN scores, which suggests that the use of the techniques described here can reliably provide a grading equivalent to an expert user. Our recommendation is that scores below 4 are almost certainly invalid, whereas scores above 6 can confidently be included. However, for scores between 4 to 6, the curves will still have to be critically evaluated, and any associated observations obtained during testing (e.g., significant eye movement) would have to be considered to determine validity. Future models with larger datasets may benefit from the use of machine learning to help close this gap in objective tonography analysis. 

In summary, we developed and tested an objective method for assessing the quality of tonography tracings based on a regression model to mimic evaluations from experienced users of tonography. This objective assessment can be performed using RMSE of a fitted line to calculate an EDEN quality score that closely predicts an expert user score of tracing quality. Further research is needed to clarify the appropriate cutoff values for acceptance/rejection and to help determine the validity of tracings with moderate scores. 

Acknowledgments

Supported by a grant from the National Eye Institute, National Institutes of Health (EY022124 to SEM), a Research to Prevent Blindness Chair Challenge Grant (SEM), and an OSU Vision Sciences Research Core Program grant (P30EY032857 to SEM). The sponsor or funding organizations had no role in the design or conduct of this research. 

This study was presented in part at the ARVO 2022 Annual Meeting, Denver, CO. 

Disclosure: A.J. Sit, None; C.B. Toris, None; V. Gulati, None; A. Kazemi, None; J. Gilbert, None; S. Fan, None; D.M. Reed, None; D.O. Hodge, None; S.E. Moroi, None