September 2023
Volume 12, Issue 9
Open Access
Glaucoma  |   September 2023
Validation of the Iowa Head-Mounted Open-Source Perimeter
Author Affiliations & Notes
  • Zachary Heinzman
    Department of Ophthalmology and Visual Sciences, Carver College of Medicine, University of Iowa, Iowa City, IA, USA
  • Edward Linton
    Department of Ophthalmology and Visual Sciences, Carver College of Medicine, University of Iowa, Iowa City, IA, USA
  • Iván Marín-Franch
    Computational Optometry, Atarfe, Spain
    Southwest Eye Institute, Tavistock, UK
  • Andrew Turpin
    Curtin School of Population Health, Curtin University, Bentley, Western Australia, Australia
  • Karam Alawa
    Department of Ophthalmology, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA
  • Anushi Wijayagunaratne
    Department of Ophthalmology and Visual Sciences, Carver College of Medicine, University of Iowa, Iowa City, IA, USA
  • Michael Wall
    Department of Ophthalmology and Visual Sciences, Carver College of Medicine, University of Iowa, Iowa City, IA, USA
  • Correspondence: Michael Wall, Department of Ophthalmology and Visual Sciences, Carver College of Medicine, University of Iowa, Iowa City, IA 52242, USA. e-mail: [email protected] 
Translational Vision Science & Technology September 2023, Vol.12, 19. doi:https://doi.org/10.1167/tvst.12.9.19
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Zachary Heinzman, Edward Linton, Iván Marín-Franch, Andrew Turpin, Karam Alawa, Anushi Wijayagunaratne, Michael Wall; Validation of the Iowa Head-Mounted Open-Source Perimeter. Trans. Vis. Sci. Tech. 2023;12(9):19. https://doi.org/10.1167/tvst.12.9.19.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose: To assess the validity of visual field (VF) results from the Iowa Head-Mounted Display (HMD) Open-Source Perimeter and to test the hypothesis that VF defects and test–retest repeatability are similar between the HMD and Octopus 900 perimeters.

Methods: We tested 20 healthy and nine glaucoma patients on the HMD and Octopus 900 perimeters using the Open Perimetry Interface platform with size V stimuli, a custom grid spanning the central 26° of the VF, and a ZEST thresholding algorithm. Historical data from the Humphrey Field Analyzer (HFA) were also analyzed. Repeatability was analyzed with the repeatability coefficient (RC), and VF defect detection was determined through side-by-side comparisons.

Results: The pointwise RCs were 2.6 dB and 3.4 dB for the HMD and Octopus 900 perimeters in ocular healthy subjects, respectively. Likewise, the RCs were 4.2 dB and 3.5 dB, respectively, in glaucomatous patients. Limits of agreement between the HMD and Octopus 900 perimeters were ±4.6 dB (mean difference, 0.4 dB) for healthy patients and ±8.9 dB (mean difference, 0.1 dB) for glaucomatous patients. Retrospective analysis showed that pointwise RCs on the HFA2 perimeter were between 3.4 and 3.7 dB for healthy patients and between 3.9 and 4.7 dB for glaucoma patients. VF defects were similar between the HMD and Octopus 900 for glaucoma subjects.

Conclusions: The Iowa Virtual Reality HMD Open-Source Perimeter is as repeatable as the Octopus 900 perimeter and is a more portable and less expensive alternative than traditional perimeters.

Translational Relevance: This study demonstrates the validity of the visual field results from the Iowa HMD Open-Source Perimeter which may help expand perimetry access.

Introduction
Quantifying the visual field through perimetry is an essential aspect of characterizing, diagnosing, and monitoring a variety of ocular diseases. Currently, the most common way to assess the visual field is through static automated perimetry, where a stimulus of fixed size and varying intensity is presented at predetermined locations throughout the visual field. Perimetry aims to assess the light sensitivity of a portion of the visual field and allows for defects to be identified leading to diagnosis, prognosis, and surveillance. 
Currently, clinical static automated perimetry most commonly uses projection-based perimeters where a light source is projected onto a background of uniform intensity. Although this process can assess the visual field, it is tedious and uncomfortable for patients.1 Costing upwards of $30,000, traditional perimeters are large, bulky, and expensive with poor patient ergonomics. Newer, non-projection–based perimetry options such as the IMOvifa visual field analyzer (CREWT Medical Systems, Tokyo, Japan) and COMPASS (CenterVue, Padova, Italy) attempt to circumvent some of these problems, although cost limitations are still present. 
These limitations are significant and restrict visual field testing to specialized clinics, thus limiting access for patients in rural locations, low-income communities, and many developing countries. These access limitations can result in poor disease outcomes, as many diseases, such as glaucoma, are more readily treated and vision is more likely to be preserved with early diagnosis.2 Portable and affordable screening devices in these communities could help identify individuals who would benefit from a more complete ophthalmic assessment if sufficient infrastructure is available to support the use of data from the devices. 
The use of virtual reality (VR) headsets has exploded in recent years with applications ranging from video games to the exploration of how microgravity aboard the International Space Station affects an astronaut's orientation and perception.3 The possibility of using VR headsets to assess the visual field has also been explored. Various studies have aimed to assess the feasibility and practicality of using VR headsets in place of traditional projection-based perimeters. For example, multiple studies have assessed the performance of the VisuALL device (Olleyes, Summit, NJ) when compared to the Humphrey Field Analyzer (HFA; ZEISS, Jena, Germany).4,5 Other studies, such as those from Tsapakis et al.6 and Stapelfeldt et al.7 compared a VR headset to the HFA-2 perimeter and the Octopus 900 perimeter (Haag-Streit, Bern, Germany), respectively. However, no open-source virtual reality headset systems for visual field testing have been published. With open-source visual field testing systems, the scientific community can develop a set of minimum standards that companies and individuals can expand upon, helping to prevent suboptimal solutions from being implemented and improving access through reduced implementation costs. The repeatability of these VR systems remains largely unexplored, highlighting an important deficit in this arena. 
We have developed a fully open-source, head-mounted display (HMD) perimeter capable of visual field testing that utilizes a Google Daydream VR headset and an Android smartphone (Samsung Electronics Co., Suwon, South Korea). With a cost of around $100, plus the cost of a consumer smartphone and computer, this system is portable and far less costly than traditional perimeters. We compared this new HMD perimeter to the Octopus 900 perimeter in two ways. First, the ability of the system to detect glaucomatous visual field defects was qualitatively compared to the Octopus 900. Second, the repeatability of the HMD was compared to the Octopus 900. Finally, we analyzed past HFA-2 perimeter data and compared its repeatability to that of the HMD. 
Methods
Test Participants
The study protocol was approved by the University of Iowa Institutional Review Board and adhered to the tenets of the Declaration of Helsinki. Written, informed consent was obtained from all participants after explanation of the nature and possible consequences of the study, and compensation was provided. Twenty ocular healthy control subjects and nine subjects with primary open-angle glaucoma (POAG) were offered participation. Ocular healthy subjects were recruited from a campus-wide University of Iowa advertisement. POAG subjects from prior University of Iowa Visual Field Laboratory studies who indicated a willingness to return for future research and met the inclusion and exclusion criteria were offered participation. 
Inclusion criteria for ocular healthy participants were as follows: age, 20 to 79 years; corrected Snellen visual acuity ≥ 20/30; and a normal eye exam within the last 2 years. Inclusion criteria for glaucoma patients were as follows: age, 20 to 79 years; corrected Snellen visual acuity ≥ 20/40, confirmed clinical diagnosis of glaucoma, and a mean deviation of ≥−15 dB. Participants were excluded if they had any coexisting condition affecting vision other than refractive error, had pupil size < 2.5 mm, and had spectacles with a prescription higher than ±8 D sphere or >±3 D cylinder. Participants were also excluded if trial lenses for the Octopus 900 perimeter were higher than ±6 D sphere or ±3 D cylinder. Both ocular healthy and glaucoma patients were included in the study if they met the inclusion and exclusion criteria. All glaucoma participants had prior experience in perimetry, whereas most ocular healthy participants did not. No pupil dilation was performed. The average age for ocular healthy participants was 52.2 ± 13.8 years (range, 25–75) and 71.3 ± 4.6 years (range, 65–79) for glaucoma patients. Supplementary Table S1 highlights POAG subject characteristics. 
Hardware Calibration
The smartphone brightness was programmatically set to 100% to ensure that adaptive brightness settings could not interfere during testing. The brightness of the screen was measured using a luminance meter (TES Electronic Corp., Taipei, Taiwan) at grayscale values ranging from 0 to 255 with three trials, and the averages were fit to a gamma function. The same measurements were then repeated through the lenses of the Daydream headset with the smartphone mounted. A lookup table was generated from this gamma function and used during testing to convert grayscale values to luminance. A comparison of the two showed that a percentage of luminance is lost to the lenses through reflection or scattering that scales with intensity. At a background of 10 cd/m2, approximately 10.8% of light is reflected or scattered and this increases to 25.8% at the maximum luminance the smartphone can display. 
The Daydream headset (in addition to other mobile virtual reality headsets containing a Google Cardboard-compatible QR code) includes parameters that define its field of view. To ensure the accurate location and size of each stimulus, the dimensions of the phone screen in pixels were obtained, and, using the field of view parameter from the headset, the location and size of stimuli could be determined. The distortion induced by the VR headset lenses was corrected by the native Daydream software using parameters from the above-mentioned QR code. 
To determine the necessary dynamic range for testing with size V stimuli, two healthy subjects underwent custom perimetric testing using the method of constant stimuli, and frequency-of-seeing curves were constructed for central and peripheral locations. To increase the minimum contrast at a background of 10 cd/m2, we placed a neutral density 0.9 (ND9) filter (FIEWSZIHU brand, manufactured by Bcolor) between the screen of the smartphone and the VR headset. We then remeasured the gamma function through the assembled headset and the ND filter and used this to generate test stimuli. We raised the background luminance on the phone, resulting in a higher minimum contrast, and the ND filter reverted the perceived background luminance to the standard level of 10 cd/m2. This final setup was used in all of the HMD testing in this study. 
Visual Testing
The HMD consisted of a Samsung Galaxy S9 smartphone (Samsung Electronics Co., Suwon, South Korea) and a second-generation Google Daydream View VR headset. The smartphone ran a custom Android application (open-source code available at https://github.com/imarinfr/opiPhoneHMD). A separate computer ran an R (R Foundation for Statistical Computing, Vienna, Austria) shiny application8 (open-source code available at https://github.com/imarinfr/opiApp) which allowed direct control of both the HMD (through the custom Android application on the smartphone) and the Octopus 900 using the Open Perimetry Interface (OPI)—an open-source system designed to control perimeters with the goal of decreasing barriers to perimetry research and increasing access to visual field testing.8,9 The zippy estimation of thresholds (ZEST) algorithm implemented within the OPI was used to determine individual location sensitivities and was coupled with the Central 26 grid pattern (Fig. 1) included within the OPI software.10,11 ZEST thresholding parameters were identical to a prior study by Marín-Franch et al.12 where they were described in detail. All tests used a Goldmann size V stimulus (1.72° diameter) as opposed to the more conventional size III stimulus (0.43°) due to the greater repeatability and approximately 1 log unit of increased dynamic range observed with size V stimuli.1315 Similar to the Octopus 900, subjects entered responses by pressing a single button remote connected to the smartphone via Bluetooth, creating a completely wireless perimeter. Although trial lenses were used for the Octopus 900, there was no refractive correction for patients on the HMD, and they did not wear spectacles while testing. In summary, testing was kept as identical as possible between the Octopus 900 and HMD with the same control software, thresholding algorithm, grid pattern, and similar patient response input. 
Figure 1.
 
Custom visual field testing grid used from the OPI software. Shading indicates the zones used for analysis as discussed in Methods. Zones 1, 2, and 3 were 2.8° to 10°, 11.3° to 20.4°, and 21.4° to 28.2° from center fixation, respectively.
Figure 1.
 
Custom visual field testing grid used from the OPI software. Shading indicates the zones used for analysis as discussed in Methods. Zones 1, 2, and 3 were 2.8° to 10°, 11.3° to 20.4°, and 21.4° to 28.2° from center fixation, respectively.
The study was comprised of two arms. In the first arm, ocular healthy and POAG subjects were tested twice on both the Octopus 900 and HMD. Test eye and test order were randomized (if both eyes for POAG patients met inclusion and exclusion criteria), and the perimeters were alternated with a minimum 5-minute break between each visual field test. Following testing on both the HMD and Octopus 900 perimeters, all participants were asked which system they preferred. 
The second arm of the study consisted of retrospective data analysis collected from previously published studies. We used visual sensitivities from three previous research studies: VIPI14, VIPII15, and the OPI Octopus 900 control database.12 These data were analyzed as described below in the Statistical Analysis section, and their relation to the results from the HMD were explored. 
Statistical Analysis
All HMD sensitivities were adjusted to the decibel scale of the Octopus 900 perimeter by contrast sensitivity. Conversion equations can be found in the appendix of Sun et al.16 In short, if Lb is the background luminance (10 cd/m2), ∆LO900 is the maximum luminance differential that can be presented on the Octopus 900 (3183.1 cd/m2), and ∆LHMD is the maximum luminance differential that can be presented on the HMD (52 cd/m2), then  
\begin{eqnarray*}- 10\ {\rm{log}}\frac{{\Delta {L_{{\rm{O}}900}}}}{{\Delta {L_{{\rm{HMD}}}}}} = 17.9\ {\rm{dB}}\end{eqnarray*}
is the minimum decibel level that can be presented on the HMD. In other words, the stimulus with maximum luminance on the HMD corresponds to an attenuation in luminance of 17.9 dB in the Octopus 900, and an attenuation of dHMD dB from the maximum luminance of the HMD would correspond to an attenuation of dHMD + 17.9 dB on the Octopus 900. 
As shown in multiple prior studies, sensitivities below 20 dB do not significantly contribute to determining glaucomatous progression and are not clinically significant in a screening device.17,18 To focus analysis on the repeatability of clinically significant results and equalize the floors of the perimeters, all sensitivities below 20 dB were censored by setting these values to 20 dB. Repeatability coefficients (RCs) were calculated for all arms of the study for the HMD, Octopus 900, and HFA-2 perimeters. The RC, also known as the smallest real difference, is related to the 95% limits of agreement (LOAs) on Bland–Altman plots and is defined as the range where you would expect 95% of repeated test results to lie between.19 This is presented in the same unit as the original measurement and was used to assess absolute agreement between repeated tests on each perimeter.20 The RC was utilized, as it has been shown to be more informative than correlation coefficients for test-retest repeatability since correlation can be biased by systematic differences such as potential learning effects between repeated tests.19 
RCs were calculated pointwise (e.g., comparing each location between test 1 and test 2 for each subject) and using mean sensitivities. For arm 1, we also divided test locations into three zones of nearly equal size based on test location distance from the center fixation target, and RCs were likewise calculated for each zone. From the center of fixation, zones 1, 2, and 3 consisted of 24 locations 2.8° to 10° from center, 22 locations 11.3° to 20.4° from center, and 18 locations 21.4° to 28.2° from center, respectively. For retrospective datasets containing more than two tests per participant, only the first two tests were included in the data analysis to improve comparisons between retrospective data and the results from this study. The effect that age has on the RC was also explored through the OPI Octopus 900 control dataset.12. Pointwise RCs and mean sensitivity RCs grouped by decade (e.g., 20s, 30s) were generated for each patient, and linear regression was performed by graphing age versus RC. 
Along with RCs to compare the test–retest repeatability of each perimeter, Bland–Altman plots were created for arm 1 using the R programming library blandr.21 This consisted of graphs comparing pointwise results for test 1 and test 2 for both the HMD and Octopus 900 perimeters. Finally, using the visualFields R package,22 grayscale visual field plots were created for each participant on the Octopus 900 and HMD to aid in the comparison of visual field defect patterns between devices. 
Results
The majority of patients in both the ocular healthy and glaucoma groups preferred testing on the HMD compared to the Octopus 900. Of the 20 ocular healthy patients, 17 (85%) preferred testing on the headset. Similarly, seven out of nine (78%) glaucoma patients preferred the HMD. The most cited reasons for preferring the HMD over the Octopus 900 perimeter were comfort and ease of use, and the most common reason for Octopus 900 preference included headset weight. The pointwise Bland–Altman plots comparing the HMD and Octopus 900 perimeters for arm 1 of the study are shown in Figure 2. Additionally, a pointwise comparison of the HMD to the Octopus 900 to assess LOAs for both ocular healthy and POAG patients is shown in Figure 3
Figure 2.
 
Pointwise Bland–Altman plots for arm 1 of the study for both the Octopus 900 and HMD perimeters. Both devices were controlled through the Open Perimetry Interface using the ZEST thresholding algorithm, Central 26 grid pattern, and Goldmann size V stimuli. Sensitivities less than 20 dB were censored.
Figure 2.
 
Pointwise Bland–Altman plots for arm 1 of the study for both the Octopus 900 and HMD perimeters. Both devices were controlled through the Open Perimetry Interface using the ZEST thresholding algorithm, Central 26 grid pattern, and Goldmann size V stimuli. Sensitivities less than 20 dB were censored.
Figure 3.
 
Pointwise Bland–Altman plots directly comparing the HMD perimeter to the Octopus 900 perimeter. Both devices were controlled through the OPI using the ZEST thresholding algorithm, Central 26 grid pattern, and Goldmann size V stimuli. Sensitivities less than 20 dB were censored.
Figure 3.
 
Pointwise Bland–Altman plots directly comparing the HMD perimeter to the Octopus 900 perimeter. Both devices were controlled through the OPI using the ZEST thresholding algorithm, Central 26 grid pattern, and Goldmann size V stimuli. Sensitivities less than 20 dB were censored.
When the Octopus 900 was directly compared to the HMD, pointwise LOAs for ocular healthy and POAG patients were 4.6 dB and 8.9 dB, respectively. Mean sensitivity LOAs for ocular healthy and POAG patients were 1.6 dB and 1.8 dB, respectively. Table 1 shows the pointwise and mean sensitivity RCs for both the HMD and Octopus 900 perimeters. Also displayed in this table are the RCs for the concentric zone analysis where the Central 26 grid was divided into three zones based on distance from center fixation. Similarly, pointwise and mean sensitivity RCs for the retrospective data analysis in arm 2 of the study using past studies VIPI14 and VIPII15 are shown below in Table 2
Table 1.
 
Head-Mounted Display and Octopus 900 Repeatability
Table 1.
 
Head-Mounted Display and Octopus 900 Repeatability
Table 2.
 
Historical Repeatability of Humphrey Field Analyzer 2
Table 2.
 
Historical Repeatability of Humphrey Field Analyzer 2
Frequency-of-seeing curves illustrated clearly that there was initially going to be a ceiling effect when measuring central locations with a size V stimulus (Fig. 4). The threshold of the most sensitive location was ∼38 dB, and the lowest contrast stimulus that could be generated at a background of 10 cd/m2 was about 37 dB stimulus. By increasing the background to 50 cd/m2 we were able to generate a lower contrast stimulus (42 dB); however, this is quite an uncomfortable level of luminance for most subjects. We reduced the amount of light reaching the eye with an ND filter such that the background was still seen at 10 cd/m2 and the contrast between the stimulus and the background was unchanged. This resulted in a new dynamic range from 17.8 to 42.0 dB (Fig. 4). 
Figure 4.
 
HMD frequency-of-seeing curves for central, mid-peripheral, and peripheral locations using Goldmann size V stimuli. Stimulus contrast levels are presented as HFA-equivalent decibels. A ceiling effect is seen for central locations, subsequently lifted with use of a neutral density filter.
Figure 4.
 
HMD frequency-of-seeing curves for central, mid-peripheral, and peripheral locations using Goldmann size V stimuli. Stimulus contrast levels are presented as HFA-equivalent decibels. A ceiling effect is seen for central locations, subsequently lifted with use of a neutral density filter.
Finally, to evaluate the impact that age has on the RC when using the ZEST thresholding algorithm and Central 26 grid pattern, the OPI dataset12 was used. First, age was graphed against the pointwise RC for each patient. The slope of these data was 0.009 dB/y with an R2 of 0.031. Second, patients were grouped by decade (e.g., 20s, 30s) and a mean sensitivity RC was determined for each decade of patients. The slope of mean age for each decade of patients graphed against RC was 0.011 dB/y with an R2 of 0.68. Supplementary Figures S1 and S2 show side-by-side comparisons between the HMD and Octopus 900 for all control and glaucoma subjects included in arm 1, respectively. 
Discussion
Validation of a new perimetric test includes ensuring that the dynamic range is adequate, the repeat variability is low, agreement with comparable standards is good, and the sensitivity and specificity for the detection of disease are useful. Recent years have seen the development of many new head-mounted perimetry devices, as well as the addition of perimetry to HMDs designed for other uses. Validation studies are sparse for many of these devices being developed or currently in use, and most focus on agreement of summary statistics or sensitivity and specificity for the detection of glaucoma without evaluating other aspects of the perimeter's function. In this study, we began this process by measuring the dynamic range of the HMD and found that it required an adjustment to avoid a ceiling effect in normal subjects with a size V stimulus. We then evaluated the repeat variability of the HMD using a ZEST threshold test and compared it to the Octopus 900 in the same subjects and to historical data from the HFA. 
Retest variability with size III stimuli is increased by fixation drift, which can be improved by using larger stimuli, as the area of the retina sampled changes less between tests with larger stimuli.23 Also, larger stimuli are robust against refractive error, have better repeatability, and have a larger useful dynamic range.24 A potential problem with the large stimulus approach highlighted by our work is that large stimuli are detected at lower contrast, so ceiling effects may be encountered depending on the display hardware used. 
The dynamic range of stimulus contrasts on the Octopus 900 spans 5 log units from 0 dB (3193 cd/m2 stimulus against a 10 cd/m2 background) to 50 dB (10.032 cd/m2 stimulus against a 10 cd/m2 background), but not all of it is typically used, and less of it is useful. Retinal locations with sensitivities below 20 dB may still be seen, but in this range, test–retest variability increases so much that it becomes difficult to distinguish further reductions of retinal sensitivity from noise.14,15 Therefore, floor effects in a perimeter may be tolerable until the floor rises above this level. 
Ceiling effects are not as well tolerated by threshold tests, as detecting glaucoma requires measurement of deviations from normal when defects develop or enlarge. Normal ranges of retinal sensitivity have been established, with age-adjusted central and paracentral thresholds expected to be in the range of 30 to 35 dB for size III and 34 to 39 dB for size V.25 Staircase-based algorithms make use of stimuli that are dimmer even than this as part of the process of threshold estimation. Because the HMD uses a commercial smartphone with an 8-bit display and does not employ any kind of dithering, the minimum contrast is constrained to a one-unit increase in RGB pixel density on the 0 to 255 scale. At baseline, the background luminance of 10 cd/m2 is generated with a pixel density of 53, so the dimmest stimulus possible is generated with a pixel density of 54, which gives an HFA-equivalent contrast of 37 dB. 
Frequency-of-seeing curves (Fig. 4) measured in two healthy observers showed us clearly that the minimum contrast was suprathreshold for central locations using a size V stimulus, indicating that a ceiling effect would be present. There are several options for lowering the minimum contrast on a digital display, including dithering stimuli, using 10-bit or 12-bit displays, and increasing the background luminance. Dithering is not currently supported by the OPI, and very few commercial smartphones are made with 10-bit displays. We increased the background luminance to ∼50 cd/m2, which improved the minimum contrast to 42 dB, and we added an ND filter that reduced the brightness to a standard background of 10 cd/m2 without changing the contrast. This did sacrifice some dynamic range by lowering the maximum brightness, but the floor of the HMD was 17.8 dB so we felt this was an acceptable tradeoff. To compare the HMD to the Octopus 900, we censored all sensitivities below 20 dB by replacing these sensitivities with a value of 20 dB, reducing variability in the Octopus 900 data as expected. 
We found that the retest variability of the HMD was low and comparable to that of the Octopus 900, which in turn was comparable to published data from the HFA using size V stimuli. Mean differences were very small (–0.2 to 0.2 dB) on the Octopus 900 and the HMD, indicating minimal learning effects. Repeat testing gave a threshold within approximately 3 dB of the initial measurement at a given location 95% of the time for ocular healthy patients. Average sensitivities on retest were within 1 dB of the original. This was comparable to the Octopus 900 and to published data with the HFA using size V, indicating that the test is as repeatable in normal subjects as the gold standard. When compared to a similar study comparing the COMPASS fundus perimeter to the HFA (COMPASS mean sensitivity RCs of 1.5 dB and 2.7 dB for ocular healthy and glaucoma patients, respectively), the HMD demonstrated good repeatability.26 In the same study, mean differences for control (1.85 dB) and glaucoma patients (1.46 dB) were higher than the 0.4 dB and 0.1 dB for control and glaucoma patient with the Iowa Perimeter, respectively.26 A post hoc power analysis using a one-tailed matched Wilcoxon signed-rank test that the differences in mean sensitivity were greater than those in this similar study26 yielded a power of 1 for the control group and 0.99 for the glaucoma group. Repeat testing in glaucoma subjects with the HMD gave pointwise repeatability coefficients of 4.2 dB overall, similar to what we found with the Octopus 900 (3.5 dB) and to prior data from the HFA (4.7 dB). Retest variability increased with eccentricity somewhat more in the HMD than the Octopus 900. This may be due to a combination of factors, including headset positioning or distortion of the optics, and is a finding worthy of further investigation. Perimetric artifacts inherent to a headset perimeter have yet to be detailed. 
In addition to dynamic range and repeat variability, we evaluated agreement between the HMD and the Octopus 900. Similar studies comparing standard automated perimetry on novel devices to HFA often limit the assessment of agreement to that of summary statistics like mean deviation or sector averages. Examples published recently have shown 95% LOAs in the range of ±4 to 5 dB for mean sensitivity.27,28 Our results using this approach showed that the 95% limits of agreement for mean sensitivities were ±1.6 dB and ±1.8 dB for normal and glaucoma subjects, respectively, with minimal bias. Given the repeatability of the devices, the minimum expected limits of agreement would be around ±1 dB, so a measured agreement of ±1.6 dB and ±1.8 dB is not perfectly interchangeable but is felt to be quite good. 
Agreement between devices is incompletely described by summary statistics, so we also included an analysis of agreement at each retinal location. As with repeatability, agreement was expected to be lower for the pointwise analysis, and we found that the pointwise LOAs in normal subjects and glaucoma patients were ±4.6 dB and ±8.9 dB, respectively, again with minimal bias. This value is not consistently reported in similar studies, so we do not have a clear basis for direct comparison with other devices. We did qualitatively compare the visual field defects measured by each device, finding that the majority of defects were reproduced well, although some small shifts and a few larger differences were seen (e.g., GVR07 OS; Supplementary Fig. S2). Disagreement does not appear systematic and may represent fixation drift, poor headset positioning, or other variable sources of artifact. 
These findings help validate test results on this HMD system as well as similar systems using the OPI open-source software.12 It has previously been demonstrated that visual field tests using Goldmann size V stimuli are more repeatable than those using size III.13 Results shown in Table 1 indicate that this is also true when using the RC as a measure of test–retest absolute agreement for both glaucoma patients and ocular healthy control subjects, which helps provide additional evidence for the use of the RC when analyzing visual field data. Although an older age may worsen repeatability, such as the increased average age in the glaucoma group, it does not affect the interpretation of these results. This study is focused on repeatability when comparing patients to themselves, not comparing between cohorts of patients. Both groups had strong repeatability when using this approach. Interestingly, this study found no significant correlation between RCs and patient age. 
Additionally, for a screening device to be successful, patients must be willing to use them. Our subjects had a strong preference for testing on the HMD compared to the Octopus 900. This is consistent with other studies showing that better ergonomics contribute to higher patient satisfaction with headset-based tests.29 However, patients were not asked about their comfort in administering the test themselves in non-clinical settings. Future work into optimizing the software for patient-administered testing will help to expand upon the accessibility and portability of the device. Consisting of a smartphone, headset, and secondary computer such as a laptop, this system is far easier to travel with than current projection-based perimeters. The device can easily be taken on an airplane and transported to other countries for underserved patients, transported throughout the hospital, or even taken into patient homes, unlike current perimeters in use today. 
Our study is limited in scope to assessment of the dynamic range, retest variability of the HMD, and the pointwise threshold comparison to the Octopus 900. We did not evaluate sensitivity or specificity for the detection of visual field defects or their progression. This, along with the collection of a larger set of normative data with the HMD, will be the subject of a future study and are necessary steps in assessing the clinical value of the device. The floor effect in this device prevents distinguishing locations with poor sensitivity from those that are perimetrically blind, which may be clinically important information with relevance in glaucoma progression. A further limitation of the device tested is that there is no eye tracking, so it is impossible to determine whether the increased retest variability in the periphery in glaucoma subjects was associated with fixation drift or headset positioning. 
Conclusions
With adequate calibration, high-quality perimetry can be performed with inexpensive hardware using the Open Perimetry Interface. We demonstrated a practical solution to lower the minimum contrast of a smartphone display and showed that perimetry with the HMD is highly repeatable with excellent pointwise agreement with the Octopus 900. At a cost of around 3% of current automated static perimeters, our open-source perimetry platform has the potential to decrease barriers to visual field testing and protect sight by increasing access to early detection of degenerative eye diseases. 
Acknowledgments
The authors thank Trina Eden for her generous contributions to data collection. 
Supported by a VA Merit Review Grant (I01 RX-001821-01A1) and by an unrestricted grant to the University of Iowa Department of Ophthalmology from Research to Prevent Blindness (New York, NY). Software and manuscript development was also partially supported by Computational Optometry (Atarfe, Spain; www.optocom.es). 
Disclosure: Z. Heinzman, None; E. Linton, None; I. Marín-Franch, None; A. Turpin, Heidelberg Engineering (F), iCare (C); K. Alawa, None; A. Wijayagunaratne, None; M. Wall, None 
References
Crabb DP, Russell RA, Malik R, et al. Frequency of Visual Field Testing When Monitoring Patients Newly Diagnosed With Glaucoma: mixed methods and modelling. Southampton, UK: NIHR Journals Library; 2014. [PubMed]
Buys YM, Jin YP. Socioeconomic status as a risk factor for late presentation of glaucoma in Canada. Can J Ophthalmol. 2013; 48: 83–87. [CrossRef] [PubMed]
Harris L. The Effect of Long Duration Hypogravity on the Perception of Self-Motion. Washington, DC: NASA; 2020.
Montelongo M, Gonzalez A, Morgenstern F, Donahue SP, Groth SL. A virtual reality-based automated perimeter, device, and pilot study. Transl Vis Sci Technol. 2021; 10: 20. [CrossRef] [PubMed]
Razeghinejad R, Gonzalez-Garcia A, Myers JS, Katz LJ. Preliminary report on a novel virtual reality perimeter compared with standard automated perimetry. J Glaucoma. 2021; 30: 17–23. [CrossRef] [PubMed]
Tsapakis S, Papaconstantinou D, Diagourtas A, et al. Visual field examination method using virtual reality glasses compared with the Humphrey perimeter. Clin Ophthalmol. 2017; 11: 1431–1443. [CrossRef] [PubMed]
Stapelfeldt J, Kucur SS, Huber N, Höhn R, Sznitman R. Virtual reality-based and conventional visual field examination comparison in healthy and glaucoma patients. Transl Vis Sci Technol. 2021; 10: 10. [CrossRef] [PubMed]
Marín-Franch I, Turpin A, Artes PH, et al. The Open Perimetry Initiative: a framework for cross-platform development for the new generation of portable perimeters. J Vis. 2022; 22: 1. [CrossRef] [PubMed]
Turpin A, Artes PH, McKendrick AM. The Open Perimetry Interface: an enabling tool for clinical visual psychophysics. J Vis. 2012; 12: 22. [CrossRef] [PubMed]
Wall M, Lee EJ, Wanzek RJ, Chong LX, Turpin A. Temporal wedge defects in glaucoma: structure/function correlation with threshold automated perimetry of the full visual field. J Glaucoma. 2020; 29: 191–197. [CrossRef] [PubMed]
Wall M, Subramani A, Chong LX, et al. Threshold static automated perimetry of the full visual field in idiopathic intracranial hypertension. Invest Ophthalmol Vis Sci. 2019; 60: 1898–1905. [CrossRef] [PubMed]
Marín-Franch I, Artes PH, Chong LX, Turpin A, Wall M. Data obtained with an open-source static automated perimetry test of the full visual field in healthy adults. Data Brief. 2018; 21: 75–82. [CrossRef] [PubMed]
Wall M, Woodward KR, Doyle CK, Artes PH. Repeatability of automated perimetry: a comparison between standard automated perimetry with stimulus size III and V, matrix, and motion perimetry. Invest Ophthalmol Vis Sci. 2009; 50: 974–979. [CrossRef] [PubMed]
Wall M, Woodward KR, Doyle CK, Zamba G. The effective dynamic ranges of standard automated perimetry sizes III and V and motion and matrix perimetry. Arch Ophthalmol. 2010; 128: 570–576. [CrossRef] [PubMed]
Wall M, Doyle CK, Eden T, Zamba KD, Johnson CA. Size threshold perimetry performs as well as conventional automated perimetry with stimulus sizes III, V, and VI for glaucomatous loss. Invest Ophthalmol Vis Sci. 2013; 54: 3975–3983. [CrossRef] [PubMed]
Sun H, Dul MW, Swanson WH. Linearity can account for the similarity among conventional, frequency-doubling, and gabor-based perimetric tests in the glaucomatous macula. Optom Vis Sci. 2006; 83: 455–465. [CrossRef] [PubMed]
Wall M, Zamba GKD, Artes PH. The effective dynamic ranges for glaucomatous visual field progression with standard automated perimetry and stimulus sizes III and V. Invest Ophthalmol Vis Sci. 2018; 59: 439–445. [CrossRef] [PubMed]
Gardiner SK, Swanson WH, Demirel S. The effect of limiting the range of perimetric sensitivities on pointwise assessment of visual field progression in glaucoma. Invest Ophthalmol Vis Sci. 2016; 57: 288–294. [CrossRef] [PubMed]
Vaz S, Falkmer T, Passmore AE, Parsons R, Andreou P. The case for using the repeatability coefficient when calculating test-retest reliability. PLoS One. 2013; 8: e73990. [CrossRef] [PubMed]
Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986; 1: 307–310. [PubMed]
Deepankar D . blandr: a Bland-Altman method comparison package for R. Available at: https://github.com/deepankardatta/blandr. Accessed September 18, 2023.
Marín-Franch I, Swanson WH. The visualFields package: a tool for analysis and visualization of visual fields. J Vis. 2013; 13: 10. [CrossRef] [PubMed]
Maddess T. The influence of sampling errors on test-retest variability in perimetry. Invest Ophthalmol Vis Sci. 2011; 52: 1014–1022. [CrossRef] [PubMed]
Anderson RS, McDowell DR, Ennis FA. Effect of localized defocus on detection thresholds for different sized targets in the fovea and periphery. Acta Ophthalmol Scand. 2001; 79: 60–63. [CrossRef] [PubMed]
Wall M, Johnson CA. Morphology and repeatability of automated perimetry using stimulus sizes III, V and VI. Med Res Arch. 2020; 8.
Montesano G, Bryan SR, Crabb DP, et al. A comparison between the compass fundus perimeter and the Humphrey Field Analyzer. Ophthalmology. 2019; 126: 242–251. [CrossRef] [PubMed]
Narang P, Agarwal A, Srinivasan M, Agarwal A. Advanced vision analyzer-virtual reality perimeter: device validation, functional correlation and comparison with Humphrey Field Analyzer. Ophthalmol Sci. 2021; 1: 100035. [CrossRef] [PubMed]
Ahmed Y, Pereira A, Bowden S, et al. Multicenter comparison of the Toronto Portable Perimeter with the Humphrey Field Analyzer: a pilot study. Ophthalmol Glaucoma. 2022; 5: 146–159. [CrossRef] [PubMed]
Groth SL, Linton EF, Brown EN, Makadia F, Donahue SP. Evaluation of virtual reality perimetry and standard automated perimetry in normal children. Transl Vis Sci Technol. 2023; 12: 6. [CrossRef] [PubMed]
Figure 1.
 
Custom visual field testing grid used from the OPI software. Shading indicates the zones used for analysis as discussed in Methods. Zones 1, 2, and 3 were 2.8° to 10°, 11.3° to 20.4°, and 21.4° to 28.2° from center fixation, respectively.
Figure 1.
 
Custom visual field testing grid used from the OPI software. Shading indicates the zones used for analysis as discussed in Methods. Zones 1, 2, and 3 were 2.8° to 10°, 11.3° to 20.4°, and 21.4° to 28.2° from center fixation, respectively.
Figure 2.
 
Pointwise Bland–Altman plots for arm 1 of the study for both the Octopus 900 and HMD perimeters. Both devices were controlled through the Open Perimetry Interface using the ZEST thresholding algorithm, Central 26 grid pattern, and Goldmann size V stimuli. Sensitivities less than 20 dB were censored.
Figure 2.
 
Pointwise Bland–Altman plots for arm 1 of the study for both the Octopus 900 and HMD perimeters. Both devices were controlled through the Open Perimetry Interface using the ZEST thresholding algorithm, Central 26 grid pattern, and Goldmann size V stimuli. Sensitivities less than 20 dB were censored.
Figure 3.
 
Pointwise Bland–Altman plots directly comparing the HMD perimeter to the Octopus 900 perimeter. Both devices were controlled through the OPI using the ZEST thresholding algorithm, Central 26 grid pattern, and Goldmann size V stimuli. Sensitivities less than 20 dB were censored.
Figure 3.
 
Pointwise Bland–Altman plots directly comparing the HMD perimeter to the Octopus 900 perimeter. Both devices were controlled through the OPI using the ZEST thresholding algorithm, Central 26 grid pattern, and Goldmann size V stimuli. Sensitivities less than 20 dB were censored.
Figure 4.
 
HMD frequency-of-seeing curves for central, mid-peripheral, and peripheral locations using Goldmann size V stimuli. Stimulus contrast levels are presented as HFA-equivalent decibels. A ceiling effect is seen for central locations, subsequently lifted with use of a neutral density filter.
Figure 4.
 
HMD frequency-of-seeing curves for central, mid-peripheral, and peripheral locations using Goldmann size V stimuli. Stimulus contrast levels are presented as HFA-equivalent decibels. A ceiling effect is seen for central locations, subsequently lifted with use of a neutral density filter.
Table 1.
 
Head-Mounted Display and Octopus 900 Repeatability
Table 1.
 
Head-Mounted Display and Octopus 900 Repeatability
Table 2.
 
Historical Repeatability of Humphrey Field Analyzer 2
Table 2.
 
Historical Repeatability of Humphrey Field Analyzer 2
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×