Ability of Head-Mounted Display Technology to Improve Mobility in People With Low Vision: A Systematic Review

Abstract

Purpose: The purpose of this study was to undertake a systematic literature review on how vision enhancements, implemented using head-mounted displays (HMDs), can improve mobility, orientation, and associated aspects of visual function in people with low vision.

Methods: The databases Medline, Chinl, Scopus, and Web of Science were searched for potentially relevant studies. Publications from all years until November 2018 were identified based on predefined inclusion and exclusion criteria. The data were tabulated and synthesized to produce a systematic review.

Results: The search identified 28 relevant papers describing the performance of vision enhancement techniques on mobility and associated visual tasks. Simplifying visual scenes improved obstacle detection and object recognition but decreased walking speed. Minification techniques increased the size of the visual field by 3 to 5 times and improved visual search performance. However, the impact of minification on mobility has not been studied extensively. Clinical trials with commercially available devices recorded poor results relative to conventional aids.

Conclusions: The effects of current vision enhancements using HMDs are mixed. They appear to reduce mobility efficiency but improved obstacle detection and object recognition. The review highlights the lack of controlled studies with robust study designs. To support the evidence base, well-designed trials with larger sample sizes that represent different types of impairments and real-life scenarios are required. Future work should focus on identifying the needs of people with different types of vision impairment and providing targeted enhancements.

Translational Relevance: This literature review examines the evidence regarding the ability of HMD technology to improve mobility in people with sight loss.

Introduction

According to the World Health Organization, more than a billion people have vision impairment or blindness that cannot be treated.¹ Globally, the leading causes of low vision are uncorrected refractive error and cataract.¹ In the developed world, the leading causes are macular degeneration, glaucoma, cataract, diabetic eye disease, and retinitis pigmentosa (RP).^2,3 These eye conditions can result in central vision loss (CVL), peripheral field loss (PFL), or a combination of both with approximately 74%, 13%, and 11% of people with low vision suffering from each type of loss, respectively.³ One of the problems faced by people with low vision (LV) that has an impact on their overall quality of life and limits their participation in day-to-day activities is reduced mobility.^4–6 Mobility is the key dimension of generic health-related quality-of-life,⁷ visual disability⁸ and vision-related quality-of-life.^9,10 

Mobility is defined as the act or ability to move from one's current location to one's desired location in another part of the environment safely, gracefully, efficiently, and comfortably.¹¹ Having good orientation skills, defined as the ability to use one's residual vision and/or other senses to understand the local environment at any given time,¹² is critical to achieving good mobility performance. 

Mobility performance in people with LV is influenced by a range of visual factors including the visual field (VF),^13–15 contrast sensitivity (CS),¹³ visual acuity (VA),^13,16 and visual scanning ability.^17,18 It has been proposed that people with LV have critically impaired mobility when their VF is smaller than 15 degrees diameter, and they are at risk of having inadequate mobility when the VF is reduced to 31 degrees to 52 degrees.¹⁹ It is widely accepted that the size of the VF together with CS are the most important predictors of mobility performance.²⁰ Although there is no universal agreement on the impact of acuity on mobility performance, some studies found an association between reduced acuity and mobility impairment. Additionally, acuity and CS are necessary for detecting obstacles from a distance.²¹ 

The mobility problems experienced by people with LV depends on the nature of the underlying vision loss. For those with central vision loss, mobility problems may stem from difficulty reading signs, coping with public transport, and finding the desired destination in unfamiliar places.²² In contrast, people with PFL tend to have problems navigating stairs, detecting and avoiding people and obstacles, and avoiding trip hazards.^15,16 Regardless of the type of VF loss, mobility problems become more pronounced when the level of illumination is reduced.^17,23 Many visual conditions also result in reduced depth perception,²⁴ which is also important for mobility.²⁵ 

A fundamental concept in low vision rehabilitation is the notion of making things bigger (e.g. larger signs), bolder (e.g. use of contrasting colors to highlight obstacles), and/or brighter. However, making these modifications in the built environment is not practical in most instances. Sight substitution approaches, such as the use of long canes or guide dogs, are popular approaches in improving mobility. Guide dogs are effective mobility aids,²⁶ but their high cost and associated maintenance issues are a limitation. Canes are inexpensive and effective aids that help people survey their immediate environment (e.g. obstacles and changes in surface heights). However, their usefulness is limited by the length of the cane and to low-lying obstacles.²⁷ Telescopes can be used for orientation tasks, such as reading signs or identifying features of the environment. However, their effectiveness is limited by a restricted field of view (FOV) and the need for good physical coordination and dexterity.²⁸ 

In recent years, rapid technological advances have led to an increase in the number of electronic mobility aids. These devices use various sensors (depth and ultrasonic sensors, and cameras) to capture the environment. Using computer vision and signal processing techniques, the visual information is translated into alternative modalities, such as auditory and vibrotactile. Audio electronic travel aids provide feedback (warnings and guidance) with either verbal descriptions²⁹ or sonification.³⁰ The vibrotactile aids provide feedback through small vibrators embedded in various places, such as in the handle of an augmented cane,³¹ soles inside shoes,³² wrist bracelet,³³ and belt,³⁴ and so on. 

Although vision substitution techniques would be essential for people who are completely blind, the majority of people with LV have useful residual vision³⁵ and prefer to use it to observe the environment.³⁶ As smartphones become more ubiquitous and powerful, the number of mobile applications designed for people with vision loss is increasing.³⁷ Utilizing complex computer vision algorithms, mobile applications can apply filters to modify the brightness range or increase the contrast of edges in images or videos.³⁷ Mobile devices can also use deep learning techniques to understand the environment to detect and recognize text or people³⁸ and obstacles.^39,40 Similar computer vision techniques can be applied to head-mounted display (HMD) systems with the added benefit of them being hands-free. More than 25 years ago, Massof et al.⁴¹ developed the first head-mounted low vision aid, the Low Vision Enhancement System, which provided improved VA and CS. Since then, advances in sensors, cameras, displays, and computational hardware have led to the availability of various consumer HMD systems, such as Google Glass, Microsoft HoloLens, and Oculus Rift. These powerful devices facilitate the development and implementation of complex computer vision algorithms that may improve mobility and orientation in people with LV. 

To make the best use of these emerging technologies as LV mobility aids, it is useful to understand the current state of HMD technology, and what types of vision enhancements and information processing have already been identified as being helpful in improving mobility performance. Harper et al.⁴² reviewed the earliest HMD devices and suggested potential future developments in image enhancements. The narrative review paper by Deemers et al.⁴³ provides an excellent overview of currently available HMD devices and flags strategies that can be used to help people with LV in general. In addition, Ehrlich et al.⁴⁴ have recently provided an expert perspective on different types of HMD technologies, identifying important optical and human factor considerations. This paper complements this work by providing a systematic review of the current state of research on how image enhancements and processing techniques implemented on HMDs affect orientation and mobility in people with LV. 

Methods

Search Method

A systematic review of the literature was undertaken. Potentially relevant articles published any time before November 2, 2018, were identified via a keyword search of the following databases: MEDLINE, CHINL, Scopus, and Web of Science. A wide range of keywords, developed in conjunction with a subject librarian, was used to capture the potentially relevant literature (see the Table). Articles were screened in three stages: (1) based on the title, (2) based on the title and abstract, and (3) based on the full text of the article. 

Table.

 

Keywords – Items Within Each Column were Joined by the ‘OR’ Operator and Between Columns were Joined by the ‘AND’ Operator.

 

Study Selection

Articles reporting vision enhancement techniques, implemented using HMDs aimed at improving mobility or associated visual functions were included. Articles describing physical implants, such as retinal prosthesis, were excluded. Studies of image enhancements implemented on displays other than HMDs, mobility aids with alternative modalities other than visual, and vision enhancements for near-vision tasks were also excluded as were studies that did not involve participants. 

Data Collection

Information about the HMD devices used, type of enhancements/intervention, participant characteristics, study design, and outcomes were extracted. 

Results

The initial literature search identified 2474 potentially relevant articles, but this included 856 duplicates. After removing duplicates, the remaining 1618 articles were screened according to the methods described previously, leaving just 28 articles in the review (see Fig. 1). 

Figure 1.

 

Flowchart of the literature search process

The results presented in the following paragraphs are grouped based on the type of enhancement used in the studies. Six studies evaluated scene simplification, seven studies explored the effect of scene minification, and five studies assessed the usefulness of off-the-shelf devices. One study explored scene minification and digital zooming. A further nine studies explored various enhancements, such as digital zooming, edge enhancement, and the use of visual cues to attract attention. 

The studies in this review used different types of augmented reality (AR) technology to provide visual enhancements. Some of these enhancements were aimed at improving mobility directly, whereas the rest improved aspects of visual function that are important for orientation, such as visual search and reading signs. 

Most investigations used an observational, cross-sectional, within-subject study design. Others used a randomized placebo-controlled study design,⁴⁵ case-controlled study design,^46–48 a longitudinal within-subject design,^49,50 and a within-group design.⁵¹ To evaluate visual factors related to orientation and mobility, a range of clinical vision tests, recognition tests, and visual search tests were used. Clinical vision tests included VA, CS, and VF. Recognition tests involved participants locating and recognizing specific items (e.g. objects, poses, and obstacles). Visual search tests involved locating letters and symbols on different backgrounds in both stationary and mobile scenarios as well as following visual cues to improve scanning and search performance. To quantify mobility performance, the studies used indoor mobility courses with varying levels of difficulty. One study used a real-world outdoor environment at night.⁵² Time taken to traverse mobility courses and the number of mobility errors were predominantly used as outcome measures. Mobility errors included unintended contact with obstacles and walls, deviation from the intended path, and problems with orientation (e.g. being unable to find the path after colliding with an obstacle). Only three studies^46,53,54 used percentage of preferred walking speed (PPWS), a useful mobility efficiency outcome measure⁵⁵ that can be used as a between-participant walking efficiency measure, in addition to assessing mobility changes in a single participant.⁵⁶ Two studies used raw walking speed as the outcome measure.^47,48 Van Rheede et al.⁴⁸ used additional outcome measures: deviation distance (distance from which participants deviated from the collision course) and hesitation scores (changes in walking speed), calculated from the video data of mobility trials. One study used an orientation and mobility expert to grade the mobility performances.⁵² 

Technology

This section introduces the AR technologies used in the studies. 

AR superimposes the computer-generated virtual world in real-world environments. AR, also known as mixed reality, enhances the natural environment of a person in the real world by adding virtual elements or holograms that are dynamic and interactive. It incorporates mainly visual feedback and uses spatial or 3D sound to provide an immersive real-world experience. 

The hardware components of AR systems include a display, processor, sensors, and input devices. These systems primarily use small displays positioned close to the eyes and attached to a headset that rests on the forehead. These display systems are known as HMDs. Displays may use both occluded and see-through displays. The processor processes the state of the real-world environment and generates virtual objects and/or environments. The processor can be built-in (allows better mobility) or tethered (conventional computers attached to the display via a cable). Various sensors (e.g. cameras, infrared depth sensors, accelerometer, and global positioning system [GPS]), and input devices (e.g. keyboard, mouse, game controllers, and eye trackers) allow the AR systems to read the state of the environment and facilitate user interaction. The type of sensors used to acquire information about the environment largely determines the usability of the HMDs. The studies included in this paper used different types of cameras (CMOS and CCD) and depth sensors (based on stereo vision or infra-red). The stereo vision-based camera's usability is limited by environmental factors, such as bright intensity lights and nontextured surfaces. In contrast, infra-red sensors have a limited range and do not perform well with transparent or translucent objects. The sensitivity of CMOS and CCD sensors determines the HMD's usability under low light conditions. 

AR is used as the primary platform for visual enhancements to improve mobility in people with LV due to its ability to combine the real-world and holograms that make things easier for users to see. The two different types of HMD technologies used in AR systems are video see-through displays (VSTs) and optical see-through displays (OSTs; see Fig. 2). Optical systems for these HMD display systems can be classified based on the image formation (i.e. pupil forming and non-pupil forming designs). Pupil-forming displays, such as that used in the HoloLens device,⁵⁷ are relatively complex and involve an array of lenses and an intermediary image.⁵⁸ In non-pupil forming designs, such as the HTC Vive, there is no intermediate image and a single convex lens system is used to ensure that images formed on the display are in focus.⁴⁴ Pupil forming displays allow flexibility in the location of the image source, which enables better ergonomics. Non-pupil forming devices are relatively easier to design and fabricate with the disadvantages of adding extra weight over the eyes.⁴⁴ 

Figure 2.

 

Example of VST and OST devices. (a) HTC VIVE with occluded display. (b) Microsoft HoloLens with transparent display.

Video See-Through Displays

VSTs^48,59 are the occluded display systems that are primarily associated with virtual reality environments (e.g. Oculus Quest, Oculus Rift, and HTC Vive). When used in AR systems, VSTs block the natural view of the environment and allow users to see the real-time environment via a video feed captured by camera(s). The video feed that is displayed to the users could be modified in real-time to provide vision enhancements. The main advantage of these systems is the wide range of image manipulations possible, such as color inversion to enhance contrast and object deletion. Compared with OSTs, these displays also have larger screen sizes. One of the main disadvantages of this type of display is the inability to use peripheral vision, and in case of system failure, the occluded display will leave users completely unable to see until they are taken off. Another drawback is the size and bulkiness of the devices. They tend to protrude in front of the face and are not comfortable for long time usage. They also need to be tethered to a more powerful external hardware (e.g. laptop) to perform computationally expensive tasks, such as running computer vision algorithms. In such a situation for VSTs, being tethered is a major limitation for mobility. 

Optical See-Through Displays

OSTs^45,60 are the most commonly used type of displays in AR environments as they allow users to see the real-world enhanced with virtual objects or holograms (e.g. Magic Leap, Microsoft HoloLens, Google Glass, and Epson Moverio BT-300). The main advantage of OSTs is unimpeded peripheral vision, and in case of power failure or system failure, they do not impede the user's view. The main disadvantage is the smaller screen size compared with VSTs. In bright daylight, the visibility of holograms or virtual objects is also relatively poor due to the limited dynamic range of the display. However, visors or liquid crystal filters can be used to dim the environment to a certain extent.⁶¹ Precise image registration between virtual and real-world objects is essential to maintain the illusion that they coexist in the same environment.⁶² However, with careful design and implementation, modern OSTs can minimize image registration errors (e.g. to a level precise enough to be used as visual guides in surgery).^63,64 Due to the see-through nature, certain types of image manipulations, such as color inversion and object deletion, cannot be performed in OSTs. 

Types of Enhancements

This section will discuss various types of visual enhancement techniques used to improve mobility. 

Scene Simplification

Scene simplification is a process where objects or information that is of little importance to the current task is removed from the scene. By reducing visual clutter, users can focus on essential features, such as obstacles in the scene. Six studies,^{45–48,53,65} summarized in Appendix Table A1, assessed the effect of scene simplification on mobility and mobility-related visual tasks. 

Using a low-resolution occluded display, Hicks et al.⁴⁷ used a scene simplification approach to convey depth information by making nearer objects brighter than more distant ones. Assessing the feasibility of this representation for visual navigation involved seven participants with normal vision. The average time taken and number of collisions decreased and the average median velocity increased over 10 mobility trials. The study did not include a control condition. Eighteen people with severe sight-impairment were also able to detect 2 by 2 bright lights representing objects located within 30 degrees from the center of the VF in less than 4 seconds leading the authors to conclude that scene simplification may help people with sight-impairment avoid obstacles. However, when this scene simplification approach was evaluated in a separate study involving participants with sight-impairment and a mobility course, although obstacle avoidance was improved, the time to complete the course was increased, and participants were more hesitant while using the HMDs compared to the performance without them.⁴⁸ Another study⁵³ also simplified the scene by using low-resolution pixel intensity to represent distance (depth) and found depth-based representation to have less reduction in walking speed compared to color-based representation in a mobility course with overhanging obstacles. Using the HoloLens device, Kinateder et al.⁴⁵ represented depth in different high-contrast color and opacity patterns. The high-contrast visual patterns improved object recognition and allowed the four participants with sight impairment to detect obstacles from further away compared to using a cane or residual vision. These studies suggest that scene simplification can help people with obstacle detection by encoding depth information. 

Another simplification technique is to color-code different categories of objects.^46,65 However, although this color coding technique worked for object recognition in static photographs,⁶⁵ it caused a significant reduction in PPWS in eight participants with PFL and eight age-matched controls when evaluated using a mobility course compared to performance without HMDs.⁴⁶ 

Although the depth-based scene simplification techniques showed potential, especially in obstacle detection, the experiments were carried out in controlled laboratory environments with large obstacles, and it is not clear if this will translate to real-life scenarios where obstacles are often smaller and spaced much more closely. 

Minification

Minification aims to help people with PFL by shrinking the image so that more of the visual field is projected onto the center of the FOV. Expanding VF size is potentially useful as it is an important predictor of mobility performance.^13–15 Eight studies,^54,66–72 summarized in Appendix Table A2, assessed the effects of minification on orientation and mobility. 

Minification has been used to expand of the VF of participants with RP and PFL by 2 to 4 times.^54,66,67 Overlaying a minified contour image, using a multiplexing approach, did not reduce VA and CS in one study⁶⁷ but did in another where the minified image was a grayscale.⁵⁴ 

Comparing performance with and without HMDs, minification using grayscale images did not significantly improve mobility in an obstacle-free course under very dim conditions (< 0.1 lux) in people with night blindness and PFL.⁵⁴ However, in obstacle-filled courses at 16 lux and 2 lux, HMD use significantly decreased PPWS for the same participants and increased numbers of obstacle contacts.⁵⁴ Conversely, using four times minified black and white image overlays, another study observed a significant reduction in the number of collisions without a reduction in walking speed in eight people with PFL and night-blindness caused by RP compared to the performance without the HMD.⁶⁸ This study did not provide clinical information about the participants, and the apparent contradiction could result from differences in participants or differences in the complexity of the experimental environments. 

Although it was expected that overlaid minified images in the central FOV would be a source of distraction during walking and reduce collision judgment performance, no significant change was observed between the performance with and without the HMD.^69,70 However, it is unclear how applicable these results are in the real-world as the studies were conducted in a virtual environment that simulated walking through a supermarket corridor without requiring participants to walk physically. 

In comparison with no enhancement, minification with contour images also improved visual search performance when the participants’ original VF was not too limited and auditory cues decreased search time for all the participants by 54% on average.⁷¹ Of 12 participants with PFL, those with a VF ≥ 10 degrees experienced a 22% reduction in search time when identifying low-contrast letters using minification. However, minification had a significant adverse effect on search time (177% increase) for those with VF < 10 degrees. Visual search task performance was better using minified images that were based on colored contours rather than black and white contours.⁷² These studies showed that minification techniques could also improve visual scanning, which is important for obstacle detection. The experiments in these studies used widely differing outcome measures, from VF size assessment to visual search and mobility course performance. Although these various measurements may be important predictors of mobility performance, more extensive experiments on actual mobility performance should be carried out before judging the usability of the minification technique for mobility. 

If HMDs can provide VF expansion using minification in a way that does not compromise residual VA or CS, there is potential to improve mobility for people with PFL. 

Other Alternative Visual Enhancements

This section will discuss the studies, summarized in Appendix Table A3, that used alternative visual enhancements, such as edge enhancement, digital zooming, and visual cues to direct the users’ attention to an area of interest. 

The contrast of visual scenes can be improved by superimposing bright outlines or edges of the objects in the scene in real time.^60,73 These contrast enhancements resulted in significant improvements in the CS of participants with CVL and in normal participants using diffusor films. Providing dynamic magnification also improved VA and CS measurements proportionate to the magnification level.^67,73 These enhancements were useful in improving orientation and mobility performance (e.g. locating and moving to objects in a large room⁷³ and reading signs from a distance).⁷⁴ 

Jang et al.⁵⁰ also reported improved near and distant VA measurements and claimed that one participant went from being not able to walk independently to being able to drive after 3 months of using the device. However, the experimental outcomes were not clearly presented, and it was unclear if this improvement was because of using HMD alone or the result of cataract surgery. 

HMDs can also be used to generate visual cues that can improve visual scanning performance. In comparison to using standard glasses with an unenhanced visual scene, an HMD using algorithms that can recognize objects and superimpose visual cues, such as flashing lights, to highlight or to direct the user's gaze, can reduce search times by approximately 46% while simultaneously improving search accuracy from approximately 93% to 100%.⁵⁹ In two further studies,^75,76 an HMD that detects obstacles and identifies a safe path used visual cues as a directional indicator and/or various audio messages to aid navigation in indoor environments. Using this device, participants with amblyopia showed improved performance in terms of time taken and number of collisions, especially in unfamiliar settings.⁷⁶ Although these visual cues showed potential, their success is contingent on the performance of the underlying computer vision algorithms. 

Another study⁷⁷ showed that having to recognize virtual elements in a scene, such as shapes and text in AR environments, significantly reduced walking speed for 18 participants with LV and 18 normal vision controls. Walking time for participants with LV increased by approximately 12% and approximately 10% when they were viewing text or shapes respectively. This study highlights the need to understand what information improves safe mobility without negatively impacting on efficiency. 

The enhancements used in these studies show that visual function parameters, such as VA, CS, and scanning ability, can be usefully enhanced to improve mobility. 

Off-The-Shelf Devices

This section discusses the studies, summarized in Appendix Table A4, that explored the effects of off-the-shelf devices on visual function. All these devices provided general vision enhancements, such as variable image magnification, contrast enhancement, and color reversal. 

Two studies,^78,79 using some of the world's foremost head-mounted low vision devices, recorded improvements in VA and CS and near-distance task performance, such as reading and writing. Despite the limited functionality and bulkiness of the devices, self-reported mobility performance decreased only in a minority of participants with the rest having better mobility performance due to increased VA. 

Compared to using conventional optical devices, Culham et al. (2004)⁵¹ used four commercially available devices (Flipperport, Jordy, Maxport, and NuVision) and found reduced performance in near-distant and far-distant tasks, including a visual search test. Flipperport, Maxport, and NuVision devices would not have been suitable for mobility tasks as Flipperport and Maxport are table-mounted and handheld camera systems, respectively, and NuVision lacked auto-focus functionality. Jordy, which produced a comparable performance to optical aids for visual search task, could be useful as an orientation aid. However, a study⁴⁹ with a device called eSight recorded instant performance increases in clinical vision tests and the Melbourne ADL score, but this did not improve further after 3 months of home use. After 3 months, the mobility subscale from the Veterans Affairs Low Vision Visual Functioning Questionnaire (VA LV VFQ) did not improve. However, other subscales (i.e. reading, visual information, and visual motor) did improve significantly, suggesting the device may be useful for orientation. 

A study assessing the usefulness of the MultiVision night vision device, in comparison with no device, showed reduced mobility performance as scored by an orientation and mobility expert based on the number of cane contacts, body contacts, and mobility errors while walking around poorly lit (14.5 lux and 2.5 lux) city blocks.⁵² Only two participants with night blindness took part in this study and, therefore, these results should be treated with caution. 

Many of the studies that used off-the-shelf devices used devices which are now 10 to 15 years old. The relatively poor performance recorded in these studies suggests why these visual aids have not been adopted more widely. 

Human Computer Interaction Aspects

This section will describe how the HMDs were perceived in terms of ergonomics and performance. The usability of devices will play a significant role in determining the success or otherwise of HMD-mobility aids, and yet most of the studies did not mention human computer interaction (HCI) aspects. 

Color-based scene simplification may be problematic for people with poor color perception. Additionally, they remove potentially useful visual cues, for example, surface texture, original colors, and shadows. Minification techniques require users to have good central vision and, therefore, are unlikely to be useful for people with CVL. Additionally, this technique may also cause similar problems to those caused by prismatic field expansion (i.e. confusion due to different overlapping views and double vision).⁸⁰ However, there is potential to improve usability of these minification techniques with long-term usage. For example, a study with 4 participants with RP who used a minification enhancement for 2 weeks at home showed improved orientation and mobility skills to a limited extent.⁶⁹ Other techniques, such as magnification, edge enhancement, and visual cues could be useful for people with LV regardless of the type of field loss. Due to the various limitations discussed previously, HMDs remain impractical as independent mobility aids. However, they may be able to supplement current mobility aids by enabling people with LV to access information beyond that is available by long canes or guide dogs. 

In the experiments with older and bulkier devices, some participants rejected them due to their fear of drawing unwanted attention to themselves and feeling different from others.⁷⁹ One common positive that stood out across all the studies over two decades was the customization provided by HMDs ranging from older HMDs’ dynamic magnification to newer ones’ ability to combine different enhancements,⁷⁴ highlighting the different needs of different individuals. 

OSTs made it harder for some participants (both CVL and PFL) to recognize shapes and text when the visual augmentations and environment had similar background colours.⁷⁷ Participants with night blindness and PFL also did not like having transparency in poorly lit situations; however, they did not feel the small screen size of HMD to be an issue.⁶⁶ Participants with PFL reported the need for color displays at pedestrian crossings with signal lights, which were not distinguishable by shape.⁵⁴ 

In the study with the HoloLens that color-coded object distance, only one participant with RP and PFL found it to be helpful at night.⁴⁵ The form factor of HoloLens along with display lag was also reported to be unhelpful. 

Lack of familiarity with new technologies and visual enhancements could also affect performance. However, the majority of the studies described here did not specify how much time was given for training. When training information was included, training time appeared to be rather limited, ranging from 2 to 3 minutes to 30 minutes. The impact of extended continuous usage of HMDs could be a critical ergonomic issue, but it has not yet been evaluated. Some experiments took up to 90 minutes to complete, but it was not clear if breaks were factored in the experimental design and so fatigue may be another experimental variable. 

Discussion

Only 10 of the studies reviewed involved a quantitative assessment of mobility performance and of these, 8 of them^{45–48,52–54,77} recorded a significant reduction in efficiency with vision enhancement. Of the other two studies, which observed similar or better efficiency, in both of them,^68,76 the mobility courses were relatively simple with large walkable areas and a small number of relatively large obstacles. Therefore, currently, there is no good evidence that vision enhancements using HMDs improve walking efficiency. However, vision enhancements, such as simplification and minification, did seem to help with obstacle detection and collision avoidance. This could result from the benefits of the enhancement, such as improved CS and VF but also could be due to other factors, such as increased participant concentration, the relatively large obstacles used in the experiments, or the participants merely taking their time to complete the mobility course safely. Although the effect on mobility has not been tested thoroughly, the minification technique proved to be useful in expanding VF^66,67 and improving visual search performance^71,72 in people with RP. 

Typically, the experiments involved only a small number of people with LV (the mean sample size in all studies was just 16, and the mean sample size of participants with LV was 10). Some studies simulated LV in people with normal vision using occlusion foils,⁴⁵ diffuser films,^60,67 or by avoiding using optical corrections⁸¹ to reduce VA and CS and simulate tunnel vision.⁷² Although these low vision simulations can be useful, they cannot simulate other visual problems, such as light sensitivity, color vision impairment, or patchy vision. Besides, many visual problems progress slowly over time, so people with LV may adapt to make use of less visual information and rely more heavily on other senses. In contrast, participants with simulated LV have not had the same chance to adapt to any visual deficiencies. Only two studies^52,66 involved mobility testing in the real world, and the experimental setups varied widely between studies. Often, the mobility courses did not reflect real-life environments. That is, they used large, high-contrast obstacles, large walkable areas, and did not include surface level changes, light level changes, or dynamically moving obstacles and stairs.^{46–48,53,68} Therefore, it is difficult to understand how the results might apply in real-world situations. Another limitation of the existing literature is the failure to compare mobility performance with HMDs with that achieved using the person's usual mobility aids. Most of the mobility experiments used unaided mobility performance as the comparator. Only two studies compared the performance of the technology against that achieved with a cane.^45,52 Hence, most studies have failed to assess the real-world benefit of HMDs. Measuring mobility performance using HMDs in conjunction with habitual aids, as three studies did,^52,77,82 could also be useful to understand how HMD technology can augment existing aids. The familiarization period was not specified in most of the studies and was also limited when mentioned, ranging from 2 to 3 minutes to 30 minutes. The resulting lack of familiarity with the technology may also have had a significant impact on measured performance. 

Another challenge in this area of research is how success is measured. Whereas time taken, number of contacts with obstacles, PPWS, and raw walking speed were mainly used as outcome measures,^{46–48,53,54} other studies used some extrinsic measures, like confidence, hesitation,⁴⁸ and perceived safe passage distance.^69,70 Outcome measures related to visual search tasks included gaze efficiency, gaze directness, gaze path, and time taken to find the item. However, more evidence confirming the validity of these secondary measures is required. It would be instructive to find out what people with LV think are important mobility-related outcomes. Due to the use of different outcome measures and mobility courses, it is difficult to compare study results. 

Although the observational, cross-sectional, within-subject study design is sufficient for preliminary studies, ultimately, future research should strive to evaluate new technologies using the randomized control trial study design. Future studies should assess the impact of HMDs on both broad classes of vision loss (i.e. PFL and CVL) using participants with low vision. Studies should also have sufficient power (sample size) to deliver meaningful results. Future studies should describe the biographical characteristics of the cohort in terms of VA, CS, VF, and underlying visual problems, and the adaptation time with the HMDs as these parameters have the potential to impact the outcomes of the studies significantly. 

Future studies should consider measuring mobility performance using PPWS, a widely adopted outcome measure in mobility experiments, to enable comparison between different studies and different rehabilitation methods. Mobility performance with HMDs should be compared against the mobility performance with habitual aids and against the performance with the HMDs in conjunction with the habitual aids to better understand the benefits of HMDs and how they can supplement existing mobility aids. The mobility courses should also reflect the challenging aspects of the real-world environment experienced by people with LV, for example, variations in surface level, different sizes of obstacles at different heights, different light levels, and stairs. Currently, there is no well-established method to measure orientation performance. Nonetheless, studies should report how the HMDs affect VA, CS, VF, and the perception of real-world targets. For successful long-term adoption of HMDs, it is also essential to understand how acceptable the devices are to people with low vision.⁴² Therefore, it is of value to report subjective feedback regarding usability issues, for example, comfort, ease of use, confidence in using the HMD, and incorporate results from validated questionnaires (e.g. the mobility subscale from the Impact of Vision Impairment [IVI] questionnaire)⁸³ where applicable, in addition to the quantitative results. 

Future Directions and Conclusion

HMD technology offers an unprecedented opportunity to enhance vision in people with LV. However, to realize the full potential of this technology as a mobility aid, some fundamental questions need to be answered. For example, how does the user's underlying visual condition impact on the value of different image enhancements? Which enhancements are best suited to those with CVL, PVL, reduced CS, and reduced acuity? In which environmental conditions can HMDs offer an advantage? Are they more helpful in high or low luminance conditions, and is this dependent on the underlying eye condition? It may also be helpful to understand how the users’ age, gender, and familiarity with technology affects mobility performance when using HMDs. From this perspective, HMD-based enhancements can be tailored to the needs of specific users. Additionally, it would be helpful to know how long it takes for people with LV to adapt to new HMD technology. 

With powerful HMDs becoming increasingly available, researchers in this area can focus on implementing and optimizing various image processing algorithms.⁸⁴ Enhancements may adopt the traditional approach of making things bigger, bolder, and brighter. They can also offer different types of information that transcend what is possible with traditional low vision aids — for example, providing distance information, identifying objects, and providing a safe route to follow to reach the desired destination. Another challenge is designing efficient and intuitive augmented environments. As an extensive amount of customization will be required to adapt these enhancements to an individual's needs, it will be essential to identify a suitable means of achieving this. Possible input methods to explore range from speech commands to hand-tracking, gaze input, and gesture recognition. It is also possible to leverage the power of machine learning to understand the adjustments users would like to make based on their preferences and the environment. There is also room to improve accuracy, efficiency, and robustness of machine learning techniques in computer vision, such as SLAM techniques,⁸⁵ to understand the environment and to generate a safe path to follow, segmentation techniques⁸⁶ to highlight stairs or surface-level changes, and image style transformation techniques⁸⁷ to maximize contrast and perceivability. 

In addition, electronic vision aids could modify the enhancement dynamically on the basis of user commands (gestures or speech) or the users’ needs. For example, by zooming out to increase the FOV before a person moves, displaying visual cues to aid obstacle avoidance, and identifying a safe path. It would be useful to understand how enhancements can be used together and what are the best transition methods to switch functionality seamlessly to optimize mobility for people with LV. 

With all these possible enhancements mentioned above, HMD-based AR technologies hold great potential as future mobility aids to provide accessible and essential information via different mediums (visual and spatial audio) as standalone aids or as complementary aids to existing mobility aids. 

In conclusion, although current studies show improved visual scanning performance, object recognition, and obstacle avoidance with HMDs, there is currently no evidence that this translates to improved mobility performance. Nonetheless, there appears to be significant potential. HMD technology is advancing rapidly, and this combined with a better understanding of what enhancements work for people with low vision should lead to improved mobility. 

Acknowledgments

Disclosure: H.M. Htike, None; T.H. Margrain, None; Y.-K. Lai, None; P. Eslambolchilar, None 

Appendix

Scene Simplification

  

Table A1.

 

Summary of Scene Simplification

 

Scene Minification Table

  

Table A2.

 

Summary of Scene Minification Studies

 

Other Visual Enhancements Table

  

Table A3.

 

Summary of Various Other Visual Enhancements

 

OTS Devices Table

  

Table A4.

 

Summary of Studies Using OTS Devices