Instrumented Contact Lens to Detect Gaze Movements Independently of Eye Blinks

Abstract

Purpose: To test on humans an eye tracker based on the use of a contact lens with active infrared sources and to evaluate whether this system can track the eye behind the eyelid.

Methods: The system is made up of a scleral contact lens embedding two vertical cavity self-emitting lasers (VCSELs) remotely driven by eyewear comprised of electronics and cameras. Tests on humans were carried out on five subjects to assess performance and to determine whether the VCSEL spot could be detected behind the eyelid.

Results: The lens was worn and used without difficulty. The device allowed accurate tracking of the eye (0.83° ± 0.59°), and the VCSELs can be detected behind closed eyelids.

Conclusions: These results confirm those previously obtained in terms of tracking and demonstrate that the device can be used safely to monitor eye movements even when the eyelids are closed.

Translational Relevance: The VCSELs could be useful for a variety of applications to reduce data missed due to blinks, particularly with regard to interactive systems, fundamental studies on closed eye movements, or finding ways to improve clinical diagnostic precision for disorders of consciousness.

Introduction

There is growing interest in being able to analyze movements of the eye behind the eyelid, whether to understand better how the visual system works¹ (e.g., the effect of blink on ocular movements^2,3), to study cognitive processes⁴ such as mental representations^5,6 for clinical applications (e.g., comatose patients,⁷ nystagmus⁸), or to better manage the display of information in virtual reality.⁹ A natural approach to tracking the eye behind the eyelid has been using magnetic scleral search coils.¹⁰ This method offers several advantages (e.g., high resolution and accuracy, fast response time) but also presents several disadvantages¹¹ in that the subject's cornea must be anesthetized, and, even with anesthetic, the coils cannot be worn for much longer than 30 minutes.¹² In addition, the subject's eyes must be in a weak, oscillating magnetic field, which strongly limits the portability of the system. Since the first version was proposed by Robinson,¹³ several variations have been developed to address these issues, such as utilizing wearable primary coils¹⁴ or wireless systems.^15,16 These prototypes remain relatively complex in their implementation and potentially sensitive to their magnetic environment but should improve with technological progress. 

An alternative and potentially more robust approach is to use an optical method. The use of a contact lens is an alternative that is currently an ideal platform for a number of applications (e.g., drug delivery,¹⁷ human–machine interfaces¹⁸), particularly eye tracking.¹⁹ In this context, IMT Atlantique's Optics Department and Cylensee have collaborated since 2021 on the development of smart contact lenses for various uses, including eye tracking. In particular, we have proposed the use of a contact lens with an embedded infrared (IR) laser,²⁰ and we have demonstrated that such a contact lens could be used to determine gaze direction²¹ through the use of its easily trackable features,²² with the potential to significantly outperform conventional eye trackers when used outdoors.²³ However, in these studies, the focus was on demonstrating and characterizing the system; therefore, the contact lens was used on model eyes to precisely control the measurement conditions and to avoid any uncertainty on the direction of gaze. 

The current study aimed to test the device on human eyes and demonstrate that, because it uses IR light sources, it can track the eye even with blinking or closed eyelids. In this paper, we first briefly recall, for the sake of clarity, how the lens is made and how the device works. We then present the different tests carried out to guarantee safe use of the contact lens, as well as the experimental protocol used to address our objectives. Results obtained for the different test conditions on human eyes are then presented followed by a discussion on the achieved performance and future work. 

Methods

Device Description and Operating Principles

The system has been described elsewhere,²² but we present it here briefly for the sake of clarity. The contact lens pointer (CLP) system is made up of three parts: a contact lens with two integrated lasers, eyewear to power the contact lens by inductive coupling and to carry cameras to track the lasers, and a computer to process the gathered data (Fig. 1). Here, we use the term eyewear to refer to a device worn over a person's eyes. We prefer this term to that of glasses, because this eyewear serves only to hold the primary antenna, so it could just as well be an augmented reality (AR) or virtual reality (VR) headset rather than glasses. The scleral contact lens is made of polymethyl methacrylate and has a diameter of 16.5 mm. The lasers are vertical cavity self-emitting lasers (VCSELs) emitting at a wavelength of 850 nm (i.e., in the IR spectrum). Like any scleral contact lens, the lens is particularly stable on the eye.^24,25 

Figure 1.

 

CLP elements. (a) The contact lens on an ocular prosthesis (the black arrow shows the position of one VCSEL). (b) The eyewear with the primary antenna inside and the Pupil Core camera, Raspberry PI, and its associated battery. (c) Example of an eye camera capture of the VCSEL spots.

This system features the conventional architecture of a wearable eyetracker, with one camera per eye mounted on both sides of the eyewear, facing the VCSELs, and another one mounted on top of it aimed at capturing the surrounding scene. Here, we used the Pupil Core architecture (Pupil Labs, Berlin, Germany) for its flexibility, open software, and performance. The use of the Pupil Core also allows comparison of the contact lens system with a traditional system. 

As the eye cameras are equipped with IR filters, the information captured is a series of black frames with two light spots corresponding to the VCSELs (Fig. 1c). In this respect, tracking of the eye can be simplified to the detection of these spots and can be easily run in real time via a Python script on a Raspberry Pi (Cambridge, UK) (Fig. 1b). The positions of the detected points are related to the position of the gaze through a calibration procedure. The calibration process and the spot detection algorithm have been detailed previously.²³ 

Safety and Regulation Processes

As a medical device, the CLP must be compliant with European Economic Community (EEC) directive 2017/745 MD Class IIa. A test campaign to obtain EC marking is underway, and several tests have already been carried out by certified external organizations with regard to physicochemical characterization (International Organization for Standardization [ISO] 10993), toxicologic risks (ISO 10993-1:2020; PR NF EN ISO 10993-17:2022), cytotoxicity and eye irritation potential (ISO 10993-5), radio electromagnetic compatibility and electrical safety (International Electrotechnical Commission [IEC] 60601-1-2, International Special Committee on Radio Interference [CISPR] 11), and optical safety (British Standard [BS] EN 60825-1:1994). Preclinical tests have been performed on three male New Zealand rabbits (Fig. 2a) to assess physiological tolerance of the CLP while inactivated and when in operation, particularly with regard to positioning of the lens, animal agitation, and cornea integrity. Fitting tests with unpowered lenses were also performed on humans to evaluate lens comfort and to design a range of lenses allowing for different corneal clearances (Fig. 2b). 

Figure 2.

 

(a) Tests on rabbits with the lens powered. (b) Slit-lamp examination with the lens unpowered.

Experimental Setup and Protocol

Tests on humans were carried out at Brest Hospital under the supervision of an ophthalmologist, after approval from the IMT Atlantique Internal Ethics Committee. Five volunteers, including the authors (MO, VN, J-LB, FMR, and AP; four males, one female) participated in the study. This protocol adhered to the tenets of the Declaration of Helsinki. 

Test 1

The first test aimed to test the contact lens pointer on human eyes and to assess if the VCSEL spot could be detected behind the eyelids. Because no binocular measurements were considered, subjects wore the CLP only on their right eye. The use of a printed circuit board rather than a flexible circuit currently limits the curvature of the lens. As a result, the lens has a negative optical power (–6 diopters) when worn. Furthermore, two of the five subjects were ametropic, which means that, because they did not wear their glasses, their vision had to be corrected. The vision of the eye wearing the contact lens was corrected with a trial frame to assess vision quality through the CLP, but tests were performed without correction. 

Regarding the evaluation of lens wear by human subjects, optical coherence tomography (OCT) and slit-lamp examinations were used to assess the lens fitting and the cornea integrity at the end of the session. A questionnaire with a ranked symptoms scale (RSS)²⁶ was used to assess the subjects’ perceptions of ocular comfort. To observe the influence of the eyelid on VCSEL detection, the following protocol was used. Tests were carried out in a neon light environment of 400 lux. The subject was seated, and a chin rest was used to limit head movements. A camera (UI-1245LE-M-GL; IDS Imaging, Obersulm, Germany) sensitive to visible and near-infrared wavelengths was placed in front of the subject's eye at 27 cm. We used an additional camera and not the one mounted on the eyewear (Fig. 1b) because that camera was mainly adapted to VCSEL detection, and we wanted to observe both the eye and VCSEL emissions. Subjects were asked to stare at the camera first, then (voluntarily) blink three times, and finally move their eyes (high, right, down, left) behind closed eyelids. 

Test 2

The second test aimed to assess the tracking performance of the CLP. The same environment as for Test 1 was used but no camera was placed in front of the subject; instead, only the camera on the eyewear was used to track the CLP (Fig. 1b). Contact lens movements were analyzed using dedicated Python software.²³ After a five-step calibration procedure²² (see calibration targets in Fig. 3), subjects were asked to follow a target projected at 2.1 m using a video projector. In this pursuit exercise, the target drew (three times) a horizontal infinite sign (25° large) starting from the center, in a rightward direction. 

Figure 3.

 

Lab configuration during the calibration step.

Results

Test 1

For all subjects, there was a slight decentering of the pupil due to the lens weight but it did not affect the subjects’ vision. Subjects could successfully be corrected with trial frames (Snellen acuity 20/20); with such correction, no impact on vision was reported (e.g., porthole effect). The axis of the VCSEL pair seemed to position itself around a preferential axis, although there were strong interindividual variations (Fig. 4). Future work will focus on modifying the toricity of the back surface to better control the lens orientation; however, when the lens had been set on the eye, no rotation was observed during the time of the study. No noticeable impact on the integrity of the cornea was observed at the end of the study, after approximately 45 minutes. 

Figure 4.

 

CLP worn by four of the participants (left). Because the lenses are not rotationally stabilized (e.g., with prism ballast), the orientation of the VCSEL on the eye may vary among users.

The overall RSS scores varied across subjects: 1, 1, 3, 5, and 7 on a 10-point scale, where 1 is the most comfortable and 10 most uncomfortable. These results are not surprising and can be considered positive, as the same generic lens model was used for all subjects without prior consideration of their cornea profiles, and only one subject had experience wearing contact lenses. The most common remarks across all subjects were an awareness of the lens and discomfort because of the blurry vision in one eye (because the lens had a negative power). The lack of proper adjustment to the cornea was mildly noticeable (e.g., “eyes felt hot,” “watering eye”), but all subjects wore the lens for approximately 45 minutes without difficulties. 

Regarding the influence of the eyelid on the VCSELs detection, we observed that the laser beam was detected by both open and closed eyes and in all participants, as illustrated in Figure 5 (subject J-LB). When the eyelid was closed, we observed a spot broadening due to tissue scattering. The spot widened by a factor between 7 and 8. However, this value is indicative of camera saturation despite the VCSEL working at the lasing threshold. It is difficult to give an attenuation value for the same reason, but, if we refer to the literature,²⁷ eyelid attenuation at 850 nm for an adult is estimated at 30%. 

Figure 5.

 

Image captures from Supplementary Movie S1. (a) View of the eye and the spot with open eyes. (b) View with closed eyes (subject J-LB).

One can notice in Supplementary Movie S1 that, at the moment the eyelid passes over the VCSEL, there is a strong spot deformation that could lead to incorrect calculation of the VCSEL position. This could be addressed numerically. For example, because the eyelid passage time on the VCSEL is very short, the positions before and after the eyelid passage could be used to improve the calculation. More significantly, for the task when subjects had to fixate on the camera and voluntarily blink, we noticed a slight upper eye movement of 12.91° ± 0.47° when the eye closed, as seen in Supplementary Movie S1 (13.1° as illustrated in Fig. 5). The average amplitude of this movement across participants was 16.22° ± 3.59°. Such movement had no impact on the centroid extraction, which was correctly detected in both cases. This upper eye movement had already been observed with a voluntary blink and is referred to as Bell's phenomenon.²⁸ 

When subjects moved their eye behind the close eyelid, the VCSEL spot could also be tracked successfully, as illustrated in Supplementary Movie S2. This movement behind the eyelid was observed for all participants over a 45° range, on both the vertical and horizontal axes. This range is associated with the camera used and the directional characteristic of the VSCELs.²² 

Test 2

As presented in previous sections, for this test the VCSEL spot was tracked with the eye camera fixed on the eyewear (Fig. 6). To establish a reliable correspondence between the real gaze direction and the measured features in the eye camera images, a calibration chart of five points was placed at the extremities of a rectangle of 25° by 12° and centered on (0°, 0°). The subject had to sequentially fixate the five targets in clockwise order starting with the central point and then moving to the top right corner to finish with the top left one. Figure 7 shows the corresponding eye motion trajectories. 

Figure 6.

 

(a) Subject VN wearing the CLP. The IR sensitivity of the camera allows observation of the emission of the IR VCSELs (only one spot is observable due to the directivity of the VCSEL). (b) Subject MO during the test.

Figure 7.

 

Gaze mapping for the calibration process (blue dots represent the gaze positions, and black crosses represent the calibration targets).

As expected from the results for Test 1, the gaze direction could be tracked continuously without data loss due to blinking. As an illustration, the pixel coordinates of the spot position on the eye camera sensor are represented as a function of time in Figure 8 for subject AP (for the same data illustrated in Fig. 7). Four jumps in the spot position (0.13 ± 0.02 seconds long) can be observed at the following time stamps: 1.9 seconds (central target fixation), 5.9 seconds (top-right target fixation), 11.1 seconds (bottom-left target fixation), and 13.5 seconds (top-left target fixation). The amplitudes of these movements on the x-axis and y-axis are, respectively, 1.91° and 2.87°, 1.27° and 1.83°, 1.27° and 2.09°, and 0.85° and 2.87°. 

Figure 8.

 

Coordinates of VSCEL spots (degrees) as a function of time and target position.

Because we have shown that our device is able to detect the gaze direction despite the eye being closed, we assume that these jumps correspond to brief eye saccades accompanying involuntary eyelid blink—that is, a brief downward movement (when the CLP is worn on the right eye, the central target is nasal), followed by an upward return movement of the eye as mentioned in the literature.²⁹ 

Results for the pursuit exercise are illustrated in Figure 9 for subject AP. Again, there was no data loss associated with undetected VCSEL spots, and the CLP demonstrates an accuracy of 0.83° and a precision of 0.59° These values were averaged over all users³⁰ (using the mean of all points for each person, similar values were obtained: 0.70° ± 0.52°). In the example illustrated in Figure 9, the 2.88° downward trajectory shift that appears during the first loop is progressively corrected in the next loop sequences. This could be explained by a natural oculometric training of the trajectory pursuit. 

Figure 9.

 

Detected eye trajectory (blue dots) and stimulus (continuous black line). Arrows on the bottom right indicate the loop number.

Discussion

We tested on humans an eye tracker based on the use of a contact lens embedding active light sources. The lens was worn and used without difficulty. Two out of five subjects experienced mild discomfort, which is not surprising as only three lens models with different clearances were produced. This limitation is temporary, as we can easily increase the range of lenses to better adapt the geometry of the lens to a subject's eye. The design of the lens still requires some improvement to better control the orientation of the axis of the VCSEL on the eye and to ensure that the lens does not decenter due to gravity, but this can be achieved utilizing recently developed manufacturing techniques (e.g., dual thin zones, prism ballast, deep toric with sinusoidal cleats). 

In terms of tracking, these tests confirmed the good results obtained on artificial eyes. These results are not yet significantly better than high-end wearable eye trackers but better performance (accuracy, precision) could be easily obtained by improving the eye camera and the mapping function. Also, the positioning of the two light sources on either side of the pupil could facilitate the detection of torsional eye movements. 

More importantly, the use of infrared light sources allows tracking the eye despite obturation by the eyelid. This could be useful for fundamental applications, such as studying closed eye movements, or, for example reducing the amount of missing data due to blinks. Missing data in these cases are usually ignored, which can lead to lag and reduced data usability, and requiring interpolation to fill in the gaps presents another problem. Therefore, being able to continuously track the eye despite blinks could be useful for various applications such as human–machine interfaces.³¹ 

Acknowledgments

The authors thank A. Piélot and F.-M. Robert for taking part in the experiments alongside the authors, Erwin Caillet, MD, from the Ophthalmology Department of Brest Hospital for supervising the tests on humans, and Laure Adam, PhD, from LCS Laboratoire for providing and customizing the contact lenses for the subjects. 

Supported by the French National Research Agency (ANR-21-CE19-0053). 

Disclosure: M. Othéguy, None; V. Nourrit, Cylensee (I, P); J.-L. de Bougrenet de la Tocnaye, Cylensee (O, I, P)