Effects of Sleep Deprivation, Non-Dominant Hand Employment, Caffeine and Alcohol Intake During Surgical Performance: Lessons Learned From the Retina Eyesi Virtual Reality Surgical Simulator

Abstract

Purpose: Determine whether real-life surgical experience correlates with scores on a retina virtual simulator and the effects of various challenges on surgical performance.

Methods: The study was performed using the Eyesi Surgical Simulator, a virtual reality retina surgical simulator. Residents, fellows, and retina staff were assessed on surgical simulations to determine surgical skills. Participants were assigned baseline scores on speed, efficiency of movement, and ability to avoid retinal damage. After receiving their baseline scores, participants were challenged to use their non-dominant hand or to use the simulator after sleep deprivation or the intake of caffeine or alcohol.

Results: At baseline, junior residents had an average score of 943; senior residents, 1045; retina fellows, 1153; and surgical retina staff, 1161. A 12.5% overall improvement in scores was achieved when comparing baseline 1 to baseline 2; a major improvement was recorded in residents (14.5%) compared with fellows and retina staff (9.97%). A statistically significant difference was observed between residents and fellows (P = 0.027), as well as between residents and retina staff (P = 0.04). A significant decrease in performance (15.7%) was observed when the non-dominant hand was used (P = 0.043). Performance after sleep deprivation and alcohol intake decreased, but not with a statistically significant difference (P = 0.6 and P = 0.5, respectively). A trend toward increasing performance was observed after caffeine intake (6.1%; P = 0.06).

Conclusions: The retina virtual simulator is a novel teaching tool for retinal surgery. A significant association was observed between real surgical experience and the retina surgery virtual simulator results based on surgical experience.

Translational Relevance: An association between real surgical experience and retina surgery virtual simulator results was demonstrated. A complete training program with a cut-off threshold score should be validated for retina training programs.

Introduction

In recent years, virtual reality simulation (VRS) technology has experienced exponential growth and expanded application into various areas. The use of VRS as a tool for developing and assessing surgical competencies in different fields of medicine has been studied extensively. In ophthalmology, significant effects on the improvement of microsurgical skills have been reported in the literature.^1–8 Some advantages for VRS training include that simulators provide a safe environment in teaching laboratories, unlimited exposure to rare or unusual clinical events, planning and shaping training scenarios¹ rather than waiting for similar situations, and objective results on surgical evaluations.² 

The Eyesi Surgical Simulator (VRmagic GmbH, Mannheim, Germany) is a well-known method for documenting training with objective outcomes, in contrast with other traditional teaching methods.² It uses advanced computer technology to give surgeons three-dimensional views of intraocular anatomy, thus providing the experience of actual vitreoretinal surgery. Because a complete training platform for cataract surgery has been validated, an increasing number of U.S. ophthalmology residency programs are now incorporating VRS into their surgical training programs.^1–3 However, only a few studies have attempted to validate vitreoretinal VRS platforms against actual surgical experience or under such challenging scenarios as caffeine or alcohol intake, sleep deprivation, or use of the non-dominant hand during surgery.^2–7 

Therefore, the purpose of the present study was to determine whether real-life surgical experience correlates with scores on the Eyesi Surgical Simulator. Taking advantage of the safe environment provided by VRS, we assessed the secondary outcomes of various challenging scenarios, such as caffeine and alcohol intake, sleep deprivation, and use of the non-dominant hand during surgery. 

Materials and Methods

Participants

Ophthalmology residents, retina fellows, and surgical retina staff from St. Michael's Hospital in Toronto, Canada, were eligible for enrollment in this study. The inclusion criteria included individuals older than 18 years who were physician trainees or attending physicians at St. Michael's Hospital. Exclusion criteria included subjects younger than 18 years of age or subjects with any cardiac condition. Subjects who chose to participate and signed an informed consent were asked to complete a training program on the vitreoretinal VRS to determine their manual dexterity, degree of tremor, and ability to perform surgical tasks. 

Simulator and Performance Scores

We used the retina Eyesi Surgical Simulator, which simulates in vivo surgical tasks, thus making the experience seem like actual surgery. The vitreoretinal dummy contains a model eye that might move and rotate during surgery. Three virtual instruments are available for insertion at two predetermined port entries in the training eye. A virtual binocular image is generated and projected as a binocular image that would appear under an actual operating microscope. For vitreoretinal surgery simulation, a binocular indirect ophthalmic microscope mimic was provided with the Eyesi Surgical Simulator (Fig. 1). Performance at the simulator was evaluated by the predetermined built-in scoring metrics. 

Figure 1.

 

(A) Eyesi Surgical Simulator. (B) Model of an eye that might move and rotate while surgical tasks are being performed; two instruments are available for insertion at two predetermined port entries in the training eye.

Training Modules

A total of four training programs were included in our study. The navigation test (level five of eight) evaluates depth perception by making dots at different planes change color when touched by a virtual instrument (Fig. 2A). The forceps training task (level five of eight) includes the virtual introduction of a rectangular object into a hollow oval object to help with manipulating the virtual forceps and light pipe (Fig. 2B). The anti-tremor task (level five of eight) requires guiding a dot through a spiral path, and the trajectory is compared to the actual path at the end (Fig. 2C). Finally, the internal limiting membrane (ILM) peeling task (level five of eight) was carried out using virtual ILM forceps (Fig. 2D). 

Figure 2.

 

Training modules employed during the tests: (A) navigation test evaluating depth perception; (B) forceps training test; (C) anti-tremor test; and (D) ILM peeling test.

The score in each module was assessed based on three different parameters, and a score was calculated by the Eyesi Surgical Simulator software. The parameters evaluated were speed, the time required to complete the task; efficiency, completion of all tasks; and tissue treatment, the ability to maintain the condition of the tissue without damage. A maximum of 1600 points could be scored after a complete session, with 400 possible points per completed task. 

Study Design

Subjects were given a brief set of standardized instructions in how to use the Eyesi Surgical Simulator for each training module. Instructions included a demo video and information about microscope adjustment, positioning of the table and chair, and foot pedal and instrument use. The same person gave these instructions to the subjects throughout the study. Each group completed two baseline program sessions of approximately 1.5 hours each, with 1 week between sessions. After completing the two baseline program sessions, participants were challenged to repeat the same program session under challenging scenarios, including using their non-dominant hand, after a quantified moderate sleep deprivation (17 hours), 45 minutes after caffeine intake (391 mg), or 45 minutes after alcohol intake (two shots of tequila, each 44.5 mL). Tasks were performed in the order mentioned before. A lag of 45 minutes was required after the caffeine or alcohol intake to allow the systemic effects of these substances to become noticeable. Each challenging scenario was separated by 1 week from one another, in order to avoid motor or muscle memory that could bias the outcomes. 

Statistical Analysis

Variable distributions were assessed using the Kolmogorov–Smirnov test. Means and standard deviations (SDs) were determined for continuous parametric variables; medians and interquartile ranges were calculated for continuous non-parametric variables. Score differences from baseline were evaluated using a two-tailed, paired Student's t-test or Wilcoxon test, according to normality. Between-group comparisons were assessed by independent two-tailed Student's t-tests and Mann–Whitney U tests, according to normality. For all inferential tests, P ≤ 0.05 was considered significant. Statistical analysis was performed using SPSS Statistics 25.0 (IBM Corporation, Chicago, IL). 

Results

Of all 22 participants evaluated at baseline, the average score of junior residents was 943 ± 21; senior residents, 1044 ± 14; retina fellows, 1153 ± 12; and surgical retina staff, 1161± 13. A statistically significant difference was observed when comparing all residents versus retina fellows (P = 0.027), all residents versus retina staff (P = 0.04), junior residents versus retina fellows (P = 0.023), and junior residents versus retina staff (P = 0.031). No statistically significant difference was observed when comparing retina fellows versus retina staff (P = 0.87). 

A significant increase of 12.5% in performance was observed when comparing baseline 1 to baseline 2 scores in the overall group scores (P = 0.029). Residents had an improvement of 14.8%, whereas retina fellows and retina staff improved only 9.97%. A statistically significant decrease in performance of 15.7% was observed for use of the non-dominant hand (P = 0.043). Both sleep deprivation and alcohol intake resulted in decreases in performance of 3.36% and 5.2%, respectively, although no statistically significant difference was observed (P = 0.6 and P = 0.5, respectively). After caffeine intake, a trend toward increased performance scores of 6.8% was observed (P = 0.06); interestingly, only two out of the 22 participants showed a decrease in performance scores with caffeine intake, but neither of them was a regular coffee drinker. 

Discussion

This study examined and demonstrated a correlation between real-life surgical experience and scores obtained on the Eyesi Surgical Simulator, which provides a set of simulated surgical maneuvers that closely resemble skills required in actual vitreoretinal surgery. A statistically significant difference between tasks was obtained for baseline scores when comparing residents and retina fellows (P = 0.027), as well as residents and retina staff (P = 0.04). Although outcomes for retina staff were higher than those for retina fellows, a statistically significant difference was not achieved (P = 0.87). This finding might be explained because of the choice of difficulty level for the tasks (level five out of eight); a higher difficulty level might have helped reveal more subtle differences between in-training surgeons and experienced surgeons. Vergmann et al.,⁴ compared similar Eyesi simulator tasks among medical students, ophthalmology residents, and trained vitreoretinal surgeons. All groups showed statistically significant differences, but no differentiation between experienced vitreoretinal surgeons and in-training vitreoretinal residents was assessed. Although a correlation between real-life surgical experience and virtual simulation exists, it seems crucial to include different modules or specific difficulty ranks to be able to distinguish between levels of expertise. 

As seen in the Table, a significant improvement was obtained between the first and second sessions in the overall group performance scores (P = 0.029). Ophthalmology residents showed greater improvement (14.8%) than did retina fellows and staff (9.9%), likely because ophthalmology residents had a greater opportunity for improvement than retina fellows or staff. On the other hand, Cissé et al.,⁹ published a similar study and found no improvement in ophthalmology residents on repetitive tasks; however, the outcomes might have been biased because all tasks were done in the same day, thus allowing fatigue and loss of concentration to play an important role in the results. Even though constant training improves VRS performance, the main goal should be to determine an objective score to determine when an in-training surgeon is adequately qualified to perform a surgery on a patient. Further evaluation should be addressed in order to determine acceptable performance test scores. 

Table.

 

Retina Eyesi Training Scores

 

A statistically significant decrease of 15.7% in performance scores was observed when tasks were performed with the non-dominant hand (P = 0.043), but, surprisingly, no difference was observed between any group. It could be expected that retina fellows and staff would have better bimanual dexterity, but this was not reflected in our scores. González-González et al.,⁵ interestingly showed how the learning curve for use of the non-dominant hand for anterior segment surgery was steeper when compared to that of the dominant hand, which might be the reason for our outcomes. The sleep deprivation consisted of 17 continuous hours without sleeping, a time span intentionally selected by analyzing the times after completed night shifts when our retina fellows performed surgery. The outcomes revealed a decrease of 3.36% on performance scores, although no statistically significant difference was observed (P = 0.6). Our study resembles the outcomes reported by Ellman et al.,¹⁰ who demonstrated that acute sleep deprivation among thoracic surgical residents did not affect operative efficiency, morbidity, or mortality in cardiac surgical operations. Grantcharov,¹¹ however, found significantly more errors and longer surgical times with regard to VRS surgical skills on post-call mornings. Our study showed only a moderate decline in sleep-deprived surgeon performance (comparable to the level for simulated surgical skills when they were rested), but these outcomes may not necessarily translate into acceptable performance under actual surgical conditions, which might differ. 

Surprisingly, when participants were tested 45 minutes after having a cup of coffee with 391 mg of caffeine (Health Canada recommends an intake of no more than 400 mg per day), an increase in performance scores of 6.1% was observed, and a trend toward statistical significance was achieved (P = 0.06), Interestingly, only two of 22 participants showed a decrease in performance scores, neither of whom was a regular coffee drinker. Both of these participants described feeling more alert, but they reported feeling tremors, although no statistical difference was seen for their anti-tremor test scores. To our knowledge, the only study evaluating the effects of caffeine in ophthalmic surgeons was published by Humayun et al.,¹² who reported that after an intake of 200 mg of caffeine tremor increased, although no statistical significance was achieved; they did not specify if participants were or not regular coffee drinkers. Although our study showed no statistically significant decrease in performance after a small alcohol intake (5.2%; P = 0.5), tests of other factors important to a safe surgery such as cognitive skills or surgical judgment were not tested. These factors are at least as important as performance skills, so outcomes might not necessarily be extrapolated to real surgery. When Kocher et al.,¹³ evaluated VSR surgical performance after sleep deprivation and alcohol intake, simulating a night out with a 0.86% breath alcohol level, they observed a significant deterioration in performance. Evaluation of breath or blood alcohol levels was not performed in our study, and the intake of alcohol was smaller; it is possible that a significant decrease in performance could have been achieved with higher alcohol intake. 

Limitations of our study include the small number of participants, as we were limited by the total numbers of trainees and attendings in our program. Besides proving correlation between real-life experience and VRS surgical skills, challenging scenarios evaluated by computer-simulated abilities may not accurately reflect true surgical performance, as other factors should be taken into consideration. Our study evaluated moderate sleep deprivation, not the chronic sleep deprivation that might be more common among residents and fellows. A small amount of alcohol intake was evaluated, and breath or blood alcohol levels should have been tested in order to determine objective systemic effects. A strength of our study is the comparison between real-life surgery and scores obtained in VRS, not only on regular scenarios but also challenging scenarios, such as after sleep deprivation, use of the non-dominant hand, and caffeine or alcohol intake. 

Conclusions

Our data suggest that an association exists between real-life surgical experiences and scores obtained on a virtual reality simulator. A decrease in VRS surgical performance was observed for operating with the non-dominant hand, as well as for moderate sleep deprivation and alcohol intake. Regular caffeine intake tended to improve performance, except for those who do not normally drink coffee. Further evaluations should attempt to determine VRS surgery scores that reflect the ability to perform surgery on a patient, as well as the breath alcohol levels and sleep deprivation times that cause a significant decrease in surgical performance. 

Acknowledgments

Disclosure: F.A. Adatia, None; G. González-Saldivar, None; D.R. Chow, None