Evaluation of the Hand Motion and Peeling Force in Inner Limiting Membrane Peeling

Abstract

Purpose: Robot assistance in membrane peeling may improve precision and dexterity or prevent complications by task automation. To design robotic devices, surgical instruments’ velocity, acceptable position/pose error, and load ability need to be precisely quantified.

Methods: A fiber Bragg grating and inertial sensors are attached to forceps. Data collected from forceps and microscope images are used to quantify a surgeon's hand motion (tremor, velocity, posture perturbation) and operation force (voluntary and involuntary) in inner limiting membrane peeling. All peeling attempts are performed on rabbit eyes in vivo by expert surgeons.

Results: The root mean square (RMS) of the tremor amplitude is 20.14 µm (transverse, X), 23.99 µm (transverse, Y), and 11.68 µm (axial, Z). The RMS posture perturbation is 0.43° (around X), 0.74° (around Y), and 0.46° (around Z). The RMS angular velocities are 1.74°/s (around X), 1.66°/s (around Y), and 1.46°/s (around Z), whereas the RMS velocities are 1.05 mm/s (transverse) and 1.44 mm/s (axial). The RMS force is 7.39 mN (voluntary force), 7.41 mN (operation force), and 0.5 mN (involuntary force).

Conclusions: Hand motion and operation force are measured in membrane peeling. These parameters provide a potential baseline for determining a surgical robot's accuracy, velocity, and load capacity.

Translational Relevance: Baseline data are obtained that can be used to guide ophthalmic robot design/evaluation.

Introduction

An epiretinal membrane (ERM) is an avascular, fibrocellular membrane, such as a scar, that may form on the retina's inner surface and cause blurred and distorted central vision.¹ The prevalence of ERM is 2% in individuals under age 60 and 12% in those over age 70.² 

ERM and inner limiting membrane (ILM) peeling procedures are challenging in surgical treatment. In these procedures, surgeons peel the extremely fragile membrane from eye tissue. Forces less than 10 millinewtons (below human tactile perception) can lead to irreversible structural and functional damage.¹ Moreover, the outcome of ERM/ILM peeling is impacted by many factors, including but not limited to the absence of force sensing and the surgeon's physiological hand tremor, proficiency, and manipulative skills. 

Robotic technology provides a potential way to assist surgeons. Various teleoperated,³ hand-held,⁴ hands-on operated,⁵ and untethered⁶ robotic systems have been developed to enhance and expand the capabilities of surgeons. In robot-assisted ERM/ILM peeling, reasonable performance metrics are required to evaluate the robot's performance and facilitate controller design. 

Figure 1 illustrates ERM/ILM peeling and the movements of the instrument: translation along axis z (feed), rotation around axis z (roll), rotation around axis y (pitch), and rotation around axis x (yaw). Performance metrics related to the movements include the range, velocity, and acceptable position/pose error. In addition, robot performance metrics also include the load capacity and acceptable force error. 

Figure 1.

 

Illustration of robot-assisted membrane peeling.

In robot-assisted membrane peeling, instrument movement is constrained by the eyeball. Therefore the range metrics can be directly derived from the eyeball structure. The surgeon's operation behaviors (hand motion, operation force) can provide baseline data for determining other metrics. For instance, surgeon hand tremor has been used to determine the acceptable position error of a hand-held robot (Micron).⁴ The operation force has been used to determine the acceptable force error in the force control loop.⁷ 

Therefore it is necessary to measure hand motion (hand tremor, velocity, posture perturbation) and operation force in ERM/ILM peeling. The measured triaxial hand tremor and triaxial posture perturbation provide baselines for the acceptable position/pose error. 

The measured velocity comprises the triaxial angular velocity, transverse velocity and axial velocity. These parameters provide baselines for determining the instrument velocity. Operation force below/above 2 Hz can be regarded as voluntary force (VF)/involuntary force (IF) because the bandwidth of human eye-hand feedback is usually from 0.5 Hz to 2 Hz.⁸ Voluntary force and IF can provide baselines for the load capacity and acceptable force error. 

In previous measurements, Singh et al.⁹ measured the hand tremor amplitude and velocity during epiretinal membrane peeling. However, the test included only one sample. Researchers have also measured the force in peeling artificial models (rubber band,¹⁰ polydimethylsiloxane⁷) and the ILM of chicken eggs.^7,11 For the operation force, the VF and IF were not distinguished. In addition, measurement results of the IF, posture perturbation, and angular velocity are still lacking. Therefore more in vivo tests are required to provide additional support. 

For the force measurement method, a research group at Johns Hopkins University developed microsurgical instruments based on fiber Bragg grating (FBG) sensors (resolution: 0.25 mN) to measure the transverse force at the tooltip,¹² the location, and the amount of force on the tool shaft.¹³ 

Optical tracking systems⁴ and inertial sensors¹⁴ are two standard methods of measuring hand motion. Optics can be highly precise but require multiple sightlines to the tracker targets and fail to measure posture. The limited sensing workspace and frequent sightline obstructions also limit the application of optics.¹⁵ Inertial sensors require no sightlines and can measure posture, but inertial sensor accuracy is hampered by many factors, such as gyroscope drift, accelerometer noise, and accumulated error. Data fusion techniques and appropriate algorithms are required to calculate hand tremor.¹⁴ 

It is difficult to replicate the ERM in an animal model. Therefore this article evaluates hand motion and operation force in ILM peeling. All procedures were performed on rabbits in vivo. To record the hand motion and operation force, we placed inertial sensors and FBG sensors on forceps. Inertial sensors were used to measure the hand tremor/angular velocity/posture perturbation. FBG sensors were used to measure the operation force. The velocity of the forceps was calculated from microscope images and inertial sensor data. Note that the measured velocity includes two parts: the transverse velocity and the axial velocity. 

Material and Methods

Hardware and Experimental Setup

This study is intended to measure hand motion and force in membrane peeling. The measured results serve as a potential baseline for robots. Table 1 lists the performance metrics of the robots and the related potential baseline. In Table 1, the involuntary force/posture perturbation refers to the force/posture components caused by hand tremor. 

Table 1.

 

Performance Metrics of Robots and Potential Baselines

 

A group of tests are built to measure the hand motion/operation force during ILM peeling. Figure 2 shows the experimental setup. 

Figure 2.

 

Experimental setup.

We invite three expert surgeons to perform three-port pars plana vitrectomy combined with ILM peeling on New Zealand white rabbits (weighing 2.5–5.0 kg, eight months). Each surgeon performs ILM peeling on three rabbits in vivo (six eyes). 

Each rabbit is intramuscularly injected with xylazine (5 mg/kg) and ketamine hydrochloride (50 mg/kg) as anesthesia (take about 1–2 minutes). The pupil is dilated with Compound Tropicamide Eye Drops (Santen, Osaka, Japan) (takes about 10–20 minutes). Then, the animal is placed on the operating table under a surgical microscope (takes about 5–6 minutes). All procedures are performed using standard 23-gauge three-port pars plana vitrectomy (takes about 20–25 minutes). After core and peripheral vitrectomy are competed through a conventional contact lens, the ILM, with an approximately 2- to 3-disc diameter area at the temporal side of the disc, is removed in all eyes using the proposed forceps with indocyanine green (ICG) staining (takes about 4–5 minutes). 

After ICG staining, if the diameter of the ILM is approximately 2 to 3 disc diameters, the attempt is considered successful. During the peeling process, small blebs sometimes occur because of retinal damage caused by forceps, but these blebs do not affect the operation, and there are no other intraoperative complications leading to surgical failure. 

The Capital Medical University of China Ethics Committee approves all animal procedures. A binocular microscope (M520 MC1; Leica, Wetzlar, Germany) is used to provide a magnified view. A camera (EOS 5D; Canon, Inc., Tokyo, Japan) is attached to the microscope for recording purposes. 

The forceps is used to measure hand tremor, angular velocity, posture, and operation force. As shown in Figure 2(b), the forceps is composed of three parts: an inertial measurement unit (IMU) at the distal end, a force-sensing tip (ϕ0.6 mm × 35 mm) at the proximal end, and a disposable handle. 

The IMU (MTi-630, Xsens, Netherland) measures the hand tremor, posture, and time derivative of the Euler angles. The IMU contains a triaxial accelerometer, a triaxial gyroscope, and a triaxial magnetometer. The data fusion engine of the IMU combines all sensor inputs and provides the measured acceleration and a drift-free estimation of the Euler angles. The measured Euler angles represent the posture of the forceps. In this article, the IMU sampling frequency is 200 Hz. According to the surgeons′ feedback, the IMU mass (approximately 8.9 grams) is acceptable during the operation. 

A force-sensing tip is used to measure the operation force. As shown in Figures 2(c) and 2(d), three FBGs are radially attached to the exterior surface of a 0.6 mm stainless steel tube at 120° intervals (Fig. 2(d)). The diameter and length of the FBG sensor are 0.11 mm and 10 mm, respectively. While applying force on the end of the force-sensing tip, the FBGs deform to shift the center wavelength. After calibration, the relationship between the applied force and the wavelength shift of the FBG can be established.¹¹ 

In this study, an optical sensing interrogator (Si155; Micron Optics Inc., Atlanta, GA, USA) is used to monitor the wavelength shift of the FBG sensors. The sampling frequency of the optical sensing interrogator is 1000 Hz. 

Video recorded by the camera is used to measure the velocity of the forceps. Using the synchronized video, the coordinates of four points (a₁′, a₂′, b₁′, b₂′ in Fig. 2(e)) can be obtained. By combining the coordinates obtained from video and posture of the forceps, the velocity of the forceps can be calculated. A detailed description of the algorithm is given in the next part. In this study, the video resolution and frame rates are 1920 × 1080 and 50 fps, respectively. 

Algorithm

Tremor Calculation Algorithm

We use the algorithm proposed by Saxena et al.¹⁴ to calculate hand tremor. Here, we give a brief introduction. 

Figure 2(b) shows the kinematic representation of the forceps. We attach frame {B} and frame {S} at the measurement origin of the IMU and the nontremulous position of the forceps tip, respectively. The orientation of {S} is identical to {B}. {W} is the world coordinate system. 

The signal fusion engine of the IMU provides the measured acceleration ^BA and a drift-free estimation of the Euler angles (φ, θ, ψ). Angles φ, θ, ψ are defined as rotations about X_B, Y_B, and Z_B, respectively. 

Then, the acceleration at the force-sensing tip (A_tip) can be calculated as follows:  

\begin{eqnarray}{A_{tip}} = {}^BA + \frac{{{}^B{\omega _{t,i,k}} - {}^B{\omega _{t,i,k - 1}}}}{t} \times {}^B{P_{tip}}\end{eqnarray}

(1)

 

\begin{eqnarray}{}^B\omega = \left[ {\begin{array}{@{}*{1}{c}@{}} {{}^B{\omega _x}}\\ {{}^B{\omega _y}}\\ {{}^B{\omega _z}} \end{array}} \right] = \left[ {\begin{array}{@{}*{3}{c}@{}} 1&0&{ - s\theta }\\ 0&{c\varphi }&{c\theta s\varphi }\\ 0&{ - s\varphi }&{c\theta c\varphi } \end{array}} \right]\left[ {\begin{array}{@{}*{1}{c}@{}} {\dot \varphi }\\ {\dot \theta }\\ {\dot \psi } \end{array}} \right]\end{eqnarray}

(2)

where i = X_B, Y_B, Z_B; t is the sampling time; k is the current time step; and k-1 is the previous time step. 

Next, the acceleration due to tremor (^SA_tip) is estimated with the band-limited multiple Fourier linear combiner.¹⁴ The position of the forceps tip due to tremor (^SP_tip = [^SP_tip,x, ^SP_tip,y, ^SP_tip,z]^T) is calculated by double integration of A_tip. A detailed description of the tremor calculation algorithm, calibration procedure, and results can be found in our previous work.¹⁶ 

Force Calculation Algorithm

We can obtain the operation force F by multiplying the FBG wavelength shift ΔS with the calibration matrix (K⁺):  

\begin{eqnarray}{\bf{F}} = {{\bf{K}}^ + }\Delta {\bf{S}}\end{eqnarray}

(3)

where F = [F_x, F_y]^T, \(\Delta {\bf{S}} = {[ {\begin{array}{*{20}{c}} {\Delta {S_1}}&{\Delta {S_2}}&{\Delta {S_3}} \end{array}} ]^{\rm{T}}}\) and where ΔS₁, ΔS₂ and ΔS₃ are the wavelength shifts of FBG1, FBG2, and FBG3, respectively. The meaning of x and y are shown in Figure 2(d). 

Then, sharp high-pass filtering¹⁷ is applied to the operation force F to extract the involuntary force F_inv. The voluntary force F_vol is calculated by F_vol = F − F_inv. The cutoff frequency is set as 2 Hz.⁸ Following the calibration procedure,¹⁸ the relative error and RMSE of the FBG sensors are 0.6% and 0.3 mN, respectively. 

Posture Perturbation/Differentiated Euler Angle Calculation Algorithm

The IMU provides the reduced-noise estimation of the Euler angles (φ, θ, ψ), which describes the posture of the forceps. The nonlinearity of the IMU is 0.1%. 

Similar with Force calculation algorithm, the posture perturbation (φ_dis, θ_dis, ψ_dis) can be obtained by applying a sharp high-pass filter¹⁷ on the measured Euler angles (φ, θ, ψ). The cutoff frequency of the sharp high-pass filter is 2 Hz because the bandwidth of human eye-hand feedback usually spans from 0.5 Hz to 2 Hz.⁸ Posture changes occurring above 2 Hz can be regarded as posture perturbation. 

Velocity Calculation Algorithm

As shown in Figure 2(e), we define four points (a₁, a₂, b₁, b₂) on the forceps: a₁ is the tip of the forceps, a₂ is the incision point, and b₁ and b₂ are the side points of the steel tube of the forceps. In addition, a₁′, a₂′, b₁′, b₂′ are the projections of the abovementioned points on the record image. The length of vector b₁′b₂′ is equal to the diameter of the steel tube d. 

For each image, we define a local coordinate {C} and axial coordinate {A}. Axes Z_C, Z_A and Z_W are identical. 

In {C}, the coordinates of a₁′ and a₂′ can be defined as a₁′ = [a_1x, a_1y, 0]^T and a₂′ = [a_2x, a_2y, 0]^T, respectively. Then, we can obtain the tip velocity v_tip as follows:  

\begin{eqnarray}{v_{tip}} = \gamma \sqrt {\dot a_{1x}^2 + \dot a_{1y}^2} \end{eqnarray}

(4)

where \(\gamma = d/\| {{\boldsymbol{b}}_1^{\prime}{\boldsymbol{b}}_2^{\prime}} \|\). 

Next, we describe the calculation process of the axial velocity. In {A}, vector a₁′a₂′ = [0, 0, γl_axial]^T, l_axial = || a₁′ - a₂′||. The length of a₁a₂ can be derived as follows:  

\begin{eqnarray}{{\boldsymbol{a}}_1}{{\boldsymbol{a}}_2} = \left\| {{}^B{\boldsymbol{a}}} \right\|\end{eqnarray}

(5)

 

\begin{eqnarray}{}^B{\boldsymbol{a}} = \left[ {\begin{array}{@{}*{1}{c}@{}} { - \sin \theta }\\ {\cos \theta \sin \varphi }\\ {\cos \theta \cos \varphi } \end{array}} \right]\gamma {l_{{\rm{axial}}}}\end{eqnarray}

(6)

where a₁a₂ is the real length on the forceps and γl_axial is the projection on the image. 

We can obtain the axial velocity v_axial by differentiating the length of a₁a₂ over time. After calibration, the relative error is approximately 3.4%. A detailed description of the calibration process can be found in Supplemental Material S1. 

Results

All data processing is completed using MATLAB software version R2017a. The boxplots are plotted with Origin 2018. The mean value and SD value of the operation time are 489 seconds and 238 seconds, respectively. It should be noted the operation time indicates the timespan from “insert forceps” (Fig. 3(a)) to “ICG staining” (Fig. 3(e)). 

Figure 3 shows the snapshots during ILM peeling. Figure 3(a) shows a snapshot after inserting the forceps. Figures 3(b)–(d) are snapshots during ILM peeling. Figure 3(e) shows a snapshot after ICG staining. 

Figure 3.

 

Snapshots of ILM peeling: (a) Forceps insertion. (b)-(d) ILM peeling. (e) ICG staining.

Table 2 gives the root mean square (RMS), SD, and maximum (MAX) value for each expert surgeon. Figure 4 shows all measurement results with boxplots. 

Table 2.

 

Statistical Results

 

Figure 4.

 

Boxplots of the measurement results.

Figure 5.

 

Spectrum plots of unfiltered tip acceleration and sample time history of hand tremor: (a)-(c): spectrum plot, X_s, Y_s, Z_s; (d)-(e): sample history, X_s, Y_s, Z_s.

These statistics are counted in the following manner. For instance, expert 1 performed ILM peeling on six eyes in total. We calculate the RMS, SD and MAX tremor along X_S of each eye. After all tests, we obtain multiple sets of RMS, SD, and MAX tremor along X_S. The obtained ranges of RMS, SD, and MAX are 16.29 µm to 25.60 µm, 15.76 µm to 22.33 µm, and 101.8 µm to 152.4 µm, respectively. Therefore, in Table 2, the RMS, SD, and MAX of ^SP_tip,x (expert 1) are written as “16.29–25.60,” “15.76–22.33,” and “101.8–152.4”, respectively. The rest are similar. The statistical results of each eye can be found in Supplemental Material S2. 

Figure 5 shows the spectrum plots of unfiltered tip acceleration and sample time history of hand tremor. The sample history is a part of the data collected from expert 2, the second rabbit, left eye. 

For the velocity, the mean RMS ± SD of v_tip and v_axial are 1.05 ± 0.52 mm/s and 1.14 ± 0.66 mm/s, respectively. The mean MAX values of v_tip and v_axial are 1.91 mm/s and 2.06 mm/s, respectively. For different surgeons, the P values of v_tip and v_axial are 0.22 and 0.28, respectively. 

Table 3 gives the percentage of the voluntary force and operation force. The ratio is defined as follows: first, the data are grouped into 2-mN intervals. Then, we count the number of data points in each group. Finally, the ratio is calculated by dividing the number of data points in each group by the total data points. 

Table 3.

 

Percentage of VF and Operation Force

 

From Table 3, the majority of the voluntary force/operation force (approximately 90.96%/88.95%) range from 4 mN to 14 mN. The voluntary force/operation force range with the highest percentage (approximately 27.94%/28.01%) is (4 mN, 6 mN). 

Discussion

Robot technology provides a feasible way to relieve the challenge of membrane peeling. To initiate the design of robotic surgical platforms, a reasonable motion range, accuracy, velocity, and load capacity are needed. In robot-assisted membrane peeling, the end-effector of the robot is limited by the eyeball. Therefore the motion range can be determined based on the structure of the eyeball. However, reasonable estimates of the acceptable position/pose error, velocity, and load ability have been lacking. 

This article evaluates hand motion/operation force on ILM peeling of rabbit eyes in vivo. The evaluation result provides a potential baseline for the acceptable position/pose error, velocity, and load ability. FBG sensors are used to measure the operation force. The measured hand motion includes the triaxial hand tremor, Euler angles, and velocity. The IMU measures the triaxial hand tremor and Euler angles. The velocity is measured by a camera and the IMU. Then, we obtain the time derivative of the Euler angles. Next, considering that the bandwidth of human eye-hand feedback is usually from 0.5 Hz to 2 Hz, we divide the operation force into voluntary force and involuntary force. We also define angle components occurring at greater than 2 Hz as posture perturbation. 

The hand tremor/posture perturbation/involuntary force provide baselines for acceptable position/pose error. The mean RMS ± SD of the hand tremor (20.14 ± 19.33 µm, 23.99 ± 22.29 µm, 11.68 ± 10.17 µm), posture perturbation (0.43° ± 0.37°, 0.74° ± 0.68°, 0.46° ± 0.41°), and involuntary force (0.50 ± 0.42 mN) serve as a potential acceptable positioning error, angular deviation, and force error, respectively. The RMS value of the hand tremor is close to the previous results (Xs: 24 µm; Ys: 22; µm Zs: 20 µm),⁹ as measured with inertial sensors in vitreoretinal microsurgery. 

The mean RMS of the velocity (transverse: 0.75 mm/s, axial: 0.79 mm/s) is larger than the results measured by the inertial sensors (0.1–0.5 mm/s, max: 0.7 mm/s¹). We attribute this difference partly to the difference in the velocity measurement method. The sampling frequency of cameras is smaller than that of inertial sensors, and we do not need to use integration/filtering algorithms. The surgeons’ habits also contribute to this difference. Compared with retinal microsurgery,¹ surgeons tend to operate faster during in vivo tests. 

The velocity/time derivative of Euler angles provides a baseline for velocity. The maximum velocity of the instrument should be larger than the evaluation results. This means that the MAX transverse velocity and MAX axial velocity should be larger than 2.21 mm/s and 2.76 mm/s, respectively. The MAX angular velocities should be larger than 5.71°/s, 4.29°/s, and 2.80°/s. 

The operation force/voluntary force provides baselines for load capacity. The MAX operation force (13.61 mN) and voluntary force (14.01 mN) are the MAX loads that might occur during ILM peeling. The MAX value is smaller than the MAX peeling force (chicken egg: 21.2 mN ⁷) measured by the FBG sensors. This might be caused by two reasons: (1) the membrane of rabbits is more fragile than that of chicken eggs, and (2) compared with ex vivo tests, surgeons tend to operate more cautiously during in vivo tests. 

In addition to serving as a quantitative baseline for the robot, the evaluation results also provide the following qualitative guidance for robot design and control: (1) from Table 3, the majority of the velocities/differentiated Euler angles are below 2 mm/s and 1.5°/s, respectively; (2) from Table 3, most of the voluntary force/operation force ranges from 4 mN to 14 mN. Emphasis should be placed on the force/velocity control precision and Jacobian matrix optimization over this force/velocity range. 

As shown in Figure 5, the results related to tremor (involuntary force/posture perturbation/hand tremor) contain more outliers. This might be caused by two reasons: (1) the strong stochasticity of tremor causes the measured results to deviate from a normal distribution and (2) unavoidable disturbance and measurement signal distortion. 

Previous work with microsurgical robotics has shown that there are components of positioning error at far lower frequencies (even below 1 Hz).^5,19,20 In addition, in the real world, the ground truth (intended motion) is unknown. The abovementioned two facts indicate that it is nearly impossible to capture the full position error and accuracy. Therefore, we regard the derived metrics as “potential baseline” or “acceptable lower performance limits” rather than “strict accuracy requirements.” A robot that meets all derived metrics should have similar abilities to expert surgeons. This lays a foundation for robot-assisted ILM peeling. 

For instance, the mean RMS values of hand tremor are 20.14 µm (Xs), 23.99 µm (Ys), and 11.68 µm (Zs). If the positioning error of the robot is smaller than the mentioned hand tremor, we can conclude the robot moves more stably than the expert surgeons’ hand. Therefore the robot can potentially be used in ILM peeling since expert surgeons can complete ILM peeling. 

In addition, all derived metrics include a margin of safety. In some peeling attempts, the tremor amplitude/involuntary force was higher than the mean RMS value, but the experts still completed ILM peeling successfully. However, excessive force or hand tremor increases the potential risk of damaging eye tissue. 

Although the number of surgeons is close to that of previous work,^21,22 the number is still small. The following two aspects could compensate for this shortcoming: (1) The surgeons could complete multiple peeling attempts on each eye. Therefore multiple datasets could be obtained from each eye. (2) None of the evaluation results reached statistical significance because the P value ranged from 0.22 to 0.95. Therefore the dataset number (approximately 40 peeling attempts) is larger than the number of eyes. 

In addition, we failed to measure the surgeon's hand trajectory because the cumulative error of inertial sensors is inevitable. A potential way to obtain this trajectory is to link the microscope data with the inertial sensor data. This is part of our ongoing work. 

Conclusion

This article measures the hand motion and operation force during ILM peeling. All procedures are performed on rabbit eyes in vivo, and three expert surgeons participates in the experiments. Our results serve as a supplement to previous research. The measured data can aid the development of the robot system by providing baseline data on surgeon performance. For robot system development, this performance provides potential baseline data for acceptable error (positioning error, pose error, and force error), velocity (transverse velocity range, axial velocity range, and angular velocity), and load capacity (operation force range). 

Acknowledgments

Supported by the National Natural Science Foundation of China (Grant No. 51875011), the National Hi-tech Research and Development Program of China (Grant No. 2017YFB1302702). 

Disclosure: Y. Zheng, None; X. Yang, None; B. Mo, None; Y. Qi, None; Y. Yang, None; C. Lin, None; S. Han, None; N. Wang, None; C. Guang, None; W. Liu, None