Correction for the Influence of Cataract on Macular Pigment Measurement by Autofluorescence Technique Using Deep Learning

Abstract

Purpose: Measurements of macular pigment optical density (MPOD) by the autofluorescence technique yield underestimations of actual values in eyes with cataract. We applied deep learning (DL) to correct this error.

Subjects and Methods: MPOD was measured by SPECTRALIS (Heidelberg Engineering, Heidelberg, Germany) in 197 eyes before and after cataract surgery. The nominal MPOD values (= preoperative value) were corrected by three methods: the regression equation (RE) method, subjective classification (SC) method (described in our previous study), and DL method. The errors between the corrected and true values (= postoperative value) were calculated for local MPODs at 0.25°, 0.5°, 1°, and 2° eccentricities and macular pigment optical volume (MPOV) within 9° eccentricity.

Results: The mean error for MPODs at four eccentricities was 32% without any correction, 15% with correction by RE, 16% with correction by SC, and 14% with correction by DL. The mean error for MPOV was 21% without correction and 14%, 10%, and 10%, respectively, with correction by the same methods. The errors with any correction were significantly lower than those without correction (P < 0.001, linear mixed model with Tukey's test). The errors with DL correction were significantly lower than those with RE correction in MPOD at 1° eccentricity and MPOV (P < 0.001) and were equivalent to those with SC correction.

Conclusions: The objective method using DL was useful to correct MPOD values measured in aged people.

Translational Relevance: MPOD can be obtained with small errors in eyes with cataract using DL.

Introduction

Macular pigment (MP), which consists of three carotenoids, lutein [(3R, 3′R, 6′R)-lutein], zeaxanthin [(3R, 3′R)-zeaxanthin], and meso-zeaxanthin [(3R, 3′S; meso)-zeaxanthin],^1,2 is important for maintenance of visual functions. Absorption of short-wavelength visible light (blue light) by MP improves contrast sensitivity and reduces glare disability.^3–6 Moreover, the absorption of blue light and reduction of oxygen radicals by MP attenuates light-induced oxidative damage in the photoreceptor cells and may help prevent disorders caused by photooxidative damage,^7–9 such as age-related macular degeneration.^4,10–15 Correct assessment of MP levels in human eyes is important for understanding visual function and vulnerability to age-related macular degeneration. Despite the physiological importance of MP, it is currently not examined in routine clinical practice. One major reason for this is the difficulty of measuring MP in vivo. 

Methods of measuring MP are classified as subjective and objective.¹⁶ Heterochromatic flicker photometry is a subjective method that is most widely used. This method requires the subjects’ understanding and training, which is often difficult to perform in aged subjects, and the examination time is relatively long.^17–20 Fundus reflectometry,²¹ fundus autofluorescence spectroscopy,^22,23 and resonance Raman spectroscopy^24,25 are objective methods. Fundus reflectometry and autofluorescence spectroscopy are currently used in clinical research. Fundus reflectometry is suitable for infants and children who have clear ocular media.^26,27 Two-wavelength (blue and green laser lights) fundus autofluorescence spectroscopy derives MPOD indirectly via measurement of peripheral and central lipofuscin fluorescence. This technique derives MPOD levels at certain eccentricities, as well as the spatial distribution and macular pigment optical volume (MPOV).²⁸ Using this concept, an MPOD measuring module was developed for the confocal scanning laser ophthalmoscope platform of SPECTRALIS (Heidelberg Engineering, Heidelberg, Germany) This device is suitable for adults who have lipofuscin, but it requires close attention when used in aged subjects with cataracts. Because the excitation blue light is attenuated by scattering in media opacities, MPOD levels are underestimated in eyes with cataract or other media opacities. Some correction methods to compensate for this disadvantage have been investigated. Sharifzadeh et al.²⁹ developed a method using fluorescence image pixel intensity histograms in a single-wavelength autofluorescence technique based on a fundus camera. Akuffo et al.³⁰ also developed a set of equations using the grade of opacity (nuclear opalescence, nuclear color, posterior subcapsular cataract) as parameters to quantify the effects of cataract on MPOD measurement by SPECTRALIS. In our previous study,³¹ we quantified the extent of cataract more precisely by using an anterior segment camera system and investigated the effect of cataracts on MPOD levels in a larger number of subjects. We proposed two different correction methods. One was an objective method using a regression equation (RE) with age, grade of nuclear cataract (NUC), and imaging quality index as independent variables, and the other was a subjective method classifying autofluorescence images into three grades and adopting correction factors for each grade. We verified these methods in 13 eyes that were not included in the previous study. The possible error with the RE method was between 7% and 20% of the true levels when MPOD levels were measured at certain eccentricities from 0.25° to 2°, and the error for MPOV within 9° eccentricity was 14%. The possible error by the subjective classification (SC) method was between 11% and 15% when MPOD levels were measured at certain eccentricities from 0.25° to 2° and 9% for MPOV (unpublished data). There are some shortcomings in these two methods in addition to the relatively high possible error. The objective method using a RE requires accurate evaluation of cataract grade by the anterior segment camera system, and calculation of the imaging quality index takes time and effort. Subjective classification of the autofluorescence image varies between observers. In the present study, we applied deep learning (DL) to correct nominal MP values and approach the true MP values more easily. 

Subjects and Methods

Subjects and Measurement of MPOD

We enrolled 258 patients (116 men and 142 women) who underwent cataract surgery at Seirei Hamamatsu General Hospital between September 2016 and March 2020 in this study. MPOD levels were measured in 405 eyes before cataract surgery (147 patients underwent bilateral surgery and 111 patients underwent unilateral surgery). Among them, 208 eyes were excluded from further analyses for the following reasons. Autofluorescence image quality was not good enough for calculation of MPOD values in 160 eyes before surgery because of intense cataract. Fundus disorders such as vein occlusion, multiple drusen, and pigment abnormality in the macula were identified in 21 eyes. Measurement of MP was not performed in 25 eyes. One eye had a rupture of the posterior capsule during cataract surgery, and one eye was mistakenly not included in the analysis. Consequently, 197 eyes of 148 patients (72 men and 76 women) were analyzed. Sixty-two of these eyes were included in our previous study.³¹ The mean ± standard deviation (SD) age of the patients was 73.0 ± 7.9 years (range, 47 to 93 years). 

The patients underwent visual acuity testing, measurement of intraocular pressure, slit-lamp and fundus examination, fundus photography, optical coherence tomography (OCT) (Spectralis OCT, Heidelberg Engineering), and measurement of MPOD level within two weeks before surgery and four or five days after surgery. NUCs were graded before surgery with a Konan Anterior Segment Tri-Camera System 1000 (KATS 1000; Konan Medical, Hyogo, Japan) in 192 eyes. Five eyes failed to be graded due to a problem with the camera. Details of this camera are described in our previous article.³¹ Slit-lamp and fundus examination, fundus photography, OCT, anterior segment photography, and measurement of MPOD level were performed in eyes under mydriasis induced by 2.5% phenylephrine hydrochloride and 1% tropicamide. MPOD level and spatial distribution were measured with a prototype MPOD module installed on a SPECTRALIS OCT with MultiColor in the same manner as in the previous studies.^30–32 The cutoff eccentricity was set at 9°. The average optical densities at 0.25°, 0.5°, 1°, and 2° eccentricities (local MPODs) and MPOV were analyzed. 

These prospective case series were approved by the institutional review board of Seirei Hamamatsu General Hospital (IRB No. 2251). The protocol followed the tenets of the Declaration of Helsinki. All patients provided written informed consent at enrollment. 

Defining Actual and Predicted CFs

The MPOD values measured in eyes with cataract were underestimated. In order to obtain the true MPOD values (i.e., MPOD after surgery), nominal MPOD values (i.e., MPOD before surgery) should be multiplied by correction factors (CFs). The ratios of MPOD values measured after and before surgery (i.e., local MPODs after surgery/local MPODs before surgery) were defined as actual CFs for the local MPODs. Similarly, the ratios of MPOV after and before surgery were defined as actual CFs for the MPOV. In contrast, CFs predicted by three correction methods were defined as predicted CFs for local MPODs and MPOV. 

Finding Predicted CFs by Three Different Methods

Predicted CFs were obtained by three correction methods. The first was an objective method using a RE; the second was a subjective method with SC of the autofluorescence image; the third was an objective method using DL. The details of the RE and SC methods are described in our previous study.³¹ 

The correction method using DL was as follows. In this study, the VGG16 network developed by the Visual Geometry Group at Oxford University³³ was used to find predicted CFs, as shown in Figure 1. The VGG16 network was pretrained with the ImageNet database and fine-tuned with three types of images of SPECTRALIS, autofluorescence images by blue and green light and subtraction images of these two. We tested two ways to generate subtraction images from blue and green images. One was a simple subtraction, that is, Green_x,y − Blue_x,y. The other was a logarithmic subtraction similar to SPECTRALIS algorism used for creating MP images, that is, log(Green_x,y − Offset) − log(Blue_x,y − Offset). The output layer of the original VGG16 network consists of 1000 classes, which were removed and modified to predict each CF in five classes (local MPODs at four locations and MPOV). To suppress overfitting, we augmented the data by horizontal flipping and random cropping, thereby increasing the dataset 20 times. In random cropping, the training images were cropped from 768 × 768 to 512 × 512 pixels, where the left upper corner of the cropped region ranged randomly from (0, 0) to (255, 255). For validation images, we cropped the center region where the left upper corner was fixed to (128, 128). In preprocessing, the cropped images were resized to 224 × 224 pixels by bicubic interpolation and the mean pixel value of ImageNet dataset was subtracted to be added to VGG16. The mean squared error was used as the loss function. The number of epochs and the minibatch size were 100 and 96 images, respectively, and Adam was used for optimizer. The learning rate was set to 10⁻⁵ except for that of the final fully connected layer (10⁻⁴). To stabilize the uncertainty of the network estimation, we calculated an ensemble of three VGG16 networks with different initial weights in the fully connected layer on the top of VGG16, as shown in Figure 1. 

Figure 1.

 

DL to obtain predicted CFs for the compensation of nominal MPOD at four eccentricities and MPOV. Three types of images of SPECTRALIS OCT with MultiColor, autofluorescence images by blue and green light, and subtraction images of these two were fine-tuned by the VGG16 network. An ensemble of three VGG16 networks with different initial weight in the fully connected layer on the top of VGG16 was calculated.

To evaluate the performance of the CNN, we performed leave-one-case-out cross-validation. One or both eyes of a single subject were used as validation data, and the remaining subjects were used as training data. This procedure was repeated until each subject in the original sample was used once as validation data. In other words, for each individual, only the data from all other subjects were used in the prediction. Training and validation were run on a computer with Ubuntu 16.04, a graphic processing unit (NVIDIA Quadro P600 with 24 GB of memory), and Chainer 4.4.0. 

Evaluation of the Accuracy of Correction Methods

Local MPODs and MPOV were corrected with predicted CFs. These corrected values were compared with values measured after surgery (i.e., the true values), and errors of corrected values to true values were calculated as error = (corrected value − true value)/true value. The actual error was calculated in the same manner: actual error = (nominal value − true value)/true value. 

Statistical Analyses

The differences in all local MPODs and MPOV between before and after surgery and the actual error and errors of three correction methods at four locations (0.25°, 0.5°, 1°, and 2° eccentricities) and MPOV were compared using Tukey's test and a linear mixed model. However, five eyes that failed to be graded for lens nucleus were excluded. The linear mixed model is equivalent to ordinary linear regression in that the model describes the relationship between the predictor variables and a single outcome variable. However, standard linear regression analysis makes the assumption that all observations are independent of each other. In the current study, measurements were nested within subjects and measurement locations and therefore were dependent on each other. Ignoring this grouping of measurements will result in underestimation of the standard errors of regression coefficients. The linear mixed model adjusts for the hierarchical structure of the data, grouping measurements within subjects and measurement locations to reduce the possible bias resulting from the nested structure of the data.^34,35 The differences in the actual CFs at the selected four eccentricities were analyzed by one-way repeated analysis of variance with multiple Bonferroni comparisons. 

Results

The subjects underwent cataract surgery without any complications, and intraocular lenses were fixed in the lens capsule. The decimal best corrected visual acuity (BCVA) before surgery ranged from 0.2 to 1.2 (Fig. 2) with a log minimum angle of resolution (logMAR) of −0.08 to 0.7 (mean ± SD, 0.11 ± 0.15). BCVA after surgery ranged from 0.5 to 1.2 (logMAR −0.08 to 0.3; mean ± SD, −0.06 ± 0.06). No eyes had deterioration of BCVA, severe intraocular inflammation, corneal disorders, or increased intraocular pressure at the time of measurement of MPOD levels after surgery. The grades of NUC based on the World Health Organization classification system³⁶ were NUC 0, 11 eyes; NUC 1, 158 eyes; NUC 2, 23 eyes; and NUC 3, 0 eyes. Table 1 summarizes the subjects’ demographic data. 

Figure 2.

 

Histogram of decimal BCVA before surgery. The median was 0.8, and one third of subjects had BCVA of 1.0 (logMAR 0) or more.

Table 1.

 

Subjects’ Demographics

 

MPOD and MPOV Before and After Surgery

Local MPOD at the four selected eccentricities was 0.48 ± 0.15 (mean ± SD), 0.47 ± 0.14, 0.47 ± 0.13, and 0.26 ± 0.09 at 0.25°, 0.5°, 1°, and 2° eccentricities, respectively, before surgery and 0.78 ± 0.19, 0.73 ± 0.18, 0.69 ± 0.16, and 0.35 ± 0.11 after surgery. MPOV was 16,038 ± 5049 before surgery and 20,194 ± 5990 after surgery. The mean local MPODs at four eccentricities and the mean MPOV were significantly greater after surgery than before surgery (P < 0.001, linear mixed model) (Figs. 3A, 3B). 

Figure 3.

 

(A) Local MPOD at four selected eccentricities before and after surgery. (B) MPOV before and after surgery. The local MPODs at all eccentricities and MPOV after surgery were significantly greater than those before surgery (P < 0.001, linear mixed model). The box represents the third and first quartiles of the data, and the midline represents the median. By default, the whiskers extend up to 1.5 times the interquartile range from the top or bottom of the box to the furthest datum within that distance. When there are any data beyond that distance, they are represented individually as points (“outliers”).

CFs for local MPODs and MPOV

The actual and predicted CFs by RE and DL methods for the local MPODs at four eccentricities and for the MPOV are shown in Table 2. There was a significant difference in the actual CFs for the four local MPODs, showing an increase of the required CF with decreasing eccentricity (P < 0.001, repeated analysis of variance). The results of multiple comparisons showed a significant difference between 0.25° and 0.5°, 0.25° and 1°, 0.25° and 2°, 0.5° and 2°, and 1° and 2° eccentricities (P < 0.05, Bonferroni correction). Predicted CFs by the SC method for local MPODs and MPOV obtained in our previous study³¹ are shown in Table 3. 

Table 2.

 

Actual CFs and Predicted CFs by Two Correction Methods for Local MPODs and MPOV

 

Table 3.

 

Predicted CFs by the Subjective Classification of Autofluorescence Images for Local MPODs and MPOV

 

Actual Error and Errors of the Three Correction Methods

Figure 4 shows the error without correction (i.e., actual error) and errors by the three correction methods. Compared with actual errors, errors with correction were small for all local MPODs and MPOV. Figures 5A to 5D and 6 show the comparisons between actual errors and errors with the three correction methods at four locations (0.25°, 0.5°, 1°, and 2° eccentricities) and MPOV. The actual error at 0.25° eccentricity (Fig. 5A) was significantly greater than the errors with the three correction methods (P < 0.001, linear mixed model with Tukey's test). However, no significant differences were observed among the three correction methods (P > 0.05). The actual error at 0.5° eccentricity (Fig. 5B) was significantly greater than the errors with the three correction methods (P < 0.001, linear mixed model with Tukey's test). However, no significant differences were observed among the three correction methods (P > 0.05), although the difference between the values with the SC and simple DL methods approached significance (P = 0.072). The actual error at 1° eccentricity (Fig. 5C) was significantly greater than the errors with the three correction methods (P < 0.001, linear mixed model with Tukey's test). No significant difference was observed between the values with the SC and RE methods (P > 0.05). The error with the DL methods was significantly smaller than that with the RE method (P = 0.0029) but was not significantly different from that with the SC method (P > 0.05). The actual error at 2° eccentricity (Fig. 5D) was significantly greater than the errors with the three correction methods (P < 0.001, linear mixed model with Tukey's test). However, no significant differences were observed among the three correction methods (P > 0.05), although the difference between the values with the SC and simple DL methods approached significance (P = 0.058). The actual error of MPOV (Fig. 6) was significantly greater than the errors with the three correction methods (P < 0.001, linear mixed model with Tukey's test). The error with both DL methods was significantly smaller than the error with the RE method (P < 0.001) but was not significantly different from that with the SC method (P > 0.05). 

Figure 4.

 

Error without correction (i.e., actual error) and errors with correction by three different methods for local MPOD at four selected eccentricities and MPOV. Compared with actual errors, errors with correction were small in all local MPODs and MPOV.

Figure 5.

 

Comparisons between actual error and errors with correction by three methods for local MPOD. (A) Errors at 0.25° eccentricity. (B) Errors at 0.5° eccentricity. (C) Errors at 1° eccentricity. (D) Errors at 2° eccentricity. The actual error is significantly greater than the errors with correction by all three methods at all eccentricities. **P < 0.001. At 1° eccentricity, the error with correction by DL is significantly smaller than that with correction by RE (P < 0.05) but is not significantly different from that with correction by SC. At the other three eccentricities, no significant differences were observed among the three correction methods. The box represents the third and first quartiles of the data, and the midline represents the median. By default, the whiskers extend up to 1.5 times the interquartile range from the top or bottom of the box to the furthest datum within that distance. When there are any data beyond that distance, they are represented individually as points (“outliers”).

Figure 6.

 

Comparisons between actual error and errors with correction by three methods for macular pigment optical volume. The actual error is significantly greater than the errors with correction by the three methods. **P < 0.001. The error with correction by DL is significantly smaller than that with correction by RE (P < 0.05) but is not significantly different from that with correction by SC. The box represents the third and first quartiles of the data, and the midline represents the median. By default, the whiskers extend up to 1.5 times the interquartile range from the top or bottom of the box to the furthest datum within that distance. When there are any data beyond that distance, they are represented individually as points (“outliers”).

Discussion

Because cataract disturbs the transmission of excitation and emission lights of Spectralis-MP module, local MPODs and MPOV are underestimated, and appropriate correction is needed to evaluate MPOD in aged people with cataracts. The present three methods could reduce the error and approach the true values. Statistical analyses using a linear mixed model with Tukey's test showed the superiority of the DL method to the RE method for evaluation of the local MPOD at 1° eccentricity and MPOV and the equality of the DL and SC methods for evaluation of local MPODs at all eccentricities and MPOV. The DL method with simple subtraction obtained a slightly smaller error than that with logarithmic subtraction. Therefore the DL method with simple subtraction would be a useful correction method as the SC method to reduce error in measuring MPOD and MPOV in eyes with cataract. 

The mean actual CFs for local MPODs at four eccentricities ranged from 1.36 to 1.73 and were comparable to the CFs found in our previous study, which ranged from 1.42 to 1.77. CFs increased toward the center of the fovea, as shown in the previous study.³¹ This finding was possibly due to intense nuclear opacity and posterior subcapsular opacity at the center of the lens, as proposed by Akuffo et al.³⁰ 

The actual errors for local MPODs at four eccentricities ranged from 24% to 38% (Fig. 4). These high rates of error may have been limited to subjects with intense cataract indicated for surgery. However, subjects with intense cataract whose autofluorescence image quality was not good enough for calculation of MPOD values were excluded from the study. The grade of NUC was 0 in 11 eyes and 1 in 158 eyes. The median BCVA before surgery was 0.8 (logMAR 0.1), and one third of subjects had BCVA of 1.0 (logMAR 0) or more. These findings showed that the subjects’ cataracts were relatively mild and suggested that aged people with mild cataract have impaired MPOD measurement. We propose that measurement of MPOD in aged people who have not received cataract surgery should be corrected by appropriate methods. 

In this study, we used the VGG16 network for transfer learning. Although the VGG16 won second place in the 2014 ILSVRC (ImageNet Large Scale Visual Recognition Challenge Competition),³⁷ this network had better performance in transfer learning tasks than the other well-known CNN networks,³⁸ such as AlexNet,³⁹ GoogLeNet,⁴⁰ and ResNet.⁴¹ We have input not only autofluorescence images by blue and green light but also subtraction images of these two into the VGG16. Because the MPOD is calculated from the logarithmic subtraction image, the subtraction image input would be useful for estimating CFs. The disadvantage of the proposed DL method is that images are rescaled to 224 × 224 to adapt to the ImageNet pretrained network. This may discard the high-frequency components of the original images. The high-frequency components can be used to estimate the amount of noise in the image. Because the amount of noise can be considered correlated with the CFs, the high-frequency component may be important for estimation of the CFs. To overcome this disadvantage, a new full scratch convolutional neural network instead of fine-tuned (pretrained) model could be considered. However, in this study, the amount of data was limited, and therefore we chose fine tuning of VGG16 instead of the full scratch. 

The correction method using DL showed high accuracy to compensate local MPOD at 1° eccentricity and MPOV compared with correction by the RE method and was equivalent in accuracy to correction by SC. Considering that the SC method varies between observers, the correction method using DL is considered useful to compensate MPOD values measured in aged people. However, even with correction using DL with simple subtraction, the error in MPOV was 9.6% and that in local MPODs was > 10%. Akuffo et al³⁰ achieved a mean error of 0 (SD, 0.18) for local MPOD at 0.23° eccentricity in 29 eyes using the RE as follows: CF at 0.23° = 0.222 + 0.184 × extent of nuclear color + 2.193 × local MPOD at 1.72° before surgery. Unfortunately, despite using this method, the error ranged from −32% to 46%. Parameters, such as nuclear opalescence and color, in the RE were determined according to the subjective observation using a slit-lamp, and the parameters used in RE varied in different eccentricities. The correction method by Akuffo et al.³⁰ achieved good compensation, but it was slightly complicated, nonobjective, and impractical to different eccentricities they investigated. In contrast, the DL method was computer based, and it needed no effort from physicians; however, errors in the present results were still unsatisfactory to use in scientific investigation on MP. The relatively small amount of data was a shortcoming of this study, although pretraining using ImageNet database and data augmentation by the horizontal flip and random crop was applied to compensate for the limited data. The errors may decrease with DL using much more training data. Further investigation with more subjects is warranted. 

Acknowledgments

Supported in part by Hamamatsu Photonics K.K. 

Disclosure: A. Obana, None; K. Ote, Hamamatsu Photonics K.K (E); F. Hashimoto, Hamamatsu Photonics K.K (E); R. Asaoka, None; Y. Gohto, None; S. Okazaki, Hamamatsu Photonics K.K (F); H. Yamada, Hamamatsu Photonics K.K (E)