Abstract
Purpose:
Our goal was to build a system that combined deep convolutional neural networks (DCNNs) and feature extraction algorithms, which automatically extracted and quantified vascular abnormalities in posterior pole retinal images of full-term infants clinically diagnosed with mild familial exudative retinopathy (FEVR).
Methods:
Using posterior pole retinal images taken from 4628 full-term infants with a total of 9256 eyes, we created data sets, trained DCNNs, and performed tests and comparisons. With the segmented images, our system extracted peripapillary vascular densities, mean tortuosities, and maximum diameter ratios within the region of interest. We also compared them with normal eyes statistically.
Results:
In the test data set, the trained system obtained a sensitivity of 0.78 and a specificity of 0.98 for vascular segmentation, with 0.94 and 0.99 for optic disc, respectively. While in the comparison data set, compared with normal, we found a significant increase in vascular densities in retinal images with mild FEVR (5.3211% ± 0.7600% vs. 4.5998% ± 0.6586%) and a significant increase in the maximum diameter ratios (1.8805 ± 0.3197 vs. 1.5087 ± 0.2877), while the mean tortuosities significantly decreased (2.1018 ± 0.2933 [104 cm−3] vs. 3.3344 ± 0.3890 [104 cm−3]). All values were statistically significantly different.
Conclusions:
Our system could automatically segment the posterior pole retinal images and extract from vascular features associated with mild FEVR. Quantitative analysis of these parameters may help ophthalmologists in the early detection of FEVR.
Translational Relevance:
This system may contribute to the early detection of FEVR and facilitate the promotion of artificial intelligence–assisted diagnostic techniques in clinical applications.
Retinal images were collected from full-term infants born between January 2021 and June 2021 at a tertiary care hospital (Jiaxing Maternal and Child Health Hospital, Jiaxing, China). Full-term infants born within 72 hours were examined after a 30-minute fast. After pupil dilatations, the medical staff used a Ret Cam III (Natus Medical, Inc., Pleasanton, CA, USA) to collect retinal images from different fields of view, first from the right eye and then from the left eye. The size of these retinal images was 1600 × 1200 pixels, from which we selected the posterior pole retinal images to construct three separate data sets.
Criteria for images included the following in the data sets: (1) from full-term infants with a gestation of 37 weeks or more, (2) from full-term infants without a history of asphyxia or oxygenation, and (3) with a clinical diagnosis of mild FEVR with small peripheral vascular abnormalities, such as retinal areas without perfusion, increased posterior retinal vascular branches, and retinal neovascularization.
We assigned a reference diagnostic criterion to each image (mild FEVR containing stages I and II, as well as normal). The determination of the reference relied on the diagnostic consensus from three ophthalmologists with more than 10 years of experience.
As shown in
Table 1, we created three separate data sets using 7644, 1532, and 80 retinal images collected from 3822, 766, and 40 different full-term infants. A total of 48 cases (1.03%) of mild FEVR were detected in the total of 4628.
Table 1. Characteristics of The Three Data Sets
Table 1. Characteristics of The Three Data Sets
Table 2. Independent Samples Test of the Three Features
Table 2. Independent Samples Test of the Three Features
As shown in
Figure 1, at the beginning of our system, the input image was segmented by two DCNNs separately. One DCNN segmented the optic disc (OD) for localization and calculating of the ROI, and the other DCNN segmented the retinal vascular. To train the vascular segmentation DCNN, we used 7644 images from the training data set, and we also randomly selected 1528 images from the training data set to train the OD segmentation DCNN.
Retinal images in the training and test data sets were manually annotated by a professional ophthalmologist for training and testing. The ophthalmologist used medical imaging-specific annotation software to manually label the OD or used a brush annotation tool to annotate the vascular.
For the DCNNs, our system used two open-source modified U-Nets based on an encoder-decoder architecture (Retina U-Net).
17 During training, we modified the program parameters by setting the number of interlayer convolutional kernels to 64, the momentum to 0.9, the learning rate to 0.0001, the decay rate to 1e-6, and the gradient crop to 5.0. Our system also selected a stochastic gradient and processed 16 images per batch. The two DCNNs were all initialized from a Gaussian distribution N (0, 0.01).
We used the area under curve (AUC) score under the receiver operating characteristic curve to measure the performance of the segmentation networks during the training. To avoid overfitting and underfitting, we divided the training data set into five parts, randomly selected four parts for training, and used the remaining for testing, and the cross-validations were repeated five times (fivefold cross-validation). We used the Scikit-Learn library tools (French Institute for Research in Computer Science and Automation, Rocquencourt, France) to calculate the AUC scores and calculated the 95% confidence intervals using the formula of Hanley and McNeil.
18
Based on the AUC scores, we selected the best configuration and conducted performance tests. To measure the performance of the segmentation networks, we calculated the sensitivities and specificities.
All networks were implemented in Tensorflow 1.13 (NVIDIA, Santa Clara, CA, USA) and evaluated on a computer with an NVIDIA GeForce 3090 GPU, 64G RAM, and an i9-12900KF CPU. All models and feature extraction algorithms were implemented by Python 3.6.5 (Python Software Foundation, Inc., Wilmington, DE, USA). We trained our DCNNs on the training data set, tested their performance on the test data set, and evaluated the features obtained on the comparison data set using SPSS 26.0 and the RF algorithm.
In the current study, we developed a system capable of automatically segmenting and extracting abnormal vascular features from posterior pole retinal images (acquired by Ret Cam III) of full-term infants. We also performed quantitative analysis on the features within the ROI (densities, mean tortuosities, and maximum diameter ratios). The results showed that these features differed significantly between full-term infants with mild FEVR and normal controls.
Unlike the manual extraction of abnormal features in the posterior polar retinal images of adults with mild FEVR by Yuan et al.,
6 we used two DCNNs to automatically segment the OD and the retinal vascular, respectively. The OD segmentation images were used to localize and calculate the ROI, while the vascular segmentation images were used to extract features (within the ROI). This helped to improve the disadvantages of manual extraction, which is very time-consuming, has subjective variation, and is poorly reproducible.
7
We have not found reports that automatically segmented and extracted features from posterior pole retinal images of full-term infants with mild FEVR. Meanwhile, we noted that Redd et al.
30 used a U-Net to segment retinal vascular and OD for the study of the automatic diagnosis of ROP. We also noted that Yildiz et al.
31 and Mao et al.
27 had also used U-Net, segmented retinal vascular, and OD for the studies of plus disease in retinopathy of prematurity.
Unlike them, we used the whole images when training the networks and kept the size consistent with the original image (1600 × 1200 pixels) throughout the whole process. This was advantageous for us to perform operations such as fusion, inversion, and so on. A similar study by Kim et al.
28 showed that retinal appearance assessment based on the whole image provided more accurate and reliable performance compared to quadrant-based assessment. Of course, a direct comparison of the performance was not appropriate because of the differences in data sets and tasks.
Many reports described abnormalities of retinal vascular in patients with mild FEVR.
3,6,32 Therefore, we designed the system to automatically extract the vascular features (densities, mean tortuosities, and maximum diameter ratios) within the ROI. All three parameters showed significant differences between infants with mild FEVR and normal. These quantitative features could be used as supporting evidence to assist physicians in the clinical diagnosis of mild FEVR in newborn screening.
In a previous study, Yuan et al.
6 found that for patients with mild FEVR, more vascular radiated from the OD within two times the OD diameter (24.53 ± 3.1 vs. 21.39 ± 2.65), and our study showed a significant increase in vessel densities within the ROI (three times the OD diameter): 5.3211% ± 0.7600% vs. 4.5998% ± 0.6586%.
In OMIM 133780 (Online Mendelian Inheritance in Man;
https://www.ncbi.nlm.nih.gov/omim provided in the public domain by the National Center for Biotechnology Information, Bethesda, MD, USA), abnormalities of stretched vascular in patients with mild FEVR were presented. In contrast, we found a significant reduction in mean vascular tortuosities within the ROI (2.1018 ± 0.2933 [10
4 cm
−3] vs. 3.3344 ± 0.3890 [10
4 cm
−3]). In addition to this, Kashani et al.
32 reported that patients with mild FEVR had dilated veins due to hypoxia, and the maximum vascular diameter ratios within the ROI were significantly increased in our study (1.8805 ± 0.3197 vs. 1.5087 ± 0.2877).
To our knowledge, this was the first time that automatically extracted quantitative vascular features (densities, mean tortuosities, and maximum diameter) had been applied to analyze abnormalities in full-term infants with mild FEVR.
After identifying the significant differences in vascular features in the comparison data set, we developed a RF classifier that achieved the best performance including a combination of all three features. In our model, the mean tortuosity of the vascular was the most important single feature.
At the same time, the results of our model using random forest (κ value of 0.775 with the manual reading results) also suggested that we needed more related features to improve the performance. Going further to investigate the features learned by DCNNs will be our next task.
FA and genetic diagnosis are very important to confirm the diagnosis of FEVR.
32 Performing FA in neonates requires a high threshold of collaboration between the anesthesiology and neonatology departments, pediatric angiography facilities, and infant anesthesia departments. However, many hospitals performing universal neonatal screening at the primary level do not have such a differential diagnosis. Comprehensive fundus examinations of neonates using a wide-field fundus imaging system (Ret Cam) had been reported to detect neonatal RH, FEVR, and other neonatal ocular abnormalities.
14–16 We found it interesting to study the vascular abnormalities in full-term infants with mild FEVR from images (acquired by Ret Cam III).
The authors thank Liu Jia of Jiaxing Maternal and Child Health Hospital, who provided the retinal images of preterm infants used for this study. We also thank doctors in ophthalmology who provided much help, including manual diagnosis.
Supported by the Jiaxing Science and Technology Bureau project “Retinal Vascular Quantitative Analysis Tool for Retina Images Based on Deep Convolutional Neural Networks” (2022AY10011).
Disclosure: P. Li, None; J. Liu, None