Improved Automated Foveal Avascular Zone Measurement in Cirrus Optical Coherence Tomography Angiography Using the Level Sets Macro

Abstract

Purpose: To evaluate automated measurements of the foveal avascular zone (FAZ) using the Level Sets macro (LSM) in ImageJ as compared with the Cirrus optical coherence tomography angiography (OCTA) inbuilt algorithm and the Kanno–Saitama macro (KSM).

Methods: The eyes of healthy volunteers were scanned four times consecutively on the Zeiss Cirrus HD-OCT 5000 system. The FAZ metrics (area, perimeter, and circularity) were measured manually and automatically by the Cirrus inbuilt algorithm, the KSM, and the LSM. The accuracy and repeatability of all methods and agreement between automated and manual methods were evaluated.

Results: The LSM segmented the FAZ with an average Dice coefficient of 0.9243. Compared with the KSM and the Cirrus inbuilt algorithm, the LSM outperformed them by 0.02 and 0.19, respectively, for Dice coefficients. Both the LSM (intraclass correlation coefficient [ICC] = 0.908; coefficient of variation [CoV] = 9.664%) and manual methods (ICC ≥ 0.921, CoV ≤ 8.727%) showed excellent repeatability for the FAZ area, whereas the other methods presented moderate to good repeatability (ICC ≤ 0.789, CoV ≥ 15.788%). Agreement with manual FAZ area measurement was excellent for both the LSM and KSM but not for the Cirrus inbuilt algorithm (LSM, ICC = 0.930; KSM, ICC = 0.928; Cirrus, ICC = 0.254).

Conclusions: The LSM exhibited greater accuracy and reliability compared to the KSM and inbuilt automated methods and may be an improved and accessible option for automated FAZ segmentation.

Translational Relevance: The LSM may be a suitable automated and customizable tool for FAZ quantification of Cirrus HD-OCT 5000 images, providing results comparable to those for manual measurement.

Introduction

The foveal vascular network is composed of interconnected capillaries perfusing the inner retinal layer. This network forms a ring at the margin of the fovea and produces a capillary-free region referred to as the fovea avascular zone (FAZ).¹ The size and shape of FAZ can indicate the ischemic severity and the capillary dropout of the macula, which are associated closely with visual prognosis.² Therefore, quantification of FAZ metrics is essential for the diagnosis and follow-up of retinal vascular diseases, such as diabetic retinopathy and retinal vein occlusion.^3–5 

Optical coherence tomography angiography (OCTA) is a fast and non-invasive imaging technique that can provide three-dimensional maps of the retinal and choroidal microvasculature.^6,7 Furthermore, the acquisition of high-resolution images allows quantifying the vasculature and providing information useful for diagnosing and following up on retinal diseases.^8–10 

The FAZ has been traditionally measured via a manual method that requires observers to plot all endpoints of vascular signals at the boundary of the FAZ.¹¹ It has been reported that the manual method has demonstrated excellent repeatability and reproducibility in various OCTA systems.^12–17 However, manually capturing the contour of the FAZ is time consuming and labor intensive, especially when thousands of images must be analyzed. The development of automated algorithms would reduce subjective bias to some extent, and they would provide convenient and quick approaches to image analysis. Some commercial OCTA systems have been equipped with automated algorithms for FAZ metrics, presenting results comparable to those of manual measurements.¹⁸ However, it has been reported that the reliability of the automated embedded algorithm for the FAZ metrics on the Cirrus HD-OCT 5000 (Carl Zeiss Meditec, Dublin, CA) is low.¹⁹ Some automated customized methods have been reported for other OCTA systems. Ishii et al.²⁰ reported that the Kanno–Saitama macro (KSM) produced reliable results in the Zeiss PLEX Elite 9000; however, the reliability of the macro in the Cirrus HD-OCT 5000 has not yet been determined, and other image processing methods should be explored for more reliable measurements in the Cirrus HD-OCT 5000. In this study, we utilized the Level Sets macro (LSM), a plugin for ImageJ software (National Institutes of Health, Bethesda, MD) to measure the FAZ, and we compared the results with measurements by the Cirrus inbuilt algorithm and the KSM. 

Methods

Study Subjects

This cross-sectional study was conducted at the Joint Shantou International Eye Center of Shantou University and The Chinese University of Hong Kong with the permission of the Institutional Review Board. All investigations followed the tenets of the Declaration of Helsinki. Each subject signed an informed consent before enrollment. The recruited volunteers were older than 18 years of age and had no evidence of retinal diseases or ocular media opacity. Their best-corrected visual acuity was at least 20/20 using the Snellen chart. Also, the refractive error was within ± 6 diopters (D) with intraocular pressure less than 21 mm Hg. 

Although the LSM method was not learning based, a training dataset and a test dataset were used to optimize the parameters and evaluate the performance, respectively. We randomly enrolled 20 subjects for the training dataset. The sample size required for the test dataset was calculated as discussed below. In the repeatability analysis, there was an assumption that 95% confidence intervals (CIs) of within-subject standard deviations (S_w) were estimated within 15% of S_w, \(1.96{\rm{ }} \times {S_w}/\sqrt {2n(m - 1)} = 15\% {\rm{ }} \times {S_w}\), n = (1.96/0.15)²/[2(m – 1)], where n and m represent the number of subjects and number of measurements, respectively. We measured each subject four times, and n was calculated to be 28.46.²¹ In the agreement analysis, we assumed that the discordance rate (a) was equal to 0.05 and the tolerance probability (β) was 80%. The sample size (n) was determined by the formula n ≥ log(1 – β)/log(1 – a), thus n ≥ 32.²² In this study, we enrolled 37 subjects in the test dataset. 

OCTA Imaging

The OCTA imaging was performed using the Cirrus HD-OCT 5000 with AngioPlex software. The Cirrus HD-OCT 5000 is a spectral-domain optical coherence tomography device with an increased scan rate of 68,000 A-scans per second.²³ The viewing software can automatically detect the tissue boundary for segmenting preset layers of interest at the posterior pole.¹⁵ One eye was randomly selected from each subject. After the pupil was dilated with tropicamide, the eye was scanned using the foveal 3 mm × 3 mm protocol by a single technician under the same conditions. Each eye was scanned consecutively four times in the testing group but only once in the training group. Eye tracking was applied using the Cirrus HD-OCT 5000 FastTrac retinal tracking technology. The cube images of vasculature were obtained using optical microangiography algorithm.²⁴ The en face images of the superficial retinal layer (SRL) were generated from the ganglion cell layer to the inner plexiform layer. Images with low image quality (lower than 6) were excluded. 

FAZ Segmentation and Quantification

We used four different methods to determine the FAZ metrics (area, perimeter, and circularity). The definitions of FAZ metrics in the automated methods were consistent with the manual method. The area was defined as the size of the segmented FAZ region, and the perimeter was determined by the length of the FAZ contour. The circularity was then calculated as an index using the formula where circularity = 4π(area/perimeter²), indicating the compactness of shape relative to a circle. A ratio closer to 0 indicates a more irregular shape far away from a circle.²⁵ In ImageJ, the measurement results were shown in pixel units. Because the images for the 3 mm × 3 mm protocol exported from the Cirrus HD-OCT 5000 were 719 × 719 pixels, the unit of pixel was converted to millimeters at a ratio of 719 to 3. For the Cirrus embedded algorithm, the results were provided in millimeters in the OCTA report. 

Manual Measurement

The exported en face SRL images were reordered in sequence to reduce the contextual bias and then analyzed by two masked observers. The observers imported all images into ImageJ software and outlined the contour of the FAZ using the ImageJ Freehand Selection Tool. After the FAZ was enclosed, the area, perimeter, and circularity were then calculated. The final manually obtained values were averages of the measurements of each observer. 

Level Sets Macro

The LSM is a plugin utilizing the theory of partial differential equations that can progressively compare pixel differences with neighboring pixels and converge at the boundary where the differences are the highest (available at https://imagej.net/Level_Sets). After importing the 8-bit grayscale image into ImageJ, inserting an oval to act as an initial seed inside the FAZ is required before running the program (Fig. 1a). The shape and the size of the initial seed are not particularly important, but it is essential that the seed be entirely inside the FAZ, preferably located at the center of the FAZ. Based on the optimized parameters, the contour advances and can be viewed in the progress window (Fig. 1b). When the contour hits the boundary, segmentation of the FAZ is finished (Fig. 1c). Finally, the LSM automatically measures and outputs the FAZ metrics (area, perimeter, and circularity). 

Figure 1.

 

Procedure for FAZ segmentation by the LSM. (a) An initial seed at the center of the FAZ is required. (b) After running the LSM, the contour advances and can be viewed in the processing window. (c) When it hits the boundary, the FAZ segmentation is finished.

The parameters of the LSM were optimized as follows. The Active Contours method was chosen for the LSM rather than Fast Marching because the latter method is prone to leaking, especially when there is a gap at the FAZ boundary. This program advances the contour like a rubber band, with the strength being controlled by the nature of the curvature. The convergence serves as the convergence criterion and compares the changes in contour between two iterations. The LSM segmented the FAZ under different settings of curvature and convergence in the training images, and the segmentation performance was evaluated by visual inspection and accuracy analyses, in which the manual segmentation served as the ground truth. The training results showed that curvature = 1.00 and convergence = 0.0100 provided the best segmentation results, with an average accuracy of 0.9937 and Dice coefficient of 0.9012, respectively (see Supplementary Figure S1 and Table S1). The various grayscale tolerance values presented similar segmentation in the clear boundary, but they differed in the blurred border due to the low signal strength. We trained various grayscales and found that the grayscale of 30 performed best (Dice coefficient, 0.9012) compared to the values of 10 (Dice coefficient, 0.2371) and 50 (Dice coefficient, 0.8911) (see Supplementary Figure S2). The detailed macro script can be found in the Supplementary Material. 

Kanno–Saitama Macro

The KSM is a new, automated approach for determining the FAZ with ImageJ that was proposed in a study by Ishii et al.²⁰ Because the en face images exported from the Cirrus HD-OCT 5000 were 719 × 719 pixels, there was no need to downsize the images. We followed the instructions in the Ishii et al.²⁰ study for the other procedures. After binarization and skeletonization, the images were processed by repeatedly dilating and eroding them so that the FAZ was extracted. Finally, the FAZ area, perimeter, and circularity were measured. 

Cirrus Inbuilt Algorithm

The Cirrus HD-OCT 5000 (version 10.0.0.14618) provides an automated inbuilt algorithm to measure the FAZ via the AngioPlex Metrix toolbox, which is only available when selecting the superficial preset. If the algorithm identifies a FAZ border from the en face SRL images, a yellow overlay is applied to the FAZ. The area, perimeter, and circularity of the FAZ are then shown in tabular form. 

Analysis of the Segmentation Performance

The accuracy (ACC), sensitivity (SEN), specificity (SPE),²⁶ and Dice coefficient²⁷ metrics were used to evaluate the performance of segmentation. The ground truth was defined as the manual segmentation performed by the first observer. The segmentation results of the second observer and the automated algorithms were compared with the ground truth. The metrics were calculated as follows:  

\begin{eqnarray*}{\rm{ACC}}&\, =& \frac{{TP + TN}}{{FN + {\rm{\;}}FP + TP + TN}},\,{\rm{SEN}} = \frac{{TP}}{{TP + FN}},\nonumber\\ {\rm{SPE}} &\,=& \frac{{TN}}{{TN + FP}},\,{\rm{Dice}} = \frac{{2TP}}{{2TP + FP + FN}}\end{eqnarray*}

where TP = true positive, TN = true negative, FP = false positive, and FN = false negative. The Dice coefficient has been reported to be the most commonly used performance metrics.²⁸ It is a more adequate indicator of intuitive object segmentation, reflecting both size and localization agreement rather than pixel-wise accuracy.²⁹ Dice coefficients for various image quality levels were calculated, as well. 

Statistical Analysis

In the repeatability analysis, the measurement results of four scans for each subject in all methods were evaluated by coefficient of variation (CoV) and intraclass correlation coefficient (ICC). The within-subject standard deviation (S_w) was calculated as the square root of the within-subject variance. CoV was calculated as (S_w/average of the measurements) × 100%, with values less than 10% indicating good repeatability.³⁰ Agreement between the first measurements of each subject by the various methods was analyzed using the paired t-test, linear agreement, and Bland–Altman plots. P < 0.05 was considered to be statistically significant. ICC was calculated with the single-measurement, absolute-agreement, two-way mixed-effects model in the repeatability and agreement analyses.¹⁹ The ICC values were classified as poor (ICC < 0.50), moderate (0.50 ≤ ICC < 0.75), good (0.75 ≤  ICC < 0.90), or excellent (ICC ≥ 0.90). All statistical analyses were performed using SPSS Statistics 19 (IBM, Armonk, NY) and GraphPad Prism 5.01 (GraphPad Software, San Diego, CA). 

Results

Thirty-seven eyes of 37 healthy subjects were included initially. Among them, three were excluded: One subject could not cooperate; for the second subject, measurement of the FAZ metrics by the Cirrus automated algorithm was not possible because the FAZ could not be identified; and the image quality was low for the third subject. Ultimately, 34 eyes (21 right eyes and 13 left eyes) of 34 volunteers (12 men and 22 women) were analyzed in our study, and 136 images were used for the test dataset. The mean ± SD of ages was 25.0 ± 6.3 years (range, 20–35), the mean ± SD of the image quality index was 8.44 ± 1.94 (range, 6 to 10), and the mean ± SD of spherical equivalent was –2.28 ± 4.03 D. 

Segmentation Performance

The segmentation performance of the LSM compared with that of the KSM, Cirrus inbuilt algorithm, and second observer is shown in Table 1. The manual segmentation by the second observer performed best in all segmentation methods, with the highest value of the accuracy (0.9947) and Dice coefficient (0.9308). The performance of the LSM was second best (accuracy, 0.9944; Dice coefficient, 0.9243). Compared to the KSM and the Cirrus inbuilt algorithm, the LSM outperformed them with Dice coefficients of 0.02 and 0.19, respectively. 

Table 1.

 

Segmentation Performance Comparisons of the Manual and Automated Methods

 

The Dice coefficients of the Cirrus algorithm and KSM improved with increased image quality. In contrast, the Dice coefficients of the second observer and LSM were comparable and remained stable at different image quality levels (Fig. 2). For images with low image quality (6 to 8), the LSM acquired the best segmentation result among the automated methods, the KSM was second to the LSM, and the Cirrus algorithm showed the lowest Dice coefficient. For images with good image quality (9 to 10), both the LSM and KSM exhibited better than the Cirrus algorithm. 

Figure 2.

 

Comparison of average Dice coefficient values for various image quality levels.

Repeatability of FAZ Metrics Measurement

Figure 3 shows representative OCTA images in which the FAZ was segmented by the various methods. FAZ segmentation by the LSM was more comparable to the manual measurements than the other two automated methods. The Cirrus inbuilt algorithm obviously segmented the FAZ erroneously. Table 2 shows the results of measuring the FAZ area by the various methods. The mean ± SD of the FAZ areas measured by one observer was 0.322 ± 0.101 mm²; for the other observer, it was 0.337 ± 0.103 mm². FAZ areas measured manually (0.329 ± 0.101mm²) were smaller than those measured by the LSM and KSM (0.346 ± 0.109 mm² and 0.353 ± 0.120 mm², respectively) but larger than those measured by the Cirrus algorithm (0.257 ± 0.112 mm²). Both the LSM (ICC, 0.908; CoV, 9.664%) and manual methods (ICC, 0.963 and 0.921; CoV, 6.109% and 8.727% for the two observers, respectively) had excellent repeatability; the Cirrus algorithm (ICC, 0.603; CoV, 27.798%) and the KSM (ICC, 0.789; CoV, 15.788%) achieved moderate to good repeatability. 

Figure 3.

 

Segmentation and quantitative measurements of the foveal avascular zone by various methods.

Table 2.

 

Repeatability of FAZ Metrics Measurement by Various Methods

 

A Table 2 shows, with regard to the FAZ perimeter the manual methods demonstrated excellent repeatability (ICC, 0.935; CoV, 4.023%), and the LSM had good repeatability (ICC, 0.802; CoV, 8.946%), but the other automated algorithms showed poor repeatability (ICC, ≤0.412; CoV, ≥21.186%). For FAZ circularity, the repeatability was moderate for the manual methods (ICC, 0.687; CoV, 4.222%) but poor for all automated methods (ICC, ≤0.401; CoV, ≥10.256%). 

Agreement of FAZ Metrics Measurement

Table 3 and Supplementary Figure S3 show the interobserver and automated/manual agreement for FAZ area. Although there was a statistically significant difference between each paired comparison (P < 0.05), agreement was generally excellent (ICC, 0.928–0.933), except for the Cirrus inbuilt algorithm, which had poor agreement with the manual method (ICC, 0.254). Bland–Altman plots showed that agreement ranged from –0.046 to 0.087 mm² for the LSM, from –0.044 to 0.089 mm² for the KSM, and from –0.355 to 0.158 mm² for the Cirrus inbuilt algorithm. 

Table 3.

 

Agreement of FAZ Metrics Measurements by the Various Methods

 

Interobserver agreement was excellent for the FAZ perimeter (ICC, 0.907, P = 0.389) and moderate for the circularity (ICC, 0.676; P = 0.134). All of the automated algorithms had poor agreement for the FAZ perimeter and circularity (ICC, ≤0.452) (Table 3; Supplementary Figs. S4, S5). 

Discussion

In the current study, we evaluated the feasibility of utilizing the LSM to determine FAZ metrics on OCTA images obtained with the Cirrus HD-OCT 5000 in healthy subjects. The LSM demonstrated excellent repeatability and agreement with the manual measurements for the FAZ area. The LSM showed much higher Dice coefficients compared to the Cirrus inbuilt algorithm, but it performed better compared to the KSM when image quality was poor. 

The reliability of automated embedded algorithms for FAZ metrics that are available with commercial OCTA systems has been investigated, including the Cirrus HD-OCT 5000 system. Linderman et al.³¹ reported better performance for manual segmentation (ICC, 0.994) compared to semiautomatic (ICC, 0.969) or automatic (ICC, 0.948) embedded segmentation for AngioVue OCTA. We have previously reported low reliability of the Cirrus inbuilt algorithm, which incorrectly outlined the border of the FAZ in 22.9% of cases.¹⁹ In the study by Enders et al.,³² Bland–Altman plots showed significant differences between manual measurement and the AngioPlex 9.5 module in the Cirrus HD-OCT 5000. Lee et al.³⁰ reported that the repeatability of automated FAZ measurements in the Cirrus HD-OCT 5000 (version 10.0) was disappointing (ICC, <0.75; CoV, >10.0%). Shiihara et al.,¹⁶ however, reported excellent automated/manual agreement (ICC, 0.987), but they neither presented the Bland–Altman plots nor investigated repeatability. 

The inconsistent results on agreement may be due to varying image quality or signal strength used in the various studies. Lee et al.³⁰ suggested that signal strength is associated with the repeatability of OCTA measurements and should be considered in analyses of the FAZ. Our previous studies have demonstrated that FAZ area and vessel density measurements are affected by signal strength.^33,34 The current study found that the accuracy of the algorithms was affected by image quality, especially for the Cirrus inbuilt algorithm and the KSM: the poorer the image quality, the worse the accuracy. The LSM, though, performed well for both good and poor image quality. We also examined two examples with image quality of 8 (Fig. 3) and 10 (see Supplementary Fig. S6). The Cirrus inbuilt algorithm performed well (Supplementary Fig. S6), although it erroneously outlined the FAZ border as shown in Figure 3. 

It should be noted that we did not select only high-quality images for analysis. The mean image quality index of 8.44 (range, 6 to 10) could lead to unsatisfactory results and account for the low reliability of the Cirrus inbuilt algorithm. Furthermore, ICC values obtained by the various models differed, so each model should be used appropriately and described clearly.³⁵ 

Several customized automated methods have been investigated for various OCTA systems but few for the Cirrus HD-OCT 5000. Ishii et al.²⁰ investigated the KSM for the Zeiss PLEX Elite 9000 and reported satisfactory results for FAZ area, but the macro appeared unsuitable for the Cirrus HD-OCT 5000 because it demonstrated unsatisfactory repeatability. After binarization, the repeated dilation of vessel signals at the FAZ border can fill up the disruption of the capillary ring by the KSM; however, high signal noise located inside the FAZ near the boundary could be mistaken for a vessel signal. This may possibly explain why the KSM performed worse than the LSM, showing lower Dice coefficient values in images with poor image quality. Xu et al.³⁶ developed a graph theoretic-based algorithm, and the results showed excellent repeatability for the FAZ perimeter (ICC, 0.930) and circularity (ICC, 0.969) in two sequential scans; however, details regarding the parameters were not provided, and the algorithm cannot be validated with our dataset. 

The LSM does not use binarized images. It can process the initial images and detect the FAZ, possibly avoiding the errors due to pixel differences between the blood vessels and the background. Some disruptions in the FAZ boundary usually occur with the Cirrus HD-OCT 5000, leading to growing region leaks. The LSM can prevent the contour from leaking by adjusting the curvature. It seems contradictory that there was a statistically significant difference between the FAZ area measured by the LSM and by the manual methods, but the agreement was excellent. The key problem is that the paired t-test compares the overall mean values of a pair of samples, whereas the ICC is a ratio of the variance of interest over the sum of the variance of interest plus error.³⁵ Therefore, ICC is a better reflection of the agreement between two measurement methods and reliability. 

Manual measurement has been shown to be reliable in published reports,^37–39 but it is impossible for observers to capture every point of the boundary of the FAZ. Thus, manual segmentation usually appears smoother than the automated algorithms and much closer to a circle, as Figure 3 shows, resulting in a shorter perimeter and larger circularity, as well as poor manual/automated agreement, except for the FAZ area. 

The LSM is practical and has potential for being utilized with the Cirrus HD-OCT 5000 for determining FAZ metrics; however, our study does have some limitations. First, the LSM was evaluated only in the 3 mm × 3 mm scanning mode, and of the other modes require further investigation. Second, the reliability of this program for eyes with ocular diseases has not yet been evaluated. Third, the optimized parameters were only assessed in the Cirrus HD-OCT 5000; therefore, the reliability of this program when used with other OCTA systems has yet to be validated. 

In conclusion, the LSM exhibited better performance with improved accuracy and reliability compared to the KSM and Cirrus inbuilt algorithm, providing repeatable measurements of FAZ metrics on Cirrus HD-OCT 5000 images. The LSM method may prove to be a reliable and accessible alternative tool for automated FAZ quantification. 

Acknowledgments

Supported by the National Key R&D Program of China (2018YFA0701700), Department of Education of Guangdong Province (2020KZDZX1086), and Grant for Key Disciplinary Project of Clinical Medicine under the Guangdong (002-18119101). 

Disclosure: A. Lin (P); D. Fang, None; C. Li, None; C.Y. Cheung, None; H. Chen (P)