Manual Versus Semi-Automated Measurement of Geographic Atrophy Area in Eyes With Age-Related Macular Degeneration

Abstract

Purpose: To investigate the agreement between and correlation of manual and semi-automated area measurements of geographic atrophy (GA) in eyes with age-related macular degeneration (AMD) using Heidelberg Eye Explorer and ImageJ software.

Methods: Fundus autofluorescence (FAF) images of eyes with GA secondary to AMD were analyzed. Two graders measured the atrophic area using Heidelberg Eye Explorer manual and semi-automated (RegionFinder) software, as well as ImageJ manual and semi-automated (Color Threshold) software.

Results: Fifty-four FAF images were analyzed. The mean (SD) areas were 10.55 (11.4) mm² and 9.6 (9.8) mm² using the Heidelberg manual and semi-automated tools, respectively. The mean (SD) areas were 11.04 (12.25) mm² and 9.75 (10.3) mm² using ImageJ manual and semi-automated tools, respectively. Compared with the semi-automated Heidelberg RegionFinder (gold standard) area measurements, Bland–Altman plots showed mean differences of 0.96 mm², 1.4 mm², and 0.16 mm² with manual Heidelberg, manual ImageJ, and semi-automated ImageJ measurements, respectively. Homogeneous GA lesions showed less disparity in area measurements across modalities compared with non-homogeneous lesions.

Conclusions: ImageJ appears to be a reliable tool for GA area measurements when proprietary OCT software is unavailable. Manual measurements with Heidelberg Eye Explorer and ImageJ were comparable, as were semi-automated measurements with Heidelberg RegionFinder and ImageJ Color Threshold.

Translational Relevance: Novel GA measurement techniques using open-source software appear to be comparable to established techniques using proprietary platform-specific software, which may permit more widespread analysis of GA progression from multiple platforms and databases.

Introduction

Geographic atrophy (GA) is a late-stage sequela of non-exudative age-related macular degeneration (AMD). It is characterized by localized loss of the outer retina, the retinal pigment epithelium (RPE), and choriocapillaris.¹ Unlike exudative AMD, which is often characterized by an acute decrease in vision, GA progression and associated visual loss are generally considered a gradual process. GA typically starts as a perifoveal or extrafoveal lesion that may be asymptomatic or associated with only minor vision changes. Over time, it may progress into the foveal center, leading to a severe, irreversible decline in vision with a central scotoma.² Currently, there is no US Food and Drug Administration–approved treatment to prevent GA formation or reduce GA progression, although multiple clinical trials are currently underway. Recent randomized phase 2/phase 3 clinical trials have shown promise in reducing the mean rate of GA growth and have included studies assessing avacincaptad pegol (Zimura; IVERIC bio, Inc., New York, NY), a C5 inhibitor, and pegcetacoplan (APL-2; Apellis Pharmaceuticals, Waltham, MA), a selective C3 inhibitor.^3,4 

Currently, fundus autofluorescence (FAF) is the preferred method for diagnosing and monitoring GA progression. FAF images detect the autofluorescent signal emitted from lipofuscin pigment associated with RPE cells. With atrophy of lipofuscin-containing RPE cells, distinct dark (hypoautofluorescent) areas become apparent on FAF images.⁵ Currently, measurements of GA area based on FAF images can be achieved using manual or semi-automated methods.^6–8 The semi-automated methods have demonstrated superior interobserver and intraobserver agreements and are less time consuming than manual measurements.⁶ In 2011, Schmitz-Valckenberg et al.⁹ introduced a semi-automated method using Heidelberg Eye Explorer (Heidelberg Engineering, Heidelberg, Germany) named RegionFinder. The proprietary software detects the decrease in FAF signal intensity compared to the adjacent normal area using a region-growing algorithm, delineating the GA area. Since then, RegionFinder has been used in research studies and clinical settings as a reliable and reproducible GA measurement method.^5,10–12 

ImageJ (National Institutes of Health, Bethesda, MD) is an open-source image analysis software with several tools measuring length and area in different images.¹³ ImageJ has been used in ophthalmology for foveal avascular zone measurements using OCT angiography images.^14,15 The Comparison of Age-Related Macular Degeneration Treatment Trial used ImageJ to manually measure GA area on fluorescein angiography images.¹⁶ However, we are not aware of any studies using the ImageJ software package to measure GA area on FAF images. The purpose of this study was to compare the area of GA measured by the semi-automated Heidelberg RegionFinder software with the manual Heidelberg Eye Explorer tool, as well as a manual and novel semi-automated methods using ImageJ software. 

Methods

Patient Selection

The Institutional Review Board of Wills Eye Hospital approved the study protocol. The study was performed in accordance with the tenets of the Declaration of Helsinki and complied with the Health Insurance Portability and Accountability Act. International Classification of Disease 10th Revision codes H35.3113, H35.3114, H35.3123, H35.3124, H35.3133, and H35.3134 were used to generate a list of subjects diagnosed with GA having FAF images at Wills Eye Hospital from January 2017 to January 2019. Consecutive patients were reviewed for inclusion in this study until an adequate sample size had been obtained. Patients were excluded if there was any history of choroidal neovascularization in either eye or if the FAF image quality was deemed to be poor. Moreover, GA lesions continuous with the peripapillary atrophy or those that extended beyond the boundaries of the FAF image were excluded. 

Image Acquisition

FAF images were obtained with a SPECTRALIS HRA+OCT (Heidelberg Engineering) and analyzed using the proprietary Heidelberg Eye Explorer software (version 1.10.4). The FAF images had an excitation of 488 nm and an emission of 500 to 700 nm. The field of view was 30° × 30° with the image centered on the fovea and a resolution of 768 × 768 pixels. Each scan was averaged using the automatic retinal tracking mode of the device set to 20. 

GA Measurements

Two independent graders (MS and RM) evaluated all images. Four different methods were used to measure GA: two manual and two semi-automated. Measurements were performed in two phases. In the first phase, the two manual techniques were performed. Using the FAF images, each grader manually outlined the GA area using the region overlay tool built into the Heidelberg Eye Explorer software. These images were then saved in Joint Photographic Experts Group (JPEG) format and transferred to ImageJ 1.53a. The boundaries of the lesion already drawn in the FAF images using the region overlay tool were used as a template for manual measurements in ImageJ. The 200-µm scale accompanying the FAF images was used to standardize distances. Each grader did the manual measurement on two separate days. When using ImageJ, the graders were masked to the numerical value of the GA area that had been measured with the manual Heidelberg technique. 

In the second phase, two semi-automated measurements were performed. FAF images were opened in RegionFinder 2.6.4, which was then used by each grader to perform semi-automated GA measurements based on a previously described method.⁹ The software recognizes the reduced FAF signal in the GA area as represented by darker regions compared with adjacent normal regions that have a brighter FAF signal. The user manually defines a point within the GA lesion which the software highlights in color based on the signal intensity. All pixels adjacent to that point with a signal intensity below the threshold of the user-defined point are also highlighted. Using a region growing algorithm, the user manually adjusts the threshold until the highlighted area matches the area of the GA. For images with multiple GA lesions, graders repeated the same selection and growing steps to cover each area. The final number reported is the sum of each measured GA area. To obtain more precise results, graders utilized the “block” tool embedded in the RegionFinder software. Because blood vessels and the optic disc/peripapillary atrophy exhibit contrast relatively similar to that of GA in FAF images, expanding algorithms might incorporate the blood vessels and/or disc in the measurements, thereby misrepresenting the results. With the block tool, blood vessels around the GA and the optic nerve/peripapillary area can be excluded by drawing lines to separate them from the GA area. For small, fovea-sparing lesions, a block circle was used to exclude the fovea. 

A novel contrast-based, semi-automated method was assessed to measure the GA area using ImageJ. Previously saved JPEG images were used for this step. FAF images were uploaded into the ImageJ software individually. Similar to the manual measurements with ImageJ, the embedded 200-µm scale was essential for correct measurement calibration. First, the approximate GA area was cropped so as to exclude extraneous vessels and the peripapillary area using the “clear outside” tab. Next, the “color threshold” tab and “brightness” section were employed to detect the GA area in a growing pattern, similar to RegionFinder. Figure 1 shows how the GA lesion was isolated in ImageJ, analogous to the block tool in RegionFinder. After sufficient saturation of the hypoautofluorescent area, the “measure” tab was used to quantify the GA area. Semi-automated measurements with ImageJ were performed on a different day than the RegionFinder measurements, with the graders masked to the RegionFinder measurements. Lesions were classified as homogeneous GA (homogeneous hypoautofluorescence within the lesion) and non-homogeneous GA (mixed hypo- and hyperautofluorescence within the lesion) for sub-analysis. 

Figure 1.

 

The semi-automated method using ImageJ. (A) Original Heidelberg fundus autofluorescence image uploaded to ImageJ software. (B) The approximate borders were drawn around the lesion to exclude other adjacent hypoautofluorescent elements such as vessels and the optic nerve/peripapillary area. The “clear outside” option in ImageJ was used to clear the area outside the desired lesion. (C) Using “color threshold,” the area of geographic atrophy was determined. The color white was chosen to highlight the hypoautofluorescent areas while sparing the hyperautofluorescent area inside the non-homogeneous lesion.

Statistical Analysis

SPSS Statistics 24 (IBM, Armonk, NY) was used to perform the statistical analysis and illustrate the Bland–Altman plots. Agreement between the two graders was assessed using intraclass correlation coefficients (ICCs) for each of the four methods. The mean value of the measured GA area for each of the different methods was calculated for each grader. Bland–Altman plots were used to compare the various methods. For each technique, the mean and standard deviation (SD) were calculated for the GA area measured with each of the methods, as well as differences in GA areas among each of the methods. Pearson’s correlation was used to assess the correlation between the mean difference in GA area with each method and the lesion size. Mann–Whitney test was used to compare the difference in GA area measured with RegionFinder and the other methods in homogeneous and non-homogeneous GA. P < 0.05 was considered to be statistically significant. 

Results

Fifty-four eyes of 40 patients were included in this study. Mean (SD) age was 82 (8.2) years (range, 65–99), and 25 patients were female (62.5%). The right eye was involved in 31 eyes (57.4%). The mean GA area measurements performed by the two graders and the ICCs for each of the four methods are provided in Table 1. Excellent agreement between graders was obtained for each technique. The mean (SD) areas were 10.55 (11.4) mm² and 9.6 (9.8) mm² using the Heidelberg manual and semi-automated measuring methods, respectively. The mean (SD) areas were 11.04 (12.25) mm² and 9.75 (10.3) mm² using the ImageJ manual and semi-automated methods, respectively. Table 2 shows the difference between various measurement methods based on homogeneous (n = 21, 38.9%) versus non-homogeneous (n = 33, 61.1%) GA lesions. Figures 2 and 3 demonstrate the four methods of GA measurement in a non-homogeneous and homogeneous lesion, respectively. 

Table 1.

 

Correlation of Measurements for Each of the Four Methods Tested Between Two Independent Graders

 

Table 2.

 

Mean Difference Between Heidelberg RegionFinder as the Gold Standard Method With Different Measurement Methods in Homogeneous and Non-Homogeneous Lesions

 

Figure 2.

 

A non-homogeneous geographic atrophy lesion seen on fundus autofluorescence with multiple hyperautofluorescent areas within the hypoautofluorescent area was measured using the semi-automated Heidelberg RegionFinder (gold standard), manual Heidelberg region overlay tool, manual ImageJ, and semi-automated ImageJ tool. (A) RegionFinder was used to measure the hypoautofluorescent area inside the lesion. The blocking lines were used to limit the lesion and exclude the vessels. The size of the measured area is 5.85 mm². (B) The manual Heidelberg Eye Explorer region overlay tool was used to draw the borders of the lesion but could not spare the hyperautofluorescent signal within the lesion. The size of the measured area is 6.94 mm². (C) Semi-automated ImageJ was used to measure the hypoautofluorescent area. The white area is representative of the hypoautofluorescent area inside the GA. The size of the measured area is 5.83 mm². (D) The manual ImageJ tool was used to draw the borders of the lesion but could not spare the hyperautofluorescent signal within the lesion. The total size of the measured area is 7.49 mm², as seen in the adjacent measurement box.

Figure 3.

 

A homogeneous lesion was measured by RegionFinder (gold standard), the manual Heidelberg region overlay tool, manual ImageJ, and the semi-automated ImageJ tool. (A) RegionFinder was used to measure the hypoautofluorescent area inside the lesion. The blocking lines were used to limit the lesion and exclude the vessels. The size of the measured area is 6.91 mm². (B) The manual Heidelberg Eye Explorer region overlay tool was used to draw the borders of the lesion. The size of the measured area is 6.90 mm² on the image. (C) The semi-automated ImageJ tool is used to measure the hypoautofluorescent area. The size of the measured area is 6.95 mm², as seen in the adjacent measurement box. (D) The manual ImageJ tool was used to draw the borders of the lesion. The size of the measured area is 7.06 mm², as seen in the adjacent measurement box.

Bland–Altman plots were used to demonstrate the level of agreement between the four measurement methods (Figs. 4 and 5). Compared with semi-automated Heidelberg RegionFinder measurements (gold standard), Bland–Altman plots showed mean differences of 0.96 mm² with manual Heidelberg, 1.4 mm² with manual ImageJ, and 0.16 mm² with semi-automated ImageJ measurements, respectively. All methods showed a close agreement with RegionFinder values. 

Figure 4.

 

Bland–Altman plots show the agreement between different measurement methods with semi-automated Heidelberg RegionFinder (gold standard). (A) Bland–Altman plot shows a mean difference of 0.96 mm² between manual Heidelberg and Heidelberg RegionFinder. (B) Bland–Altman plot shows a mean difference of 0.16 mm² between semi-automated ImageJ and Heidelberg RegionFinder. (C) Bland–Altman plot shows a mean difference of 1.4 mm² between manual ImageJ and Heidelberg RegionFinder.

Figure 5.

 

Bland–Altman plots show the agreement between two manual methods with each other and each manual method with semi-automated ImageJ. (A) Bland–Altman plot shows a mean difference of 0.48 mm² between manual Heidelberg and manual ImageJ. (B) Bland–Altman plot shows a mean difference of 1.3 mm² between semi-automatic ImageJ and manual ImageJ. (C) Bland–Altman plot shows a mean difference of 0.80 mm² between manual Heidelberg and semi-automated ImageJ.

Comparing manual ImageJ with semi-automated ImageJ, the Bland–Altman plot demonstrated a mean difference in GA area measurement of 1.3 mm². Comparing the manual Heidelberg region overlay tool and semi-automated ImageJ method, the mean difference in GA area measurement was 0.80 mm². The manual Heidelberg and manual ImageJ methods showed a mean difference in GA area measurement of 0.48 mm². Pearson’s correlation showed that an increase in GA area was strongly correlated with an increase in the mean difference between Heidelberg RegionFinder and each of the manual methods (r = 0.80, P < 0.001 for Heidelberg manual; r = 0.84, P < 0.001 for ImageJ manual). However, the mean difference between the two semi-automated methods (Heidelberg vs. ImageJ) was only moderately correlated with an increase in GA area (r = 0.60, P < 0.001). 

Discussion

This study demonstrates that semi-automated area measurements using public-domain ImageJ software can produce results comparable to the proprietary Heidelberg RegionFinder software. The same pattern of agreement was observed when comparing manual measurements made with ImageJ and those made with the Heidelberg Eye Explorer region overlay tool. A high degree of interobserver agreement was also found within each of the four methods tested. Similar to prior studies,^6,17 we found semi-automated methods to be more accurate and reproducible than manual outlining methods in detecting and monitoring GA progression, particularly when measuring non-homogeneous GA lesions. 

Panthier et al.¹¹ showed that, with increasing GA size, the variability of measurements even with RegionFinder increases. They showed that excellent intraobserver agreement could be achieved for lesions with size up to 15.75 mm². With larger lesions, small changes in the semi-automated algorithm may lead to a greater rise in the measured area of GA. In similar fashion, the difference in area measurements when comparing RegionFinder values to each of the manual techniques was strongly correlated with increasing GA lesion size in this study. However, when comparing RegionFinder to semi-automated ImageJ, the difference in area measurements was only moderately correlated with increasing GA lesion size, suggesting that the semi-automated methods may more closely match even as the lesion size increases. Area measurements outside the agreement limits in the Bland–Altman plots were typically related to lesions larger than 17.7 mm². The smallest overall differences occurred when comparing Heidelberg RegionFinder with semi-automated ImageJ (0.16 mm²), followed by comparing manual Heidelberg with manual ImageJ (0.48 mm²). It was also noteworthy that the mean difference between the two semi-automated methods was only moderately correlated with increasing GA size (r = 0.60, P < 0.001), whereas the mean difference between RegionFinder and each of the two manual methods was more strongly correlated with GA lesion size (r = 0.80, P < 0.001 for Heidelberg manual; r = 0.84, P < 0.001 for ImageJ manual). This further suggests that, as GA size increases, semi-automated ImageJ measurements were more accurate than manual methods. 

In this study, we examined the effect of homogeneous lesions versus non-homogeneous lesions on different measurement methods. Previous studies by Holz et al.⁸ have defined different patterns of GA based on abnormalities in the junctional zone. They concluded that different FAF patterns have an impact on disease progression and can be considered as prognostic factors. Unlike the previously described junctional patterns, we defined homogeneous lesions as those that are more uniformly hypoautofluorescent compared with non-homogeneous lesions, which were defined as those that had mixed areas of hyper- and hypoautofluorescence within the lesion itself rather than focusing only on the borders. Area measurements using each of the manual methods were in close agreement with RegionFinder for homogeneous lesions but yielded higher values for non-homogeneous lesions. In contrast, the difference in area measurements was not significant when comparing RegionFinder with semi-automated ImageJ for both homogeneous and non-homogeneous lesions. Because both semi-automated methods use a similar detection algorithm that is based on pixel intensity, they are able to measure only the hypoautofluorescent areas even within non-homogeneous lesions compared with the manual methods, which demarcate the boundaries of the lesion and therefore may incorporate some relatively more hyperautofluorescent areas within it. These differences in measurement technique likely explain the discrepancies in area measurements seen between the manual and semi-automated methods, particularly for non-homogeneous lesions, and suggest that the manual methods may not be as reliable for accurate measurement of GA area. 

The primary benefits of ImageJ are that it is platform agnostic and can be used to process many popular image file types, including JPEG, Portable Network Graphics (PNG), and Digital Imaging and Communications in Medicine (DICOM). In contrast, platform-based proprietary software (e.g., Heidelberg Eye Explorer, RegionFinder) typically requires a license agreement and the availability of specially formatted image files (e.g., E2E with Heidelberg). In addition, ImageJ is compatible with all popular computer operating systems, such as Windows (Microsoft Corporation, Redmond, WA), macOS (Apple, Inc., Cupertino, CA), and Linux (Linux Foundation, San Francisco, CA). Based on the present study, it appears to be comparable to the proprietary software for measuring the area of GA in FAF images. However, ImageJ could be more efficient if compiling images from multiple sites and/or multiple platforms, as comparatively smaller sized non-proprietary files can be imported into the software. 

This study has several limitations. The agreement of different measurement methods in detecting the progression of GA lesions over time was not investigated in this study. Therefore, we could not evaluate how lesion growth might vary with each of those methods. Similarly, we did not explore the prognostic value of homogeneous versus non-homogeneous lesions in predicting the progression of GA. Finally, the sample size was relatively small, and further studies with a larger sample size and also more diverse lesion sizes will be necessary to better elucidate the agreement level of different measurement methods. 

In summary, this study demonstrated that manual and semi-automated public domain software tools using ImageJ could be used to measure the area of GA lesions accurately when compared with corresponding methods using proprietary software tools. Given the current multitude of clinical trials focusing on developing new therapies for GA and the possibility that treatments to delay the progression of GA may soon become available, the ability to measure lesion size and monitor progression will become increasingly important. Adapting open-source software to analyze large image databases, such as those being collected by the American Academy of Ophthalmology's Intelligent Research in Sight (IRIS) registry, will be important to understanding disease progression and risk factors across large numbers of patients. 

Acknowledgments

Disclosure: R. Mahmoudzadeh, None; M. Salabati, None; M.A. Khan, None; S.J. Garg, None, J. Hsu, None