Open Access
Retina  |   July 2023
Inter-Rater Reliability of Proliferative Diabetic Retinopathy Assessment on Wide-Field OCT-Angiography and Fluorescein Angiography
Author Affiliations & Notes
  • Payal N. Shah
    Department of Vitreoretinal and Ocular Oncology, Sankara Eye Hospital, Kundalahalli Gate, Bangalore, India
  • Divyansh K. Mishra
    Department of Vitreoretinal and Ocular Oncology, Sankara Eye Hospital, Kundalahalli Gate, Bangalore, India
  • Peyman Falahat
    Department of Ophthalmology, University Hospital Bonn, Venusberg-Campus 1, Bonn, Germany
  • Lars Fischer
    Department of Ophthalmology, University Hospital Bonn, Venusberg-Campus 1, Bonn, Germany
  • Gabriela Guzman
    Department of Ophthalmology, University Hospital Bonn, Venusberg-Campus 1, Bonn, Germany
  • Jan H. Terheyden
    Department of Ophthalmology, University Hospital Bonn, Venusberg-Campus 1, Bonn, Germany
  • Frank G. Holz
    Department of Ophthalmology, University Hospital Bonn, Venusberg-Campus 1, Bonn, Germany
  • Tim U. Krohne
    Department of Ophthalmology, University Hospital Bonn, Venusberg-Campus 1, Bonn, Germany
    Department of Ophthalmology, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany
  • Robert P. Finger
    Department of Ophthalmology, University Hospital Bonn, Venusberg-Campus 1, Bonn, Germany
  • Maximilian W. M. Wintergerst
    Department of Ophthalmology, University Hospital Bonn, Venusberg-Campus 1, Bonn, Germany
Translational Vision Science & Technology July 2023, Vol.12, 13. doi:https://doi.org/10.1167/tvst.12.7.13
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Payal N. Shah, Divyansh K. Mishra, Peyman Falahat, Lars Fischer, Gabriela Guzman, Jan H. Terheyden, Frank G. Holz, Tim U. Krohne, Robert P. Finger, Maximilian W. M. Wintergerst; Inter-Rater Reliability of Proliferative Diabetic Retinopathy Assessment on Wide-Field OCT-Angiography and Fluorescein Angiography. Trans. Vis. Sci. Tech. 2023;12(7):13. https://doi.org/10.1167/tvst.12.7.13.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose: To assess inter-rater reliability in the detection of proliferative diabetic retinopathy (PDR) changes using wide-field optical coherence tomography angiography (WF-OCTA) versus fluorescein angiography (FA).

Methods: This retrospective, cross-sectional study included patients with severe nonproliferative and PDR. Images were acquired with 12 × 12 mm WF-OCTA and FA with a 55° lens. Images were cropped to represent the exact same field of view. Qualitative (detection of neovascularization at the disc [NVD] and elsewhere [NVE], enlarged foveal avascular zone [FAZ], vitreous hemorrhage [VH]) and quantitative analyses (FAZ area, horizontal, vertical, and maximum FAZ diameter) were performed by 2 masked graders using ImageJ. Inter-rater reliability was calculated using unweighted Cohen's kappa coefficient (κ) for qualitative analyses and intraclass correlation coefficients (ICC) for quantitative analyses.

Results: Twenty-three eyes of 17 patients were included. Inter-rater reliability was higher for FA than for WF-OCTA in qualitative analyses: κ values were 0.65 and 0.78 for detection of extended FAZ, 0.83 and 1.0 for NVD, 0.78 and 1.0 for NVE, and 0.19 and 1 for VH for WF-OCTA and FA, respectively. In contrast, inter-rater reliability was higher for WF-OCTA than for FA in the quantitative analyses: ICC values were 0.94 and 0.76 for FAZ size, 0.92 and 0.79 for horizontal FAZ diameter, 0.82 and 0.72 for vertical FAZ diameter, and 0.88 and 0.82 for maximum FAZ diameter on WF-OCTA and FA, respectively.

Conclusions: Inter-rater reliability of FA is superior to WF-OCTA for qualitative analyses whereas inter-rater reliability of WF-OCTA is superior to FA for quantitative analyses.

Translational Relevance: The study highlights the specific merits of both imaging modalities in terms of reliability. FA should be preferred for qualitative parameters, whereas WF-OCTA should be preferred for quantitative parameters.

Introduction
Diabetic retinopathy (DR) is the leading cause for visual impairment and blindness in the working population worldwide and prevalence is increasing.1,2 The role of various investigative modalities like optical coherence tomography (OCT) and fluorescein angiography (FA) in the management of DR is well established. Although OCT helps to identify macular edema, FA is primarily used to identify leaking microaneurysms, macular ischemia, and nonperfusion areas and to differentiate between intraretinal microvascular abnormalities and neovascularization elsewhere (NVE).3,4 Although FA is still considered the gold standard for the detection of severe nonproliferative diabetic retinopathy and proliferative diabetic retinopathy, it is an invasive imaging modality and, it is associated with a small risk of allergy to fluorescein dye.5 With the advent of optical coherence tomography angiography (OCTA), a non-invasive and high-resolution imaging modality for analysis of perfusion and axial stratification in different retinal and choroidal layers has become available.68 
Studies have shown that OCTA can be used as an alternative tool to FA in detection of DR changes, especially where a wide-field OCTA (WF-OCTA) can be obtained to identify non-perfusion and neovascularisation.912 Furthermore, quantitative DR-related biomarkers like a greater foveal avascular zone (FAZ) perimeter, increased FAZ circularity, and extended morphological classification of neovascularization have also been established by OCTA.1315 Some studies are even suggestive that OCTA can achieve higher sensitivities and specificities (96% and 79% sensitivity / 100% and 96% specificity for detection of non-perfusion area and neovascularization's, respectively) than FA for detection of DR-related biomarkers.16 
However, so far, studies comparing OCTA and FA applied relatively small OCTA scan-frames (e.g., 3 × 3 mm or 4.5 × 4.5 mm), and, compared to FA, these implicate a narrowed field of view. Consequently, possible vascular changes in the periphery relevant for therapeutic management can therefore not be detected, and the level of detail between the two imaging modalities cannot be directly compared.17 With the advent of swept-source OCTA, WF-OCTA has become available, allowing for wider fields of view (e.g., 12 × 12 mm), which is comparable to FA with ∼55°. Such wider fields of view are clinically relevant to assess proliferative DR (PDR) in terms of nonperfused areas and NVDs/NVEs. Several reports have been published to date on detection of PDR using WF-OCTA; however, so far, inter-rater reliability of PDR evaluation with WF-OCTA is unclear.11,12,18,19 Hence, we assessed the inter-rater reliability of WF-OCTA for qualitative and quantitative analyses of severe nonproliferative and proliferative DR and made a comparison with the reliability of FA. 
Methods
Subject Recruitment
This was a retrospective, cross-sectional case series at the University Hospital Bonn, Department of Ophthalmology, Germany. The study design adhered to the tenets of the Declaration of Helsinki. Ethical approval was obtained from the ethics committee of the University of Bonn (approval ID 548/20). Inclusion criteria for the study were patients with severe DR, defined as severe nonproliferative or proliferative DR who received both WF-OCTA and FA. Grading of DR was recorded from the clinical and FA records of the patient. Exclusion criteria were patients with prior vitrectomy and impaired image quality (artefacts/blurriness, e.g., due to motion, blinking, and projection artifacts). 
Image Acquisition
Wide-field 12 × 12 mm swept-source OCT angiography (PLEX Elite 9000, Carl Zeiss Meditec, Germany) and confocal scanning laser ophthalmoscope fluorescein angiography (Spectralis, Heidelberg Engineering, Germany) with a 55° lens were performed. We have used a 55° lens for FA to best match the field of view of both imaging modalities. 
Image Analysis
Four images from each eye from the same location were used: two WF-OCTA images and two FA images. The vitreoretinal interface (VRI) slab (top boundary = 300 µm above the internal limiting membrane [ILM], bottom boundary = 10 µm above the ILM) and a composite retinal slab (top boundary = internal limiting membrane, bottom boundary = 70 µm above Bruch’s membrane) of WF-OCTA combining the superficial and deep retinal plexus, and early and late frames of FA images (in total four images for each eye) were used for comparative analyses. An early-phase FA (20–30 seconds) and a late-frame FA (3–5 minutes) were selected for the analysis. 
FA images were superimposed on the WF-OCTA images using Keynote (Apple; performed by co-author DKM). The FA image was kept as background making it transparent and then superimposing the WF-OCTA image over it, the size of the disc was aligned to avoid errors of magnification / proportion, rotated until branching retinal blood vessels matched and aligned in both modalities, and FA images were cropped in a square format as in WF-OCTA (see Supplementary Fig. S1). Once both the images (FA and WF-OCTA) were matched, they were exported to ImageJ for further quantitative analysis. Qualitative (P.S. and M.W.) and quantitative (P.S. and P.F.) masked analysis of these images was performed by two graders, respectively. ImageJ version 1.51a (National Institutes of Health, Bethesda, MD, USA) was used for quantitative analyses.20 
For analyses of the inter-rater reliability of qualitative analyses, the detection of an extended FAZ, NVD, NVE, and vitreous hemorrhage on WF-OCTA and FA were compared. Identification of NVD and NVE was based on the VRI slabs of WF-OCTA images whereas the rest of the parameters were evaluated on the retinal composite slabs. NVs located on the optic disc or within one disc diameter from the margin were classified as NVD while the rest were classified as NVE. Detection and quantitative measurements of FAZ were performed on the early FA images. Comparison of early and late frames was done to identify NVD, NVE and VH. 
For analyses of the inter-rater reliability of quantitative analyses, the measurements of FAZ size and horizontal, vertical, and maximum FAZ diameter on WF-OCTA and FA were compared. FAZ was outlined using the ImageJ software by annotating the foveal avascular zone which was defined as the foveal and any directly connected non-perfused area. Detection and quantitative measurements of FAZ were performed on FA from the early frames. The integrated line tool from ImageJ software was used to calculate horizontal, vertical and maximal diameters. Using the integrated marking tool, FAZ was marked along the boundaries of nonperfused fovea and analyzed to measure the area on ImageJ. All analysis were performed on 27-inch full-high-definition (1920 × 1080 pixels) monitors with a 200% magnification. 
Statistical Analyses
Data was statistically analyzed using R version 4.0.5. Inter-rater reliability between two raters was calculated using the unweighted Cohen's kappa (κ) for the qualitative analysis (binary data) or the intraclass correlation coefficient (ICC) (model = “two-way,” type = “agreement,” unit = “single”) for the quantitative analysis. 
ICC is a measure of the reliability of a rating when several raters are involved. It calculates the proportion of total variance in the ratings that is due to the differences between the raters, and the proportion due to differences between the subjects being rated. The ICC value ranges from 0 to 1, with higher values indicating greater reliability.21 Cohen's kappa (κ) is a statistic used to measure inter-rater agreement for categorical items. It takes into account the agreement occurring by chance and returns a value between −1 and 1. A value of 1 indicates perfect agreement, a value of 0 indicates agreement no better than chance, and negative values indicate less agreement than chance. The interpretation of kappa values is based on the following thresholds: less than 0.20 is considered poor agreement, 0.21 to 0.40 is considered fair agreement, 0.41 to 0.60 is considered moderate agreement, 0.61 to 0.80 is considered substantial agreement, and greater than 0.80 is considered almost-perfect agreement.22,23 All subjects were rated by all graders, which means a two-way model ICC should be used because the design is fully crossed. Furthermore, it should be an average-measures unit ICC because there is an interest in the reliability of the mean ratings provided by all coders. The 95% confidence interval for ICC was calculated using the bootstrapping method. Based on the 95% confidence interval of the ICC estimate, values <0.5, between 0.5 and 0.75, between 0.75 and 0.9, and >0.90 indicate poor, moderate, good, and excellent reliability, respectively.24 
Results
Twenty-three eyes from 17 patients were included in the study. The characteristics of the cohort are summarized in Table 1 and an exemplary comparison of FA and OCTA images is illustrated in the Figure
Figure.
 
Comparison of fluorescein angiography and optical coherence tomography angiography. Exemplary early (first column) and late (second column) FAs and retinal (third column) and VRI WF-OCTA slabs (fourth column) for seven exemplary eyes. This figure illustrates the original images. Images were cropped to the same field of view for the analyses. Neovascularization at the disc are visible on FA and WF-OCTA in subject A and B (arrows). Artifacts in the VRI slab (red arrow, subject F) can suggest presents of neovascularization, but are not conclusive, which can be a reason for decreased inter-reader reliability. The identification of the foveal avascular zone extend (arrow heads) was more challenging on FA compared to WF-OCTA (subjects A, B, F, and G). Preretinal hemorrhage (asterisk, subject A) can be well recognized on FA, whereas identification on WF-OCTA can be very limited. Additionally, capillary nonperfusion (arrows) is noted in subjects C–E and G.
Figure.
 
Comparison of fluorescein angiography and optical coherence tomography angiography. Exemplary early (first column) and late (second column) FAs and retinal (third column) and VRI WF-OCTA slabs (fourth column) for seven exemplary eyes. This figure illustrates the original images. Images were cropped to the same field of view for the analyses. Neovascularization at the disc are visible on FA and WF-OCTA in subject A and B (arrows). Artifacts in the VRI slab (red arrow, subject F) can suggest presents of neovascularization, but are not conclusive, which can be a reason for decreased inter-reader reliability. The identification of the foveal avascular zone extend (arrow heads) was more challenging on FA compared to WF-OCTA (subjects A, B, F, and G). Preretinal hemorrhage (asterisk, subject A) can be well recognized on FA, whereas identification on WF-OCTA can be very limited. Additionally, capillary nonperfusion (arrows) is noted in subjects C–E and G.
Table 1.
 
Characteristics of the Sample
Table 1.
 
Characteristics of the Sample
Inter-rater reliability was higher for qualitative analyses for FA than for WF-OCTA for all assessed parameters (Table 2). The numeric comparison of the qualitative parameters between WF-OCTA and FA for both graders including the detailed proportion of eyes that showed the pathologies is shown in Supplementary Tables S1 and S2. The presence of an extended FAZ, NVDs, and NVEs were underestimated on WF-OCTA, whereas the presence of vitreous hemorrhage was overestimated on WF-OCTA. 
Table 2.
 
Comparison of Inter-Rater Reliability of Qualitative Analyses Between WF-OCTA and FA
Table 2.
 
Comparison of Inter-Rater Reliability of Qualitative Analyses Between WF-OCTA and FA
In the quantitative comparison, the inter-rater reliability for WF-OCTA tended to be higher than for FA for all quantitative parameters measured, as shown in Table 3. For all four parameters, the 95% confidence intervals overlapped. Based on the 95% confidence intervals, the inter-rater reliability is “good” to “excellent” (FAZ and the horizontal diameter) and “moderate” to “excellent” (vertical and maximum diameter) for WF-OCTA. For FA, the inter-rater reliability is “moderate” to “good” (FAZ size and vertical FAZ diameter) and “moderate” to “excellent” (horizontal and maximum FAZ diameter), respectively. Supplementary Figures S2 and S3 illustrate a Bland-Altman plot for the comparison of the measurements of both graders on WF-OCTA (Supplementary Fig. S2) and FA (Supplementary Fig. S3). Difference tends to be higher for larger FAZ measurements. The numeric comparison of the quantitative analyses for both graders is shown in Supplementary Tables S3 and S4, and assessed parameters were not significantly different between WF-OCTA and FA. Overall, reliability for qualitative parameters was higher on FA, whereas reliability for quantitative parameters tended to be higher on WF-OCTA. 
Table 3.
 
Comparison of Inter-Rater Reliability of Quantitative Analyses Between WF-OCTA and FA
Table 3.
 
Comparison of Inter-Rater Reliability of Quantitative Analyses Between WF-OCTA and FA
Discussion
We have evaluated inter-rater agreement of qualitative and quantitative imaging parameters for severe DR on WF-OCTA and FA. So far, only studies comparing conventional (e.g., 3 × 3 mm) field-of-view OCTA with FA have been reported. However, a larger field of view FA, such as the one with 55° applied in our study, is clinically more relevant for assessment of severe DR not restricted to the macula/posterior pole, and reliability has not been analyzed for such wide field of views.25,26 Results of our study indicate that FA has a higher reliability for qualitative measurements (as presence/absence of an extended foveal avascular zone, NVD/NVEs, and vitreous hemorrhage), whereas WF-OCTA has a higher reliability for quantitative geometrical measurements (as parameters of FAZ extend) in severe DR. 
Quantitative analyses on angiography images require a clear discriminability of the quantitative imaging biomarker of interest compared to the background image. In the case of measurements of nonperfused areas such as the FAZ, identification of the capillaries bordering the avascular zone is therefore crucial. Therefore the scan quality has a significant impact on quantitative OCTA parameters.27 Given sufficient image quality, our results indicate that this identification of the exact avascular border has likely a higher reliability on WF-OCTA as compared to FA. The superiority of WF-OCTA may be due to partial FAZ masking on FA because of microaneurysm leakage, which is not visible on WF-OCTA. 
Other studies, analyzing the general applicability of OCTA to diabetic retinopathy, found that the FAZ and areas of capillary dropout are usually well delineated on OCTA, whereas FA often fails to clearly demarcate nonperfused areas.2833 Presence of NVDs and NVEs is indicated by vascular morphology in both FA and WF-OCTA and additionally by leakage on FA and three-dimensional structure (presence of perfusion in the VRI slab) on WF-OCTA. The higher reliability of analysis on FA that we found likely indicates a higher reliability of NVD/NVE detection by leakage on FA as compared to the detection of only morphological alterations observed on WF-OCTA. 
Our results are in line with a previous study on reliability of quantitative measurement, which found an ICC of 0.92 for FAZ area measurements, which however, compared OCTA with FA with a small field of view (30°).25,2833 Furthermore, our results on the accuracy of NVD/NVE detection on OCTA compared to FA are in line with a previous study by Wang et al.34 
Another study by Al-Khersan et al.26 found no statistically significant differences in the overall mean percentage of correct grading of NV when comparing residents, fellows, and attending retinal specialists using either SS-OCTA or FA (87.8% vs. 86.2%), although their false negative rates were as high as 77%. Our study showed a κ value of 1 between observers for detection of NV on FA, while the κ values were 0.83 and 0.77 for detection of NVD and NVE on OCTA. Identifying neovascularization on OCTA requires proper segmentation techniques, which can be easily missed otherwise. VRI images can sometimes give false negative results due to artefacts caused by segmentation errors at the level of ILM.35 Another reason for poor identification can be low flow within the NVs. Thus FA is still the gold standard for detection of neovascularization since it is almost always associated with leakage. In contrast, the NV network identification on OCTA may depend on the flow signal strength in these new vessels and proper segmentation of slabs. This is also reflected by our results indicating a higher reliability of NV detection on FA. 
The strengths of our study are its masked design and that all images from WF-OCTA and FA were standardized by cropping to the same field of view to avoid any bias related to field of view/magnification. We have comprehensively evaluated various parameters including qualitative and quantitative measurements using ImageJ. Such comprehensive inter-rater analyses for wide-field FA and OCTA have not been published so far. Limitations of our study include a relatively small sample size. However, as we only included patients with severe nonproliferative and proliferative diabetic retinopathy, and all patients needed both fluorescein and OCT angiography, recruitment was relatively challenging. Furthermore, both our qualitative and quantitative results showed a consistent difference/trend between the fluorescein and OCT angiography analysis for all analyzed parameters, despite the relatively small sample size. Also, we have not analyzed the detailed OCT-B scans with flow overlay, which may have given better NV detection rates on OCTA; however, it is more time consuming and, hence, less applicable to real-world settings. Although intra-reader reliability is another relevant parameter for reliability, the primary aim of this study was to assess inter-reader reliability, because one would expect that inter-reader reliability is more susceptible and clinically more relevant. 
Conclusion
Inter-rater reliability of FA is superior to WF-OCTA for qualitative analyses including detection of NVDs, NVEs, vitreous hemorrhage, and extended FAZ, whereas inter-rater reliability of WF-OCTA is superior to FA for quantitative geometrical analyses including FAZ measurements. Our study illustrates that both imaging modalities have specific merits in terms of reliability for analysis in severe DR. Therefore FA should be preferred for qualitative parameters, whereas WF-OCTA should be the preferred for quantitative parameters in terms of reliability. 
Acknowledgments
The optical coherence tomography angiography device was provided by Carl Zeiss Meditec. 
Disclosure: P.N. Shah, None; D.K. Mishra, None; P. Falahat, None; L. Fischer, None; G. Guzman, None; J.H. Terheyden, None; F.G. Holz, None; T.U. Krohne, None; R.P. Finger, None; M.W.M. Wintergerst, None 
References
Antonetti DA, Klein R, Gardner TW. Diabetic retinopathy. N Engl J Med. 2012; 366: 1227–1239. [CrossRef] [PubMed]
Cheung N, Mitchell P, Wong TY. Diabetic retinopathy. Lancet. 2010; 376: 124–136. [CrossRef] [PubMed]
Wessel MM, Aaker GD, Parlitsis G, Cho M, D'Amico DJ, Kiss S. Ultra-wide-field angiography improves the detection and classification of diabetic retinopathy. Retina. 2012; 32: 785–791. [CrossRef] [PubMed]
Rabiolo A, Parravano M, Querques L, et al. Ultra-wide-field fluorescein angiography in diabetic retinopathy: a narrative review. Clin Ophthalmol. 2017; 11: 803–807. [CrossRef] [PubMed]
López-Sáez MP, Ordoqui E, Tornero P, et al. Fluorescein-induced allergic reaction. Ann Allergy Asthma Immunol. 1998; 81: 428–430. [CrossRef] [PubMed]
Gao SS, Jia Y, Zhang M, et al. Optical coherence tomography angiography. Invest Ophthalmol Vis Sci. 2016; 57: OCT27–OCT36. [CrossRef] [PubMed]
Spaide RF, Klancnik JM, Jr., Cooney MJ. Retinal vascular layers imaged by fluorescein angiography and optical coherence tomography angiography. JAMA Ophthalmol. 2015; 133: 45–50. [CrossRef] [PubMed]
Cennamo G, Romano MR, Nicoletti G, Velotti N, de Crecchio G. Optical coherence tomography angiography versus fluorescein angiography in the diagnosis of ischaemic diabetic maculopathy. Acta Ophthalmol. 2017; 95: e36–e42. [CrossRef] [PubMed]
Hirano T, Kakihara S, Toriyama Y, Nittala MG, Murata T, Sadda S. Wide-field en face swept-source optical coherence tomography angiography using extended field imaging in diabetic retinopathy. Br J Ophthalmol. 2018; 102: 1199–1203. [CrossRef] [PubMed]
Soares M, Neves C, Marques IP, et al. Comparison of diabetic retinopathy classification using fluorescein angiography and optical coherence tomography angiography. Br J Ophthalmol. 2017; 101: 62–68. [CrossRef] [PubMed]
Cui Y, Zhu Y, Wang JC, et al. Comparison of widefield swept-source optical coherence tomography angiography with ultra-widefield colour fundus photography and fluorescein angiography for detection of lesions in diabetic retinopathy. Br J Ophthalmol. 2021; 105: 577–581. [CrossRef] [PubMed]
Hirano T, Hoshiyama K, Takahashi Y, Murata T. Wide-field swept-source OCT angiography (23  ×  20 mm) for detecting retinal neovascularization in eyes with proliferative diabetic retinopathy. Graefes Arch Clin Exp Ophthalmol. 2023; 261: 339–344. [CrossRef] [PubMed]
Li Z, Alzogool M, Xiao J, Zhang S, Zeng P, Lan Y. Optical coherence tomography angiography findings of neurovascular changes in type 2 diabetes mellitus patients without clinical diabetic retinopathy. Acta Diabetol. 2018; 55: 1075–1082. [CrossRef] [PubMed]
Lu Y, Simonett JM, Wang J, et al. Evaluation of automatically quantified foveal avascular zone metrics for diagnosis of diabetic retinopathy using optical coherence tomography angiography. Invest Ophthalmol Vis Sci. 2018; 59: 2212–2221. [CrossRef] [PubMed]
Pan J, Chen D, Yang X, et al. Characteristics of neovascularization in early stages of proliferative diabetic retinopathy by optical coherence tomography angiography. Am J Ophthalmol. 2018; 192: 146–156. [CrossRef] [PubMed]
Wang M, Garg I, Miller JB. Wide field swept source optical coherence tomography angiography for the evaluation of proliferative diabetic retinopathy and associated lesions: a review. Semin Ophthalmol. 2021; 36: 162–167. [CrossRef] [PubMed]
Schaal KB, Munk MR, Wyssmueller I, Berger LE, Zinkernagel MS, Wolf S. Vascular abnormalities in diabetic retinopathy assessed with swept-source optical coherence tomography angiography widefield imaging. Retina. 2019; 39: 79–87. [CrossRef] [PubMed]
Pichi F, Smith SD, Abboud EB, et al. Wide-field optical coherence tomography angiography for the detection of proliferative diabetic retinopathy. Graefes Arch Clin Exp Ophthalmol. 2020; 258: 1901–1909. [CrossRef] [PubMed]
Russell JF, Flynn HW, Jr., Sridhar J, et al. Distribution of diabetic neovascularization on ultra-widefield fluorescein angiography and on simulated widefield OCT angiography. Am J Ophthalmol. 2019; 207: 110–120. [CrossRef] [PubMed]
Schneider CA, Rasband WS, Eliceiri KW. NIH Image to ImageJ: 25 years of image analysis. Nat Methods. 2012; 9: 671–675. [CrossRef] [PubMed]
Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979; 86: 420–428. [CrossRef] [PubMed]
Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Measurement. 1960; 20: 37–46. [CrossRef]
Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33: 159–174. [CrossRef] [PubMed]
Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016; 15: 155–163. [CrossRef] [PubMed]
Garcia JM, Lima TT, Louzada RN, Rassi AT, Isaac DL, Avila M. Diabetic macular ischemia diagnosis: comparison between optical coherence tomography angiography and fluorescein angiography. J Ophthalmol. 2016; 2016: 3989310. [CrossRef] [PubMed]
Al-Khersan H, Russell JF, Lazzarini TA, et al. Comparison between graders in detection of diabetic neovascularization with swept source optical coherence tomography angiography and fluorescein angiography. Am J Ophthalmol. 2021; 224: 292–300. [CrossRef] [PubMed]
Czakó C, István L, Ecsedy M, et al. The effect of image quality on the reliability of OCT angiography measurements in patients with diabetes. Int J Retina Vitreous. 2019; 5: 46. [CrossRef] [PubMed]
Ishibazawa A, Nagaoka T, Takahashi A, et al. Optical coherence tomography angiography in diabetic retinopathy: a prospective pilot study. Am J Ophthalmol. 2015; 160: 35–44. [CrossRef] [PubMed]
Hwang TS, Jia Y, Gao SS, et al. Optical coherence tomography angiography features of diabetic retinopathy. Retina. 2015; 35: 2371–2376. [CrossRef] [PubMed]
Sawada O, Ichiyama Y, Obata S, et al. Comparison between wide-angle OCT angiography and ultra-wide field fluorescein angiography for detecting non-perfusion areas and retinal neovascularization in eyes with diabetic retinopathy. Graefes Arch Clin Exp Ophthalmol. 2018; 256: 1275–1280. [CrossRef] [PubMed]
Couturier A, Rey PA, Erginay A, et al. Widefield OCT-angiography and fluorescein angiography assessments of nonperfusion in diabetic retinopathy and edema treated with anti-vascular endothelial growth factor. Ophthalmology. 2019; 126: 1685–1694. [CrossRef] [PubMed]
Enders C, Baeuerle F, Lang GE, et al. Comparison between findings in optical coherence tomography angiography and in fluorescein angiography in patients with diabetic retinopathy. Int J Ophthalmol. 2020; 243: 21–26.
Shaikh NF, Vohra R, Balaji A, et al. Role of optical coherence tomography-angiography in diabetes mellitus: utility in diabetic retinopathy and a comparison with fluorescein angiography in vision threatening diabetic retinopathy. Ind J Ophthalmol. 2021; 69: 3218–3224.
Wang XN, Zhou J, Cai X, Li T, Long D, Wu Q. Optical coherence tomography angiography for the detection and evaluation of optic disc neovascularization: a retrospective, observational study. BMC Ophthalmol. 2022; 22: 125. [CrossRef] [PubMed]
Shiraki A, Sakimoto S, Eguchi M, et al. Analysis of progressive neovascularization in diabetic retinopathy using widefield OCT angiography. Ophthalmol Retina. 2022; 6: 153–160. [CrossRef] [PubMed]
Figure.
 
Comparison of fluorescein angiography and optical coherence tomography angiography. Exemplary early (first column) and late (second column) FAs and retinal (third column) and VRI WF-OCTA slabs (fourth column) for seven exemplary eyes. This figure illustrates the original images. Images were cropped to the same field of view for the analyses. Neovascularization at the disc are visible on FA and WF-OCTA in subject A and B (arrows). Artifacts in the VRI slab (red arrow, subject F) can suggest presents of neovascularization, but are not conclusive, which can be a reason for decreased inter-reader reliability. The identification of the foveal avascular zone extend (arrow heads) was more challenging on FA compared to WF-OCTA (subjects A, B, F, and G). Preretinal hemorrhage (asterisk, subject A) can be well recognized on FA, whereas identification on WF-OCTA can be very limited. Additionally, capillary nonperfusion (arrows) is noted in subjects C–E and G.
Figure.
 
Comparison of fluorescein angiography and optical coherence tomography angiography. Exemplary early (first column) and late (second column) FAs and retinal (third column) and VRI WF-OCTA slabs (fourth column) for seven exemplary eyes. This figure illustrates the original images. Images were cropped to the same field of view for the analyses. Neovascularization at the disc are visible on FA and WF-OCTA in subject A and B (arrows). Artifacts in the VRI slab (red arrow, subject F) can suggest presents of neovascularization, but are not conclusive, which can be a reason for decreased inter-reader reliability. The identification of the foveal avascular zone extend (arrow heads) was more challenging on FA compared to WF-OCTA (subjects A, B, F, and G). Preretinal hemorrhage (asterisk, subject A) can be well recognized on FA, whereas identification on WF-OCTA can be very limited. Additionally, capillary nonperfusion (arrows) is noted in subjects C–E and G.
Table 1.
 
Characteristics of the Sample
Table 1.
 
Characteristics of the Sample
Table 2.
 
Comparison of Inter-Rater Reliability of Qualitative Analyses Between WF-OCTA and FA
Table 2.
 
Comparison of Inter-Rater Reliability of Qualitative Analyses Between WF-OCTA and FA
Table 3.
 
Comparison of Inter-Rater Reliability of Quantitative Analyses Between WF-OCTA and FA
Table 3.
 
Comparison of Inter-Rater Reliability of Quantitative Analyses Between WF-OCTA and FA
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×