Clinically Meaningful Change Estimates for the National Eye Institute Visual Function Questionnaire-25 in Patients With Diabetic Macular Edema

Abstract

Purpose: To derive estimates of clinically meaningful change (improvement) on the 25-item National Eye Institute Visual Function Questionnaire (NEI VFQ-25) in patients with diabetic macular edema (DME) using anchor- and distribution-based methods.

Methods: In this exploratory post hoc analysis of data from the RIDE/RISE (NCT00473382/NCT00473330) clinical trials of ranibizumab for DME, the NEI VFQ-25 was completed at baseline and six, 12, 18, and 24 months. Anchor-based (≥5-, ≥10-, and ≥15-letter gain in best-corrected visual acuity [BCVA]) and distribution-based estimates were calculated. Subgroup analyses included outcomes when the study eye was the better- or worse-seeing eye.

Results: Baseline characteristics were balanced between the trials (RIDE, N = 382; RISE, N = 377). Anchor-based estimates of clinically meaningful improvement in composite scores (for ≥15-letter gain in BCVA) were 3.78 and 2.23 for RIDE and RISE, respectively. Estimates appeared similar for most subscales: near activities (4.11 and 3.31), distance activities (3.53 and 3.74), driving difficulties (5.15 and 3.15), and vision-specific dependency (4.70 and 1.83). Supportive distribution-based meaningful change composite score estimates also were similar between RIDE and RISE for values based on 0.5 standard deviation (9.85 and 9.70, respectively) or standard error of the mean (5.10 and 4.82, respectively).

Conclusions: These analyses suggest improvement of three to five points on the NEI VFQ-25 composite score and four individual subscales as clinically meaningful in patients with DME.

Translational Relevance: This analysis supports considering these thresholds when assessing the clinical risk-benefit of DME treatment from the patient perspective using the NEI VFQ-25.

Introduction

Self-reported vision-related quality of life in patients with diabetic eye disease shows that worsening diabetic retinopathy severity is associated with increased vision-related functional burden.¹ Patient-reported outcomes (PROs) specific to retinal eye diseases represent a quantitative way of estimating the quality of a patient's functional vision as related to the patient's perception, and change on the scale can be used to evaluate disease progression, as well as outcomes of treatments. Some PROs have been accepted for use as valid functional endpoints by regulatory agencies for inclusion in product labeling and, with or without inclusion in product labeling, have been leveraged by payors to support consideration of reimbursement for treatments. 

The 25-item National Eye Institute Visual Function Questionnaire (NEI VFQ-25), although not accepted currently by the U.S. Food and Drug Association (FDA) for inclusion in product labeling, is a commonly used PRO measure of vision-related function in ophthalmology clinical trials² and has been used widely in study participants with acquired retinal diseases, including diabetic macular edema (DME). Various publications have shown that the NEI VFQ-25 is responsive to treatment in common retinal diseases,^3–7 and it has been taken into consideration by government payors such as the Centers for Medicare and Medicaid Services when determining coverage of anti-vascular endothelial growth factor (anti-VEGF) agents for retinal diseases.⁸ 

To date, the NEI VFQ-25 has been included in a number of clinical trials as an efficacy endpoint to help demonstrate the benefit of anti-VEGF treatments from the DME patient perspective. RIDE (NCT00473382) and RISE (NCT00473330),^9,10 and VISTA (NCT01363440) and VIVID (NCT01331681)^11,12 were large randomized phase 3 superiority trials comparing active treatment with an anti-VEGF agent to sham or laser control arm in participants with DME. These trials used the NEI VFQ-25 as a supportive efficacy measure of participant-reported vision-related function. In RIDE and RISE,^9,10 participants treated with ranibizumab showed greater improvement in the NEI VFQ-25 compared with the control sham-treated participants at year 2 (macular laser was permitted starting at month 6). In VISTA, participants treated with aflibercept had improved near and distance activity subscale scores on the NEI VFQ-25 compared with the control participants treated with a sham intravitreal injection and laser, whereas in the identically designed VIVID trial, subscale scores were similar between treatment arms after 2 years.¹¹ The large randomized Diabetic Retinopathy Clinical Research Network Protocol S¹³ compared ranibizumab treatment with panretinal photocoagulation (PRP) control in patients with diabetic retinopathy. Although differences in some work productivity and driving-related outcomes favored ranibizumab over PRP, no differences were noted between the two treatment options in most of the NEI VFQ-25 outcomes at 2 years associated with excellent mean visual acuities in both treatment groups at baseline and follow-up.¹³ Two other large, randomized phase 3 trials, KESTREL (NCT03481634) and KITE (NCT03481660),¹⁴ evaluated brolucizumab versus control aflibercept treatment and showed relatively comparable results across NEI VFQ-25 scores with brolucizumab arms compared with aflibercept at year 2.¹⁵ 

A recent phase 3 program in participants with DME comprised two large randomized clinical trials evaluating faricimab compared with aflibercept (YOSEMITE [NCT03622580] and RHINE [NCT03622593]). Across the studies, both faricimab treatment regimens showed improvements in NEI VFQ-25 composite, near activities, distance activities, and driving scores at years 1 and 2 that were comparable with aflibercept.¹⁶ 

Determining whether changes in a PRO are meaningful is dependent on whether the changes are considered to be clinically relevant. A meaningful change threshold is the smallest amount of change in a PRO score that might be considered clinically meaningful to patients. Various approaches are used to evaluate meaningful change, including anchor-based and distribution-based methods. The anchor-based method involves anchoring changes in the PRO measure of interest to an external criterion measure that identifies patients who have experienced an important change in their condition. This approach is considered a preferred methodology for deriving meaningful change estimates because it links change in the target PRO measure to a degree of change on the anchor measure that is recognized as meaningful. In retinal disorders, best-corrected visual acuity (BCVA) is a well-established functional endpoint and is frequently used as the primary efficacy outcome in registrational clinical trials, with individual patient changes of ≥15 letters considered clinically meaningful by regulatory authorities across the range of BCVAs typically noted in common retinal diseases.¹⁷ Previous studies have shown that BCVA is at least moderately correlated with NEI VFQ-25 in DME and other ophthalmic disorders.^2,18 Furthermore, a ≥15-letter change in BCVA has been used previously as an anchor to estimate clinically meaningful changes in NEI VFQ-25 scores in patients with neovascular age-related macular degeneration (nAMD), with an identified range of 4 to 6 points.⁷ Some ophthalmologists have proposed that smaller changes in BCVA (e.g., visual gain of ≥ 10 letters on the Early Treatment Diabetic Retinopathy Study [ETDRS] chart) also may be clinically relevant¹⁷; however, the relevance of these smaller changes to patients is not currently well established and not always considered meaningful to regulatory agencies, such as the U.S. FDA. 

Distribution-based methods use the distribution of the PRO score to classify the magnitude of a meaningful change, rather than the prespecified statistical significance or clinical relevance of that change from the patient's perspective.^19,20 Given their lack of connection to external criteria to contextualize meaningfulness, they are considered supportive only. A distribution-based estimate of meaningful change for an NEI VFQ-25 composite score of three to six points in DME has been reported, with higher ranges for the individual subscales,¹⁸ but to our knowledge, no anchor-based analyses have been described to date. Such analyses could add to our understanding of the clinical relevance of changes in NEI VFQ-25 scores in retinal diseases. Therefore the aim of this post hoc analysis of data from RIDE and RISE was to derive an estimate of clinically meaningful change on the NEI VFQ-25 in patients with DME using anchor- (with BCVA gain of ≥ 5, ≥ 10, or ≥ 15 ETDRS letters) and distribution-based methods. 

Methods

RIDE and RISE

RIDE and RISE were parallel, randomized, multicenter, double-masked, sham-injection controlled, phase 3 clinical trials that evaluated the 24-month (primary outcome endpoint) efficacy and safety of ranibizumab compared with sham injections in patients with center-involved DME.¹⁰ Both trials were conducted in accordance with the Declaration of Helsinki and were compliant with the Health Insurance Portability and Accountability Act. Protocols were approved by institutional review boards or ethics committees, as applicable, and all patients provided written informed consent. 

Patients ≥18 years of age with diabetes mellitus (type 1 or 2), BCVA 20/40 to 20/320 Snellen equivalent, and retinal thickening secondary to diabetes mellitus (DME) involving the center of the fovea, with central macular edema ≥275 µm on optical coherence tomography, were enrolled in RIDE/RISE.¹⁰ Participants were randomized 1:1:1 to receive monthly intravitreal ranibizumab 0.3 mg, ranibizumab 0.5 mg, or sham treatment through month 24 in one eye designated as the study eye. In participants with bilateral DME, the eye with the worse visual acuity as assessed at screening was selected as the study eye, unless, based on medical reasons, the investigator deemed the other eye to be more appropriate for treatment and study per the protocol.¹⁰ Additional details on the patient populations have been published previously.¹⁰ 

From month 3 onward, all participants were eligible to receive macular laser therapy according to protocol-specified criteria: central foveal thickness ≥250 µm with a <50-µm change from the previous month; no prior macular laser in the previous 3 months; and an assessment by the evaluating physician that macular laser would be beneficial. The primary efficacy endpoint of RIDE and RISE was the proportion of participants who gained ≥ 15 ETDRS letters in BCVA from baseline at month 24.¹⁰ Secondary efficacy endpoints relevant to the present analysis included mean change from baseline BCVA over time and the proportion of participants who achieved BCVA of 20/40 or better (equivalent to an ETDRS letter score ≥ 74). 

The NEI VFQ-25, a PRO measure that is usually driven by binocular vision,⁴ as opposed to the regular monocular clinical assessment of BCVA, was interviewer-administered in person at baseline and at months 6, 12, 18, and 24.²¹ The NEI VFQ-25 is valid for use in patients with chronic eye diseases and comprises 11 vision-related subscales, including general vision (one item), difficulty with near vision (three items) and distance vision activities (three items), vision-specific dependency on others (three items), vision-specific limitations in social functioning (two items) and role difficulties (two items), vision-specific mental health symptoms (four items), driving difficulties (three items), limitations with peripheral (one item) and color vision (one item), ocular pain (two items), and one general health question.²² The NEI VFQ-25 also includes 13 optional appendix items that can be used to supplement the subscales. In RIDE and RISE, six appendix items (See Supplementary Table S1) were used to enhance the reliability of the near and distance activities subscales. Subscale scores are calculated according to published guidelines and range from 0 to 100, with higher scores representing better vision-related functioning.² The overall composite score is calculated by taking the mean of all the vision-related subscale scores, excluding the general health question.²³ 

Data Analyses and Statistical Methods

For this post hoc analysis, treatment groups were pooled within each study to try to increase the robustness of the clinically meaningful change estimates. This was judged to be an appropriate approach because the trial data, including outcomes, appeared similar. As the purpose of this analysis was to generate estimates for use in benchmarking clinical trial results, the study eye was used for the primary analyses. Results are also reported for when the study eye was the better- or the worse-seeing eye relative to the fellow eye per previously described definitions.²⁴ 

The anchor-based estimates of clinically meaningful change on the NEI VFQ-25 composite score and near activities, distance activities, driving difficulties, and vision-specific dependency subscales were estimated using ≥5-, ≥10-, and ≥15-letter gains in BCVA as clinical anchors. Analysis of covariance (ANCOVA) models were estimated by regressing NEI VFQ-25 change from baseline to month 24 in RIDE and RISE on change in BCVA from baseline to month 24. These analyses used visual acuity in the study eye assessed at 24 months. The least-squares mean change in NEI VFQ-25 for each visual acuity subgroup, with associated 95% confidence intervals, was derived from ANCOVA models. All models were controlled for age, self-reported sex, and baseline NEI VFQ-25 score. 

Distribution-based thresholds were calculated using 0.5 standard deviation (SD) at baseline and standard error of measurement (SEM). Estimates from distribution- and anchor-based methods were triangulated to determine an appropriate range, with anchor-based estimates prioritized. Triangulation is the process by which estimates generated across the different methods are considered and a threshold, or narrow range of thresholds, are selected where there is convergence. When considering estimates together, those generated via anchor-based methods using the most clinically relevant anchors tend to be weighted more strongly than those generated using less clinically relevant anchors or distribution-based methods.^25–27 All results were considered post hoc exploratory outcomes; hence, there was no adjustment to the analyses for multiplicity. 

Results

Baseline and Clinical Characteristics in RIDE/RISE

As previously described,¹⁰ 759 patients with DME were enrolled in RIDE (N = 382) and RISE (N = 377) and randomized 1:1:1 to receive monthly intravitreal ranibizumab 0.3 mg (RIDE, n = 125; RISE, n = 125), ranibizumab 0.5 mg (RIDE, n = 127; RISE, n = 125), or sham (RIDE, n = 130; RISE, n = 127) injections through month 24. Baseline participant demographic and clinical characteristics (pooled across treatment arms), including visual acuity, appeared generally well balanced in RIDE and RISE (Tables 1, 2). In RIDE, 99.5% (380 of 382) of participants completed the NEI VFQ-25 at baseline, 84.6% (323 of 382) at 12 months, and 81.2% (310 of 382) at 24 months.⁴ In RISE, 99.2% (374 of 377) of participants completed the NEI VFQ-25 at baseline, 82.8% (312 of 377) at 12 months, and 78.2% (295 of 377) at 24 months.⁴ The baseline NEI VFQ-25 composite score and subscale scores (pooled across treatment arms) were similar between RIDE and RISE (Table 3). 

Table 1.

 

Baseline Demographics and Clinical Characteristics for Participants With DME in RIDE and RISE, Pooled Across Treatment Groups

 

Table 2.

 

Baseline BCVA Scores for Participants With DME in RIDE and RISE, Pooled Across Treatment Groups

 

Table 3.

 

Baseline NEI VFQ-25 Composite Score and Subscale Scores for Participants With DME in RIDE and RISE, Pooled Across Treatment Groups

 

Anchor-Based Meaningful Change Estimates

The anchor-based meaningful change (improvement) estimates based on ≥15-, ≥10-, and ≥5-letter gains in visual acuity in the study eye for NEI VFQ-25 scores in RIDE and RISE are detailed in Table 4. For ≥ 15-letter gain, composite score estimates were 3.78 and 2.23 for RIDE and RISE, respectively. Estimates were generally similar for the near activities subscale (4.11 and 3.31), distance activities subscale (3.53 and 3.74), driving difficulties subscale (5.15 and 3.15), and vision-specific dependency subscale (4.7 and 1.83). For ≥10-letter gain, composite score estimates were 2.52 and 1.49 for RIDE and RISE, respectively, and subscale estimates ranged from 1.22 to 3.43. For ≥5-letter gain, composite score estimates were 1.26 and 0.74 for RIDE and RISE, respectively, and subscale estimates ranged from 0.61 to 1.72. Anchor-based estimates were generally higher compared with overall estimates when the study eye was the better-seeing eye and generally lower when the study eye was the worse-seeing eye (Table 5). 

Table 4.

 

Anchor-Based Meaningful Change Estimates for NEI VFQ-25 Composite and Subscale Scores at 24 Months Based on ≥ 15-Letter, ≥10-Letter, and ≥5-Letter Gain in Visual Acuity in the Study Eye

 

Table 5.

 

Anchor-Based Meaningful Change Estimates for NEI VFQ-25 Composite and Subscale Scores at 24 Months Based on ≥15-Letter Gain, ≥10-Letter Gain, and ≥5-Letter Gain in Visual Acuity in the Study Eye

 

Distribution-Based Meaningful Change Estimates

The distribution-based meaningful change estimates based on 0.5 SD and SEM are detailed in Table 6. Composite score estimates were similar between trials (0.5 SD: 9.85 and 9.70; SEM: 5.10 and 4.82). Subscale score estimates were also similar between trials, ranging from 11.37 to 17.09 for 0.5 SD and 7.52 to 21.77 for SEM. The higher scores observed in the subscales compared with the composite could be associated with the former comprising fewer items, which could contribute to higher variability. Also of note, whereas estimates based on the 0.5 SD at baseline solely reflected the group-based distribution, the SEM took measurement error into account. Since the composite and most subscale scores had levels of reliability (α ≥ 0.70²⁸) judged acceptable, the SEM estimate was associated with smaller estimates than those generated using 0.5 SD alone. An exception was the driving subscale (α = 0.67 for RIDE and 0.60 for RISE), for which the lower reliability resulted in higher SEM estimates compared with those based on 0.5 SD. Distribution-based estimates when the study eye was the worse-seeing eye relative to the fellow eye and when the study eye was the better-seeing eye relative to the fellow eye were generally consistent with the overall estimates (Table 7). 

Table 6.

 

Distribution-Based Meaningful Change Estimates for NEI VFQ-25 Composite and Subscale Scores at Baseline Based on 0.5 SD at Baseline and SEM in Visual Acuity in the Study Eye

 

Table 7.

 

Distribution-Based Meaningful Change Estimates for NEI VFQ-25 Composite and Subscale Scores at Baseline Based on 0.5 SD at Baseline and SEM in Visual Acuity in the Study Eye

 

Triangulation of Meaningful Change

Estimates from anchor- and distribution-based methods were triangulated to determine a meaningful change estimate, with anchor-based estimates prioritized. Across subscales, estimates from the anchor-based analyses were highest for a ≥15-letter gain (range 1.83 to 5.15) and successively smaller for ≥10- (range 1.22 to 3.43) and ≥5-letter gains (range 0.61 to 1.72). Because a ≥15-letter gain in BCVA from baseline is considered clinically meaningful in most patients with DME and the relevance of smaller letter gains may or may not be clinically meaningful in an individual, the former was given more weight in selecting a threshold range. A range of 3 to 5 encompassed the estimates derived using an anchor of a ≥ 15-letter gain, while exceeding most of the estimates derived using the smaller letter gain anchors. 

Discussion

The NEI VFQ-25 is frequently used in ophthalmology clinical trials owing to the utility of this questionnaire for understanding the relationship between treatment approaches and patient-reported vision-related function. Although not accepted currently by the FDA for inclusion in product labeling, it has been taken into consideration by government payors such as Centers for Medicare and Medicaid Services when determining coverage of anti-VEGF agents for retinal diseases. The NEI VFQ-25 has been shown repeatedly to be responsive in clinical trials for anti-VEGF therapies (e.g., VIVID, KESTREL^11,15). To our knowledge, however, the magnitude of clinically relevant meaningful change in NEI VFQ-25 outcomes has not been anchored to objectively measured, clinically relevant ETDRS BCVA measurements in patients with DME. The establishment of meaningful change thresholds can add to the interpretability of clinical trial data.²⁹ Our post hoc analyses aimed to determine what magnitude of change (specifically improvement) on NEI VFQ-25 might be clinically meaningful based on anchoring of the NEI VFQ-25 changes to an objectively measured visual acuity outcome and distribution-based estimates. 

Comparing the results and weighing the anchor- and distribution-based approaches, we suggest a range of three to five points on the NEI VFQ-25 composite and four individual subscales as representing clinically meaningful change in patients with DME. These findings are in line with previously identified meaningful change thresholds on the NEI VFQ-25 composite score and near activities, distance activities, and dependency subscales for nAMD (four to six points).¹⁷ They also align with distribution-based estimates for the NEI VFQ-25 composite (three to six points) and subscale scores for DME.¹⁸ Distribution-based estimates, while not recommended to be the sole basis of deriving meaningful change thresholds, can be considered as supporting evidence. Ideally, clinically meaningful change thresholds should be the smallest amount of change identified by patients as meaningful. In this study, the lowest distribution-based estimate was around five for the study eye, as well as the better and worse seeing eyes. These findings were consistent with the three to five range identified using anchor-based methods. By comprehensively assessing meaningful change in a DME trial population, the results of this study might support clinical risk-benefit assessment of DME treatment interventions in the future. 

This study has several limitations. These analyses were not planned in the original trial designs and were post hoc and exploratory in nature and therefore should be considered as hypothesis-generating. The analyses were limited to the variables and assessment time points available in the data sets. Additionally, as in other clinical trials, the participants in RIDE and RISE were recruited based on specific eligibility criteria, which may not be representative of the broader DME population, and they received only one anti-VEGF agent (ranibizumab 0.3 or 0.5 mg), thus potentially limiting generalizability of the results to other patients with DME. The goal of the analyses was to generate thresholds to benchmark future clinical trial results. As such, the analyses were conducted with a relevant clinical trial sample and the study eye was used for all primary analyses for this investigation, which in most cases was the worse-seeing eye as defined previously²⁴ (59.2% [226 of 382] in RIDE and 66.6% [251 of 377] in RISE; Table 1). Whether the thresholds obtained can be used in observational or real-world studies where the patient criteria are less stringent than in investigational clinical trials is a topic for future research. Findings from our subanalyses of data when the study eye was the better- or worse-seeing eye (relative to the fellow eye) generally were consistent with the overall study eye findings, although some of the better-seeing eye estimates were higher than the overall estimates and generally lower when the study eye was the worse-seeing eye relative to the fellow eye. Given the suggested range of three to five points, we recommend researchers select the threshold on the higher or lower end of this range, depending on whether they are investigating change scores in the better or worse seeing eye, respectively. An additional limitation is that meaningful change estimates were not calculated for all subscales of the NEI VFQ-25, but only those considered most proximal to the disease and most likely to be responsive to change in BCVA. Most participants in RIDE and RISE experienced some degree of improvement in their BCVA over the course of 24 months of treatment. Therefore these analyses anchored participants to ≥5-, ≥10-, and ≥15-letter gains rather than losses. As such, the resulting meaningful change threshold range may be applicable only to benchmark improvement, not loss of visual acuity. Further research would be needed to determine the degree of visual acuity decline that is clinically meaningful to patients. 

In conclusion, this exploratory post hoc analysis confirmed previous estimates of clinically meaningful improvement in the association of change in the visual acuity of the study eye with changes in NEI VFQ-25 composite and subscale scores. This investigation expanded on these previous studies as an attempt to comprehensively derive clinically meaningful improvement on the NEI VFQ-25 composite and four individual subscales, which was identified as an improvement of three to five points in patients with DME. These thresholds may be considered when assessing the clinical risk-benefit of DME treatment in clinical trials from the patient perspective using the NEI VFQ-25. Future studies using additional large data sets from the most recent ongoing phase 3 programs in patients with DME might be used to confirm and expand upon the findings of this analysis. 

Acknowledgments

Supported by Genentech, Inc., a member of the Roche Group (South San Francisco, California). The sponsor participated in the design of the study; collection, management, analysis, and interpretation of the data; and preparation, review, and approval of the manuscript. Third-party writing assistance, provided by Luke Carey, PhD, CMPP, of Envision Pharma Group, was funded by Genentech, Inc. 

Disclosure: N. Bressler, Principal Investigator of grants at Johns Hopkins University for research: Bayer (F), Boehringer Ingelheim (F), Genentech, Inc. (F), Regeneron (F), Samsung Bioepis (F), American Medical Association as Editor in Chief of JAMA Ophthalmology (C), Emmes LLC as Chair of the Data and Safety Monitoring Committee for intramural trials with the National Eye Institute (C), US Food and Drug Association as Chair of the Ophthalmic Devices Panel (C), volunteer for Board of Directors of the JAEB Center for Health Research; Z. Haskova, Genentech, Inc. (E), Roche (I); A. Kapre, Genentech, Inc. (E) at the time of study; B. Gentile, Genentech, Inc. (E)