A Screening Tool for Self-Evaluation of Risk for Age-Related Macular Degeneration: Validation in a Spanish Population

Abstract

Purpose: The objectives of this study were the creation and validation of a screening tool for age-related macular degeneration (AMD) for routine assessment by primary care physicians, ophthalmologists, other healthcare professionals, and the general population.

Methods: A simple, self-administered questionnaire (Simplified Théa AMD Risk-Assessment Scale [STARS] version 4.0) which included well-established risk factors for AMD, such as family history, smoking, and dietary factors, was administered to patients during ophthalmology visits. A fundus examination was performed to determine presence of large soft drusen, pigmentary abnormalities, or late AMD. Based on data from the questionnaire and the clinical examination, predictive models were developed to estimate probability of the Age-Related Eye Disease Study (AREDS) score (categorized as low risk/high risk). The models were evaluated by area under the receiving operating characteristic curve analysis.

Results: A total of 3854 subjects completed the questionnaire and underwent a fundus examination. Early/intermediate and late AMD were detected in 15.9% and 23.8% of the patients, respectively. A predictive model was developed with training, validation, and test datasets. The model in the test set had an area under the curve of 0.745 (95% confidence interval [CI] = 0.705–0.784), a positive predictive value of 0.500 (95% CI = 0.449–0.557), and a negative predictive value of 0.810 (95% CI = 0.770–0.844).

Conclusions: The STARS questionnaire version 4.0 and the model identify patients at high risk of developing late AMD.

Translational Relevance: The screening instrument described could be useful to evaluate the risk of late AMD in patients >55 years without having an eye examination, which could lead to more timely referrals and encourage lifestyle changes.

Introduction

Age-related macular degeneration (AMD) is a progressive and degenerative disease of the retina and choroid with a high and growing incidence in the elderly population of industrialized nations.^1,2 In the age range of 45 to 85 years, the prevalence of AMD has been estimated at 8.7%,² and at 17.6% in those aged ≥85 years.³ According to the severity of the retinal findings, AMD can be categorized as early, intermediate, and late AMD, a classification which helps clinicians to track progression, estimate risk, and provide advice on possible interventions.⁴ There are two subtypes of late AMD, geographic atrophy and neovascular disease, and both are associated with blindness and major impact on quality of life.⁵ Although there is no current curative therapy for AMD, the neovascular forms of the disease can be treated with antivascular endothelial growth factor injections.^6,7 However, treatment is burdensome, some patients do not respond adequately, and progressive vision loss is common. 

Due to the high incidence and major impact of AMD on quality of life,⁸ the development of a general and reliable early risk evaluation for AMD has been the focus of much research. Accurate analysis of risk factors could lead to timely monitoring, lifestyle changes, such as dietary modifications, and possible clinical interventions. A number of prognostic models using statistical methods have been developed that use features such as genetic polymorphisms, clinical history of the patient, baseline grade of AMD, and behavioral factors.^9–12 Some of these models allow for missing data, such as genotypes, and all are meant to assess likelihood of progressing to late AMD longitudinally over time. More recently, deep learning has been applied that analyzes imaging data,^13–17 but these technologies are rarely applicable or available in daily clinical practice. 

Another approach is the development of relatively simple questionnaires with the objective of identification of individuals currently at risk. The Simplified Théa AMD Risk-Assessment Scale (STARS) questionnaire was developed and validated as a tool to be used by ophthalmologists in routine clinical practice or as a self-assessment for identification of patients at risk of AMD.¹⁸ The STARS questionnaire was derived from large samples of subjects aged ≥50 years from European ancestry and showed good discrimination of patients with and without AMD. The STARS questionnaire evaluated well-established AMD risk factors, such as ethnicity, family history of AMD, smoking, and personal medical history, but did not include information on possible diet-related risk factors. 

There is interest in including diet and nutritional information in the algorithms, as diet is a modifiable factor that could be applied early to slow disease progression.¹⁹ In a single large randomized clinical trial, some nutritional supplements were found to have a potential role in delaying the progression of intermediate forms of AMD,^20,21 and numerous other studies have suggested that diet could have a role in delaying the progression of AMD.^19,22–26 For these reasons, a new version of the STARS questionnaire (version 4.0) was developed that incorporated dietary factors into the algorithm to predict AMD risk. 

Here, we present the development and validation of a new screening tool based on the modified STARS questionnaire. This instrument, which includes dietary information, was used to collect the data among individuals aged ≥55 years, and to establish the relationship between risk factors and the presence of clinical signs of AMD. The new predictive model was designed to identify probability of a high Age-Related Eye Disease Study (AREDS) score (2–4),²⁷ which provides an estimation of the risk of developing late AMD in 5 years. The model was implemented in an online platform that could be used by ophthalmologists, primary health physicians, other healthcare professionals, and the general population, as a screening tool for risk of AMD without genetic testing and a detailed eye examination. 

Methods

This study was promoted by the Spanish Retinal and Vitreous Society (SERV), the Health Research Cooperative Network in Ophthalmology (RETICS-OFTARED), and the Retina+ Foundation. Théa Laboratoires provided technical and logistics support for the development of the study. The protocol was approved by the Clinical Research Ethical Committee of the University Hospital of Valladolid (Valladolid, Spain) on July 26, 2013. The study was conducted in accordance with the ethical principles of the Declaration of Helsinki, good pharmaco-epidemiology practices (GPPs), and the applicable laws and regulations of Spain. All individuals were informed of the objectives of the study and authorized their inclusion by signing a consent form prior to participation. 

Patient Population and Study Design

A cross-sectional survey was carried out with the participation of 44 ophthalmologists working in 44 public or private hospitals and eye clinics throughout Spain, who recruited patients from September 2013 to November 2015. Inclusion criteria were patients aged ≥55 years attending a routine clinical ophthalmological visit. Patients were invited to participate in the study regardless of the reason for the visit to the ophthalmologist (e.g. periodic review of chronic disease, acute symptoms, preparation for surgery, follow-up, etc.), and the invitation was unrelated to the possible presence of AMD symptoms. All patients completed the STARS questionnaire version 4.0 consisting of topics related to possible AMD risk factors: age, gender, medical history (myocardial infarction, hypertension, atherosclerosis, and hypercholesterolemia), body mass index (BMI), family history of AMD (father, mother, and siblings), smoking habits, iris color, ocular risk factors (emmetropia, hyperopia, and myopia), and dietary factors, such as consumption of alcohol, oily fish, eggs, green-leaf vegetables, fruit, and omega-3 rich oils (see Supplementary Figs. S1 and S2 in Supplementary Information). Next, the ophthalmologist performed a dilated fundus examination to verify the presence of large soft drusen (>125 µm), pigmentary abnormalities, or late AMD. For each patient, a single visit was necessary, no specific treatment was prescribed related to the study, and there was no follow-up. All data were anonymous and collected in an electronic database for further processing. 

Risk Model Development

The model was designed to identify probability of a high AREDS score (2–4) without having a clinical examination. This score correlates with an estimated risk of developing late AMD in 5 years. To develop the model, data from the STARS questionnaire and the fundus examinations of patients were used. Patients with missing data on the fundus examination were not included in the model training. AREDS score was derived and the presence of early/intermediate and late AMD were assessed. The database was split randomly into three independent datasets (60% training, 20% validation, and 20% testing). Database feature and dimensionality reduction were performed by mutual information, and correlation analysis by principal component analysis, respectively. 

Some independent variables in the STARS questionnaire version 4.0 were not used in further analysis due to low variability (geographic origin of ancestors and skin color), inconsistency (consumption of supplements of vitamin C, omega-3 oils, or lutein/zeaxanthin), and because it lowered the accuracy of the final model (cataract surgery). 

The machine learning techniques that were evaluated to derive the predictive algorithm were logistic regression, decision tree analysis, random forest, support vector machine, K-nearest neighbor’s algorithm, naïve Bayes, and artificial neural networks. An exhaustive search of different parameters was applied to the training dataset. Best parameters were chosen according to the performance of each fit model applied on the validation set. The performance was assessed based on metrics that quantifies the quality of predictions: true positive (TP), true negative (TN), false negative (FN) and false positive (FP), receiving operating characteristic (ROC), and area under the curve (AUC). To select the best model, performance of the model in the test group (accuracy and AUC) were considered (see Supplementary Tables S1 and S2, and Supplementary Fig. S2 for comparative between logistics regression and neural network models). The dependent variables were the presence of large drusen and pigmentary abnormalities. The relationship was established based on the AREDS score.²⁷ The AREDS scoring scale for the risk of developing late AMD in the next 5 years relies on the presence of pigmentary abnormalities and large drusen (defined as >125 µm). This scale consists of 5 levels, from level 0 (lack of risk factors in both eyes) up to level 4 (presence of large drusen and pigment changes in both eyes), and each level is associated with a probability of developing late AMD in the next 5 years overall, but is not eye specific. These probabilities are: score 0 (0–1% probability), score 1 (1–5%), score 2 (5–19%), score 3 (19–31%), and score 4 (31–50%). In this study, risk of late AMD was categorized as “low” for AREDS scores 0 and 1 and as “high” for AREDS scores 2 to 4. 

The analysis was performed using software R Project 3.5 and Python 3.7. 

Web Application

The web application tool was created using HTML and Django 2, a high-level Python Web framework. The design and views of the user interface were developed in cooperation with ophthalmologists with the objective being ease of use. The main view of the application is the STARS questionnaire version 4.0, which should be completed by the user. The interface then returns a prediction of the risk of developing late AMD in the next 5 years. The model evaluates the risk of the patient at different ages with the same items that he/she completed in the questionnaire. If the risk of the user is predicted to worsen in the future, the interface returns a prediction about the risk factor of developing late AMD at that age. Additionally, the web tool provides recommendations according to the risk level of developing late AMD in the next 5 years. The interface was developed in two languages, English and Spanish. 

Statistical Analysis

All the variables in the database were discrete. Histograms and contingency tables were performed to analyze its distribution and variability. Contingency tables included frequencies and relative percentages. 

Results

A total of 3854 patients who had completed the STARS version 4.0 questionnaire during a visit to their ophthalmologists and who had undergone a fundus examination (identification of soft drusen [>125 µm], pigmentary abnormalities, or a diagnosis of late AMD) were included in the study. Table 1 summarizes the demographic characteristics of the patients and the results from the survey and the eye examination. Most patients (78.5%) were aged ≥65 years and 57.2% were women. A large number of patients were overweight (BMI >25, 58.8%), had hypertension (52.4%), and hypercholesterolemia (38.5%). Upon fundus examination, early/intermediate and late AMD were diagnosed in 15.9% and 23.8% of the patients, respectively. No AMD was detected in 60.3% of patients. 

Table 1.

 

STARS Questionnaire Version 4.0.^a

 

Of all machine learning techniques tested, logistic regression was chosen for predictive modeling. The independent variables used were: “age,” “gender,” “BMI,” “father with AMD,” “mother with AMD,” “sibling with AMD,” “myocardial infarction,” “presence of atherosclerosis,” “‘high cholesterol,” “smoking habit,” “beer consumption,” “wine consumption,” “fish consumption,” “fruit consumption,” “vegetables consumption,” “eggs consumption,” “omega-3-rich oils,” “iris color,” “hyperopia,” and “myopia.” Data from 2312 patients (59.8%) was used in model training, information from 771 patients (18.4%) was included in model validation, and 771 patients (18.4%) were represented in model testing. 

The performance of the logistic regression model was evaluated by the area under ROC curves (AUC), as shown in Figure 1. The AUC was 0.75 (95% CI = 0.71–0.78). This model suggested a high AREDS score with an accuracy of 67.70% (95% CI = 64.4–71.0). Table 2 shows the metrics obtained from the logistic regression model, with a positive predictive value of 0.500 (95% CI = 0.449–0.557) and a negative predictive value of 0.810 (95% CI = 0.770–0.844). 

Figure 1.

 

Area under ROC (AUC) curve of the logistic regression model derived from the test set.

Table 2.

 

Performance of the Logistic Regression Model in the Test Group

 

The model was implemented online at www.Test4Retina.com. To be able to relate the probabilities derived from the logistic regression model with the AREDS scores and the 5-year probabilities of developing late AMD,⁴ the probabilities from the model were distributed by using the specificity and sensitivity metrics. The results are shown in Table 3 and Figure 2. To determine the result for the model probabilities for each range, linear extrapolations were performed. The website indicates that it is not a diagnostic tool. Rather, the results obtained are estimates of the current AREDS score that provide the risk of developing late AMD in 5 years. An eye examination by an ophthalmologist is recommended, especially for older individuals. 

Table 3.

 

Model Correspondence With the AREDS Score

 

Figure 2.

 

Relationship between the logistic regression model probabilities and the AREDS score's associated probabilities (Table 3).

Discussion

In this study, we have described the development and evaluation of a screening tool that includes dietary factors for the assessment of AMD risk among individuals aged ≥55 years. The data on which the logistic regression model was based includes the results of a survey with the STARS questionnaire version 4.0, which was carried out at multiple centers throughout Spain. The results suggest that the model performed well (AUC = 0.75 [0.72; 0.79]) for identifying high risk of late AMD (AREDS score 2–4) in this patient population. 

In the absence of curative therapies for AMD, a major goal has been the early identification of risk factors of AMD. This strategy could help patients at risk to modulate modifiable risk factors that could delay the onset or slow the progression of their disease. In the STARS questionnaire version 4.0 age, presence of AMD in siblings, high cholesterol, and high BMI were evaluated as risk factors for late AMD. Age is naturally the most critical nonmodifiable risk factor in all populations studied. For example, a study of 14 different European populations showed that late AMD had a prevalence of 0.1% in people aged 55 to 59 years compared to 9.8% in people aged ≥85.³ In contrast, no gender-dependent differences have been found associated with late AMD.³ 

In the STARS questionnaire version 4.0, factors related to a high risk of developing late AMD included presence of AMD in siblings and in parents, highlighting the critical role of genetic factors in the development of AMD. Early studies with twins showed that 46% to 71% of the variation in the overall severity of AMD could be explained by genetics,²⁸ and the risk of AMD is greatly increased by having an affected first-degree relative.^29,30 A family history of AMD was also significantly associated with an increased risk for early and late AMD in the survey carried out with the original STARS questionnaire (odds ratio [OR] = 3.93 and OR = 6.99, respectively; P < 0.0001).¹⁸ Currently, there is a large list of genetic polymorphisms associated with AMD development, mostly related to genes involved in the complement system, extracellular matrix remodeling, and lipid metabolism.^31–34 

The STARS questionnaire version 4.0 also evaluated BMI, atherosclerosis, hypertension, hypercholesterolemia, prior myocardial infarction, and smoking habits. The original STARS questionnaire showed a significant increased risk for AMD in subjects with cardiovascular disease and risk factors, such as obesity, hypercholesterolemia, and hypertension, as in previous studies.¹⁸ Other studies suggested an association with cholesterol,³⁵ but other studies have not.³⁶ The association of BMI has been observed before and this is a parameter usually included in prediction models.^12,18,32,37 Smoking is also a notoriously important and widely reported predictive factor for AMD.^{12,32,36,38–40} Although regular physical exercise has been shown to be a protective factor for AMD,^37,32,41 this was not included in the questionnaire. However, recommendations about physical exercise are given in the web page when the results of the questionnaire are shown. 

Because diet is an important modifiable lifestyle factor, a strong interest has been placed in associating various dietary factors with AMD progression. In addition to studies more than 25 years ago, more recent studies of nutrition showed that adherence to a Mediterranean diet was associated with a reduced risk of incident late AMD,^23,24,26 and for this reason dietary factors were included in the STARS questionnaire version 4.0. The beneficial effects of the Mediterranean diet were confirmed in studies of populations in Portugal and Italy.^41,42 A few studies reported that moderate wine or other alcohol consumption was associated with decreased odds of developing AMD,^42–44 but some studies did not adjust for cigarette smoking. Other studies did not find a beneficial effect of moderate alcohol intake, but very heavy alcohol consumption (more than 3 standard drinks per day) was possibly associated with an increased risk of early AMD.^45,46 Many studies have shown a protective effect of eating fish, which is rich in omega-3 fatty acids, but it is not clear if omega-3 supplementation is beneficial. 

The major advantage of the model presented here is that it is based on self-assessment and on basic biomedical parameters that do not require detailed medical or ophthalmological analysis. Because it does not rely on genetic testing or fundus examination, which are the basis for most current risk models, there are few direct comparisons of its performance parameters. Previous studies showed the highest AUC was 0.68 in models with only age, gender, education, smoking, and BMI.^11,47 The AUC (0.75 [95% CI = 0.71–0.78]) in the model described herein is higher, but somewhat lower compared to previous models which include macular phenotype and genetics (ranging between 0.80 and 0.90),^9,11,12 but it is comparable to that of the original STARS questionnaire (0.78 and 0.72 for the testing and validation samples, respectively).¹⁸ The high negative predictive value in our study (81%) should ensure that most subjects with low risk of late AMD are correctly classified. Because the STARS questionnaire version 4.0 was designed as a screening tool, false negative results were minimized to avoid misclassification of patients at risk of developing AMD. The consequence of minimizing false negatives is the increase of false positives, however, these can be corrected in subsequent assessments. 

The study presented here has some limitations. The data on which the risk model was based was derived from a highly homogeneous population in Spain, which may not be representative of other populations. Previous epidemiologic studies have found differences in prevalence between geographic regions.² For example, Europeans have higher prevalence of late AMD than Africans (0.5% vs. 0.3%), and a higher prevalence of the geographic atrophy subtype than Asians, Africans, and Hispanics.² 

Although previous studies of prevalence of AMD in Europe have shown only small differences between countries,^48,49 country-specific validation of the STARS questionnaire would be desirable. Another limitation is that, although the questionnaire and web tool was designed to be used by primary care physicians, specialists, and the general public, it was validated on a population attending an ophthalmology clinic, and over-represents subjects already aware of their disease. This was evident by the fact that about 40% of the patients who completed the questionnaire were diagnosed with AMD. Applicability to the general population would therefore need to be validated in further studies. Finally, an important limitation is its self-reported nature of diet and medical history. The questionnaire was designed to be concise and easy to complete, and more detailed assessment of lifestyle patterns could, if necessary, be performed by the physician in those cases in which elevated risk is observed. The physician could also subsequently perform a more detailed examination of such parameters as BMI, blood pressure, or blood lipids to advise the patients on possible lifestyle changes or medication. Finally, it would have been preferable to test the model in a prospective dataset. 

In Spain, one of the objectives of the SERV, in collaboration with primary care and general practitioners, is to reduce blindness caused by AMD. The Society has proposed a mass screening program in the population, similar to those carried out to detect diabetic retinopathy or early colon or breast cancer, by means of non-mydriatic retinal photography. However, as there are >15 million people >55 years old in Spain, this strategy is not feasible at present. Prior selection by means of the STARS questionnaire could allow screening efforts to be directed to the population at greatest risk. This campaign would also increase knowledge of AMD, the risk factors involved, and encourage lifestyle changes that could help reduce the impact of AMD blindness in the long term. 

In summary, the screening tool presented here may be useful to evaluate the risk of later stages of AMD in patients aged ≥55 years without having an eye examination. It was designed to be used by primary care physicians, ophthalmologists, other healthcare professionals, and the general population as a rapid and basic assessment, with the objective of detecting those patients at high risk that should seek further evaluation. The role of primary care physicians to identify early signs of AMD in individuals at risk is critical.⁵⁰ The primary care physicians can provide timely referrals and encourage behavioral modifications that reduce risk factors. Prospective studies will be needed to determine the general applicability of the tools described here and their effectiveness in meeting these goals. 

Acknowledgments

The authors thank the participating ophthalmologists and their affiliations: Maribel López (Hospital Universitario de Valladolid, Valladolid), Francisco Cabrera López (Hospital Insular de Gran Canarias, Gran Canaria), Antonio Piñero Busamante (Hospital Virgen de Valme, Sevilla), Eduardo Esteban González (Hospital Virgen Macarena, Sevilla), José Manuel García Campos (Hospital Clínico Virgen de la Victoria, Málaga), Jose Ma Ruiz Moreno (Vissum Alicante, Alicante), Josep García Arumí (Hospital del Valle de Hebrón, Barcelona), Lluís Arias Barquet (Hospital de Bellvitge, L'Hospitalet de Llobregat, Barcelona), Laura Sararols (Hospital General de Cataluña, Barcelona), Rosa Piñón Mosquera (Hospital Universitario de Burgos, Burgos), Sara Velilla Oses (Hospital de San Pedro, Logroño), Belén Puente Luna (Hospital Virgen Blanca, Bilbao), Félix Armadá Maresa (Hospital de la Paz, Madrid), Alfredo García Layana (Clínica Universitaria de Navarra, Pamplona), Begoña Goldaracena Tanco (Hospital Virgen del Camino, Pamplona), Javier Montero Moreno (Hospital Río Ortega, Valladolid), Francisco Javier Ascaso Puyuelo (Hospital Clínico Lozano Blesa, Zaragoza), Clemen Torrón Fernández (Hospital Universitario Miguel Servet, Zaragoza), Miguel Serrano García (Hospital Universitario de Canarias, Santa Cruz de Tenerife), Inmaculada Sellés Navarro (Hospital General Universitario Reina Sofía´, Murcia), Luis Matarán Macía (Hospital Comarcal de Antequera, Antequera, Málaga), Saideh Khaouly Alonso (Hospital de la Esperanza, Barcelona), Mercedes Lorenzo Soto (Hospital Costa del Sol, Marbella, Málaga), José Mª Naranjo Rodríguez (Hospital Punta Europa, Algeciras, Cádiz), Teresa Torrent Solans (Hospital Josep Trueta, Girona), Demian Antolín García (Hospital Torrejón de Ardoz, Torrejón, Madrid), Roberto Martínez Rodríguez (Hospital Cabueñes, Gijón, Asturias), Javier Torresano Rodríguez (Hospital Un. Gregorio Marañón, Madrid), Mar Prieto del Cura (Hospital del Tajo, Aranjuez, Madrid), María Muxima Acebes García (Hospital Infanta Leonor, Madrid), Inés Contreras Martín (Hospital Universitario Ramón y Cajal, Madrid), Eduardo Solé Forteza (Hospital Juan XIII, Tarragona), Julián Fernández Ruiz (Hospital General Ciudad Real, Ciudad Real), María Aurora Muriel (Hospital de la Princesa, Madrid), Kazem Mousavi (Hospital Mutua Terrassa, Terrassa, Barcelona), Mª Josefa Domínguez Fernández (Hospital Gral La Mancha Centro, Alcázar de San Juan, Ciudad Real), Nélida Muñoz Sainz (Hospital Fundación Jiménez Díaz, Madrid), Mercedes García Torre (Ambulatorio Pontones, Madrid), Nuria Valdés Sanz (Hospital Puerta del Hierro, Madrid), Diana Iturralde Errea (Hospital Comarcal Alto Debra, Arrasate, Guipúzcoa), Pedro Romero (Hospital Sant Joan de Reus, Reus, Tarragona), Mouafk Asaad (Consorci Sanitari de Terrassa, Terrassa, Barcelona), Roberto Gallego Pinazo (Hospital Universitario La Fe, Valencia), and Maribel Fernández (Instituto Oftalmológico Gómez-Ulla, Madrid). 

The authors thank Francisco López de Saro (Trialance SCCL) for medical writing support with the preparation of this manuscript, funded by Théa Laboratoires. 

Disclosure: A. García-Layana, Allergan (C), Bayer (C), Novartis (C), Roche (C), and Théa Laboratoires (C); M. López-Gálvez, Allergan (C), Roche (C), Novartis (C), Bayer (C), and Ferrer (C); J. García-Arumí, Allergan (C), Bayer (C), Novartis (C), and Roche (C); L. Arias, Allergan (C), Bayer (C), Novartis (C), Roche (C), and Topcon (C); A. Gea-Sánchez, None; J.J. Marín-Méndez, None; O. Sayar-Beristain, None; G. Sedano-Gil, None; T.M. Aslam, Bayer (C), Théa Laboratoires (C), Allergan (C), Novartis (C), and Roche (C); A.M. Minnella, Théa Laboratoires (C); I.L. Ibáñez, None; J.M. de Dios Hernández, None; J.M. Seddon, Scientific Co-Founder and equity in Gemini Therapeutics, Inc; equity in Apellis Pharmaceuticals; Théa Laboratoires (C)