Identifying Content for an Item Bank to Measure the Quality-of-Life Impact of Myopia Refractive Interventions

Abstract

Purpose: To report on the content generation and item refinement phases for a myopia refractive intervention-specific quality-of-life (QoL) item bank that will be operationalized using computerized adaptive testing.

Methods: Myopia refractive intervention-specific QoL domains and items were generated from (1) a literature search of existing refractive–intervention QoL questionnaires; (2) semistructured interviews with myopic patients corrected using spectacles, contact lenses and/or refractive surgery (n = 32); (3) and myopia experts (n = 9) recruited from the Singapore National Eye Centre. After a thematic analysis, items were systematically refined and tested using cognitive interviews with 24 additional patients with corrected myopia.

Results: Of the 32 participants with myopia interviewed (mean ± standard deviation age, 35.6 ± 9.0 years; 71.9% female; 78.1% Chinese), 12 (37.5%) wore spectacles, 7 (21.9%) used contact lenses, and 20 (62.5%) had undergone laser refractive surgery. Initially, 912 items within 7 independent QoL domains were identified. After refinement, 204 items were retained, including those relating to mobility challenges and work-related difficulties that are not well-represented in current refractive intervention-specific questionnaires.

Conclusions: Through a rigorous item generation and selection process, we have developed a 204-item and 7-domain myopia refractive intervention-specific item bank that will now undergo rigorous psychometric testing to generate item calibrations for the validation of a novel computerized adaptive testing instrument designed for use in research and routine clinical practice.

Translational Relevance: Once psychometrically validated and operationalized using computerized adaptive testing, this myopia refractive intervention-specific instrument will enable researchers and clinicians to quickly and comprehensively assess the impact of myopic refractive interventions across seven QoL domains.

Introduction

Against the backdrop of a global shift toward individualized value-based care, it is becoming crucial to understand the impact of interventions for myopia, the leading cause of global visual impairment,¹ from the patients’ perspective in research and clinical practice.² These outcomes have traditionally been quantified using generic or vision-specific patient-reported outcome measures (PROMs).^3,4 However, because uncomplicated myopia is fully correctable with a range of refractive interventions available on the market, including optical corrections (spectacles, contact lenses) and laser refractive surgery (i.e., laser in-situ keratomileusis [LASIK]),⁵ the advantages that such interventions have over each other are mostly nonvisual, such as convenience.⁶ As such, vision-specific PROMs may not be sensitive enough to quantify these nonvisual patient-reported changes. This point is particularly important given the frequent and iterative improvements made to currently available optical corrections, such as the introduction of daily and extended wear contact lenses; and the rapid improvements in laser refractive surgery techniques such as the introduction of femtosecond LASIK (Femto-LASIK) and small incision lenticule extraction, which may have similar visual outcomes yet differ in patient-reported measures.⁵ 

Several myopia intervention-specific PROMs have already been developed and validated, such as the Contact Lens Impact on Quality of Life scale⁷ and the Patient-Reported Outcomes With Laser In Situ Keratomileusis questionnaire.⁸ However, they are fixed-length and able to only quantify the QoL impact of one or two specific types of interventions, limiting the ability to conduct comprehensive comparisons for existing and future intervention strategies.⁹ Although the Quality of Life Impact of Refractive Correction questionnaire is brief (20 items)¹⁰ and has been shown to be able to differentiate between the QoL impact of the three main refractive interventions,¹¹ it has a complicated scoring algorithm that makes it difficult to use in busy clinical settings. There is, therefore, a need for a quick, comprehensive, and valid refractive intervention-specific PROM that can be scored and reported in real time to measure and compare the QoL outcomes for the different myopia interventions currently available. 

These issues can potentially be addressed with item banking and computerized adaptive testing (CAT), an integrated approach to generate instruments (questionnaires) that are able to comprehensively and efficiently quantify QoL estimates using relatively few items (questions), resulting in reduced test length and administrative burden.^12–16 Although refractive error-specific item banks have been developed in a combined Australian and Nepalese population,¹⁷ these item banks were designed to broadly assess QoL issues across a range of corrected and uncorrected refractive errors, which limits their ability to quantify intervention-related changes. In this study, we therefore developed myopia refractive intervention-specific item banks to assess the impact of myopia interventions on all aspects of QoL by leveraging previously published literature, together with qualitative data collected in Asian persons with myopia corrected with refractive interventions (MyoRI-CAT). We report here on the initial phases of generating and refining items and domains for the item banks, and also compare the content coverage of our novel item banks with that of existing qualitative myopia studies and available validated refractive error PROMs used in the assessment of myopia intervention-specific QoL. 

Methods

Study Design and Population

Qualitative data from 21 English- and/or Mandarin-speaking patients aged 21 years or older with corrected myopia, defined as less than −0.50 diopters spherical equivalent (sphere plus half negative cylinder) in one or both eyes and either using optical correction (spectacles and/or contact lenses) or who had undergone laser refractive surgery, were extracted from a qualitative study conducted between 2013 and 2015 at the Singapore National Eye Centre, the largest tertiary ophthalmic care center in Singapore, to understand the QoL impact of visual impairment and associated eye diseases in Asian individuals (VisBank, CIRB#2010/573/A). Realizing the need for more contemporary patient data, as well as feedback from the treating clinicians and allied health practitioners (AHP), we recruited another 11 patients and 9 clinicians/AHP between September 2020 and September 2021 (MyoRI-CAT, CIRB #2020/2322). A more detailed overview, including the definition and assessment of visual impairment and myopia, study exclusion criteria, and sampling strategy, is provided in the Supplementary Materials. 

Both studies received ethical approval from the SingHealth Centralized Institutional Review Board and the study protocols were conducted in accordance with the Declaration of Helsinki. 

Development of the Myopia Refractive Intervention-Specific QoL Item Bank

Phase 1: Domain and Item Generation

Literature Review

We searched PubMed and Google Scholar databases and the bibliographies of relevant English language papers for the impact of myopia refractive interventions on QoL. The search terminology used is listed in Supplementary Table S1. These findings, together with preliminary analyses of VisBank transcripts, were then used to develop our MyoRICAT qualitative moderator's guide comprising key interview questions, subquestions, and prompts for participants with myopia, researchers, and clinicians (see the Supplementary Materials), as well as our comparison of the content coverage between our item banks and that of published qualitative studies and validated refractive error PROMs used in myopia intervention research. The moderator's guide was translated into Mandarin by certified translators. 

Qualitative Data Collection

The VisBank study used face-to-face focus groups and semistructured interviews to obtain qualitative data, whereas the MyoRI-CAT study used only telephone-based semistructured interviews owing to the ongoing coronavirus disease 2019 pandemic. These interview sessions were held in accordance with the Consolidated Criteria for Reporting Qualitative Research Criteria,¹⁸ and the questions posed to participants were designed to align with the World Health Organization definition of QoL.¹⁹ The 21 individuals with corrected myopia from the VisBank study participated either in 1 of 3 focus groups (2 in English with 5 participants each, 1 in Mandarin with 4 participants; average length, 42 minutes) or one of 87 semistructured interviews (all in English; average length, 26 minutes). Of the 17 participants recruited for MyoRI-CAT, 11 were interviewed in English (average length, 34 minutes). Participants were asked open-ended questions about how their myopia and its associated interventions had affected different aspects of their QoL, with the interviewers summarizing the main points raised by the participant(s) after each focus group or interview and identifying themes that emerged. Interviews were conducted until thematic saturation was reached, that is, once two consecutive focus group and interviews were completed without any new themes emerging. The interview sessions were audio recorded and transcribed verbatim by a professionally certified transcription company, with Mandarin transcripts translated into English. Translated transcripts were then back-translated and checked for accuracy by the interviewers, who were proficient in both English and Mandarin (minimum of a “B” or better grade on the Singapore–Cambridge General Certificate of Education Ordinary Level [O-Level] examinations taken at Secondary 4 or Grade 12 equivalent). 

Two researchers (K.G. and J.L.) analysed participant transcripts separately and health care practitioner feedback using the constant comparative method.²⁰ Participant quotes were analysed for relevance to myopia interventions and any disagreements in coding were resolved through discussions with three senior researchers (E.L.P.X., R.M., and E.F.). 

Phase 2: Item Refinement

Binning and Winnowing

To develop minimally representative pilot item pools for the myopia intervention-specific item banks, we used a validated process of item binning (grouping items similar in content or meaning) and winnowing (identifying potential items for removal) using systematic criteria.²¹ These processes were undertaken by two researchers (K.G. and J.L.) under the guidance of the expert panel comprising a psychologist (E.L.P.X.), psychometricians (E.F. and E.L.L.), and clinicians and myopia researchers (R.M., S.S.M., W.C.W., M.A., and Q.V.H.). 

Cognitive Interviews

After the finalization of the item pools, we tested the preliminary instrument using cognitive interviews,²² which ensures that the items and instructions to participants are worded in a lay language; are clear, comprehensible, and culturally relevant; and are interpreted by participants as intended before actual pilot instrument testing.²³ Because it was not viable to explore all 213 items (after binning and winnowing; see Results) in detail, a list of 16 items with a high potential for interpretation issues (i.e., complex concepts, technical examples, and new items not generated in other instrument development work) were chosen for cognitive interview testing. 

We used the think aloud and verbal probing techniques for the cognitive interview process.²⁴ To mitigate the effect of item ordering,²⁵ we randomized the sequence of questions for every participant. Summarized feedback from every batch of three interviews was reviewed by the study team members and the items were discussed during a consensus meeting with regards to (a) item amendment (b) item retest, (c) item removal, or (d) item confirmation and removal from testing (if no significant issues arose for two consecutive batches). 

Results

Phase 1: Domain and Item Generation

Qualitative Interviews

Of the 21 VisBank participants (mean ± standard deviation [SD], age 32.71 ± 7.12 years; range, 23–44 years; 71.4% female; 66.67% Chinese), 5 (22.8%) wore spectacles, 4 (19.0%) wore contact lenses regularly, and 16 (76.2%) had undergone laser refractive surgery. None of the participants had visual impairment while wearing their habitual correction. Seven (33.3%), 10 (47.6%), and 4 (19.1%) patients had mild, moderate, and high myopia in the worse eye, respectively (Table 1). 

Table 1.

 

Sociodemographic Characteristics of Individuals With Myopia Participating in Qualitative Interviews (N = 32)

 

For the 11 MyoRI-CAT participants (mean ± SD age, 41.6 ± 12.5; range, 26–54 years; 72.7% female; 90.9% Chinese), 7 (63.6%) wore spectacles, 3 (27.3%) used contact lenses regularly, and 4 (36.4%) had undergone laser refractive surgery (Table 1). None of the participants had presenting binocular visual impairment, although 3 (27.3%), 3 (27.3%), and 5 (45.5%) had mild, moderate, and high myopia in the worse eye, respectively (Table 1). 

Of the nine interviews with clinicians and AHPs (mean ± SD age, 35.1 ± 7.6 years; range, 26.0–45.5 years; 55.6% female; 88.9% Chinese; 8.7 ± 5.4 years of professional experience), three were ophthalmologists (33.3%), five were optometrists (55.5%), and one (11.1%) was a psychologist involved in myopia research (Supplementary Table S2). All clinicians and AHPs were familiar with the range of myopia treatment options (both optical and surgical) available to patients. 

A total of 665 (339 from VisBank and 326 from MyoRI-CAT) and 200 items were extracted from the 32 patients with myopia and 9 health care practitioner transcripts, respectively (Table 2). 

Table 2.

 

Items Generated Across Four Sources for Item Generation

 

Literature Review

With the approach outlined in the methods, we identified an additional 47 extant items from 4 published qualitative interviews and 15 validated refractive error PROMs,^8,9,26–30 that the expert panel deemed to be related to the impact of myopia refractive interventions on an individual's QoL (Table 2). 

Domain Generation

Using both a top-down approach, as informed by the World Health Organization definition of QoL,¹⁹ previous item banking work,^15,17,31 and our comprehensive literature search, as well as a bottom-up (data-driven) strategy to categorize similar items into distinct domains of QoL, we isolated seven key domains of myopia refractive intervention-specific QoL from our item pool, namely activity limitation (AL), emotional (EM), comfort (CM), convenience (CV), concerns (CN), mobility (MB), and work (WK). Findings for each domain are outlined below, with supporting quotes provided in Supplementary Table S3. 

AL

Participants using prescriptive spectacles and/or contact lenses almost unanimously reported having difficulty carrying out activities that made it inconvenient for them to wear their correction, such as swimming. Lighting issues, including halos from street lights and glare from incoming headlights, were also reported, from both spectacle and contact lens wearers and, more rarely, from individuals who underwent laser refractive surgery. 

EM

Many participants described experiencing negative emotions (e.g., embarrassment, frustration, self-consciousness) and reported reluctance to carry out certain tasks (e.g., socializing or wearing their myopia correction socially at night). 

CM

Spectacle wearers often reported discomfort from wearing their correction (e.g., spectacle-related soreness around their nose and ears). Moreover, many contact lens users after long hours of correction use and some participants who had undergone laser refractive surgery experienced visual and/or ocular symptoms (e.g., blurred vision, dry eyes, redness, and tired eyes). 

CV

Many participants wearing spectacles and/or contact lenses highlighted having difficulty completing certain tasks when not wearing their correction (e.g., having trouble finding their correction after just waking up), as well as difficulty finding suitable types of prescriptive eyewear (e.g., prescriptive sunglasses and goggles). Having their correction slip off or fog up, having to remove or put on their correction (e.g., when applying make-up), and having to maintain backup correction (in case the first one became misplaced or damaged) were also common inconveniences associated with myopia correction use. 

CN

Some participants were worried about being able to adapt to new correction (spectacles or contact lenses), or laser refractive procedures not meeting their perceived expectations. In addition, participants were concerned with the upfront cost of laser refractive surgery and/or the ongoing costs associated with correction upkeep (e.g., replacement of spectacles or contact lenses). 

MB

A number of participants, particularly individuals wearing thick prescriptive lenses and those who underwent laser refractive surgery, reported having issues with moving around in dim lighting (e.g., walking at night or in dimly lit areas), while using steps or stairs (particularly if they were of a uniform color), and when navigating difficult terrain (e.g., avoiding obstacles or uneven ground). 

WK

Several issues that impacted participants’ work performance were highlighted, including the need to adjust their correction while wearing external optical aids required for work tasks such as protective goggles, and reduced efficiency in completing work tasks. Some participants also mentioned concern about having their choice of refractive intervention compromising their career progression (e.g., having to take time off work owing to refractive surgery procedures and follow-up appointments). 

Item Generation Summary

At the end of phase 1, we generated a total of 912 items across 7 independent domains of QoL (AL, n = 162; CM, n = 102; CN, n = 272; CV, n = 156; EM, n = 144; MB, n = 33; and WK, n = 43). 

Comparison With Existing Qualitative Studies

Many myopia intervention-specific QoL issues, such as intervention-related discomfort and psychosocial CN (e.g., low self-esteem and a feeling of inadequacy when wearing glasses), reported by our multiethnic Asian population were similar to those reported in the four qualitative studies conducted in three Western and one Asian population in the literature.^27–30 Key differences were the additional emphasis raised by our participants on MB and WK-related issues. 

Phase 2: Item Refinement

Binning and Winnowing

We organized two expert panel sessions where every item generated was evaluated and either retained, shifted to another domain, rephrased, or removed. To allow this PROM to be applicable to people with myopia across the various types of correction and treatment, we used the term correction to describe the different types of optical corrections currently available on the market (i.e., spectacles, soft and hard contact lenses), and treatment to mean laser refractive surgery. After the two sessions of binning and winnowing, the number of items was decreased from 912 to 213 (Table 3). 

Table 3.

 

The Process of Refining the Initial Item Pools to the Final Seven Item Banks

 

Development of Item Stems, Preceding Statement, and Response Options

Based on previous item banking work^15,17 and empirical evidence,³² preceding statements (e.g., “Because of your myopia correction/treatment...”), item stems (e.g., “how often do you feel…”) and response options (4-/5-point Likert-type responses) were developed. 

Cognitive Interviews

For the 24 cognitive interview participants (mean ± SD age, 55.4 ± 15.3 years; 70.8% female; 100% Chinese), 21 (87.5%) wore spectacles, 3 (12.5%) wore contact lenses regularly, and 5 (20.8%) had undergone refractive surgery (LASIK, laser-assisted subepithelial keratectomy, or photorefractive keratectomy) (Table 4). Six (35.3%), 4 (23.5%), and 14 (82.4%) patients had mild, moderate, and high myopia, respectively (Table 4). 

Table 4.

 

Sociodemographic and Clinical Characteristics of Cognitive Interview Participants (N = 24)

 

After the cognitive interviews, two items were maintained without change, three items were deleted, one item was merged with a similar item, one item was recategorized into another domain, and nine items were rephrased. We next did a pretesting of the full item banks with the study team, as well as their friends, family members, and colleagues (n = 18), to check for any issues in the remaining items. In this pretesting phase, one item was merged with a similar item and four items were deleted. With these adjustments, the final instrument contained 204 items across 7 independent domains of QoL (AL, n = 42; CM, n = 20; CN, n = 53; CV, n = 25; EM, n = 34; MB, n = 17; and WK, n = 13) (Table 5). 

Table 5.

 

Description of Domains, Item Stems, and Response Options in the Final Myopia Intervention-Specific Item Banks

 

Comparison With Existing Refractive Error QoL PROMs

QoL issues belonging to the AL, EM, and CM domains were relatively well-represented in the 16 validated refractive error PROMs included in our literature review,^{7,8,10,17,27,33–43} although the number of items within each of these domains was quite limited (median, 10 items; range, 1–80 items; Table 6). Conversely, the CN, CV, MB, and WK domains were largely under-represented. Although the refractive error item banks developed by Kandel et al.¹⁷ covered similar QoL domains, none of the items were specific to myopia interventions. 

Table 6.

 

Content Coverage of Our MyoRICAT Item Banks Compared With Validated Refractive Error Quality-of-Life Questionnaires^a

 

Discussion

Our team generated 204 items across 7 myopia intervention-specific QoL item banks that will enable the comprehensive assessment of a range of functional ALs, psychosocial challenges, inconveniences, MB issues, and WK CN associated with spectacles, contact lenses, and laser refractive surgery previously reported qualitatively. Once calibrated and operationalized via CAT, these item banks will offer a rapid and precise way to quantify myopia intervention-specific QoL across the range of management modalities. 

Although many challenges related to refractive interventions, such as soreness around the eyes and ears related to prolonged spectacle wear, and dryness and discomfort resulting from contact lens overwear and/or laser refractive surgery, are covered to some extent by current refractive error PROMs,^17,44 most of these instruments are designed to assess the QoL impact of a single form of intervention only, precluding a more complete and holistic comparison between the different forms of refractive interventions currently available. Our myopia refractive intervention–specific item banks, once validated, will allow for a comprehensive comparison of the QoL burden of the different myopia interventions from the patients’ perspective, something that is critically needed in clinical trials to inform intervention efficacy. 

Many of our working participants raised work-related difficulties and concerns, such as having to take time off work for eye appointments/treatment and difficulty wearing their usual correction (e.g., spectacles) while working owing to the need for specialized eye protection. These issues are not well-covered in existing questionnaires, which is a critical gap, given the rapidly ageing global population and the resultant need to empower older adults to remain in, or re-enter the workforce.⁴⁵ As such, we have generated a dedicated item banking covering these work-specific issues, which will allow for the quantification of work-related challenges associated with the different myopia interventions. These data can then be used to inform government schemes and health care policies to help people with myopia intervention-related issues to remain employable. 

Interestingly, several of our participants, particularly those with high myopia wearing glasses and individuals who have undergone laser refractive surgery, reported MB issues that were particularly evident at night. Given that MB issues are not well-represented in current refractive error questionnaires, with most focusing on AL challenges,^{7,8,10,17,27,33–42} our MB domain will enable a more comprehensive understanding of the impact of the different myopia interventions on an individual's MB and independence. 

As a part of the ongoing WK to develop the MyoRI-CAT, we will be administering these items to a group of participants across the various refractive interventions, which will then allow us to evaluate the psychometric validity of the different item banks, as well as obtain the relative item difficulty rankings (calibrations) for each individual item, using Rasch analysis. CAT algorithms will then use these calibrations to present items targeted to a respondent's ability level based on their previous responses, allowing for a precise estimate of intervention-specific QoL with relatively few items being administered, resulting in time savings of up to 80%.⁴⁶ Importantly, the challenge with having items that may not be applicable to all respondents (e.g., intervention-specific CN) are easily circumvented in a CAT instrument by having a precursor question relating to the type of intervention or treatment the respondent is currently on, with the CAT algorithm tasked to administer only these intervention-specific items. Furthermore, item banking and CAT allow new items to be added by simply estimating the calibration of these new items relative to existing ones using a Rasch analysis,⁴⁷ hence future proofing its use for novel intervention strategies. The item banks and resulting CAT system will be hosted on a cloud-based platform (PROMinsight) using Concerto open source software that can be administered from any internet-enabled digital device, enabling real-time scoring and reporting of data.⁴⁸ 

Our sampling strategy, which enabled us to explore in-depth and detailed qualitative information from health care practitioners and participants across the different corrective interventions, is a key study strength that increases the generalizability of our data. Our thematic analysis and item generation process also followed a systematic and standardized protocol⁴⁹ and were guided by empirical evidence.³² Moreover, we conducted extensive cognitive interviews to improve item comprehensibility. Limitations include a relative paucity of participants of Malay and Indian ethnicities and those of male gender, which could mean that we may have missed out some key ethnic- or gender-related challenges. The flexibility of item banking and CAT, however, offers the option of adding additional relevant ,items as detailed previously.⁴⁷ Moreover, we decided not to include QoL challenges associated with behavioral, pharmacological, or pathological myopia–related interventions, because the target population for these interventions are very different from that of our clinical sample (young children for behavioral and pharmacological interventions; older adults with pathological myopia). Finally, we decided to omit items related to presbyopia, such as difficulties with near WK, because inclusion might lead to problems with model fit, targeting, and measurement integrity because these items will not be applicable to the prepresbyopic population. Last, the relative paucity of contact lens users in our participant pool for both the qualitative and cognitive interviews may suggest that we could have missed out key QoL challenges associated exclusively with contact lens wear. However, we used a strict study methodology to decrease this risk, including ceasing recruitment of particular patient groups only after two consecutive interviews, comprising on average two to three pages of qualitative data, failed to bring up new QoL issues (thematic saturation). We also supplemented our participant data with a comprehensive review of published qualitative WK and contact lens-specific questionnaires (e.g., CLIQ)⁷ to further minimize the risk of missing out on key contact lens-related QoL issues. Moreover, none of the items that underwent cognitive interview testing for comprehensibility were contact lens specific; as such, the small number of contact lens wearers in this group is unlikely to have affected the integrity of the cognitive interview process. 

In conclusion, our MyoRI-CAT QoL instrument, comprising 204 items across 7 domains of QoL, allows for a better understanding of the impact of myopia refractive interventions on patients’ QoL. Future WK will focus on evaluating the psychometric properties of these item banks in a large population of patients with myopia using or undergoing the different types of refractive interventions, and obtaining item calibrations to develop a CAT system that can quickly, yet comprehensively, assess the QoL impact of these refractive interventions. 

Acknowledgments

Supported by the National Medical Council (NMRC) Transition Award (#MOH-TA19may-0002 – Ryan Man). Lamoureux is supported by the NMRC Senior-Clinician Scientist Award (#NMRC/CSASI/0009/2016). The funding body had no role in the design and conduct of this research. 

Disclosure: R.E.K. Man, None; K.J.L. Goh, None; E.P.X. Lee, None; J.H.X. Lim, None; M. Ang, None; Q.V. Hoang, None; C.W. Wong, None; S.M. Saw, None; E.K. Fenwick, None; E.L. Lamoureux, None