Validation of the Waterloo Differential Visual Acuity Test (WatDAT) and Comparison With Existing Pediatric Tests of Visual Acuity

Abstract

Purpose: The new Waterloo Differential Acuity Test (WatDAT) is designed to allow recognition visual acuity (VA) measurement in children before they can typically undertake matching tests. The study purpose was to validate WatDAT in adults with normal and reduced VA.

Methods: Eighty adults (18 to <40 years of age) participated (32 normal VA, 12 reduced VA, and 36 simulated reduced VA). Monocular VA was measured on two occasions in random order for WatDAT (versions with 3 and 5 distractors for Faces and Patti Pics house among circles), Lea Symbols, Kay Pictures and Patti Pics matching tests, Teller Acuity Cards, Cardiff Acuity Test, and Early Treatment Diabetic Retinopathy Study (ETDRS) letter chart. Pediatric tests were validated against ETDRS using limits of agreement (LoA), sensitivity, and specificity. The LoA for repeatability were also determined.

Results: WatDAT showed minimal bias compared with ETDRS, and LoAs, which were similar to pediatric matching tests (0.241–0.250). Both preferential looking tests showed higher bias and LoAs than ETDRS. Matching tests showed good agreement with ETDRS, except for Kay Pictures and Lea Uncrowded test, which overestimated VA. WatDAT showed high sensitivity (>0.96) and specificity (>0.79), which improved with criterion adjustment and were significantly higher than for the preferential looking tests. LoA for repeatability for WatDAT 3 Faces and WatDAT 5 Faces were comparable with the ETDRS.

Conclusions: WatDAT demonstrates good agreement and repeatability compared with the gold-standard ETDRS letter chart, and performed better than preferential looking tests, the alternative until a child can undertake a matching VA test.

Translational Relevance: Good validity of the Waterloo Differential Acuity Test was demonstrated in adults as a first step to showing its potential for detecting childhood visual disorders.

Introduction

The early detection of treatable vision disorders such as amblyopia and congenital cataracts is important because these conditions may lead to permanent vision loss if not treated early.^1–3 Visual acuity (VA) is a key component of a visual examination, alerting the clinician to the presence of high refractive error, amblyopia, or eye disease and aids in understanding the visual function status of the patient. 

However, not all tests of VA are equally effective to detect these disorders. VA can be categorized into tests of recognition or form VA and resolution VA.⁴ Tests of recognition VA require the patient to recognize and identify the form of the optotypes presented, whereas resolution VA requires the patient to detect or optically resolve the target, but not to identify it.⁵ Preferential looking (PL) tests, such as the Cardiff Acuity Test (CAT) and the Teller Acuity Cards (TAC) utilize resolution VA and were developed for babies and young children who have not yet developed the cognitive skills to undertake a test of recognition. 

Compared with recognition acuity, resolution acuity is less sensitive for detecting amblyopia,^4,6 optical blur,^7–9 and other causes of reduced VA, because it overestimates acuity compared with recognition acuity.^4,6,10 Therefore, it is ideal to measure recognition VA as early in life as possible. Current pediatric clinical tests use a matching paradigm for young children who are not able to verbalize letters, shapes, or pictures. These tests are possible with typically developing children from the age of 2.5 to 3.0 years of age and upward. 

To allow measurement of recognition acuity at an earlier age, Leat et al.⁵ developed and validated the concept of “differential acuity,” which requires the observer to detect the position of one “odd-one-out” target among similar distractors. This task is thought to be cognitively simpler than matching a target; children 12 months of age can undertake simple search tasks and children of 2.5 years can choose an odd-one-out colored spot.^11–13 Based on this concept, the Waterloo Differential Acuity Test (WatDAT) was specifically designed for children aged 18 months to approximately 2.5 to 3.0 years of age, although it could be used up to the age when a child can undertake a letter test. The testability of the task in young children has been demonstrated. Most young children of 18 months upward were able to undertake the task with the Patti Pics target (Patti Pics house among circle distractors) and children of 22 months upward were able to do the task with a face versus a nonface target.¹⁴ 

Validation of new tests is essential before they are introduced clinically and is typically undertaken by considering agreement with existing clinical tests, which are commonly used or accepted as a gold standard. Agreement between two tests is typically measured in terms of bias (the average difference between two tests),^9,15–17 the limits of agreement (LoA), based on the standard deviation of individual differences between the two tests,^5,16,18–20 and sensitivity and specificity for detecting reduced VA.^6,8,21–23 Test–retest repeatability of the new test is also an important measure.^19,20,24,25 A good test of VA will perform well on all these metrics. 

This study aimed to evaluate a newly developed VA test, WatDAT, in people with poor vision in one or both eyes and in those with normal vision. Ultimately, the goal of this new test is to allow measurement of recognition VA in children younger than 3 years, which is not possible with currently available tests. First, we determined the validity of WatDAT with adult observers who are able to undertake the gold-standard adult test (Early Treatment Diabetic Retinopathy Study [ETDRS]).^26,27 Because there are few studies that compare a larger number of pediatric tests with each other, or with the ETDRS, we have also included three pediatric matching tests (five versions) and two tests of PL, the CAT and the TAC. In particular, there are few studies that have measured the sensitivity and specificity of pediatric tests.^6,21,22 WatDAT and all other pediatric tests were evaluated against the ETDRS for agreement, test–retest repeatability, sensitivity, and specificity. 

Material and Methods

Participants

Inclusion criteria were adults aged 18 years to <40 years old who had either normal corrected VA (with contact lenses or spectacles) or reduced VA owing to conditions such as amblyopia or other disorders. Exclusion criteria were any medical diagnoses or medications that might result in fluctuating vision (such as diabetes); cognitive impairment, developmental delay, neurodevelopmental disorder, or brain injury (e.g., concussion with on-going symptoms); or reported unstable or fluctuating vision. For the normal vision group, additional exclusion criterion were high spherocylindrical refraction (>+3 diopters [D] hyperopia, 5 D or more of myopia, or >1.5 D astigmatism), or >1.5 D anisometropia, strabismus, or any disorder causing vision loss in the eye to be tested. 

We aimed for a total sample size of ≥70, to include 30 participants with normal VA, 30 with simulated reduced VA, and 10 with actual reduced VA due to a disorder. The sample size was based on a review of other laboratory-based validation studies of the agreement between VA tests which included adults. The sample sizes varied between 20 and 128.^{6,9,16,28–32} Participants were recruited from the optometry clinics, among the staff and students, and through lists of potential participants held by the Centre for Ocular Research and Education (CORE), School of Optometry and Vision Science at the University of Waterloo. The study was reviewed and approved by the University of Waterloo Research Ethics Board. All eligible participants gave written consent to participate in the study and the study adhered to the tenets of the Declaration of Helsinki. The study took place at the University of Waterloo, School of Optometry and Vision Science where participants attended for two study visits within a 2-week interval. 

WatDAT Specifications

WatDAT software, developed at the School of Optometry and Vision Science, University of Waterloo, ran on a Raspberry Pi computer. Targets were presented on a Corkea 4K 14-inch 3840 × 2160 resolution touch screen. The software allows the examiner to choose the target type (Faces or Patti Pics targets [Precision Vision, used with permission]) and the number of distractors. The range of acuity available was logarithm of the Minimum Angle of Resolution (logMAR) 1.7 to 0.3 for Face VA targets and 1.7 to 0.2 for the Patti Pics house among circles (HC) targets at a 30-cm viewing distance, which could be extended by increasing the viewing distance. The spacing between the inner edges of the targets remained constant at 43.6 mm (8.27° at 30 cm). This spacing was chosen so that the targets gave the perception of remaining in the same position and to allow a constant active touch area of 24 mm (4.57°) around each target (Fig. 1). For each presentation of Face VA, the target was a schematic face target randomly positioned among either three or five nonface distractors and for HC the target was a house among circles. These four versions were developed, because it was not known a priori which version would result in the best response from young children. 

Figure 1.

 

Faces and House/circle symbols used as the WatDAT targets. Permission to use Patti Pics house symbols from Precision Vision.

General Procedure

Corrected monocular VA was initially measured with the participant's current vision correction (spectacles or contact lenses if required) using the ETDRS chart R (Precision Vision) at a 4-m distance. Chart R was not used for the outcome VA measurement, but for the initial VA measurement and refraction. Ocular alignment was evaluated with a unilateral cover test to indicate presence and type of strabismus. For the normal vision group and those with simulated low vision, one eye was picked randomly as the study eye by flipping a coin. For those with monocular vision loss (e.g., unilateral amblyopia), the eye with reduced vision was selected as the study eye. For those with reduced VA in both eyes, an eye to be tested was chosen to give a good spread of VA among the participants. Once the study eye was selected, the refraction of that eye was verified by subjective trial frame refraction. For those wearing contact lenses, an over-refraction was performed. The final corrected VA of the chosen eye was remeasured with ETDRS chart R. For those in the simulated low vision group, a Bangerter filter (with nominal density ranging from 0.2 to 0.6) was placed in front of the study eye to reduce VA. We aimed for a range of ETDRS VA from 6/9 (0.2 logMAR) to approximately 6/36 (0.8 logMAR), with the reasoning that a VA poorer than this would easily be detected by all the charts and would not add value to the sensitivity and specificity measures and that this gives a good range for estimating the LoA for people with normal and reduced VA. 

Using the study eye with the best-corrected refraction in place, VA was measured for the following VA tests: Lea Uncrowded and Lea Massachusetts Visual Acuity Test (MassVAT) cards (Good-Lite Vision), Kay Pictures (Kay Pictures), Patti Pics Uncrowded and Patti Pics MassVAT cards (Precision Vision), TAC (Precision Vision), Cardiff Acuity Cards (Good-Lite Vision), ETDRS (Precision Vision), WatDAT with three distractors (WatDAT3), and WatDAT with five distractors (WatDAT5) and with either Face or House/circle symbols.¹⁴ For efficiency, the following tests were paired together: the two Patti Pics tests, the two Lea tests, the PL tests (Teller and Cardiff), WatDAT with three distractors, and WatDAT with five distractors. These paired tests, together with the ETDRS and the Kay Pictures resulted in seven test types. The order of these test types was according to a predetermined random order and was separately randomized for the first and second study visits. Within each pair of the paired tests, the order was also randomized using a flipped coin. All tests used high-contrast black symbols on a white background between 80 and 120 cd/m² measured with a Konica-Minolta photometer (Ramsey, NJ). One experimenter administered all the tests. 

For all acuity tests, the participant was asked to guess if they were uncertain. For all VA tests except the ETDRS, there were two phases (phase 1 and phase 2). Phase 1 determined the approximate threshold and phase 2 refined the exact threshold. Testing started with presentation of 0.7 logMAR targets (0.8 for the TAC) for those with normal VA and approximately two to three octaves above the initial ETDRS VA for those with reduced acuity. When a level during phase 1 was failed, the experimenter selected 0.2 logMAR steps higher to start phase 2. To reach a true threshold for adult participants, the testing distance was increased for the smaller acuity levels. 

Patti Pics, Kay Pictures, Lea Matching Tests, and WatDAT

For the Patti Pics, Kay, and Lea matching tests, the participant was first shown the matching card so they were aware of the options of symbols or pictures in each test. Then the experimenter moved back to the 3-m testing distance and the participant was asked to name the symbol shown. For acuities better than the smallest available for each test, the experimenter moved back to 6 m. During phase 1, one symbol was presented at each level. Testing continued until the participant made an error, at which point the experimenter started phase 2. Phase 2 started at two logMAR levels above the first error. During phase 2, a total of four symbols were shown at each level (including those shown in phase 1). The exception was Patti Pics MassVAT and Lea symbols MassVAT, which have five symbols at each acuity level, and for these a total of five symbols per level were presented, because this is how the test is designed to be used. At the start of phase 2, the experimenter ensured that the participant achieved a line fully correct, and testing continued until only one or zero symbols were named correctly or until the smallest acuity level was reached. 

WatDAT

For both the Faces or HC, with either three or five distractors, the participant was asked to touch the schematic Face or house among the distractors. Initially, the experimenter presented a large demonstration version of the task (1.7 logMAR) to familiarize the participant with the task and then continued with phase 1 and 2 as described elsewhere in this article. The initial testing distance was 30 cm. To measure acuity better than 0.3 logMAR for Faces and 0.2 logMAR for Patti Pics, it was necessary to increase the testing distance to 60 cm owing to limitations in screen resolution, and then to 120 cm for acuities better than 0.0 and −0.1, respectively. The correct distance for the participant's eyes from the screen was marked and monitored by the experimenter. At the longer distances, participants verbalized the position of the target and the experimenter entered the response. 

PL Tests (Teller and Cardiff)

The experimenter randomized the position of the grating for the TAC by rotating the card without looking at the grating side, before presenting the card, and the participant was asked to state the position of the grating. The three CAT cards were shuffled, and the experimenter did not look to see the position of the picture until the participant had responded. After three presentations, the cards were reshuffled to randomize the position of the picture such that the experimenter did not know the position of the picture for the fourth presentation. The initial testing distance for the TAC was 84 cm and the starting level was 3.2 cy/cm (0.8 logMAR). The initial testing distance for the CAT Cards was 1 m, starting with the 0.7 logMAR card. Because the guessing rate for these tests is 50% (as opposed to 20% or 25% for the matching tests), two cards at each acuity level were presented during phase 1. Both responses needed to be correct to proceed to the next smaller acuity level. If the first presentation was incorrect, the experimenter would not proceed. For phase 2, a total of four presentations was shown at each level (including levels tested during phase 1). At the start of phase 2, the experimenter ensured that the participant achieved four correct responses at one acuity level and testing continued until only one or zero were correct or until the smallest acuity level was reached. 

ETDRS Test

The ETDRS was administered at a 4-m distance, using a different chart on visits 1 and 2 (the order was randomized for each participant). Participants were asked to read the letters from left to right, starting with the smallest line they could easily read. If the participant made an error and corrected themselves before reading the subsequent letter, the letter was counted as correct. If they read the subsequent letter and then corrected themselves on the initial error, it was counted as an error. The experimenter ensured that there was at least one line with all five letters correct. The testing continued until the participant correctly read one or no letters on a line. This stopping rule was used because previous experience with participants with low vision indicated that it is more accurate to use a stopping rule of one or no correct responses. Participants with low vision can often correctly identify several more letters after they get three wrong on a line. Also, it is consistent with the other VA tests used in this study. Each letter was marked as correct or incorrect. 

Data Analysis

The final acuity was calculated by-presentation, giving each correct response a logMAR weighting of 0.025 when there were four presentations at each level, or 0.02 when there were five. By-letter scoring was used to calculate the final VA for the ETDRS chart.²⁶ Because the TAC does not have exactly equal logMAR steps between acuity sizes,³³ the by-presentation acuity was calculated by taking one-fourth of the step size between the acuity levels in question. For example, the log step between 0.03 logMAR and −0.11 logMAR is 0.14 log units. So, each response between these levels was given a weight of 0.140/4.000 = 0.035. 

Bland–Altman plots were created for each test in comparison with the ETDRS. The 95% LoA (1.96 × standard deviation of the differences) and mean difference (bias) were calculated. Scatterplots were generated, and Spearman's rank correlation coefficient was used to calculate correlation, as VA data were not normally distributed. Test–retest repeatability for each test was determined with the mean bias and LoA for repeatability (LoA-R). Sensitivity and specificity for each test was determined using different cut-offs for normal or abnormal VA. Normal or abnormal grouping was determined by the ETDRS, using a cut-off of 0.1 (normal VA) versus 0.11 and greater (abnormal VA).³⁴ Receiver operating characteristic (ROC) curves were plotted, the area under each curve was determined, and dot-density graphs were examined to determine the optimum cut-off to give best sensitivity with good specificity for each test. Analysis was undertaken with Excel (version 2210), Jamovi (version 1.6), and Sigma Plot (version 11.0) and a P value of 0.05 was used for significance. 

Results

Eighty participants between 18 and 39 years of age participated in the study, including 32 with normal VA, 12 with reduced VA (7 amblyopia, 1 high myopia, 2 keratoconus, 1 nystagmus, and 1 Best's disease), and 36 with simulated reduced VA. The bias, LoA, and Spearman's correlation coefficient for all the tests in comparison to the ETDRS, are presented in Table 1. Similar results were found between the first and second visits in all tests, so only the results of the first visit are shown here. 

Table 1.

 

Agreement of Children's VA Tests With Standard ETDRS VA

 

Figures 2A–D show Bland–Altman and scatter plots of the agreement of WatDAT HC for 3 and 5 distractors with ETDRS. Figures 3A–D show the same plots comparing WatDAT Faces and ETDRS. WatDAT 3 HC and WatDAT 3 Faces (Figs. 2A and 3A) showed the best agreement with ETDRS among the WatDAT tests (bias = 0.0712 [LoA = 0.241] and bias = 0.0678 [LoA = 0.247], respectively). The positive biases indicate that WatDAT slightly underestimated VA compared with ETDRS logMAR VA, that is, giving poorer values of VA. For all versions of WatDAT 3, this underestimation was <1 logMAR line of acuity. The WatDAT 5 gave slightly more bias and poorer agreement (bias = 0.121 [LoA = 0.250] and bias = 0.0853 [LoA = 0.250]) for HC and Faces, respectively. The slightly higher bias may be because the guessing rate was lower for the five distractors compared with three, making the test a little harder. 

Figure 2.

 

Agreement of WatDAT HC with ETDRS. (A and B) Bland–Altman plots for WatDAT HC and ETDRS for the 3 and 5 distractors, respectively. The difference between the WatDAT and ETDRS is plotted against the mean of the WatDAT and ETDRS. (C and D) Scatterplots of WatDAT against ETDRS. LV, low vision; Simulated, simulated low vision; LLOA, lower limit of agreement; ULOA, upper limit of agreement.

Figure 3.

 

Agreement of WatDAT Faces with ETDRS. (A and B) Bland–Altman plots for WatDAT Face and ETDRS for the 3 and 5 distractors, respectively. The difference between the WatDAT and ETDRS is plotted against the mean of the WatDAT and ETDRS. (C and D) Scatterplots of WatDAT against ETDRS. LV, low vision; Simulated, simulated low vision; LLOA, lower limit of agreement; ULOA, upper limit of agreement.

Patti Pics and Lea MassVAT matching tests showed slightly better agreement with ETDRS than WatDAT (Table 1). The Kay and Lea uncrowded matching tests both overestimated VA compared with the ETDRS VA by more than one line of logMAR acuity (bias = −0.179 and −0.119, respectively), although the LoA was good for both. Figures 4A–F show the agreement of Kay Pictures and the two PL tests with ETDRS. The CAT and TAC considerably over-estimated acuity compared with ETDRS (bias of −0.421 and −0.226, respectively) and also had wide LoA (0.375 and 0.389, respectively) (Table 1 and Figs. 4B, C, E, and F). The horizontal line at 0.1 shows the participants in the low vision and simulated low vision group and who clearly had reduced ETDRS VA but had VA of ≥6/7.5 with the Kay or PL tests. The CAT and TAC were also compared directly with WatDAT. This analysis showed very poor agreement with the bias ranging between 0.294 and 0.543 and the LoA between 0.357 and 0.444. 

Figure 4.

 

Agreement of Kay pictures and PL tests (Cardiff acuity cards and TAC) and with ETDRS. (A–C) Bland–Altman plots for Kay Pictures, Cardiff and TAC, and ETDRS, respectively. The difference between each test and ETDRS is plotted against the mean of each test and ETDRS. (D–F) Scatterplots of Kay pictures, Cardiff, and TAC against ETDRS. The horizontal line at 0.1 shows the participants with low vision or simulated LV who demonstrated VA of 6/7.5 or better with the Kay or PL tests. LV, low vision; Simulated, simulated low vision; LLOA, lower limit of agreement; ULOA, upper limit of agreement.

WatDAT showed high sensitivity and specificity (sensitivity ≥0.94 and specificity ≥0.84). Sensitivity, specificity, and the area under curve are shown in comparison with ETDRS in Table 2. The ROC plots and dot histograms are shown in Figures 5A and B for WatDAT tests using the same cut-off as the ETDRS. Sensitivity using this criterion was also high for all the matching tests, except the Kay and Lea uncrowded (sensitivity of 0.69 and 0.81, respectively). A ROC analysis showed that, for these tests, the criterion to the nearest logMAR line for optimum sensitivity with good specificity for detecting participants with vision of <0.1 logMAR, was a logMAR of −0.1 and 0, respectively (Table 2). The PL tests had poorer sensitivity (0.46 for CAT and 0.65 for TAC) and gave the best sensitivity at lower cut-off levels of −0.5 logMAR for the CAT and −0.1 logMAR for the TACs. There was a significant difference for the area under the ROC between the ETDRS (area under the curve = 1) and each of the PL tests (CAT, P = 0.019; TAC, P = 0.013). The area under the ROC curves for all other tests did not differ significantly from that for the ETDRS. 

Table 2.

 

Sensitivity, Specificity, Area Under the Curve in Comparison With ETDRS Using a Cut-Off of ≤0.1 versus >0.1

 

Figure 5.

 

(A) ROC plots for WatDAT tests in comparison with ETDRS using a cut-off of ≤0.1 versus >0.1. A, Area under the curve. (B) Dot histogram. Filled circles (0) = those with normal VA based on ETDRS. Open circles (1) = those with reduced VA.

WatDAT 3 Faces and WatDAT 5 Faces had good test–retest repeatability (LoA-R of 0.193 and 0.203 logMAR, respectively), followed by the WatDAT 3 HC (LoA-R = 0.209) and WatDAT 5 HC (LoA-R = 0.232). This result compares with an LoA-R of 0.167 logMAR for the ETDRS (Fig. 6 and Table 3). The LoA-R of matching tests were found to be in a similar range. In fact, the Kay and Lea MassVAT had a better LoA-R (0.143 and 0.145 logMAR, respectively) than the ETDRS. PL tests, CAT, and TAC were found to have the poorest test–retest repeatability with an LoA-R of 0.286 and 0.362 log units (Table 3). 

Figure 6.

 

Test–retest repeatability for WatDAT tests and ETDRS. (A, B, D, and E) Bland–Altman plots for the test and retest of WatDAT HC 3 and 5 distractors and WatDAT Faces 3 and 5 distractors, respectively. (C) Bland–Altman plot for the test and retest of ETDRS. The difference between the test and the retest is plotted against the mean of the test and retest. LV, low vision; Simulated, simulated low vision; LLOA, lower limit of agreement; ULOA, upper limit of agreement.

Table 3.

 

Test–Retest Repeatability of VA Tests

 

Discussion

This study aimed to validate the newly developed WatDAT in an adult population with normal and reduced vision against the ETDRS, by considering LoA with the ETDRS and sensitivity and specificity in detecting reduced VA. The accuracy (agreement) of WatDAT relative to currently available charts for measuring VA in children was also evaluated along with the LoA-R for each test. 

Bias of the Tests

WatDAT showed good agreement with the ETDRS (small bias), with WatDAT 3 HC and Faces versions showing the best agreement. All WatDAT versions slightly underestimated ETDRS VA (showed a positive bias, indicating slightly poorer VA). For WatDAT 3 HC and Faces, this difference was <1 logMAR line, which is within the repeatability of the ETDRS itself,³⁵ approximately 1 line (0.1 logMAR) for people with normal vision, and between 0.122 and 0.21 for those with reduced VA.^19,24,25 

The bias of WatDAT, except WatDAT 5 HC, was similar to those of Patti Pics and Lea matching tests, which ranged from 0.001 for Patti Pics UC to −0.12 for the Lea UC. In the present study, Patti Pics had minimal bias compared with ETDRS, although Richardson et al.²⁹ reported that Patti Pics underestimated VA by 1 line. The slightly higher positive bias of WatDAT HC compared with the Patti Pics is likely because the printed Patti Pics are increased by 18% from the true Snellen size, which was done to give equivalent thresholds to ETDRS (Ed Kopidlansky, personal communication (April 27, 2023)), whereas those used in the WatDAT followed the Snellen principle of subtending 50 minutes of arc for the 1.0 logMAR target. An 18% increase would be expected to make a difference of approximately 0.70 of a logMAR line or 0.07 logMAR, which agrees with the difference found here. The Lea MassVAT was also found to have a low bias, which agrees with results from Paudel et al.⁹ WatDAT 5 HC, Lea UC, and Kay Pictures all showed a larger bias (0.12, −0.12, and −0.18, respectively). Previous studies have also found an overestimation of the Lea and Kay. Dobson et al.¹⁵ reported that the logMAR chart version of the Lea symbols overestimated ETDRS VA in children by 0.21 logMAR,¹⁶ whereas several other studies have reported that Kay Pictures overestimate ETDRS VA by 1 to 2 lines.^{16,17,19,29,36} Anstice et al.¹⁶ found significantly better VA measures with crowded Kay Pictures than all the other charts they tested (ETDRS, crowded MassVAT Lea Symbols, crowded Keeler logMAR, crowded HOTV, and Tumbling E) in a group of adults and children using blur to reduce VA. It has been suggested that the overall shape or one component of the Kay Pictures gives cues that make them easier to identify and less susceptible to blur than standard optotypes.^16,17 Another explanation is that the induced contour interaction may be less consistent with the Kay Pictures compared with standard letter charts with similar interoptotype separation.³⁶ The Kay Pictures have been redesigned since these studies.³⁷ Milling et al.,²⁰ using the newly designed Kay Pictures, found that there was minimal bias compared with ETDRS. However, it appears that they maintained a 3-m testing distance, which may have resulted in a floor effect with their normal adult participants. In the current study, it was necessary to increase the distance to 6 m for most participants with normal vision. The current results indicate that the problem of overestimation remains. 

In the present study, the CAT and TAC were found to overestimate VA measures approximately by 0.4 and 0.2 logMAR, respectively; that is, these two PL tests have poor agreement with ETDRS. Significant overestimation has been reported previously for TAC⁶ and the CAT.^8,9,23 Paudel et al.⁹ reported even greater disparities between the CAT and ETDRS with spherical and astigmatic blur. This overestimate is clearly shown in Figure 4, where a significant number of people with abnormal VA would be undetected if a cut-off of 0.1 is applied to either PL test. Overestimation of VA is problematic because it may indicate good vision among children with potential visual deficits, thus creating a false sense of security, unless the pass/fail criterion for failure is adjusted. 

Limits of Agreement

Compared with the ETDRS, WatDAT had slightly higher LoAs than the crowded matching tests (which ranged between 0.17 and 0.20), but was similar to the Lea UC and Patti Pics UC (0.23 and 0.21, respectively). The PL tests had poorer LoAs (0.38 and 0.39 for CAT and TAC, respectively). The LoAs for the crowded matching tests in this study are similar to reported ranges for adults. Anstice et al.¹⁶ found the 95% LoA to range between 0.13 and 0.18 when comparing results on the ETDRS, HOTV, Lea symbols, and Kay Pictures, whereas Mercer et al.¹⁸ found LoA of 0.20 and 0.11 comparing Sloan letters with Lea and Patti Pics symbols, respectively. Shah et al.¹⁹ found LoAs of 0.19 and 0.21 between the ETDRS and two versions of the Kay Pictures, which are very close to the present study (0.198). Leat et al.⁵ compared an early version of the differential acuity test with the ETDRS using blur to reduce VA and found an LoA of 0.27. Most of these studies used crowded symbols. It is possible that using single uncrowded symbols gives more variability, resulting in poorer LoA compared with the ETDRS (which is a crowded test), than those using crowded symbols, such as the MassVAT tests and crowded Kay Pictures. The number of participants with normal or reduced vision will also impact the LoA. For example, it seems that more of Milling et al.’s participants had normal vision than in the present study and the LoA tends to increase for participants with reduced vision.^5,20,24 However, it is important to note that the WatDAT was found to have significantly better LoA than both PL tests, which is the alternative for children who cannot undertake matching tests. 

Sensitivity and Specificity

When applying the initial cut-off for abnormal VA of <0.1 logMAR, the sensitivity of WatDAT for detecting abnormal ETDRS VA was very high (>0.96). Sensitivity was also good for all the matching tests, except for the Lea UC and Kay Pictures (Table 3). The specificity of WatDAT Faces was also high with the initial cut-off, although lower for WatDAT HC. Increasing the cut-off to <0.2 logMAR for WatDAT 3 HC and to <0.3 for WatDAT 5 HC improved the specificity to 0.94 while maintaining the high sensitivity (0.98). With such good sensitivity and specificity, WatDAT has good potential for measuring recognition acuity in children. The poorer sensitivity of the Kay Pictures and the Lea UC is likely caused by the negative bias (overestimation of VA). The sensitivity of both these tests can also be improved by changing the criterion (reducing the cut-off) (Table 3). 

Our analysis showed that, although the specificity of the CAT and TAC was high (1.00), sensitivity was extremely poor at 0.46 and 0.65, respectively. This finding is unlike that from the report of an earlier study in children by Drover et al.,²² where TAC was successful in detecting amblyopia, yielding a sensitivity of 0.80. However, Kushner et al.⁶ agreed that TAC had low sensitivity for detecting reduced vision in children and adults; sensitivity ranged from 0.24 to 0.58 for participants with different levels of VA. Geer et al.²³ reported a sensitivity for the CAT of 0.42 for detecting amblyopia and Howard et al.⁸ found a sensitivity of 0.56 with a cut-off of 0.0 logMAR. In the current study, to optimize the sensitivity of CAT and TAC in detecting abnormal VA, the cut-offs required adjustment to levels better than that of nominal normal VA (−0.5 and −0.1, respectively) (Table 3). This process resulted in some improvement in sensitivity, but poorer specificity for both tests. This finding agrees with Geer et al.²³ and Paudel et al.,⁹ who concluded that a simple correction factor would not improve the agreement between CAT and logMAR letter tests. However, to apply this cut-off for the CAT (−0.5 logMAR), the testing distance would need to be increased to 4 m, which is not done clinically, and testing with the TAC would need to be undertaken at 84 cm. The poorer performance of the PL tests is also indicated by their significantly smaller area under the ROC curve compared with all other tests. These findings confirm that a normal finding on a PL test is not reliable, but an abnormal result is a clear indication of abnormal VA.⁶ 

Repeatability

WatDAT showed good LoA-R (0.193 and 0.209 logMAR, respectively) compared with the ETDRS (0.167 logMAR). In fact, all the matching tests except Patti Pics UC had good test–retest repeatability. The LoA-R for WatDAT 3 and 5 Faces and WatDAT 3 HC are comparable with those reported for the ETDRS chart when participants with reduced VA are included (LoA-R between 0.122 and 0.21).^19,24,25 Our LoA-Rs for the Kay and Lea tests are similar to those of Shah et al.¹⁹ and Milling et al.²⁰ for adult populations. Our results found poor LoA-R for CAT and TAC (LoA-R = 0.286 and 0.362 log units, respectively). There was minimal test–retest bias for all the tests. 

Limitations

The use of adult participants is a potential limitation for evaluating a test designed for young children. This strategy was necessary to compare WatDAT with the ETDRS chart and matching tests, which the youngest children cannot perform.^9,16,18 It was also necessary to compare among so many pediatric tests of VA—young children would not be able to hold their attention for the duration of the testing required. Because young children experience a greater crowding effect, the use of adults may selectively affect the results for the crowded pediatric tests, but it is unlikely to affect the results for WatDAT and other uncrowded tests. The results from young children may be more variable, resulting in a larger LoA and LoA-R than reported here, because adults generally have more mature cognition, gaze control, and attention to perform VA tests.^36,38 However, Mercer et al.¹⁸ and Shah et al.¹⁹ found no significant differences between adults and children who were able to perform the tests. If anything, the use of adults would be expected to result in better LoAs, so the results presented here represent the best performance of all the included tests. Evaluation of the WatDAT for young children requires a different approach, such as measuring testability and the ability to detect known eye disorders. These results are forthcoming. 

Another limitation was including some participants with simulated low vision, as well as those with true low vision. Those with simulated low vision may perform differently. However, it can be seen from Figures 2, 3, 4, and 6 that the scatter among the simulated low vision participants is similar to that for those with true low vision. Figures 4E and F show that true low vision participants were able to achieve better acuity on the CAT compared with those with simulated low vision and similar VA with the TAC. So, the poor sensitivity of the CAT would be poorer still for those with true low vision, but similar for the TAC. 

A third limitation is the possibility of examiner bias; the test administrators were not completely blinded to the results of the previous VA scores during the second assessment. This factor was minimized because the test sequence was randomized in each visit and the examiner did not refer to the previous test results. 

Last, WatDAT is conducted at a near or intermediate distance. The testing distance varied from 30 to 120 cm depending on the participant's VA. This finding is also true for the TAC and CAT, which had testing distances from 84 cm to 2 m. However, for the WatDAT, there was no significant decrease of VA for any of the versions when plotted against age, which indicates that accommodation or age are unlikely to influence the data in this population. Nonetheless, accommodation is known to be reduced in some clinical populations for whom WatDAT may be a useful measure of VA, such as Down syndrome and cerebral palsy.^39,40 Reduced accommodation may result in an underestimate of distance VA in these cases (although an accurate estimation of near VA). 

Conclusions

This study is unique in providing validity and repeatability for the new differential acuity tests, as well as for several pediatric VA charts, including tests based on matching and on PL, compared with the ETDRS. 

The study demonstrates that WatDAT has similar agreement with ETDRS as several other matching tests for young children and has better agreement than the Kay Pictures and the LEA UC. WatDAT HC 3 and both WatDAT Faces versions have better repeatability than Patti Pics UC. WatDAT also has very good sensitivity and specificity when compared with the ETDRS. Sensitivity and specificity can be improved for WatDAT HC versions by a slight change of the criterion. The data presented here indicate that WatDAT Faces and WatDAT 3 HC would be the best choices of the available WatDAT versions. The performance of WatDAT was superior to both PL tests, the alternative for children who are not able to undertake matching tests. 

Both PL tests and the Kay Pictures significantly overestimated VA in adults and have poor sensitivity for detecting reduced VA. The sensitivity of the CAT, TAC, and Kay Pictures could be improved by changing the criterion, but this was not possible for the TAC and CAT without sacrificing specificity and would mean applying a criterion which was better than 6/6 (0.0 logMAR) to detect adults with VA poorer than 0.1 LogMAR. Our findings corroborate earlier reports that caution must be exercised in the use of CAT and TAC in assessing vision in children because they are not reliable for detecting mild and moderate reductions of VA.^6,23,41 

This study demonstrates that WatDAT has good validity when compared with the ETDRS and, therefore, has the potential to be a useful clinical test of recognition VA. It was designed to make it possible to measure recognition VA in children from the age of 18 months, before they are able to undertake a matching test.¹⁴ 

Acknowledgments

The authors thank Aashi Saraf for software development and Jeff Hovis for help with the ROC analysis. 

Funding Information: Fighting Blindness Canada and University of Waterloo School of Optometry and Vision Science Seed Funding. 

Disclosure: V. Opoku-Yamoah, None; L.W. Christian, None; E.L. Irving, Leat and Irving: 2020 US copyright. Copyright for Face and Non-Face visual acuity symbols (FaceVA). Leat SJ, Irving EL. Registration number VAu 1-411-289. Feb 21 2020. 2019 Canadian copyright. Copyright for Face and Non-Face visual acuity symbols (FaceVA). Leat SJ, Irving EL. Registration number 1160819. Jul 4th 2019; D. Jones, None; D. McCulloch, None; K. Rose, None; S.J. Leat, Leat and Irving: 2020 US copyright. Copyright for Face and Non-Face visual acuity symbols (FaceVA). Leat SJ, Irving EL. Registration number VAu 1-411-289. Feb 21 2020. 2019 Canadian copyright. Copyright for Face and Non-Face visual acuity symbols (FaceVA). Leat SJ, Irving EL. Registration number 1160819. Jul 4th 2019