September 2018
Volume 7, Issue 5
Open Access
Articles  |   October 2018
A Smartphone-Based Tool for Rapid, Portable, and Automated Wide-Field Retinal Imaging
Author Affiliations & Notes
  • Tyson N. Kim
    Department of Ophthalmology and Visual Sciences, University of Michigan School of Medicine, Ann Arbor, MI, USA
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
    Department of Ophthalmology, University of California, San Francisco, CA, USA
  • Frank Myers
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
  • Clay Reber
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
  • PJ Loury
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
  • Panagiota Loumou
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
  • Doug Webster
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
  • Chris Echanique
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
  • Patrick Li
    Department of Ophthalmology and Visual Sciences, University of Michigan School of Medicine, Ann Arbor, MI, USA
  • Jose R. Davila
    Department of Ophthalmology and Visual Sciences, University of Michigan School of Medicine, Ann Arbor, MI, USA
  • Robi N. Maamari
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
    Department of Ophthalmology and Visual Sciences, Washington University School of Medicine in St. Louis, St. Louis, MO, USA
  • Neil A. Switz
    Department of Physics and Astronomy, San José State University, San Jose, CA, USA
  • Jeremy Keenan
    Department of Ophthalmology, University of California, San Francisco, CA, USA
  • Maria A. Woodward
    Department of Ophthalmology and Visual Sciences, University of Michigan School of Medicine, Ann Arbor, MI, USA
  • Yannis M. Paulus
    Department of Ophthalmology and Visual Sciences, University of Michigan School of Medicine, Ann Arbor, MI, USA
  • Todd Margolis
    Department of Ophthalmology and Visual Sciences, Washington University School of Medicine in St. Louis, St. Louis, MO, USA
  • Daniel A. Fletcher
    Department of Bioengineering and Biophysics Program, University of California, Berkeley, Berkeley, CA, USA
    Chan-Zuckerberg Biohub, San Francisco, CA, USA
  • Correspondence: Daniel A. Fletcher, University of California, Berkeley, Department of Bioengineering and Biophysics Program, 608B Stanley Hall #1762, Berkeley, CA 94720, USA. e-mail: fletch@berkeley.edu 
  • Footnotes
     TNK, FM, and CR contributed equally to this work.
Translational Vision Science & Technology October 2018, Vol.7, 21. doi:10.1167/tvst.7.5.21
  • Views
  • PDF
  • Share
  • Tools
    • Alerts
      ×
      This feature is available to authenticated users only.
      Sign In or Create an Account ×
    • Get Citation

      Tyson N. Kim, Frank Myers, Clay Reber, PJ Loury, Panagiota Loumou, Doug Webster, Chris Echanique, Patrick Li, Jose R. Davila, Robi N. Maamari, Neil A. Switz, Jeremy Keenan, Maria A. Woodward, Yannis M. Paulus, Todd Margolis, Daniel A. Fletcher; A Smartphone-Based Tool for Rapid, Portable, and Automated Wide-Field Retinal Imaging. Trans. Vis. Sci. Tech. 2018;7(5):21. doi: 10.1167/tvst.7.5.21.

      Download citation file:


      © ARVO (1962-2015); The Authors (2016-present)

      ×
  • Supplements
Abstract

Purpose: High-quality, wide-field retinal imaging is a valuable method for screening preventable, vision-threatening diseases of the retina. Smartphone-based retinal cameras hold promise for increasing access to retinal imaging, but variable image quality and restricted field of view can limit their utility. We developed and clinically tested a smartphone-based system that addresses these challenges with automation-assisted imaging.

Methods: The system was designed to improve smartphone retinal imaging by combining automated fixation guidance, photomontage, and multicolored illumination with optimized optics, user-tested ergonomics, and touch-screen interface. System performance was evaluated from images of ophthalmic patients taken by nonophthalmic personnel. Two masked ophthalmologists evaluated images for abnormalities and disease severity.

Results: The system automatically generated 100° retinal photomontages from five overlapping images in under 1 minute at full resolution (52.3 pixels per retinal degree) fully on-phone, revealing numerous retinal abnormalities. Feasibility of the system for diabetic retinopathy (DR) screening using the retinal photomontages was performed in 71 diabetics by masked graders. DR grade matched perfectly with dilated clinical examination in 55.1% of eyes and within 1 severity level for 85.2% of eyes. For referral-warranted DR, average sensitivity was 93.3% and specificity 56.8%.

Conclusions: Automation-assisted imaging produced high-quality, wide-field retinal images that demonstrate the potential of smartphone-based retinal cameras to be used for retinal disease screening.

Translational Relevance: Enhancement of smartphone-based retinal imaging through automation and software intelligence holds great promise for increasing the accessibility of retinal screening.

Introduction
Retinal photography is used extensively to assist with diagnosis and monitoring of retinal diseases. However, access to traditional tabletop retinal cameras is limited by their high-cost, bulky design and need for skilled operators. Conventional imaging approaches also require patient cooperation with stabilized, upright head positioning, which can be difficult among sick, wheelchair-bound, and immobilized patients, as well as children.1 Handheld ophthalmoscopes with digital image capture offer an alternative. Recently, smartphone-based retinal imaging approaches have been described that leverage the phone for its portability, high-resolution camera, and wireless data transfer capabilities to capture diagnostic images for real-time or remote anterior2 or posterior segment312 evaluation at a fraction of the cost of traditional instruments.810,12 These investigations nicely demonstrate the growing potential for smartphone imaging to expand the accessibility of ophthalmic care and photographic screening of vision-threatening diseases. 
There is particularly great interest in the validation and integration of smartphone-based retinal photography in local community screening programs for diseases, such as glaucoma and diabetic retinopathy (DR),1316 as well as in the emergency department and inpatient settings where the fundus examination is underperformed.1719 DR is the most common microvascular complication of diabetes and the leading cause of vision loss in working-age adults aged 20 to 74 in the United States, accounting for 12% of new cases annually.2023 The World Health Organization estimates more than 346 million people worldwide have diabetes and that 552 million people will be affected by 2030.24 Physicians have effective treatments for complications of DR, and it has been shown that patients have better vision outcomes with early detection and treatment.2527 However, despite well-accepted guidelines,28 nearly half of diabetic adults in the United States do not receive recommended annual screening for DR, and vulnerable populations with less access to specialty medical care have an estimated screening rate of between 10% and 20% per year.29 Ophthalmologists recognize that telehealth screening of DR via retinal photography in the primary care setting, with remote ophthalmologist consultation,30 is a mechanism to improve access and thereby improve outcomes in a cost-effective manner.31,32 A remaining challenge is how to increase access to such retinal photography. 
The gold-standard photographic screening technique for DR is seven-field, 30° mydriatic tabletop retinal photography33 in which 14 images per eye from seven standard fields comprising the posterior 90° of the retina are evaluated to determine the risk of vision loss and retinopathy progression. Subsequent studies have demonstrated that three-field, 45° nonmydriatic fundus photography was effective in grading DR to determine referral-warranted disease, whereas a single 45° photograph was insufficient to accurately grade DR.3436 Importantly, imaging by nonexpert operators may affect image quality and is an important consideration for community-based screening efforts.37,38 Several studies have investigated smartphone-based screening of DR4,16,39 and reported sensitivities of 80% or more with high specificities, as recommended by the British Diabetic Association for new imaging devices in population-based screening.40 However, other investigators have found insufficient sensitivities of below 60% for detecting DR37 suggesting smartphone imaging is not universally reliable. In light of this, smartphone-based retinal imaging has continued to undergo validation against more traditional photographic methods.37,4143 
We previously reported on the development of the Ocular CellScope, a handheld, smartphone-integrated imaging device capable of capturing high-quality, wide-field images of the retina.3 From extensive field-testing with that device, we identified several challenges facing the performance and use of that device in particular and smartphone-based retinal photography in general. First, surveying wide regions of the retina with a handheld device is technically challenging and requires extensive operator experience to comprehensively image the peripheral retina. Second, sustained levels of bright white-light illumination for high-resolution images of the retina can be uncomfortable for patients, resulting in poor image localization caused by gaze shifts and decreased image quality from motion artifacts. Third, smartphone imaging approaches have typically required two-handed operation and been asymmetric, with different orientation for the right and left eye, increasing operator instability and motion. Fourth, the lack of streamlined data management for acquiring, viewing, and storing images slows workflow and increased examination time. 
Here, we demonstrate a retinal imaging system, CellScope Retina, that addresses these issues by incorporating automation to improve image quality and reliability. We describe the application of CellScope Retina in feasibility studies in inpatient and outpatient settings, where images of various retinal pathologies, including DR, were captured. The 100° field was chosen to ensure that a larger portion of the retina is imaged with CellScope Retina than the gold-standard Early Treatment Diabetic Retinopathy Study (ETDRS) screening technique, which evaluates the posterior 90° using seven individual 30° field images. Additionally, CellScope Retina images have a resolution of 52.3 pixels per retinal degree, surpassing the minimum resolution requirement of 30 pixels per degree described by the National Health Service for diabetic retinopathy.44 To our knowledge, CellScope Retina is the first demonstration of an automated smartphone-based system capable of imaging, stitching, and reviewing a wide-field retinal montage in a fully handheld platform without requiring an external computer. 
Methods
Hardware Design
The CellScope Retina weighs approximately 310 g and consists of a smartphone and three-dimensional (3D)-printed plastic housing containing optics for illuminating and imaging the retina in conjunction with the camera of a smartphone (Fig. 1A). A polarizing wire grid beam splitter (Moxtek HCPBF02A, Orem, UT) is used to illuminate the retina with polarized light and minimize unwanted reflections as previously described.3 A custom printed circuit board (PCB) provides illumination with a single white light-emitting diode (LED; Lumileds Luxeon LXML-PWN2, Brantford, Ontario, Canada) at the center and a ring of radius 3.15 mm with three deep red (655-nm peak wavelength) LEDs (Lumileds Luxeon LXM3-PD01, San Jose, CA). An aspheric condenser lens (Thorlabs ACL12708U-A, Newton, NJ) transmits light from the LEDs to a diffuser (Thorlabs ED1-C50), through a polarizer, an annular mask with 8-mm inner diameter and 15-mm outer diameter, and a 50-mm focal length condenser lens (Thorlabs LA1171-A). The illumination light is reflected by a beam splitter and passes through a 54-diopter ophthalmic lens (Ocular Instruments OI54-A, Bellevue, WA) to form an annulus with a 4.8-mm inner diameter and 9.6-mm outer diameter at the surface of the cornea. The illumination light defocuses and uniformly illuminates the retina, from which it is scattered with accompanying depolarization. Light from the eye is collected and transmitted through a second polarizer (Edmund Optics 85919, Barrington, NJ), thereby preferentially blocking reflected (and orthogonally polarized) illumination light. An achromatic lens with a focal length of 20 mm (Edmund Optics 47661) relays the signal to the smartphone camera module. 
Figure 1
 
CellScope Retina schematic and workflow. (A) Schematic of the optical system. Light from LEDs is directed through a mask and forms an annulus that passes through the peripheral cornea and focuses through the pupil. In propagating through the eye the light becomes defocused, providing even illumination at the retina. Polarization filters minimize unwanted reflections from anterior ocular surfaces, enabling the smartphone to capture a clear image of the retina. (B) The compact optical system and custom-control electronics fit inside a handheld enclosure. (C) Red LED illumination of 655-nm peak emission is used for focusing on the retina, which is within the spectral range of the iPhone camera but outside the peak photopic response of the eye. (D) A white LED with a broad emission spectrum is flashed for recording images of the retina. LED spectra in (C) and (D) are from respective datasheets; photopic response is from CIE 1931 standard; phone response is approximate for a CMOS phone sensor. (E) Smartphone user interface enables (1) patient data capture, (2) preview during focus/alignment with swipe gestures adjusting camera settings, and (3) exam data review with pinch and swipe gestures for browsing stitched image montage. (F) Photos can be uploaded directly from the smartphone to a cloud database allowing remote diagnosis with a web interface.
Figure 1
 
CellScope Retina schematic and workflow. (A) Schematic of the optical system. Light from LEDs is directed through a mask and forms an annulus that passes through the peripheral cornea and focuses through the pupil. In propagating through the eye the light becomes defocused, providing even illumination at the retina. Polarization filters minimize unwanted reflections from anterior ocular surfaces, enabling the smartphone to capture a clear image of the retina. (B) The compact optical system and custom-control electronics fit inside a handheld enclosure. (C) Red LED illumination of 655-nm peak emission is used for focusing on the retina, which is within the spectral range of the iPhone camera but outside the peak photopic response of the eye. (D) A white LED with a broad emission spectrum is flashed for recording images of the retina. LED spectra in (C) and (D) are from respective datasheets; photopic response is from CIE 1931 standard; phone response is approximate for a CMOS phone sensor. (E) Smartphone user interface enables (1) patient data capture, (2) preview during focus/alignment with swipe gestures adjusting camera settings, and (3) exam data review with pinch and swipe gestures for browsing stitched image montage. (F) Photos can be uploaded directly from the smartphone to a cloud database allowing remote diagnosis with a web interface.
The device is powered by a rechargeable lithium polymer battery and includes a compact, custom-printed circuit board containing a microcontroller module (Arduino Micro, Somerville, MA), Bluetooth transceiver (RedBearLabs BLEMini, Hong Kong, China), two buck/boost controlled-current LED drivers, battery management controller, power supply, and several status indicator LEDs. The battery, electronics, and illumination optics are miniaturized and arranged in linear fashion along the optical axis to fit within the handle of the device (Fig. 1B). An external organic LED display (4D Systems μOLED-128-G2, Minchinbury, Australia) provides a fixation target for the contralateral eye (Fig. 1B). This external display attaches to either side of the CellScope Retina housing using magnets, allowing symmetrical use while imaging either eye. Spring-loaded gold pins provide electrical contact for power and communication to the microcontroller. To conserve in-the-field battery life, the system automatically shuts off after 5 minutes of inactivity. Under normal daily use, the device runs for more than a week on a single charge and, like the smartphone, can be recharged with a standard USB cable. 
Light hazards were assessed based on the ISO 15004-2:2007 standard, and informed by Sliney et al.45 In regard to the standard, the red (see Figs. 1C and 1D for LED spectra) focusing LEDs are a continuous source while in use (CW; though in practice not continuously illuminated), while the white LED flash is a pulsed source. The red LEDs produce 0.67 mW or less maximum at the cornea, and the LEDs illuminate 0.1 cm2 or more of the cornea and 1 cm2 or more of the retina, placing the red illumination more than 100× below the ISO 15004 §5.4 safety limits for Group 1 CW instruments. The white LED flash produces 80-ms pulses with peak powers of 19.4 mW (software limited to 8.1 mW in use); this places the combined illumination safely below the §5.4 safety limits for Group 1 Pulsed instruments even for extremely high pulse counts: assuming 250 flashes (a large overestimate of likely use – a typical 5-image montage would require ∼5 pulses) and flash rates of 5/second (again overestimated; 1 per second is a likely maximum), the safety factor is over 5× for §5.4.2.1.a, and over 20× for the other limits. For completeness, we also compared the white LED flash illumination with the §5.5.2 Group 2 Limits for Pulsed Instruments and the §5.5.1 Group 2 Limits for CW Instruments, and the device complies with both. The radiant exposure guideline (§5.5.1.6.a) falls at the equivalent of well over 1000 pulses. 
Application and Software Design
A custom iPhone (Apple Inc., Cupertino, CA) application (app) was developed which communicates with the electronic hardware of CellScope Retina via Bluetooth low-energy (BLE). To begin an exam, the operator enters basic patient information and then selects an eye for imaging. The app provides a preview window, which allows the operator to employ ergonomic touch and swipe motions to adjust focus, zoom, and exposure prior to initiating the image capture sequence. This allows reduction of operator motion during imaging, which can degrade image quality when using a handheld platform (Fig. 1E). Additionally, during surveying and focusing onto the retina the device uses far-red LEDs that fall outside the peak photopic response of the eye but remain within the receptive range of the iPhone camera and filters (Fig. 1C). This approach minimizes patient discomfort and unwanted gaze-induced motion artifacts. Once camera positioning and focus are optimized, the initiation of image capture signals a high-intensity white LED to flash during acquisition (Fig. 1D). The images can be uploaded to a secure server directly from the app using WiFi or cellular service for remote reviewing (Fig. 1F). 
The CellScope Retina hardware-software integration enables wide-field imaging of the retina in a semiautomated fashion. The device is designed to allow the operator to comfortably hold and operate the touchscreen app using one hand (Fig. 2A). The magnetically mounted screen displays a software-driven target for eye fixation to minimize unwanted eye movement during examination (Fig. 2B). This green, 2-mm diameter fixation dot directs the nonimaged eye to software-specified locations on the display. Conjugate eye movements simultaneously reposition the imaged eye for rapid and precise imaging of multiple retinal fields (Fig. 2C). With the current automated program, five overlapping images are captured of the central, inferior, superior, nasal, and temporal retina in rapid sequence (Fig. 2D and Supplementary Movie S1). Each image has an approximately 50° field of view and may be computationally merged with a custom algorithm that runs directly on the smartphone to create an approximately 100°, wide-field montage of the retina (Fig. 2E). 
Figure 2
 
CellScope Retina is a handheld, smartphone-based retinal camera that enables fully integrated wide-field retinal imaging. (A, B) The device is designed for easy handheld operation. (B) A detachable display may be magnetically mounted to either side of the device and provides a green dot as a fixation target. (C) The fixation target is translated through a series of positions, re-orienting the patient's eyes and retina in a rapid and controllable fashion. (D) Each individual image is approximately 50°. With the current automated arrangement, five overlapping images are captured of the central, superior, nasal, temporal, and inferior retina. (E) These five images are computationally merged on the phone using custom software to create a wide-field image of the retina spanning approximately 100°.
Figure 2
 
CellScope Retina is a handheld, smartphone-based retinal camera that enables fully integrated wide-field retinal imaging. (A, B) The device is designed for easy handheld operation. (B) A detachable display may be magnetically mounted to either side of the device and provides a green dot as a fixation target. (C) The fixation target is translated through a series of positions, re-orienting the patient's eyes and retina in a rapid and controllable fashion. (D) Each individual image is approximately 50°. With the current automated arrangement, five overlapping images are captured of the central, superior, nasal, temporal, and inferior retina. (E) These five images are computationally merged on the phone using custom software to create a wide-field image of the retina spanning approximately 100°.
Image-stitching could be performed either directly on the smartphone using custom software (Fig. 2E) or using the DualAlign i2k software package (Clifton Park, NY; Fig. 4). The custom on-phone algorithm used the OpenCV C++ image-processing library for a stepwise process of correcting optical distortion, cropping the circular retinal field of view, and estimating affine transforms between neighboring fields using Speeded Up Robust Features (SURF) keypoints matching. These transforms describe the translation, rotation, and skew of each peripheral field relative to the central field. To save on computational resources, the transform is estimated using downsampled grayscale images derived from the green color channel. Finally, a full-resolution mosaic is generated using the estimated transforms, and overlapping regions are linearly blended. 
Processing time to generate a full resolution, five-image montage covering approximately 100 retinal degrees (52.3 pixels/retinal degree) was approximately 5 minutes with our custom software on an iPhone 5s. We subsequently worked with DualAlign, LLC to generate a high-performance solution and improve on-phone processing time. This allows a rapid preview 100° montage to be produced in approximately 5 seconds on an iPhone 5s using downsampled images (6.4 pixels/retinal degree), allowing near real-time assessment of wide-field image data. A full-resolution montage may be generated once the operator is satisfied with image data. Initial implementation of the i2k DualAlign library has already pushed processing time for full resolution montage below 50 seconds on an iPhone 5s, with further improvements expected. All image acquisition, processing, and review steps may be performed on the device without requiring an external computer. Individual images or montages may be reviewed on the smartphone using touchscreen pinch, zoom, and pan gestures. 
Study Participants
Study participants were recruited at the University of Michigan Kellogg Eye Center Retina Clinic and the Ophthalmology Consultation Service at the University of Michigan Hospital, Ann Arbor, MI, in accordance with the University of Michigan Institutional Review Board, and adhered to the tenets of the Declaration of Helsinki (ClinicalTrials.gov Identifier NCT03076697). For feasibility testing in DR, patients (≥18-years old) with diabetes and no obvious media opacity (e.g., vitreous hemorrhage, advanced cataract) were recruited while undergoing routine examination in the University of Michigan Retina Clinic. Patients in the retina clinic had a higher prevalence of retinal disease than might be expected in the general population. A medical student and medical intern acquired images of DR with the CellScope Retina. For imaging of various other retinal abnormalities, patients over 18 years of age were recruited by our university-based consultation service by ophthalmology resident physicians in the emergency room or inpatient setting. A convenience sample of patients was enrolled from the consultation service based on the presence of retinal abnormalities identified from standard clinical examination. All consenting patients underwent a gold-standard dilated eye examination as part of routine care. Participants' demographic data, clinical findings, and diagnoses were recorded. 
Photographic and Clinical Interpretation
Patients underwent dilated fundus imaging in a dimmed room, where CellScope Retina was used to capture five sequential images (central, inferior, superior, nasal, and temporal). Both eyes were imaged unless only one eye was pharmacologically dilated for that visit. Participants were asked to assess the comfort level for red illumination and white flash from CellScope Retina (‘uncomfortable', ‘bearable', or ‘comfortable'), and to compare this with illumination from slit-lamp biomicroscopy. CellScope Retina images were evaluated for the presence and severity of diabetic retinopathy by two masked ophthalmologists who were experienced with telemedicine grading and following review of the modified Airlie House classification grading criteria defined by the ETDRS.33 These graders were masked to all patient information and performed analysis per eye in a randomized fashion. Each 50° image was evaluated in a controlled environment on high-resolution (1600 × 1200 pixels) 19-inch displays with standard luminance and contrast on a black background in accordance with United Kingdom National Health Service guidelines.44 
A grading template was used that assessed the quality of the image and the severity of DR. The image quality was categorized as excellent, fair, acceptable, or not gradable. DR severity was graded as mild, moderate, or severe nonproliferative diabetic retinopathy (NPDR), proliferative diabetic retinopathy (PDR), or no DR in accordance with ETDRS criteria. Because CellScope Retina acquires nonstereo images and thus cannot determine retinal thickening, photographic grading of clinically significant diabetic macular edema (CSDME) was based on the presence of hard exudates within 1 disc diameter of the macula center, as previously described.46,47 Clinically, the CSDME was defined by ETDRS criteria, including retinal thickening within 500 μm of the macula center, hard exudates within 500 μm of macula center with adjacent retinal thickening, or more than 1 disc diameter of retinal thickening partly within 1 disc diameter of the macula center. Photographic diagnosis was based solely on the smartphone image acquisition, whereas clinical diagnosis was aided by optical coherence tomography (OCT) and included patients actively receiving intravitreal anti-vascular endothelial growth factor therapy or intravitreal steroid therapy for CSDME. 
The sensitivity and specificity in detection of referral-warranted diabetic retinopathy (RWDR) is of fundamental importance for screening programs. RWDR was defined as equal to or greater than moderate NPDR or the presence of CSDME, as previously established.33 Traditional in-clinic diagnosis was considered the reference standard and acquired retrospectively by masked chart review of same-day visits, which had included CellScope Retina imaging. In the event that the severity of DR was not explicitly documented in the chart, a masked expert reviewed the documented note, exam, and traditional imaging studies to determine disease severity. Intergrader kappa and percent agreements were calculated to assess identical grading as well as grading within one stage. Confidence intervals (CIs) for sensitivity and specificity were based on exact Clopper-Pearson CIs. Statistical analysis was performed using SPSS Statistics software (IBM, Armonk, NY). CIs for the predictive values are the standard logit confidence intervals given by Mercaldo et al.48 
Data Availability
The datasets generated during the current study include patient information, and may be made available from the corresponding author on reasonable request complying with and approved by the relevant institutional review board committees. 
Results
Device
The CellScope Retina is a hand-held, battery-powered 3D printed optical and hardware system built around a smartphone (Fig. 1). The retina is imaged via a standard ophthalmic lens and relay lenses to the cellphone camera, and the illumination and collection optical paths feature polarization-based rejection of reflections from intermediate surfaces (including the cornea), as well as far-red illumination (Fig. 1C) to increase patient comfort during focusing steps. An external side-mounted display (Figs. 1A, 2B) provides a moveable fixation target to the contralateral eye to direct patient gaze (Fig. 2C); target position may be automatically stepped between positions during sequential imaging. This display is magnetically affixed, allowing switching between sides for imaging either eye. Hardware (fixation target, illumination LEDs) is controlled via Bluetooth communication with the smartphone, which runs a custom iPhone app allowing ergonomic touch and swipe motions to adjust image focus, zoom, and exposure. An automated program then collects five overlapping retinal images (Fig. 2D), which are computationally merged on the smartphone to create and display an approximately 100° montage of the retina (Fig. 2E). Image acquisition takes approximately 1 minute for cooperative patients with pharmacologic mydriasis. Multi-image montages may be generated directly on the smartphone; to our knowledge this is the first implementation of retinal montage on-phone. Our custom software produces full resolution 5-image montages with a field of view of approximately 100° (52.3 pixels/retinal degree) in approximately 5 minutes, with image stitching running in the background on an iPhone 5s. Initial implementation of the i2k DualAlign library has already pushed this below 50 seconds, with further improvements expected. Additionally, rapid preview 100° montage may be produced in approximately 5 seconds on an iPhone 5s using downsampled images (6.4 pixels/retinal degree). This feature allows near real-time assessment of wide-field image data during patient examination and may guide image selection for high-resolution montage. The software and mobile connectivity of the device allow images to be directly uploaded to cloud-based platforms for remote analysis or consultation. 
Although the devices tested used the iPhone 5s (set early in development to ensure consistency with our initial clinical testing), updating to newer iPhones involves only adaptation of the phone mounting bracket and updating of the software—neither the optics nor (being based on fixed LED hardware) the ISO 15004-2:2007 safety assessment is affected. Furthermore, the improved camera sensitivity and processing speeds will be advantageous. 
Diagnostic-Quality Imaging of Various Retinal Pathologies
CellScope Retina captured diagnostic-quality images spanning a wide range of retinal findings. We demonstrate several examples including hypertensive retinopathy with a ruptured retinal arterial macroaneurysm (RAM) and hemorrhages, rhegmatogenous retinal detachment, peripapillary myelinated nerve fiber layer, circinate macular exudates, acute optic nerve edema with disc hemorrhage, NPDR with dot-blot and disc hemorrhages, cytomegalovirus (CMV) retinitis, Morning Glory disc, and PDR with preretinal fibrovascular membrane (Fig. 3). All images were acquired after pharmacologic mydriasis of the pupil. The optics are designed such that the image nearly fills the vertical dimension of the 3:4 aspect-ratio smartphone image sensor to maximize retinal field of view in both the vertical and horizontal dimensions. Images were of sufficient quality to allow rapid diagnosis by a retinal specialist who was masked to patients' medical records. 
Figure 3
 
CellScope Retina produces diagnostic-quality photographs of retinal abnormalities. (A) Hypertensive retinopathy with ruptured RAM along the inferior arcade with hemorrhage in multiple layers. (B) Superotemporal bullous macula-involving rhegmatogenous retinal detachment in the right eye extending from 9:00 to 1:00 o'clock. (C) Peripapillary myelinated nerve fiber layer with flame-shaped appearance and feathered borders in the left eye. (D) Circinate retinal exudates in the inferotemporal macula of the right eye. (E) Acute papilledema with flame hemorrhages extending from the optic disc and proximally along the inferior arcade of the left eye. (F) DR with dot-blot and disc hemorrhages in the right eye. (G) CMV retinitis demonstrating broad retinal whitening along the superior arcade of the left eye. (H) Morning Glory disc anomaly in the left eye. (I) Epiretinal membrane with fibrovascular changes in the left eye. All images were acquired after pharmacologic mydriasis.
Figure 3
 
CellScope Retina produces diagnostic-quality photographs of retinal abnormalities. (A) Hypertensive retinopathy with ruptured RAM along the inferior arcade with hemorrhage in multiple layers. (B) Superotemporal bullous macula-involving rhegmatogenous retinal detachment in the right eye extending from 9:00 to 1:00 o'clock. (C) Peripapillary myelinated nerve fiber layer with flame-shaped appearance and feathered borders in the left eye. (D) Circinate retinal exudates in the inferotemporal macula of the right eye. (E) Acute papilledema with flame hemorrhages extending from the optic disc and proximally along the inferior arcade of the left eye. (F) DR with dot-blot and disc hemorrhages in the right eye. (G) CMV retinitis demonstrating broad retinal whitening along the superior arcade of the left eye. (H) Morning Glory disc anomaly in the left eye. (I) Epiretinal membrane with fibrovascular changes in the left eye. All images were acquired after pharmacologic mydriasis.
Figure 4
 
Wide-field retinal imaging with CellScope Retina under typical and challenging imaging conditions in the inpatient setting. (A) Retinal montage of presumed fungal endophthalmitis in a patient with a well-dilated 8.0-mm pupil who was alert and cooperative with examination demonstrating three creamy white, well-circumscribed 500-μm lesions within 2 mm of the optic nerve. (A') Representative images used for retinal montage. Images A and A' were acquired in less than 1 minute by guiding the patient's eyes with device-assisted fixation. (B) Retinal montage of presumed fungal endophthalmitis in a hospitalized patient with a poorly dilated 4.5-mm pupil and altered mental status demonstrating a 16-mm large creamy white chorioretinal lesion with associated retinal hemorrhages, retinal vasculitis, and vitritis extending from the superior arcade into the macula and threatening the fovea. (B') Representative images demonstrating glare and restricted retinal view as present with small pupil size. These images were acquired in approximately 4 minutes by manually repositioning the angle of the device relative to the eye due to patient's inability to fixate. Multi-image processing steps are employed to improve wide-field viewing of the retina. All images were taken of bed-bound patients under normal inpatient care conditions without additional head stabilization or repositioning. (B, B′) Image stitching was performed using the i2k DualAlign software package.
Figure 4
 
Wide-field retinal imaging with CellScope Retina under typical and challenging imaging conditions in the inpatient setting. (A) Retinal montage of presumed fungal endophthalmitis in a patient with a well-dilated 8.0-mm pupil who was alert and cooperative with examination demonstrating three creamy white, well-circumscribed 500-μm lesions within 2 mm of the optic nerve. (A') Representative images used for retinal montage. Images A and A' were acquired in less than 1 minute by guiding the patient's eyes with device-assisted fixation. (B) Retinal montage of presumed fungal endophthalmitis in a hospitalized patient with a poorly dilated 4.5-mm pupil and altered mental status demonstrating a 16-mm large creamy white chorioretinal lesion with associated retinal hemorrhages, retinal vasculitis, and vitritis extending from the superior arcade into the macula and threatening the fovea. (B') Representative images demonstrating glare and restricted retinal view as present with small pupil size. These images were acquired in approximately 4 minutes by manually repositioning the angle of the device relative to the eye due to patient's inability to fixate. Multi-image processing steps are employed to improve wide-field viewing of the retina. All images were taken of bed-bound patients under normal inpatient care conditions without additional head stabilization or repositioning. (B, B′) Image stitching was performed using the i2k DualAlign software package.
Retinal Imaging in Hospital Settings
The CellScope Retina was also evaluated outside traditional eye care facilities to assess its feasibility of use for retinal image acquisition in the inpatient and emergency department setting. Here, we demonstrate wide-field retinal photography using CellScope Retina in the inpatient setting (Fig. 4). Images of two patients with presumed fungal endophthalmitis were taken at the bedside without either repositioning the patient or using head stabilization devices. The first patient was cooperative with the examination and dilated well to 8 mm with pharmacologic mydriasis, enabling 100° image acquisition in less than 1 minute using device-assisted fixation. Fundus photography demonstrates three creamy white, well-circumscribed 500-μm lesions within 2 mm of the optic nerve (Fig. 4A). The second patient was critically ill with altered mental status and had a poorly-dilated pupil of 4.5 mm despite pharmacologic mydriasis. Multiple images were acquired over the span of approximately 4 minutes by repositioning the angle of the device relative to the eye due to the patient's inability to fixate (Fig. 4B). Under these conditions, individual images exhibited significant glare and restricted views of the retina (Fig. 4B'). Multi-image stitching was employed to produce a greater than 100° view of the retina. Fundus photography demonstrates a 16-mm large, creamy-white chorioretinal lesion with associated retinal hemorrhages, retinal vasculitis, and vitritis extending from the superior arcade into the macula and threatening the fovea (Fig. 4B). 
Utility in Diabetic Retinopathy Screening
The CellScope Retina was further evaluated as a DR screening tool to assess its ability to resolve retinal findings in DR. We demonstrate two examples of retinal imaging in patients with RWDR (Fig. 5). The first patient demonstrates clear retinal abnormalities including panretinal photocoagulation (PRP) in both eyes. The right eye remains normal while the left eye shows re-activation of PDR with preretinal hemorrhage (Figs. 5A, 5B, respectively). The second patient is diabetic without a known prior history of retinopathy. Imaging with CellScope Retina in the left eye resolves trace neovascularization and sparse microaneurysms (Figs. 5D, 5E, respectively). Additionally, post processing of these images with red-subtraction was employed to enhance contrast of vascular structure within the superficial retinal layers (Figs. 5D', 5E'), an approach often used in traditional fundus photography. 
Figure 5
 
Imaging of RWDR with CellScope Retina. Fundus photographs of a patient with diabetes mellitus with right (A) and left (B) eye demonstrating prior PRP and re-activation of quiescent diabetic retinopathy with preretinal hemorrhage in the left eye. (C) Fundus photograph of the left eye in a patient with diabetes mellitus but no known history of DR. CellScope Retina resolves trace signs of neovascularization (D) and sparse microaneurysms (E). Postprocessing with red-subtraction can be used to enhance contrast of vascular structures in the superficial retina, including neovascularization (D') and microaneurysms (E').
Figure 5
 
Imaging of RWDR with CellScope Retina. Fundus photographs of a patient with diabetes mellitus with right (A) and left (B) eye demonstrating prior PRP and re-activation of quiescent diabetic retinopathy with preretinal hemorrhage in the left eye. (C) Fundus photograph of the left eye in a patient with diabetes mellitus but no known history of DR. CellScope Retina resolves trace signs of neovascularization (D) and sparse microaneurysms (E). Postprocessing with red-subtraction can be used to enhance contrast of vascular structures in the superficial retina, including neovascularization (D') and microaneurysms (E').
Pilot Testing for Diabetic Retinopathy Screening
In order to evaluate the potential utility of CellScope Retina for DR grading, a pilot test of 71 subjects and 142 eyes were enrolled from the University of Michigan Kellogg Eye Center Retina Clinic. Forty-five subjects were male, 26 were female, and the mean age was 56.7 years (standard deviation 16.9 years). CellScope Retina imaging was performed in 121 eyes (84%). The remaining 21 eyes were not imaged because patients were pharmacologically dilated in only one eye or were discharged from clinic prior to imaging both eyes. Of the 121 images evaluated by the graders, a grade of “Excellent” was given 21 times, a grade of “acceptable” was given 98 times, a grade of “fair” was given 52 times, and “ungradable” was given 2 times. 
Detecting Referral-Warranted Diabetic Retinopathy
The sensitivity and specificity of referral-warranted DR detection by grader 1 were 91.4% (95% CI 83.0, 96.5), and 64.9% (95% CI 47.5, 79.8), respectively (Table 2). The positive and negative predictive values were 85.1% (95% CI 78.5–89.9), and 77.4% (95% CI 61.9–87.9), respectively. For grader 2, the sensitivity and specificity for referral-warranted DR detection were 95.1% (95% CI 88.0–98.7), and 48.7% (95% CI 31.9–65.6), respectively (Table 2). The positive and negative predictive values were 80.4% (95% CI 74.9–84.9), and 81.8% (95% CI 62.1–92.5), respectively. 
Agreement between Image-Based Grading and Clinical Exam
Compared with the clinical examination, for grader 1, there was exact agreement in DR severity grading in 57.3% of the eyes, and agreement within one clinical stage in 88.9% of the eyes, weighted κ = 0.63 (95% CI 0.53–0.73, Table 1). For grader 2, there was exact agreement in 52.9% of the eyes, and agreement within one clinical stage in 81.5% of the eyes, weighted κ = 0.55 (95% CI 0.45–0.64, Table 1). The two graders agreed perfectly 56.3% of the time and were within one step of perfect agreement 95.0% of the time (Fig. 6), with weighted κ = 0.66 (95% CI 0.57–0.74; c.f. Supplementary Table S1). 
Table 1
 
Grader Assessments of Diabetic Retinopathy Severity of CellScope Retina Images Versus Clinical Fundus Exam
Table 1
 
Grader Assessments of Diabetic Retinopathy Severity of CellScope Retina Images Versus Clinical Fundus Exam
Table 2
 
Grader Sensitivity and Specificity of Referral-Warranted Diabetic Retinopathy
Table 2
 
Grader Sensitivity and Specificity of Referral-Warranted Diabetic Retinopathy
Figure 6
 
Agreement in DR severity between masked reading of CellScope Retina images and clinical examination. DR severity was determined by two masked graders and compared with grading from the clinical examination. For grader 1, there was exact agreement in 57.3% of the eyes, and agreement within one step in 88.9% of the eyes (kappa coefficient was 0.63, 95% CI, 0.53–0.73). Grader 1 was also able to detect 91.4% of RWDR as compared with the dilated fundus examination. For grader 2, there was exact agreement in 52.9% of the eyes, and agreement within one step in 81.5% of the eyes (kappa coefficient was 0.55, 95% CI, 0.45–0.65). Grader 2 was also able to detect 95.1% of RWDR as compared with the dilated fundus examination.
Figure 6
 
Agreement in DR severity between masked reading of CellScope Retina images and clinical examination. DR severity was determined by two masked graders and compared with grading from the clinical examination. For grader 1, there was exact agreement in 57.3% of the eyes, and agreement within one step in 88.9% of the eyes (kappa coefficient was 0.63, 95% CI, 0.53–0.73). Grader 1 was also able to detect 91.4% of RWDR as compared with the dilated fundus examination. For grader 2, there was exact agreement in 52.9% of the eyes, and agreement within one step in 81.5% of the eyes (kappa coefficient was 0.55, 95% CI, 0.45–0.65). Grader 2 was also able to detect 95.1% of RWDR as compared with the dilated fundus examination.
Detecting Clinically Significant Diabetic Macular Edema
For grader 1, the sensitivity and specificity for detecting CSDME was 29.4% (95% CI 10.3–56.0) and 98.0% (95% CI 92.9–99.8). The positive and negative predictive values were 71.4% (95% CI 34.5–92.2) and 89.0% (95% CI 85.6–91.7), respectively. For grader 2, the sensitivity and specificity for detecting CSDME were 68.8% (95% CI 41.3–89.0) and 83.3% (95% CI 74.4–90.2), respectively. The positive and negative predictive values were 40.7% (95% CI 28.3–54.5) and 94.1% (95% CI 88.5–97.1), respectively (c.f. Supplementary Table S2). CSDME with or without DR was classified as referral-warranted disease. The in-clinical diagnosis of CSDME was assisted by use of macular OCT. 
Patient Comfort
Patient comfort levels during examination with the red illumination used to focus on the retina (Fig. 1C) and the white flashes for image capture (Fig. 1D) were surveyed according to a three-point scale, including ‘comfortable', ‘bearable', or ‘uncomfortable'. All of the surveyed patients (Supplementary Fig. S1) reported the continuous red illumination to be comfortable throughout the examination (N = 87/87). Of patients, 90.8% reported the white, image-capture LED flash to be comfortable (N = 79/87), 8.0% reported the white flash as bearable (N = 7/87), and 1.1% reported the white flash to be uncomfortable (N = 1/87). Additionally, 100% of patients who also underwent slit-lamp examination reported the CellScope Retina illumination system to be more comfortable than the illumination from the slit lamp (N = 72/72). 
Discussion
Smartphone-based retinal photography holds great promise in ophthalmic care and telemedicine, via leveraging of the smartphone for its portability, low-cost, ease-of-use, high-resolution camera, and wireless connectivity in order to capture diagnostic retinal images. This study highlights two important aspects of a novel, smartphone-based retinal imaging tool, the CellScope Retina. First, CellScope Retina was capable of capturing and stitching wide-field, 100° images of a broad variety of retinal pathologies in the nonoptimal imaging conditions of an ophthalmic consultation service and emergency department setting. Second, the CellScope Retina achieved good sensitivity and moderate specificity (with high intergrader agreement) when operated by nonophthalmic personnel for detection of RWDR in a feasibility study in this population. Furthermore, the portability and ease of one-handed operation of CellScope Retina make it possible for the device to increase access to high-quality retinal imaging when compared with benchtop fundus cameras. 
There has been substantial effort in recent years to incorporate smartphone-based devices in screening and monitoring of diseases, such as glaucoma and DR.1315 In particular, three recent studies demonstrate that smartphone-based retinal imaging can be effective in screening for DR.4,16,39 Rajalakshmi et al.4 compared grading of DR from images acquired by the Fundus on Phone (FOP) system with traditional benchtop fundus photography. Using the FOP system, sensitivity and specificity for any DR was reported as 92.7% and 98.4%, respectively, and for sight-threatening DR (defined as proliferative DR or CSDME) reported as 87.9% and 94.9%, respectively.4 Russo et al.39 compared grading of DR from images acquired by the D-Eye smartphone system against clinical examination with slit-lamp biomicroscopy. They reported exact agreement in grading of severity of DR in 85% of cases and within one step of agreement in 96.7%, and a sensitivity and specificity for clinically significant macular edema as 81% and 98%, respectively.39 Toy et al.16 compared grading of DR from images acquired by a smartphone with macrolens adapter and external light source compared with clinical examination. They reported that smartphone-acquired photos demonstrated 91% sensitivity and 99% specificity in detection of moderate nonproliferative DR and worse DR.16 
While these results are encouraging, the prior literature has not addressed whether handheld retinal photography could be made simple and intuitive for nonexpert operators in nonideal settings. Indeed, experience with our first generation Ocular CellScope prototype demonstrated that smartphone imaging of the retina, while highly possible, could also be highly variable. The FOP system used an indirect ophthalmoscopy arrangement with a field of view of up to 45° with mydriasis, which they reported could be used as a handheld system or adapted to a slit-lamp frame.4 However, the FOP system has a relatively large size and the study did not disclose who performed photography or whether the device was mounted to the slit lamp. Rigid stabilization of the camera and patient's head helps with imaging quality but is not readily available in many community-based screening environments. The D-Eye uses a direct ophthalmoscopy arrangement with a field-of-view of up to 20° with mydriasis. This requires significant manual adjustment and familiarity with retinal anatomy to perform complete retinal imaging, and images in their study were acquired by a retina specialist.39 
Variability in image quality that could arise from nonideal settings and nonexpert operators is a critical consideration when screening in the community.37,38 In order to assess this potential limitation with the CellScope Retina, we performed a feasibility study wherein image acquisition for grading of DR was performed by a medical student and medicine intern rather than ophthalmologists or ophthalmic photographers. We found that even when image acquisition was performed by nonexpert operators, grading of CellScope Retina images demonstrated good agreement for DR and CSDME compared with clinical examination by slit-lamp biomicroscopy, consistent with other photographic-based methods,49 and met British Diabetic Association guidelines for sensitivity in screening tools used in DR.40 This was achieved, in part, by leveraging the processing power of the phone to simplify operational steps in image acquisition. In particular, we focused on portable, wide-field imaging due to the importance of peripheral retina surveillance in diseases, such as DR. Previous studies have found that a single 45° field of view retinal image had relatively good detection of any DR but was inadequate to effectively determine severity of DR as needed for referral.3436 Additionally, a prospective study using ultrawide field imaging with 200-degree field of view (Optos plc) found that approximately one-third of lesions (hemorrhages, microaneurysms, intraretinal microvascular abnormalities, neovascularization) were outside traditional ETDRS seven-field photography (90° field of view), and 10% of patients had a more severe grade of DR than determined by ETDRS photography.50 These studies suggest that wider-field surveillance of the retina may improve accuracy in screening of DR. At the same time, cost and speed are equally important considerations for screening in community settings. Commercially available, handheld systems can provide very high-quality imaging even under nonmydriatic conditions. However, these dedicated systems can cost many thousands of dollars, require moderate training and experience to operative reliably, and need to be paired with an external computer to store, interpret, and transmit images. These conditions may be prohibitive in resource limited settings where retinal screening is most needed. Therefore, the CellScope Retina was designed to maximize retinal field of view while being fast, easy to use, and inexpensive. We leverage the familiar interface and processing power of smartphones to simplify and automate elements of wide-field imaging for even inexperienced operators. Images can be stored, reviewed, and wirelessly transmitted directly from the smartphone without an external computer. Additionally, the processing and wireless capabilities of smartphone-based systems hold great promise for implementing advanced features, such as artificial intelligence-assisted evaluation of retinal images. 
This study has several important strengths. First, masked grading of DR was performed without providing patients' medical history, helping to eliminate bias and ensuring assessment was based purely on photographic findings. Second, nonophthalmic operators acquired images for DR grading and, as a result, images of lesser quality were included in analysis and more closely simulated conditions that may be encountered when screening in the primary care setting. Nevertheless, CellScope Retina images both demonstrated good agreement for DR and met British Diabetic Association guidelines for sensitivity of screening tools used in DR.40 Third, complex imaging tasks, such as imaging multiple regions of the retina for wide-field analysis, were simplified and standardized by leveraging computational capabilities of the phone to guide imaging. Further work in this direction has the potential to drastically simplify and improve the efficacy of retinal screening in the community. 
There are several limitations to this study. First, participants for the DR study were recruited from the retina clinic in a tertiary care eye hospital, where the prevalence of patient DR is much higher than in the general population. While our feasibility study shows promising results, additional work is required to validate the accuracy and utility of the CellScope Retina in the primary care and community-based settings where the technology is most needed. Second, CellScope Retina is currently designed as a mydriatic device that requires patients' eyes to be pharmacologically dilated. This can be time-consuming and uncomfortable for patients while also being unfamiliar to primary care physicians. Third, a medical student and a medicine intern acquired the images for the DR study, and additional validation for ease of imaging would be called for if shifting to use by medical assistants and primary care personnel. 
Smartphone-based retinal imaging offers the possibility of dramatically increasing the accessibility of ophthalmic care by remote screening and diagnosis of vision-threatening diseases among millions of individuals not receiving regular eye examinations. Achieving this goal will rely not only on the familiarity and portability of smartphones, but also on the development of novel techniques that enhance image quality and reliability. We aimed to make wide-field retinal imaging intuitive and reproducible for the nonexpert operator. The resulting CellScope Retina device harnesses hardware and software automation controlled by a smartphone to simplify retinal imaging, potentially improving access to ophthalmic care through handheld and portable retinal disease screening and diagnosis. 
Acknowledgments
This work was supported in part by the Michigan Translational Research and Commercialization program with support from MEDC and the UM Medical School (TNK), National Eye Institute Grant no. K23 EY023596 (MAW), QB3 Bridging the Gap Awards from the Rogers Family Foundation (DAF), and a Bakar Fellows Award (DAF). DAF is a Chan-Zuckerberg Biohub investigator. 
Competing financial interests 
Disclosure: T.N. Kim, is an inventor on US Patent Application 20160296112, “Retinal CellScope Apparatus”, (P); F. Myers, is an inventor on US Patent Application 20160296112, “Retinal CellScope Apparatus”, (P); C. Reber, is an inventor on US Patent Application 20160296112, “Retinal CellScope Apparatus”, (P); P.J. Loury, None; P. Loumou, None; D. Webster, None; C. Echanique, None; P. Li, None; J.R. Davila, None; R.N. Maamari, is an inventor on several issued US patents and related applications regarding “High Numerical Aperture Telemicroscopy Apparatus”, and on US Patent Application 20160296112, “Retinal CellScope Apparatus”, (P); N.A. Switz, holds shares in CellScope, Inc. The shares are held by The Regents of the University of California and disposition is determined by them according to a fixed formula (I), CellScope, Inc. neither had nor has any involvement with the project described in this paper. He is an inventor on several issued US patents and related applications regarding “High Numerical Aperture Telemicroscopy Apparatus”, and on US Patent Application 20160296112, “Retinal CellScope Apparatus”, (P); J. Keenan, None; M.A. Woodward, None; Y.M. Paulus, None; T. Margolis, is an inventor on US Patent Application 20160296112, “Retinal CellScope Apparatus”, (P); D.A. Fletcher, is a co-founder of CellScope, Inc., a company commercializing a cellphone-based otoscope. He holds shares in CellScope, Inc. (I), CellScope, Inc. neither had nor has any involvement with the project described in this paper. He is an inventor on several issued US patents and related applications regarding “High Numerical Aperture Telemicroscopy Apparatus”, on a US Patent Application 20160296112, “Retinal CellScope Apparatus”, (P) and on US Patent Application 20160296112, “Retinal CellScope Apparatus” (P). 
References
Shields CL, Materin M, Shields JA. Panoramic imaging of the ocular fundus. Arch Ophthalmol. 2003; 121: 1603–1607.
Suto S, Hiraoka T, Okamoto Y, Okamoto F, Oshika T. Photography of anterior eye segment and fundus with smartphone [in Japanese]. Nippon Ganka Gakkai Zasshi. 2014; 118: 7–14.
Maamari RN, Keenan JD, Fletcher DA, Margolis TP. A mobile phone-based retinal camera for portable wide field imaging. Br J Ophthalmol. 2014; 98: 438–441.
Rajalakshmi R, Arulmalar S, Usha M, et al. Validation of smartphone based retinal photography for diabetic retinopathy screening. PLoS One. 2015; 10: e0138285.
Lord RK, Shah VA, San Filippo AN, Krishna R. Novel uses of smartphones in ophthalmology. Ophthalmology. 2010; 117: 1274–1274, e1273.
Haddock LJ, Kim DY, Mukai S. Simple, inexpensive technique for high-quality smartphone fundus photography in human and animal eyes. J Ophthalmol. 2013; 2013: 518479.
Bastawrous A. Smartphone fundoscopy. Ophthalmology. 2012; 119: 432–433, e432; author reply 433.
Giardini ME, Livingstone IA, Jordan S, et al. A smartphone based ophthalmoscope. Conf Proc IEEE Eng Med Biol Soc. 2014; 2014: 2177–2180.
Hong SC. 3D printable retinal imaging adapter for smartphones could go global. Graefes Arch Clin Exp Ophthalmol. 2015; 253: 1831–1833.
Russo A, Morescalchi F, Costagliola C, Delcassi L, Semeraro F. A novel device to exploit the smartphone camera for fundus photography. J Ophthalmol. 2015; 2015: 823139.
Bastawrous A, Giardini ME, Bolster NM, et al. Clinical validation of a smartphone-based adapter for optic disc imaging in Kenya. JAMA Ophthalmol. 2016; 134: 151–158.
Myung DJ, Jais A, He L, Blumenkranz MS, Chang RT. 3D Printed smartphone indirect lens adapter for rapid, high quality retinal imaging. J MTM. 2014; 3: 9–15.
Bolster NM, Giardini ME, Bastawrous A. The diabetic retinopathy screening workflow: potential for smartphone imaging. J Diabetes Sci Technol. 2016; 10: 318–324.
Garg SJ. Applicability of smartphone-based screening programs. JAMA Ophthalmol. 2016; 134: 158–159.
Micheletti JM, Hendrick AM, Khan FN, Ziemer DC, Pasquel FJ. Current and next generation portable screening devices for diabetic retinopathy. J Diabetes Sci Technol. 2016; 10: 295–300.
Toy BC, Myung DJ, He L, et al. Smartphone-based dilated fundus photography and near visual acuity testing as inexpensive screening tools to detect referral warranted diabetic eye disease. Retina. 2016; 36: 1000–1008.
Bruce BB, Thulasi P, Fraser CL, et al. Diagnostic accuracy and use of nonmydriatic ocular fundus photography by emergency physicians: phase II of the FOTO-ED study. Ann Emerg Med. 2013; 62: 28–33, e21.
Lamirel C, Bruce BB, Wright DW, Newman NJ, Biousse V. Nonmydriatic digital ocular fundus photography on the iPhone 3G: the FOTO-ED study. Arch Ophthalmol 2012; 130: 939–940.
Woodward MA, Bavinger JC, Amin S, et al. Telemedicine for ophthalmic consultation services: use of a portable device and layering information for graders. J Telemed Telecare. 2017; 23: 365–370.
Harris MI. Diabetes in America: epidemiology and scope of the problem. Diabetes Care. 1998; 21 (Suppl 3): C11–C14.
Mokdad AH, Ford ES, Bowman BA, et al. Diabetes trends in the U.S.: 1990-1998. Diabetes Care. 2000; 23: 1278–1283.
Duh EJ, Aiello LP. Basic Pathobiology of the eye and its complications. In: Porte DE, Jr. et al., eds. Ellenberg & Rifkin's Diabetes Mellitus. New York, NY: McGraw-Hill, 2002.
Diabetic Retinopathy: What you should know. US Dept. of Health and Human Services, NIH, NEI. https://nei.nih.gov/sites/default/files/Diabetic-Retinopathy-What-You-Should-Know-508.pdf. Accessed September 11, 2017.
Facts about diabetic eye disease. National Eye Institute. https://nei.nih.gov/health/diabetic/retinopathy. Accessed September 11, 2017.
Techniques for scatter and local photocoagulation treatment of diabetic retinopathy: early Treatment Diabetic Retinopathy Study Report no. 3. The Early Treatment Diabetic Retinopathy Study Research Group. Int Ophthalmol Clin. 1987; 27: 254–264.
Diabetic Retinopathy Clinical Research Network, Elman MJ, Aiello LP, et al . Randomized trial evaluating ranibizumab plus prompt or deferred laser or triamcinolone plus prompt laser for diabetic macular edema. Ophthalmology. 2010; 117: 1064–1077, e1035.
Photocoagulation for diabetic macular edema. Early Treatment Diabetic Retinopathy Study report number 1. Early Treatment Diabetic Retinopathy Study research group. Arch Ophthalmol. 1985; 103: 1796–1806.
American Diabetes Association. Standards of medical care in diabetes. Diabetes Care. 2004; 27 (Suppl 1): S15–S35.
Harris MI, Klein R, Cowie CC, Rowland M, Byrd-Holt DD. Is the risk of diabetic retinopathy greater in non-Hispanic blacks and Mexican Americans than in non-Hispanic whites with type 2 diabetes? A U.S. population study. Diabetes Care. 1998; 21: 1230–1235.
MacCuish AC. Early detection and screening for diabetic retinopathy. Eye (Lond). 1993; 7 (Pt 2): 254–259.
Li Z, Wu C, Olayiwola JN, Hilaire DS, Huang JJ. Telemedicine-based digital retinal imaging vs standard ophthalmologic evaluation for the assessment of diabetic retinopathy. Conn Med. 2012; 76: 85–90.
Pasquel FJ, Hendrick AM, Ryan M, Cason E, Ali MK, Narayan KM. Cost-effectiveness of different diabetic retinopathy screening modalities. J Diabetes Sci Technol. 2015; 10: 301–307.
Grading diabetic retinopathy from stereoscopic color fundus photographs–an extension of the modified Airlie House classification. ETDRS report number 10. Early Treatment Diabetic Retinopathy Study Research Group. Ophthalmology. 1991; 98: 786–806.
Vujosevic S, Benetti E, Massignan F, et al. Screening for diabetic retinopathy: 1 and 3 nonmydriatic 45-degree digital fundus photographs vs 7 standard early treatment diabetic retinopathy study fields. Am J Ophthalmol. 2009; 148: 111–118.
Aptel F, Denis P, Rouberol F, Thivolet C. Screening of diabetic retinopathy: effect of field number and mydriasis on sensitivity and specificity of digital fundus photography. Diabetes Metabol. 2008; 34: 290–293.
Murgatroyd H, Ellingford A, Cox A, et al. Effect of mydriasis and different field strategies on digital image screening of diabetic eye disease. Br J Ophthalmol. 2004; 88: 920–924.
Ryan ME, Rajalakshmi R, Prathiba V, et al. Comparison Among Methods of Retinopathy Assessment (CAMRA) study: smartphone, nonmydriatic, and mydriatic photography. Ophthalmology. 2015; 122: 2038–2043.
Ludwig CA, Murthy SI, Pappuru RR, Jais A, Myung DJ, Chang RT. A novel smartphone ophthalmic imaging adapter: user feasibility studies in Hyderabad, India. Indian J Ophthalmol. 2016; 64: 191–200.
Russo A, Morescalchi F, Costagliola C, Delcassi L, Semeraro F. Comparison of smartphone ophthalmoscopy with slit-lamp biomicroscopy for grading diabetic retinopathy. Am J Ophthalmol. 2015; 159: 360–364, e361.
British Diabetic Association. Retinal Photographic Screening for Diabetic Eye Disease. A British Diabetic Association Report. London: British Diabetic Association. 1997.
Darma S, Zantvoord F, Verbraak FD. The quality and usability of smartphone and hand-held fundus photography, compared to standard fundus photography. Acta Ophthalmol. 2015; 93: e310–e311.
Adam MK, Brady CJ, Flowers AM, et al. Quality and diagnostic utility of mydriatic smartphone photography: the Smartphone Ophthalmoscopy Reliability Trial. Ophthalmic Surg Lasers Imaging Retina. 2015; 46: 631–637.
Panwar N, Huang P, Lee J, et al. Fundus Photography in the 21st century–a review of recent technological advances and their implications for worldwide healthcare. Telemed J E Health. 2016; 22: 198–208.
Facey K, Cummins E, Macpherson K, Morris A, Reay L, Slattery J. Organisation of Services for diabetic retinopathy screening. Health Technology Assessment Report 1 Glasgow: Health Technology Board for Scotland 2002. http://www.ndrs-wp.scot.nhs.uk/wp-content/uploads/2013/04/hta1.pdf. Accessed September 11, 2017.
Landry RJ, Bostrom RG, Miller SA, Shi D, Sliney DH. Retinal phototoxicity: a review of standard methodology for evaluating retinal optical radiation hazards. Health Phys. 2011; 100: 417–434.
Bresnick GH, Mukamel DB, Dickinson JC, Cole DR. A screening approach to the surveillance of patients with diabetes for the presence of vision-threatening retinopathy. Ophthalmology. 2000; 107: 19–24.
Cuadros J, Bresnick G. EyePACS: an adaptable telemedicine system for diabetic retinopathy screening. J Diabetes Sci Technol. 2009; 3: 509–516.
Mercaldo ND, Lau KF, Zhou XH. Confidence intervals for predictive values with an emphasis to case-control studies. Stat Med. 2007; 26: 2170–2183.
Williams GA, Scott IU, Haller JA, Maguire AM, Marcus D, McDonald HR. Single-field fundus photography for diabetic retinopathy screening: a report by the American Academy of Ophthalmology. Ophthalmology. 2004; 111: 1055–1062.
Silva PS, Cavallerano JD, Sun JK, Soliman AZ, Aiello LM, Aiello LP. Peripheral lesions identified by mydriatic ultrawide field imaging: distribution and potential impact on diabetic retinopathy severity. Ophthalmology. 2013; 120: 2587–2595.
Figure 1
 
CellScope Retina schematic and workflow. (A) Schematic of the optical system. Light from LEDs is directed through a mask and forms an annulus that passes through the peripheral cornea and focuses through the pupil. In propagating through the eye the light becomes defocused, providing even illumination at the retina. Polarization filters minimize unwanted reflections from anterior ocular surfaces, enabling the smartphone to capture a clear image of the retina. (B) The compact optical system and custom-control electronics fit inside a handheld enclosure. (C) Red LED illumination of 655-nm peak emission is used for focusing on the retina, which is within the spectral range of the iPhone camera but outside the peak photopic response of the eye. (D) A white LED with a broad emission spectrum is flashed for recording images of the retina. LED spectra in (C) and (D) are from respective datasheets; photopic response is from CIE 1931 standard; phone response is approximate for a CMOS phone sensor. (E) Smartphone user interface enables (1) patient data capture, (2) preview during focus/alignment with swipe gestures adjusting camera settings, and (3) exam data review with pinch and swipe gestures for browsing stitched image montage. (F) Photos can be uploaded directly from the smartphone to a cloud database allowing remote diagnosis with a web interface.
Figure 1
 
CellScope Retina schematic and workflow. (A) Schematic of the optical system. Light from LEDs is directed through a mask and forms an annulus that passes through the peripheral cornea and focuses through the pupil. In propagating through the eye the light becomes defocused, providing even illumination at the retina. Polarization filters minimize unwanted reflections from anterior ocular surfaces, enabling the smartphone to capture a clear image of the retina. (B) The compact optical system and custom-control electronics fit inside a handheld enclosure. (C) Red LED illumination of 655-nm peak emission is used for focusing on the retina, which is within the spectral range of the iPhone camera but outside the peak photopic response of the eye. (D) A white LED with a broad emission spectrum is flashed for recording images of the retina. LED spectra in (C) and (D) are from respective datasheets; photopic response is from CIE 1931 standard; phone response is approximate for a CMOS phone sensor. (E) Smartphone user interface enables (1) patient data capture, (2) preview during focus/alignment with swipe gestures adjusting camera settings, and (3) exam data review with pinch and swipe gestures for browsing stitched image montage. (F) Photos can be uploaded directly from the smartphone to a cloud database allowing remote diagnosis with a web interface.
Figure 2
 
CellScope Retina is a handheld, smartphone-based retinal camera that enables fully integrated wide-field retinal imaging. (A, B) The device is designed for easy handheld operation. (B) A detachable display may be magnetically mounted to either side of the device and provides a green dot as a fixation target. (C) The fixation target is translated through a series of positions, re-orienting the patient's eyes and retina in a rapid and controllable fashion. (D) Each individual image is approximately 50°. With the current automated arrangement, five overlapping images are captured of the central, superior, nasal, temporal, and inferior retina. (E) These five images are computationally merged on the phone using custom software to create a wide-field image of the retina spanning approximately 100°.
Figure 2
 
CellScope Retina is a handheld, smartphone-based retinal camera that enables fully integrated wide-field retinal imaging. (A, B) The device is designed for easy handheld operation. (B) A detachable display may be magnetically mounted to either side of the device and provides a green dot as a fixation target. (C) The fixation target is translated through a series of positions, re-orienting the patient's eyes and retina in a rapid and controllable fashion. (D) Each individual image is approximately 50°. With the current automated arrangement, five overlapping images are captured of the central, superior, nasal, temporal, and inferior retina. (E) These five images are computationally merged on the phone using custom software to create a wide-field image of the retina spanning approximately 100°.
Figure 3
 
CellScope Retina produces diagnostic-quality photographs of retinal abnormalities. (A) Hypertensive retinopathy with ruptured RAM along the inferior arcade with hemorrhage in multiple layers. (B) Superotemporal bullous macula-involving rhegmatogenous retinal detachment in the right eye extending from 9:00 to 1:00 o'clock. (C) Peripapillary myelinated nerve fiber layer with flame-shaped appearance and feathered borders in the left eye. (D) Circinate retinal exudates in the inferotemporal macula of the right eye. (E) Acute papilledema with flame hemorrhages extending from the optic disc and proximally along the inferior arcade of the left eye. (F) DR with dot-blot and disc hemorrhages in the right eye. (G) CMV retinitis demonstrating broad retinal whitening along the superior arcade of the left eye. (H) Morning Glory disc anomaly in the left eye. (I) Epiretinal membrane with fibrovascular changes in the left eye. All images were acquired after pharmacologic mydriasis.
Figure 3
 
CellScope Retina produces diagnostic-quality photographs of retinal abnormalities. (A) Hypertensive retinopathy with ruptured RAM along the inferior arcade with hemorrhage in multiple layers. (B) Superotemporal bullous macula-involving rhegmatogenous retinal detachment in the right eye extending from 9:00 to 1:00 o'clock. (C) Peripapillary myelinated nerve fiber layer with flame-shaped appearance and feathered borders in the left eye. (D) Circinate retinal exudates in the inferotemporal macula of the right eye. (E) Acute papilledema with flame hemorrhages extending from the optic disc and proximally along the inferior arcade of the left eye. (F) DR with dot-blot and disc hemorrhages in the right eye. (G) CMV retinitis demonstrating broad retinal whitening along the superior arcade of the left eye. (H) Morning Glory disc anomaly in the left eye. (I) Epiretinal membrane with fibrovascular changes in the left eye. All images were acquired after pharmacologic mydriasis.
Figure 4
 
Wide-field retinal imaging with CellScope Retina under typical and challenging imaging conditions in the inpatient setting. (A) Retinal montage of presumed fungal endophthalmitis in a patient with a well-dilated 8.0-mm pupil who was alert and cooperative with examination demonstrating three creamy white, well-circumscribed 500-μm lesions within 2 mm of the optic nerve. (A') Representative images used for retinal montage. Images A and A' were acquired in less than 1 minute by guiding the patient's eyes with device-assisted fixation. (B) Retinal montage of presumed fungal endophthalmitis in a hospitalized patient with a poorly dilated 4.5-mm pupil and altered mental status demonstrating a 16-mm large creamy white chorioretinal lesion with associated retinal hemorrhages, retinal vasculitis, and vitritis extending from the superior arcade into the macula and threatening the fovea. (B') Representative images demonstrating glare and restricted retinal view as present with small pupil size. These images were acquired in approximately 4 minutes by manually repositioning the angle of the device relative to the eye due to patient's inability to fixate. Multi-image processing steps are employed to improve wide-field viewing of the retina. All images were taken of bed-bound patients under normal inpatient care conditions without additional head stabilization or repositioning. (B, B′) Image stitching was performed using the i2k DualAlign software package.
Figure 4
 
Wide-field retinal imaging with CellScope Retina under typical and challenging imaging conditions in the inpatient setting. (A) Retinal montage of presumed fungal endophthalmitis in a patient with a well-dilated 8.0-mm pupil who was alert and cooperative with examination demonstrating three creamy white, well-circumscribed 500-μm lesions within 2 mm of the optic nerve. (A') Representative images used for retinal montage. Images A and A' were acquired in less than 1 minute by guiding the patient's eyes with device-assisted fixation. (B) Retinal montage of presumed fungal endophthalmitis in a hospitalized patient with a poorly dilated 4.5-mm pupil and altered mental status demonstrating a 16-mm large creamy white chorioretinal lesion with associated retinal hemorrhages, retinal vasculitis, and vitritis extending from the superior arcade into the macula and threatening the fovea. (B') Representative images demonstrating glare and restricted retinal view as present with small pupil size. These images were acquired in approximately 4 minutes by manually repositioning the angle of the device relative to the eye due to patient's inability to fixate. Multi-image processing steps are employed to improve wide-field viewing of the retina. All images were taken of bed-bound patients under normal inpatient care conditions without additional head stabilization or repositioning. (B, B′) Image stitching was performed using the i2k DualAlign software package.
Figure 5
 
Imaging of RWDR with CellScope Retina. Fundus photographs of a patient with diabetes mellitus with right (A) and left (B) eye demonstrating prior PRP and re-activation of quiescent diabetic retinopathy with preretinal hemorrhage in the left eye. (C) Fundus photograph of the left eye in a patient with diabetes mellitus but no known history of DR. CellScope Retina resolves trace signs of neovascularization (D) and sparse microaneurysms (E). Postprocessing with red-subtraction can be used to enhance contrast of vascular structures in the superficial retina, including neovascularization (D') and microaneurysms (E').
Figure 5
 
Imaging of RWDR with CellScope Retina. Fundus photographs of a patient with diabetes mellitus with right (A) and left (B) eye demonstrating prior PRP and re-activation of quiescent diabetic retinopathy with preretinal hemorrhage in the left eye. (C) Fundus photograph of the left eye in a patient with diabetes mellitus but no known history of DR. CellScope Retina resolves trace signs of neovascularization (D) and sparse microaneurysms (E). Postprocessing with red-subtraction can be used to enhance contrast of vascular structures in the superficial retina, including neovascularization (D') and microaneurysms (E').
Figure 6
 
Agreement in DR severity between masked reading of CellScope Retina images and clinical examination. DR severity was determined by two masked graders and compared with grading from the clinical examination. For grader 1, there was exact agreement in 57.3% of the eyes, and agreement within one step in 88.9% of the eyes (kappa coefficient was 0.63, 95% CI, 0.53–0.73). Grader 1 was also able to detect 91.4% of RWDR as compared with the dilated fundus examination. For grader 2, there was exact agreement in 52.9% of the eyes, and agreement within one step in 81.5% of the eyes (kappa coefficient was 0.55, 95% CI, 0.45–0.65). Grader 2 was also able to detect 95.1% of RWDR as compared with the dilated fundus examination.
Figure 6
 
Agreement in DR severity between masked reading of CellScope Retina images and clinical examination. DR severity was determined by two masked graders and compared with grading from the clinical examination. For grader 1, there was exact agreement in 57.3% of the eyes, and agreement within one step in 88.9% of the eyes (kappa coefficient was 0.63, 95% CI, 0.53–0.73). Grader 1 was also able to detect 91.4% of RWDR as compared with the dilated fundus examination. For grader 2, there was exact agreement in 52.9% of the eyes, and agreement within one step in 81.5% of the eyes (kappa coefficient was 0.55, 95% CI, 0.45–0.65). Grader 2 was also able to detect 95.1% of RWDR as compared with the dilated fundus examination.
Table 1
 
Grader Assessments of Diabetic Retinopathy Severity of CellScope Retina Images Versus Clinical Fundus Exam
Table 1
 
Grader Assessments of Diabetic Retinopathy Severity of CellScope Retina Images Versus Clinical Fundus Exam
Table 2
 
Grader Sensitivity and Specificity of Referral-Warranted Diabetic Retinopathy
Table 2
 
Grader Sensitivity and Specificity of Referral-Warranted Diabetic Retinopathy
Supplement 1
Supplement 2
×
×

This PDF is available to Subscribers Only

Sign in or purchase a subscription to access this content. ×

You must be signed into an individual account to use this feature.

×