Demographics Prediction and Heatmap Generation From OCT Images of Anterior Segment of the Eye: A Vision Transformer Model Study

Abstract

Purpose: To predict demographic characteristics from anterior segment optical coherence tomography (AS-OCT) images of eyes using a Vision Transformer (ViT) model.

Methods: A total of 2970 AS-OCT images were used to train, validate, and test a ViT to predict age and sex, and 2616 images were used for height, weight, and body mass index (BMI). The main outcome measure was the area under the receiver operating characteristic curve (AUC) of the ViT.

Results: The ViT achieved the largest AUC (0.910) for differentiating age ≤75 versus >75 years, followed by age ≤60 versus 60–75 versus >75 years (AUC, 0.844), and for discriminating sex (AUC, 0.665). The prediction abilities for the other demographic characteristics were lower: an AUC of 0.521 for classifying height ≤170 versus >170 cm in males and ≤155 versus >155 cm in females; 0.522 for weight <70 versus ≥70 kg in males and 0.503 for <55 versus ≥55 kg in females, and 0.517 for BMI <23 versus 23–25 versus ≥25 kg/m². Heatmaps highlighted the area of the iridocorneal angle for its contribution to the prediction of age ≤75 versus >75 years.

Conclusions: Although the ViT demonstrated a good ability to classify age from AS-OCT images, it performed poorly for sex, height, weight, and BMI. The heatmap obtained of the prediction will provide clues to understanding the age-related anterior segment changes in eyes.

Translational Relevance: The ViT can determine age-related anterior segment structural changes using AS-OCT images, which will aid clinicians in the management of ocular diseases.

Introduction

Anterior-segment optical coherence tomography (AS-OCT) is an imaging modality that enables non-invasive, rapid, and high-resolution visualization of the anterior segment structures of the eye.¹ On the strength of its technological breakthrough, AS-OCT is demonstrating its clinical utility for diagnosis, monitoring, and treatment of various ocular diseases.^2–13 Given the premise that the diagnosis of diseases and determination of their progression are fundamentally based on differentiating abnormalities from the norm, it is vital to figure out natural changes and differences in anterior segment structures related to demographic characteristics. 

Previous AS-OCT imaging studies have revealed anatomic differences of anterior segment structures including the cornea, anterior chamber, iris, iridocorneal angle, and trabecular meshwork as they relate to demographic factors such as age, sex, height, or weight.^14–26 For example, the iris width has been reported to increase with age and to be significantly greater in males.¹⁶ Also, corneal diameter and vault have shown positive correlations with height,²⁷ and anterior chamber depth has been significantly lower in obese subjects.²⁸ 

Recent studies using deep learning (DL) also have reported that the DL algorithm employed successfully predicted demographic characteristics including age and sex from OCT images (Supplementary Table).^29–32 In addition, Vision Transformer (ViT), which is an extended application of Transformer to computer vision, has shown its usefulness in ophthalmology, as well.^33,34 

Notwithstanding the former efforts to relate anterior segment structural differences to demographic characteristics,^14–26 research focusing on diverse demographic variables is lacking, and the results that have been obtained are not validated. Furthermore, whereas there have been several studies predicting demographic characteristic from ocular images using DL, those utilizing AS-OCT images are lacking. Thus motivated, we aimed to validate and test DL models for prediction of demographic characteristics including age, sex, height, weight, and body mass index (BMI) from AS-OCT images of normal eyes and to identify the key structures contributory to such prediction. Our findings will help clinicians to understand how anterior segment structures differ according to demographic characteristics, which in turn will facilitate evaluation and management of ocular diseases in clinical practice. 

Methods

The protocol of this study was prospectively registered at the Open Science Framework (registration no. 10.17605/OSF.IO/FQ46X). The study followed the tenets of the Declaration of Helsinki and was approved by the Big Data Review Board of Seoul National University Hospital (SNUH; Institutional Review Board registration no. H-2104-085-1212). Informed consent was waived due to the retrospective nature of the study. 

Participants and Data Collection

From SUPREME, a clinical data warehouse of SNUH, a list of eligible participants who had undergone AS-OCT imaging (Visante; Carl Zeiss Meditec, Dublin, CA) from January 2008 to December 2020 at SNUH and for whom height and weight measurements taken within 6 months of (before or after) imaging were available was obtained. The AS-OCT images and demographic data on age, sex, height, weight, and BMI were extracted from an electronic medical record database of BESTCare (ezCaretech, Seoul, Korea), an SNUH hospital information system (Fig. 1). As for the demographic data, we obtained them through SUPREME, which matches each patient's data with an identification number. 

Figure 1.

 

Study flowchart. CDW, clinical data warehouse; EMR, electronic medical record; Grad-CAM, gradient-weighted class activation mapping.

For inclusion in the study, participants were required to be 40 years of age or older and to have an open-angle on gonioscopic examinations. The exclusion criteria were as follows: history of prior ocular surgery, laser treatment, or trauma; any ocular or systemic diseases that could affect anterior segment structures, including corneal disease (e.g., corneal opacity, corneal dystrophy, keratoconus), iridocorneal angle abnormality (e.g., angle-closure glaucoma), or ocular inflammatory disease (e.g., uveitis); any medications that could affect anterior segment structures; any systemic diseases or therapies affecting height (e.g., Marfan syndrome) or weight (e.g., metabolic disease, cancer); or AS-OCT images of poor quality or with artifacts. Participants were then independently assessed by two investigators (YJL and SC) for final eligibility. Discrepancies were resolved through discussion and consensus or, if needed, adjudication by a third investigator (YKK). 

AS-OCT Imaging

All of the participants underwent Visante AS-OCT imaging by an experienced examiner at the Department of Ophthalmology, SNUH. Seated, they were asked to fixate on the internal fixation light in the primary gaze position under darkroom conditions. Cross-sectional AS-OCT scanning was performed for each eye along the horizontal axis (nasal–temporal angles at 0°–180°) using the standard anterior segment single-scan protocol entailing 256 A-scans per line obtained at an image acquisition speed of 2000 A-scans per second, an axial resolution of 18 µm, a transverse resolution of 60 µm, and in-scan dimensions of 16 × 6 mm. 

Image Datasets and Preprocessing

Among the 3866 eligible AS-OCT images from 1322 participants, a total of 2970 images from 1030 were used to predict age and sex, and 2616 images from 857 were used to predict height, weight, and BMI. A total of 896 images of 292 participants were excluded: 808 images of 259 due to poor quality or artifacts, 44 images of 22 due to history of prior ocular surgery, and 44 images of 11 due to corneal disease. As for analysis of height, weight, or BMI, a total of 354 images of 173 participants were further excluded due to their missing data (Fig. 1). To estimate the number of images required to train a DL model, we referred to earlier studies^35–37 that utilized medical images, including AS-OCT images, for DL. 

For image preprocessing, each of the horizontal AS-OCT images was split in half along the vertical midline and rescaled to 384 × 384 pixels using bicubic interpolation. The entire datasets were randomly shuffled and apportioned into training, validation, and test datasets at a ratio of 8:1:1 to ensure that no images from the same participant ended up in two or more of those datasets. The training dataset was used to train the DL model, the validation dataset to determine when to end the training, and the test dataset to evaluate the performance of the DL model. For data augmentation to improve the performance of the DL model, the right side of the split images in the training dataset were flipped horizontally to align with the left-side images. We applied only flipping, because other geometric transformation techniques such as rotation did not improve the consistency of the configuration of the images. 

DL Model Development

We used a ViT model, ViT-B/16, to predict the demographic variables. The ViT, as recently applied for image classification, has achieved state-of-the-art, best accuracy among models classified with ImageNet.³⁸ The model was pretrained with ImageNet-21k and fine-tuned with ImageNet-1k (available at https://github.com/lukemelas/PyTorch-Pretrained-ViT). 

A schematic of the ViT model is provided in Figure 2. Images of 384 × 384 pixels were fed into the network as multiple 16 × 16 pixel patches (576 patches in total) flattened to one-dimensional vectors to serve as inputs to the encoder. A total of 768 (16 × 16 × 3) one-dimensional vectors were converted to 768-dimension embedding vectors through linear projection (i.e., patch embeddings). The other vectors were classification tokens, which act as one-dimensional representation vectors for images. To retain positional information, position embeddings were added to the patch embeddings. The encoder consists of alternating layers of multi-head self-attention and multi-layer perceptron blocks to which LayerNorm and residual connections are applied before and after each block, respectively.³⁸ In multi-head self-attention, 768 vectors were multiplied by weight matrices of W_q, W_k, and W_v to obtain query (Q), key (K), and value (V), respectively. A dot product value of Q and K was normalized using a softmax function, and a final attention value was obtained by multiplying the softmax of Q^T · K by V. The output of the encoder was then passed through the multi-layer perceptron head, and the scores were obtained for each class. 

Figure 2.

 

Schematic of ViT model.

For each of the demographic variables, we used the following cut-offs: ≤75 and >75, ≤60, 60–75, and >75 years for age; ≤170 and >170 cm for male height; ≤155 and >155 cm for female height; <70 and ≥70 kg for male weight; <55 and ≥55 kg for female weight; and <23, 23–25, and ≥25 kg/m² for BMI. As for the cut-off values for age, we referred to the definitions of “older age” used by the United Nations³⁹ and by the Japan Gerontological Society and the Japan Geriatrics Society.⁴⁰ Regarding height and weight, we referred to the average data of the adult Korean population as provided by the National Health Insurance Service of Korea.⁴¹ For BMI, we referred to the Asia-Pacific guidelines issued by the World Health Organization⁴² and the guidelines of the Korean Society for the Study of Obesity.⁴³ 

Visual Explanation of DL Model Decisions

To highlight the regions in the images that had contributed to the prediction of the trained DL model, heatmaps were generated using gradient-weighted class activation mapping (Grad-CAM),⁴⁴ which was extracted from the first LayerNorm of the last block of the Transformer encoder. 

External Validation

To validate the performance of the DL model, we used external datasets from the SNUH Healthcare System Gangnam Center, an independent and remote external center for regular check-ups. A total of 174 AS-OCT images from 69 participants were used for prediction of age. 

Statistical Analyses and Performance Evaluation

To evaluate the performance of the ViT model, the area under the receiver operating characteristic (ROC) curve (AUC), sensitivity, and specificity were calculated with 95% confidence intervals (CIs) for each of the demographic variables. The sensitivity and specificity were computed by thresholding the output value of the designed network after normalization (0–1) by the softmax function. The data were analyzed using R 3.3.3 (R Foundation for Statistical Computing, Vienna, Austria) and Python 3.7 (Python Software Foundation, Wilmington, DE). In the analyses, P < 0.05 was considered statistically significant. 

Results

The demographic data on the participants are shown in Table 1. A total of 2970 AS-OCT images from 1030 participants were used to train, validate, and test the ViT model for prediction of age and sex, and 2616 images from 857 participants likewise for prediction of height, weight, and BMI. The mean age was 70.5 ± 14.5 years, and a total of 592 females (57.5%) were included among the participants. The mean height was 159.1 ± 9.9 cm: 167.1 ± 5.9 cm for males and 153.8 ± 8.3 cm for females. The mean weight was 61.6 ± 11.2 kg: 68.0 ± 11.1 kg for males and 57.7 ± 9.1 kg for females. Histograms that present the number of participants in each group of demographic characteristics are provided in Supplementary Figure S1. 

Table 1.

 

Demographics of Participants

 

Table 2 presents the performance of the ViT model for prediction of demographic characteristics. For differentiating age ≤75 versus >75 years, the model achieved an AUC of 0.910 (95% CI, 0.875–0.944), a sensitivity of 0.914 (95% CI, 0.767–0.967), and a specificity of 0.773 (95% CI, 0.671–0.898). For classifying age into the three groups of ≤60 versus 60–75 versus >75 years, the model obtained an AUC of 0.844 (95% CI, 0.794–0.890), a sensitivity of 0.747 (95% CI, 0.674–0.877), and a specificity of 0.866 (95% CI, 0.701–0.945). For differentiating sex, an AUC of 0.665 (95% CI, 0.600–0.730), a sensitivity of 0.736 (95% CI, 0.449–0.949), and a specificity of 0.563 (95% CI, 0.295–0.821) were obtained. 

Table 2.

 

Performance of DL Algorithm for Prediction of Demographic Characteristics

 

Further analysis for height revealed that, for classifying height ≤170 versus >170 cm in males, the AUC was 0.521 (95% CI, 0.403–0.639) with a sensitivity of 0.765 (95% CI, 0.118–1.000) and a specificity of 0.443 (95% CI, 0.129–1.000). For classifying height ≤155 versus >155 cm in females, the AUC was 0.521 (95% CI, 0.436–0.605) with a sensitivity of 0.702 (95% CI, 0.362–0.968) and a specificity of 0.789 (95% CI, 0.456–0.789). Regarding weight, specifically for discriminating weight <70 versus ≥70 kg in males, the ViT model showed an AUC of 0.522 (95% CI, 0.407–0.638), a sensitivity of 0.479 (95% CI, 0.188–0.979), and a specificity of 0.741 (95% CI, 0.111–0.926); For discriminating weight <55 versus ≥55 kg in females, the model showed an AUC of 0.503 (95% CI, 0.408–0.599), a sensitivity of 0.773 (95% CI, 0.125–0.932), and a specificity of 0.371 (95% CI, 0.145–0.936). Also, for classifying BMI into the three groups of <23 versus 23–25 versus ≥25 kg/m², an AUC of 0.517 (95% CI, 0.489–0.595), a sensitivity of 0.582 (95% CI, 0.144–0.892), and a specificity of 0.533 (95% CI, 0.188–0.931) were obtained. The ROC curves of the ViT model for prediction of age and sex are plotted in Figure 3 and for height, weight, and BMI in Figure 4. 

Figure 3.

 

ROC curves of DL model for prediction of age and sex. (A) ROC curves for prediction of age ≤75 versus >75 years (red), ≤60 versus 60–75 versus >75 years (blue), and sex (green). (B) ROC curves for prediction of age ≤60 versus 60–75 versus >75 years (red) and for each class (dotted black).

Figure 4.

 

ROC curves of DL model for prediction of height, weight, and BMI. (A) ROC curves for prediction of height ≤170 versus >170 cm in males (solid green), height ≤155 versus >155 cm in females (dotted green), weight <70 versus ≥70 kg in males (solid blue), weight <55 versus ≥55 kg in females (dotted blue), and BMI <23 versus 23–25 versus ≥25 kg/m² (red). (B) ROC curves for prediction of BMI <23 versus 23–25 versus ≥25 kg/m² (red) and for each class (dotted black).

In the final stage, heatmaps, examples of which are presented in Figure 5, were generated with Grad-CAM to highlight the regions of the images that had contributed to the decisions of the trained ViT model. With respect to prediction of age ≤75 versus >75 years, the heatmaps indicated that the area of iridocorneal angle including the limbus and iris had been the main contributor to the determinations of the model. 

Figure 5.

 

Examples of heatmaps for prediction of age ≤75 versus >75 years by trained DL model. The heatmaps of participants of differing ages, including an 86-year-old male (A), a 75-year-old female (B), and a 73-year-old female (C), revealed that the area of iridocorneal angle including the limbus and iris was mainly highlighted, being the main contributor to the determinations of the model redundant.

For external validation, 174 AS-OCT images from 69 participants were used to predict age. The mean age was 61.0 ± 14.4 years, and 34 females (49.3%) were included. For differentiating age ≤75 versus >75 years, the model achieved an AUC of 0.904 (95% CI, 0.852–0.956), a sensitivity of 0.688 (95% CI, 0.500–0.828), and a specificity of 0.482 (95% CI, 0.109–0.791). For classifying of age into the three groups of ≤60 versus 60–75 versus >75 years, the model obtained an AUC of 0.772 (95% CI, 0.696–0.838), a sensitivity of 0.434 (95% CI, 0.139–0.562), and a specificity of 0.332 (95% CI, 0.177–0.484) (Table 3). The ROC curves are provided in Figure 6. 

Table 3.

 

Performance of DL Algorithm for Prediction of Age With External Datasets

 

Figure 6.

 

ROC curves of DL model for prediction of age with external datasets. (A) ROC curve for prediction of age ≤75 versus >75 years (red). (B) ROC curves for prediction of age ≤60 versus 60–75 versus >75 years (red) and for each class (dotted black).

Discussion

In the present study, we developed a ViT model to predict demographic characteristics from AS-OCT images. Our results reveal that the ViT model can predict age category accurately and that the area of the iridocorneal angle is the key structure related to aging. To our best knowledge, this is the first large-scale study that has utilized AS-OCT images, in their entirety, for prediction of diverse demographic variables and that has applied DL methods to the prediction. 

Our results are in line with previous studies that have revealed the relationship between age and anterior segment structures, including the iridocorneal angle, limbus, and iris, in normal eyes.^{15,16,21,22,24,45} As for iridocorneal angle–associated structures, angular measurements such as iridocorneal angle, scleral spur angle, angle-opening distance, and trabecular–iris space area have been reported to decrease with age.^21,22 Several studies also have revealed age-related changes in the trabecular meshwork or its interface shadow,^15,45 as well as ciliary muscle thickness and location.²² With regard to the limbus, the study by Yang et al.²⁴ reported that the nasal and temporal limbal epithelia became thinner with aging, and that of Xie et al.²² showed that central corneal thickness had no significant relationship with age, findings that further support the results of our study. 

Changes in iris morphology, in which iris width increases with aging while volume and thickness remain unaffected, also have been reported.¹⁶ Although other factors such as corneal diameter, anterior chamber depth or width, and lens thickness or vault also have been reported to be associated with age,^22,27 our results indicate that they had not been the main anatomies contributing to age prediction. As for the study by Ma et al.,⁴⁶ which to our knowledge is the only one to have developed predictive models of aging with AS-OCT using machine learning, their results are problematic in terms of their representativeness for normal eyes, because they did not exclude diseases (other than corneal pathologies) possibly affecting anterior segment structures. Few studies have investigated the association of anterior segment structures with sex, height, weight, or BMI, and their results remain inconclusive.^{15–17,20,23,27,28,45} 

We found that our DL model was able to predict age accurately from AS-OCT images, whereas it performed poorly for sex, height, weight, and BMI. Although several earlier studies using OCT images reported that a DL model accurately predicted demographic characteristics of age or sex,^29–32 there are several limitations that should be noted (Supplementary Table). First, whereas Shigueoka et al.²⁹ reported that a DL algorithm was able to accurately predict age from whole peripapillary OCT, with high correlation between predicted and true chronological ages, their patient-to-patient results were highly variable. Second, notwithstanding the report by Hassan et al.³⁰ that accurate prediction for age and acceptable performance for sex had been obtained using DL, their results also were highly variable by patient. Third, Munk et al.³¹ concluded that age and sex could be classified from OCT using DL-based methods for a broad spectrum of patients irrespective of underlying disease or image quality. For age prediction, however, they used a combination of methods whereby the network outputs age bins that are normalized using a softmax activation w and multiplied by the lower edge of the bins, d_x, which could cause overfitting and result in a DL model vulnerable to domain shift. Finally, although Chueh et al.³² reported that age and sex could be identified from macular OCT with good accuracy using DL, they did not separate a test dataset when applying 10-fold cross-validation, which could have exaggerated the performance of the DL model. Moreover, all of these studies utilized OCT images in their analyses, which limits the direct comparability of their results with our AS-OCT–based ones. 

In light of the indecisive results from earlier studies, our findings certainly will provide important, well-founded clues to determining the relationships between certain demographic variables and anterior segment structures. Of note, the finding that age can be predicted from AS-OCT images suggests that anterior segment structures could potentially play a role as biomarkers for aging. Moreover, combining the advantages of DL with those of AS-OCT imaging, as demonstrated in our study, will help to enhance the clinical utilization of AS-OCT, an increasingly potent imaging modality for evaluation of ocular diseases. Also, it is of value that our DL model demonstrated its potential clinical usefulness by distinguishing age-related structural changes on imaging, a task that is challenging for practitioners using the naked eye (Supplementary Fig. S2). 

The strength of our study is its utilization of DL techniques enabling automated and fast analysis of big image data obtained from a large population. The limitation of the current study is that, in our attempt to predict demographic values quantitatively with a regression model, the network converged to the class mean value. This might be attributable to the fact that multiple classes are required for regression,⁴⁷ because the performance of the DL model becomes poorer as the number of classes increases due to fewer samples.³¹ Also, although the heatmaps indicated the area of iridocorneal angle as the main contributor to the age differentiation of the model, there have been difficulties in understanding why and how it produced its outputs. This is due to a “black box problem” associated with the DL model which refers to the difficulty of inspecting its internal state. As for external validation, the DL model showed poorer performance overall with the external datasets than with the internal ones. This might have been due to changes in the distribution of the data (i.e., the domain shift), which could have negatively affected the performance of the model.⁴⁸ Additional studies are required to further evaluate its external validity. Also, future studies comparing ethnic and racial differences would further expand our knowledge of normal anterior segment structures. Moreover, additional, longitudinal studies monitoring structural changes and exploring molecular changes would better elucidate the effects of aging on anterior segment structures. 

In conclusion, the ViT model could predict age category accurately from AS-OCT images, and the area of the iridocorneal angle was the key structure related to aging. Our findings will help clinicians to better understand age-related anterior segment structural changes in normal eyes, which should ultimately aid in the evaluation and management of ocular diseases in clinical practice. Additionally, our study suggests that anterior segment structures could potentially serve as biomarkers for aging. 

Acknowledgments

Disclosure: Y.J. Lee, None; S. Choe, None; S. Wy, None; M. Jang, None; J.W. Jeoung, None; H.J. Choi, None; K.H. Park, None; S. Sun, None; Y.K. Kim, None