AI-Based Decision-Support System for Diagnosing Acanthamoeba Keratitis Using In Vivo Confocal Microscopy Images

Abstract

Purpose: In vivo confocal microscopy (IVCM) of the cornea is a valuable tool for clinical assessment of the cornea but does not provide stand-alone diagnostic support. The aim of this work was to develop an artificial intelligence (AI)-based decision-support system (DSS) for automated diagnosis of Acanthamoeba keratitis (AK) using IVCM images.

Methods: The automated workflow for the AI-based DSS was defined and implemented using deep learning models, image processing techniques, rule-based decisions, and valuable input from domain experts. The models were evaluated with 5-fold-cross validation on a dataset of 85 patients (47,734 IVCM images from healthy, AK, and other disease cases) collected at a single eye clinic in Sweden. The developed DSS was validated on an additional 26 patients (21,236 images).

Results: Overall, the DSS uses as input raw unprocessed IVCM image data, successfully separates artefacts from true images (93% accuracy), then classifies the remaining images by their corneal layer (90% accuracy). The DSS subsequently predicts if the cornea is healthy or diseased (95% model accuracy). In disease cases, the DSS detects images with AK signs with 84% accuracy, and further localizes the regions of diagnostic value with 76.5% accuracy.

Conclusions: The proposed AI-based DSS can automatically and accurately preprocess IVCM images (separating artefacts and sorting images into corneal layers) which decreases screening time. The accuracy of AK detection using raw IVCM images must be further explored and improved.

Translational Relevance: The proposed automated DSS for experienced specialists assists in diagnosing AK using IVCM images.

Introduction

In vivo confocal microscopy (IVCM) of the cornea has been used as a complementary clinical imaging technique in assessing different corneal diseases, such as ocular surface disease and corneal infections.^1–5 IVCM is a noninvasive imaging technique which generates transversal images (also called en face images) with a lateral resolution of 1 to 2 µm and an axial resolution of about 4 µm.⁶ These characteristics make IVCM an excellent technique for observing the cornea's tissue and its cellular structure. 

A confocal scanning laser microscope can generate more than a thousand IVCM images during a diagnostic session of a single patient. However, images can be artefacts (e.g. very dark, very bright, reflective, and blurry) or not useful/uninformative (showing tissue structures, but lacks diagnostic value). Some images might be from tissue and structures that lack diagnostic value, such as images from the conjunctiva, when investigating corneal infections, such as Acanthamoeba keratitis (AK). Diagnosing a specific condition using IVCM images involves several sub-tasks starting from collecting IVCM raw images, removing artefact images, finding useful images, finding disease features, and making the final diagnostic decision (positive, negative, or suspected case). This process of manually separating good images with diagnostic value from images without diagnostic value is time-consuming and prone to human error. A possible solution for this challenge is the use of artificial intelligence (AI)-based decision support systems (DSS) using, for example, a two-step approach: first, the software separates good from unuseful images, and second, it selects images based on the disease features that are clinically significant for the condition being investigated. 

Previous attempts to develop AI-based DSS for diagnosing different cornea conditions using IVCM images have had the common limitation of requiring a manual selection of good images for diagnosis as part of the preprocessing step.^2–6 In addition, previous AI-studies focused on one sub-task, such as finding disease features, and manually perform or omit the other sub-tasks of the DSS.^1,7 However, in prior studies, investigators used images at the group level and not on an individual level, hence, omitting the possibility to provide a diagnostic prediction for a given patient.^1,2 These two examples of DSS with their merits and limitations provide a good basis for further improvements in AI-based DSS.^1,6,8 One aspect that we consider crucial to be addressed by an improved AI-based DSS for corneal IVCM is a fully automated process for diagnosing conditions using IVCM images. Therefore, the aim of this work was to develop such an AI-based DSS for automated diagnosis of AK using IVCM images. 

The proposed AI-based DSS system using IVCM images differs from prior AI-based systems in several ways. First, we used raw input images from each patient taken directly from the IVCM diagnostic session as input, without human intervention to, for example, remove artefact images. Second, we introduce the possibility of having a domain expert in the loop. This involves the expert correcting the predictions; corrections are then used to model retraining. Third, our system includes a “suspected positive” diagnosis class. This means that our concept accounts for the uncertainty of a diagnosis and is also designed to handle the uncertainty. Last, our system implements all sub-tasks mentioned above including the final diagnostic decision per the patient. 

Methods

Clinical Condition and Motivation

Raw IVCM image datasets were extracted from patients with various eye diseases, but for the purpose of this study we focused on detecting one disease, AK. AK is a serious corneal condition accompanied by intense eye pain, sensitivity to light, and severely impaired vision. If not treated within a few weeks after infection, the parasite can cause permanent corneal damage, necessitating subsequent corneal transplantation. If allowed to further progress, all eye structures can eventually become infected, resulting in a total loss of the eye.⁹ Contact lens wear-related infection is the most frequent etiology of AK, accounting for up to 90% of all cases.¹⁰ Given their popularity, particularly in Scandinavia, with around 12% of adults in Sweden wearing contact lenses, AK, once considered rare, is becoming more common.¹¹ 

We defined that Acanthamoeba cysts or trophozoites confirmed in three or more nonoverlapping IVCM images can be used to diagnose AK. AK structures have been categorized into four types¹²: trophozoites, double-walled cysts, target sign, and bright spots (pre-cysts), as shown in Figure 1. 

Figure 1.

 

Four typical phenotypic forms of acanthamoeba cysts as observed in IVCM (reproduced in part from Ref. 14 with permission from the publisher). (A) Suspected trophozoites (arrows) of different sizes. (B) Double-walled cysts (arrows). (C) The pre-cysts presenting as round, bright spots (arrows; triple arrow indicating a chain formation). (D) Target signs (arrows) of cysts in the epithelial wing cell layer. All images are 400 × 400 µm.

The microscopic appearance of AK cysts is well described in the related literature.⁵ AK cysts can differ from other inflammatory cells in terms of shape, distribution, and microscopic features of the cell wall and/or nucleus.⁵ Little research to date has focused on detecting AK in IVCM images using a deep learning approach.^1,6 Most studies have explored fungal keratitis excluding images with AK and have used slit-lamp or IVCM images.^13–18 Using information about the microscopic features of cysts and knowledge of image processing techniques (such as edge detection, texture features, and image segmentation), we hypothesized that it should be possible to develop a DSS for AK cyst detection and AK diagnosis. 

Data Collection

This study utilizes IVCM datasets extracted from a database of images at the Department of Ophthalmology, Linköping University, Sweden. The use of anonymized IVCM datasets for research was approved by the Swedish Ethical Review Authority under ethical permits (No. 2021-05050 and 2022-00365-01). An IVCM system (Heidelberg Retinal Tomograph 3 with Rostock Corneal Module [HRT3-;CM]; Heidelberg Engineering, Heidelberg, Germany) was used to acquire all images. Images were acquired at various scanning depths, depending on the pathology, but generally the corneal epithelium and stromal layers were scanned at least once. For AK cases, the scanning was focused on the epithelium and anterior stromal regions and thus full thickness scans for those cases were generally not available. Prior to examination, each subject received topical anesthetic (Minims Tetracaine Hydrochloride 0.5% w/v, eye drops solution; Bausch & Lomb, UK). Images were taken from the central or paracentral cornea using sequence mode, with images acquired at five to eight frames per second. After each sequence (defined as 100 consecutive images), images were automatically stored within the instrument's software. Images were saved in TIF format, with 8-bit grey levels and a single image frame size of 384 × 384 pixels, representing an imaged corneal area of 400 × 400 µm. On average, there were 716 images per eye. All images were exported via the instrument's software into a single folder marked with a randomized study number. 

Dataset

The image dataset for this study included a total of 68,970 images (including artefact images) obtained from 85 patients (30 men and 55 women) who had 94 separate clinical visits between January 2008 and January 2023 at the Department of Ophthalmology, Linköping University. The dataset contained images acquired from subjects with healthy corneas and from subjects with various corneal conditions, such as inflammation, AK, other noninfectious disease, suspected positive AK, and suspected negative AK. The category “other noninfectious disease” included dry eye disease, corneal ulceration, neurotrophic deficit, the post-coronavirus disease 2019 (COVID-19) syndrome and posterior polymorphous dystrophy, one patient for each condition, with conditions identified from the confocal microscopy and patient records. From these images, five datasets (summarized in Table 1) were prepared and labeled by three domain experts (experienced confocal microscopist – co-author N.L., and 2 trained optometrists – co-authors J.R. and A.F.M.) and used as ground truth for training the models (see Table 2). In addition, one more dataset (see Table 3) was prepared for DSS validation. 

Table 1.

 

Training Datasets

 

Labeling and Preprocessing

The IVCM images were sorted and prepared by domain experts into different datasets, as presented in Table 1. 

Deep Learning Models Design and Training

In the current study, all models are deep residual networks, particularly the ResNet101V2,¹⁹ where transfer learning was used. Transfer learning is a powerful tool for improving the performance of machine learning models, allowing them to leverage knowledge gained from previous tasks and apply it to new problems. Rather than training a model from scratch on a new problem, transfer learning involves taking features learned from one problem and leveraging them to create a model that is specifically tailored to the new task. Transfer learning works by taking the weights of a pretrained model trained on a large dataset and then fine-tuning them on a smaller dataset associated with a new task. This allows the new model to leverage the features already learned by the pretrained model while also allowing it to adjust to the specific characteristics of the new dataset. The main advantage of transfer learning is that the model retains the weights that were used in the training of millions of images, saving model training time and resources. Numerous pretrained deep neural networks for image classification and segmentation tasks exist. Previous research has shown that the ResNet101V2 model has high classification accuracy on different medical images, such as X-rays,^21–23 whole slide images,²⁴ histopathological images,²⁵ and IVCM images.⁶ We experimented with different pretrained networks using our image datasets including InceptionV3, EfficientNets, Xception, VGG-16, VGG-19, ResNet50V2, and ResNet152V2. The ResNet101V2 outperformed the others and was selected for all five models in the proposed DSS (Table 2). 

Table 2.

 

ML Models

 

ResNet101V2 is a convolutional neural network with 101 layers deep and has been pretrained on the ImageNet database. We added some new layers on top of the pretrained ones and trained them specifically on our IVCM images to adjust ResNET101V2 to our classification tasks. The new layers added for each model are presented in Table 2. 

Each dataset described in Table 1 was split into training (80%) and test set (20%). An image data generator was used for training the models with the following augmentation techniques: horizontal flip, and 0.5 for height and width shift range. All models were trained for 100 epochs with an initial learning rate of 0.001, the early stopping training when the validation accuracy stopped improving. Training of the models was performed using 5-fold cross-validation, leading to the 80%/20% training/test data split for each of the 5 cross-validated models, and standard model performance assessments (accuracy, F1-score, precision, recall, false negative [FN], and false positive [FP]²⁶) were measured, averaged over the 5 models, and reported. External validation was performed on a new dataset of 26 patients (not previously seen by the model) to assess the DSS performance. 

All models were developed in Python version 3.9 programming language using the TensorFlow version 2.9.0 and Keras version 2.9.0 machine learning library and PyCharm version 2023.1 software by JetBrains. The models were trained on Mac M1 Pro 16-Core GPU (development environment) and on a 40-CPU Ubuntu 22.04 Server (deployment environment). 

An Approach for Diagnosing IVCM Images

Diagnosing using IVCM images includes several sub-tasks which were organized into nine steps (schematically shown in Fig. 2): 

• Step 0. Export the raw IVCM image dataset into a single patient folder.
• Step 1. Sort images into “good” and “artifacts” folders using a binary classification model (trained on artefact and non-artefact images [model 1]).
• Step 2. Sort images into four corneal layers using a multi-classification model (model 2) trained on a labeled dataset of corneal layers: epithelium, nerve plexus, stroma, and oblique slice.
• Step 3 (Optional). In this step, a domain expert can check the images in the artefacts folder (in case some useful image was falsely sorted into that folder) and other layers and correct the sorting by moving the image of each misplaced layer to the correct folder. The image is then automatically copied to the folder containing training images for model 2. This correction will be used in the next retraining session to improve or maintain the sorting at a satisfactory level.
• Step 4. Another binary classification model (model 3) trained on epithelial images with and without disease, uses only images from the epithelial layer to predict if the cornea is healthy or diseased. This step is only relevant for some diseases. The model also provides the confidence level (as a percentage) to indicate how confident a decision is. If the cornea is healthy, the diagnostic process is stopped, and the final decision suggested by the system is “healthy cornea.”
• Below this point, steps are related to AK detection, other corneal diseases will require some modifications (such as another image preprocessing technique for finding disease features, a trained model [with same or similar architecture], and perhaps using images from other cornea layers).
• Step 5. If model 3 suggests the result “diseased epithelium,” then a multi-classification model (model 4) is initiated to scan images from two layers, epithelium and stroma, to detect signs of AK. This classifier was trained on the labeled dataset with three classes of confocal microscopy images: containing cysts (positive), likely to contain cysts (suspected), and without cysts (not informative). Model 4 sorts images into these three categories, and the system provides to the user the number of images classified into each category and the images themselves.
• Step 6 (Optional). In this step, a domain expert can check the classification into categories provided by model 4 (step 5) and correct the misclassified images. The performed corrections are automatically copied to the respective folder with training images to improve the model accuracy.
• Step 7. Images with positive and suspected AK cases are preprocessed by the edge detection algorithm to emphasize the cyst's shape, morphology, and distribution within the image. After this preprocessing step, the image is scanned with a sliding square window of size 75 × 75 pixels, and a binary classification model (model 5) classifies these square regions of the image as containing cyst shapes or not. This step can be improved in future implementations by using image segmentation techniques to accentuate the regions of interest in the image.
• Step 8 (Optional). The domain expert can correct the AK localization by moving/removing the 75 × 75 pixel square to regions of interest in the image that contain cysts. The corrected 75 × 75 pixel squares are then copied to the folder of training images to improve the model accuracy. In future implementations, this step can be improved by adding an image annotation technique to delineate the regions with cysts in an image more precisely.
• Step 9. It is generally not recommended to make a final decision (AK positive, AK negative, and AK suspected) based on one IVCM image, as a single image represents only 0.15% of the total corneal area⁴ and could contain artefacts. The current state-of-the-art method for decision requires that an experienced grader performs an image selection to collect the necessary evidence for making the decisions. These can be called “AK-positive corneal images” and they are called as such because they contain cysts. The minimum number of AK-positive corneal images (without overlapping) required for a positive diagnosis is three. This has been implemented in our system; we incorporated a ranking algorithm to find the best “AK positive” classified images. This automation was based on the prediction confidence (probability >0.90) for ranking the best images and the percentage of overlapping pixels in the image (should be <10%). If the system finds three or more diagnostic images, the final decision is “AK positive case” and useful images are presented to the user. In cases of a negative AK decision, currently, the diagnostic process is stopped, and the final decision is “AK negative.” In future implementations, the system can continue to detect other corneal diseases.

Figure 2.

 

Automated workflow for diagnosing AK in IVCM images.

Results

The results have been divided into three subsections: results I - the AI-based DSS implementation, results II - evaluation of the AI-based DSS for AK diagnosis - training session, and results III - evaluation of the AI-based DSS for AK diagnosis - validation session. 

Results I: The AI-Based DSS Implementation

We developed a web application using the Angular 15 web framework (developed by Google). This application allows end-users to work with the AI-based DSS. The machine learning models were deployed on a Flask version 2.2.3 Python-based web framework,²⁷ as shown in Figure 3. 

Figure 3.

 

AI-based decision support architecture overview. The front-end contains several angular components which represent a simple user interface (UI), and a HTTPS service is used for communication between the front-end and the back-end. The backend implements the automated workflow presented in Figure 2 (app.py) with a machine learning component (machine learning scripts), and files (trained ML models, images, and other files such as text files for notes, and JSON files for saving suggested decisions for each patient).

As the system uses a folder with IVCM images and a unique numeric value as the patient identifier, this architecture removes the need for a database. The web application interface shown in Figure 4A accepts a single folder containing all raw images extracted from an IVCM examination without the need to preprocess images in any way. The model then automatically classifies each image in the folder into a given corneal layer (category) according to the layers shown in the tabs of Figure 4B. At this stage, the AI-based system already performed step 1 and step 2 using the respective models, as described in Figure 2. 

Figure 4.

 

Decision support system's user interface. (A) New patients can be added by dropping the folder of raw IVCM images in the rectangle at the top, and (B) automatic corneal layer classification results showing the five categories (shown in tabs) defined for the case study (AK detection and diagnosis), images shown are all classified as “Epithelium.”

As shown in Figure 2, during step 3 experts can review images in each category, with the possibility to manually correct the classification of any image deemed to be wrongly classified. Wrong classification occurs in some cases, particularly when the training of the model has been limited. Correction is performed by moving images manually to the correct category. Each corrected classification (image is moved to the correct layer) is then automatically copied to the corresponding training folder and used for re-training the models according to this domain expert input. 

This constitutes the concept of the DSS that can manage IVCM images in general. The next steps are specific to the diagnosis of AK based on IVCM images. The user can press the button “AK Detection” in the AK Detection panel shown in Figure 5. 

Figure 5.

 

AK detection panel, the DSS generates a classification: healthy or disease condition, giving a confidence level from 0 (no confidence) to 100 (certain).

This executes steps 4 and 5 given in Figure 2 using their respective models. The system provides a suggestion for the diagnosis, that is, results from model 3 classify the corneal epithelium as healthy/diseased with a level of confidence graded from 0 to 100% (see Fig. 5, Prediction 1) and the model 4 classifies all images, as shown in Figure 6. A summary of the amounts of “positive,” “suspicious,” and “not informative” is displayed below the AK diagnosis prediction (see Fig. 5). 

Figure 6.

 

Classification results visualization: AK positive images (red border), AK suspected images (orange border), and noninformative images (without border).

The system generates the final decision suggestion (bottom of Fig. 5, Prediction 2) by finding a minimum of three nonoverlapping images with AK cysts. The images can be seen in the dialog window by pressing “View AK Images.” The user can click the “Cysts Localization” button to highlight the regions of interest (e.g. AK cysts) within the positive and suspected images, as shown in Figure 7. 

Figure 7.

 

Example of AK cyst localization. By clicking the button “View AK Images” in the web interface the user can see the images with suspected cysts that the system based the decision suggestion on (left, “Original Image”). Clicking the “Cysts Localization” the system highlights the regions of interest (rightmost image, “Localized Cysts image”) within the positive and suspected images.

Results II: Evaluation of the AI-Based DSS for AK Diagnosis - Testing Session

The developed AI-based DSS system with a fully automated diagnostic process (see Fig. 2, steps 1-5) was tested by three domain experts. The testing procedure was as follows: 

• • Image folders from five additional patients (not included in the initial training of the models) were uploaded to the system (Fig. 2, steps 1 and 2). The ground truth for these patients was known: two patients with suspected AK, and three patients positively diagnosed with AK. The baseline accuracy for the models trained on the dataset before the testing session is recorded in Table 3 in the “Accuracy (baseline)” column.
• • Domain experts first screened the artefact images and checked if informative images were incorrectly categorized there, and, if yes, these images were moved to the correct cornea layer category (dataset 1, step 3). Images categorized as “oblique slice,” “epithelium,” “stroma,” and “nerves” were also inspected and moved to the correct layers if needed. Each moved image was also added to the training dataset (dataset 2, step 3).
• • Then, the “Diagnose AK” button was clicked to first predict healthy or diseased cornea (see Fig. 2, step 4). In case of a prediction of a diseased cornea, the system starts to detect AK positive images, AK suspected images, and noninformative images (see Fig. 2, step 5). The predicted images (positive AK and suspected AK) were examined by domain experts, corrections were made where needed, and the images were copied to the respective training dataset (dataset 4, see Fig. 2, step 6).

Table 3.

 

The Deep Learning Models Performance

 

After the testing session was completed, the models were retrained with 5-fold cross-validation and the models’ performances were recorded as given in Table 3. Precision (or “sensitivity”) measures a model's ability to correctly identify images of the positive class, whereas recall (or “specificity”) measures a model's ability to correctly identify images of the negative class.²⁸ F1-score combines the precision and recall scores of a mode (geometric mean of the two) to determine the total correct prediction rate (positive and negative classes). For models 2 and 4, the FP and FN values were converted to a one-versus-all matrix for each class, and the percentage was taken instead of the absolute value to better compare the models’ errors. The “Accuracy (after retraining)” column shows the retrained models’ accuracy, that is, retrained after the testing session with the domain experts. 

The first three models perform the tasks described in Figure 2 and are useful in any IVCM image-based DSS. These models achieved a high accuracy (≥90%) using raw images without feature engineering or additional image preprocessing techniques. As summarized in Table 4, the precision and recall scores are well balanced in the models 1 and 3, which means that these models are classifying similar amounts of FPs and FNs. In model 2, the images belonging to the “nerves” class were the most accurately classified ones with an average F1-score of 97%, and with well-balanced precision (FP = 2.8%) and recall (FN = 2.4%). That means the model found accurate classification boundaries for this class. Images belonging to the “stroma” class had the lowest classification accuracy with an average F1-score of 86%, FP = 17.0%, and FN = 11.9%, meaning the model was less successful in finding accurate classification boundaries for this class. Most of the FP and FN errors here were made by classifying the “stroma” image as “epithelium” and vice-versa, which means the classifier was unable to identify boundaries between these two classes clearly. 

Table 4.

 

Validation Dataset

 

The other two models related to AK detection (models 4 and 5) had a lower accuracy (84.2% and 76.5%, respectively) using raw images. Model 4 had unbalanced precision and recall in all classes and had a large percentage of FN errors, meaning the model tended to misclassify the positive AK images as the “not-informative” class. Classification boundaries for the AK “suspicious” and AK “positive” classes need to be improved because these classes yielded the lowest accuracies, 48% and 67%, respectively. There is also a need to improve the recall of the cyst “positive” class in model 5 (44%). In general, the models 4 and 5 were less accurate than the models 1 to 3 due to: (a) not having a perfect balance of images in the training dataset, (b) the lack of sufficient training data for model 5, and (c) not performing model tuning. It is also possible that (d) using raw images is not enough to capture the AK cysts, meaning that additional features should be extracted or additional image preprocessing techniques, such as cell segmentation, should be further explored. 

Overall, the baseline accuracy was increased for models 1 and 4 after retraining them. The accuracy of model 2 decreased after retraining because there were mislabeled training data in the baseline model (labeled by a data science expert) and some varieties of images belonging to the same layer were not included in the baseline training dataset. In addition, some images did not have a clear/dominant corneal layer. Thus, the training data for the baseline model had mislabeled data. Then, after the domain expert input in the test session, it become clear which images should belong to which layer type. There is no difference in the accuracy (baseline and after retraining) for the models 3 and 5, because the DSS does not yet support the correction of the prediction results of model 3, nor does it allow for image annotation implemented for correcting the results of model 5. The same training data for the models 3 and 5 are used before (baseline) and after retraining the models. 

Results III: Evaluation of the AI-Based DSS for AK Diagnosis - Validation Session

The validation dataset consisted of 26 new patients, never used in the training or testing of the DSS. A summary of the characteristics of the dataset is given in Table 4. This dataset was used for the final validation of the models (more specifically the models retrained after one testing session) and the DSS. 

For the validation of the DSS to diagnose AK, we requested the system to perform prediction 1 (cornea is healthy or diseased, Fig. 2, step 4) and prediction 2 (final diagnostic decision: AK positive or negative case, Fig. 2, step 9) as described in detail at the end of section Results I. The decisions suggested by the DSS were measured as a proportion of correct and incorrect predictions presented in Table 5. 

Table 5.

 

Validation Results

 

Only one out of six patients with confirmed AK (type 1) received a correct prediction of positive AK diagnosis, which corresponds to a 16.6% accuracy. The three nonoverlapping images criteria was not met for the remaining five patients with AK because few images with cysts were identified by the DSS (model 4). This can be explained by the low model accuracy (model 4) in identifying images with AK cysts. One patient was classified correctly and another incorrectly with type 4 (suspected AK), thus the accuracy was 50%. Two patients with type 2 (healthy cornea) were misclassified as patients with corneal disease but not with AK. This can be explained by having perfect epithelium images in the training dataset, whereas the validation dataset contained oblique images depicting several different corneal layers in a single image and were classified as an epithelium even though the epithelium typically comprised only 20% of the image. This can be corrected in future implementations by adding imperfect epithelium images to the training dataset. The DSS made correct decisions for all patients belonging to type 3 and type 5 categories. 

Discussion

In this study, we proposed a general approach for fully automated AK diagnosis based on IVCM images, that was implemented using deep learning, image processing techniques, and rule-based decisions, referred to as an AI-based DSS. The AI-based DSS was tested with clinical data and evaluated for AK disease detection. The results show that the DSS was able to substantially reduce the time needed to sort IVCM images and find useful ones for clinical decision making, which reduces time and human effort to analyze IVCM images. The diagnostic capabilities for AK, however, need further improvement. 

The proposed DSS utilizes domain expert input in the intermediate steps. The concept allows experienced IVCM image assessors to provide input to correct the model classifications where necessary. We believe this input will improve the model performance, stability, and reliability over time and will lead to a fully autonomous and precise DSS. The proposed concept is generalizable as it can easily be extended to other types of corneal infections or eye conditions. This, however, will require training and validation datasets and domain expert input for each new condition. The proposed DSS saves considerable screening and sorting time by preprocessing the IVCM images without human input, which can be beneficial for an eventual clinical translation. 

For the current study, we used data collected at a single eye clinic using the HRT-RCM system (Heidelberg Engineering, Germany) and that might, currently, limit its widespread use. For example, there are other devices used for in vivo microscopy of the cornea, such as the Nidek Confoscan,²⁹ and our DSS was not tested with images from that device. The HRT-RCM system is the most commonly used and the images from other clinics would have similar contrast and resolution. Image quality, however, could still be influenced by the operator's experience. Nevertheless, this study sought to establish an initial proof-of-concept by utilizing a local dataset, with the intention of expanding its use in the future. Notably, the expert evaluation and retraining processes were conducted remotely, involving input from experts from different clinics. Our DSS platform enables the remote uploading of data from different clinics, along with remote expert evaluation for fine-tuning the models based on specific datasets. 

Overall, performance of the first three models was better because an entire image represented a specific class (e.g. “artefact,” “good,” “healthy,” and “diseased”) meaning that all pixels of an image carry informative features. With a reasonable number of images, this allows distinguishing each class successfully with contemporary deep convolutional models. In the case of AK, areas with cysts tend to be very small in proportion to the image as a whole. Thus, considering the entire image for detecting AK is less accurate, because there are many noninformative features (pixels), which can impact the classification outcome. An alternative method would be to scan the images for a small region and to detect cysts in these regions; if one or more regions with cysts are detected, the image would be classified as positive. Scanning every image using small regions (including noninformative images without AK cysts), however, would be substantially more time-consuming for the software. Therefore, we decided to first perform coarse filtering and then to scan smaller regions within the image for localizing cysts. With our approach only relevant images were scanned and the process was rapid. Generally, informative AK feature need to be integrated into algorithms to classify AK images more accurately. Several options for better detection of AK feature areas have been identified, such as normalizing the brightness in the images, and testing different image preprocessing techniques and cell segmentation techniques to filter out noninformative regions from the images and highlight the AK cysts. Following these steps, model 4 can be retrained on the normalized and preprocessed images. 

We found that the edge detection (with the Sobel operator) was a useful preprocessing technique for identifying cysts or suspicious features within an image, and this technique is also used for detecting the outer corneal contour.³⁰ In a future version, both edge detection and cell segmentation techniques could be combined to better locate features of cysts in an image. Because the current concept was optimized for diagnosis of AK, the “endothelium” layer is currently missing in the layer sorting model (model 2). This layer might be important for other corneal diagnoses⁸ and, for this reason, we plan to include this class in the next version of the DSS. 

The proposed DSS is not purely data-driven but is a combination of AI classification and expert-defined rules. Prediction 1 (healthy/disease) was made by calculating how many epithelial images were classified as “healthy” or “diseased.” In this case, the decision suggestion is dependent on whether the number of “healthy” images exceed the number of “disease” images. This rule could give misleading conclusions, for instance, in cases where only a few images contain the pathologic tissue or cellular changes whereas the surrounding cornea is normal. Previous studies, however, have not considered accounting for the health status of the epithelium in the diagnostic decision process. 

The final AK decision suggestion for a patient is also rule-based and relies on the AI classifications made by model 4. The DSS ranks the images by prediction probability, then applies a threshold to select the best images (where the model is 100% sure that this image contains AK feature), and the final step is to select the best images with minimal overlap. To suggest a positive AK decision, the number of best and nonoverlapping images must be at least three, otherwise, a negative decision is suggested. Even this rule could require adjustment, as our datasets with positive cases showed that the threshold of three positive and nonoverlapping images is sometimes difficult to meet. In such cases, the cornea is highly inflamed with edema and patients can experience significant pain and/or photophobia, resulting in poor compliance with the IVCM examination, thereby directly impacting quality and quantity of the images obtained. We opted for a stringent threshold, which resulted in a low degree of accuracy and thereby a greater risk for leaving a positive case untreated. Alternatively, a less stringent threshold used in previous systems could improve the positive detection rate,¹ but at the expense of increasing the FP rate and applying antiparasitic treatment in patients without cysts. Ultimately the system must be adjusted for an acceptable balance. 

In addition, we did not compile the IVCM datasets for this study with the DSS in mind. Instead, the DSS was applied to retrospective image datasets. Knowing the conditions for optimal screening by the DSS could enable future IVCM examinations to focus on obtaining as many suspected cyst-laden images as possible, with minimal overlap, thereby improving the training potential and usefulness of the DSS. 

Finally, whereas the results of this first prototype (which implements the complete cycle of diagnosing AK) are promising, the numbers of patients (8 with and 8 without AK) are too small to draw general and reliable conclusions. After implementing improvements in the AK model, we plan to validate the DSS with more AK cases, also including cases from different eye clinics. 

Conclusions

In this study, we present an AI-based DSS for diagnosing AK in the clinic based on IVCM images obtained from clinical examinations. A user-friendly interface was developed that is designed to learn from domain expert input to re-classify images and continually improve the models and their predictive ability. At this stage, the DSS can be used to filter out noninformative images and predict whether the corneal epithelium is healthy or associated with a limited set of diseases with a high accuracy. Furthermore, the DSS can predict with high accuracy if a cornea is negative for AK, giving clinicians a useful tool in cases where AK is suspected but IVCM evidence is nonconfirmatory. In such cases, the clinician could choose not to instate anti-amebic treatment or to use other complementary diagnostic techniques. In the future, using larger datasets, domain expert input, and annotations will improve the accuracy of the DSS. Finally, the possibilities of using this DSS platform for assessing other corneal diseases presents a large untapped potential. 

Further work is required to (a) explore different image preprocessing techniques to filter out uninformative regions (pixels) from the image and to highlight the informative regions (AK cyst pixels); (b) improve step 7 by using cell segmentation techniques (e.g. using the U-Net deep learning model) instead of a sliding window; (c) improve the detection accuracy for suspected AK positive cases; (d) implement software improvements, such as adding image annotation for correcting the localization of the cyst and enabling domain expert input for model 5; (e) including more testing sessions with domain experts; and (f) a validation of the complete diagnostic cycle using data from a larger number of patients and different eye clinics. 

Acknowledgments

This publication is the result of a seed-project with funding from The Linnaeus University Center for Data Intensive Sciences and Applications, DISA, lnu.se/disa, which is a research excellence center, and constitutes a strategic research profiling effort at Linnaeus University (LNU), Sweden. 

Disclosure: A. Lincke, None; J. Roth, None; A.F. Macedo, None; P. Bergman, None; W. Löwe, None; N.S. Lagali, None