Development of a Machine-Learning–Based Tool for Overnight Orthokeratology Lens Fitting

Abstract

Purpose: Orthokeratology (ortho-K) is widely used to control myopia. Overnight ortho-K lens fitting with the selection of appropriate parameters is an important technique for achieving successful reductions in myopic refractive error. In this study, we developed a machine-learning model that could select ortho-K lens parameters at an expert level.

Methods: Machine-learning models were established to predict the optimal ortho-K parameters, including toric lens option (toric or non-toric), overall diameter (OAD; 10.5 or 11.0 mm), base curve (BC), return zone depth (RZD), landing zone angle (LZA), and lens sagittal depth (LensSag). The analysis included 547 eyes of 297 Korean adolescents with myopia or astigmatism. The dataset was randomly divided into training (80%, n = 437 eyes) and validation (20%, n = 110 eyes) sets at the patient level. The model was trained based on clinical ortho-K lens fitting performed by highly experienced experts and ophthalmic measurements.

Results: The final machine-learning models showed accuracies of 92.7% and 86.4% for predicting the toric lens option and OAD, respectively. The mean absolute errors for the BC, RZD, LZA, and LensSag predictions were 0.052 mm, 2.727 µm, 0.118°, and 5.215 µm, respectively. The machine-learning model outperformed the manufacturer's conventional initial lens selector in predicting BC and RZD.

Conclusions: We developed an expert-level machine-learning–based model for determining comprehensive ortho-K lens parameters. We also created a web-based application.

Translational Relevance: This model may provide more accurate fitting parameters for lenses than those of conventional calculations, thus reducing the need to rely on trial and error.

Introduction

Orthokeratology (ortho-K) has proven effective in slowing the progression of myopia in children and adolescents.¹ It is a non-invasive method that does not require surgery or drugs, and good eyesight can be maintained without the wearing of optical devices such as glasses or contact lenses.² However, customizing ortho-K lenses to suit individual patients’ needs can be challenging depending on the practitioner's proficiency. The accurate selection of lens parameters is crucial for effective myopia correction, comfortable lens wear, and prevention of side effects.³ Traditional ortho-K fitting methods often overlook individual ophthalmologic information and rely on a trial-and-error approach. This process can be time consuming and may necessitate multiple visits to the practitioner's office, resulting in reduced efficiency and a poor treatment experience for the patient.⁴ 

Traditionally, most of the initial ortho-K lens parameters are determined using a simple nomogram-based initial lens selector (ILS) provided by the manufacturer.⁵ The ILS uses manifest refraction and keratometry values to guide parameter selection. However, it is crucial for practitioners to consider various factors such as refractive errors, corneal shape, and the patient’s comfort in depth.⁶ Prescribing a lens that does not properly fit the patient's cornea can lead to side effects such as reduced visual acuity, abnormal corneal fluorescent staining, or corneal erosion. To address these problems, follow-up interventions may be required, involving changes in the lens parameters and types and adjustments to the wearing schedule. If the initial diagnostic lens (the first fitted test lens), selected according to the manufacturer's recommendations, proves unsuitable during the evaluation process, a new lens should be ordered, and the fitting and evaluation processes should be repeated. Therefore, proper lens parameter selection is important to minimize the potential risk of side effects and alleviate the burden on patients in terms of time and effort. 

Recently, artificial intelligence–based approaches have been used to learn from expert clinical decisions in the medical field.⁷ Machine learning is a field of artificial intelligence that involves training models to map data.^8,9 In the field of ortho-K, machine-learning techniques have also been employed. For example, a study utilized machine learning to predict the return zone depth (RZD) and landing zone angle (LZA), which are two factors associated with corneal refractive therapy (CRT) ortho-K lens (Paragon Vision Sciences, Gilbert, AZ).¹⁰ However, there are other crucial ortho-K lens parameters, such as the toric option, overall diameter (OAD), and base curve (BC). 

To select the optimal ortho-K lens parameters based on optical and clinical measurements, we developed an expert-level decision support system to recommend CRT ortho-K lens parameters using machine learning. Specifically, we constructed a comprehensive prediction model that encompasses selection of the parameters, including the toric option, OAD, BC, RZD, LZA, and lens sagittal depth (LensSag), thereby providing a comprehensive support tool. The machine-learning model was trained based on the clinical decisions of highly experienced experts. We also developed a web-based application for practical implementation. 

Methods

Study Design

This retrospective study included 547 eyes of 297 Korean patients who had visited the B&VIIT Eye Center (Seoul, South Korea) between June 2020 and October 2022 for myopia treatment with ortho-K. Only patients 7 to 20 years old were included, and those with ophthalmic diseases or a history of ophthalmic surgery were excluded. We retrospectively reviewed the patients’ charts to collect final lens-fitting information. The initial ophthalmological examination data before the wearing of the lens were collected and used. This study was approved by the institutional review board of the Korean National Institute for Bioethics Policy (No. 2022-1554-002). This study adhered to the tenets of the Declaration of Helsinki. 

Data Collection

Lens fittings were performed by two Korean board-certified ophthalmologists (WKK and YKP) with an average experience of 10 years in ortho-K. All eyes were initially fitted using CRT lenses with ILS (Supplementary Fig. S1), and clinical evaluations were performed using corneal fluorescent staining and manifest refraction during the follow-up periods. After evaluation of lens fitting during the follow-up observation periods, ortho-K lens parameters were adjusted if necessary. Ortho-K lens parameters that were confirmed to be stable for more than 6 months were determined as the final prescription results. In this study, the first-visit data before treatment and the final ortho-K lens parameter data of patients with stable lens fitting were analyzed. All of the patients underwent pretreatment corrected distance visual acuity measurements, autorefraction, autokeratometry, manifest refraction, and slit-lamp examinations. Central corneal thickness (CCT) and anterior chamber depth (ACD) were measured using an AL-Scan device (NIDEK, Gamagori, Japan). In this study, ACD was defined as the distance from the corneal surface to the front of the crystalline lens. The Keratograph 4 (OCULUS, Wetzlar, Germany) was used to measure the corneal topography. We extracted the white-to-white (WTW) and corneal eccentricity values, including flat e (horizontal e), steep e (vertical e), and mean e, from the corneal topography images (Supplementary Fig. S2). 

We used anonymized medical records and ocular measurement data to train and validate the machine-learning algorithms. The input features included various parameters such as age, spherical and cylindrical powers of autorefraction (AR Sph and AR Cyl, respectively), and manifest refraction (MR Sph and MR Cyl, respectively), as well as flattest and steepest keratometry (K1 and K2, respectively), mean keratometry, keratometry astigmatism, WTW, e value, flat e value, steep e value, axial length, CCT, and ACD. The prediction targets in this study were the toric options, OAD, BC, RZD1 (primary return zone depth), RZD2 (return zone depth for the additional astigmatism axis in toric lenses), LZA, and LensSag, as shown in Figure 1. 

Figure 1.

 

Parameters of ortho-K lens fitting. The parameters include toric lens option (toric or non-toric), OAD (10.5 mm or 11.0 mm), BC, RZD, LZA, and LensSag.

Considering the characteristics of each parameter and dataset, we approached the prediction tasks differently. The toric options and OAD were treated as binary classification problems because of their distinct categories. The toric options encompassed the choices of toric (CRT dual-axis) or non-toric (CRT) lenses. As for the OAD, although OADs in CRTs are manufactured in units of 0.5 mm, in the Korean context the prevalent choices are limited to 10.5 mm and 11.0 mm. Hence, predicting the OAD involves a binary classification task that considers the two available options. On the other hand, the prediction tasks of the continuous values of BC, RZD1, RZD2, LZA, and LensSag were considered as regression problems, and machine-learning models were developed using regression algorithms. 

Datasets

Figure 2 shows the data management workflow used in this study. Initially, 585 eyes were included. Among the Korean patients, 38 eyes were excluded because of a history of ocular surgery, corneal diseases, or missing values. The final dataset consisted of 547 eyes of 297 Korean patients. The training and test datasets were randomly split at the patient level to avoid inter-eye correlations. We assigned 80% of the Korean patient data to the training dataset (N = 437 eyes), and 20% of the patient data (N = 110 eyes) were used as the internal validation (test) dataset. During the training process, we performed fivefold cross-validation to assess the performance of and optimize the prediction models. We tuned the hyperparameters of the algorithms via fivefold cross-validation to avoid overfitting. 

Figure 2.

 

Data management workflow used for the development and validation of machine-learning models to predict the optimal ortho-K lens parameters.

Machine-Learning Methods

The machine-learning algorithms used to solve the binary classification problems (toric option and OAD) were CatBoost, Extra Trees, XGBoost, and random forest. The remaining regression models (BC, RZD1, RZD2, and LZA) used least-angle regression, Ridge, Lasso, CatBoost, random forest, XGBoost, and Extra Trees regressors. Decision-tree–based algorithms, including CatBoost, Extra Trees, XGBoost, and random forest, have been recognized as robust and powerful nonlinear mapping functions in various fields.^11,12 Least-angle regression, Ridge, and Lasso are robust regression methods based on linear regressors and have been widely used for prediction tasks in the medical field.^13,14 To search for the optimal hyperparameters for each machine-learning algorithm, we adopted a grid search (Cartesian method), in which various tunable parameter values were tested via the fivefold cross-validation. We calculated the importance of permutation features in the trained models to determine the impact of each input feature. Based on the feature importance, a partial dependence plot was used to visualize the variable interaction effects.¹⁵ It should be noted that the input and output data were not normalized before training the models for real-world inference. Additionally, tree-based algorithms were not affected by data normalization due to the rule-based characteristics of each variable. We also developed a web-based hands-on application for this task. The scikit-learn Python library was used to develop the algorithms.¹⁶ 

Statistical Analysis

In the binary classification tasks, metrics including accuracy, precision, recall, and F1 were calculated between the achieved and predicted parameters. To evaluate the prediction performance of the regression tasks, we compared the mean absolute error (MAE), root mean square error (RMSE), and R² (the coefficient of determination) between the achieved and predicted parameters. All statistical comparisons were performed using a paired t-test with the significance level set at P  <  0.05. The lens fitting prediction values of the developed models and ILS were compared with the final prescription results. For fair comparisons with ILS, the output values of the models were rounded to the same units as ILS, with BC set to 0.1 mm, LZA set to 1.0°, and RZD1 and RZD2 set to 25 µm before performance evaluation. 

Results

The initial measurements of the participants in the dataset are listed in Table 1. The final ortho-K lens parameters obtained after completing the lens fitting process are listed in Table 2. Of the 297 patients, 121 were male and 176 were female. Among the 547 eyes analyzed, an OAD of 10.5 mm was prescribed for 368 eyes and an OAD of 11.0 mm was prescribed for 179 eyes. 

Table 1.

 

Demographics and Baseline Ocular Biometric Data of the Study Participants

 

Table 2.

 

Final Orthokeratology Lens Parameters Determined by Expert Ophthalmologists

 

The fivefold cross-validation results for the binary classification problems (toric option and OAD prediction) in the training dataset are shown in Figure 3. Extensive performance data are presented in Supplementary Table S1. Toric prediction determines whether an astigmatic lens (CRT dual-axis) is required. In this task, among the classifiers evaluated, the CatBoost classifier demonstrated the best performance, achieving accuracy, precision, recall, and F1 scores of 0.940, 0.941, 0.940, and 0.940, respectively. In addition, for the OAD classification model, the Extra Trees classifier outperformed the others, achieving accuracy, precision, recall, and F1 scores of 0.872, 0.895, 0.872, and 0.875, respectively. 

Figure 3.

 

Fivefold cross-validation performance of the top three models on the training dataset (N = 437). Results for the toric option (A) and OAD (B) predictions. The metrics are shown as mean ± SD from the subset validations.

As shown in Figure 4, these algorithms exhibited similar performance on the test dataset. The detailed metrics are listed in Table 3. Performance was evaluated using a test set containing 110 eyes, and the CatBoost classifier showed an accuracy of 0.927, precision of 0.931, recall of 0.927, and F1 of 0.929. In the confusion matrix, 86 of the 91 cases in which the final lens was non-toric (CRT) in the test dataset, and 16 of the 19 cases in which the final lens was CRT dual-axis (toric) were correctly classified. For the OAD classification in the test dataset, the Extra Trees classifier showed accuracy, precision, recall, and F1 scores of 0.864, 0.868, 0.864, and 0.865, respectively. The receiver operating characteristic (ROC) curves for the binary classification problems are shown in Supplementary Figure S3. The area under the ROC curves showed 0.970 and 0.920 for the toric option and OAD prediction tasks, respectively. 

Figure 4.

 

Classification performance for determining the toric lens and OAD of ortho-K. (A) Ground truth versus prediction of toric versus non-toric lens. (B) Ground truth versus prediction of 10.5-mm versus 11.0-mm OADs.

Table 3.

 

Performance of Best Models for Toric Option and OAD Prediction on the Test Dataset (N = 110)

 

Figure 5 presents the performance of the regression tasks (BC, RZD1, RZD2, and LZA) via fivefold cross-validation of the training dataset. The detailed performance data are shown in Supplementary Table S2. For BC prediction, the least-angle regression showed the highest performance, with MAE, RMSE, and R² metrics of 0.054 mm, 0.083 mm, and 0.948, respectively. In the RZD1 prediction, the least-angle regression was the most accurate, with an MAE of 3.031 µm, RMSE of 8.574 µm, and R² of 0.708. The best model for the RZD2 prediction was the CatBoost regressor, which recorded an MAE of 3.775 µm, RMSE of 11.703 µm, and R² of 0.791. For the LZA prediction, the CatBoost regressor showed an MAE of 0.121°, RMSE of 0.346°, and R² of 0.798. In the case of LensSag, Extra Trees exhibited the best performance, with MAE, RMSE, and R² values of 4.372 µm, 6.008 µm, and 0.921, respectively. 

Figure 5.

 

Fivefold cross-validation performance of the top three models on the training dataset (N = 437). Results are shown for BC (A), RZD1 (B), RZD2 (C), LZA (D), and LensSag (E) predictions. The metrics are shown as mean ± SD from the subset validations.

As shown in Table 4, these regression algorithms generally showed slightly lower performance in the test dataset compared to that in the fivefold cross-validation. For BC prediction using least-angle regression, the MAE, RMSE, and R² metrics were 0.052 mm, 0.105 mm, and 0.943, respectively. In the RZD1 prediction using least-angle regression, the MAE, RMSE, and R² metrics were 2.727 µm, 8.257 µm, and 0.718, respectively. For the RZD2 prediction, CatBoost showed MAE, RMSE, and R² values of 7.045 µm, 13.693 µm, and 0.704, respectively. For the LZA prediction, CatBoost showed MAE, RMSE, and R² values of 0.118°, 0.344°, and 0.723, respectively. The LensSag predictions obtained using Extra Trees showed MAE, RMSE, and R² values of 5.215 µm, 6.875 µm, and 0.921, respectively. 

Table 4.

 

Performance of Best Models for BC, RZD1, RZD2, LZA, and LensSag Prediction on the Test Dataset (N = 110)

 

The developed methods were compared with the ILS based on manifest refraction and keratometry, using a test dataset consisting of 110 cases (Figure 6). Regarding BC prediction, there was no significant difference between the machine-learning prediction and the achieved lens BCs (P = 0.817). However, the ILS-based BC calculation was significantly lower than the values ascertained by both the machine-learning prediction (P < 0.001) and the achieved lens BCs (P < 0.001). Similar results were observed for the prediction of RZD1. The machine-learning prediction and the achieved lens RZD1 showed no significant difference, but the ILS-based calculations showed significant differences. In the LZA prediction, there were no significant differences among the machine-learning prediction, ILS-based calculation, and achieved lens LZAs. 

Figure 6.

 

Comparison of the machine-learning (M/L) method and ILS in the test dataset. (A) BC; (B) RZD1; (C) LZA.

Figure 7 shows the feature importance and two-way partial dependence plots for each prediction task. Toric option prediction is primarily affected by cylindrical refraction and keratometric astigmatism. WTW and ACD were the most important predictors of OAD. Spherical and cylindrical refraction, flattest and steepest keratometry, and flat e values were the major factors predicting BC, RZD1, RZD2, and LZA. Finally, a machine-learning calculator was developed for the web-based interface (https://visuworks-dev.github.io/ParagonCRT-Calculator/) based on this analysis. A screenshot of the developed calculator is shown in Figure 8. 

Figure 7.

 

Permutation feature importance (PFI) estimates selected using the machine-learning method and two-way partial dependence plots (PDPs). (A) Toric option prediction based on cylindrical refraction (MR Cyl) and keratometry astigmatism (dK) values (threshold = 0.5; 0 = non-toric; 1 = toric). (B) OAD prediction based on WTW and ACD values (threshold = 0.5; 0 = 10.5 mm; 1 = 11 mm). (C) BC prediction based on MR Sph (spherical refraction) and K1 (flattest keratometry). (D) RZD1 prediction based on K1 (flattest keratometry) and MR Sph. (E) RZD2 prediction based on MR Cyl and K2 (steepest keratometry). (F) LZA prediction based on K1 and flat e (flat or horizontal eccentricity). (G) LensSag prediction based on K1 and K2.

Figure 8.

 

Schematic illustrating our proposed machine-learning model for ortho-K lens fitting. A web-based calculator was developed for practice (https://visuworks-dev.github.io/ParagonCRT-Calculator/).

Discussion

Ortho-K is an effective treatment option to slow myopia progression for children and adolescents.¹⁷ We applied a machine-learning algorithm to develop a CRT ortho-K lens parameter prediction model. The machine-learning models effectively learned the implicit lens prescription patterns in the learning dataset. Our algorithm outperformed the traditional fitting calculator, ILS, in predicting the BC and RZD1. To the best of our knowledge, this is the first study to comprehensively predict the ortho-K lens parameters at an expert level using artificial intelligence. The algorithm of the previous study predicted only two factors (RZD and LZA); therefore, it might be limited to ordering commercial CRT lenses in the real world.¹⁰ The algorithm we developed aims to assist in real-world clinical practice by predicting all of the relevant lens parameters and being accessible through a web-based interface. According to a previous study, ortho-K requires a long learning curve for novice practitioners compared to those required for other types of contact lenses.¹⁸ Our algorithm will be useful for novice practitioners because it can provide expert-level ortho-K lens parameters based on a data-driven approach. 

In this study, we have presented a novel ortho-K fitting support tool to predict the lens design that best suited each patient using machine-learning models that successfully fit the data. We believe that the machine-learning method showed a level of performance applicable to actual clinical practice in determining ortho-K parameters such as toric option, OAD, BC, RZD1, RZD2, LZA, and LensSag. By leveraging clinical factors such as refraction, corneal topography, and AL-scan measurements, the machine-learning methods significantly enhanced the accuracy of prediction compared to those of existing approaches. The CRT fitting method using machine-learning models has the potential to replace the traditional ILS method, which relies solely on manifest refraction and keratometry, thereby reducing the time-consuming trial-and-error processes involved in lens fitting. Reducing trial-and-error processes improves patient compliance and achieves myopia suppression.¹⁹ 

We analyzed how the features were involved in the prediction process of each model and attempted to understand the reasoning process of the models. Although some machine-learning algorithms are considered black-box models that cannot be interpreted,²⁰ our approach provided insights into the decision-making processes. As shown in Figure 7, the factors used in the machine learning were consistent with the variables considered important in actual clinical practice. This analysis showed that the inference process of the machine-learning model was consistent with the ortho-K fitting principle. Machine-learning methods can analyze not only manifest refraction and keratometry but also additional clinical information simultaneously and extract the necessary prediction information from nonlinear variable spaces, as shown in the two-way partial dependence plots. In our experiments, most algorithms of the decision-tree–based ensemble methods exhibited the highest performance. These algorithms successfully modeled the distribution of a complex mixture of linear and nonlinear boundaries. Recent studies have demonstrated the most robust performances of the decision-tree–based methods for a variety of engineering and medical problems.^21,22 

Considering the differences in eye characteristics between East Asians and Caucasians, the ILS method based on the characteristics of Caucasian eye is likely less suitable for East Asians. For example, unlike ILS, which uses only MR Sph and K1 as factors in LZA predictions, the machine-learning model uses an additional flat e (horizontal eccentricity) factor. Thus, the use of factors related to corneal morphology, such as eccentricity, may help address the racial bias inherent in the existing methods. Therefore, our novel machine-learning algorithms, specifically tailored for East Asians, demonstrate their superiority in this regard. Notably, East Asians and Caucasians exhibit distinct clinical characteristics in ortho-K studies,²³ and further studies are needed to develop more accurate ortho-K lens fitting models. 

A major advantage of the proposed model is that it can use multimodal measurements to achieve an accurate ortho-K lens fitting. However, many input parameters can be considered disadvantageous because of their inconvenience. Another advantage is that we developed a web-based interface for a practical and fast hands-on experience (https://visuworks-dev.github.io/ParagonCRT-Calculator/). This tool is publicly available on GitHub, which provides free repositories. If more complex machine-learning–based tools for ortho-K lens parameter selection are developed in the future, they will be commercialized due to server operating costs. 

However, this method has some limitations that must be addressed in future studies. First, it is necessary to include a wider dataset to ensure generalizability of the machine-learning model. The dataset collected in this study consisted of data from Korean patients between the ages of 7 and 20 years who were treated at a single clinic. Therefore, to ensure that the developed model is effective for more diverse age groups and ethnic backgrounds,^24,25 extensive data reflecting different population characteristics are required. Accordingly, external validation should be performed using independent sets. Second, a prospective study is required to determine whether the developed method reduces the need for a trial-and-error approach by practitioners.²⁶ The machine-learning method was more accurate than the traditional method; however, further validation is required to determine whether it reduces the number of lens fittings until a lens suitable for each patient is found in the clinical field. Third, we used permutation feature importance and two-way partial dependence plots for feature analysis. However, these methods are insufficient to represent the decision-making process in machine learning. As the number of features increased, it became difficult to visualize and interpret all interactions²⁷; therefore, in this study, only the two features with the highest importance scores for each model were analyzed. 

In conclusion, we demonstrated the potential of machine learning in determining CRT ortho-K lens parameters, particularly in supporting data-driven decision-making. We developed an expert-level machine-learning–based model for determining the comprehensive ortho-K lens parameters. We also created a web-based application for this task (https://visuworks-dev.github.io/ParagonCRT-Calculator/). This model may provide lens fitting parameters more accurate than those provided by conventional calculations and can reduce the need for a trial-and-error approach. Future work should focus on applying a broader dataset and exploring machine-learning techniques to further improve the predictive capabilities and applicability of the model in real-world scenarios. 

Acknowledgments

VISUWORKS received research grants for SMILE surgery from Carl Zeiss Meditec and WooJeon Co. The research grants did not affect this study. 

Disclosure: S. Koo, VISUWORKS (E); W.K. Kim, None; Y.K. Park, None; K. Jun, VISUWORKS (E); D. Kim, VISUWORKS (E); I.H. Ryu, VISUWORKS (E); J.K. Kim, VISUWORKS (E), Korea Medical Devices Industry Association (C); T.K. Yoo, VISUWORKS (E)