Global Model Selection for Semi-Supervised Support Vector Machine via Solution Paths.

Semi-supervised support vector machine (S 3 VM) is important because it can use plentiful unlabeled data to improve the generalization accuracy of traditional SVMs. In order to achieve good performance, it is necessary for S 3 VM to take some effective measures to select hyperparameters. However, model selection for semi-supervised models is still a key open problem. Existing methods for semi-supervised models to search for the optimal parameter values are usually computationally demanding, especially those ones with grid search. To address this challenging problem, in this article, we first propose solution paths of S 3 VM (SPS 3 VM), which can track the solutions of the nonconvex S 3 VM with respect to the hyperparameters. Specifically, we apply incremental and decremental learning methods to update the solution and let it satisfy the Karush-Kuhn-Tucker (KKT) conditions. Based on the SPS 3 VM and the piecewise linearity of model function, we can find the model with the minimum cross-validation (CV) error for the entire range of candidate hyperparameters by computing the error path of S 3 VM. Our SPS 3 VM is the first solution path algorithm for nonconvex optimization problem of semi-supervised learning models. We also provide the finite convergence analysis and computational complexity of SPS 3 VM. Experimental results on a variety of benchmark datasets not only verify that our SPS 3 VM can globally search the hyperparameters (regularization and ramp loss parameters) but also show a huge reduction of computational time while retaining similar or slightly better generalization performance compared with the grid search approach.

Global Model Selection via Solution Paths for Robust Support Vector Machine.

Robust support vector machine (RSVM) using ramp loss provides a better generalization performance than traditional support vector machine (SVM) using hinge loss. However, the good performance of RSVM heavily depends on the proper values of regularization parameter and ramp parameter. Traditional model selection technique with gird search has extremely high computational cost especially for fine-grained search. To address this challenging problem, in this paper, we first propose solution paths of RSVM (SPRSVM) based on the concave-convex procedure (CCCP) which can track the solutions of the non-convex RSVM with respect to regularization parameter and ramp parameter respectively. Specifically, we use incremental and decremental learning algorithms to deal with the Karush-Khun-Tucker violating samples in the process of tracking the solutions. Based on the solution paths of RSVM and the piecewise linearity of model function, we can compute the error paths of RSVM and find the values of regularization parameter and ramp parameter, respectively, which corresponds to the minimum cross validation error. We prove the finite convergence of SPRSVM and analyze the computational complexity of SPRSVM. Experimental results on a variety of benchmark datasets not only verify that our SPRSVM can globally search the regularization and ramp parameters respectively, but also show a huge reduction of computational time compared with the grid search approach.

Long-Term Axial Length Shortening in Myopic Orthokeratology: Incident Probability, Time Course, and Influencing Factors.

Purpose: Long-term axial length (AL) shortening in myopia is uncommon but noteworthy. Current understanding on the condition is limited due to difficulties in case collection. The study reported percentage, probability, and time course of long-term AL shortening in myopic orthokeratology based on a large database. Methods: This study reviewed 142,091 medical records from 29,825 subjects in a single-hospital orthokeratology database that were collected over 10 years. Long-term AL shortening was defined as a change in AL of -0.1 mm or less at any follow-up beyond 1 year. Incident probability was calculated based on multivariate logistic regression. Time course was estimated using mixed-effect regression model. Results: A total of 10,093 subjects (mean initial age, 11.70 ± 2.52 years; 58.8% female) with 80,778 visits were included. The number of subjects experienced long-term AL shortening was 1,662 (16.47%; 95% confidence interval, 15.75%-17.21%). Initial age showed significant impact on the incident occurrence (OR, 1.37; 95% confidence interval, 1.34-1.40; P < 0.001). The estimated probability of AL shortening was approximately 2% for subjects with initial age of 6 years and 50% for those aged 18. Among the 1662 AL shortening cases, the median magnitude of the maximum AL reduction was 0.19 mm. The shortening process mostly occurred within the initial 2 years. Subject characteristics had limited associations with the shortening rate. Conclusions: Long-term AL shortening is possible in subjects receiving myopic orthokeratology. Although age notably affect the incident probability, the time course seems to not vary significantly.

Kernel Path for ν-Support Vector Classification.

It is well known that the performance of a kernel method is highly dependent on the choice of kernel parameter. However, existing kernel path algorithms are limited to plain support vector machines (SVMs), which has one equality constraint. It is still an open question to provide a kernel path algorithm to ν -support vector classification ( ν -SVC) with more than one equality constraint. Compared with plain SVM, ν -SVC has the advantage of using a regularization parameter ν for controlling the number of support vectors and margin errors. To address this problem, in this article, we propose a kernel path algorithm (KP ν SVC) to trace the solutions of ν -SVC exactly with respect to the kernel parameter. Specifically, we first provide an equivalent formulation of ν -SVC with two equality constraints, which can avoid possible conflicts during tracing the solutions of ν -SVC. Based on this equivalent formulation of ν -SVC, we propose the KP ν SVC algorithm to trace the solutions with respect to the kernel parameter. However, KP ν SVC traces nonlinear solutions of kernel method rather than the errors of loss function, and it is still a challenge to provide the algorithm that guarantees to find the global optimal model. To address this challenging problem, we extend the classical error path algorithm to the nonlinear kernel solution paths and propose a new kernel error path (KEP) algorithm that ensures to find the global optimal kernel parameter by minimizing the cross validation error. We also provide the finite convergence analysis and computational complexity analysis to KP ν SVC and KEP. Extensive experimental results on a variety of benchmark datasets not only verify the effectiveness of KP ν SVC but also show the advantage of applying KEP to select the optimal kernel parameter.

Kernel Path for Semisupervised Support Vector Machine.

Semisupervised support vector machine (S 3 VM) is a powerful semisupervised learning model that can use large amounts of unlabeled data to train high-quality classification models. The choice of kernel parameters in the kernel function determines the mapping between the input space and the feature space and is crucial to the performance of the S 3 VM. Kernel path algorithms have been widely recognized as one of the most efficient tools to trace the solutions with respect to a kernel parameter. However, existing kernel path algorithms are limited to convex problems, while S 3 VM is nonconvex problem. To address this challenging problem, in this article, we first propose a kernel path algorithm of S 3 VM (KPS 3 VM), which can track the solutions of the nonconvex S 3 VM with respect to a kernel parameter. Specifically, we estimate the position of the breakpoint by monitoring the change of the sample sets. In addition, we also use an incremental and decremental learning algorithm to deal with the Karush-Khun-Tucker violating samples in the process of tracking the solutions. More importantly, we prove the finite convergence of our KPS 3 VM algorithm. Experimental results on various benchmark datasets not only validate the effectiveness of our KPS 3 VM algorithm but also show the advantage of choosing the optimal kernel parameters.

Efficient Active Learning by Querying Discriminative and Representative Samples and Fully Exploiting Unlabeled Data.

Active learning is an important learning paradigm in machine learning and data mining, which aims to train effective classifiers with as few labeled samples as possible. Querying discriminative (informative) and representative samples are the state-of-the-art approach for active learning. Fully utilizing a large amount of unlabeled data provides a second chance to improve the performance of active learning. Although there have been several active learning methods proposed by combining with semisupervised learning, fast active learning with fully exploiting unlabeled data and querying discriminative and representative samples is still an open question. To overcome this challenging issue, in this article, we propose a new efficient batch mode active learning algorithm. Specifically, we first provide an active learning risk bound by fully considering the unlabeled samples in characterizing the informativeness and representativeness. Based on the risk bound, we derive a new objective function for batch mode active learning. After that, we propose a wrapper algorithm to solve the objective function, which essentially trains a semisupervised classifier and selects discriminative and representative samples alternately. Especially, to avoid retraining the semisupervised classifier from scratch after each query, we design two unique procedures based on the path-following technique, which can remove multiple queried samples from the unlabeled data set and add the queried samples into the labeled data set efficiently. Extensive experimental results on a variety of benchmark data sets not only show that our algorithm has a better generalization performance than the state-of-the-art active learning approaches but also show its significant efficiency.

Development and validation of a prediction model for axial length elongation in myopic children treated with overnight orthokeratology.

PURPOSE: To develop and validate a standardized prediction model aiming at 1-year axial length elongation and to guide the orthokeratology lens practice. METHODS: This retrospective study was based on medical records of myopic children treated with orthokeratology. Individuals aged 8-15 years (n = 1261) were included and divided into the primary cohort (n = 757) and validation cohort (n = 504). Feature selection was primarily performed to sort out influential predictors by high-throughput extraction. Then, the prediction model was developed using multivariable linear regression analysis completed by backward stepwise selection. Finally, the validation of the prediction model was performed by evaluation metrics (mean-square error, root-mean-square error, mean absolute error and R ad 2 ). RESULTS: No significant difference was found between primary and validation cohort (all p > 0.05). After the feature selection, the crude model was adjusted by demographic information in multivariable linear regression analysis, and five final predictors were identified (all p < 0.01). The interaction effect of age with 1-month change of zone-3 mm flat K was detected (p < 0.01); hence, two final prediction models were developed based on two age subgroups. The validation proved an acceptable performance. CONCLUSION: An effective multivariable prediction model aiming at 1-year axial length elongation was developed and validated. It can potentially help clinicians to predict orthokeratology efficacy and make valid adjustments. The influential variables revealed in this model can also provide designers directions to optimize the design of lens to improve the efficacy of myopia control.

Rigid Gas-Permeable Contact Lenses Fitting Philosophy for Unilateral Aphakic Infants.

PURPOSE: To provide fitting guidelines with suggested powers and base curves (BCs) and diameters for initial rigid gas-permeable (RGP) contact lenses (CLs) selection for unilateral aphakic infants based on age. METHODS: Records of 52 children (52 eyes) with RGP CLs to unilateral aphakia between 2014 and 2019 were evaluated. Refractive status was assessed by standard retinoscopy. The original BC and diameter were determined by keratometric readings and fluorescein pattern under sedation. Correlation analysis was performed between age and CLs parameters. Linear regression analysis was used to develop a model for estimating power with the help of infant's age. Subgroup analysis was performed by grouping the eyes into four groups according to age. Lens adjustments and adverse events were also evaluated. RESULTS: The median age was 9.0 months (interquartile range [IQR], 5.25-13.0 months). The mean power and BC and diameter of the initial RGP CLs were 25.46±4.83 diopters, 7.57±0.40 mm, and 9.48±0.23 mm, respectively. All these parameters showed correlations with infant's age (Pearson r=-0.676, 0.367, and 0.497, respectively; P=0.000, 0.008, and 0.000, respectively). Regression analysis revealed that CL power =31.66 to 0.62×age (P<0.001). The median follow-up was 7.50 months (IQR, 3.0-11.0 months). Lens adjustments took about every 3 months before 1 year of age and every 5 months afterward (F=3.442; P=0.024). The RGP CLs provided ideal fit characteristics, and no severe lens-related adverse event occurred except only one patient had mild conjunctivitis. CONCLUSIONS: Our empirical RGP CLs fitting philosophy presented that aphakic infant's age can be used to determine the initial lens if accurate biometry cannot be obtained initially.

Pattern of Axial Length Growth in Children Myopic Anisometropes with Orthokeratology Treatment.

PURPOSE: To compare the pattern of growth in axial length (AL) between children with anisometropia who wear orthokeratology (OK) lenses and those who wear spectacles (SP). METHODS: A retrospective study was conducted. Data of baseline and 1 year from 252 children (8-14 years old) anisomyopes who sought refraction corrections at the Zhongshan Ophthalmic Center between October 2013 and June 2017 were reviewed. Seventy-nine unilateral myopic anisometropes (UMA) and 98 bilateral myopic anisometropes (BMA) treated with OK lenses were set as study groups (OK-UMA and OK-BMA groups). Age, refraction, and AL-matched unilateral (n = 38) and bilateral myopic anisometropes (n = 37) treated with spectacles were set as control groups (SP-UMA and SP-BMA groups). The 1-year change in AL between the study and control groups (OK-UMA vs. SP-UMA and OK-BMA vs. SP-BMA) was compared. RESULTS: There were no significant differences in the baseline of age, refraction, and AL between OK-UMA and SP-UMA or OK-BMA and SP-BMA groups (all P > .05). Compared to the SP-UMA group, annual axial elongation of the myopic eyes of the OK-UMA group was smaller (0.05 ± 0.19 mm vs. 0.33 ± 0.29 mm, P < .001); however, AL elongation in the non-myopic eyes were comparable between SP-UMA and OK-UMA groups (P > .05). At the end of 1 year, the interocular difference in AL (aniso-AL) decreased by 0.29 ± 0.29 mm (P < .001) in the OK-UMA group but remained unchanged in SP-UMA group. Compared to the SP-BMA group, annual axial elongations of both eyes of the OK-BMA group were smaller (the more myopic eye, 0.05 ± 0.17 mm vs. 0.38 ± 0.21 mm; the less myopic eye, 0.15 ± 0.19 mm vs. 0.35 ± 0.28 mm; both P < .001). At the end of 1 year, aniso-AL decreased by 0.10 ± 0.15 mm (P < .001) in the OK-BMA group but remained unchanged in the SP-BMA group. CONCLUSION: Orthokeratology is effective in reducing the interocular difference in AL of children anisomyopes through greater retardation of axial elongation of the more myopic eyes.