A Preliminary Evaluation of the Diagnostic Performance of a Smartphone-Based Machine Learning-Assisted System for Evaluation of Clinical Activity Score in Digital Images of Thyroid-Associated Orbitopathy.

Background: We previously developed a machine learning (ML)-assisted system for predicting the clinical activity score (CAS) in thyroid-associated orbitopathy (TAO) using digital facial images taken by a digital single-lens reflex camera in a studio setting. In this study, we aimed to apply this system to smartphones and detect active TAO (CAS ≥3) using facial images captured by smartphone cameras. We evaluated the performance of our system on various smartphone models and compared it with the performance of ophthalmologists with varying clinical experience. Methods: We applied the preexisting ML architecture to classify photos taken with smartphones (Galaxy S21 Ultra, iPhone 12 pro, iPhone 11, iPhone SE 2020, Galaxy M20, and Galaxy A21S). The performance was evaluated with smartphone-captured images from 100 patients with TAO. Three ophthalmology residents, three general ophthalmologists with <5 years of clinical experience, and three oculoplastic specialists independently interpreted the same set of images taken under a studio environment and compared their results with those generated by the smartphone-based ML-assisted system. Reference CAS was determined by a consensus of three oculoplastic specialists. Results: Active TAO (CAS ≥3) was identified in 28 patients. Smartphone model used in capturing facial images influenced active TAO detection performance (F1 score 0.59-0.72). The smartphone-based system showed 74.5% sensitivity, 84.8% specificity, and F1 score 0.70 on top three smartphones. On images from all six smartphones, average sensitivity, specificity, and F1 score were 71.4%, 81.6%, and 0.66, respectively. Ophthalmology residents' values were 69.1%, 55.1%, and 0.46. General ophthalmologists' values were 61.9%, 79.6%, and 0.55. Oculoplastic specialists' values were 73.8%, 90.7%, and 0.75. This smartphone-based ML-assisted system predicted CAS within 1 point of reference CAS in 90.7% using facial images from smartphones. Conclusions: Our smartphone-based ML-assisted system shows reasonable accuracy in detecting active TAO, comparable with oculoplastic specialists and outperforming residents and general ophthalmologists. It may enable reliable self-monitoring for disease activity, but confirmatory research is needed for clinical application.

A machine learning-assisted system to predict thyrotoxicosis using patients' heart rate monitoring data: a retrospective cohort study.

Previous studies have shown a correlation between resting heart rate (HR) measured by wearable devices and serum free thyroxine concentration in patients with thyroid dysfunction. We have developed a machine learning (ML)-assisted system that uses HR data collected from wearable devices to predict the occurrence of thyrotoxicosis in patients. HR monitoring data were collected using a wearable device for a period of 4 months in 175 patients with thyroid dysfunction. During this period, 3 or 4 thyroid function tests (TFTs) were performed on each patient at intervals of at least one month. The HR data collected during the 10 days prior to each TFT were paired with the corresponding TFT results, resulting in a total of 662 pairs of data. Our ML-assisted system predicted thyrotoxicosis of a patient at a given time point based on HR data and their HR-TFT data pair at another time point. Our ML-assisted system divided the 662 cases into either thyrotoxicosis and non-thyrotoxicosis and the performance was calculated based on the TFT results. The sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of our system for predicting thyrotoxicosis were 86.14%, 85.92%, 52.41%, and 97.18%, respectively. When subclinical thyrotoxicosis was excluded from the analysis, the sensitivity, specificity, PPV, and NPV of our system for predicting thyrotoxicosis were 86.14%, 98.28%, 94.57%, and 95.32%, respectively. Our ML-assisted system used the change in mean, relative standard deviation, skewness, and kurtosis of HR while sleeping, and the Jensen-Shannon divergence of sleep HR and TFT distribution as major parameters for predicting thyrotoxicosis. Our ML-assisted system has demonstrated reasonably accurate predictions of thyrotoxicosis in patients with thyroid dysfunction, and the accuracy could be further improved by gathering more data. This predictive system has the potential to monitor the thyroid function status of patients with thyroid dysfunction by collecting heart rate data, and to determine the optimal timing for blood tests and treatment intervention.

Machine learning-assisted system using digital facial images to predict the clinical activity score in thyroid-associated orbitopathy.

Although the clinical activity score (CAS) is a validated scoring system for identifying disease activity of thyroid-associated orbitopathy (TAO), it may produce differing results depending on the evaluator, and an experienced ophthalmologist is required for accurate evaluation. In this study, we developed a machine learning (ML)-assisted system to mimic an expert's CAS assessment using digital facial images and evaluated its accuracy for predicting the CAS and diagnosing active TAO (CAS ≥ 3). An ML-assisted system was designed to assess five CAS components related to inflammatory signs (redness of the eyelids, redness of the conjunctiva, swelling of the eyelids, inflammation of the caruncle and/or plica, and conjunctival edema) in patients' facial images and to predict the CAS by considering two components of subjective symptoms (spontaneous retrobulbar pain and pain on gaze). To train and test the system, 3,060 cropped images from 1020 digital facial images of TAO patients were used. The reference CAS for each image was scored by three ophthalmologists, each with > 15 years of clinical experience. We repeated the experiments for 30 randomly split training and test sets at a ratio of 8:2. The sensitivity and specificity of the ML-assisted system for diagnosing active TAO were 72.7% and 83.2% in the test set constructed from the entire dataset. For the test set constructed from the dataset with consistent results for the three ophthalmologists, the sensitivity and specificity for diagnosing active TAO were 88.1% and 86.9%. In the test sets from the entire dataset and from the dataset with consistent results, 40.0% and 49.9% of the predicted CAS values were the same as the reference CAS, respectively. The system predicted the CAS within 1 point of the reference CAS in 84.6% and 89.0% of cases when tested using the entire dataset and in the dataset with consistent results, respectively. An ML-assisted system estimated the clinical activity of TAO and detect inflammatory active TAO with reasonable accuracy. The accuracy could be improved further by obtaining more data. This ML-assisted system can help evaluate the disease activity consistently as well as accurately and enable the early diagnosis and timely treatment of active TAO.