Successful real-world application of an osteoarthritis classification deep-learning model using 9210 knees-An orthopedic surgeon's view.

This study aimed to evaluate the performance of a deep-learning model to evaluate knee osteoarthritis using Kellgren-Lawrence grading in real-life knee radiographs. A deep convolutional neural network model was trained using 8964 knee radiographs from the osteoarthritis initiative (OAI), including 962 testing set images. Another 246 knee radiographs from the Far Eastern Memorial Hospital were used for external validation. The OAI testing set and external validation images were evaluated by experienced specialists, two orthopedic surgeons, and a musculoskeletal radiologist. The accuracy, interobserver agreement, F1 score, precision, recall, specificity, and ability to identify surgical candidates were used to compare the performances of the model and specialists. Attention maps illustrated the interpretability of the model classification. The model had a 78% accuracy and consistent interobserver agreement for the OAI (model-surgeon 1 К = 0.80, model-surgeon 2 К = 0.84, model-radiologist К = 0.86) and external validation (model-surgeon 1 К = 0.81, model-surgeon 2 К = 0.82, model-radiologist К = 0.83) images. A lower interobserver agreement was found in the images misclassified by the model (model-surgeon 1 К = 0.57, model-surgeon 2 К = 0.47, model-radiologist К = 0.65). The model performed better than specialists in identifying surgical candidates (Kellgren-Lawrence Stages 3 and 4) with an F1 score of 0.923. Our model not only had comparable results with specialists with respect to the ability to identify surgical candidates but also performed consistently with open database and real-life radiographs. We believe the controversy of the misclassified knee osteoarthritis images was based on a significantly lower interobserver agreement.

Training a Deep Contextualized Language Model for International Classification of Diseases, 10th Revision Classification via Federated Learning: Model Development and Validation Study.

BACKGROUND: The automatic coding of clinical text documents by using the International Classification of Diseases, 10th Revision (ICD-10) can be performed for statistical analyses and reimbursements. With the development of natural language processing models, new transformer architectures with attention mechanisms have outperformed previous models. Although multicenter training may increase a model's performance and external validity, the privacy of clinical documents should be protected. We used federated learning to train a model with multicenter data, without sharing data per se. OBJECTIVE: This study aims to train a classification model via federated learning for ICD-10 multilabel classification. METHODS: Text data from discharge notes in electronic medical records were collected from the following three medical centers: Far Eastern Memorial Hospital, National Taiwan University Hospital, and Taipei Veterans General Hospital. After comparing the performance of different variants of bidirectional encoder representations from transformers (BERT), PubMedBERT was chosen for the word embeddings. With regard to preprocessing, the nonalphanumeric characters were retained because the model's performance decreased after the removal of these characters. To explain the outputs of our model, we added a label attention mechanism to the model architecture. The model was trained with data from each of the three hospitals separately and via federated learning. The models trained via federated learning and the models trained with local data were compared on a testing set that was composed of data from the three hospitals. The micro F1 score was used to evaluate model performance across all 3 centers. RESULTS: The F1 scores of PubMedBERT, RoBERTa (Robustly Optimized BERT Pretraining Approach), ClinicalBERT, and BioBERT (BERT for Biomedical Text Mining) were 0.735, 0.692, 0.711, and 0.721, respectively. The F1 score of the model that retained nonalphanumeric characters was 0.8120, whereas the F1 score after removing these characters was 0.7875-a decrease of 0.0245 (3.11%). The F1 scores on the testing set were 0.6142, 0.4472, 0.5353, and 0.2522 for the federated learning, Far Eastern Memorial Hospital, National Taiwan University Hospital, and Taipei Veterans General Hospital models, respectively. The explainable predictions were displayed with highlighted input words via the label attention architecture. CONCLUSIONS: Federated learning was used to train the ICD-10 classification model on multicenter clinical text while protecting data privacy. The model's performance was better than that of models that were trained locally.