Data Quality Estimation Via Model Performance: Machine Learning as a Validation Tool.

In our recent study, the attempt to classify neurosurgical operative reports into routinely used expert-derived classes exhibited an F-score not exceeding 0.74. This study aimed to test how improving the classifier (target variable) affected the short text classification with deep learning on real-world data. We redesigned the target variable based on three strict principles when applicable: pathology, localization, and manipulation type. The deep learning significantly improved with the best result of operative report classification into 13 classes (accuracy = 0.995, F1 = 0.990). Reasonable text classification with machine learning should be a two-way process: the model performance must be ensured by the unambiguous textual representation reflected in corresponding target variables. At the same time, the validity of human-generated codification can be inspected via machine learning.

Multinomial Classification of Neurosurgical Operations Using Gradient Boosting and Deep Learning Algorithms.

This study aimed at testing the feasibility of neurosurgical procedures classification into 100+ classes using natural language processing and machine learning. A catboost algorithm and bidirectional recurrent neural network with a gated recurrent unit showed almost the same accuracy of ∼81%, with suggestions of correct class in top 2-3 scored classes up to 98.9%. The classification of neurosurgical procedures via machine learning appears to be a technically solvable task which can be additionally improved considering data enhancement and classes verification.

Semiautomated Approach for Muscle Weakness Detection in Clinical Texts.

The automated detection of adverse events in medical records might be a cost-effective solution for patient safety management or pharmacovigilance. Our group proposed an information extraction algorithm (IEA) for detecting adverse events in neurosurgery using documents written in a natural rich-in-morphology language. In this paper, we challenge to optimize and evaluate its performance for the detection of any extremity muscle weakness in clinical texts. Our algorithm shows the accuracy of 0.96 and ROC AUC = 0.96 and might be easily implemented in other medical domains.

Detection of Muscle Weakness in Medical Texts Using Natural Language Processing.

Identifying adverse events in clinical documents is demanded in retrospective clinical research and prospective monitoring of treatment safety and cost-effectiveness. We proposed and evaluated a few methods of semi-automated muscle weakness detection in preoperative clinical notes for a larger project on predicting paresis by images. The combination of semi-expert and machine learning methods demonstrated maximized sensitivity = 0.860 and specificity = 0.919, and largest AUC = 0.943 with a 95% CI [0.874; 0.991], outperforming each method used individually. Our approaches are expected to be effective for autoshaping a well- verified training dataset for supervised machine learning.