ABSTRACT
In our recent study, the attempt to classify neurosurgical operative reports into routinely used expert-derived classes exhibited an F-score not exceeding 0.74. This study aimed to test how improving the classifier (target variable) affected the short text classification with deep learning on real-world data. We redesigned the target variable based on three strict principles when applicable: pathology, localization, and manipulation type. The deep learning significantly improved with the best result of operative report classification into 13 classes (accuracy = 0.995, F1 = 0.990). Reasonable text classification with machine learning should be a two-way process: the model performance must be ensured by the unambiguous textual representation reflected in corresponding target variables. At the same time, the validity of human-generated codification can be inspected via machine learning.
Subject(s)
Data Accuracy , Machine Learning , HumansABSTRACT
This study aimed at testing the feasibility of neurosurgical procedures classification into 100+ classes using natural language processing and machine learning. A catboost algorithm and bidirectional recurrent neural network with a gated recurrent unit showed almost the same accuracy of â¼81%, with suggestions of correct class in top 2-3 scored classes up to 98.9%. The classification of neurosurgical procedures via machine learning appears to be a technically solvable task which can be additionally improved considering data enhancement and classes verification.
Subject(s)
Deep Learning , Algorithms , Machine Learning , Natural Language Processing , Neural Networks, ComputerABSTRACT
The automated detection of adverse events in medical records might be a cost-effective solution for patient safety management or pharmacovigilance. Our group proposed an information extraction algorithm (IEA) for detecting adverse events in neurosurgery using documents written in a natural rich-in-morphology language. In this paper, we challenge to optimize and evaluate its performance for the detection of any extremity muscle weakness in clinical texts. Our algorithm shows the accuracy of 0.96 and ROC AUC = 0.96 and might be easily implemented in other medical domains.
Subject(s)
Muscle Weakness , Natural Language Processing , Electronic Health Records , Humans , Information Storage and Retrieval , PharmacovigilanceABSTRACT
Identifying adverse events in clinical documents is demanded in retrospective clinical research and prospective monitoring of treatment safety and cost-effectiveness. We proposed and evaluated a few methods of semi-automated muscle weakness detection in preoperative clinical notes for a larger project on predicting paresis by images. The combination of semi-expert and machine learning methods demonstrated maximized sensitivity = 0.860 and specificity = 0.919, and largest AUC = 0.943 with a 95% CI [0.874; 0.991], outperforming each method used individually. Our approaches are expected to be effective for autoshaping a well- verified training dataset for supervised machine learning.