Data Quality Estimation Via Model Performance: Machine Learning as a Validation Tool.

In our recent study, the attempt to classify neurosurgical operative reports into routinely used expert-derived classes exhibited an F-score not exceeding 0.74. This study aimed to test how improving the classifier (target variable) affected the short text classification with deep learning on real-world data. We redesigned the target variable based on three strict principles when applicable: pathology, localization, and manipulation type. The deep learning significantly improved with the best result of operative report classification into 13 classes (accuracy = 0.995, F1 = 0.990). Reasonable text classification with machine learning should be a two-way process: the model performance must be ensured by the unambiguous textual representation reflected in corresponding target variables. At the same time, the validity of human-generated codification can be inspected via machine learning.

The Assessment of Glioblastoma Metabolic Activity via 11C-Methionine PET and Radiomics.

Nowadays, the quantitative analysis of PET/CT data in patients with glioblastoma is not strictly standardized in the clinic and does not exclude the human factor. This study aimed to evaluate the relationship between the radiomic features of glioblastoma 11C-methionine PET images and the tumor-to-normal brain (T/N) ratio determined by radiologists in clinical routine. PET/CT data were obtained for 40 patients (mean age 55 ± 12 years; 77.5% men) with a histologically confirmed diagnosis of glioblastoma. Radiomic features were calculated for the whole brain and tumor-containing regions of interest using the RIA package for R. We redesigned the original RIA functions for GLCM and GLRLM calculation to reduce computation time significantly. Machine learning over radiomic features was applied to predict T/N with the best median correlation between the true and predicted values of 0.73 (p = 0.01). The present study showed a reproducible linear relationship between 11C-methionine PET radiomic features and a T/N indicator routinely assessed in brain tumors. Radiomics enabled utilizing texture properties of PET/CT neuroimaging that may reflect the biological activity of glioblastoma and can potentially augment the radiological assessment.

An optimized IL-12-Fc expands its therapeutic window, achieving strong activity against mouse tumors at tolerable drug doses.

BACKGROUND: Interleukin-12 (IL-12) has emerged as one of the most potent cytokines for tumor immunotherapy due to its ability to induce interferon γ (IFNγ) and polarize Th1 responses. Clinical use of IL-12 has been limited by a short half-life and narrow therapeutic index. METHODS: We generated a monovalent, half-life-extended IL-12-Fc fusion protein, mDF6006, engineered to retain the high potency of native IL-12 while significantly expanding its therapeutic window. In vitro and in vivo activity of mDF6006 was tested against murine tumors. To translate our findings, we developed a fully human version of IL-12-Fc, designated DF6002, which we characterized in vitro on human cells and in vivo in cynomolgus monkeys in preparation for clinical trials. FINDINGS: The extended half-life of mDF6006 modified the pharmacodynamic profile of IL-12 to one that was better tolerated systemically while vastly amplifying its efficacy. Mechanistically, mDF6006 led to greater and more sustained IFNγ production than recombinant IL-12 without inducing high, toxic peak serum concentrations of IFNγ. We showed that mDF6006's expanded therapeutic window allowed for potent anti-tumor activity as single agent against large immune checkpoint blockade-resistant tumors. Furthermore, the favorable benefit-risk profile of mDF6006 enabled effective combination with PD-1 blockade. Fully human DF6002, similarly, demonstrated an extended half-life and a protracted IFNγ profile in non-human primates. CONCLUSION: An optimized IL-12-Fc fusion protein increased the therapeutic window of IL-12, enhancing anti-tumor activity without concomitantly increasing toxicity. FUNDING: This research was funded by Dragonfly Therapeutics.

Predictive model for preoperative risk calculation of cerebrospinal fluid leak after resection of midline craniofacial mass lesions.

Background: Complex anterior skull base defects produced by resection of mass lesions vary in size and configuration and may be extensive. We analyzed the largest single-center series of midline craniofacial lesions extending intra- and extracranially. The study aims at the development of a predictive model for preoperative measurement of the risk of the postoperative cerebrospinal fluid (CSF) leak based on patients' characteristics and surgical plans. Methods: 166 male and 149 female patients with mean age 40,5 years (1 year and - 81 years) operated for benign and tumor-like midline craniofacial mass lesions were retrospectively analyzed using logistic regression method (Ridge regression algorithm was selected). The overall CSF leak rate was 9.6%. The ROSE algorithm and 'glmnet' software suite in R were used to overcome the cohort's disbalance and avoid overtraining the model. Results: The most influential modifiable negative predictor of the postoperative CSF leak was the use of extracranial and combined approaches. Use of transbasal approaches, gross total resection, utilization of one or two vascularized flaps for skull base reconstruction were the foremost modifiable predictors of a good outcome. Criterium of elevated risk was established at 50% with a specificity of the model as high as 0.83. Conclusions: The performed study has allowed for identifying the most significant predictors of postoperative CSF leak and developing an effective formula to estimate the risk of this complication using data known for each patient. We believe that the suggested web-based online calculator can be helpful for decision making support in off-pattern clinical situations.

Multinomial Classification of Neurosurgical Operations Using Gradient Boosting and Deep Learning Algorithms.

This study aimed at testing the feasibility of neurosurgical procedures classification into 100+ classes using natural language processing and machine learning. A catboost algorithm and bidirectional recurrent neural network with a gated recurrent unit showed almost the same accuracy of ∼81%, with suggestions of correct class in top 2-3 scored classes up to 98.9%. The classification of neurosurgical procedures via machine learning appears to be a technically solvable task which can be additionally improved considering data enhancement and classes verification.

Artificial Intelligence: On the Way to Doctor's Trust.

The submission is devoted to reflections on the role of trust to modern IT systems, especially based on the AI technologies. Its purpose is to draw the attention of the medical informatics community to the need to achieve trust at all stages of the life cycle of MDSS and other information systems.

Predicting the Length of Stay in Neurosurgery with RuGPT-3 Language Model.

In this study, we update the evaluation of the Russian GPT3 model presented in our previous paper in predicting the length of stay (LOS) in neurosurgery. We aimed to assess the performance the Russian GPT-3 (ruGPT-3) language model in LOS prediction using narrative medical records in neurosurgery compared to doctors' and patients' expectations. Doctors appeared to have the most realistic LOS expectations (MAE = 2.54), while the model's predictions (MAE = 3.53) were closest to the patients' (MAE = 3.47) but inferior to them (p = 0.011). A detailed analysis showed a solid quality of ruGPT-3 performance based on narrative clinical texts. Considering our previous findings obtained with recurrent neural networks and FastText vector representation, we estimate the new result as important but probably improveable.

Noninvasive Glioma Grading with Deep Learning: A Pilot Study.

Gliomas are the most common neuroepithelial brain tumors, different by various biological tissue types and prognosis. They could be graded with four levels according to the 2007 WHO classification. The emergence of non-invasive histological and molecular diagnostics for nervous system neoplasms can revolutionize the efficacy and safety of medical care and radically reduce healthcare costs. Our pilot study aimed to evaluate the diagnostic accuracy of deep learning (DL) in subtyping gliomas by WHO grades (I-IV) based on preoperative magnetic resonance imaging (MRI) from Burdenko Neurosurgery Center's database. A total of 707 MRI studies was included. A "3D classification" approach predicting tumor type for the entire patient's MRI data showed the best result (accuracy = 83%, ROC AUC = 0.95), consistent with that of other authors who used different methodologies. Our preliminary results proved the separability of MR T1 axial images with contrast enhancement by WHO grade using DL.

A Comparison of Word Embeddings to Study Complications in Neurosurgery.

Our study aimed to compare the capability of different word embeddings to capture the semantic similarity of clinical concepts related to complications in neurosurgery at the level of medical experts. Eighty-four sets of word embeddings (based on Word2vec, GloVe, FastText, PMI, and BERT algorithms) were benchmarked in a clustering task. FastText model showed the best close to the medical expertise capability to group medical terms by their meaning (adjusted Rand index = 0.682). Word embedding models can accurately reflect clinical concepts' semantic and linguistic similarities, promising their robust usage in medical domain-specific NLP tasks.

Semiautomatic Identification of Pulmonary Embolism in Electronic Health Records Through Sentence Labeling.

In this study, we tested the quality of the information extraction algorithm proposed by our group to detect pulmonary embolism (PE) in medical cases through sentence labeling. Having shown a comparable result (F1 = 0.921) to the best machine learning method (random forest, F1 = 0.937), our approach proved not to miss the information of interest. Scoping the number of texts under review down to distinct sentences and introducing labeling rules contributes to the efficiency and quality of information extraction by experts and makes the challenging tasks of labeling large textual datasets solvable.

Length of Stay Prediction in Neurosurgery with Russian GPT-3 Language Model Compared to Human Expectations.

Patients, relatives, doctors, and healthcare providers anticipate the evidence-based length of stay (LOS) prediction in neurosurgery. This study aimed to assess the quality of LOS prediction with the GPT3 language model upon the narrative medical records in neurosurgery comparing to doctors' and patients' expectations. We found no significant difference (p = 0.109) between doctors', patients', and model's predictions with neurosurgeons tending to be more accurate in prognosis. The modern neural network language models demonstrate feasibility in LOS prediction.

The incidence and outcomes of healthcare-associated respiratory tract infections in non-ventilated neurocritical care patients: Results of a 10-year cohort study.

The incidence of healthcare-associated respiratory tract infections in non-ventilated patients (NVA-HARTI) in neurosurgical intensive care units (ICUs) is unknown. The impact of NVA-HARTI on patient outcomes and differences between NVA-HARTI and ventilator-associated healthcare-associated respiratory tract infections (VA-HARTI) are poorly understood. Our objectives were to report the incidence, hospital length of stay (LOS), ICU LOS, and mortality in NVA-HARTI patients and compare these characteristics to VA-HARTI in neurocritical care patients. This cohort study was conducted in a neurosurgical ICU in Moscow. From 2011 to 2020, all patients with an ICU LOS > 48 h were included. A competing risk model was used for survival and risk analysis. A total of 3,937 ICU admissions were analyzed. NVA-HARTI vs VA-HARTI results were as follows: cumulative incidence 7.2 (95%CI: 6.4-8.0) vs 15.4 (95%CI: 14.2-16.5) per 100 ICU admissions; incidence rate 4.2 ± 2.0 vs 9.5 ± 3.0 per 1000 patient-days in the ICU; median LOS 32 [Q1Q3: 21, 48.5] vs 46 [Q1Q3: 28, 76.5] days; median ICU LOS 15 [Q1Q3: 10, 28.75] vs 26 [Q1Q3: 17, 43] days; mortality 12.3% (95%CI: 7.9-16.8) vs 16.7% (95%CI: 13.6-19.7). The incidence of VA-HARTI decreased over ten years while NVA-HARTI incidence did not change. VA-HARTI was an independent risk factor of death, OR 1.54 (1.11-2.14), while NVA-HARTI was not. Our findings suggest that NVA-HARTI in neurocritical care patients represents a significant healthcare burden with relatively high incidence and associated poor outcomes. Unlike VA-HARTI, the incidence of NVA-HARTI remained constant despite preventive measures. This suggests that extrapolating VA-HARTI research findings to NVA-HARTI should be avoided.

Best Research Practice Implementation: The Experience of the N.N. Burdenko National Medical Research Center of Neurosurgery.

Implementing the best research principles initiates an important shift in clinical research culture, improving efficiency and the level of evidence obtained. In this article, we share our own view on the best research practice and our experience introducing it into the scientific activities of the N.N. Burdenko National Medical Research Center of Neurosurgery (Moscow, Russian Federation). While being adherent to the principles described in the article, the percentage of publications in the international scientific journals in our Center has increased from 7% to 27%, with an overall gain in the number of articles by 2 times since 2014. We believe it is important that medical informatics professionals equally to medical experts involved in clinical research are familiar with the best research principles.

Inter-Rater Reliability of Unstructured Text Labeling: Artificially vs. Naturally Intelligent Approaches.

Unstructured medical text labeling technologies are expected to be highly demanded since the interest in artificial intelligence and natural language processing arises in the medical domain. Our study aimed to assess the agreement between experts who judged on the fact of pulmonary embolism (PE) in neurosurgical cases retrospectively based on electronic health records and assess the utility of the machine learning approach to automate this process. We observed a moderate agreement between 3 independent raters on PE detection (Light's kappa = 0.568, p = 0). Labeling sentences with the method we proposed earlier might improve the machine learning results (accuracy = 0.97, ROC AUC = 0.98) even in those cases that could not be agreed between 3 independent raters. Medical text labeling techniques might be more efficient when strict rules and semi-automated approaches are implemented. Machine learning might be a good option for unstructured text labeling when the reliability of textual data is properly addressed. This project was supported by the RFBR grant 18-29-22085.

Semiautomated Approach for Muscle Weakness Detection in Clinical Texts.

The automated detection of adverse events in medical records might be a cost-effective solution for patient safety management or pharmacovigilance. Our group proposed an information extraction algorithm (IEA) for detecting adverse events in neurosurgery using documents written in a natural rich-in-morphology language. In this paper, we challenge to optimize and evaluate its performance for the detection of any extremity muscle weakness in clinical texts. Our algorithm shows the accuracy of 0.96 and ROC AUC = 0.96 and might be easily implemented in other medical domains.

Classification of Intracranial Hemorrhage Subtypes Using Deep Learning on CT Scans.

Intracranial hemorrhage is a pathological condition that requires fast diagnosis and decision making. Recently, a neural network model for classification of different intracranial hemorrhage types was proposed by a member of our research group Konstantin Kotik as part of the machine learning competition at Kaggle. Our current pilot study aimed to test this model on real-world CT scans from patients with intracranial hemorrhage treated at N.N. Burdenko Neurosurgery Center. The deep learning model for intracranial hemorrhage classification based on ResNexT architecture showed an accuracy of detection greater than 0.81 for every subtype of hemorrhage without any tuning. We expect further improvement in the model performance.

Detection of Muscle Weakness in Medical Texts Using Natural Language Processing.

Identifying adverse events in clinical documents is demanded in retrospective clinical research and prospective monitoring of treatment safety and cost-effectiveness. We proposed and evaluated a few methods of semi-automated muscle weakness detection in preoperative clinical notes for a larger project on predicting paresis by images. The combination of semi-expert and machine learning methods demonstrated maximized sensitivity = 0.860 and specificity = 0.919, and largest AUC = 0.943 with a 95% CI [0.874; 0.991], outperforming each method used individually. Our approaches are expected to be effective for autoshaping a well- verified training dataset for supervised machine learning.

Predicting Postoperative Hospital Stay in Neurosurgery with Recurrent Neural Networks Based on Operative Reports.

This study aimed to predict the duration of the postoperative in-hospital period in neurosurgery based on unstructured operative reports, natural language processing, and deep learning. The recurrent neuronal network (RNN-GRU) was tuned on the word-embedded reports of primary surgical cases retrieved for the period between 2000 and 2017. A new test dataset obtained for the primary operations performed in 2018-2019 was used to evaluate model performance. The mean absolute error of prediction in the final test was 3.00 days. Our study demonstrated the usability of textual EHRs data for the prediction of postoperative period length in neurosurgery using deep learning.

Blockchain as a Process Control Tool for Healthcare.

The blockchain is one of the most popular information technologies and, at the same time, it was discredited by stories about crashes of multiple cryptocurrency projects. Even though this technology has recently found application in many areas not related to cryptocurrencies, mainly for security purposes, the attitude towards it remains wary. Herein we shall try to demonstrate that blockchain is something going far beyond cryptocurrency and security issues, and may become one of the fundamental information technologies in future healthcare.

An Information Extraction Algorithm for Detecting Adverse Events in Neurosurgery Using Documents Written in a Natural Rich-in-Morphology Language.

Rich-in-morphology language, such as Russian, present a challenge for extraction of professional medical information. In this paper, we report on our solution to identify adverse events (complications) in neurosurgery based on natural language processing and professional medical judgment. The algorithm we proposed is easily implemented and feasible in a broad spectrum of clinical studies.