RESUMO
BACKGROUND: Administrative costs for billing and insurance-related activities in the United States are substantial. One critical cause of the high overhead of administrative costs is medical billing errors. With advanced deep learning techniques, developing advanced models to predict hospital and professional billing codes has become feasible. These models can be used for administrative cost reduction and billing process improvements. OBJECTIVE: In this study, we aim to develop an automated anesthesiology current procedural terminology (CPT) prediction system that translates manually entered surgical procedure text into standard forms using neural machine translation (NMT) techniques. The standard forms are calculated using similarity scores to predict the most appropriate CPT codes. Although this system aims to enhance medical billing coding accuracy to reduce administrative costs, we compare its performance with that of previously developed machine learning algorithms. METHODS: We collected and analyzed all operative procedures performed at Michigan Medicine between January 2017 and June 2019 (2.5 years). The first 2 years of data were used to train and validate the existing models and compare the results from the NMT-based model. Data from 2019 (6-month follow-up period) were then used to measure the accuracy of the CPT code prediction. Three experimental settings were designed with different data types to evaluate the models. Experiment 1 used the surgical procedure text entered manually in the electronic health record. Experiment 2 used preprocessing of the procedure text. Experiment 3 used preprocessing of the combined procedure text and preoperative diagnoses. The NMT-based model was compared with the support vector machine (SVM) and long short-term memory (LSTM) models. RESULTS: The NMT model yielded the highest top-1 accuracy in experiments 1 and 2 at 81.64% and 81.71% compared with the SVM model (81.19% and 81.27%, respectively) and the LSTM model (80.96% and 81.07%, respectively). The SVM model yielded the highest top-1 accuracy of 84.30% in experiment 3, followed by the LSTM model (83.70%) and the NMT model (82.80%). In experiment 3, the addition of preoperative diagnoses showed 3.7%, 3.2%, and 1.3% increases in the SVM, LSTM, and NMT models in top-1 accuracy over those in experiment 2, respectively. For top-3 accuracy, the SVM, LSTM, and NMT models achieved 95.64%, 95.72%, and 95.60% for experiment 1, 95.75%, 95.67%, and 95.69% for experiment 2, and 95.88%, 95.93%, and 95.06% for experiment 3, respectively. CONCLUSIONS: This study demonstrates the feasibility of creating an automated anesthesiology CPT classification system based on NMT techniques using surgical procedure text and preoperative diagnosis. Our results show that the performance of the NMT-based CPT prediction system is equivalent to that of the SVM and LSTM prediction models. Importantly, we found that including preoperative diagnoses improved the accuracy of using the procedure text alone.
RESUMO
BACKGROUND: Heart failure with reduced ejection fraction (HFrEF) is a condition imposing significant health care burden. Given its syndromic nature and often insidious onset, the diagnosis may not be made until clinical manifestations prompt further evaluation. Detecting HFrEF in precursor stages could allow for early initiation of treatments to modify disease progression. Granular data collected during the perioperative period may represent an underutilized method for improving the diagnosis of HFrEF. We hypothesized that patients ultimately diagnosed with HFrEF following surgery can be identified via machine-learning approaches using pre- and intraoperative data. METHODS: Perioperative data were reviewed from adult patients undergoing general anesthesia for major surgical procedures at an academic quaternary care center between 2010 and 2016. Patients with known HFrEF, heart failure with preserved ejection fraction, preoperative critical illness, or undergoing cardiac, cardiology, or electrophysiologic procedures were excluded. Patients were classified as healthy controls or undiagnosed HFrEF. Undiagnosed HFrEF was defined as lacking a HFrEF diagnosis preoperatively but establishing a diagnosis within 730 days postoperatively. Undiagnosed HFrEF patients were adjudicated by expert clinician review, excluding cases for which HFrEF was secondary to a perioperative triggering event, or any event not associated with HFrEF natural disease progression. Machine-learning models, including L1 regularized logistic regression, random forest, and extreme gradient boosting were developed to detect undiagnosed HFrEF, using perioperative data including 628 preoperative and 1195 intraoperative features. Training/validation and test datasets were used with parameter tuning. Test set model performance was evaluated using area under the receiver operating characteristic curve (AUROC), positive predictive value, and other standard metrics. RESULTS: Among 67,697 cases analyzed, 279 (0.41%) patients had undiagnosed HFrEF. The AUROC for the logistic regression model was 0.869 (95% confidence interval, 0.829-0.911), 0.872 (0.836-0.909) for the random forest model, and 0.873 (0.833-0.913) for the extreme gradient boosting model. The corresponding positive predictive values were 1.69% (1.06%-2.32%), 1.42% (0.85%-1.98%), and 1.78% (1.15%-2.40%), respectively. CONCLUSIONS: Machine-learning models leveraging perioperative data can detect undiagnosed HFrEF with good performance. However, the low prevalence of the disease results in a low positive predictive value, and for clinically meaningful sensitivity thresholds to be actionable, confirmatory testing with high specificity (eg, echocardiography or cardiac biomarkers) would be required following model detection. Future studies are necessary to externally validate algorithm performance at additional centers and explore the feasibility of embedding algorithms into the perioperative electronic health record for clinician use in real time.
