Comparing Precision Machine Learning With Consumer, Quality, and Volume Metrics for Ranking Orthopedic Surgery Hospitals: Retrospective Study.

BACKGROUND: Patients' choices of providers when undergoing elective surgeries significantly impact both perioperative outcomes and costs. There exist a variety of approaches that are available to patients for evaluating between different hospital choices. OBJECTIVE: This paper aims to compare differences in outcomes and costs between hospitals ranked using popular internet-based consumer ratings, quality stars, reputation rankings, average volumes, average outcomes, and precision machine learning-based rankings for hospital settings performing hip replacements in a large metropolitan area. METHODS: Retrospective data from 4192 hip replacement surgeries among Medicare beneficiaries in 2018 in a the Chicago metropolitan area were analyzed for variations in outcomes (90-day postprocedure hospitalizations and emergency department visits) and costs (90-day total cost of care) between hospitals ranked through multiple approaches: internet-based consumer ratings, quality stars, reputation rankings, average yearly surgical volume, average outcome rates, and machine learning-based rankings. The average rates of outcomes and costs were compared between the patients who underwent surgery at a hospital using each ranking approach in unadjusted and propensity-based adjusted comparisons. RESULTS: Only a minority of patients (1159/4192, 27.6% to 2078/4192, 49.6%) were found to be matched to higher-ranked hospitals for each of the different approaches. Of the approaches considered, hip replacements at hospitals that were more highly ranked by consumer ratings, quality stars, and machine learning were all consistently associated with improvements in outcomes and costs in both adjusted and unadjusted analyses. The improvement was greatest across all metrics and analyses for machine learning-based rankings. CONCLUSIONS: There may be a substantive opportunity to increase the number of patients matched to appropriate hospitals across a broad variety of ranking approaches. Elective hip replacement surgeries performed at hospitals where patients were matched based on patient-specific machine learning were associated with better outcomes and lower total costs of care.

Incorporating temporal EHR data in predictive models for risk stratification of renal function deterioration.

Predictive models built using temporal data in electronic health records (EHRs) can potentially play a major role in improving management of chronic diseases. However, these data present a multitude of technical challenges, including irregular sampling of data and varying length of available patient history. In this paper, we describe and evaluate three different approaches that use machine learning to build predictive models using temporal EHR data of a patient. The first approach is a commonly used non-temporal approach that aggregates values of the predictors in the patient's medical history. The other two approaches exploit the temporal dynamics of the data. The two temporal approaches vary in how they model temporal information and handle missing data. Using data from the EHR of Mount Sinai Medical Center, we learned and evaluated the models in the context of predicting loss of estimated glomerular filtration rate (eGFR), the most common assessment of kidney function. Our results show that incorporating temporal information in patient's medical history can lead to better prediction of loss of kidney function. They also demonstrate that exactly how this information is incorporated is important. In particular, our results demonstrate that the relative importance of different predictors varies over time, and that using multi-task learning to account for this is an appropriate way to robustly capture the temporal dynamics in EHR data. Using a case study, we also demonstrate how the multi-task learning based model can yield predictive models with better performance for identifying patients at high risk of short-term loss of kidney function.

Acoustic Voice and Speech Biomarkers of Treatment Status during Hospitalization for Acute Decompensated Heart Failure.

This study investigates acoustic voice and speech features as biomarkers for acute decompensated heart failure (ADHF), a serious escalation of heart failure symptoms including breathlessness and fatigue. ADHF-related systemic fluid accumulation in the lungs and laryngeal tissues is hypothesized to affect phonation and respiration for speech. A set of daily spoken recordings from 52 patients undergoing inpatient ADHF treatment was analyzed to identify voice and speech biomarkers for ADHF and to examine the trajectory of biomarkers during treatment. Results indicated that speakers produce more stable phonation, a more creaky voice, faster speech rates, and longer phrases after ADHF treatment compared to their pre-treatment voices. This project builds on work to develop a method of monitoring ADHF using speech biomarkers and presents a more detailed understanding of relevant voice and speech features.

ECG-guided non-invasive estimation of pulmonary congestion in patients with heart failure.

Quantifying hemodynamic severity in patients with heart failure (HF) is an integral part of clinical care. A key indicator of hemodynamic severity is the mean Pulmonary Capillary Wedge Pressure (mPCWP), which is ideally measured invasively. Accurate non-invasive estimates of the mPCWP in patients with heart failure would help identify individuals at the greatest risk of a HF exacerbation. We developed a deep learning model, HFNet, that uses the 12-lead electrocardiogram (ECG) together with age and sex to identify when the mPCWP > 18 mmHg in patients who have a prior diagnosis of HF. The model was developed using retrospective data from the Massachusetts General Hospital and evaluated on both an internal test set and an independent external validation set, from another institution. We developed an uncertainty score that identifies when model performance is likely to be poor, thereby helping clinicians gauge when to trust a given model prediction. HFNet AUROC for the task of estimating mPCWP > 18 mmHg was 0.8 [Formula: see text] 0.01 and 0.[Formula: see text] 0.01 on the internal and external datasets, respectively. The AUROC on predictions with the highest uncertainty are 0.50 [Formula: see text] 0.02 (internal) and 0.[Formula: see text] 0.04 (external), while the AUROC on predictions with the lowest uncertainty were 0.86 ± 0.01 (internal) and 0.82 ± 0.01 (external). Using estimates of the prevalence of mPCWP > 18 mmHg in patients with reduced ventricular function, and a decision threshold corresponding to an 80% sensitivity, the calculated positive predictive value (PPV) is 0.[Formula: see text] 0.01when the corresponding chest x-ray (CXR) is consistent with interstitial edema HF. When the CXR is not consistent with interstitial edema, the estimated PPV is 0.[Formula: see text] 0.02, again at an 80% sensitivity threshold. HFNet can accurately predict elevated mPCWP in patients with HF using the 12-lead ECG and age/sex. The method also identifies cohorts in which the model is more/less likely to produce accurate outputs.

Prospective evaluation of data-driven models to predict daily risk of Clostridioides difficile infection at 2 large academic health centers.

Many data-driven patient risk stratification models have not been evaluated prospectively. We performed and compared the prospective and retrospective evaluations of 2 Clostridioides difficile infection (CDI) risk-prediction models at 2 large academic health centers, and we discuss the models' robustness to data-set shifts.

Augmenting existing deterioration indices with chest radiographs to predict clinical deterioration.

IMPORTANCE: When hospitals are at capacity, accurate deterioration indices could help identify low-risk patients as potential candidates for home care programs and alleviate hospital strain. To date, many existing deterioration indices are based entirely on structured data from the electronic health record (EHR) and ignore potentially useful information from other sources. OBJECTIVE: To improve the accuracy of existing deterioration indices by incorporating unstructured imaging data from chest radiographs. DESIGN, SETTING, AND PARTICIPANTS: Machine learning models were trained to predict deterioration of patients hospitalized with acute dyspnea using existing deterioration index scores and chest radiographs. Models were trained on hospitalized patients without coronavirus disease 2019 (COVID-19) and then subsequently tested on patients with COVID-19 between January 2020 and December 2020 at a single tertiary care center who had at least one radiograph taken within 48 hours of hospital admission. MAIN OUTCOMES AND MEASURES: Patient deterioration was defined as the need for invasive or non-invasive mechanical ventilation, heated high flow nasal cannula, IV vasopressor administration or in-hospital mortality at any time following admission. The EPIC deterioration index was augmented with unstructured data from chest radiographs to predict risk of deterioration. We compared discriminative performance of the models with and without incorporating chest radiographs using area under the receiver operating curve (AUROC), focusing on comparing the fraction and total patients identified as low risk at different negative predictive values (NPV). RESULTS: Data from 6278 hospitalizations were analyzed, including 5562 hospitalizations without COVID-19 (training cohort) and 716 with COVID-19 (216 in validation, 500 in held-out test cohort). At a NPV of 0.95, the best-performing image-augmented deterioration index identified 49 more (9.8%) individuals as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission. At a NPV of 0.9, the EPIC image-augmented deterioration index identified 26 more individuals (5.2%) as low-risk compared to the deterioration index based on clinical data alone in the first 48 hours of admission. CONCLUSION AND RELEVANCE: Augmenting existing deterioration indices with chest radiographs results in better identification of low-risk patients. The model augmentation strategy could be used in the future to incorporate other forms of unstructured data into existing disease models.

Learning the Effect of Registration Hyperparameters with HyperMorph.

We introduce HyperMorph, a framework that facilitates efficient hyperparameter tuning in learning-based deformable image registration. Classical registration algorithms perform an iterative pair-wise optimization to compute a deformation field that aligns two images. Recent learning-based approaches leverage large image datasets to learn a function that rapidly estimates a deformation for a given image pair. In both strategies, the accuracy of the resulting spatial correspondences is strongly influenced by the choice of certain hyperparameter values. However, an effective hyperparameter search consumes substantial time and human effort as it often involves training multiple models for different fixed hyperparameter values and may lead to suboptimal registration. We propose an amortized hyperparameter learning strategy to alleviate this burden by learning the impact of hyperparameters on deformation fields. We design a meta network, or hypernetwork, that predicts the parameters of a registration network for input hyperparameters, thereby comprising a single model that generates the optimal deformation field corresponding to given hyperparameter values. This strategy enables fast, high-resolution hyperparameter search at test-time, reducing the inefficiency of traditional approaches while increasing flexibility. We also demonstrate additional benefits of HyperMorph, including enhanced robustness to model initialization and the ability to rapidly identify optimal hyperparameter values specific to a dataset, image contrast, task, or even anatomical region, all without the need to retrain models. We make our code publicly available at http://hypermorph.voxelmorph.net.

Predicting outcomes in patients with aortic stenosis using machine learning: the Aortic Stenosis Risk (ASteRisk) score.

OBJECTIVE: To use echocardiographic and clinical features to develop an explainable clinical risk prediction model in patients with aortic stenosis (AS), including those with low-gradient AS (LGAS), using machine learning (ML). METHODS: In 1130 patients with moderate or severe AS, we used bootstrap lasso regression (BLR), an ML method, to identify echocardiographic and clinical features important for predicting the combined outcome of all-cause mortality or aortic valve replacement (AVR) within 5 years after the initial echocardiogram. A separate hold out set, from a different centre (n=540), was used to test the generality of the model. We also evaluated model performance with respect to each outcome separately and in different subgroups, including patients with LGAS. RESULTS: Out of 69 available variables, 26 features were identified as predictive by BLR and expert knowledge was used to further reduce this set to 9 easily available and input features without loss of efficacy. A ridge logistic regression model constructed using these features had an area under the receiver operating characteristic curve (AUC) of 0.74 for the combined outcome of mortality/AVR. The model reliably identified patients at high risk of death in years 2-5 (HRs ≥2.0, upper vs other quartiles, for years 2-5, p<0.05, p=not significant in year 1) and was also predictive in the cohort with LGAS (n=383, HRs≥3.3, p<0.05). The model performed similarly well in the independent hold out set (AUC 0.78, HR ≥2.5 in years 1-5, p<0.05). CONCLUSION: In two separate longitudinal databases, ML identified prognostic features and produced an algorithm that predicts outcome for up to 5 years of follow-up in patients with AS, including patients with LGAS. Our algorithm, the Aortic Stenosis Risk (ASteRisk) score, is available online for public use.

A machine-learning algorithm for detecting seizure termination in scalp EEG.

Efforts to develop algorithms that can robustly detect the cessation of seizure activity within scalp EEGs are now underway. Such algorithms can facilitate novel clinical applications such as the estimation of a seizure's duration; the delivery of therapies designed to mitigate postictal period symptoms; or detection of the presence of status epilepticus. In this article, we present and evaluate a novel, machine learning-based method for detecting the termination of electrographic seizure activity. When tested on 133 seizures from a public database, our method successfully detected the end of 132 seizures within 10.3 ± 5.5 seconds of the time determined by an electroencephalographer to represent the electrographic end of seizure. Furthermore, by pairing our seizure end detector with a previously published seizure onset detector, we could automatically estimate the duration of 85% of test electrographic seizures within a 15-second error margin compared with electroencephalographer determinations. This article is part of a Supplemental Special Issue entitled The Future of Automated Seizure Detection and Prediction.

An algorithm for seizure onset detection using intracranial EEG.

This article addresses the problem of real-time seizure detection from intracranial EEG (IEEG). One difficulty in creating an approach that can be used for many patients is the heterogeneity of seizure IEEG patterns across different patients and even within a patient. In addition, simultaneously maximizing sensitivity and minimizing latency and false detection rates has been challenging as these are competing objectives. Automated machine learning systems provide a mechanism for dealing with these hurdles. Here we present and evaluate an algorithm for real-time seizure onset detection from IEEG using a machine-learning approach that permits a patient-specific solution. We extract temporal and spectral features across all intracranial EEG channels. A pattern recognition component is trained using these feature vectors and tested against unseen continuous data from the same patient. When tested on more than 875 hours of IEEG data from 10 patients, the algorithm detected 97% of 67 test seizures of several types with a median detection delay of 5 seconds and a median false alarm rate of 0.6 false alarms per 24-hour period. The sensitivity was 100% for 8 of 10 patients. These results indicate that a sensitive, specific, and relatively short-latency detection system based on machine learning can be employed for seizure detection from EEG using a full set of intracranial electrodes to individual patients. This article is part of a Supplemental Special Issue entitled The Future of Automated Seizure Detection and Prediction.

Do as AI say: susceptibility in deployment of clinical decision-aids.

Artificial intelligence (AI) models for decision support have been developed for clinical settings such as radiology, but little work evaluates the potential impact of such systems. In this study, physicians received chest X-rays and diagnostic advice, some of which was inaccurate, and were asked to evaluate advice quality and make diagnoses. All advice was generated by human experts, but some was labeled as coming from an AI system. As a group, radiologists rated advice as lower quality when it appeared to come from an AI system; physicians with less task-expertise did not. Diagnostic accuracy was significantly worse when participants received inaccurate advice, regardless of the purported source. This work raises important considerations for how advice, AI and non-AI, should be deployed in clinical environments.

Machine intelligence for early targeted precision management and response to outbreaks of respiratory infections.

OBJECTIVES: To evaluate the utility of machine learning (ML) for the management of Medicare beneficiaries at risk of severe respiratory infections in community and postacute settings by (1) identifying individuals in a community setting at risk of infections resulting in emergent hospitalization and (2) matching individuals in a postacute setting to skilled nursing facilities (SNFs) that are likely to reduce the risk of infections. STUDY DESIGN: Retrospective analysis of claims from 2 million Medicare beneficiaries for 2017-2019. METHODS: In the first analysis, the rate of emergent hospitalization due to respiratory infections was measured among beneficiaries predicted by ML to be at highest risk and compared with the overall average for the population. In the second analysis, the rate of emergent hospitalization due to respiratory infections was compared between beneficiaries who went to an SNF with lower predicted risk of infections using ML and beneficiaries who did not. RESULTS: In the community setting, beneficiaries predicted to be at highest risk had significantly increased rates of emergency department visits (13-fold) and hospitalizations (18-fold) due to respiratory infections. In the postacute setting, beneficiaries who received care at top-recommended SNFs had a relative reduction of 37% for emergent care and 36% for inpatient hospitalization due to respiratory infection. CONCLUSIONS: Precision management through personalized and predictive ML offers the opportunity to reduce the burden of outbreaks of respiratory infections. In the community setting, ML can identify vulnerable subpopulations at highest risk of severe infections. In postacute settings, ML can inform patient choices by matching beneficiaries to SNFs likely to reduce future risk.

Advances in the application of technology to epilepsy: the CIMIT/NIO Epilepsy Innovation Summit.

In 2008, a group of clinicians, scientists, engineers, and industry representatives met to discuss advances in the application of engineering technologies to the diagnosis and treatment of patients with epilepsy. The presentations also provided a guide for further technological development, specifically in the evaluation of patients for epilepsy surgery, seizure onset detection and seizure prediction, intracranial treatment systems, and extracranial treatment systems. This article summarizes the discussions and demonstrates that cross-disciplinary interactions can catalyze collaborations between physicians and engineers to address and solve many of the pressing unmet needs in epilepsy.

Non-invasive computerized system for automatically initiating vagus nerve stimulation following patient-specific detection of seizures or epileptiform discharges.

OBJECTIVE: To demonstrate the feasibility of using a computerized system to detect the onset of a seizure and, in response, initiate Vagus nerve stimulation (VNS) in patients with medically refractory epilepsy. METHODS: We designed and built a non-invasive, computerized system that automatically initiates VNS following the real-time detection of a pre-identified seizure or epileptiform discharge. The system detects these events through patient-specific analysis of the scalp electroencephalogram (EEG) and electrocardiogram (ECG) signals. RESULTS: We evaluated the performance of the system on 5 patients (A-E). For patients A and B the computerized system initiated VNS in response to seizures; for patients C and D the system initiated VNS in response to epileptiform discharges; and for patient E neither seizures nor epileptiform discharges were observed during the evaluation period. During the 81 hour clinical test of the system on patient A, the computerized system detected 5/5 seizures and initiated VNS within 5 seconds of the appearance of ictal discharges in the EEG; VNS did not seem to alter the electrographic or behavioral characteristics of the seizures in this case. During the same testing session the computerized system initiated false stimulations at the rate of 1 false stimulus every 2.5 hours while the subject was at rest and not ambulating. During the 26 hour clinical test of the system on patient B, the computerized system detected 1/1 seizures and initiated VNS within 16 seconds of the appearance of ictal discharges; VNS did not alter the electrographic duration of the seizure but decreased anxiety and increased awareness during the post-seizure recovery phase. During the same testing session the computerized system did not declare any false detections. SIGNIFICANCE: Initiating Vagus nerve stimulation soon after the onset of a seizure may abort or ameliorate seizure symptoms in some patients; unfortunately, a significant number of patients cannot initiate VNS by themselves following the start of a seizure. A system that automatically couples automated detection of seizure onset to initiation of VNS may be helpful for seizure treatment.

Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces.

Classical deformable registration techniques achieve impressive results and offer a rigorous theoretical treatment, but are computationally intensive since they solve an optimization problem for each image pair. Recently, learning-based methods have facilitated fast registration by learning spatial deformation functions. However, these approaches use restricted deformation models, require supervised labels, or do not guarantee a diffeomorphic (topology-preserving) registration. Furthermore, learning-based registration tools have not been derived from a probabilistic framework that can offer uncertainty estimates. In this paper, we build a connection between classical and learning-based methods. We present a probabilistic generative model and derive an unsupervised learning-based inference algorithm that uses insights from classical registration methods and makes use of recent developments in convolutional neural networks (CNNs). We demonstrate our method on a 3D brain registration task for both images and anatomical surfaces, and provide extensive empirical analyses of the algorithm. Our principled approach results in state of the art accuracy and very fast runtimes, while providing diffeomorphic guarantees. Our implementation is available online at http://voxelmorph.csail.mit.edu.

VoxelMorph: A Learning Framework for Deformable Medical Image Registration.

We present VoxelMorph, a fast learning-based framework for deformable, pairwise medical image registration. Traditional registration methods optimize an objective function for each pair of images, which can be time-consuming for large datasets or rich deformation models. In contrast to this approach, and building on recent learning-based methods, we formulate registration as a function that maps an input image pair to a deformation field that aligns these images. We parameterize the function via a convolutional neural network (CNN), and optimize the parameters of the neural network on a set of images. Given a new pair of scans, VoxelMorph rapidly computes a deformation field by directly evaluating the function. In this work, we explore two different training strategies. In the first (unsupervised) setting, we train the model to maximize standard image matching objective functions that are based on the image intensities. In the second setting, we leverage auxiliary segmentations available in the training data. We demonstrate that the unsupervised model's accuracy is comparable to state-of-the-art methods, while operating orders of magnitude faster. We also show that VoxelMorph trained with auxiliary data improves registration accuracy at test time, and evaluate the effect of training set size on registration. Our method promises to speed up medical image analysis and processing pipelines, while facilitating novel directions in learning-based registration and its applications. Our code is freely available at https://github.com/voxelmorph/voxelmorph.

Learning from Few Subjects with Large Amounts of Voice Monitoring Data.

Recently, researchers have started training high complexity machine learning models to clinical tasks, often improving upon previous benchmarks. However, more often than not, these methods require large amounts of supervision to provide good generalization guarantees. When applied to data coming from small cohorts and long monitoring periods these models are prone to overfit to subject-identifying features. Since obtaining large amounts of labels is usually not practical in many scenarios, expert-driven knowledge of the task is a common technique to prevent overfitting. We present a two-step learning approach that is able to generalize under these circumstances when applied to a voice monitoring dataset. Our approach decouples the feature learning stage and performs it in an unsupervised manner, removing the need for laborious feature engineering. We show the effectiveness of our proposed model on two voice monitoring related tasks. We evaluate the extracted features for classifying between patients with vocal fold nodules and controls. We also demonstrate that the features capture pathology relevant information by showing that models trained on them are more accurate predicting vocal use for patients than for controls. Our proposed method is able to generalize to unseen subjects and across learning tasks while matching state-of-the-art results.

SMART--an integrated wireless system for monitoring unattended patients.

Monitoring vital signs and locations of certain classes of ambulatory patients can be useful in overcrowded emergency departments and at disaster scenes, both on-site and during transportation. To be useful, such monitoring needs to be portable and low cost, and have minimal adverse impact on emergency personnel, e.g., by not raising an excessive number of alarms. The SMART (Scalable Medical Alert Response Technology) system integrates wireless patient monitoring (ECG, SpO(2)), geo-positioning, signal processing, targeted alerting, and a wireless interface for caregivers. A prototype implementation of SMART was piloted in the waiting area of an emergency department and evaluated with 145 post-triage patients. System deployment aspects were also evaluated during a small-scale disaster-drill exercise.

Ambulatory assessment of phonotraumatic vocal hyperfunction using glottal airflow measures estimated from neck-surface acceleration.

Phonotraumatic vocal hyperfunction (PVH) is associated with chronic misuse and/or abuse of voice that can result in lesions such as vocal fold nodules. The clinical aerodynamic assessment of vocal function has been recently shown to differentiate between patients with PVH and healthy controls to provide meaningful insight into pathophysiological mechanisms associated with these disorders. However, all current clinical assessment of PVH is incomplete because of its inability to objectively identify the type and extent of detrimental phonatory function that is associated with PVH during daily voice use. The current study sought to address this issue by incorporating, for the first time in a comprehensive ambulatory assessment, glottal airflow parameters estimated from a neck-mounted accelerometer and recorded to a smartphone-based voice monitor. We tested this approach on 48 patients with vocal fold nodules and 48 matched healthy-control subjects who each wore the voice monitor for a week. Seven glottal airflow features were estimated every 50 ms using an impedance-based inverse filtering scheme, and seven high-order summary statistics of each feature were computed every 5 minutes over voiced segments. Based on a univariate hypothesis testing, eight glottal airflow summary statistics were found to be statistically different between patient and healthy-control groups. L1-regularized logistic regression for a supervised classification task yielded a mean (standard deviation) area under the ROC curve of 0.82 (0.25) and an accuracy of 0.83 (0.14). These results outperform the state-of-the-art classification for the same classification task and provide a new avenue to improve the assessment and treatment of hyperfunctional voice disorders.