Predicting health crises from early warning signs in patient medical records.

The COVID-19 global pandemic has caused unprecedented worldwide changes in healthcare delivery. While containment and mitigation approaches have been intensified, the progressive increase in the number of cases has overwhelmed health systems globally, highlighting the need for anticipation and prediction to be the basis of an efficient response system. This study demonstrates the role of population health metrics as early warning signs of future health crises. We retrospectively collected data from the emergency department of a large academic hospital in the northeastern United States from 01/01/2019 to 08/07/2021. A total of 377,694 patient records and 303 features were included for analysis. Departing from a multivariate artificial intelligence (AI) model initially developed to predict the risk of high-flow oxygen therapy or mechanical ventilation requirement during the COVID-19 pandemic, a total of 19 original variables and eight engineered features showing to be most predictive of the outcome were selected for further analysis. The temporal trends of the selected variables before and during the pandemic were characterized to determine their potential roles as early warning signs of future health crises. Temporal analysis of the individual variables included in the high-flow oxygen model showed that at a population level, the respiratory rate, temperature, low oxygen saturation, number of diagnoses during the first encounter, heart rate, BMI, age, sex, and neutrophil percentage demonstrated observable and traceable changes eight weeks before the first COVID-19 public health emergency declaration. Additionally, the engineered rule-based features built from the original variables also exhibited a pre-pandemic surge that preceded the first pandemic wave in spring 2020. Our findings suggest that the changes in routine population health metrics may serve as early warnings of future crises. This justifies the development of patient health surveillance systems, that can continuously monitor population health features, and alarm of new approaching public health crises before they become devastating.

Human knowledge models: Learning applied knowledge from the data.

Artificial intelligence and machine learning have demonstrated remarkable results in science and applied work. However, present AI models, developed to be run on computers but used in human-driven applications, create a visible disconnect between AI forms of processing and human ways of discovering and using knowledge. In this work, we introduce a new concept of "Human Knowledge Models" (HKMs), designed to reproduce human computational abilities. Departing from a vast body of cognitive research, we formalized the definition of HKMs into a new form of machine learning. Then, by training the models with human processing capabilities, we learned human-like knowledge, that humans can not only understand, but also compute, modify, and apply. We used several datasets from different applied fields to demonstrate the advantages of HKMs, including their high predictive power and resistance to noise and overfitting. Our results proved that HKMs can efficiently mine knowledge directly from the data and can compete with complex AI models in explaining the main data patterns. As a result, our study reveals the great potential of HKMs, particularly in the decision-making applications where "black box" models cannot be accepted. Moreover, this improves our understanding of how well human decision-making, modeled by HKMs, can approach the ideal solutions in real-life problems.

Temporal quality degradation in AI models.

As AI models continue to advance into many real-life applications, their ability to maintain reliable quality over time becomes increasingly important. The principal challenge in this task stems from the very nature of current machine learning models, dependent on the data as it was at the time of training. In this study, we present the first analysis of AI "aging": the complex, multifaceted phenomenon of AI model quality degradation as more time passes since the last model training cycle. Using datasets from four different industries (healthcare operations, transportation, finance, and weather) and four standard machine learning models, we identify and describe the main temporal degradation patterns. We also demonstrate the principal differences between temporal model degradation and related concepts that have been explored previously, such as data concept drift and continuous learning. Finally, we indicate potential causes of temporal degradation, and suggest approaches to detecting aging and reducing its impact.

Correction: An alternative to the black box: Strategy learning.

[This corrects the article DOI: 10.1371/journal.pone.0264485.].

An alternative to the black box: Strategy learning.

In virtually any practical field or application, discovering and implementing near-optimal decision strategies is essential for achieving desired outcomes. Workflow planning is one of the most common and important problems of this kind, as sub-optimal decision-making may create bottlenecks and delays that decrease efficiency and increase costs. Recently, machine learning has been used to attack this problem, but unfortunately, most proposed solutions are "black box" algorithms with underlying logic unclear to humans. This makes them hard to implement and impossible to trust, significantly limiting their practical use. In this work, we propose an alternative approach: using machine learning to generate optimal, comprehensible strategies which can be understood and used by humans directly. Through three common decision-making problems found in scheduling, we demonstrate the implementation and feasibility of this approach, as well as its great potential to attain near-optimal results.

COVID-19: Recovery Models for Radiology Departments.

The coronavirus disease 2019 (COVID-19) pandemic has greatly affected demand for imaging services, with marked reductions in demand for elective imaging and image-guided interventional procedures. To guide radiology planning and recovery from this unprecedented impact, three recovery models were developed to predict imaging volume over the course of the COVID-19 pandemic: (1) a long-term volume model with three scenarios based on prior disease outbreaks and other historical analogues, to aid in long-term planning when the pandemic was just beginning; (2) a short-term volume model based on the supply-demand approach, leveraging increasingly available COVID-19 data points to predict examination volume on a week-to-week basis; and (3) a next-wave model to estimate the impact from future COVID-19 surges. The authors present these models as techniques that can be used at any stage in an unpredictable pandemic timeline.

Continuous Learning AI in Radiology: Implementation Principles and Early Applications.

Artificial intelligence (AI) is becoming increasingly present in radiology and health care. This expansion is driven by the principal AI strengths: automation, accuracy, and objectivity. However, as radiology AI matures to become fully integrated into the daily radiology routine, it needs to go beyond replicating static models, toward discovering new knowledge from the data and environments around it. Continuous learning AI presents the next substantial step in this direction and brings a new set of opportunities and challenges. Herein, the authors discuss the main concepts and requirements for implementing continuous AI in radiology and illustrate them with examples from emerging applications.

Imaging Volume Trends and Recovery During the COVID-19 Pandemic: A Comparative Analysis Between a Large Urban Academic Hospital and Its Affiliated Imaging Centers.

RATIONALE AND OBJECTIVES: While affiliated imaging centers play an important role in healthcare systems, little is known of how their operations are impacted by the COVID-19 pandemic. Our goal was to investigate imaging volume trends during the pandemic at our large academic hospital compared to the affiliated imaging centers. MATERIALS AND METHODS: This was a descriptive retrospective study of imaging volume from an academic hospital (main hospital campus) and its affiliated imaging centers from January 1 through May 21, 2020. Imaging volume assessment was separated into prestate of emergency (SOE) period (before SOE in Massachusetts on March 10, 2020), "post-SOE" period (time after "nonessential" services closure on March 24, 2020), and "transition" period (between pre-SOE and post-SOE). RESULTS: Imaging volume began to decrease on March 11, 2020, after hospital policy to delay nonessential studies. The average weekly imaging volume during the post-SOE period declined by 54% at the main hospital campus and 64% at the affiliated imaging centers. The rate of imaging volume recovery was slower for affiliated imaging centers (slope = 6.95 for weekdays) compared to main hospital campus (slope = 7.18 for weekdays). CT, radiography, and ultrasound exhibited the lowest volume loss, with weekly volume decrease of 41%, 49%, and 53%, respectively, at the main hospital campus, and 43%, 61%, and 60%, respectively, at affiliated imaging centers. Mammography had the greatest volume loss of 92% at both the main hospital campus and affiliated imaging centers. CONCLUSION: Affiliated imaging center volume decreased to a greater degree than the main hospital campus and showed a slower rate of recovery. Furthermore, the trend in imaging volume and recovery were temporally related to public health announcements and COVID-19 cases.

Modeling workflows: Identifying the most predictive features in healthcare operational processes.

Limited resources and increased patient flow highlight the importance of optimizing healthcare operational systems to improve patient care. Accurate prediction of exam volumes, workflow surges and, most notably, patient delay and wait times are known to have significant impact on quality of care and patient satisfaction. The main objective of this work was to investigate the choice of different operational features to achieve (1) more accurate and concise process models and (2) more effective interventions. To exclude process modelling bias, data from four different workflows was considered, including a mix of walk-in, scheduled, and hybrid facilities. A total of 84 features were computed, based on previous literature and our independent work, all derivable from a typical Hospital Information System. The features were categorized by five subgroups: congestion, customer, resource, task and time features. Two models were used in the feature selection process: linear regression and random forest. Independent of workflow and the model used for selection, it was determined that congestion feature sets lead to models most predictive for operational processes, with a smaller number of predictors.

Modeling Human Perception of Image Quality.

Humans can determine image quality instantly and intuitively, but the mechanism of human perception of image quality is unknown. The purpose of this work was to identify the most important quantitative metrics responsible for the human perception of digital image quality. Digital images from two different datasets-CT tomography (MedSet) and scenic photographs of trees (TreeSet)-were presented in random pairs to unbiased human viewers. The observers were then asked to select the best-quality image from each image pair. The resulting human-perceived image quality (HPIQ) ranks were obtained from these pairwise comparisons with two different ranking approaches. Using various digital image quality metrics reported in the literature, we built two models to predict the observed HPIQ rankings, and to identify the most important HPIQ predictors. Evaluating the quality of our HPIQ models as the fraction of falsely predicted pairwise comparisons (inverted image pairs), we obtained 70-71% of correct HPIQ predictions for the first, and 73-76%for the second approach. Taking into account that 10-14% of inverted pairs were already present in the original rankings, limitations of the models, and only a few principal HPIQ predictors used, we find this result very satisfactory. We obtained a small set of most significant quantitative image metrics associated with the human perception of image quality. This can be used for automatic image quality ranking, machine learning, and quality-improvement algorithms.

Current Applications and Future Impact of Machine Learning in Radiology.

Recent advances and future perspectives of machine learning techniques offer promising applications in medical imaging. Machine learning has the potential to improve different steps of the radiology workflow including order scheduling and triage, clinical decision support systems, detection and interpretation of findings, postprocessing and dose estimation, examination quality control, and radiology reporting. In this article, the authors review examples of current applications of machine learning and artificial intelligence techniques in diagnostic radiology. In addition, the future impact and natural extension of these techniques in radiology practice are discussed.

Machine Learning for Predicting Patient Wait Times and Appointment Delays.

Being able to accurately predict waiting times and scheduled appointment delays can increase patient satisfaction and enable staff members to more accurately assess and respond to patient flow. In this work, the authors studied the applicability of machine learning models to predict waiting times at a walk-in radiology facility (radiography) and delay times at scheduled radiology facilities (CT, MRI, and ultrasound). In the proposed models, a variety of predictors derived from data available in the radiology information system were used to predict waiting or delay times. Several machine-learning algorithms, such as neural network, random forest, support vector machine, elastic net, multivariate adaptive regression splines, k-th nearest neighbor, gradient boosting machine, bagging, classification and regression tree, and linear regression, were evaluated to find the most accurate method. The elastic net model performed best among the 10 proposed models for predicting waiting times or delay times across all four modalities. The most important predictors were also identified.

Improving Radiology Workflow with Automated Examination Tracking and Alerts.

The modern radiology workflow is a production line where imaging examinations pass in sequence through many steps. In busy clinical environments, even a minor delay in any step can propagate through the system and significantly lengthen the examination process. This is particularly true for the tasks delegated to the human operators, who may be distracted or stressed. We have developed an application to track examinations through a critical part of the workflow, from the image-acquisition scanners to the PACS archive. Our application identifies outliers and actively alerts radiology managers about the need to resolve these problems as soon as they happen. In this study, we investigate how this real-time tracking and alerting affected the speed of examination delivery to the radiologist. We demonstrate that active alerting produced a 3-fold reduction of examination-to-PACS delays. Additionally, we discover an overall improvement in examination-to-PACS delivery, evidence that the tracking and alerts instill a culture where timely processing is essential. By providing supervisors with information about exactly where delays emerge in their workflow and alerting the correct staff to take action, applications like ours create more robust radiology workflow with predictable, timely outcomes.

How Secure Is Your Radiology Department? Mapping Digital Radiology Adoption and Security Worldwide.

OBJECTIVE: Despite the long history of digital radiology, one of its most critical aspects--information security--still remains extremely underdeveloped and poorly standardized. To study the current state of radiology security, we explored the worldwide security of medical image archives. MATERIALS AND METHODS: Using the DICOM data-transmitting standard, we implemented a highly parallel application to scan the entire World Wide Web of networked computers and devices, locating open and unprotected radiology servers. We used only legal and radiology-compliant tools. Our security-probing application initiated a standard DICOM handshake to remote computer or device addresses, and then assessed their security posture on the basis of handshake replies. RESULTS: The scan discovered a total of 2774 unprotected radiology or DICOM servers worldwide. Of those, 719 were fully open to patient data communications. Geolocation was used to analyze and rank our findings according to country utilization. As a result, we built maps and world ranking of clinical security, suggesting that even the most radiology-advanced countries have hospitals with serious security gaps. CONCLUSION: Despite more than two decades of active development and implementation, our radiology data still remains insecure. The results provided should be applied to raise awareness and begin an earnest dialogue toward elimination of the problem. The application we designed and the novel scanning approach we developed can be used to identify security breaches and to eliminate them before they are compromised.

Can We Predict Patient Wait Time?

PURPOSE: The importance of patient wait-time management and predictability can hardly be overestimated: For most hospitals, it is the patient queues that drive and define every bit of clinical workflow. The objective of this work was to study the predictability of patient wait time and identify its most influential predictors. METHODS: To solve this problem, we developed a comprehensive list of 25 wait-related parameters, suggested in earlier work and observed in our own experiments. All parameters were chosen as derivable from a typical Hospital Information System dataset. The parameters were fed into several time-predicting models, and the best parameter subsets, discovered through exhaustive model search, were applied to a large sample of actual patient wait data. RESULTS: We were able to discover the most efficient wait-time prediction factors and models, such as the line-size models introduced in this work. Moreover, these models proved to be equally accurate and computationally efficient. Finally, the selected models were implemented in our patient waiting areas, displaying predicted wait times on the monitors located at the front desks. The limitations of these models are also discussed. CONCLUSIONS: Optimal regression models based on wait-line sizes can provide accurate and efficient predictions for patient wait time.

Digital perfusion phantoms for visual perfusion validation.

OBJECTIVE: Despite the increasingly broad use of perfusion applications, we still have no generally accessible means for their verification: The common sense of perfusion maps and "bona fides" of perfusion software vendors remain the only grounds for acceptance. Thus, perfusion applications are one of a very few clinical tools considerably lacking practical objective hands-on validation. MATERIALS AND METHODS: To solve this problem, we introduce digital perfusion phantoms (DPPs)--numerically simulated DICOM image sequences specifically designed to have known perfusion maps with simple visual patterns. Processing DPP perfusion sequences with any perfusion algorithm or software of choice and comparing the results with the expected DPP patterns provide a robust and straightforward way to control the quality of perfusion analysis, software, and protocols. RESULTS: The deviations from the expected DPP maps, observed in each perfusion software, provided clear visualization of processing differences and possible perfusion implementation errors. CONCLUSION: Perfusion implementation errors, often hidden behind real-data anatomy and noise, become very visible with DPPs. We strongly recommend using DPPs to verify the quality of perfusion applications.

Perfusion linearity and its applications in perfusion algorithm analysis.

Perfusion analysis computes blood flow parameters (blood volume, blood flow, and mean transit time) from the observed flow of a contrast agent passing through the patient's vascular system. Perfusion deconvolution has been widely accepted as the principal numerical tool for perfusion analysis, and is used routinely in clinical applications. The extensive use of perfusion in clinical decision-making makes numerical stability and robustness of perfusion computations vital for accurate diagnostics and patient safety. The main goal of this paper is to propose a novel approach for validating numerical properties of perfusion algorithms. The approach is based on the Perfusion Linearity Property (PLP), which is fundamental to virtually all perfusion data processing. PLP allows one to study perfusion values as weighted averages of the original imaging data. This, in turn, uncovers hidden problems with the existing perfusion techniques, and may be used to suggest more reliable computational approaches and methodology.

Body tumor CT perfusion protocols: optimization of acquisition scan parameters in a rat tumor model.

PURPOSE: To evaluate the effects of total scanning time (TST), interscan delay (ISD), inclusion of image at peak vascular enhancement (IPVE), and selection of the input function vessel on the accuracy of tumor blood flow (BF) calculation with computed tomography (CT) in an animal model. MATERIALS AND METHODS: All animal protocols and experiments were approved by the institutional animal care and use committee prior to study initiation. After injection of 0.2 or 0.4 mL of iodinated contrast material, six rats with mammary adenocarcinoma (three tumors each) were scanned in the axial mode for 5 minutes with 1-second ISD (reference scan), 2.5-mm section thickness, 2.5-mm interval, pitch of 1.3, 120 kV, 240 mA, and 0.5-second rotation time. A total of 126 dynamic data sets were created with commercial software by varying TST and ISD, including or excluding the IPVE, and using the aorta or inferior vena cava (IVC) as the input function. Comparative analyses were used to test for significant differences (t test, Wilcoxon signed rank test). Regression analysis was performed to assess the relationship between attenuation of the input function vessel and BF. RESULTS: No significant difference was observed (P > .05) when TST was as short as 30 seconds (range, 20-23 mL/100 g). In sequences performed with an ISD longer than 8 seconds, BF was significantly elevated (P < .01). Inclusion of the IPVE eliminated this difference (P > .10). Use of the IVC as the input function resulted in significantly higher BF (P < .02), with a correlation between peak attenuation and BF (R(2) = 0.43). CONCLUSION: To reduce radiation dose in tumor perfusion with CT, TST can be reduced without causing significant changes in BF calculation in an animal model. Scanning the aortic reference with peak contrast enhancement reduces variability sufficiently to allow for longer ISDs.

The fast automatic algorithm for correction of MR bias field.

PURPOSE: To develop a method for efficient automatic correction of slow-varying nonuniformity in MR images. MATERIALS AND METHODS: The original MR image is represented by a piecewise constant function, and the bias (nonuniformity) field of an MR image is modeled as multiplicative and slow varying, which permits to approximate it with a low-order polynomial basis in a "log-domain." The basis coefficients are determined by comparing partial derivatives of the modeled bias field with the original image. RESULTS: We tested the resulting algorithm named derivative surface fitting (dsf) on simulated images and phantom and real data. A single iteration was sufficient in most cases to produce a significant improvement to the MR image's visual quality. dsf does not require prior knowledge of intensity distribution and was successfully used on brain and chest images. Due to its design, dsf can be applied to images of any modality that can be approximated as piecewise constant with a multiplicative bias field. CONCLUSION: The resulting algorithm appears to be an efficient method for fast correction of slow varying nonuniformity in MR images.