Demonstrating the consequences of learning missingness patterns in early warning systems for preventative health care: A novel simulation and solution.

When using tree-based methods to develop predictive analytics and early warning systems for preventive healthcare, it is important to use an appropriate imputation method to prevent learning the missingness pattern. To demonstrate this, we developed a novel simulation that generated synthetic electronic health record data using a variational autoencoder with a custom loss function, which took into account the high missing rate of electronic health data. We showed that when tree-based methods learn missingness patterns (correlated with adverse events) in electronic health record data, this leads to decreased performance if the system is used in a new setting that has different missingness patterns. Performance is worst in this scenario when the missing rate between those with and without an adverse event is the greatest. We found that randomized and Bayesian regression imputation methods mitigate the issue of learning the missingness pattern for tree-based methods. We used this information to build a novel early warning system for predicting patient deterioration in general wards and telemetry units: PICTURE (Predicting Intensive Care Transfers and other UnfoReseen Events). To develop, tune, and test PICTURE, we used labs and vital signs from electronic health records of adult patients over four years (n = 133,089 encounters). We analyzed primary outcomes of unplanned intensive care unit transfer, emergency vasoactive medication administration, cardiac arrest, and death. We compared PICTURE with existing early warning systems and logistic regression at multiple levels of granularity. When analyzing PICTURE on the testing set using all observations within a hospital encounter (event rate = 3.4%), PICTURE had an area under the receiver operating characteristic curve (AUROC) of 0.83 and an adjusted (event rate = 4%) area under the precision-recall curve (AUPR) of 0.27, while the next best tested method-regularized logistic regression-had an AUROC of 0.80 and an adjusted AUPR of 0.22. To ensure system interpretability, we applied a state-of-the-art prediction explainer that provided a ranked list of features contributing most to the prediction. Though it is currently difficult to compare machine learning-based early warning systems, a rudimentary comparison with published scores demonstrated that PICTURE is on par with state-of-the-art machine learning systems. To facilitate more robust comparisons and development of early warning systems in the future, we have released our variational autoencoder's code and weights so researchers can (a) test their models on data similar to our institution and (b) make their own synthetic datasets.

Data collected by the electronic health record is insufficient for estimating nursing costs: An observational study on acute care inpatient nursing units.

Introduction As the electronic health record becomes more sophisticated, commensurate advances in cost accounting have risen as a top priority for hospital leaders. This study explored: 1) the average time to complete common nursing tasks documented in the electronic health record, 2) nursing-related tasks that remain undocumented, 3) the association between observation data and actual nursing documentation, and 4) considerations for model development and report design to be used for activity based cost accounting in nursing. Methods This was an observational study completed on acute care inpatient nursing units at a large academic medical center. During a five-week period, 63 nurses from 25 units were observed for over 250 h. Results Nearly 60% of the observed nursing activities did not fit into categories readily available in, and easily abstracted from, the electronic health record. The undocumented activities accounted for over half of the observation tasks and equated to nearly 130 h, in which over 40 h were spent on the activity of documentation/charting itself. Furthermore, nearly 36 h were spent on communication, followed by 13.5 h on monitoring/surveillance, two critical tasks in nursing which cannot be overlooked. Conclusions Using the electronic health record for cost accounting in nursing is a novel approach. In addition to the electronic health record, supplementary sources of data must be included to accurately capture nursing work and associated costs. Findings and lessons learned from this study will be used to guide future work and develop a model that determines the cost of nursing care and improved value in hospitalized patients.