Implementation and Analysis of an Institution-Wide EHR-Integrated Handoff Note.

EHR-Integrated Handoff Notes are becoming increasingly prevalent, especially among inpatient clinical service teams composed of physicians in training (resident physicians). We describe the implementation of such a tool at our institution, and then describe the usage of the tool, as well as changes made to its content, structure, and format, years after original implementation. We also describe frequency and temporal distribution of updates made to free text elements of the Handoff Note. At the initial implementation there were three versions available; medical/surgical, pediatric, and ICU. Years after implementation, 57% of clinical service teams continue to utilize the note, including all medical, pediatric, behavioral health, obstetrics/gynecology, and neurology services, as well as most surgical services, that serve as "primary" teams. Several interesting changes were noted to the content and structure of the Handoff Note, namely that more complicated versions were abandoned in favor of simpler versions. The Patient Summary and To Do free text boxes are updated an average of 1.0 and 1.6 times per day. Around 60% of updates to both free text boxes occur between 12 pm - 5:59 pm, likely in preparation for the daytime-nightime team handoff.

Accelerating Surgical Site Infection Abstraction With a Semi-automated Machine-learning Approach.

OBJECTIVE: To demonstrate that a semi-automated approach to health data abstraction provides significant efficiencies and high accuracy. BACKGROUND: Surgical outcome abstraction remains laborious and a barrier to the sustainment of quality improvement registries like ACS-NSQIP. A supervised machine learning algorithm developed for detecting SSi using structured and unstructured electronic health record data was tested to perform semi-automated SSI abstraction. METHODS: A Lasso-penalized logistic regression model with 2011-3 data was trained (baseline performance measured with 10-fold cross-validation). A cutoff probability score from the training data was established, dividing the subsequent evaluation dataset into "negative" and "possible" SSI groups, with manual data abstraction only performed on the "possible" group. We evaluated performance on data from 2014, 2015, and both years. RESULTS: Overall, 6188 patients were in the 2011-3 training dataset and 5132 patients in the 2014-5 evaluation dataset. With use of the semi-automated approach, applying the cut-off score decreased the amount of manual abstraction by >90%, resulting in < 1% false negatives in the "negative" group and a sensitivity of 82%. A blinded review of 10% of the "possible" group, considering only the features selected by the algorithm, resulted in high agreement with the gold standard based on full chart abstraction, pointing towards additional efficiency in the abstraction process by making it possible for abstractors to review limited, salient portions of the chart. CONCLUSION: Semi-automated machine learning-aided SSI abstraction greatly accelerates the abstraction process and achieves very good performance. This could be translated to other post-operative outcomes and reduce cost barriers for wider ACS-NSQIP adoption.

Applying Machine Learning Across Sites: External Validation of a Surgical Site Infection Detection Algorithm.

BACKGROUND: Surgical complications have tremendous consequences and costs. Complication detection is important for quality improvement, but traditional manual chart review is burdensome. Automated mechanisms are needed to make this more efficient. To understand the generalizability of a machine learning algorithm between sites, automated surgical site infection (SSI) detection algorithms developed at one center were tested at another distinct center. STUDY DESIGN: NSQIP patients had electronic health record (EHR) data extracted at one center (University of Minnesota Medical Center, Site A) over a 4-year period for model development and internal validation, and at a second center (University of California San Francisco, Site B) over a subsequent 2-year period for external validation. Models for automated NSQIP SSI detection of superficial, organ space, and total SSI within 30 days postoperatively were validated using area under the curve (AUC) scores and corresponding 95% confidence intervals. RESULTS: For the 8,883 patients (Site A) and 1,473 patients (Site B), AUC scores were not statistically different for any outcome including superficial (external 0.804, internal [0.784, 0.874] AUC); organ/space (external 0.905, internal [0.867, 0.941] AUC); and total (external 0.855, internal [0.854, 0.908] AUC) SSI. False negative rates decreased with increasing case review volume and would be amenable to a strategy in which cases with low predicted probabilities of SSI could be excluded from chart review. CONCLUSIONS: Our findings demonstrated that SSI detection machine learning algorithms developed at 1 site were generalizable to another institution. SSI detection models are practically applicable to accelerate and focus chart review.

Outcomes and influences of rural-focused integrated clerkship programs in general surgery.

BACKGROUND: A shortage of general surgeons is predicted in the future, with particular impact on rural surgery. This is an exploratory analysis on a rural-focused longitudinal integrated clerkship to determine if such clerkships can be used to increase interest and recruitment in rural general surgery. METHODS: An institutional database was reviewed to identify students who became general surgeons after completing a rural-focused longitudinal integrated clerkship. Telephone interviews were conducted on a portion of these surgeons. RESULTS: Fifty-seven students (3.6%) completing the rural-focused longitudinal integrated clerkship became general surgeons. Of those participating in phone interviews, most (90%) decided to become surgeons during their experience while all stated that preclinical years did not influence their specialty decision. CONCLUSIONS: A substantial portion of these surgeons went on to practice in rural communities. Pre-existing rural and primary care-focused education could help to address the future projected shortage of rural general surgeons.

Resident-Physician Preferences for Electronic Handoff Note Content: Implications for Implementation of a System-Wide Electronic Health Record-Integrated Handoff Tool.

Increasing attention is paid to the Handoff Process and EHR- integrated tools to enhance the Handoff Process and aid in creating Handoff Notes are becoming more prevalent. In this study, we attempted to determine the ideal content of the Handoff Note based on the preferences of the resident physicians for whom the tool was being constructed. This commenced with an initial semi-structured interview and culminated in a large-scale survey. Overall, 315 resident physicians completed the survey. Plan of Care, Illness Severity, and Patient Summary were the most important content elements to resident physicians. The importance and trustworthiness of other content elements, as well as their preferred display and level of granularity within the Handoff Note, varied considerably. Subjective assessment by a colleague of a patient's hospital course and plan of care, rather than any single piece of objective data, are preferred as ideal content for Handoff Note composition by resident physicians.

Colorectal Lymphoma: A Contemporary Case Series.

BACKGROUND: Colon and rectal lymphomas are rare and can occur in the context of posttransplant lymphoproliferative disorder. Evidence-based management guidelines are lacking. OBJECTIVE: The purpose of this study was to characterize the presentation, diagnosis, and management of colorectal lymphoma and to identify differences within the transplant population. DESIGN: This was a retrospective review of patients evaluated for colorectal lymphoma between 2000 and 2017. Patients were identified through clinical note queries. SETTINGS: Four hospitals within a single health system were included. PATIENTS: Fifty-two patients (64% men; mean age = 64 y; range, 26-91 y) were identified. No patient had <3 months of follow-up. Eight patients (15%) had posttransplant lymphoproliferative disorder. MAIN OUTCOME MEASURES: Overall survival, recurrence, and complications in treatment pathway were measured. RESULTS: Most common presentations were rectal bleeding (27%), abdominal pain (23%), and diarrhea (23%). The most common location was the cecum (62%). Most frequent histologies were diffuse large B-cell lymphoma (48%) and mantle cell lymphoma (25%). Posttransplant lymphoproliferative disorder occurred in the cecum (n = 4) and rectum (n = 4). Twenty patients (38%) were managed with chemotherapy; 25 patients (48%) underwent primary resection. Mass lesions had a higher risk of urgent surgical resection (35% vs 8%; p = 0.017). Three patients (15%) treated with chemotherapy presented with perforation requiring emergency surgery. Overall survival was 77 months (range, 25-180 mo). Patients with cecal involvement had longer overall survival (96 vs 26 mo; p = 0.038); immunosuppressed patients had shorter survival (16 vs 96 mo; p = 0.006). Survival in patients treated with surgical management versus chemotherapy was similar (67 vs 105 mo; p = 0.62). LIMITATIONS: This was a retrospective chart review, with data limited by the contents of the medical chart. This was a small sample size. CONCLUSIONS: Colorectal lymphoma is rare, with variable treatment approaches. Patients with noncecal involvement and chronic immunosuppression had worse overall survival. Patients with mass lesions, particularly cecal masses, are at higher risk to require urgent intervention, and primary resection should be considered. See Video Abstract at http://links.lww.com/DCR/A929.

Using Implementation Science to Adapt a Training Program to Assist Surgeons with High-Stakes Communication.

OBJECTIVE: Surgeons often conduct difficult conversations with patients near the end of life, yet surgical education provides little formalized communication training. We developed a communication tool, Best Case/Worst Case, and trained surgeons using a one-on-one resource intensive format that was effective but difficult to scale for widespread dissemination. We aimed to generate an implementation package to teach surgeons using fewer resources without sacrificing fidelity. DESIGN, SETTING, AND PARTICIPANTS: We used the Replicating Effectiveness Programs framework to guide our implementation strategy and tested our intervention with 39 surgical residents at 4 institutions from September 2016 to June 2017. The implementation package consisted of: (1) instructional video, (2) checklist to assess competence, (3) learner manual, and (4) instructor manual. We focused on 3 implementation outcomes: feasibility, fidelity, and acceptability to participants. RESULTS: Attendance rates ranged from 16% to 75%. Site leaders had little difficulty identifying suitable instructors; however, resident recruitment proved challenging. Sixty-nine percent of residents completed the post-training assessment and the mean score was 12.8 (range 8-15) using the 15-point checklist. Across sites, 69% strongly agreed that Best Case/Worst Case is better than how they usually approach high-stakes conversations and 100% felt prepared to use the tool after training. Instructors reported that the training provided residents with the necessary skills to perform the fundamental elements of Best Case/Worst Case. CONCLUSIONS: Using implementation science we demonstrated that a resource intensive communication training intervention can be successfully modified for group-learning and wide-scale dissemination. However, we identified barriers to implementation, including challenges with feasibility and programmatic buy-in that inform not only resident education but also communication skills training more broadly.

Creating a Rural Surgery Track and a Review of Rural Surgery Training Programs.

OBJECTIVE: The objective of this study was to present the process of developing a rural surgery training track within an established residency program and review the current rural surgery training programs in the nation. DESIGN: This study reviews current rural surgery training opportunities at Accreditation Council for Graduate Medical Education accredited surgical residencies in the United States and presents the process of creating the University of Minnesota's rural surgery training track. SETTING: This study was performed at the University of Minnesota, in Minneapolis, Minnesota, and at Essentia Health-Saint Mary's Medical Center, in Duluth, Minnesota. PARTICIPANTS: Accredited general surgery residencies were reviewed. The creation of a designated rural surgery training track added an additional rural-designated surgical resident during each postgraduation year and created a required postgraduation year 2 rural surgery rotation for all categorical surgical residents. RESULTS: Two hundred sixty-eight surgery residency programs were reviewed. Twenty-seven had required rural rotations, 10 offered only elective rural rotations, and 4 had dedicated National Resident Matching Program codes for rural training tracks. After review of national rural surgery training opportunities, the University of Minnesota's process of creating a designated rural surgery training program required attention to 5 main components: needs assessment and review of local opportunities, surgery residency review committee approval, funding, surgical education, and clinical/operative education. CONCLUSIONS: Increasing opportunities for surgical residents to train in rural settings may help with recruitment of medical students and retention of surgeons pursuing careers in rural surgery.

Medical Student Perceptions of 24-Hour Call.

OBJECTIVE: To assess the medical student perception and experience of a 24-hour call requirement, and to learn if improvements can be made to improve the 24-hour call requirement. DESIGN: Medical students completing their required surgical clerkship over 1 academic year at our institution were surveyed prior to their clerkship and on the last week of clerkship regarding their perceptions and experience with 24-hour call. SETTING: This study was performed at the University of Minnesota, in Minneapolis, Minnesota, a medical school and tertiary medical center. PARTICIPANTS: Two hundred one medical students were given the option to complete an anonymous survey before and after their required surgical clerkship. RESULTS: Response rate for the preclerkship survey was 70% (n = 140) and 58% (n = 117) for the postclerkship survey. The mean age of respondents was 26 years, and the majority of students were in their third year of medical school. After completing the clerkship, students interested in surgery more often agreed the 24-hour call requirement should remain (51% versus 31%, p = 0.01). Students rotating at a Level I Trauma Center were also more likely to agree the call requirement should remain (59% versus 33%, p = 0.008). Medical students generally had less concerns (mental health, fatigue, mistakes, and grade performance) related to 24-hour call after completion of the clerkship. Concerns about the effect of 24-hour call on study schedule remained high in both pre and postclerkship groups. CONCLUSIONS: Medical students have concerns about the experience prior to the clerkship that diminished by its completion. To improve medical student perceptions and overall experience of 24-hour call, frequency of shifts could be limited and the 24-hour call requirement sites could be shifted to Level I Trauma Centers.

Usability Evaluation of an Unstructured Clinical Document Query Tool for Researchers.

Natural Language Processing - Patient Information Extraction for Researchers (NLP-PIER) was developed for clinical researchers for self-service Natural Language Processing (NLP) queries with clinical notes. This study was to conduct a user-centered analysis with clinical researchers to gain insight into NLP-PIER's usability and to gain an understanding of the needs of clinical researchers when using an application for searching clinical notes. Clinical researcher participants (n=11) completed tasks using the system's two existing search interfaces and completed a set of surveys and an exit interview. Quantitative data including time on task, task completion rate, and survey responses were collected. Interviews were analyzed qualitatively. Survey scores, time on task and task completion proportions varied widely. Qualitative analysis indicated that participants found the system to be useful and usable in specific projects. This study identified several usability challenges and our findings will guide the improvement of NLP-PIER 's interfaces.

Characterizing Functional Health Status of Surgical Patients in Clinical Notes.

Functional health status is an important factor not only for determining overall health, but also for measuring risks of adverse events. Our hypothesis is that important functional status data is contained in clinical notes. We found that several categories of phrases related to functional status including diagnoses, activity and care assessments, physical exam, functional scores, assistive equipment, symptoms, and surgical history were important factors. Use of functional health status level terms from our chart review compared to National Surgical Quality Improvement Program determination had varying sensitivities for correct functional status category identification, with 96% for independent patients, 60% for partially dependent patients, and 44% for totally dependent patients. Inter-rater agreement assessing term relevance to functional health status was high at 91% (Kappa=0.74). Functional status-related terms in clinical notes show potential for use in future methodologies for automated detection of functional health status for quality improvement registries and other clinical assessments.

Acute Kidney Injury and Sepsis.

BACKGROUND: Worldwide, both acute kidney injury (AKI) and sepsis are significant clinical complications, particularly in critical care patients. Sepsis is an important cause of AKI, and AKI is a common complication of sepsis. METHODS: We reviewed the literature, including current practice guidelines, on sepsis-associated AKI. RESULTS: We assessed causes of renal failure, potential mechanisms of sepsis-associated acute kidney injury, current practice guidelines, diagnostic criteria and methods, prevention strategies, treatment options, and outcomes. CONCLUSION: In patients with sepsis-associated AKI, appropriate fluid resuscitation and maintenance of blood pressure are important to prevent further kidney damage. Despite multiple clinical trials, the mechanisms of sepsis-associated AKI and the best treatment options remain unclear.

Supplementing Resident Research Funding Through a Partnership With Local Industry.

OBJECTIVE: To develop a model for the supplementation of resident research funding through a resident-hosted clinical immersion with local industry. DESIGN: Designated research residents hosted multiple groups of engineers and business professionals from local industry in general surgery-focused clinical immersion weeks. The participants in these week-long programs are educated about general surgery and brought to the operating room to observe a variety of surgeries. SETTING: This study was performed at the University of Minnesota, in Minneapolis, Minnesota, at a tertiary medical center. PARTICIPANTS: Ten designated research residents hosted general surgery immersion programs. Fifty-seven engineers and business professionals from 5 different local biomedical firms have participated in this program. RESULTS: General surgery research residents (in collaboration with the University of Minnesota's Institute for Engineering in Medicine) have hosted 9 clinical immersion programs since starting the collaborative in 2015. Immersion participant response to the experiences was very positive. Two full-time resident research positions can be funded annually through participation in this program. CONCLUSIONS: With decreasing funding available for surgical research, particularly resident research, innovative ways to fund resident research are needed. The general surgery clinical immersion program at the University of Minnesota has proven its value as a supplement for resident research funding and may be a sustainable model for the future.

Penetrating Colon Trauma Outcomes in Black and White Males.

INTRODUCTION: Racial disparities have been both published and disputed in trauma patient mortality, outcomes, and rehabilitation. In this study, the objective was to assess racial disparities in patients with penetrating colon trauma. METHODS: The National Trauma Data Bank was searched for males aged ≥14years from 2010 through 2014 who underwent operative intervention for penetrating colon trauma. The primary outcomes for this study were stoma formation and transfer to rehabilitation; secondary outcomes were postoperative morbidity and mortality. Analyses were performed in 2016-2018. RESULTS: There were 7,324 patients identified (4,916 black, 2,408 white). Black and white patients underwent fecal diversion with stoma formation at a similar rate (19.6% vs 18.5%, p=0.28). Black patients were more likely than white patients to be uninsured (self-pay; 37.1% vs 29.9%) and more likely to be injured by firearms (88.3% vs 70.2%, p<0.001), but had a lower overall postoperative morbidity rate (52.6% vs 55.3%, p=0.04). The odds of stoma formation (OR=0.92, 95% CI=0.78, 1.09, p=0.35) and the odds of transfer to rehabilitation (OR=1.03, 95% CI=0.82, 1.30, p=0.78) were similar for black versus white patients. CONCLUSIONS: Black patients experienced similar rates of stoma formation and transfer to rehabilitation as white patients with penetrating colon trauma. Multivariate analysis confirmed expected findings that trauma severity increased the odds of receiving an ostomy and rehabilitation placement. The protocol-based management approach to emergency trauma care potentially decreases the risk for the racial biases that could lead to healthcare disparities. SUPPLEMENT INFORMATION: This article is part of a supplement entitled African American Men's Health: Research, Practice, and Policy Implications, which is sponsored by the National Institutes of Health.

Representation of Social History Factors Across Age Groups: A Topic Analysis of Free-Text Social Documentation.

As individuals age, there is potential for dramatic changes in the social and behavioral determinants that affect health status and outcomes. The importance of these determinants has been increasingly recognized in clinical decision-making. We sought to characterize how social and behavioral health determinants vary in different demographic groups using a previously established schema of 28 social history types through both manual analysis and automated topic analysis of social documentation in the electronic health record across the population of an entire integrated healthcare system. Our manual analysis generated 8,335 annotations over 1,400 documents, representing 24 (86%) social history types. In contrast, automated topic analysis generated 22 (79%) social history types. A comparative evaluation demonstrated both similarities and differences in coverage between the manual and topic analyses. Our findings validate the widespread nature of social and behavioral determinants that affect health status over populations of individuals over their lifespan.

Causal Phenotyping for Susceptibility to Cardiotoxicity from Antineoplastic Breast Cancer Medications.

Cardiotoxicity is a relatively common and particularly important adverse event caused by chemotherapy for breast cancer patients. Typical associative phenotypes, such as risk factors associated with diabetes, can often be detected solely based on the data elements existing in electronic health records; however, causal phenotypes, such as risk factors causing cardiotoxicity, require establishing causation between chemotherapy and determining new heart disease, and cannot be directly observedfrom EHR. We propose three phenotyping algorithms to assess breast cancer patients' susceptibility to cardiotoxicity caused by five first-line antineoplastic drugs: (1) causal phenotype model to predict the patients' risk of cardiotoxicity as the difference between the heart disease risks with exposure and nonexposure to the drugs; (2) regular predictive model; (3) combined predictive model of the above two models. Concordances for three methods were 0.60, 0.62, and 0.68. When considering all exposed patients, concordances were 0.66, 0.58 and 0.65 at 280 days after treatment. The study demonstrates the potential utility of causal phenotyping.

Phenotyping and Visualizing Infusion-Related Reactions for Breast Cancer Patients.

Infusion-related reactions (IRRs) are typical adverse events for breast cancer patients. Detecting IRRs and visualizing their occurance associated with the drug treatment would potentially assist clinicians to improve patient safety and help researchers model IRRs and analyze their risk factors. We developed and evaluated a phenotyping algorithm to detect IRRs for breast cancer patients. We also designed a visualization prototype to render IRR patients' medications, lab tests and vital signs over time. By comparing with the 42 randomly selected doses that are manually labeled by a domain expert, the sensitivity, positive predictive value, specificity, and negative predictive value of the algorithms are 69%, 60%, 79%, and 85%, respectively. Using the algorithm, an incidence of 6.4% of patients and 1.8% of doses for docetaxel, 8.7% and 3.2% for doxorubicin, 10.4% and 1.2% for paclitaxel, 16.1% and 1.1% for trastuzumab were identified retrospectively. The incidences estimated are consistent with related studies.

Characterizing Surgical Site Infection Signals in Clinical Notes.

Surgical site infections (SSIs) are the most common and costly of hospital acquired infections. An important step in reducing SSIs is accurate SSI detection, which enables measurement quality improvement, but currently remains expensive through manual chart review. Building off of previous work for automated and semi-automated SSI detection using expert-derived "strong features" from clinical notes, we hypothesized that additional SSI phrases may be contained in clinical notes. We systematically characterized phrases and expressions associated with SSIs. While 83% of expert-derived original terms overlapped with new terms and modifiers, an additional 362 modifiers associated with both positive and negative SSI signals were identified and 62 new base observations and actions were identified. Clinical note queries with the most common base terms revealed another 49 modifiers. Clinical notes contain a wide variety of expressions describing infections occurring among surgical specialties which may provide value in improving the performance of SSI detection algorithms.

Assessing Provider-Generated Free-Text Quality in EHR-Integrated Handoff Notes.

Handoff notes are increasingly integrated within electronic health record (EHR) systems and often contain data automatically generated from the EHR and free-text narratives. We examined the quality of data entered by providers in the free-text portion of our institutional EHR handoff tool. Overall, 65% of handoff notes contained at least one error (average 1.7 errors per note). Most errors were omissions in information around patient plan/management or assessment/diagnosis rather than entry of false data. Factors associated with increased error rate were increasing hospital day number; weekend note; medical (vs. surgical) service team; and authorship by a medical student, first or fourth year resident physician, or attending physician. Our findings suggest that errors are common in handoff notes, and while these errors are not completely false data, they may provide individuals caring for patients an inaccurate understanding of patient status.

Usability Evaluation of NLP-PIER: A Clinical Document Search Engine for Researchers.

NLP-PIER (Natural Language Processing - Patient Information Extraction for Research) is a self-service platform with a search engine for clinical researchers to perform natural language processing (NLP) queries using clinical notes. We conducted user-centered testing of NLP-PIER's usability to inform future design decisions. Quantitative and qualitative data were analyzed. Our findings will be used to improve the usability of NLP-PIER.