Lung Ultrasound Findings in Patients Hospitalized With COVID-19.

OBJECTIVES: Lung ultrasound (LUS) can accurately diagnose several pulmonary diseases, including pneumothorax, effusion, and pneumonia. LUS may be useful in the diagnosis and management of COVID-19. METHODS: This study was conducted at two United States hospitals from 3/21/2020 to 6/01/2020. Our inclusion criteria included hospitalized adults with COVID-19 (based on symptomatology and a confirmatory RT-PCR for SARS-CoV-2) who received a LUS. Providers used a 12-zone LUS scanning protocol. The images were interpreted by the researchers based on a pre-developed consensus document. Patients were stratified by clinical deterioration (defined as either ICU admission, invasive mechanical ventilation, or death within 28 days from the initial symptom onset) and time from symptom onset to their scan. RESULTS: N = 22 patients (N = 36 scans) were included. Eleven (50%) patients experienced clinical deterioration. Among N = 36 scans, only 3 (8%) were classified as normal. The remaining scans demonstrated B-lines (89%), consolidations (56%), pleural thickening (47%), and pleural effusion (11%). Scans from patients with clinical deterioration demonstrated higher percentages of bilateral consolidations (50 versus 15%; P = .033), anterior consolidations (47 versus 11%; P = .047), lateral consolidations (71 versus 29%; P = .030), pleural thickening (69 versus 30%; P = .045), but not B-lines (100 versus 80%; P = .11). Abnormal findings had similar prevalences between scans collected 0-6 days and 14-28 days from symptom onset. DISCUSSION: Certain LUS findings may be common in hospitalized COVID-19 patients, especially for those that experience clinical deterioration. These findings may occur anytime throughout the first 28 days of illness. Future efforts should investigate the predictive utility of these findings on clinical outcomes.

Point-of-Care Ultrasound Predicts Clinical Outcomes in Patients With COVID-19.

OBJECTIVES: Point-of-care ultrasound (POCUS) detects the pulmonary manifestations of COVID-19 and may predict patient outcomes. METHODS: We conducted a prospective cohort study at four hospitals from March 2020 to January 2021 to evaluate lung POCUS and clinical outcomes of COVID-19. Inclusion criteria included adult patients hospitalized for COVID-19 who received lung POCUS with a 12-zone protocol. Each image was interpreted by two reviewers blinded to clinical outcomes. Our primary outcome was the need for intensive care unit (ICU) admission versus no ICU admission. Secondary outcomes included intubation and supplemental oxygen usage. RESULTS: N = 160 patients were included. Among critically ill patients, B-lines (94 vs 76%; P < .01) and consolidations (70 vs 46%; P < .01) were more common. For scans collected within 24 hours of admission (N = 101 patients), early B-lines (odds ratio [OR] 4.41 [95% confidence interval, CI: 1.71-14.30]; P < .01) or consolidations (OR 2.49 [95% CI: 1.35-4.86]; P < .01) were predictive of ICU admission. Early consolidations were associated with oxygen usage after discharge (OR 2.16 [95% CI: 1.01-4.70]; P = .047). Patients with a normal scan within 24 hours of admission were less likely to require ICU admission (OR 0.28 [95% CI: 0.09-0.75]; P < .01) or supplemental oxygen (OR 0.26 [95% CI: 0.11-0.61]; P < .01). Ultrasound findings did not dynamically change over a 28-day scanning window after symptom onset. CONCLUSIONS: Lung POCUS findings detected within 24 hours of admission may provide expedient risk stratification for important COVID-19 clinical outcomes, including future ICU admission or need for supplemental oxygen. Conversely, a normal scan within 24 hours of admission appears protective. POCUS findings appeared stable over a 28-day scanning window, suggesting that these findings, regardless of their timing, may have clinical implications.

Interobserver Agreement of Lung Ultrasound Findings of COVID-19.

BACKGROUND: Lung ultrasound (LUS) has received considerable interest in the clinical evaluation of patients with COVID-19. Previously described LUS manifestations for COVID-19 include B-lines, consolidations, and pleural thickening. The interrater reliability (IRR) of these findings for COVID-19 is unknown. METHODS: This study was conducted between March and June 2020. Nine physicians (hospitalists: n = 4; emergency medicine: n = 5) from 3 medical centers independently evaluated n = 20 LUS scans (n = 180 independent observations) collected from patients with COVID-19, diagnosed via RT-PCR. These studies were randomly selected from an image database consisting of COVID-19 patients evaluated in the emergency department with portable ultrasound devices. Physicians were blinded to any patient information or previous LUS interpretation. Kappa values (κ) were used to calculate IRR. RESULTS: There was substantial IRR on the following items: normal LUS scan (κ = 0.79 [95% CI: 0.72-0.87]), presence of B-lines (κ = 0.79 [95% CI: 0.72-0.87]), ≥3 B-lines observed (κ = 0.72 [95% CI: 0.64-0.79]). Moderate IRR was observed for the presence of any consolidation (κ = 0.57 [95% CI: 0.50-0.64]), subpleural consolidation (κ = 0.49 [95% CI: 0.42-0.56]), and presence of effusion (κ = 0.49 [95% CI: 0.41-0.56]). Fair IRR was observed for pleural thickening (κ = 0.23 [95% CI: 0.15-0.30]). DISCUSSION: Many LUS manifestations for COVID-19 appear to have moderate to substantial IRR across providers from multiple specialties utilizing differing portable devices. The most reliable LUS findings with COVID-19 may include the presence/count of B-lines or determining if a scan is normal. Clinical protocols for LUS with COVID-19 may require additional observers for the confirmation of less reliable findings such as consolidations.

CMS Billing Guidelines and Student Documentation: a New Era or New Burden?

The Centers for Medicare and Medicaid Services (CMS) recently revised their Medicare Claims Processing Manual with the addition of CR 10412, a provision that permits teaching providers to fully bill for medical student notes. This change will have significant implications on the documentation duties of teaching physicians and trainees. Potential benefits of this provision include reduced documentation burden on house officers, improved medical student empowerment, and the infusion of more original content into the electronic medical record. However, these benefits may be offset by shifting the burden of documentation onto medical students, which may compromise their time spent with patients and overall wellness. In this perspective, we review the changes that occurred with CR 10412 and their potential impact on documentation across the medical education spectrum.

Evaluation of Trainee Competency with Point-of-Care Ultrasonography (POCUS): a Conceptual Framework and Review of Existing Assessments.

Point-of-care ultrasonography (POCUS) has the potential to transform healthcare delivery through its diagnostic expediency. Trainee competency with POCUS is now mandated for emergency medicine through the Accreditation Council for Graduate Medical Education (ACGME), and its use is expanding into other medical specialties, including internal medicine. However, a key question remains: how does one define "competency" with this emerging technology? As our trainees become more acquainted with POCUS, it is vital to develop validated methodology for defining and measuring competency amongst inexperienced users. As a framework, the assessment of competency should include evaluations that assess the acquisition and application of POCUS-related knowledge, demonstration of technical skill (e.g., proper probe selection, positioning, and image optimization), and effective integration into routine clinical practice. These assessments can be performed across a variety of settings, including web-based applications, simulators, standardized patients, and real clinical encounters. Several validated assessments regarding POCUS competency have recently been developed, including the Rapid Assessment of Competency in Echocardiography (RACE) or the Assessment of Competency in Thoracic Sonography (ACTS). However, these assessments focus mainly on technical skill and do not expand upon other areas of this framework, which represents a growing need. In this review, we explore the different methodologies for evaluating competency with POCUS as well as discuss current progress in the field of measuring trainee knowledge and technical skill.

Thrombophilia testing in the inpatient setting: impact of an educational intervention.

BACKGROUND: Thrombophilia testing is frequently ordered in the inpatient setting despite its limited impact on clinical decision-making and unreliable results in the setting of acute thrombosis or ongoing anticoagulation. We sought to determine the effect of an educational intervention in reducing inappropriate thrombophilia testing for hospitalized patients. METHODS: During the 2014 academic year, we implemented an educational intervention with a phase implementation design for Internal Medicine interns at Stanford University Hospital. The educational session covering epidemiology, appropriate thrombophilia evaluation and clinical rationale behind these recommendations. Their ordering behavior was compared with a contemporaneous control (non-medicine and private services) and a historical control (interns from prior academic year). From the analyzed data, we determined the proportion of inappropriate thrombophilia testing of each group. Logistic generalized estimating equations were used to estimate odds ratios for inappropriate thrombophilia testing associated with the intervention. RESULTS: Of 2151 orders included, 934 were deemed inappropriate (43.4%). The two intervention groups placed 147 orders. A pooled analysis of ordering practices by intervention groups revealed a trend toward reduction of inappropriate ordering (p = 0.053). By the end of the study, the intervention groups had significantly lower rates of inappropriate testing compared to historical or contemporaneous controls. CONCLUSION: A brief educational intervention was associated with a trend toward reduction in inappropriate thrombophilia testing. These findings suggest that focused education on thrombophilia testing can positively impact inpatient ordering practices.

A high value care curriculum for interns: a description of curricular design, implementation and housestaff feedback.

PURPOSE: Most residency programmes do not have a formal high value care curriculum. Our goal was to design and implement a multidisciplinary high value care curriculum specifically targeted at interns. DESIGN: Our curriculum was designed with multidisciplinary input from attendings, fellows and residents at Stanford. Curricular topics were inspired by the American Board of Internal Medicine's Choosing Wisely campaign, Alliance for Academic Internal Medicine, American College of Physicians and Society of Hospital Medicine. Our topics were as follows: introduction to value-based care; telemetry utilisation; lab ordering; optimal approach to thrombophilia work-ups and fresh frozen plasma use; optimal approach to palliative care referrals; antibiotic stewardship; and optimal approach to imaging for low back pain. Our curriculum was implemented at the Stanford Internal Medicine residency programme over the course of two academic years (2014 and 2015), during which 100 interns participated in our high value care curriculum. After each high value care session, interns were offered the opportunity to complete surveys regarding feedback on the curriculum, self-reported improvements in knowledge, skills and attitudinal module objectives, and quiz-based knowledge assessments. RESULTS: The overall survey response rate was 67.1%. Overall, the material was rated as highly useful on a 5-point Likert scale (mean 4.4, SD 0.6). On average, interns reported a significant improvement in their self-rated knowledge, skills and attitudes after the six seminars (mean improvement 1.6 points, SD 0.4 (95% CI 1.5 to 1.7), p<0.001). CONCLUSIONS: We successfully implemented a novel high value care curriculum that specifically targets intern physicians.

Preparing to take the USMLE Step 1: a survey on medical students' self-reported study habits.

BACKGROUND: The USA Medical Licensing Examination Step 1 is a computerised multiple-choice examination that tests the basic biomedical sciences. It is administered after the second year in a traditional four-year MD programme. Most Step 1 scores fall between 140 and 260, with a mean (SD) of 227 (22). Step 1 scores are an important selection criterion for residency choice. Little is known about which study habits are associated with a higher score. OBJECTIVE: To identify which self-reported study habits correlate with a higher Step 1 score. METHODS: A survey regarding Step 1 study habits was sent to third year medical students at Tulane University School of Medicine every year between 2009 and 2011. The survey was sent approximately 3 months after the examination. RESULTS: 256 out of 475 students (54%) responded. The mean (SD) Step 1 score was 229.5 (22.1). Students who estimated studying more than 8-11 h per day had higher scores (p<0.05), but there was no added benefit with additional study time. Those who reported studying <40 days achieved higher scores (p<0.05). Those who estimated completing >2000 practice questions also obtained higher scores (p<0.01). Students who reported studying in a group, spending the majority of study time on practice questions or taking >40 preparation days did not achieve higher scores. CONCLUSIONS: Certain self-reported study habits may correlate with a higher Step 1 score compared with others. Given the importance of achieving a high Step 1 score on residency choice, it is important to further identify which characteristics may lead to a higher score.

Persistent leukocytosis-is this a persistent problem for patients with acute ischemic stroke?

BACKGROUND: In the setting of acute ischemic stroke (AIS), leukocytosis has been shown to be an indicator of inflammatory response. Although leukocytosis on admission has been shown to correlate with initial stroke severity in AIS patients, no work has been done to assess if there are differences in transient or persistent leukocytosis in patients without infection. The objective of this study is to determine the clinical significance of persistent versus transient leukocytosis during the early phase of AIS. METHODS: Patients who presented with AIS to our center within 48 hours of symptom onset between July 2008 and June 2010 were retrospectively identified by chart review. Patients were included if they had leukocytosis on admission (defined as white blood cell count >11,000/µL based on laboratory reference range values). A logistic regression model was used to evaluate persistent leukocytosis (leukocytosis 48 hours after admission) as a predictor of several outcome measures, including good functional outcome (discharge modified Rankin Scale score of 0-2). Marginal effects were used to estimate the probability of poor functional outcome. RESULTS: Of the 438 patients screened, 49 had leukocytosis on admission and of those 24 (49%) had persistent leukocytosis. NIHSS score correlated significantly with persistence of leukocytosis (r = .306; P = .0044). More people with transient leukocytosis (leukocytosis lasting <48 hours) had a good functional outcome (44% versus 16%; P = .006). After adjusting for baseline NIHSS score, persistent leukocytosis was not a significant independent predictor of good functional outcome, but showed an association (OR, 2.5; 95% CI, .562-10.7; P = .2322). Persistent leukocytosis after adjusting for age and NIHSS score at admission is associated with a poor functional outcome, but it is not statistically significant (OR, 2.43; 95% CI, .59-9.87; P = .2151). After controlling for age and NIHSS score on admission, for patients with persistent leukocytosis, the probability of having poor functional outcome at discharge was increased by 16 percentage points. CONCLUSIONS: Persistent leukocytosis is associated with higher baseline NIHSS scores. Persistent leukocytosis is tightly linked with baseline stroke severity and is associated with poor patient outcomes. Our study found that patients with persistent leukocytosis are more likely to present with severe strokes and maintain a high NIHSS score at 24 hours after admission, unlike patients without leukocytosis or patients with transient leukocytosis. Furthermore, it appears that persistent leukocytosis outside the setting of an infection negatively impacts the short-term functional outcome of AIS patients. Identifying patients with persistent leukocytosis could help to prognosticate and target patients that may benefit from future anti-inflammatory interventions.

Identification of modifiable and nonmodifiable risk factors for neurologic deterioration after acute ischemic stroke.

BACKGROUND: Neurologic deterioration (ND) after ischemic stroke has been shown to impact short-term functional outcome and is associated with in-hospital mortality. METHODS: Patients with acute ischemic stroke who presented between July 2008 and December 2010 were identified and excluded for in-hospital stroke, presentation >48 hours since last seen normal, or unknown time of last seen normal. Clinical and laboratory data, National Institutes of Health Stroke Scale (NIHSS) scores, and episodes of ND (increase in NIHSS score ≥ 2 within a 24-hour period) were investigated. RESULTS: Of the 596 patients screened, 366 were included (median age 65 years; 42.1% female; 65.3% black). Of these, 35.0% experienced ND. Patients with ND were older (69 v 62 years; P < .0001), had more severe strokes (median admission NIHSS score 12 v 5; P < .0001), carotid artery stenosis (27.0% v 16.8%; P = .0275), and coronary artery disease (26.0% v 16.4%; P = .0282) compared to patients without ND. Patients with ND had higher serum glucose on admission than patients without ND (125.5 v 114 mg/dL; P = .0036). After adjusting for crude variables associated with ND, age >65 years, and baseline NIHSS score >14 remained significant independent predictors of ND. In a logistic regression analysis adjusting for age and serum glucose, each 1-point increase in admission NIHSS score was associated with a 7% increase in the odds of ND (odds ratio 1.07; 95% confidence interval 1.04-1.10; P < .0001). CONCLUSIONS: Older patients and patients with more severe strokes are more likely to experience ND. Initial stroke severity was the only significant, independent, and modifiable risk factor for ND, amenable to recanalization and reperfusion.

What change in the National Institutes of Health Stroke Scale should define neurologic deterioration in acute ischemic stroke?

BACKGROUND: Neurologic deterioration (ND) occurs in one-third of patients with stroke. However, the true incidence of ND and risk for adverse outcomes remains unknown because no standardized definition of ND exists. Our study compared the prognostic value of a range of definitions for ND in patients with acute ischemic stroke (AIS). METHODS: All patients who presented to our center with AIS within 48 hours of symptom onset between July 2008 and June 2010 were retrospectively identified. Patient demographics, National Institutes of Health Stroke Scale (NIHSS) scores, etiologies of ND, and outcome measures were compared between patients according to a range of ND definitions using receiver operating characteristic analyses. RESULTS: Three hundred forty-seven patients were included. The 2 definitions of ND with the highest sensitivity and specificity for several outcome measures were tested against each other: an increase in the NIHSS score by ≥2 or ≥4 points in a 24-hour period. More than one third (36.9%) of patients experienced ≥2-point ND versus 17.3% with ≥4-point ND. Patients who experienced ND by either definition had prolonged hospitalization (P < .001), poorer functional outcome (discharge modified Rankin Scale score >2; P < .001), and higher discharge NIHSS score (P < .001) compared to patients without ND. Compared to patients without ND, a ≥2-point ND was associated with a 3-fold risk of death (odds ratio 3.120; 95% confidence interval 1.231-7.905; P < .0165) after adjusting for admission NIHSS score, serum glucose, and age. CONCLUSIONS: A ≥2-point ND is a sensitive indicator of poor outcome and in-hospital mortality. An accepted definition of ND is needed to systematically study and compare results across trials for ND in patients with stroke.

Leukocytosis in patients with neurologic deterioration after acute ischemic stroke is associated with poor outcomes.

BACKGROUND: Neurologic deterioration (ND) after acute ischemic stroke (AIS) has been shown to result in poor outcomes. ND is thought to arise from penumbral excitotoxic cell death caused in part by leukocytic infiltration. Elevated admission peripheral leukocyte levels are associated with poor outcomes in stroke patients who suffer ND, but little is known about the dynamic changes that occur in leukocyte counts around the time of ND. We sought to determine if peripheral leukocyte levels in the days surrounding ND are correlated with poor outcomes. METHODS: Patients with AIS who presented to our center within 48 hours of symptom onset between July 2008 and June 2010 were retrospectively identified by chart review and screened for ND (defined as an increase in National Institutes of Health Stroke Scale score ≥ 2 within a 24-hour period). Patients were excluded for steroid use during hospitalization or in the month before admission and infection within the 48 hours before or after ND. Demographics, daily leukocyte counts, and poor functional outcome (modified Rankin Scale score 3-6) were investigated. RESULTS: Ninety-six of the 292 (33%) patients screened had ND. The mean age was 69.5 years; 62.5% were male and 65.6% were black. Patients with a poor functional outcome had significantly higher leukocyte and neutrophil levels 1 day before ND (P = .048 and P = .026, respectively), and on the day of ND (P = .013 and P = .007, respectively), compared to patients with good functional outcome. CONCLUSIONS: Leukocytosis at the time of ND correlates with poor functional outcomes and may represent a marker of greater cerebral damage through increased parenchymal inflammation.

Infections present on admission compared with hospital-acquired infections in acute ischemic stroke patients.

BACKGROUND: To date, few studies have assessed the influence of infections present on admission (POA) compared with hospital-acquired infections (HAIs) on neurologic deterioration (ND) and other outcome measures in acute ischemic stroke (AIS). METHODS: Patients admitted with AIS to our stroke center (July 2010 to December 2010) were retrospectively assessed. The following infections were assessed: urinary tract infection, pneumonia, and bacteremia. Additional chart review was performed to determine whether the infection was POA or HAI. We assessed the relationship between infections in ischemic stroke patients and several outcome measures including ND and poor functional outcome. A mediation analysis was performed to assess the indirect effects of HAI, ND, and poor functional outcome. RESULTS: Of the 334 patients included in this study, 77 had any type of infection (23 POA). After adjusting for age, National Institutes of Health Stroke Scale at baseline, glucose on admission, and intravenous tissue plasminogen activator, HAI remained a significant predictor of ND (odds ratio [OR]=8.8, 95% confidence interval [CI]: 4.2-18.7, P<.0001) and poor functional outcome (OR=41.7, 95% CI: 5.2-337.9, P=.005), whereas infections POA were no longer associated with ND or poor functional outcome. In an adjusted analysis, we found that 57% of the effect from HAI infections on poor functional outcome is because of mediation through ND (P<.0001). CONCLUSIONS: Our data suggests that HAI in AIS patients increases the odds of experiencing ND and subsequently increases the odds of being discharged with significant disability. This mediated effect suggests a preventable cause of ND that can thereby decrease the odds of poor functional outcomes after an AIS.

Performance of ChatGPT on free-response, clinical reasoning exams.

Importance: Studies show that ChatGPT, a general purpose large language model chatbot, could pass the multiple-choice US Medical Licensing Exams, but the model's performance on open-ended clinical reasoning is unknown. Objective: To determine if ChatGPT is capable of consistently meeting the passing threshold on free-response, case-based clinical reasoning assessments. Design: Fourteen multi-part cases were selected from clinical reasoning exams administered to pre-clerkship medical students between 2019 and 2022. For each case, the questions were run through ChatGPT twice and responses were recorded. Two clinician educators independently graded each run according to a standardized grading rubric. To further assess the degree of variation in ChatGPT's performance, we repeated the analysis on a single high-complexity case 20 times. Setting: A single US medical school. Participants: ChatGPT. Main Outcomes and Measures: Passing rate of ChatGPT's scored responses and the range in model performance across multiple run throughs of a single case. Results: 12 out of the 28 ChatGPT exam responses achieved a passing score (43%) with a mean score of 69% (95% CI: 65% to 73%) compared to the established passing threshold of 70%. When given the same case 20 separate times, ChatGPT's performance on that case varied with scores ranging from 56% to 81%. Conclusions and Relevance: ChatGPT's ability to achieve a passing performance in nearly half of the cases analyzed demonstrates the need to revise clinical reasoning assessments and incorporate artificial intelligence (AI)-related topics into medical curricula and practice.

Handheld Ultrasound Device Usage and Image Acquisition Ability Among Internal Medicine Trainees: A Randomized Trial.

BACKGROUND: There is insufficient knowledge about how personal access to handheld ultrasound devices (HUDs) improves trainee learning with point-of-care ultrasound (POCUS). OBJECTIVE: To assess whether HUDs, alongside a yearlong lecture series, improved trainee POCUS usage and ability to acquire images. METHODS: Internal medicine intern physicians (n = 47) at a single institution from 2017 to 2018 were randomized 1:1 to receive personal HUDs (n = 24) for patient care/self-directed learning vs no-HUDs (n = 23). All interns received a repeated lecture series on cardiac, thoracic, and abdominal POCUS. Main outcome measures included self-reported HUD usage rates and post-intervention assessment scores using the Rapid Assessment of Competency in Echocardiography (RACE) scale between HUD and no-HUD groups. RESULTS: HUD interns reported performing POCUS assessments on patients a mean 6.8 (SD 2.2) times per week vs 6.4 (SD 2.9) times per week in non-HUD arm (P = .66). There was no relationship between the number of self-reported examinations per week and a trainee's post-intervention RACE score (rho = 0.022, P = .95). HUD interns did not have significantly higher post-intervention RACE scores (median HUD score 17.0 vs no-HUD score 17.8; P = .72). Trainee confidence with cardiac POCUS did not correlate with RACE scores. CONCLUSIONS: Personal HUDs without direct supervision did not increase the amount of POCUS usage or improve interns' acquisition abilities. Interns who reported performing more examinations per week did not have higher RACE scores. Improved HUD access and lectures without additional feedback may not improve POCUS mastery.

Portable Ultrasound Device Usage and Learning Outcomes Among Internal Medicine Trainees: A Parallel-Group Randomized Trial.

BACKGROUND: Little is known about how to effectively train residents with point-of-care ultrasonography (POCUS) despite increasing usage. OBJECTIVE: This study aimed to assess whether handheld ultrasound devices (HUDs), alongside a year-long lecture series, improved trainee image interpretation skills with POCUS. METHODS: Internal medicine intern physicians (N = 149) at a single academic institution from 2016 to 2018 participated in the study. The 2017 interns (n = 47) were randomized 1:1 to receive personal HUDs (n = 24) for patient care vs no-HUDs (n = 23). All 2017 interns received a repeated lecture series regarding cardiac, thoracic, and abdominal POCUS. Interns were assessed on their ability to interpret POCUS images of normal/abnormal findings. The primary outcome was the difference in end-of-the-year assessment scores between interns randomized to receive HUDs vs not. Secondary outcomes included trainee scores after repeating lectures and confidence with POCUS. Intern scores were also compared with historical (2016, N = 50) and contemporaneous (2018, N = 52) controls who received no lectures. RESULTS: Interns randomized to HUDs did not have significantly higher image interpretation scores (median HUD score: 0.84 vs no-HUD score: 0.84; P = .86). However, HUD interns felt more confident in their abilities. The 2017 cohort had higher scores (median 0.84), compared with the 2016 historical control (median 0.71; P = .001) and 2018 contemporaneous control (median 0.48; P < .001). Assessment scores improved after first-time exposure to the lecture series, while repeated lectures did not improve scores. CONCLUSIONS: Despite feeling more confident, personalized HUDs did not improve interns' POCUS-related knowledge or interpretive ability. Repeated lecture exposure without further opportunities for deliberate practice may not be beneficial for mastering POCUS.

Physician Usage and Acceptance of a Machine Learning Recommender System for Simulated Clinical Order Entry.

Clinical decision support tools that automatically disseminate patterns of clinical orders have the potential to improve patient care by reducing errors of omission and streamlining physician workflows. However, it is unknown if physicians will accept such tools or how their behavior will be affected. In this randomized controlled study, we exposed 34 licensed physicians to a clinical order entry interface and five simulated emergency cases, with randomized availability of a previously developed clinical order recommender system. With the recommender available, physicians spent similar time per case (6.7 minutes), but placed more total orders (17.1 vs. 15.8). The recommender demonstrated superior recall (59% vs 41%) and precision (25% vs 17%) compared to manual search results, and was positively received by physicians recognizing workflow benefits. Further studies must assess the potential clinical impact towards a future where electronic health records automatically anticipate clinical needs.

OrderRex clinical user testing: a randomized trial of recommender system decision support on simulated cases.

OBJECTIVE: To assess usability and usefulness of a machine learning-based order recommender system applied to simulated clinical cases. MATERIALS AND METHODS: 43 physicians entered orders for 5 simulated clinical cases using a clinical order entry interface with or without access to a previously developed automated order recommender system. Cases were randomly allocated to the recommender system in a 3:2 ratio. A panel of clinicians scored whether the orders placed were clinically appropriate. Our primary outcome included the difference in clinical appropriateness scores. Secondary outcomes included total number of orders, case time, and survey responses. RESULTS: Clinical appropriateness scores per order were comparable for cases randomized to the order recommender system (mean difference -0.11 order per score, 95% CI: [-0.41, 0.20]). Physicians using the recommender placed more orders (median 16 vs 15 orders, incidence rate ratio 1.09, 95%CI: [1.01-1.17]). Case times were comparable with the recommender system. Order suggestions generated from the recommender system were more likely to match physician needs than standard manual search options. Physicians used recommender suggestions in 98% of available cases. Approximately 95% of participants agreed the system would be useful for their workflows. DISCUSSION: User testing with a simulated electronic medical record interface can assess the value of machine learning and clinical decision support tools for clinician usability and acceptance before live deployments. CONCLUSIONS: Clinicians can use and accept machine learned clinical order recommendations integrated into an electronic order entry interface in a simulated setting. The clinical appropriateness of orders entered was comparable even when supported by automated recommendations.

Timing and Predictors of Subspecialty Career Choice Among Internal Medicine Residents: A Retrospective Cohort Study.

BACKGROUND: Internal medicine residents face numerous career options after residency training. Little is known about when residents make their final career choice. OBJECTIVE: We assessed the timing and predictive factors of final career choices among internal medicine residents at graduation, including demographics, pre-residency career preferences, and rotation scheduling. METHODS: We conducted a retrospective study of graduates of an academic internal medicine residency program from 2014 to 2017. Main measures included demographics, rotation schedules, and self-reported career choices for residents at 5 time points: recruitment day, immediately after Match Day, end of postgraduate year 1 (PGY-1), end of PGY-2, and at graduation. RESULTS: Of the 138 residents eligible for the study, 5 were excluded based on participation in a fast-track program for an Accreditation Council for Graduate Medical Education subspecialty fellowship. Among the remaining 133 residents, 48 (36%) pursued general internal medicine fields and 78 (59%) pursued fellowship training. Career choices from recruitment day, Match Day, and PGY-1 were only weakly predictive of the career choice. Many choices demonstrated low concordance throughout training, and general medicine fields (primary care, hospital medicine) were frequently not decided until after PGY-2. Early clinical exposure to subspecialty rotations did not predict final career choice. CONCLUSIONS: Early career choices before and during residency training may have low predictability toward final career choices upon graduation in internal medicine. These choices may continue to have low predictability beyond PGY-2 for many specialties. Early clinical exposure may not predict final career choice for subspecialties.