Development and Establishment of Initial Validity Evidence for a Novel Tool for Assessing Trainee Admission Notes.

BACKGROUND: Documentation is a key component of practice, yet few curricula have been published to teach trainees proper note construction. Additionally, a gold standard for assessing note quality does not exist, and no documentation assessment tools integrate with established competency-based frameworks. OBJECTIVE: To develop and establish initial validity evidence for a novel tool that assesses key components of trainee admission notes and maps to the Accreditation Council for Graduate Medical Education (ACGME) milestone framework. DESIGN: Using an iterative, consensus building process we developed the Admission Note Assessment Tool (ANAT). Pilot testing was performed with both the supervising attending and study team raters not involved in care of the patients. The finalized tool was piloted with attendings from other institutions. PARTICIPANTS: Local experts participated in tool development and pilot testing. Additional attending physicians participated in pilot testing. MAIN MEASURES: Content, response process, and internal structure validity evidence was gathered using Messick's framework. Inter-rater reliability was assessed using percent agreement. KEY RESULTS: The final tool consists of 16 checklist items and two global assessment items. Pilot testing demonstrated rater agreement of 72% to 100% for checklist items and 63% to 70% for global assessment items. Note assessment required an average of 12.3 min (SD 3.7). The study generated validity evidence in the domains of content, response process, and internal structure for use of the tool in rating admission notes. CONCLUSIONS: The ANAT assesses individual components of a note, incorporates billing criteria, targets note "bloat," allows for narrative feedback, and provides global assessments mapped to the ACGME milestone framework. The ANAT can be used to assess admission notes by any attending and at any time after note completion with minimal rater training. The ANAT allows programs to implement routine note assessment for multiple functions with the use of a single tool.

Entrustment and mapping of observable practice activities for resident assessment.

Entrustable Professional Activities (EPAs) and the Next Accreditation System reporting milestones reduce general competencies into smaller evaluable parts. However, some EPAs and reporting milestones may be too broad to use as direct assessment tools. We describe our internal medicine residency curriculum and assessment system, which uses entrustment and mapping of observable practice activities (OPAs) for resident assessment. We created discrete OPAs for each resident rotation and learning experience. In combination, these serve as curricular foundation and tools for assessment. OPA performance is measured via a 5-point entrustment scale, and mapped to milestones and EPAs. Entrustment ratings of OPAs provide an opportunity for immediate structured feedback of specific clinical skills, and mapping OPAs to milestones and EPAs can be used for longitudinal assessment, promotion decisions, and reporting. Direct assessment and demonstration of progressive entrustment of trainee skill over time are important goals for all training programs. Systems that use OPAs mapped to milestones and EPAs provide the opportunity for achieving both, but require validation.

Evaluation of a Novel Multidisciplinary Preoperative Workup Strategy for Geriatric Hip Fractures.

OBJECTIVE: To determine the financial and clinical impact of a standardized, multidisciplinary team for surgical clearance and optimization in geriatric hip fracture patients. DESIGN: Retrospective case series. SETTING: Level-1 trauma center. PATIENTS: One hundred twenty-four geriatric patients (age >65 years old) in the preprotocol group (cohort 1; January 2017-December 2018) and 98 geriatric patients in the postprotocol group (cohort 2; October 2019-January 2021) with operative hip fractures. INTERVENTION: Implementation of a multidisciplinary team protocol consisting of Anesthesiology, Internal Medicine and Orthopedic Surgery departments for the assessment of medical readiness and optimization for surgical intervention in geriatric hip fractures. MAIN OUTCOME MEASURES: Rate of cardiology consultation, need for cardiac workup (echocardiography stress testing, heath catheterization), time to medical readiness (TTMR), time to surgery, case-cancellation rate, length of stay (LOS), and total hospitalization charges. RESULTS: Following implementation of the new protocol, there were significant ( P < 0.001) decreases in TTMR (19 vs. 11 hours), LOS (149 vs. 120 hours), case cancellation rate, and total hospital charges ($84,000 vs. $62,000). There were no significant differences with respect to in-hospital complications or readmission rates/mortality rates at 1 year. CONCLUSIONS: Following implementation of a protocolized, multidisciplinary approach to optimizing geriatric fracture patients, we were able to demonstrate a reduction in unnecessary preoperative testing, TTMR for surgery, case cancellation rate, LOS, and total hospitalization charge-without a concomitant increase in complications or mortality. This study highlights that standardization of the perioperative care for geriatric hip fracture patients can provide effective patient care while also lowering financial and logistical burden in care for these injuries. LEVEL OF EVIDENCE: Therapeutic Level III. See Instructions for Authors for a complete description of levels of evidence.

Improving trainee clinical documentation through a novel curriculum in internal medicine.

BACKGROUND: Clinical documentation is a key component of practice. Trainees rarely receive formal training in documentation or assessment of their documentation. Effective methods of improving documentation remain unknown. OBJECTIVE: The objective of this study was to determine if the implementation of a documentation curriculum led to improvement in admission note quality. DESIGNS: Admission notes written prior to implementation of the curriculum and after the curriculum intervention were assessed. Notes were assessed from two-time frames for both years to account for improvement with time not associated with the intervention. SETTINGS AND PARTICIPANTS: Admission notes written by University of Cincinnati interns were assessed. INTERVENTIONS: The documentation curriculum consisted of educational sessions and routine admission note assessments with feedback. MAIN OUTCOMES AND MEASURES: Admission notes were assessed via the 16 checklist items and two global assessment items of the Admission Note Assessment Tool (ANAT). RESULTS: Six ANAT items showed statistically significant differences. The review of systems item improved with the intervention only (odds ratio: 3.61, p < .001) while the assessment and plan item 1 and global assessment item 2 improved with time only (ß = .08, p = .03 and ß = .25, p = .02, respectively) in univariate models. In univariate models the physical exam item, diagnostic data item 2, and global assessment item 1 showed improvement with both intervention and time, respectively, with additive effects seen in models with both intervention and time. CONCLUSION: Several aspects of documentation can improve with a formal documentation curriculum which includes a routine assessment with feedback, and some aspects of documentation improve with time.

Critical Deficiency Ratings in Milestone Assessment: A Review and Case Study.

PURPOSE: The Accreditation Council for Graduate Medical Education (ACGME) requires programs to report learner progress using specialty-specific milestones. It is unclear how milestones can best identify critical deficiencies (CDs) in trainee performance. Specialties developed milestones independently of one another; not every specialty included CDs within milestones ratings. This study examined the proportion of ACGME milestone sets that include CD ratings, and describes one residency program's experiences using CD ratings in assessment. METHOD: The authors reviewed ACGME milestones for all 99 specialties in November 2015, determining which rating scales contained CDs. The authors also reviewed three years of data (July 2012-June 2015) from the University of Cincinnati Medical Center (UCMC) internal medicine residency assessment system based on observable practice activities mapped to ACGME milestones. Data were analyzed by postgraduate year, assessor type, rotation, academic year, and core competency. The Mantel-Haenszel chi-square test was used to test for changes over time. RESULTS: Specialties demonstrated heterogeneity in accounting for CDs in ACGME milestones, with 22% (22/99) of specialties having no language describing CDs in milestones assessment. Thirty-three percent (63/189) of UCMC internal medicine residents received at least one CD rating, with CDs accounting for 0.18% (668/364,728) of all assessment ratings. The authors identified CDs across multiple core competencies and rotations. CONCLUSIONS: Despite some specialties not accounting for CDs in milestone assessment, UCMC's experience demonstrates that a significant proportion of residents may be rated as having a CD during training. Identification of CDs may allow programs to develop remediation and improvement plans.

Entrusting Observable Practice Activities and Milestones Over the 36 Months of an Internal Medicine Residency.

PURPOSE: Competency-based medical education and milestone reporting have led to increased interest in work-based assessments using entrustment over time as an assessment framework. Little is known about data collected from these assessments during residency. This study describes the results of entrustment of discrete work-based skills over 36 months in the University of Cincinnati internal medicine (IM) residency program. METHOD: Attending physician and peer/allied health assessors provided entrustment ratings of resident performance on work-based observable practice activities (OPAs) mapped to Accreditation Council for Graduate Medicine Education/American Board of Internal Medicine reporting milestones for IM. These data were translated into milestones data and tracked longitudinally. The authors analyzed data from this new entrustment system's first 36 months (July 2012-June 2015). RESULTS: During the 36-month period, assessors made 364,728 milestone assessments (mapped from OPAs) of 189 residents. Residents received an annualized average of 83 assessment encounters, producing means of 3,987 milestone assessments and 4,325 words of narrative assessment. Mean entrustment ratings (range 1-5) from all assessors for all milestones rose from 2.46 for first-month residents to 3.92 for 36th-month residents (r = 0.9252, P < .001). Attending physicians' entrustment ratings were lower than peer/allied health assessors' ratings. Medical knowledge and patient care milestones were rated lower than professionalism and interpersonal and communication skills milestones. CONCLUSIONS: Entrustment of milestones appears to rise progressively over time, with differences by assessor type, competency, milestone, and resident. Further research is needed to elucidate the validity of these data in promotion, remediation, and reporting decisions.

Preferences regarding the computerized delivery of lecture content:a survey of medical students.

Available technology allows for the capture and rebroadcast of lectures via computer-based tools. Such tools have the potential to enhance medical education. Medical schools are beginning to offer such services, but little is known about end-user preferences. We surveyed students at one US medical school to gather their preferences for the availability and use of computer-based lecture presentation. These findings add to the limited literature regarding use of such tools for medical education.

Hierarchical requirement of SWI/SNF in retinoblastoma tumor suppressor-mediated repression of Plk1.

Plk1 (Polo-like kinase 1) is a critical regulator of cell cycle progression that harbors oncogenic activity and exhibits aberrant expression in multiple tumors. However, the mechanism through which Plk1 expression is regulated has not been extensively studied. Here we demonstrate that Plk1 is a target of the retinoblastoma tumor suppressor (RB) pathway. Activation of RB and related pocket proteins p107/p130 mediate attenuation of Plk1. Conversely, RB loss deregulates the control of Plk1 expression. RB pathway activation resulted in the repression of Plk1 promoter activity, and this action was dependent on the SWI/SNF chromatin remodeling complex. Although SWI/SNF subunits are lost during tumorigenesis and cooperate with RB for transcriptional repression, the mechanism through which SWI/SNF impinges on RB action is unresolved. Therefore, we delineated the requirement of SWI/SNF for three critical facets of Plk1 promoter regulation: transcription factor binding, corepressor binding, and histone modification. We find that E2F4 and pocket protein association with the Plk1 promoter is independent of SWI/SNF. However, these analyses revealed that SWI/SNF is required for histone deacetylation of the Plk1 promoter. The importance of SWI/SNF-dependent histone deacetylation of the Plk1 promoter was evident, because blockade of this event restored Plk1 expression in the presence of active RB. In summary, these data demonstrate that Plk1 is a target of the RB pathway. Moreover, these findings demonstrate a hierarchical role for SWI/SNF in the control of promoter activity through histone modification.