Validity evidence for the clinical communication skills assessment tool (CCSAT) from 9 years of implementation in a high stakes medical student OSCE.

OBJECTIVES: Communication and other clinical skills are routinely assessed in medical schools using Objective Structured Clinical Examinations (OSCEs) so routinely that it can be difficult to monitor and maintain validity. We report on the accumulation of validity evidence for the Clinical Communication Skills Assessment Tool (CCSAT) based on its use with 9 cohorts of medical students in a high stakes OSCE. METHODS: We describe the implementation of the CCSAT including information on the underlying model, the tool's items, domains, scales and scoring, and its role in curriculum. Internal structure is explored through item, internal consistency, and confirmatory factor analyses. Evidence for CCSAT validity is synthesized within prevailing frameworks (Messick12 and Kane13) based on continuous quality improvement and use of the CCSAT for feedback, remediation, curricular design, and research. RESULTS: Implementation of the CCSAT over time has facilitated our communication skills curriculum and training. Thoughtful case development and investment in standardized patient training has contributed to data quality. Item analysis supports our behaviorally anchored scale (not done, partly and well done) and the skills domains suggested by an a priori evidence-based clinical communication model were confirmed via analysis of actual student data. Evidence synthesized across the frameworks suggests consistent validity of the CCSAT for generalization inferences (that it captures the construct), responsiveness (sensitivity to change/difference), content validity/internal structure, relationships to other variables, and consequences/implications. More evidence is needed to strengthen validity of CCSAT scores for understanding extrapolation inferences and real-world implications. CONCLUSIONS AND PRACTICE IMPLICATIONS: This pragmatic approach to evaluating validity within a program of assessment serves as a model for medical schools seeking to continuously monitor the quality of clinical skill assessments, a need made particularly relevant since the US NBME no longer requires the Step 2 Clinical Skills exam, leaving individual schools with the responsibility for ensuring graduates have acquired the requisite core clinical skills. We document strong evidence for CCSAT validity over time and across cohorts as well as areas for improvement and further examination.

Addressing social determinants of health in primary care: a quasi-experimental study using unannounced standardised patients to evaluate the impact of audit/feedback on physicians' rates of identifying and responding to social needs.

BACKGROUND: Although efforts are underway to address social determinants of health (SDOH), little is known about physicians' SDOH practices despite evidence that failing to fully elicit and respond to social needs can compromise patient safety and undermine both the quality and effectiveness of treatment. In particular, interventions designed to enhance response to social needs have not been assessed using actual practice behaviour. In this study, we evaluate the degree to which providing primary care physicians with feedback on their SDOH practice behaviours is associated with increased rates of eliciting and responding to housing and social isolation needs. METHODS: Unannounced standardised patients (USPs), actors trained to consistently portray clinical scenarios, were sent, incognito, to all five primary care teams in an urban, safety-net healthcare system. Scenarios involved common primary care conditions and each included an underlying housing (eg, mould in the apartment, crowding) and social isolation issue and USPs assessed whether the physician fully elicited these needs and if so, whether or not they addressed them. The intervention consisted of providing physicians with audit/feedback reports of their SDOH practices, along with brief written educational material. A prepost comparison group design was used to evaluate the intervention; four teams received the intervention and one team served as a 'proxy' comparison (no intervention). Preintervention (February 2017 to December 2017) rates of screening for and response to the scripted housing and social needs were compared with intervention period (January 2018 to March 2019) rates for both intervention and comparison teams. RESULTS: 108 visits were completed preintervention and 183 during the intervention period. Overall, social needs were not elicited half of the time and fully addressed even less frequently. Rates of identifying the housing issue increased for teams that received audit/feedback reports (46%-60%; p=0.045) and declined for the proxy comparison (61%-42%; p=0.174). Rates of responding to housing needs increased significantly for intervention teams (15%-41%; p=0.004) but not for the comparison team (21%-29%; p=0.663). Social isolation was identified more frequently postintervention (53%) compared with baseline (39%; p=0.041) among the intervention teams but remained unchanged for the comparison team (39% vs 32%; p=0.601). Full exploration of social isolation remained low for both intervention and comparison teams. CONCLUSIONS: Results suggest that physicians may not be consistently screening for or responding to social needs but that receiving feedback on those practices, along with brief targeted education, can improve rates of SDOH screening and response.

Gasping for air: measuring patient education and activation skillsets in two clinical assessment contexts.

Objective structured clinical examinations (OSCEs) provide a controlled, simulated setting for competency assessments, while unannounced simulated patients (USPs) measure competency in situ or real-world settings. This exploratory study describes differences in primary care residents' skills when caring for the same simulated patient case in OSCEs versus in a USP encounter. Data reported describe a group of residents (n=20) who were assessed following interaction with the same simulated patient case in two distinct settings: an OSCE and a USP visit at our safety-net clinic from 2009 to 2010. In both scenarios, the simulated patient presented as an asthmatic woman with limited understanding of illness management. Residents were rated through a behaviourally anchored checklist on visit completion. Summary scores (mean % well done) were calculated by domain and compared using paired sample t-tests. Residents performed significantly better with USPs on 7 of 10 items and in two of three aggregate assessment domains (p<0.05). OSCE structure may impede assessment of activation and treatment planning skills, which are better assessed in real-world settings. This exploration of outcomes from our two assessments using the same clinical case lays a foundation for future research on variation in situated performance. Using both assessments during residency will provide a more thorough understanding of learner competency.

Clinical problem solving and social determinants of health: a descriptive study using unannounced standardized patients to directly observe how resident physicians respond to social determinants of health.

Objectives While the need to address patients' social determinants of health (SDoH) is widely recognized, less is known about physicians' actual clinical problem-solving when it comes to SDoH. Do physicians include SDoH in their assessment strategy? Are SDoH incorporated into their diagnostic thinking and if so, do they document as part of their clinical reasoning? And do physicians directly address SDoH in their "solution" (treatment plan)? Methods We used Unannounced Standardized Patients (USPs) to assess internal medicine residents' clinical problem solving in response to a patient with asthma exacerbation and concern that her moldy apartment is contributing to symptoms - a case designed to represent a clear and direct link between a social determinant and patient health. Residents' clinical practices were assessed through a post-visit checklist and systematic chart review. Patterns of clinical problem solving were identified and then explored, in depth, through review of USP comments and history of present illness (HPI) and treatment plan documentation. Results Residents fell into three groups when it came to clinical problem-solving around a housing trigger for asthma: those who failed to ask about housing and therefore did not uncover mold as a potential trigger (neglectors ­ 21%; 14/68); those who asked about housing in negative ways that prevented disclosure and response (negative elicitors ­ 23%, 16/68); and those who elicited and explored the mold issue (full elicitors ­ 56%; 38/68) [corrected]. Of the full elicitors 53% took no further action, 26% only documented the mold; and 21% provided resources/referral. In-depth review of USP comments/explanations and residents' notes (HPI, treatment plan) revealed possible influences on clinical problem solving. Failure to ask about housing was associated with both contextual factors (rushed visit) and interpersonal skills (not fully engaging with patient) and with possible differences in attention ("known" vs. unknown/new triggers, usual symptoms vs. changes, not attending to relocation, etc.,). Use of close-ended questions often made it difficult for the patient to share mold concerns. Negative responses to sharing of housing information led to missing mold entirely or to the patient not realizing that the physician agreed with her concerns about mold. Residents who fully elicited the mold situation but did not take action seemed to either lack knowledge or feel that action on SDoH was outside their realm of responsibility. Those that took direct action to help the patient address mold appeared to be motivated by an enhanced sense of urgency. Conclusions Findings provide unique insight into residents' problem solving processes including external influences (e.g., time, distractions), the role of core communication and interpersonal skills (eliciting information, creating opportunities for patients to voice concerns, sharing clinical thinking with patients), how traditional cognitive biases operate in practice (premature closure, tunneling, and ascertainment bias), and the ways in which beliefs about expectancies and scope of practice may color clinical problem-solving strategies for addressing SDoH.

Evolution of an Accelerated 3-Year Pathway to the MD Degree: The Experience of New York University Grossman School of Medicine.

The revision of the curriculum at New York University Grossman School of Medicine in 2010, with a reduction of the preclerkship curriculum to 18 months, made it possible to offer an accelerated 3-year pathway in 2013 for students who know their career path. The goals of the program include individualizing education, reducing student debt, and integrating undergraduate and graduate medical education. This accelerated 3-year doctor of medicine (3YMD) pathway is the first program of its kind in the United States to offer conditional acceptance to residency programs in all specialties through the National Resident Matching Program. Since inception of the pathway 6 years ago, 81 students have graduated. Critical components to successfully launch and implement the program are described.Unwavering commitment to the program as a high institutional priority by the dean and vice dean for education facilitated the support required by department chairs and residency program directors and the flexibility needed for success. Alignment between the 3- and 4-year pathways has made it possible to add points of entry into the 3-year pathway during the second and third years and to shift back into the 4-year pathway, as warranted. Modifications to how 3YMD students are mentored included changing the role of the departmental advisor and adding a dedicated 3YMD pathway advisor who serves as an advocate for both the students and the program. Having a relatively large number of 3YMD students has contributed to the success of the program and facilitated acceptance by the residencies.

Disruptive behavior in the workplace: Challenges for gastroenterology fellows.

AIM: To assess first-year gastroenterology fellows' ability to address difficult interpersonal situations in the workplace using objective structured clinical examinations (OSCE). METHODS: Two OSCEs ("distracted care team" and "frazzled intern") were created to assess response to disruptive behavior. In case 1, a fellow used a colonoscopy simulator while interacting with a standardized patient (SP), nurse, and attending physician all played by actors. The nurse and attending were instructed to display specific disruptive behavior and disregard the fellow unless requested to stop the disruptive behavior and focus on the patient and procedure. In case 2, the fellow was to calm an intern managing a patient with massive gastrointestinal bleeding. The objective in both scenarios was to assess the fellows' ability to perform their duties while managing the disruptive behavior displayed by the actor. The SPs used checklists to rate fellows' performances. The fellows completed a self-assessment survey. RESULTS: Twelve fellows from four gastrointestinal fellowship training programs participated in the OSCE. In the "distracted care team" case, one-third of the fellows interrupted the conflict and refocused attention to the patient. Half of the fellows were able to display professionalism despite the heated discussion nearby. Fellows scored lowest in the interprofessionalism portion of post-OSCE surveys, measuring their ability to handle the conflict. In the "frazzled intern" case, 68% of fellows were able to establish a calm and professional relationship with the SP. Despite this success, only half of the fellows were successfully communicate a plan to the SP and only a third scored "well done" in a domain that focused on allowing the intern to think through the case with the fellow's guidance. CONCLUSION: Fellows must receive training on how to approach disruptive behavior. OSCEs are a tool that can assess fellow skills and set a culture for open discussion.

Opioid vs nonopioid prescribers: Variations in care for a standardized acute back pain case.

BACKGROUND: Opioid analgesics are effective and appropriate therapy for many types of acute pain. Epidemiologic evidence supports a direct relationship between increased opioid prescribing and increases in opioid use disorders and overdoses. OBJECTIVE: To tailor our residency curriculum, we designed and fielded an unannounced standardized patient (USP) case involving a patient with acute back pain who is requesting Vicodin (5/325 mg). We describe residents' case management and examine whether their management decisions, including opioid prescribing, were related to their core clinical skills. METHODS: Results are based on 50 (USP) visits with residents in 2 urban primary care clinics. Highly trained USPs portrayed a patient with acute lower back pain who was taking leftover Vicodin with effective pain relief but was running out. We describe how residents managed this case, using both USP report and chart review data, and compare summary clinical skills scores between those who prescribed Vicodin and those who did not. RESULTS: Of the 50 residents, 18 prescribed Vicodin (10-60 pills). Among those who did not prescribe (32/50), most (50%) prescribed ibuprofen. Eighty-three percent of the prescribers and 72% of nonprescribers ordered physical therapy (nonsignificant). Of the 18 prescribers, 13 documented checking the prescription monitoring database. Prescribers had significantly better communication scores than nonprescribers (relationship development: 80% vs. 58% well done, P = .029; patient education: 59% vs. 31% well done, P = .018). Assessment summary scores were also higher (60% vs. 46%) but not significantly (P = .060). Patient satisfaction and activation scores were higher in the prescribers than nonprescribers (71% vs. 39%, P = .004 and 48% vs. 26%, P = .034, respectively). CONCLUSIONS: Most Vicodin prescribers did not follow prescribing guidelines, and they demonstrated better communication and assessment skills than the nonprescribers. Results suggest the need to guide residents in using a systematic approach to prescribing opioids safely and to develop an acceptable alternative pain management plan when they decide against prescribing.

Assessment of Abilities of Gastroenterology Fellows to Provide Information to Patients With Liver Disease.

BACKGROUND & AIMS: Patient education is critical in ensuring patient compliance and good health outcomes. Fellows must be able to effectively communicate with their patients, delivering enough information for the patient to understand their medical problem and maximize patient compliance. We created an objective structured clinical examination (OSCE) with 4 liver disease cases to assess fellows' knowledge and ability to inform standardized patients (SPs) about their clinical condition. METHODS: We developed 4 cases highlighting different aspects of liver disease and created a 4-station OSCE: hepatitis B, acute hepatitis C, new diagnosis of cirrhosis, and an end-stage cirrhotic nontransplant candidate. The SP with hepatitis B was minimizing the fact that she could not read English. The acute hepatitis C SP was a nursing student who is afraid that having hepatitis C might jeopardize her career. The SP with the new diagnosis of alcoholic cirrhosis needed to stop drinking, and the end-stage liver disease patient had to grapple with his advanced directives. Twelve fellows from 4 GI training programs participated. Our focus was to assess the fellows' knowledge about liver diseases and the Accreditation Council for Graduate Medical Education competencies of health literacy, shared decision making, advanced directives, and goals of care. The goal for the fellows was to communicate effectively with the SPs, and acknowledge that each patient had an emotionally charged issue to overcome. The SPs used a checklist to rate fellows' performance. Faculty and the SPs observed the cases and provided feedback. The fellows were surveyed on their performance regarding the case. RESULTS: The majority of fellows were able to successfully summarize findings and discuss a plan with the patient in the new diagnosis of cirrhosis (76.92%) and hepatitis C case (100%), but were less successful in the hepatitis B case (30.77%) and the end-of-life case (41.67%). Overall, a small percentage of fellows reflected that they did a good job (22%-33%), except at the end-of-life case (67%). The fellows' greatest challenge was trying to cover a lot of information in a single outpatient visit. CONCLUSIONS: Caring for patients with liver diseases can be complex and time consuming. The patients and fellows' observations were discordant in several areas: for example, the fellows believed they excelled in the end-of-life case, but the SP thought only a small percentage of fellows were able to successfully summarize and discuss the plan. This discrepancy and others highlight important areas of focus in training programs. OSCEs are important to help the fellows facilitate striking the right balance of information delivery and empathy, and this will lead to better patient education, compliance, rapport, and satisfaction.

Assessment of Developmental Progress Using an Objective Structured Clinical Examination-Simulation Hybrid Examination for Obstetrics and Gynecology Residents.

OBJECTIVE: The Test of Integrated Professional Skills (TIPS) is an objective structured clinical examination-simulation hybrid examination that assesses resident integration of technical, cognitive, and affective skills in Obstetrics and Gynecology (OBGYN) residents. The aim of this study was to analyze performance patterns and reactions of residents to the test to understand how it may fit within a comprehensive assessment program. DESIGN: A retrospective, mixed methods review of the design and implementation of the examination, patterns of performance of trainees at different levels of training, focus group data, and description of use of TIPS results for resident remediation and curriculum development. SETTING: OBGYN residents at New York University Langone Medical Center, a tertiary-care, urban academic health center. PARTICIPANTS: OBGYN residents in all years of training, postgraduate year-1 through postgraduate year, all residents completing the TIPS examination and consenting to participate in focus groups were included. RESULTS: In all, 24 residents completed the TIPS examination. Performance on the examination varied widely among individuals at each stage of training, and did not follow developmental trends, except for technical skills. Cronbach α for both standardized patient and faculty ratings ranged from 0.69 to 0.84, suggesting internal consistency. Focus group results indicated that residents respond to the TIPS examination in complex ways, ranging from anxiety about performance to mixed feelings about how to use the data for their learning. CONCLUSION: TIPS assesses a range of attributes, and can support both formative and summative evaluation. Lack of clear developmental differences and wide variation in performance by learners at the same level of training support the argument for individualized learning plans and competency-based education.

Assessing cultural competency skills in gastroenterology fellowship training.

AIM: To assess and teach cultural competency skills at the fellowship training level through the use of objective structured clinical examinations (OSCEs). METHODS: We revised four scenarios to infuse a specific focus on cross-cultural care, and to render them appropriate for gastroenterology fellows. Three are discussed here: (1) Poor Health Literacy; (2) Disclosing/Apologizing for a Complication to a Patient Who Mistrusts the Healthcare System; and (3) Breaking Bad News to a Fatalistic Patient. A fourth case emphasizing shared decision-making will be described elsewhere. Four stations were completed by fellows and observed live by four faculty members, and the fellows' performance was assessed. RESULTS: Eleven fellows from four programs participated in the four OSCE. In the "Poor Health Literacy" case, 18% (2/11) of participants recognized that the standardized patient (SP) had below-basic health literacy. None successfully evaluated the SP's reading skills in a culturally-sensitive manner. In "Disclosing/Apologizing for a Complication", 4/11 (36%) personally apologized for the complication. 1/11 recognized the SP's mistrust of the medical system. With "Breaking Bad News", 27% (3/11) explored the patient's values to identify her fatalistic beliefs. CONCLUSION: OSCEs can be used to assess deficiencies in culturally-competent care at the fellowship level. OSCEs also afford fellowships the opportunity to inform future training curricula.

Self-assessment and goal-setting is associated with an improvement in interviewing skills.

PURPOSE: Describe the relationship between medical students' self-assessment and goal-setting (SAGS) skills and development of interviewing skills during the first-year doctoring course. METHOD: 157 first-year medical students completed three two-case standardized patient (SP) interviews. After each of the first two, students viewed videotapes of their interview, completed a SAGS worksheet, and reviewed a selected tape segment in a seminar. SAGS was categorized into good and poor quality and interviewing skills were rated by trained raters. RESULTS: SAGS improved over time (37% good week 1 vs. 61% good week 10). Baseline SAGS and interviewing skills were not associated. Initial SAGS quality was associated with change in interviewing skills - those with poor-quality SAGS demonstrated a decrease and those with good-quality SAGS demonstrated an increase in scores by 17 weeks (ANOVA F=4.16, p=0.024). For students whose SAGS skills were good at both week 1 and 10, interviewing skills declined in weeks 1-10 and then increased significantly at week 17. For those whose SAGS remained 'poor' in weeks 1-10, interviewing skills declined in weeks 10-17. CONCLUSIONS: In general, the quality of students' SAGS improved over time. Poor baseline SAGS skills and failure to improve were associated with a decrease in interviewing skills at 17 weeks. For students with better SAGS, interviewing skills increased at week 17. Improvement in SAGS skills was not associated with improved interviewing skills. Understanding structured self-assessment skills helps identify student characteristics that influence progressive mastery of communication skills and therefore may inform curriculum and remediation tailoring.

Test of integrated professional skills: objective structured clinical examination/simulation hybrid assessment of obstetrics-gynecology residents' skill integration.

BACKGROUND: Assessment of obstetrics-gynecology residents' ability to integrate clinical judgment, interpersonal skills, and technical ability in a uniform fashion is required to document achievement of benchmarks of competency. An observed structured clinical examination that incorporates simulation and bench models uses direct observation of performance to generate formative feedback and standardized evaluation. METHODS: The Test of Integrated Professional Skills (TIPS) is a 5-station performance-based assessment that uses standardized patients and complex scenarios involving ultrasonography, procedural skills, and evidence-based medicine. Standardized patients and faculty rated residents by using behaviorally anchored checklists. Mean scores reflecting performance in TIPS were compared across competency domains and by developmental level (using analysis of variance) and then compared to standard faculty clinical evaluations (using Spearman ρ). Participating faculty and residents were also asked to evaluate the usefulness of the TIPS. RESULTS: Twenty-four residents participated in the TIPS. Checklist items used to assess competency were sufficiently reliable, with Cronbach α estimates from 0.69 to 0.82. Performance improved with level of training, with wide variation in performance. Standard faculty evaluations did not correlate with TIPS performance. Several residents who were rated as average or above average by faculty performed poorly on the TIPS (> 1 SD below the mean). Both faculty and residents found the TIPS format useful, providing meaningful evaluation and opportunity for feedback. CONCLUSIONS: A simulation-based observed structured clinical examination facilitates observation of a range of skills, including competencies that are difficult to observe and measure in a standardized way. Debriefing with faculty provides an important interface for identification of performance gaps and individualization of learning plans.

Presenting quality data to vulnerable groups: charts, summaries or behavioral economic nudges?

OBJECTIVES: Despite the increased focus on health care consumers' active choice, not enough is known about how to best facilitate the choice process. We sought to assess methods of improving this process for vulnerable consumers in the United States by testing alternatives that emphasize insights from behavioral economics, or 'nudges'. METHODS: We performed a hypothetical choice experiment where subjects were randomized to one of five experimental conditions and asked to choose a health center (location where they would receive all their care). The conditions presented the same information about health centers in different ways, including graphically as a chart, via written summary and using behavioral economics, 'nudging' consumers toward particular choices. We hypothesized that these 'nudges' might help simplify the choice process. Our primary outcomes focused on the health center chosen and whether consumers were willing to accept 'nudges'. RESULTS: We found that consumer choice was influenced by the method of presentation and the majority of consumers accepted the health center they were 'nudged' towards. CONCLUSIONS: Consumers were accepting of choices grounded in insights from behavioral economics and further consideration should be given to their role in patient choice.

Telephone care coordination for smokers in VA mental health clinics: protocol for a hybrid type-2 effectiveness-implementation trial.

BACKGROUND: This paper describes an innovative protocol for a type-II hybrid effectiveness-implementation trial that is evaluating a smoking cessation telephone care coordination program for Veterans Health Administration (VA) mental-health clinic patients. As a hybrid trial, the protocol combines implementation science and clinical trial methods and outcomes that can inform future cessation studies and the implementation of tobacco cessation programs into routine care. The primary objectives of the trial are (1) to evaluate the process of adapting, implementing, and sustaining a smoking cessation telephone care coordination program in VA mental health clinics, (2) to determine the effectiveness of the program in promoting long-term abstinence from smoking among mental health patients, and (3) to compare the effectiveness of telephone counseling delivered by VA staff with that delivered by state quitlines. METHODS/DESIGN: The care coordination program is being implemented at six VA facilities. VA mental health providers refer patients to the program via an electronic medical record consult. Program staff call referred patients to offer enrollment. All patients who enroll receive a self-help booklet, mailed smoking cessation medications, and proactive multi-call telephone counseling. Participants are randomized to receive this counseling from VA staff or their state's quitline. Four primary implementation strategies are being used to optimize program implementation and sustainability: blended facilitation, provider training, informatics support, and provider feedback. A three-phase formative evaluation is being conducted to identify barriers to, and facilitators for, program implementation and sustainability. A mixed-methods approach is being used to collect quantitative clinical effectiveness data (e.g., self-reported abstinence at six months) and both quantitative and qualitative implementation data (e.g., provider referral rates, coded interviews with providers). Summative data will be analyzed using the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework. DISCUSSION: This paper describes the rationale and methods of a trial designed to simultaneously study the clinical effectiveness and implementation of a telephone smoking cessation program for smokers using VA mental health clinics. Such hybrid designs are an important methodological design that can shorten the time between the development of an intervention and its translation into routine clinical care.

Engaging health professional students in substance abuse research: development and early evaluation of the SARET program.

OBJECTIVE: There is a need to build the ranks of health care professionals engaged in substance abuse (SA)-focused clinical research. The authors simultaneously developed and evaluated SARET, the Substance Abuse Research Education and Training program. The fundamental goal of this interprofessional program is to stimulate medical, dental, and nursing student interest and experience in SA research. Evaluation aims to understand program feasibility and acceptability and to assess short-term impact. METHODS: SARET comprises 2 main components: stipend-supported research mentorships and a Web-based module series, consisting of 6, interactive, multimedia modules addressing core SA research topics, delivered via course curricula and in the research mentorships. Authors assessed program feasibility and impact on student interest in conducting SA research by tracking participation and conducting participant focus groups and online surveys. RESULTS: Thirty early health care professional students completed mentorships (25 summer, 5 yearlong) and 1324 completed at least 1 Web-module. SARET was considered attractive for the opportunity to conduct clinically oriented research and to work with health care professionals across disciplines. Mentorship students reported positive impact on their vision of SA-related clinical care, more positive attitudes about research, and, in some cases, change in career plans. Web-based modules were associated with enhanced interest in SA (35% increase, P = 0.005, in those somewhat/very interested for neurobiology module) and SA research (+38%, P < 0.001 for activation, +45%, P < 0.001 for personal impact, +7%, P = 0.089 for neurobiology). CONCLUSIONS: The SARET program stimulates SA clinical and research interest among students of nursing, medicine, and dentistry and may lend itself to dissemination.

Assessment of medical student clinical reasoning by "lay" vs physician raters: inter-rater reliability using a scoring guide in a multidisciplinary objective structured clinical examination.

BACKGROUND: To determine whether a "lay" rater could assess clinical reasoning, interrater reliability was measured between physician and lay raters of patient notes written by medical students as part of an 8-station objective structured clinical examination. METHODS: Seventy-five notes were rated on core elements of clinical reasoning by physician and lay raters independently, using a scoring guide developed by physician consensus. Twenty-five notes were rerated by a 2nd physician rater as an expert control. Kappa statistics and simple percentage agreement were calculated in 3 areas: evidence for and against each diagnosis and diagnostic workup. RESULTS: Agreement between physician and lay raters for the top diagnosis was as follows: supporting evidence, 89% (κ = .72); evidence against, 89% (κ = .81); and diagnostic workup, 79% (κ = .58). Physician rater agreement was 83% (κ = .59), 92% (κ = .87), and 96% (κ = .87), respectively. CONCLUSIONS: Using a comprehensive scoring guide, interrater reliability for physician and lay raters was comparable with reliability between 2 expert physician raters.

An intervention to improve care and reduce costs for high-risk patients with frequent hospital admissions: a pilot study.

BACKGROUND: A small percentage of high-risk patients accounts for a large proportion of Medicaid spending in the United States, which has become an urgent policy issue. Our objective was to pilot a novel patient-centered intervention for high-risk patients with frequent hospital admissions to determine its potential to improve care and reduce costs. METHODS: Community and hospital-based care management and coordination intervention with pre-post analysis of health care utilization. We enrolled Medicaid fee-for-service patients aged 18-64 who were admitted to an urban public hospital and identified as being at high risk for hospital readmission by a validated predictive algorithm. Enrolled patients were evaluated using qualitative and quantitative interview techniques to identify needs such as transportation to/advocacy during medical appointments, mental health/substance use treatment, and home visits. A community housing partner initiated housing applications in-hospital for homeless patients. Care managers facilitated appropriate discharge plans then worked closely with patients in the community using a harm reduction approach. RESULTS: Nineteen patients were enrolled; all were male, 18/19 were substance users, and 17/19 were homeless. Patients had a total of 64 inpatient admissions in the 12 months before the intervention, versus 40 in the following 12 months, a 37.5% reduction. Most patients (73.3%) had fewer inpatient admissions in the year after the intervention compared to the prior year. Overall ED visits also decreased after study enrollment, while outpatient clinic visits increased. Yearly study hospital Medicaid reimbursements fell an average of $16,383 per patient. CONCLUSIONS: A pilot intervention for high-cost patients shows promising results for health services usage. We are currently expanding our model to serve more patients at additional hospitals to see if the pilot's success can be replicated. TRIAL REGISTRATION: Clinicaltrials.gov Identifier: NCT01292096.

Observing handoffs and telephone management in GI fellowship training.

OBJECTIVES: Gastroenterology (GI) training programs are mandated to teach fellows interpersonal communication and professionalism as basic competencies. We sought to assess important skill sets used by our fellows but not formally observed or measured: handoffs, telephone management, and note writing. We designed an Observed Standardized Clinical Examination (OSCE) form and provided the faculty with checklists to rate fellows' performance on specific criteria. METHODS: We created two new scenarios: a handoff between a tired overnight senior fellow on call and a more junior fellow, and a telephone management case of an ulcerative colitis flare. Fellows wrote a progress notes documenting the encounters. To add educational value, we gave the participants references about handoff communication. Four OSCE stations-handoff communication, telephone management, informed consent, and delivering bad news-were completed by fellows and observed by faculty. RESULTS: Eight faculty members and eight fellows from four GI training programs participated. All the fellows agreed that handoffs can be important learning opportunities and can be improved if they are structured, and that handoff skills can improve with practice. CONCLUSIONS: OSCEs can serve as practicums for assessing complex skill sets such as handoff communication and telephone management.