Predicting Medical Student Performance With a Situational Judgment Test in Admissions.

PURPOSE: To explore correlations between AAMC situational judgment test (SJT) scores, other admissions data, and learners' medical school performance. METHOD: First- and second-year medical students from 8 U.S. MD-granting medical schools completed a prototype version of the AAMC SJT in 2017. Outcomes included research-only faculty ratings of student performance, final course grades, and faculty evaluations of student performance, 2017-2018 and 2018-2019 academic years. Bivariate correlations were used to investigate the relationship between SJT scores and student performance outcomes and hierarchical regressions to investigate whether SJT scores provided incremental validity over MCAT total scores and cumulative undergraduate grade point averages (UGPAs) for predicting student performance outcomes. RESULTS: In general, there were small positive correlations with research-only faculty ratings from the first year of medical school, with the highest for social skills/service orientation ( rcorrected = .33, P < .05). Correlations were higher, with the highest for social skills/service orientation and cultural competence ( rcorrected = .33 and .36, respectively, P < .05) in the second year in medical school. SJT scores improved prediction of research-only faculty ratings over MCAT total scores and UGPAs for reliability and dependability/capacity for improvement, cultural competence, social skills/service orientation, and the overall composite score in the first year and for resilience and adaptability, social skills/service orientation, cultural competence, and the overall composite score in the second year. SJT scores demonstrated small correlations with course grades ( rsample-weighted = .10, P = ns) and faculty evaluations related to professionalism skills ( rsample-weighted = .14, P < .05); however, MCAT total scores explained most of the variance associated with course outcomes. CONCLUSIONS: These studies provide initial evidence that SJT scores may add value to the medical school admissions process because scores were related to faculty ratings of professional behaviors and provided unique information relative to MCAT scores and UGPAs.

Impact of patterns of language use and socio-economic status on a constructed response Situational Judgment Test (SJT).

The study explored the impacts of patterns of language use (PLU) and socio-economic status (SES) on Casper, a constructed-response situational judgment test (SJT). 10,266 applicants from two U.S. medical schools were grouped into self-reported balanced bilinguals, unbalanced bilinguals, English monolinguals, and English as a Second Language (ESL) students. A multicomponent SES composite was used to assess the degree of socioeconomic disadvantage (DSD). Results from a hierarchical regression analysis showed that after accounting for demographic variables, both PLU and DED were significant factors on applicants' Casper performance. Bilingualism was associated with better Casper performance compared to English monolinguals and ESL students. No significant effect of speaking English as a native language was found on applicants' Casper performance. English monolinguals and ESL students performed equivalently on Casper. Finally, high DSD was associated with better Casper performance than low DSD, and the impact of DSD on Casper held the same across all four language groups. These findings provide evidence that socio-cultural factors, such as PLU and DSD have important impacts on SJT performance. Further research is needed to understand the role of differences in language construction across socio-cultural factors on constructed-response SJT performance.

Shaping Perceptions of Basic Science Education by Utilizing Real Patient Encounters.

BACKGROUND: Integrating basic science into clinical teaching has been a struggle for medical schools. However, early exposure to clinical experience has been associated with an increased understanding of the importance of basic science, positive attitudes, and developing clinical skills faster. Furthermore, early clinical exposure can help students reconnect with what drove them into medicine in the first place, especially when they are starting to feel burned out by the volume of lecture material. As a result, increasing patient experience during the first year has become a goal of many medical schools. METHODS: At Rutgers Robert Wood Johnson Medical School, interprofessional case discussions (ICDs) begin with a lecture that explicitly integrates basic science with a disease, followed by a discussion with a patient, their family, the healthcare team, and first-year students. Our objective is to explore whether ICDs enhanced the learning experience of basic science. CONTEXT: ICD satisfaction was assessed using evaluations from two different courses (2013-2016). Responses were analyzed quantitatively using descriptive statistics and qualitatively using a grounded-theory-content analysis. Study 2: A follow-up measure with current third- and fourth-year students on long-term retention of basic science was analyzed using a Wilcoxon signed rank test. Relative rankings of three different case-based teaching modalities were assessed using chi-square. RESULTS: Students reported significantly higher satisfaction with ICDs (93%) for reinforcing concepts and integrating materials compared to Flipped Classrooms (66%) and Jigsaws (65%), x 2 = 120.9, p < .001. Student comments fit into five categories: enjoyment, learning/retention, the clinical usefulness of basic science, affirming passion to be in medicine, and others. The follow-up measure indicated significantly greater retention of the biochemical basis of diseases covered during ICDs. CONCLUSIONS: While other teaching modalities integrate basic science into a clinical context, ICDs go further by displaying interprofessional care and the manifestation of the disease on the patient and the lives of their family. As a result, ICDs lead to a positive learning environment in which students feel comfortable, have a sense of rapport with the patients and health care providers, and feel motivated to learn basic science.

Development of an empathy and clarity rating scale to measure the effect of medical improv on end-of-first-year OCSE performance: a pilot study.

Patients want empathetic physicians who listen and understand. How do you teach and measure empathy? Medical educators, including those inspired by Alan Alda, have turned to theater to teach skills in empathetic communication. Improvisation-informedcurriculum (medical improv) draws upon foundational actors training: deep listening, emotional understanding, connections, authenticity. Arating scale to measure the impact of medical improv on empathetic and clear communication does not exist. Objective: To develop aframework and instrument, the Empathy and Clarity Rating Scale (ECRS), for measuring communication elements used by actors and physicians, and pilot ECRS to test effectiveness of medical improv on first-yearstudents' communication skills. Design: Four medical schools collaborated. USMLE Step 2 Communication and Interpersonal Skills (CIS) domains were used as framework for discussion among three focus groups, each with clinicians, actors, communication experts, and community members with patient experience. Audiotaped discussions were transcribed; open coding procedures located emerging themes. The initial coding scheme was compared with the Consultation and Relational Empathy (CARE) measure. ECRS content was aligned with CARE, CIS and focus group themes. Modified nominal processes were conducted to finalize the scale. We implemented procedures to establish content validity and interrater reliability. Final ECRS was used to study student performance across three levels of experience with medical improv. Results: The final ECRS was comprised of seven five-pointscale items. Narrative comments precede behaviorally anchored ratings: 5=desired, 1=ineffective, 2-4=developing based upon adjustment needed. Rater agreement across all items was 84%. There was asmall correlation between the ECRS and another measure interviewing (r=0.262, p=0.003). Students with advanced medical improv training outperformed those without (F=3.51, p=.042). Conclusion: Acommunication scale enlightened by experiences of actors, clinicians, scholars and patients has been developed. The ECRS has potential to detect the impact of medical improv on development of empathetic and clear communication.

How good are we at selecting students that meet our mission? Outcomes of the 2011 and 2012 entering classes selected by a locally developed multiple mini interview.

BACKGROUND: Can a locally developed multiple mini interview (MMI) process lead to outcomes reflective of local values and mission? METHODS: In 2017, the authors performed a retrospective analysis of the relationship of MMI with multiple-choice-based outcomes and non-multiple-choice-based outcomes, including clerkship competencies, OSCE, scholarship/service/leadership, academic honor society induction, peer and faculty humanism nominations, and overall performance at graduation for two entering classes with acceptance decisions based exclusively on a locally developed MMI. RESULTS: There was no association between MMI and performance on multiple-choice-based examinations. For other outcomes, the effect size of MMI for OSCE was small and leadership/service and scholarship did not correlate with MMI score. For clerkship competencies, there was medium effect size for patient care, practice-based learning and improvement, interpersonal and communication skills, and cultural competence. Highest and lowest quartile MMI scorers were no different in academic honor society induction; however, top quartile MMI scorers received more humanism votes versus last quartile and were more likely rated outstanding or excellent graduates. CONCLUSIONS: Local development of MMI and of admissions processes with sole reliance on MMI for final acceptance decisions will not affect academic preparation/medical school performance in multiple-choice-based assessments but can lead to locally desired attributes in students.

The role of professional identity in graduate school success for under-represented minority students.

We examined the relationship between sense of professional identity and academic success among under-represented minority graduate students in a biomedical doctoral program. We found that a sense of professional identity is related to science success among under-represented minority students, but not for non-underrepresented minority students. Sense of professional identity appears to be linked to the use of mentors and to development as a scientist and member of the professional community, rather than someone who simply performs laboratory experiments.

A course for nurses to handle difficult communication situations. A randomized controlled trial of impact on self-efficacy and performance.

OBJECTIVE: to evaluate the impact of a communication skills course for nurses on how to handle difficult communication situations in their daily work. METHODS: a 7-h course was developed using a construct of "Awareness, Feelings, Listen, Solve" (AFLS). A pedagogy of experiential, learner-centered learning was adopted. The course evaluation used a randomized controlled design with pre- and post-measures of self-efficacy and performance. RESULTS: forty-one nurses volunteered and thirty-three nurses completed all assigned parts of the study. On self-assessment, there was significant improvement for self-efficacy (F=24.43, p<0.001), but not for emotional awareness. On performance, there was no significant improvement between intervention and control groups (F=3.46, p=0.073). CONCLUSION: a short course for nurses on handling difficult communication situations achieved significant improvements in self-efficacy but not in performance. PRACTICE IMPLICATIONS: teaching communication skills in community-based settings is important for the safety and effectiveness of patient care. Sponsoring organizations should weigh trade-offs between feasibility and achievement of measurable improvements in performance. One possible approach is to focus on specific communication skills rather than a full suite of skills.

A birth-to-death view of mRNA from the RNA recognition motif perspective.

RNA binding proteins are a large and varied group of factors that are the driving force behind post-transcriptional gene regulation. By analogy with transcription factors, RNA binding proteins bind to various regions of the mRNAs that they regulate, usually upstream or downstream from the coding region, and modulate one of the five major processes in mRNA metabolism: splicing, polyadenylation, export, translation and decay. The most abundant RNA binding protein domain is called the RNA Recognition Motif (RRM)1. It is probably safe to say that an RRM-containing protein is making some contact with an mRNA throughout its existence. The transcriptional counterpart would likely be the histones, yet the multitude of specific functions that are results of RRM based interactions belies the universality of the motif. This complex and diverse application of a single protein motif was used as the basis to develop an advanced graduate level seminar course in RNA:protein interactions. The course, utilizing a learner-centered empowerment model, was developed to dissect each step in RNA metabolism from the perspective of an RRM containing protein. This provided a framework to discuss the development of specificity for the RRM for each required process.

The war-related illness and injury study centers: a resource for deployment-related health concerns.

Combat veterans often return from deployment having experienced a wide range of exposures, symptoms, and medical conditions. The Department of Veterans Affairs established war-related illness and injury study centers to serve combat veterans with unexplained illnesses. We report the exposures, clinical status, and utilization of 53 combat veterans who participated in the National Referral Program (NRP) from January 2002 until March 2004. Participants were primarily male (81%) and served in the Persian Gulf War (79%). Common diagnoses were chronic fatigue syndrome (n = 23, 43%), neurotic depression (n = 21, 40%), and post-traumatic stress disorder (n = 20, 38%). Self-reported exposures related to weaponry, disease prophylaxis, environmental hazards, stress, and poor hygiene. A small increase in mean SF-36V mental component scores (2.8 points, p = 0.009) and use of rehabilitation therapies (1.6 additional visits, p = 0.018) followed the NRP referral. The small gain in mental function suggests that the NRP may benefit combat veterans with long and complex medical histories.

Integrative medicine: implementation and evaluation of a professional development program using experiential learning and conceptual change teaching approaches.

OBJECTIVE: To meet the increasing patient interest in complementary and alternative medicine (CAM), conventional physicians need to understand CAM, be willing to talk with their patients about CAM, and be open to recommending selected patients to appropriate CAM modalities. We aimed to raise physicians' awareness of, and initiate attitudinal changes towards CAM in the context of integrative medical practice. We developed and implemented a professional development program involving experiential learning and conceptual change teaching approaches. METHODS: A randomized controlled study with a pre-post design in a large academic medical center. The 8-hour intervention used experiential and conceptual change educational approaches. Forty-eight cardiologists were randomized to participant and control groups. A questionnaire measured physicians' conceptions of, and attitudes to CAM, the likelihood of changing practice patterns, and the factors most important in influencing such changes. The questionnaire included an embedded control question on a topic that was not the focus of this program. We administered the questionnaire before (pretest) and after (posttest) the intervention. We compared differences in pre- and post-intervention scores between the participant (N = 20) and control (N = 16) groups. We used both groups to identify factors that influenced their practice patterns. The study was NIH-funded and IRB-exempt. RESULTS: Both groups initially had little knowledge about, and negative attitudes to CAM. The participant group had significant positive changes in their conceptions about, and attitudes to CAM after the program, and significant improvements when compared with the control group. Participant physicians significantly increased in their willingness to integrate CAM in their practices. Physicians (combined groups) rated research evidence as the most important factor influencing their willingness to integrate CAM. They requested more research evidence for CAM efficacy, and more information on non-conventional pharmacology. Participants reflected enthusiasm for the experiential program. CONCLUSIONS: The participants were able to experience the positive effects of selected CAM modalities. It is possible to increase physician knowledge and change attitudes towards integrative medicine with an eight-hour intervention using experiential and conceptual change teaching approaches. PRACTICE IMPLICATIONS: Professional development on integrative medicine can be offered to medical practitioners using experiential learning and conceptual change teaching approaches, with the help of local CAM practitioners.

Development and evaluation of a peer-tutoring program for graduate students*.

Many interdisciplinary Ph.D. programs admit students of different educational backgrounds who receive a first year of a general curriculum education. However, student preparation for this curriculum varies, and methods are needed to provide academic support. Graduate student peer tutoring was piloted as an initiative funded by a National Institutes of Health (NIH) Initiative for Minority Student Development award to the University of Medicine and Dentistry of New Jersey-Robert Wood Johnson Medical School (UMDNJ-RWJMS) and is now offered to all students in the interdisciplinary Molecular Biosciences Ph.D. program between Rutgers, The State University of New Jersey, and UMDNJ-RWJMS. Tutoring occurs individually or in small groups and has grown over the past 5 years in the number of students tutored and hours of tutoring. The program was evaluated by surveying and interviewing both tutors and students concerning process variables (e.g. awareness, frequency) and impact variables (e.g. perceived benefits, motivators), as well as by assessing changes in exam scores for the four core courses of the first-year graduate curriculum.

National practices regarding payment to research subjects for participating in pediatric research.

OBJECTIVE: Payment to subjects for participation in research is reportedly common, but no published data documents the nature of this practice. Institutional review boards (IRBs) are responsible for ensuring both the safety and voluntary participation of research subjects, yet guidance from federal and expert pediatric sources regarding appropriate payment approaches is conflicting. Ethical issues of payment for participation of adult versus pediatric research subjects may differ. This empirical study sought to examine current payment practices for participation in pediatric research as reported by IRBs. DESIGN: An 18-question survey regarding payment practices for participation in pediatric research was sent to IRB chairs at member institutions of the National Association of Children's Hospitals and Related Institutions, and to a systematic random sample of IRB chairs listed with the Office for Protection From Research Risks. Descriptive, nonparametric, and qualitative analyses were used to describe institution types, payment practices, and correlations among responses. RESULTS: Data from 128 institutions that conduct pediatric research revealed that payment for participation in pediatric research was allowed by 66% of responding institutions, and practices varied widely among institutions. Most responding IRBs that allowed payment required disclosure of payment before enrollment (during the consent process), following federal guidelines more closely than American Academy of Pediatrics guidelines. An IRB's perception of potential benefits or harms of a study correlated with the amount of payment approved. CONCLUSIONS: IRBs must balance the need to recruit pediatric research subjects against the risk of undue influence during the recruitment process. Federal guidelines and expert pediatric opinion differ in recommendations regarding payment; responding IRBs appeared to follow federal guidelines more closely than guidelines proposed by the American Academy of Pediatrics.