Twelve tips for embedding assessment for and as learning practices in a programmatic assessment system.

Programmatic assessment supports the evolution from assessment of learning to fostering assessment for learning and as learning practices. A well-designed programmatic assessment system aligns educational objectives, learning opportunities, and assessments with the goals of supporting student learning, making decisions about student competence and promotion decisions, and supporting curriculum evaluation. We present evidence-based guidance for implementing assessment for and as learning practices in the pre-clinical knowledge assessment system to help students learn, synthesize, master and retain content for the long-term so that they can apply knowledge to patient care. Practical tips are in the domains of culture and motivation of assessment, including how an honour code and competency-based grading system can support an assessment system to develop student self-regulated learning and professional identity, curricular assessment structure, such as how and when to utilize low-stakes and cumulative assessment to drive learning, exam and question structure, including what authentic question and exam types can best facilitate learning, and assessment follow-up and review considerations, such exam retake processes to support learning, and academic success structures. A culture change is likely necessary for administrators, faculty members, and students to embrace assessment as most importantly a learning tool for students and programs.

PBL and beyond: trends in collaborative learning.

Building upon the disruption to lecture-based methods triggered by the introduction of problem-based learning, approaches to promote collaborative learning are becoming increasingly diverse, widespread and generally well accepted within medical education. Examples of relatively new, structured collaborative learning methods include team-based learning and just-in-time teaching. Examples of less structured approaches include think-pair share, case discussions, and the flipped classroom. It is now common practice in medical education to employ a range of instructional approaches to support collaborative learning. We believe that the adoption of such approaches is entering a new and challenging era. We define collaborate learning by drawing on the broader literature, including Chi's ICAP framework that emphasizes the importance of sustained, interactive explanation and elaboration by learners. We distinguish collaborate learning from constructive, active, and passive learning and provide preliminary evidence documenting the growth of methods that support collaborative learning. We argue that the rate of adoption of collaborative learning methods will accelerate due to a growing emphasis on the development of team competencies and the increasing availability of digital media. At the same time, the adoption collaborative learning strategies face persistent challenges, stemming from an overdependence on comparative-effectiveness research and a lack of useful guidelines about how best to adapt collaborative learning methods to given learning contexts. The medical education community has struggled to consistently demonstrate superior outcomes when using collaborative learning methods and strategies. Despite this, support for their use will continue to expand. To select approaches with the greatest utility, instructors must carefully align conditions of the learning context with the learning approaches under consideration. Further, it is critical that modifications are made with caution and that instructors verify that modifications do not impede the desired cognitive activities needed to support meaningful collaborative learning.

Natural Language Processing of Learners' Evaluations of Attendings to Identify Professionalism Lapses.

Unprofessional faculty behaviors negatively impact the well-being of trainees yet are infrequently reported through established reporting systems. Manual review of narrative faculty evaluations provides an additional avenue for identifying unprofessional behavior but is time- and resource-intensive, and therefore of limited value for identifying and remediating faculty with professionalism concerns. Natural language processing (NLP) techniques may provide a mechanism for streamlining manual review processes to identify faculty professionalism lapses. In this retrospective cohort study of 15,432 narrative evaluations of medical faculty by medical trainees, we identified professionalism lapses using automated analysis of the text of faculty evaluations. We used multiple NLP approaches to develop and validate several classification models, which were evaluated primarily based on the positive predictive value (PPV) and secondarily by their calibration. A NLP-model using sentiment analysis (quantifying subjectivity of the text) in combination with key words (using the ensemble technique) had the best performance overall with a PPV of 49% (CI 38%-59%). These findings highlight how NLP can be used to screen narrative evaluations of faculty to identify unprofessional faculty behaviors. Incorporation of NLP into faculty review workflows enables a more focused manual review of comments, providing a supplemental mechanism to identify faculty professionalism lapses.