Assessment Practices in Continuing Professional Development Activities in Health Professions: A Scoping Review.

CONTEXT: In continuing professional development (CPD), educators face the need to develop and implement innovative assessment strategies to adhere to accreditation standards and support lifelong learning. However, little is known about the development and validation of these assessment practices. We aimed to document the breadth and depth of what is known about the development and implementation of assessment practices within CPD activities. METHODS: We conducted a scoping review using the framework proposed by Arksey and O'Malley (2005) and updated in 2020. We examined five databases and identified 1733 abstracts. Two team members screened titles and abstracts for inclusion/exclusion. After data extraction, we conducted a descriptive analysis of quantitative data and a thematic analysis of qualitative data. RESULTS: A total of 130 studies were retained for the full review. Most reported assessments are written assessments (n = 100), such as multiple-choice items (n = 79). In 99 studies, authors developed an assessment for research purpose rather than for the CPD activity itself. The assessment validation process was detailed in 105 articles. In most cases, the authors examined the content with experts (n = 57) or pilot-tested the assessment (n = 50). We identified three themes: 1-satisfaction with assessment choices; 2-difficulties experienced during the administration of the assessment; and 3-complexity of the validation process. CONCLUSION: Building on the adage "assessment drives learning," it is imperative that the CPD practices contribute to the intended learning and limit the unintended negative consequences of assessment. Our results suggest that validation processes must be considered and adapted within CPD contexts.

Considering the Structured Oral Examinations Beyond Its Psychometrics Properties.

Decisions to set aside Structured Oral Examinations (SOE) are, almost invariably, based on their poor psychometric properties. However, considering the perspectives of the stakeholders might help us to understand its potential contribution. To explore this, we conducted focus groups and individual interviews with stakeholders: students, assessors, and administrators. Students and assessors perceived the SOE as a window on students' clinical reasoning, as an authentic assessment, but as a subjective and stressful method. Administrators emphasized the organizational consequences such as logistical challenges. Consequences must be considered when making decisions about SOE and our results support important positive consequences.

Validity as a social imperative: users' and leaders' perceptions.

Introduction: Recently, validity as a social imperative was proposed as an emerging conceptualization of validity in the assessment literature in health professions education (HPE). To further develop our understanding, we explored the perceived acceptability and anticipated feasibility of validity as a social imperative with users and leaders engaged with assessment in HPE in Canada. Methods: We conducted a qualitative interpretive description study. Purposeful and snowball sampling were used to recruit participants for semi-structured individual interviews and focus groups. Each transcript was analyzed by two team members and discussed with the team until consensus was reached. Results: We conducted five focus group and eleven interviews with two different stakeholder groups (users and leaders). Our findings suggest that the participants perceived the concept of validity as a social imperative as acceptable. Regardless of group, participants shared similar considerations regarding: the limits of traditional validity models, the concept's timeliness and relevance, the need to clarify some terms used to characterize the concept, the similarities with modern theories of validity, and the anticipated challenges in applying the concept in practice. In addition, participants discussed some limits with current approaches to validity in the context of workplace-based and programmatic assessment. Conclusion: Validity as a social imperative can be interwoven throughout existing theories of validity and may represent how HPE is adapting traditional models of validity in order to respond to the complexity of assessment in HPE; however, challenges likely remain in operationalizing the concept prior to its implementation.

Contexte: Une nouvelle manière de concevoir la validité en matière d'évaluation des apprentissages dans les programmes en sciences de la santé a récemment été proposée : la validité en tant qu'impératif social. Pour mieux la comprendre, nous avons exploré l'acceptabilité perçue et la faisabilité anticipée de la validité en tant qu'impératif social auprès d'utilisateurs et de leaders en matière d'évaluation en éducation des professions de la santé au Canada. Méthodes: Nous avons mené une étude qualitative descriptive interprétative. Pour recruter les participants aux entrevues individuelles semi-structurées et aux groupes de discussion, nous avons utilisé l'échantillonnage par choix raisonné et l'échantillonnage boule de neige. Les transcriptions ont été analysées par deux membres de l'équipe et discutées avec l'ensemble de l'équipe jusqu'à l'obtention d'un consensus. Résultats: Nous avons mené cinq groupes de discussion et onze entretiens avec deux groupes de parties prenantes, l'un composé d'utilisateurs, l'autre de leaders. Nos résultats suggèrent que les participants estiment acceptable le concept de validité comme impératif social. Quel que soit le groupe, les participants ont partagé des considérations similaires concernant : les limites des modèles de validité traditionnels, l'actualité et la pertinence du concept, la nécessité de clarifier certains termes utilisés pour définir le concept, les similitudes avec les théories modernes de la validité, et les défis anticipés de son application. En outre, les participants ont soulevé certaines limites des approches actuelles de la validité dans le contexte de l'évaluation en milieu de travail et de l'évaluation programmatique. Conclusion: La notion de validité comme impératif social peut être incorporée dans les théories existantes pour traduire l'adaptation des modèles traditionnels de la validité à la complexité de l'évaluation en éducation des professions de la santé; cependant, certains défis liés à l'opérationnalisation du concept seraient à résoudre avant sa mise en œuvre.

Nursing Interventions Designed to Improve Healthcare in General Healthcare Settings for Adults Living with Mental Illness: A Scoping Review.

People living with mental illness have difficulty obtaining quality physical care services. Nurses find it hard to fully assume their role and adopt a person-centered approach when they care for people with both physical and mental health problems. A scoping review was carried out to document the breadth and depth of literature on nurse interventions designed to improve nursing care in general healthcare settings for adults living with mental illness. Two researchers independently screened titles, abstracts and full documents to obtain publications, and the relevant data was extracted from individual studies. Of the 384 studies identified, 16 documents were included in the study. Few interventions have been the subject of scientific publications and they are scattered in their objectives and very heterogeneous in their content. Interventions are often included in a large program that may be difficult to apply in various care settings. There are many different nursing roles and activities in care-management and in support of behavioral changes although their main measured outcomes differ. This study highlights a tendency for interventions to relegate people with mental health problems to specialized services, which reinforces the dichotomization of services and does not favor a holistic and person-centered approach. There is an urgent need to develop and evaluate nursing interventions in general healthcare settings for adults living with mental illness. The development of relevant continuing education and training must take into account both the point of view of the people living with mental illness and that of nurses.

Exploration of the occupational and personal dimensions impacted by the COVID-19 pandemic for nurses: A qualitative analysis of survey responses.

AIM: To explore the occupational and personal life dimensions that have been impacted by the COVID-19 pandemic for registered nurses (RN) and licensed practical nurses (LPN). DESIGN: Qualitative interpretive description approach. METHODS: Between July and September 2020, a web-based cross-sectional study was conducted among RNs and LPNs in Quebec, Canada. Included in this survey was an open-ended question allowing nurses to describe the occupational and personal life dimensions that were impacted by the COVID-19 pandemic. Thematic analysis was used to interpret the qualitative data from this open-ended question. Reporting followed the Standards For Reporting Qualitative Research (SRQR). RESULTS: Of the 1860 survey respondents, 774 RNs and 43 LPNs responded to the open-ended question (total n = 819). For the occupational dimension, six themes were identified: impacts of infection control on work, change in daily work tasks, offloading and reorganization of care, deterioration of working conditions, increased stress at work and issues related to the profession. For the personal dimension, four themes were found: impacts on the family, dealing with changes about leisure and personal life, impacts on physical and psychological health. CONCLUSION: Knowing the dimensions affected by the COVID-19 pandemic could help to identifying appropriate interventions to support RNs and LPNs. IMPACT: The COVID-19 pandemic has significantly impacted the occupational and personal lives RNs and LPNs working in the Quebec healthcare system. More specifically, Quebec's nurses experienced a major reorganization of care generated by important government decisions. Knowing how the pandemic affected different life dimensions will help in the development of support adapted to nurses' realities. Even in a pandemic context, improvements in the work environment or appropriate support could lead to an improved psychological health for nurses.

COVID-19 as the tipping point for integrating e-assessment in higher education practices.

The COVID-19 pandemic provoked an urgency for many educators to integrate digital information and communication technologies in their educational practices. We explored how faculty members tackled the task of adapting their assessment practices during the pandemic to identify what is required to sustain and favour future quality development and implementation of e-assessment in higher education. Employing a qualitative descriptive approach, we conducted semi-structured interviews with thirty-one individuals six months into the COVID-19 pandemic. We identified four major themes in participants' discourse about the integration of e-assessment during the COVID-19 pandemic: (a) the considerations they had for the potential consequences on students and how they considered this while deciding how to move forward, (b) the preoccupations for the potential for cheating, (c) the importance of pedagogical alignment, and (d) the affordances available to them. While the COVID-19 pandemic highlighted the fact that higher education institutions were not prepared for a pivot to- or greater integration of- e-assessment, it also provided the tipping-point to do so. In other words, it offered an unprecedented opportunity to critically appraise and change assessment practices, this opportunity was also a very challenging balancing act of considering the social consequences of assessment, and the alignment within set affordances.

Validity as a social imperative for assessment in health professions education: a concept analysis.

CONTEXT: Assessment can have far-reaching consequences for future health care professionals and for society. Thus, it is essential to establish the quality of assessment. Few modern approaches to validity are well situated to ensure the quality of complex assessment approaches, such as authentic and programmatic assessments. Here, we explore and delineate the concept of validity as a social imperative in the context of assessment in health professions education (HPE) as a potential framework for examining the quality of complex and programmatic assessment approaches. METHODS: We conducted a concept analysis using Rodgers' evolutionary method to describe the concept of validity as a social imperative in the context of assessment in HPE. Supported by an academic librarian, we developed and executed a search strategy across several databases for literature published between 1995 and 2016. From a total of 321 citations, we identified 67 articles that met our inclusion criteria. Two team members analysed the texts using a specified approach to qualitative data analysis. Consensus was achieved through full team discussions. RESULTS: Attributes that characterise the concept were: (i) demonstration of the use of evidence considered credible by society to document the quality of assessment; (ii) validation embedded through the assessment process and score interpretation; (iii) documented validity evidence supporting the interpretation of the combination of assessment findings, and (iv) demonstration of a justified use of a variety of evidence (quantitative and qualitative) to document the quality of assessment strategies. CONCLUSIONS: The emerging concept of validity as a social imperative highlights some areas of focus in traditional validation frameworks, whereas some characteristics appear unique to HPE and move beyond traditional frameworks. The study reflects the importance of embedding consideration for society and societal concerns throughout the assessment and validation process, and may represent a potential lens through which to examine the quality of complex and programmatic assessment approaches.