Trainee-supervisor collaboration, progress-visualisation, and coaching: a survey on challenges in assessment of ICU trainees.

BACKGROUND: Assessing trainees is crucial for development of their competence, yet it remains a challenging endeavour. Identifying contributing and influencing factors affecting this process is imperative for improvement. METHODS: We surveyed residents, fellows, and intensivists working in an intensive care unit (ICU) at a large non-university hospital in Switzerland to investigate the challenges in assessing ICU trainees. Thematic analysis revealed three major themes. RESULTS: Among 45 physicians, 37(82%) responded. The first theme identified is trainee-intensivist collaboration discontinuity. The limited duration of trainees' ICU rotations, large team size operating in a discordant three-shift system, and busy and unpredictable day-planning hinder sustained collaboration. Potential solutions include a concise pre-collaboration briefing, shared bedside care, and post-collaboration debriefing involving formative assessment and reflection on collaboration. The second theme is the lack of trainees' progress visualisation, which is caused by unsatisfactory familiarisation with the trainees' development. The lack of an overview of a trainee's previous achievements, activities, strengths, weaknesses, and goals may result in inappropriate assessments. Participants suggested implementing digital assessment tools, a competence committee, and dashboards to facilitate progress visualisation. The third theme we identified is insufficient coaching and feedback. Factors like personality traits, hierarchy, and competing interests can impede coaching, while high-quality feedback is essential for correct assessment. Skilled coaches can define short-term goals and may optimise trainee assessment by seeking feedback from multiple supervisors and assisting in both formative and summative assessment. Based on these three themes and the suggested solutions, we developed the acronym "ICU-STAR" representing a potentially powerful framework to enhance short-term trainee-supervisor collaboration in the workplace and to co-scaffold the principles of adequate assessment. CONCLUSIONS: According to ICU physicians, trainee-supervisor collaboration discontinuity, the lack of visualisation of trainee's development, and insufficient coaching and feedback skills of supervisors are the major factors hampering trainees' assessment in the workplace. Based on suggestions by the survey participants, we propose the acronym "ICU-STAR" as a framework including briefing, shared bedside care, and debriefing of the trainee-supervisor collaboration at the workplace as its core components. With the attending intensivists acting as coaches, progress visualisation can be enhanced by actively collecting more data points. TRIAL REGISTRATION: N/A.

Positive Health dialogue tool and value-based healthcare: a qualitative exploratory study during residents' outpatient consultations.

OBJECTIVE: To explore how residents experienced the application of the Positive Health dialogue tool (PH-tool) during outpatient consultations and its influence on the delivery of value-based healthcare (VBHC). DESIGN: Qualitative study using non-participant observations of outpatient consultations during which residents used the PH-tool, followed by longitudinal individual, semistructured interviews. To analyse the data from observations and interviews, observational form notes' summarisation and categorisation, and an iterative-inductive thematic approach was used. PARTICIPANTS: Eight residents-five from the ear, nose, and throat-department and three from the gastroenterology-hepatology-department-were selected through convenience sampling, accounting for 79 observations and 79 interviews. RESULTS: Residents had bivalent experiences with using the PH-tool. Residents discussed three main benefits: a gained insight into the individual patient's context and functioning, a changed dynamics in resident-patient communication, and an increased awareness regarding value in terms of patient-related outcomes and healthcare costs. Three barriers became apparent: doubts regarding the PH-tool's relevance and scope, boundaries of superspecialised medical professionals, and a lack of demarcation in clinical practice. CONCLUSION: The PH-tool use can be beneficial for residents during outpatient consultations with new patients and follow-up in cases of multidimensional problems, particularly in cases of chronic conditions and generalist care. In these situations, the tool yielded valuable patient information beyond physical health, helped foster patient engagement, and enabled tailoring the treatment plan to individual patients' needs. On the other hand, the PH-tool was not a good fit for simple problems, clearly demarcated help requests, periodic follow-up consultations, or verbose patients. In addition, it was not suitable for superspecialised care, because it yielded an abundance of general information. For particular patients and problems, using the PH-tool seems a promising strategy to increase VBHC delivery. Nevertheless, further research and detailing is needed to better align the PH-tool's broad intent and clinical practice.

Web-assisted assessment of professional behaviour in problem-based learning: more feedback, yet no qualitative improvement?

Although other web-based approaches to assessment of professional behaviour have been studied, no publications studying the potential advantages of a web-based instrument versus a classic, paper-based method have been published to date. This study has two research goals: it focuses on the quantity and quality of comments provided by students and their peers (two researchers independently scoring comments as correct and incorrect in relation to five commonly used feedback rules (and resulting in an aggregated score of the five scores) on the one, and on the feasibility, acceptability and perceived usefulness of the two approaches on the other hand (using a survey). The amount of feedback was significantly higher in the web-based group than in the paper based group for all three categories (dealing with work, others and oneself). Regarding the quality of feedback, the aggregated score for each of the three categories was not significantly different between the two groups, neither for the interim, nor for the final assessment. Some, not statistically significant, but nevertheless noteworthy trends were nevertheless noted. Feedback in the web-based group was more often unrelated to observed behaviour for several categories for both the interim and final assessment. Furthermore, most comments relating to the category 'Dealing with oneself' consisted of descriptions of a student's attendance, thereby neglecting other aspects of personal functioning. The survey identified significant differences between the groups for all questionnaire items regarding feasibility, acceptability and perceived usefulness in favour of the paper-based form. The use of a web-based instrument for professional behaviour assessment yielded a significantly higher number of comments compared to the traditional paper-based assessment. Unfortunately, the quality of the feedback obtained by the web-based instrument as measured by several generally accepted feedback criteria did not parallel this increase.

Prognosis of patients with haematological malignancies admitted to the intensive care unit: Sequential Organ Failure Assessment (SOFA) trend is a powerful predictor of mortality.

BACKGROUND: The prognosis of patients with haematological malignancies who are admitted to the ICU is generally poor. In order to optimize care, it is important to be able to determine which patients are most likely to benefit from continuation of treatment after ICU admission. METHODS: Data of 86 patients with a haematological malignancy consecutively admitted to the ICU of Maastricht University Medical Centre were examined in a retrospective cohort study in order to identify clinically useful prognostic parameters. RESULTS: ICU mortality was 56% and in-hospital mortality was 65%. Non-survivors had higher APACHE-II and SOFA scores compared with survivors (32±8.0 versus 25±6.5 and 11.5±3.1 versus 8.5±3.0, respectively). The mortality rate was significantly higher in patients with an increasing SOFA score (≥2 points) compared with patients with an unchanged or decreasing SOFA score (72% versus 58% and 21%, respectively). Mortality was also higher in patients requiring invasive mechanical ventilation or inotropic/vasopressor therapy. CONCLUSION: The mortality rate among patients with haematological malignancies who are admitted to the ICU is high and mainly associated with the severity of illness, as reflected by more severe and worsening organ failure and a need for mechanical ventilation or inotropic/vasopressor therapy. Several factors appear to be associated with a poor outcome, but no absolute predictors of mortality could be identified, although the results suggest that changes in the SOFA score during the stay in the ICU can be helpful in the decision making about the continuation or discontinuation of treatment.

Combined formative and summative professional behaviour assessment approach in the bachelor phase of medical school: a Dutch perspective.

BACKGROUND: Teaching and assessment of professional behaviour (PB) has been receiving increasing attention in the educational literature and educational practice. Although the focus tends to be summative aspects, it seems perfectly feasible to combine formative and summative approaches in one procedural approach. AIMS AND METHOD: Although, many examples of frameworks of professionalism and PB can be found in the literature, most originate from North America, and only few are designed in other continents. This article presents the framework for PB that is used at Maastricht medical school, the Netherlands. RESULTS: The approach to PB used in the Dutch medical schools is described with special attention to 4 years (2005-2009) of experience with PB education in the first 3 years of the 6-year undergraduate curriculum of Maastricht medical school. Future challenges are identified. CONCLUSIONS: The adages 'Assessment drives learning' and 'They do not respect what you do not inspect' [Cohen JJ. 2006. Professionalism in medical education, an American perspective: From evidence to accountability. Med Educ 40, 607-617] suggest that formative and summative aspects of PB assessment can be combined within an assessment framework. Formative and summative assessments do not represent contrasting but rather complementary approaches. The Maastricht medical school framework combines the two approaches, as two sides of the same coin.

Approaches to professional behaviour assessment: tools in the professionalism toolbox.

There is general agreement that professionalism and professional behaviour should be (formatively and summatively) assessed, but consensus on how this should be done is still lacking. After discussing some of the remaining issues and questions regarding professionalism assessment, this article discusses the importance of qualitative comments to the assessment of professional behaviour, focuses on the currently most frequently used tools, as well as stresses the need for triangulation (combining) of these tools.

General considerations regarding assessment of professional behaviour.

Core medical knowledge has been assessed for over fifty years and technical and communication skills for at least twenty. The assessment of professionalism however has failed to achieve recognition within this time frame. The interest in the assessment of professionalism and professional behaviour thus is a fairly recent development. This article will firstly clarify how professional behaviour assessment relates to other assessment methods using the framework proposed by Miller6. Thereafter a brief overview will be provided of the current "tool box" of methods available to assess professionalism. Data on the validity, reliability, feasibility, acceptability and educational utility of these "tools" as derived from published evidence will be reviewed. Subsequently a general overview of the way forward in the assessment of professionalism and professional behaviour will be given.

Factors inhibiting assessment of students' professional behaviour in the tutorial group during problem-based learning.

CONTEXT: We addressed the assessment of professional behaviour in tutorial groups by investigating students' perceptions of the frequency and impact of critical incidents that impede this assessment and 5 factors underlying these critical incidents. METHODS: A questionnaire asking students to rate the frequency and impact of 40 critical incidents relating to effective assessment of professional behaviour on a 5-point Likert scale was developed and sent to all undergraduate medical students in Years 2-4 of a 6-year undergraduate curriculum. RESULTS: The response rate was 70% (n = 393). Important factors underlying critical incidents are: lack of effective interaction; lack of thoroughness; tutors' failure to confront students with unprofessional behaviour; lack of effort to find solutions, and lack of student motivation. Confirmatory factor analysis showed a good model fit. Because the relationship between frequency of occurrence and degree of impediment varies, the best information about the true impact of critical incidents and the underlying factors is provided by the product of frequency and degree of impediment. Frequency of occurrence remains stable and degree of impediment increases in Years 2-4. CONCLUSIONS: The results of this study can be used to design and improve faculty development programmes aimed at improving assessment of professional behaviour. Training programmes should motivate tutors by providing background information as to why and how sound assessment of professional behaviour is to be performed and encourage tutors to confront students with and discuss all aspects of professional behaviour, as well as provide appropriate feedback.