Twelve tips for completing quality in-training evaluation reports.

Assessing learners in the clinical setting is vital to determining their level of professional competence. Clinical performance assessments can be documented using In-training evaluation reports (ITERs). Previous research has suggested a need for faculty development in order to improve the quality of these reports. Previous work identified key features of high-quality completed ITERs which primarily involve the narrative comments. This aligns well with the recent discourse in the assessment literature focusing on the value of qualitative assessments. Evidence exists to demonstrate that faculty can be trained to complete higher quality ITERs. We present 12 key strategies to assist clinical supervisors in improving the quality of their completed ITERs. Higher quality completed ITERs will improve the documentation of the trainee's progress and be more defensible when questioned in an appeal or legal process.

The CNDR: collaborating to translate new therapies for Canadians.

BACKGROUND: Patient registries represent an important method of organizing "real world" patient information for clinical and research purposes. Registries can facilitate clinical trial planning and recruitment and are particularly useful in this regard for uncommon and rare diseases. Neuromuscular diseases (NMDs) are individually rare but in aggregate have a significant prevalence. In Canada, information on NMDs is lacking. Barriers to performing Canadian multicentre NMD research exist which can be overcome by a comprehensive and collaborative NMD registry. METHODS: We describe the objectives, design, feasibility and initial recruitment results for the Canadian Neuromuscular Disease Registry (CNDR). RESULTS: The CNDR is a clinic-based registry which launched nationally in June 2011, incorporates paediatric and adult neuromuscular clinics in British Columbia, Alberta, Ontario, Quebec, New Brunswick and Nova Scotia and, as of December 2012, has recruited 1161 patients from 12 provinces and territories. Complete medical datasets have been captured on 460 "index disease" patients. Another 618 "non-index" patients have been recruited with capture of physician-confirmed diagnosis and contact information. We have demonstrated the feasibility of blended clinic and central office-based recruitment. "Index disease" patients recruited at the time of writing include 253 with Duchenne and Becker muscular dystrophy, 161 with myotonic dystrophy, and 71 with ALS. CONCLUSIONS: The CNDR is a new nationwide registry of patients with NMDs that represents an important advance in Canadian neuromuscular disease research capacity. It provides an innovative platform for organizing patient information to facilitate clinical research and to expedite translation of recent laboratory findings into human studies.

Quality evaluation reports: Can a faculty development program make a difference?

BACKGROUND: The quality of medical student and resident clinical evaluation reports submitted by rotation supervisors is a concern. The effectiveness of faculty development (FD) interventions in changing report quality is uncertain. AIMS: This study assessed whether faculty could be trained to complete higher quality reports. METHOD: A 3-h interactive program designed to improve evaluation report quality, previously developed and tested locally, was offered at three different Canadian medical schools. To assess for a change in report quality, three reports completed by each supervisor prior to the workshop and all reports completed for 6 months following the workshop were evaluated by three blinded, independent raters using the Completed Clinical Evaluation Report Rating (CCERR): a validated scale that assesses report quality. RESULTS: A total of 22 supervisors from multiple specialties participated. The mean CCERR score for reports completed after the workshop was significantly higher (21.74 ± 4.91 versus 18.90 ± 5.00, p = 0.02). CONCLUSIONS: This study demonstrates that this FD workshop had a positive impact upon the quality of the participants' evaluation reports suggesting that faculty have the potential to be trained with regards to trainee assessment. This adds to the literature which suggests that FD is an important component in improving assessment quality.

Peer assessment of outpatient consultation letters--feasibility and satisfaction.

BACKGROUND: Written correspondence is one of the most important forms of communication between health care providers, yet there is little feedback provided to specialists. The objective of this study was to determine the feasibility and satisfaction of a peer assessment program on consultation letters and to determine inter-rater reliability between family physicians and specialists. METHODS: A rating scale of nine 5-point Likert scale items including specific content, style items, education value of the letter and an overall rating was developed from a previous validated tool. Nine Internal Medicine specialists/subspecialists from two tertiary care centres submitted 10 letters with patient and physician identifiers removed. Two Internal Medicine specialists, and 2 family physicians from the other centre rated each letter (to protect writer anonymity). A satisfaction survey was sent to each writer and rater after collation of the results. A follow-up survey was sent 6-8 months later. RESULTS: There was a high degree of satisfaction with the process and feedback. The rating scale information was felt to be useful and appropriate for evaluating the quality of consultation letters by 6/7 writers. 5/7 seven writers felt that the feedback they received resulted in immediate changes to their letters. Six months later, 6/9 writers indicated they had maintained changes in their letters. Raters rank ordered letters similarly (Cronbach's alpha 0.57-0.84) but mean scores were highly variant. At site 1 there were significant differences in scoring brevity (p < 0.01) between family physician and specialist raters; whereas, at site 2 there were differences in scoring of history (p < 0.01), physical examination (p < 0.01) and educational value (p < 0.01) of the letter. CONCLUSION: Most participants found peer assessment of letters feasible and beneficial and longstanding changes occurred in some individuals. Family physicians and specialists appear to have different expectations on some items. Further studies on reliability and validity, with a larger sample, are required before high stakes professional assessments include consultation letters.

Feedback to Supervisors: Is Anonymity Really So Important?

PURPOSE: Research demonstrates that physicians benefit from regular feedback on their clinical supervision from their trainees. Several features of effective feedback are enabled by nonanonymous processes (i.e., open feedback). However, most resident-to-faculty feedback processes are anonymous given concerns of power differentials and possible reprisals. This exploratory study investigated resident experiences of giving faculty open feedback, advantages, and disadvantages. METHOD: Between January and August 2014, nine graduates of a Canadian Physiatry residency program that uses open resident-to-faculty feedback participated in semistructured interviews in which they described their experiences of this system. Three members of the research team analyzed transcripts for emergent themes using conventional content analysis. In June 2014, semistructured group interviews were held with six residents who were actively enrolled in the program as a member-checking activity. Themes were refined on the basis of these data. RESULTS: Advantages of the open feedback system included giving timely feedback that was acted upon (thus enhancing residents' educational experiences), and improved ability to receive feedback (thanks to observing modeled behavior). Although some disadvantages were noted, they were often speculative (e.g., "I think others might have felt …") and were described as outweighed by advantages. Participants emphasized the program's "feedback culture" as an open feedback enabler. CONCLUSIONS: The relationship between the feedback giver and recipient has been described as influencing the uptake of feedback. Findings suggest that nonanonymous practices can enable a positive relationship in resident-to-faculty feedback. The benefits of an open system for resident-to-faculty feedback can be reaped if a "feedback culture" exists.

Can written communication skills be tested in an objective structured clinical examination format?

PURPOSE: To design and evaluate an objective structured clinical examination (OSCE) station on dictating a consult letter as part of a formative OSCE for internal medicine residents. METHOD: A 22-minute station for the dictation of a consult letter was included in a ten-station OSCE. Two raters completed a 34-item rating scale for 36 letters. The rating scale involved content and style. The station's score was derived from an overall rating of each section of the letter (history, physical examination, impression/plan) and a global rating of the complete letter. The exam also contained a physical exam station on the same patient problem and a verbal communication station. Residents provided written feedback following the station. RESULTS: The inter-rater reliability for the station's score was .72. The generalizability coefficient for the two-rater four-part rating scale was .79. The correlation between the consult letter and the total exam scores was .56, the highest for all ten stations. A significant correlation existed between the verbal communication station's score and the letter station's score (r =.37, p <.005), but no correlation occurred between the physical exam station's score on a similar patient problem and the letter station's score. The feedback from residents was favorable regarding amount of information provided and time allotted. CONCLUSION: An OSCE station is a feasible way to examine written communication skills. Letter-writing skills appear to be distinct from knowledge (physical exam station), but somewhat linked to verbal communication skills.

Writing effective consultation letters: 12 tips for teachers.

Written correspondence is the standard mode of communication between healthcare providers. Despite the importance of this skill and increased emphasis on ambulatory care, communication skills and professionalism in training programs, there has been very little written on the teaching and evaluation of consultation letter writing. Consultation letter writing is an essential skill that cannot be learned simply by reading others' letters and should be taught in a formal manner. This article describes the authors' experience in teaching the skill of writing effective consultation letters to residents and describes strategies for evaluating this skill.

Communication skills, cultural challenges and individual support: challenges of international medical graduates in a Canadian healthcare environment.

Physicians require good communication skills to develop effective patient-physician relationships. Externally funded international medical graduates (IMGs) move directly from their home countries to complete residency training at the University of Ottawa, Canada. They must learn quickly how to work with patients, families and colleagues. A detailed needs assessment was designed to assess IMGs' communication skill needs through focus groups, interviews and surveys with IMGs, program directors, allied healthcare professionals and experts in communication skills. There was a high degree of consensus amongst all participants concerning specific educational needs for communication skills and training issues related to the healthcare system for externally funded IMGs. Specific recommendations include (1) English-language skills; (2) how to get things done in the hospital/healthcare system; (3) opportunities to practise specific skills, e.g. negotiating treatment, (4) adequate support system for IMGs; (5) faculty and staff education on the cultural challenges faced by IMGs.