The making of a (new) surgeon: Opportunities for repeated messaging on NOTSS in the Canadian medical system.

Proficiency in nontechnical skills (NTS) contributes to reduction in critical safety incidents and improvement in patient safety outcomes. Despite evidence demonstrating the importance of NTS in patient safety, there remains limited NTS specific curricula and formal teaching in Canadian surgical programs. We propose a three-stage longitudinal approach to education surrounding NTS using the Nontechnical skills for surgeons (NOTSS) framework.

Examining the equity and diversity characteristics of academic general surgeons hired in Canada.

BACKGROUND: Job competition and underemployment among surgeons emphasize the importance of equitable hiring practices. The purpose of this study was to describe some of the demographic characteristics of academic general surgeons and to evaluate the gender and visible minority (VM) status of those recently hired. METHODS: Demographic information about academic general surgeons across Canada including gender, VM status, practice location and graduate degree status was collected. Location of residency was collected for recently hired general surgeons (hired between 2013 and 2020). Descriptive statistics were performed on the demographic characteristics at each institution. Pearson correlation coefficients and hypothesis testing were used to determine the correlation between various metrics and gender and VM status. RESULTS: A total of 393 general surgeons from 30 academic hospitals affiliated with 14 universities were included. The percentage of female general surgeons ranged from 0% to 47.4% and the percentage of VM general surgeons ranged from 0% to 66.7% at the hospitals. This heterogeneity did not correlate with city population (gender: r = 0.06, p = 0.77; VM: r = 0.04, p = 0.83). The percentage of VM general surgeons at each hospital did not correlate with the percentage of VM population in the city (r = 0.13, p = 0.49). Only 34 of 120 recently hired academic general surgeons (28.3%) did not have a graduate degree. The percentage of recently hired academic general surgeons who did not have a graduate degree was approximately 1.5 times higher among male hirees than female hirees. With respect to academic promotion, the percentage of female full professors ranged from 0% to 40.0% and did not correlate with the percentage of female general surgeons at each institution (r = 0.11, p = 0.70). The percentage of VM full professors ranged from 0% to 44.4% and was moderately correlated with the percentage of VM surgeons at each institution (r = 0.40, p = 0.16). CONCLUSION: The academic general surgery workforce appears to be somewhat diverse. However, there was substantial heterogeneity in diversity between hospitals, leaving room for improvement. We must be willing to examine our hiring processes and be transparent about them to build an equitable surgical workforce.

When Illness and Loss Hit Close to Home-Do Health Care Providers Learn How to Cope?

INTRODUCTION: Up to 85% of newly qualified physicians report loss or illness in themselves or a loved one. These experiences can intensify feelings of grief in the professional setting, but the range of formal training that addresses personal illness or loss is unknown. This study aimed to explore interventions that teach health care providers and trainees about personal illness experience. METHODS: A scoping review was conducted by searching three bibliographic databases using the terms "illness," "personal," "education," and synonyms. Article screening was performed in duplicate to identify studies that described an intervention that included teaching or learning on personal experiences with illness or loss for health care providers and trainees. RESULTS: The search yielded 4168 studies, of which 13 were included. Education most often targeted medical students (54%), resident or attending physicians (31%), and nurses (31%). Other participants included social workers and psychologists. Personal illness was most frequently taught for reflection in the context of palliative care curricula (54%). Only two studies' primary purpose was to teach about coping with grief related to personal experiences. No studies within the scope of our defined methodology described training on how to support colleagues or trainees facing personal illness or loss. Reported outcomes included improved coping skills, decreased stress, and better ability to support bereaving patients. DISCUSSION: Specific training on personal illness experience is limited, with gaps in continuity of learning, particularly for continuing medical education. Future curricula can equip providers with coping strategies while enabling improved resilience and patient care.

The effect of assessor professional background on interview evaluation during residency selection: A mixed-methods study.

BACKGROUND: Residency interviewer scores are greatly variable and seems to be influenced by personal characteristics of assessors, although factors contributing to variability remain unclear. The study sought to determine how different professional backgrounds influence assessors' scores. METHODS: Fifty-five general surgery applicants rotated through an interview station assessing teamwork. They were scored by surgeons, human-resource managers, pilots, athletes. Pearson's correlation and a repeated-measures ANOVA were used to determine correlations between professions. Structured interviews were used to probe for scoring rationale. RESULTS: Interview scores differed significantly between professions (F (3, 159) = 11.12, p < 0.001. Qualitative analysis revealed that due to the challenge of distinguishing between similarly performing candidates, assessors rely on global impressions informed by personal values. CONCLUSION: Assessor variability is ubiquitous, in part due to the subjective nature of interviews and is associated with personal values. When selecting assessors, programs should choose diverse assessors to assess to ensure a reliable selection process.

Assessment of laparoscopic skills: comparing the reliability of global rating and entrustability tools.

Background: Competence by design (CBD) residency programs increasingly depend on tools that provide reliable assessments, require minimal rater training, and measure progression through the CBD milestones. To assess intraoperative skills, global rating scales and entrustability ratings are commonly used but may require extensive training. The Competency Continuum (CC) is a CBD framework that may be used as an assessment tool to assess laparoscopic skills. The study aimed to compare the CC to two other assessment tools: the Global Operative Assessment of Laparoscopic Skills (GOALS) and the Zwisch scale. Methods: Four expert surgeons rated thirty laparoscopic cholecystectomy videos. Two raters used the GOALS scale while the remaining two raters used both the Zwisch scale and CC. Each rater received scale-specific training. Descriptive statistics, inter-rater reliabilities (IRR), and Pearson's correlations were calculated for each scale. Results: Significant positive correlations between GOALS and Zwisch (r = 0.75, p < 0.001), CC and GOALS (r = 0.79, p < 0.001), and CC and Zwisch (r = 0.90, p < 0.001) were found. The CC had an inter-rater reliability of 0.74 whereas the GOALS and Zwisch scales had inter-rater reliabilities of 0.44 and 0.43, respectively. Compared to GOALS and Zwisch scales, the CC had the highest inter-rater reliability and required minimal rater training to achieve reliable scores. Conclusion: The CC may be a reliable tool to assess intraoperative laparoscopic skills and provide trainees with formative feedback relevant to the CBD milestones. Further research should collect further validity evidence for the use of the CC as an independent assessment tool.

Contexte: Les programmes de résidence structurés autour de la compétence par conception (CPC) dépendent de plus en plus d'outils qui fournissent des évaluations fiables, nécessitent une formation minimale des évaluateurs et mesurent la progression dans les étapes de la CPC. Pour évaluer les compétences peropératoires, les échelles d'évaluation globale et de confiance sont couramment utilisées mais peuvent nécessiter une formation approfondie. Le Continuum des compétences (CC) est un cadre de la CPC qui peut être utilisé comme outil d'évaluation des compétences laparoscopiques. L'étude visait à comparer le CC à deux autres outils d'évaluation : l'évaluation globale opératoire des compétences laparoscopiques (GOALS) et l'échelle de Zwisch. Méthodes: Quatre chirurgiens experts ont évalué trente vidéos de cholécystectomie laparoscopique. Deux évaluateurs ont utilisé l'échelle GOALS tandis que les deux autres ont utilisé l'échelle Zwisch et le CC. Chacun d'eux avait reçu une formation spécifique à l'échelle utilisée. Des statistiques descriptives, la fiabilité inter-évaluateurs (FIÉ) et des corrélations de Pearson ont été calculées pour chaque échelle. Résultats: Des corrélations positives significatives ont été trouvées entre les échelles GOALS et Zwisch (r=0.75, p<0.001), CC et GOALS (r=0.79, p<0.001), et CC et Zwisch (r=0.90, p<0.001). Le CC avait une fiabilité inter-évaluateurs de 0,74 tandis que les échelles GOALS et Zwisch avaient des fiabilités inter-évaluateurs de 0,44 et 0,43, respectivement. Par rapport aux échelles GOALS et Zwisch, le CC avait la fiabilité inter-évaluateurs la plus élevée et ne nécessitait qu'une formation minimale des évaluateurs pour obtenir des scores fiables. Conclusion: Le CC constituerait un outil fiable pour évaluer les compétences laparoscopiques peropératoires et pour fournir aux stagiaires une rétroaction formatrice pertinente pour les étapes de la CPC. Des recherches supplémentaires devraient être entreprises pour recueillir plus de preuves de validité pour l'utilisation du CC comme outil d'évaluation indépendant.

Trust Me, I Know Them: Assessing Interpersonal Bias in Surgery Residency Interviews.

Background: Residency selection integrates objective and subjective data sources. Interviews help assess characteristics like insight and communication but have the potential for bias. Structured multiple mini-interviews may mitigate some elements of bias; however, a halo effect is described in assessments of medical trainees, and degree of familiarity with applicants may remain a source of bias in interviews. Objective: To investigate the extent of interviewer bias that results from pre-interview knowledge of the applicant by comparing file review and interview scores for known versus unknown applicants. Methods: File review and interview scores of applicants to the University of Ottawa General Surgery Residency Training Program from 2019 to 2021 were gathered retrospectively. Applicants were categorized as "home" if from the institution, "known" if they completed an elective at the institution, or "unknown." The Kruskal-Wallis H test was used to compare median interview scores between groups and Spearman's rank-order correlation (rs) to determine the correlation between file review and interview scores. Results: Over a 3-year period, 169 applicants were interviewed; 62% were unknown, 31% were known, and 6% were home applicants. There was a statistically significant difference (P=.01) between the median interview scores of home, known, and unknown applicants. Comparison of groups demonstrated higher positive correlations between file review and interview scores (rs=0.15 vs 0.36 vs 0.55 in unknown, known, and home applicants) with increasing applicant familiarity. Conclusions: There is an increased positive correlation between file review and interview scores with applicant familiarity. The interview process may carry inherent bias insufficiently mitigated by the current structure.

Self-Assessment in General Surgery Applicants: An Insight Into Interview Performance.

INTRODUCTION: Selecting medical students for residency is a competitive process, with a narrow range of scores separating middle-ranked applicants. Self-assessment is a fundamental skill for any competent physician with a demonstrated correlation to diagnostic ability, examination scores, and technical skills, but has yet to be investigated in residency selection. The objective of this study was to investigate the relationship between self-assessment and interview performance as a potential adjunct to discriminate between applicants. METHODS: At the University of Ottawa in 2020, 55 applicants completed a 9-station interview circuit assessing different characteristics or skills important for a career in general surgery, followed by a self-assessment questionnaire evaluating their perceived performance at each station. Pearson's correlation was used to determine the relationship between self-assessment scores (SASs) and interviewer scores (ISs). RESULTS: There was a negative correlation between SASs and ISs for all interview stations. High performers underestimated their interview performance, and low performers overestimated their performance. Seven of the nine stations reached statistical significance (r = 0.60-0.73, P < 0.001). There was significant variability in the SAS of middle-ranked applicants, with a range three times greater than the range of ISs and demonstrating distinct self-assessment skills in candidates with very similar scores. CONCLUSIONS: Although we strive to select applicants who will succeed in residency to become competent physicians, self-assessment skills may be a useful adjunct during the interview process to assist in discriminating between applicants with similar scores.

A chance for reform: The environmental impact of travel for general surgery residency interviews.

BACKGROUND: In light of the global climate emergency, it is worth reconsidering the current practice of medical students traveling to interview for residency positions. We sought to estimate carbon dioxide (CO2) emissions associated with travel for general surgery residency interviews in Canada, and the potential avoided emissions if interviews were restructured. METHODS: An eight-item survey was constructed to collect data on cities visited, travel modalities, and costs incurred. Applicants to the University of Ottawa General Surgery Program during the 2019/20 Canadian Resident Matching Service (CaRMS) cycle were invited to complete the survey. Potential reductions in CO2 emissions were modeled using a regionalized interview process with either one or two cities. RESULTS: Of a total of 56 applicants, 39 (70%) completed the survey. Applicants on average visited 10 cities with a mean total cost of $4,866 (95% CI=3,995-5,737) per applicant. Mean CO2 emissions were 1.82 (95% CI=1.50-2.14) tonnes per applicant, and the total CO2 emissions by applicants was estimated to be 101.9 (95% CI=84.0 - 119.8) tonnes. In models wherein interviews are regionalized to one or two cities, emissions would be 57.9 tonnes (43.2% reduction) and 84.2 tonnes (17.4% reduction), respectively. Overall, 74.4% of respondents were concerned about the environmental impact of travel and 46% would prefer to interview by videoconference. CONCLUSION: Travel for general surgery residency interviews in Canada is associated with a considerable environmental impact. These findings are likely generalizable to other residency programs. Given the global climate crisis, the CaRMS application process must consider alternative structures.

CONTEXTE: Compte tenu de la situation d'urgence climatique mondiale, il convient de reconsidérer l'usage actuel selon lequel les étudiants en médecine se déplacent pour se présenter aux entrevues en vue d'obtenir un poste de résidence. Nous avons tenté d'estimer les émissions de dioxyde de carbone (CO2) causées par les déplacements pour les entretiens de résidence en chirurgie générale au Canada, et les émissions potentielles évitées si les entretiens étaient organisés autrement. MÉTHODES: Un sondage comportant huit questions a été élaboré pour recueillir les données sur les villes visitées, les modalités de voyage et les coûts encourus. Les candidats au programme de chirurgie générale de l'Université d'Ottawa au cours du cycle 2019/20 du Service canadien de jumelage des résidents (CaRMS) ont été invités à y répondre. Les réductions potentielles des émissions de CO2 ont été modélisées à l'aide d'un processus d'entrevue régionalisé avec une ou deux villes. RÉSULTATS: Sur un total de 56 candidats, 39 (70 %) ont répondu au sondage. Les candidats ont visité en moyenne 10 villes, pour un coût total moyen de 4 866 dollars (IC 95 % = 3 995-5 737) par candidat. Les émissions moyennes de CO2 étaient de 1,82 (IC 95 % = 1,50-2,14) tonne par candidat, et le total des émissions de CO2 pour l'ensemble des candidats était estimé à 101,9 (IC 95 % = 84,0 - 119,8) tonnes. D'après les modèles où les entrevues sont régionalisées avec une ou deux villes, les émissions seraient respectivement de 57,9 tonnes (43,2 % de réduction) et 84,2 tonnes (17,4 % de réduction). Dans l'ensemble, 74,4 % des personnes interrogées se disent préoccupées par l'impact environnemental des déplacements et 46 % préféreraient que l'entretien se fasse par vidéoconférence. CONCLUSION: Les déplacements pour les entrevues de résidence en chirurgie générale au Canada ont un impact environnemental considérable. Ces conclusions sont probablement généralisables à d'autres programmes de résidence. Compte tenu de la crise climatique mondiale, il conviendrait d'envisager d'autres modalités d'organisation des entrevues pour le processus de candidatures du CaRMS.

The Impact of Surgeon Experience on Script Concordance Test Scoring.

OBJECTIVE: The Script Concordance Test (SCT) is a test of clinical decision-making that relies on an expert panel to create its scoring key. Existing literature demonstrates the value of specialty-specific experts, but the effect of experience among the expert panel is unknown. The purpose of this study was to explore the role of surgeon experience in SCT scoring. DESIGN: An SCT was administered to 29 general surgery residents and 14 staff surgeons. Staff surgeons were stratified as either junior or senior experts based on years since completing residency training (<15 versus >25 years). The SCT was scored using the full expert panel, the senior panel, the junior panel, and a subgroup junior panel in practice <5 years. A one-way ANOVA was used to compare the scores of first (R1) and fifth (R5) year residents using each scoring scheme. Cognitive interviews were analyzed for differences between junior and senior expert panelist responses. RESULTS: There was no statistically significant difference between the mean score of six R1s and five R5s using the full expert panel (R1 69.08 versus R5 67.06, F1,9 = 0.10, P = 0.76), the junior panel (R1 66.73 versus R5 62.50, F1,9 = 0.35, P = 0.57), or the subgroup panel in practice <5 years (R1 61.07 versus R5 58.79, F1,9 = 0.18, P = 0.75). However, the average score of R1s was significantly lower than R5s when using the senior faculty panel (R1 52.04 versus R5 63.26, F1,9 = 26.90, P = 0.001). Cognitive interview data suggests that some responses of junior experts demonstrate less confidence than those of senior experts. CONCLUSIONS: SCT scores are significantly affected by the responses of the expert panel. Expert differences between first and fifth year residents were only demonstrated when using an expert panel consisting of senior faculty members. Confidence may play a role in the response selections of junior experts. When constructing an SCT expert panel, consideration must be given to the experience of panel members.

Comparing 2 Approaches for the File Review of Residency Applications.

BACKGROUND: The residency selection process relies on subjective information in applications, as well as subjective assessment of applications by reviewers. This inherent subjectivity makes residency selection prone to poor reliability between those reviewing files. OBJECTIVES: We compared the interrater reliability of 2 assessment tools during file review: one rating applicant traits (ie, leadership, communication) and the other using a global rating of application elements (ie, curriculum vitae, reference letters). METHODS: Ten file reviewers were randomized into 2 groups, and each scored 7 general surgery applications from the 2019-2020 cycle. The first group used an element-based (EB) scoring tool, while the second group used a trait-based (TB) scoring tool. Feedback was collected, discrimination capacities were measured using variation in scores, and interrater reliability (IRR) was calculated using intraclass correlation (ICC) in a 2-way random effects model. RESULTS: Both tools identified the same top-ranked and bottom-ranked applicants; however, discrepancies were noted for middle-ranked applicants. The score range for the 5 middle-ranked applicants was greater with the TB tool (6.43 vs 3.80), which also demonstrated fewer tie scores. The IRR for TB scoring was superior to EB scoring (ICC [2, 5] = 0.82 vs 0.55). The TB tool required only 2 raters to achieve an ICC ≥ 0.70. CONCLUSIONS: Using a TB file review strategy can facilitate file review with improved reliability compared to EB, and a greater spread of candidate scores. TB file review potentially offers programs a feasible way to optimize and reflect their institution's core values in the process.

Nontechnical Skills (NTS) in the Undergraduate Surgical and Anesthesiology Curricula: Are We Adequately Preparing Medical Students?

OBJECTIVES: Nontechnical skills (NTS) encompass interpersonal, cognitive, and personal resource skills that can mitigate surgical errors and improve patient outcomes. However, inconsistencies in medical student awareness around NTS suggest limited exposure to these skills. This study aimed to determine the prevalence and content of NTS in medical school surgery and anesthesiology education. DESIGN AND SETTING: Learning objectives from clerkship core surgery and anesthesiology rotations were collected from Canadian anglophone medical schools. Two raters independently classified each objective under one of the Non-Technical Skills for Surgeons (NOTSS) or Anaesthetists' Non-Technical Skills (ANTS) "Categories" and "Elements" of NTS, or as a non-NTS objective. Rater disagreements were resolved by group consensus. Group discussion was also held to identify examples of objectives that could help develop future curricula. Descriptive statistics were used to determine the number of NTS objectives from each school and within each NOTSS and ANTS Categories and Elements. RESULTS: Learning objectives were obtained from 12 out of 14 Canadian medical schools. A total of 2116 surgery objectives and 571 anesthesiology objectives were reviewed. Of these, 16 (0.76%) and 26 (4.55%) were identified as NTS objectives in surgery and anesthesiology, respectively. Of the NOTSS and ANTS Categories, "Situation Awareness" and "Decision Making" were represented by only one objective each in both specialties. Approximately half of the NOTSS and ANTS Elements were not represented by a single objective. Group discussion yielded examples of NTS objectives that were excellent, could use improvement, or were too vague to be classified as NTS. CONCLUSIONS: A paucity of objectives in the clerkship perioperative curricula involve NTS. These findings suggest that NTS are unlikely being adequately introduced as critical skillsets of surgeons and anesthesiologists in undergraduate perioperative education. Future curriculum development should involve greater medical student exposure to NTS as key components of their surgery and anesthesiology education.

How do cognitive processes influence script concordance test responses?

INTRODUCTION: The script concordance test (SCT) is a test of clinical decision-making (CDM) that compares the thought process of learners to that of experts to determine to what extent their cognitive 'scripts' align. Without understanding test-takers' cognitive process, however, it is unclear what influences their responses. The objective of this study was to gather response process validity evidence by studying the cognitive process of test-takers to determine whether the SCT tests CDM and what cognitive processes may influence SCT responses. METHODS: Cases from an SCT used in a national validation study were administered and semi-structured cognitive interviews were conducted with ten residents and five staff surgeons. A retrospective verbal probing technique was used. Data was independently analysed and coded by two analysts. Themes were identified as factors that influence SCT responses during the cognitive interview. RESULTS: Cognitive interviews demonstrated variability in CDM among test-takers. Consistent with dual process theory, test-takers relied on scripts formed through past experiences, when available, to make decisions and used conscious deliberation in the absence of experience. However, test-takers' response process was also influenced by their comprehension of specific terms, desire for additional information, disagreement with the planned management, underlying knowledge gaps and desire to demonstrate confidence or humility. CONCLUSION: The rationale behind SCT answers may be influenced by comprehension, underlying knowledge and social desirability in addition to formed scripts and/or conscious deliberation. Having test-takers verbalise their rationale for responses provides a depth of assessment that is otherwise lost in the SCT's current format. With the improved ability to standardise CDM assessment using the SCT, consideration of test-makers improving the SCT construction process and combining the SCT question format with verbal responses may improve the use of the SCT for CDM assessment.

Everyone Is Awesome: Analyzing Letters of Reference in a General Surgery Residency Selection Process.

BACKGROUND: The resident selection process involves the analysis of multiple data points, including letters of reference (LORs), which are inherently subjective in nature. OBJECTIVE: We assessed the frequency with which LORs use quantitative terms to describe applicants and to assess whether the use of these terms reflects the ranking of trainees in the final selection process. METHODS: A descriptive study analyzing LORs submitted by Canadian medical graduate applicants to the University of Ottawa General Surgery Program in 2019 was completed. We collected demographic information about applicants and referees and recorded the use of preidentified quantitative descriptors (eg, best, above average). A 10% audit of the data was performed. Descriptive statistics were used to analyze the demographics of our letters as well as the frequency of use of the quantitative descriptors. RESULTS: Three hundred forty-three LORs for 114 applicants were analyzed. Eighty-five percent (291 of 343) of LORs used quantitative descriptors. Eighty-four percent (95 of 113) of applicants were described as above average, and 45% (51 of 113) were described as the "best" by at least 1 letter. The candidates described as the "best" ranked anywhere from second to 108th in our ranking system. CONCLUSIONS: Most LORs use quantitative descriptors. These terms are generally positive, and while the use does discriminate between different applicants, it was not helpful in the context of ranking applicants in our file review process.

Prioritizing resident and patient safety while maintaining educational value: emergency restructuring of a Canadian surgical residency program during COVID-19.

Summary: Surgical programs are facing major and fluctuating changes to the resident workforce because of decreased elective volumes and high exposure risk during the coronavirus disease 2019 pandemic. Rapid restructuring of a residency program to protect its workforce while maintaining educational value is imperative. We describe the experience of the Division of General Surgery at the University of Ottawa in Ontario, Canada. The residency program was restructured to feature alternating "on" and "off" weeks, maintaining a healthy resident cohort in case of exposure. Teams were restructured and subdivided to maximize physical distancing and minimize resident exposure to pathogens. Educational initiatives doubled, with virtual sessions targeting every resident year and incorporating intraoperative teaching. The divisional research day and oral exams proceeded uninterrupted, virtually. A small leadership team enabled fast and flexible restructuring of a system for patient care while prioritizing resident safety and maintaining a commitment to resident education in a pandemic.

Gender and academic promotion of Canadian general surgeons: a cross-sectional study.

BACKGROUND: Gender disparities in faculty rank have yet to be studied among Canadian physicians. The purpose of this study was to determine whether differences in region, training, research productivity and years in practice explain gender differences in academic promotion among Canadian general surgeons. METHODS: We developed a cross-sectional database of faculty-appointed general surgeons practising in the hospitals affiliated with the 17 universities within the Association of Faculties of Medicine of Canada in 2017 using publicly available directories, university and hospital websites, and direct communication. The data were collected between October and December 2018 and included gender, residency completion year, graduate education, fellowships, number of publications and Scopus h-index; faculty lists and professorship status were verified by program administrators or division heads of their respective divisions. The dependent variable was binary: full professor or not. A combined outcome of associate or full professor was also analyzed. We analyzed all variables in a multivariable logistic regression model. RESULTS: Of the 17 institutions contacted, all but 1 confirmed the faculty lists and professorship status. A total of 405 surgeons were included, of whom 111 (27.4%) were women. Sixty-eight women (61.3%) and 120 men (40.8%) were assistant professors, and 9 women (8.1%) and 75 men (25.5%) were full professors. Although on average women had completed residency more recently than men (15.2 yr v. 19.2 yr, p < 0.001), there was no difference between men and women in the mean number of publications as residents (2.98 v. 2.74, p = 0.7) or per year of practice (3.12 v. 2.09, p = 0.2), number of fellowships pursued (p = 0.7) or graduate education (p = 0.2). In the multivariable model (C-statistic = 0.88), gender remained significantly associated with full professorship (odds ratio 2.79, 95% confidence interval 1.13 to 6.92), along with years in practice (odds ratio 1.61, 95% confidence interval 1.13 to 2.30). INTERPRETATION: After controlling for years in practice, training and research productivity measures, we found that female surgeons with faculty appointments in Canada were less likely than their male counterparts to receive promotion to full professor. Pervasive inequities in systems of promotion must be addressed.

The cognitive process of test takers when using the script concordance test rating scale.

CONTEXT: Clinical decision making (CDM) skills are important to learn and assess in order to establish competence in trainees. A common tool for assessing CDM is the script concordance test (SCT), which asks test takers to indicate how a new clinical finding influences a proposed plan using a Likert-type scale. Most criticisms of the SCT relate to its rating scale but are largely theoretical. The cognitive process of test takers when selecting their responses using the SCT rating scale remains understudied, but is essential to gathering validity evidence for use of the SCT in CDM assessment. METHODS: Cases from an SCT used in a national validation study were administered to 29 residents and 14 staff surgeons. Semi-structured cognitive interviews were then conducted with 10 residents and five staff surgeons based on the SCT results. Cognitive interview data were independently coded by two data analysts, who specifically sought to elucidate how participants mapped their internally generated responses to any of the rating scale options. RESULTS: Five major issues were identified with the response matching cognitive process: (a) the meaning of the '0' response option; (b) which response corresponds to agreement with the planned management; (c) the rationale for picking '±1' versus '±2'; (d) which response indicates the desire to undertake the planned management plus an additional procedure, and (e) the influence of time on response selection. CONCLUSIONS: Studying how test takers (experts and trainees) interpret the SCT rating scale has revealed several issues related to inconsistent and unintended use. Revising the scale to address the variety of interpretations could help to improve the response process validity of the SCT and therefore improve the SCT's ability to be used in CDM skills assessments.