[Script concordance test for knowledge assessment of lung cancer screening]. / Évaluation des connaissances sur le dépistage du cancer par test de concordance de script.

BACKGROUND: Annual screening for lung cancer using low-dose CT-scans is associated with decreased mortality. A survey conducted in Rhône-Alpes area in France found that clinicians need education and information on this topic. Script concordance tests (SCT) are a tool for assessing clinical reasoning in situations of uncertainty. They have not previously been used in France in the context of continuing medical education. METHOD: We created a questionnaire with 5 multiple-choice questions (MCQ) and two SCT scenarios. The questionnaire was sent to all clinicians and residents who are members of French-Speaking Respiratory Society or the French Young Pulmonologist Association. RESULTS: One hundred and ninety answers were analyzed. Seventy percent stated that decreasing mortality was the best criterion for assessing the effectiveness of a cancer screening policy, and 75% that low-dose CT scan was the best test to achieve this in lung cancer screening. Forty-five percent knew the eligibility criteria of the population, and 62% that low-dose CT scan should be performed annually. Participation in tumor boards and certification in oncology were significantly associated with a better score at MCQ and SCT. SCT and MCQ scores were significantly correlated (Spearman's Rho 0.339; P<0.0001). CONCLUSION: SCT are feasible by electronic survey and seem relevant. Improving knowledge of clinicians on lung cancer screening is still critical.

Script concordance testing in continuing professional development: local or international reference panels?

Context The PRACTICUM Institute has developed large-scale international programs of on-line continuing professional development (CPD) based on self-testing and feedback using the Practicum Script Concordance Test© (PSCT). Aims To examine the psychometric consequences of pooling the responses of panelists from different countries (composite panels) and the effect of increasing composite panel size. Method 97 cardiologists in Mexico answered a set of 62 PSCT cases/305 questions. A local panel was recruited in Mexico (n = 7). Other panelists were recruited in Argentina (n = 10) and Brazil (n = 11). Together they constituted a composite panel of 28 experts. Random panels of reference of increasing sizes (5, 10, 15, 20, and 25) were generated. Participants' scores were computed for each panel sample. Units of analysis were means of participants' scores per case. Discrimination, ranking and reliability of the scores obtained with each panel were estimated. Descriptive statistics, Pearson correlation coefficient, generalizability analysis, computation of Cronbach's alpha were used in the analyses. Results Correlation coefficients between the local and the composite panels ranged from 0.951 to 0.981. Cronbach alpha coefficient values were above 0.85 for all panels. The value of the relative G coefficient from the generalizability analyses varied from 0.91 to 0.93, indicating very high and stable ranking of participants, though absolute value of scores increased with increasing composite panel size. Conclusions In CPD environments, and with panel members selected with the highest standards, composite panels can be used. Panels of all sizes yielded high psychometric qualities. Absolute scores should be interpreted with caution.

Assessment in the context of uncertainty: how many members are needed on the panel of reference of a script concordance test?

PURPOSE: The script concordance test (SCT) assesses clinical reasoning in the context of uncertainty. Because there is no single correct answer, scoring is based on a comparison of answers provided by examinees with those provided by members of a panel of reference made up of experienced practitioners. This study aims to determine how many members are needed on the panel to obtain reliable scores to compare against the scores of examinees. METHODS: A group of 80 residents were tested on 73 items (Cronbach's alpha: 0.76). A total of 38 family doctors made up the pool of experienced practitioners, from which 1000 random panels of reference of increasing sizes (5, 10, 15, 20, 25 and 30) were generated with a resampling procedure. Residents' scores were computed for each panel sample. Units of analysis were means of residents' score, test reliability coefficient and correlation coefficient between scores obtained with a given panel of reference versus the scores obtained with the full panel of 38. Statistics were averaged across the 1000 samples for each panel size for the mean and test reliability computations, and across 100 samples for the correlation computation. RESULTS: For sample variability, there was a 3-fold increase in standard deviation of means between a sample panel size of 5 (SD=1.57) and a panel size of 30 (SD=0.50). For reliability, there was a large difference in precision between a panel size of 5 (0.62) and a panel size of 10 (0.70). When the panel size was over 20, the gain became negligible (0.74 for 20 and 0.76 for 38). For correlation, the mean correlation coefficient values were 0.90 with 5 panel members, 0.95 with 10 members and 0.98 with 20 members. CONCLUSION: Any number over 10 is associated with acceptable reliability and good correlation between the samples versus the full panel of 38. For high stake examinations, using a panel of 20 members is recommended. Recruiting more than 20 panel members shows only a marginal benefit in terms of psychometric properties.

[Self-assessment for neurosurgery residents by script concordance test (SCT). The process of test elaboration]. / Auto-évaluation des internes en neurochirurgie par tests de concordance de script (TCS). Processus d'élaboration des tests.

BACKGROUND: The script concordance test (SCT) is a new tool to assess clinical reasoning in a context of uncertainty. It was chosen for computer-based self-assessment training for neurosurgery residents. PURPOSE: The aim of this work is to describe the construction of a first series of questions and provide teachers practical information needed to build a SCT. METHOD: The theorical background of the SCT is explained. We review the steps we followed when writing of a series of more than two hundred questions. RESULTS: We present the different steps of our work, including writing clinical cases, validation of questions and elaboration of scoring systems. We explain the main difficulties we encountered. CONCLUSION: The script concordance test is an interesting easy-to-construct assessment tool useful for evaluating clinical competence. This work summarizes our experience in building a SCT, which should be helpful for teachers interested in this new assessment tool, especially in neurosurgery.

[Selection and training in Canadian ophthalmology residency. A concept to be meditated?]. / La sélection et la formation des ophtalmologistes au Canada.

The authors describe in detail the modalities for the selection of students applying to the ophthalmology programs in the various Canadian universities; the university of Montréal will serve as a model. The characteristics of their 5-year training assured by numerous teachers and short training rotations as well as by a compulsory research project are explained. They highlight the meticulous organization and the compulsory constraints for students, teachers and universities. In absence of evaluation criteria on the efficiency of interventionist training compared to less detailed apprenticeship-program training such as that dispensed in France, they note the differences between the two systems, a necessary precursor to the standardization of training programs which would allow more complete recognition of degrees in the context of a globalization of professional activities.

Measurement of clinical reflective capacity early in training as a predictor of clinical reasoning performance at the end of residency: an experimental study on the script concordance test.

BACKGROUND: The script concordance (SC) test was conceived to measure knowledge organization, the presence of links between items of knowledge which allow for interpretation of data in clinical decision making situations. Earlier studies have shown that the SC test has good psychometric qualities and overcomes some of the limitations of simulation clinical testing. This study explores the predictive validity of the test. OBJECTIVES: To verify whether scores obtained by students at the end of clerkship predict their clinical reasoning performance at the end of residency. DESIGN: Comparison of scores obtained on a SC test taken at the end of clerkship with those obtained 2 years later at the end of residency on two clinical reasoning assessments of known validity, called the short-answer management problems (SAMPs) and the simulated office orals (SOOs), and an objective structured clinical examination (OSCE) aimed at assessing hands-on skills and clinical reasoning. Data were treated by Pearson correlation analysis. SUBJECTS AND SETTING: A cohort of 24 students from a medical school in Quebec was followed up to the end of their residency in family medicine, completed in several schools across Quebec. RESULTS: The observed Pearson correlation coefficients of the SC test were statistically significant (0.451, P=0.013; 0.447; P=0.015) when compared with the SAMPs and the SOOs, respectively. They were not statistically significant (0.340, P=0.052) when compared with the OSCE. CONCLUSION: The authors assumed that the richness of knowledge organization, as indicated by SC test scores, would predict part of the performance on the measures of clinical reasoning (SAMP and SOO), but would predict less well performance on the OSCE which measures both clinical skills and clinical reasoning. Data found in the study are coherent with this hypothesis. This is evidence in favour of the construct validity of the SC test. It also indicates that, in the context of certification assessment, if a candidate has shown good organization of clinical knowledge at an early point in training, it can be expected that he/she will show good organization at subsequent measurements of this kind of knowledge. This appears to be true even if the later measures bear on a wider clinical domain.

[Consultation skills in urology: a model for teaching and evaluation]. / Les compétences de consultant en urologie: une modélisation pour l'enseignement et l'évaluation.

OBJECTIVE: Consultation activity is an important aspect of urological practice, but the specific teaching of this activity is underdeveloped. The objective of this study was to establish a list of consultancy skills as a basis for a planned and structured approach to teaching and evaluation of consultancy skills in urology. MATERIAL AND METHOD: Two-step qualitative protocol: 1) Establishment of an initial list of skills based on data of the literature; 2) Submission of this list to a series of "focus groups" (urologists, interns, referring physicians) in order to validate and progressively refine the model. RESULTS: The items identified were classified into 3 distinct lists: 1) theoretical knowledge; 2) technical skills specific to urology, predominantly performed in the consulting setting, 3) interpersonal skills exclusively concerning the consultant-referring physician relationship. CONCLUSIONS: The consensual specification of these skills can be used to objectively define teaching and evaluation strategies for urology consultancy skills.

[Evaluation of clinical reasoning in urology: contribution of the Script Concordance Test]. / Evaluation du raisonnement clinique en urologie: l'apport du Test de Concordance de Script.

OBJECTIVE: The Script Concordance test is designed to measure knowledge organization, to allow interpretation of data in clinical decision making. The objective of this study was to assess the value of this new written evaluation instrument to evaluate clinical reasoning in urology. MATERIAL AND METHOD: An 80-item examination was completed by a group of medical students (n = 15), a group of urology interns (n = 11), a group of registrars-assistants (n = 7), and a group of experienced urologists (n = 10). The scores obtained were compared by analysis of variance. The reliability of the test was studied by calculating Cronbach's coefficient alpha. RESULTS: The mean score was 46.95 +/- 6.80 for students, 56.18 +/- 1.73 for interns, 66.27 +/- 4.92 for registrars and 63.38 +/- 2.19 for urologists. The differences observed between the scores for students, interns and urologists were significant. The reliability coefficient was 0.79 for the entire examination. CONCLUSIONS: This test is able to discriminate various levels of experience in urology. It proposes a simple and direct approach to evaluation of knowledge organization. Further studies are necessary to confirm the contribution of this test to the strategy of evaluation of the clinical skills in urology.

Scripts and medical diagnostic knowledge: theory and applications for clinical reasoning instruction and research.

Medical diagnosis is a categorization task that allows physicians to make predictions about features of clinical situations and to determine appropriate course of action. The script concept, which first arose in cognitive psychology, provides a theoretical framework to explain how medical diagnostic knowledge can be structured for diagnostic problem solving. The main characteristics of the script concept are pre-stored knowledge, values acceptable or not acceptable for each illness attribute, and default values. Scripts are networks of knowledge adapted to goals of clinical tasks. The authors describe how scripts are used in diagnostic tasks, how the script concept fits within the clinical reasoning literature, how it contrasts with competing theories of clinical reasoning, how educators can help students build and refine scripts, and how scripts can be used to assess clinical competence.

The Script Concordance test: a tool to assess the reflective clinician.

BACKGROUND: The Script Concordance (SC) test is a new assessment tool. It is designed to probe whether knowledge of examinees is efficiently organized for clinical actions. That kind of organization of knowledge is named a script. The SC test places examinees in written, but authentic, clinical situations in which they must interpret data to make decisions. PURPOSE: The SC test is designed to measure the degree of concordance that exists between examinees' scripts and scripts of a panel of experts. The objective of this article is to provide interested educators with the practical "how to" information needed to build and use an SC test. METHODS: The theoretical background of the SC test is described. The principles of construction of an SC test are presented, including the writing of clinical cases, the choice of item format, the validation of the test, and the elaboration of the scoring system. RESULTS: A series of studies have shown that the SC test has interesting psychometric properties, in terms of reliability, face validity, and construct validity. Results from these studies are succinctly presented and commented. CONCLUSION: The SC test is a simple and direct approach to testing organization and use of knowledge. It has the strong advantage for a testing method of being relatively easy to construct and use and to be machine-scorable. It can be either paper- or computer-based and can be used in undergraduate, postgraduate, or continuing medical education.

[Teaching resources of a hospital Gynecology-Obstetrics service. Review of the literature and practical applications]. / Les ressources pédagogiques d'un service hospitalier de Gynécologie-Obstétrique. Revue de la littérature et applications pratiques.

OBJECTIVES: To summarize the methods encountered in a gynecological department for teaching medical students. STUDY: Review of the Medline literature underlying the benefits and disadvantages of each method using the issues of the modern theories of teaching. RESULTS: All the methods are helpful for learning, with different and complementary objectives. Students can constitute a set of skills using a teaching program containing clear objectives and evaluation on which the future medical practice will be based. CONCLUSION: Students have immediate benefits from an active clinical learning involving them and are prepared to the Continued Medical Education.

[Apprenticeship in radiology at the University Hospital Center in Grenoble: research on the most effective methods]. / Apprentissage de la radiologie au CHU de Grenoble: à la recherche de plus d'efficacité.

From analyses of strengths and weaknesses of the apprenticeship of radiology in France, of existing pedagogic methods, of time constraints within clinical settings and of scientific theories of teaching and learning, the authors define objectives that teaching should target and propose a new method of instruction problem and practise-based. The method appears applicable in French radiology departments. It should forster early acquisition by residents of professional attitudes and way of thinking, and make them active self-directed learners.

Consultation skills in radiology: a qualitative study.

OBJECTIVE: Consultation is an important part of radiologic practice. Recently, many concerns have been expressed about the lack of emphasis on consultation skills as part of the training requirements for radiologists. In order to improve the teaching and assessment of trainees, we designed a qualitative study to specify the consultation skills expected from a trainee by the end of residency. METHODS: Three successive focus groups were held with professors of radiology, residents in radiology, and referring physicians. Participants were asked to identify the various competencies required from radiologic consultants (1) spontaneously, (2) by reacting to a previously generated list, and (3) by reacting to common clinical problems encountered in radiology. RESULTS: Consultation skills thus identified were organized in a framework consisting of 2 groups: observable skills and standards of practice. Observable skills were subdivided into 5 "problem-setting" skills and 6 "results-management" skills. Seven qualities and attitudes identified were combined under the rubric "standards of practice." CONCLUSION: The specification of these abilities and competencies through a consensus process should permit the teaching of these skills and their evaluation through objective, structured clinical examinations.

The Diagnosis Script Questionnaire: A New Tool to Assess a Specific Dimension of Clinical Competence.

The Diagnosis Script Questionnaire (DSQ) assesses a specific skill of clinical competence: the ability to weigh collected information in light of entertained hypotheses. The questionnaire presents a clinical vignette for which several hypotheses are relevant. The model of the questions is: if you are thinking of hypothesis A and you find sign Z, what is the effect on your hypothesis? Answers are placed on a 7-point Likert scale, with values ranging from "it can only be this hypothesis" to "this hypothesis is definitely rejected." The scoring process is innovative and reflects the variability of answers among experts.The questionnaire was administered in gynecology-obstetrics; 103 respondents, divided into three groups, 15 faculty, 12 residents, and 76 clerkship students volunteered. Mean global scores were 45.3 for faculty, 40.5 for residents, and 35.8 for students. The differences between the three groups were statistically significant with Welch ANOVA (p < 0.001). The Bonferroni post-hoc correction however indicated that the only significant difference was between student and faculty groups (p < 0.001). Cronbach's alpha was 0.822 for the total group; for the student and resident groups, 0.794 and 0.812 respectively. The proportion of the total variance explained by the interaction items/participants as estimated by generalizability was 42.1%, 65.4% and 73.4% for the faculty, resident and student groups respectively.Results agree with the theories of development of clinical competence which states that knowledge structures specifically adapted to diagnostic tasks appear with clinical experience. This new assessment tool appears promising and warrants future development.

[Evaluation of clinical competence in urology: innovative approach based on performance observation]. / Evaluation de la compétence clinique en urologie: approche innovatrice basée sur l'observation de la performance.

OBJECTIVE: The authors present a pilot project for the evaluation of clinical skills in urology using a method of evaluation based on observation of real performance. MATERIAL AND METHODS: An objective and structured clinical examination (OSCE) applied to urology was developed according to a precise predetermined design: 1) Identification of the objectives to be evaluated. 2) Choice of sampling of clinical situations representative of routine urological practice. 3) Construction, on the basis of these cases, of physician-patient interaction stations and question stations, with, for each case, weighting of objectives, construction of observation grids and writing of instructions for candidates, simulated patients and observers. RESULTS: An OSCE circuit of 10 clinical cases and 16 stations was constructed. The main poles of activity and urological settings were represented. Objective complementary investigations, diagnosis and treatment received the highest weightings. The reliability coefficient, the content validity and the construct validity will be verified on the basis of the overall score obtained by candidate. CONCLUSION: Establishment of the score and the expected results in terms of reliability, validity, and feasibility are discussed. The psychometric qualities of the OSCE have been demonstrated. Although a single instrument is not sufficient, it is currently the measuring tool which most closely approximates ideal evaluation of clinical skills. If the feasibility of this method is confirmed, this pilot project in urology could provide a new approach to evaluation strategies and could participate in the current reflection concerning urology training.

Congenital fistula of the apex of the pyriform sinus: an overlooked phenomenon of debatable origin.

Fistula of the pyriform sinus apex is an often overlooked entity which generally manifests itself as acute suppurative thyroiditis or recurrent deep neck abscesses in children or young adults. Two cases are reported. Arguments in favor of a fourth pharyngeal pouch origin are stressed and the surgical management is described.

Assessment of laryngeal cancer: CT scan versus endoscopy.

In order to assess the value of the systematic use of CT scan in the workup for laryngeal cancer, a retrospective study was conducted on 66 consecutive previously untreated cases of laryngeal cancer. Endoscopic and CT scan findings were systematically compared. The areas of particular difficulty in CT scan assessment are described. CT scan alone understaged laryngeal cancers in 10.6% of cases, all of them being superficial spreading tumors within the larynx or in the juxtalaryngeal areas. CT scan worsened the staging in 22.7% of cases, all of them being deep invasions overlooked by endoscopy. CT was most useful in lesions initially classified as T2 and T3, which included all those reclassified by CT. None of the T1 lesions was upgraded after CT, and systematic use of CT for this stage is not warranted. CT scan workup changed the therapeutic attitude in 10 of the 66 patients (15.1%), causing a switch to conservative surgery in seven patients and total laryngectomy with radiotherapy for the three others. CT was also valuable in choosing the most suitable technique for conservation surgery.

Assessment of the cricoarytenoid joint: high-resolution CT scan study with histo-anatomical correlation.

Eight normal larynges from autopsy cases were CT scanned and 1.5 mm contiguous slices in the cricoarytenoid joint region were taken. The larynges were then sectioned and stained for histological correlation. Different degrees of calcification, ossification and bone marrow cavitation of the cricoid and arytenoid cartilages were observed. This explained the variety of presentations of these structures on CT films. For example, the cricoarytenoid articular interspace was not always visible due to the saddle-shaped configuration of the joint superimposing cartilages on axial views. Features of normal CA joints on CT scan are discussed.

Aseptic necrosis of the cricoid: a complication of tracheal intubation.

Necrosis of the cricoid is a rare complication of tracheal intubation. Almost all cases previously reported were purulent. A 25-year-old man presented with a respiratory distress, two months after 2 days of tracheal intubation. The posterior larynx was the site of a major inflammation with fixity of the arytenoids. After an emergency control of the airway in the acute phase, a posterior cricoid split with widening by a laryngeal stent was performed for treatment of sequelae. Evolutional data from clinical examinations, direct fiberoptic laryngoscopy, CT scan, respiratory function tests, and pathology are described. Successive phases of latency, then acute manifestations, then sequelae can be distinguished. The work-up did not reveal any sign in favor of a septic necrosis. A physiopathologic hypothesis is suggested, explaining the necrosis as an ischemic one, due to destruction of the internal perichondrium vascular network of the cricoid.