Competence in Decision Making: Setting Performance Standards for Critical Care.

BACKGROUND: Health care professionals must be able to make frequent and timely decisions that can alter the illness trajectory of intensive care patients. A competence standard for this ability is difficult to establish yet assuring practitioners can make appropriate judgments is an important step in advancing patient safety. We hypothesized that simulation can be used effectively to assess decision-making competence. To test our hypothesis, we used a "standard-setting" method to derive cut scores (standards) for 16 simulated ICU scenarios targeted at decision-making skills and applied them to a cohort of critical care trainees. METHODS: Panelists (critical care experts) reviewed digital audio-video performances of critical care trainees managing simulated critical care scenarios. Based on their collectively agreed-upon definition of "readiness" to make decisions in an ICU setting, each panelist made an independent judgment (ready, not ready) for a large number of recorded performances. The association between the panelists' judgments and the assessment scores was used to derive scenario-specific performance standards. RESULTS: For all 16 scenarios, the aggregate panelists' ratings (ready/not ready for independent decision making) were positively associated with the performance scores, permitting derivation of performance standards for each scenario. CONCLUSIONS: Minimum competence standards for high-stakes decision making can be established through standard-setting techniques. We effectively identified "front-line" providers who are, or are not, ready to make independent decisions in an ICU setting. Our approach may be used to assure stakeholders that clinicians are competent to make appropriate judgments. Further work is needed to determine whether our approach is effective in simulation-based assessments in other domains.

Description and function of a difficult airway service.

The goal of the Pediatric Difficult Airway Service (DAS) is to improve the care of children with airway abnormalities primarily through identification of children at risk for failed airway management. The airway service encourages early recognition and provides consultation, a plan for airway management, expertise in airway management, and follow-up care for children who have a difficult airway. The service has improved the education of healthcare professionals and heightened awareness about the consequences of failed airway management.

Simulation-Based Assessment of Critical Care "Front-Line" Providers.

OBJECTIVES: Develop a standardized simulation method to assess clinical skills of ICU providers. DESIGN: Simulation assessment. SETTING: Simulation laboratory. SUBJECTS: Residents, Critical Care Medicine fellows, acute care nurse practitioner students. INTERVENTIONS: Performance scoring in scenarios from multiple Critical Care Medicine competency domains. MEASUREMENTS AND MAIN RESULTS: Three-hundred eighty-four performances by 48 participants were scored using checklists (% correct) and holistic "global" ratings (1 [unprepared] to 9 [expert]). One-hundred eighty were scored by two raters. Mean checklist and global scores (± SD) ranged from 65.0% (± 16.3%) to 84.5% (± 17.3%) and 4.7 (± 1.4) to 7.2 (± 1.2). Checklist and global scores for Critical Care Medicine fellows and senior acute care nurse practitioner students (Experienced group, n = 26) were significantly higher than those for the Novice acute care nurse practitioner students (Novice group, n = 14) (75.6% ± 15.6% vs 68.8% ± 21.0% and 6.1 ± 1.6 vs 5.4 ± 1.5, respectively; p < 0.05). Residents (Intermediate group, n = 8) scored between the two (75.4% ± 18.3% and 5.7 ± 1.7). 38.5% of the Experienced group scored in the top quartile for mean global score, compared with 12.5% of the Intermediate and 7.1% of the Novice groups. Conversely, 50% of the Novice group scored in the lower quartile (< 5.3), compared with 37.5% of the Intermediate and 11.5% of the Experienced groups. Psychometric analyses yielded discrimination values greater than 0.3 for most scenarios and reliability for the eight-scenario assessments of 0.51 and 0.60, with interrater reliability of 0.71 and 0.75, for checklist and global scoring, respectively. CONCLUSIONS: The simulation assessments yielded reasonably reliable measures of Critical Care Medicine decision-making skills. Despite a wide range of performance, those with more ICU training and experience performed better, providing evidence to support the validity of the scores. Simulation-based assessments may ultimately prove useful to determine readiness to assume decision-making roles in the ICU.

Tertiary Care Referrals for Fractures in Children.

PURPOSE: With limited local access to pediatric subspecialty care outside major metropolitan areas, tertiary care hospitals treat many children originally seen at outside facilities for relatively brief but urgent surgical procedures. This referral-based care imposes significant financial and psychological stress on the families. DESIGN: Prospective, survey methodology was used. METHODS: Families of children aged 0-18 years admitted to the St. Louis Children's Hospital for surgical repair of fractures were surveyed. The questionnaire was developed by the research team and measured a variety of fields. FINDINGS: The operative procedure in the majority of these children was relatively brief in both groups, often less than one hour. The time of injury to their discharge from our hospital, however, extended to 36 hours. Families missed several days of work. Many children were kept NPO longer than needed. CONCLUSIONS: Our preliminary evaluation suggests that a relatively minor unexpected surgery of a child can impose significant financial, organizational, and psychological burden on the family.

The Simulation-Based Assessment of Pediatric Rapid Response Teams.

OBJECTIVE: To create scenarios of simulated decompensating pediatric patients to train pediatric rapid response teams (RRTs) and to determine whether the scenario scores provide a valid assessment of RRT performance with the hypothesis that RRTs led by intensivists-in-training would be better prepared to manage the scenarios than teams led by nurse practitioners. STUDY DESIGN: A set of 10 simulated scenarios was designed for the training and assessment of pediatric RRTs. Pediatric RRTs, comprising a pediatric intensive care unit (PICU) registered nurse and respiratory therapist, led by a PICU intensivist-in-training or a pediatric nurse practitioner, managed 7 simulated acutely decompensating patients. Two raters evaluated the scenario performances and psychometric analyses of the scenarios were performed. RESULTS: The teams readily managed scenarios such as supraventricular tachycardia and opioid overdose but had difficulty with more complicated scenarios such as aortic coarctation or head injury. The management of any particular scenario was reasonably predictive of overall team performance. The teams led by the PICU intensivists-in-training outperformed the teams led by the pediatric nurse practitioners. CONCLUSIONS: Simulation provides a method for RRTs to develop decision-making skills in managing decompensating pediatric patients. The multiple scenario assessment provided a moderately reliable team score. The greater scores achieved by PICU intensivist-in-training-led teams provides some evidence to support the validity of the assessment.

A French description of German psychology laboratories in 1893 by Victor Henri, a collaborator of Binet.

There is a rich tradition of writings about the foundation of psychology laboratories, particularly in the United States but also in France. Various documents exist concerning former German laboratories in American and French literature. But the most interesting French paper was certainly written by a young psychologist named Victor Henri (1872-1940) who was a close collaborator of Alfred Binet (1857-1911) in the 1890s. Visiting various psychology laboratories, he wrote, in 1893, a clear description of the laboratories of Wundt, G. E. Müller, Martius and Ebbinghaus. An English translation is given of Henri's paper and the historical importance of his contribution is here expounded by contrasting the German and French psychologies of the time.

Difficult airway consultation service for children: steps to implement and preliminary results.

BACKGROUND: Failed airway management remains one of the most common causes of cardiopulmonary arrest in the pediatric population. Practice guidelines addressing the difficult airway (DAW) in adults provide anesthesiologists a framework for managing the airway during the perioperative period; however, similar consensus guidelines are lacking in the pediatric population. Many of the adverse events associated with difficult pediatric airway management occur outside the perioperative setting and often result in worse outcomes. The lower frequency of DAW management required in children, lesser awareness of pediatric health care professionals about DAW management, and the need for guiding principles led us to develop a DAW consultative service. This report outlines the steps to establish the Difficult Airway Service (DAS) and the initial experiences with this new consultation service. METHODS: The mission of the DAS is to identify children with known or anticipated DAWs, communicate the diagnosis and collaborate with referring medical and surgical services, and to manage children in those settings that airway management might be required in the context of the patient's ongoing medical care. RESULTS: The initial 3-month experience confirmed that a majority of pediatric DAW events are associated with congenital or acquired abnormalities. Through appropriate consultation and leadership, the DAS was able to physically and electronically identify pediatric patients with a DAW and provide management. Hospital-wide participation was instrumental in the success and exponential growth of DAS: planned preoperative tracheostomy in complicated posterior spinal fusion candidates, participation in EXIT procedures, standardization of airway carts, and implementation of education forums. CONCLUSION: In developing the DAS, our goal was to provide a more comprehensive approach to caring for a child with a DAW that included their entire hospital stay and follow-up care. We believe this approach has improved health care professional awareness as well as the safe management of routine and difficult pediatric airway. Additional studies are needed to determine whether measurable changes in morbidity and mortality are observed over time.

Progress in simulation education: developing an anesthesia curriculum.

PURPOSE OF REVIEW: Simulation's role in anesthesia education is expanding to include more advanced skills and training for subspecialty practice. This review will provide an overview of many recent studies that expand the simulation curriculum for anesthesia education. RECENT FINDINGS: Recent studies describe a curriculum that uses a range of simulation modalities, including part-task trainers, mannequin-based simulation, virtual reality, in-situ techniques, screen-based simulations as well as encounters with 'standardized' patients, nurses or physician colleagues. A variety of studies describe the use of task-training devises to more effectively acquire skills, such as fibre-optic intubation, ultrasound-guided regional anesthesia and transthoracic echocardiography as well as expand on a variety of teamwork skills particularly in subspecialty anesthesia practice. SUMMARY: A curriculum is emerging that utilizes a variety of simulation modalities as part of a more comprehensive educational strategy for anesthesia specialty training.

Acute care of pediatric patients with sickle cell disease: a simulation performance assessment.

BACKGROUND: Sickle cell disease (SCD) is a rare disorder with cardinal features including hospitalization for vaso-occlusive pain episodes, acute pulmonary injury, and increased infection rates. For physician-trainees, learning optimal SCD management is challenging because of limited exposure to life threatening complications requiring timely interventions. PROCEDURE: To create, demonstrate reliability, and validate simulation-based, acute care SCD scenarios for physician-trainees, seven scenarios were derived from SCD patient cases. For each scenario, participants had 5 minutes to complete diagnostic and treatment interventions. Participants were divided into two groups based on clinical experience: interns or residents/fellows. Two raters scored performances using diagnostic and therapeutic checklists--indicating whether specific actions were performed and a global, 1 (poor) to 9 (excellent), rating. Scenario scores were calculated by averaging rater scores on each metric. Reliability was defined through uniformity in rater scoring and consistency of participant performance over scenarios. Validity was demonstrated by the performance gradient where the more experienced trainees outperform those early in training. RESULTS: Twenty-eight pediatric residents and hematology fellows took part in the study. Reliability for assessment scores overall was moderate. Performance on all but one scenario was moderately predictive of overall performance. Senior resident/fellows performed significantly better than interns. Positive associations existed between overall performance scores (P < 0.01) and months of postgraduate training (P < 0.01). CONCLUSIONS: Mannequin-based simulation is a novel method for teaching pediatric residents SCD-specific acute care skills. The assessment provided reliable and valid measures of trainees' performance. Further studies are needed to determine simulation's utility in education and evaluation.

Cognitive Load Assessment Scales in Simulation: Validity Evidence for a Novel Measure of Cognitive Load Types.

INTRODUCTION: Cognitive load (CL) theory provides a framework to inform simulation instructional design. Reliable measures of CL types (intrinsic [IL], extraneous [EL], and germane load [GL]) in simulation are lacking. We developed the novel Cognitive Load Assessment Scales in Simulation (CLAS-Sim) and report validity evidence using Kane's framework. METHODS: This quasi-experimental study tested the effect of a segmented/pause-and-debrief or standard/end-of-case-debrief intervention on pediatric residents' performance and self-rated CL in 2 complex- and simple-case simulations. After each simulation, participants completed 22 items measuring CL types. Three validity inferences were examined: scoring (instrument development and principal component analysis); generalization (internal consistency reliability of CL-component items across cases); and extrapolation [CLAS-Sim correlations with the single-item Paas scale, which measures overall CL; differences in primary task performance (high vs low); and discriminant validity of IL under different instructional-design conditions]. RESULTS: Seventy-four residents completed both simulations and postcase CLAS-Sim measures. The principal component analysis yielded 3 components: 4-item IL, 4-item EL, and 3-item GL scales (Cronbach's α, 0.68-0.77). The Paas scores correlated with CLAS-Sim IL and total CL scores in both cases ( rs range, 0.39-0.70; P ≤ 0.001). High complex-case performers reported lower IL and total CL (analyses of variance, each P < 0.001). In multivariate analyses of variance, CLAS-Sim IL, GL, and total CL varied across both cases by arm (each P ≤ 0.018); the segmented-debrief arm reported lower IL than the standard-debrief arm in both cases (each P ≤ 0.01). CONCLUSIONS: The CLAS-Sim demonstrates preliminary validity evidence for distinguishing 3 CL types but requires further study to evaluate the impact of simulation-design elements on CL and learning.

Simulation in pediatric anesthesiology.

Simulation-based training, research and quality initiatives are expanding in pediatric anesthesiology just as in other medical specialties. Various modalities are available, from task trainers to standardized patients, and from computer-based simulations to mannequins. Computer-controlled mannequins can simulate pediatric vital signs with reasonable reliability; however the fidelity of skin temperature and color change, airway reflexes and breath and heart sounds remains rudimentary. Current pediatric mannequins are utilized in simulation centers, throughout hospitals in-situ, at national meetings for continuing medical education and in research into individual and team performance. Ongoing efforts by pediatric anesthesiologists dedicated to using simulation to improve patient care and educational delivery will result in further dissemination of this technology. Health care professionals who provide complex, subspecialty care to children require a curriculum supported by an active learning environment where skills directly relevant to pediatric care can be developed. The approach is not only the most effective method to educate adult learners, but meets calls for education reform and offers the potential to guide efforts toward evaluating competence. Simulation addresses patient safety imperatives by providing a method for trainees to develop skills and experience in various management strategies, without risk to the health and life of a child. A curriculum that provides pediatric anesthesiologists with the range of skills required in clinical practice settings must include a relatively broad range of task-training devises and electromechanical mannequins. Challenges remain in defining the best integration of this modality into training and clinical practice to meet the needs of pediatric patients.

Charpentier's papers of 1886 and 1891 on weight perception and the size-weight illusion.

The French physiologist Augustin Charpentier (1852-1916) published the first accounts of the size-weight illusion-the observation that if two objects differ in size but have equal mass, the smaller will feel heavier when lifted. In the current paper, translations are presented of Charpentier's much-cited 1891 paper on weight perception and the size-weight illusion, and his little-known brief 1886 paper which contains the earliest experimental data on the illusion. Charpentier explained weight illusions in terms of the sense of effort involved in lifting the object and the contrast with the expected effort. Modern research shows that people quickly adapt and use the appropriate force to pick up objects, but the illusion persists even when appropriate force is used; expectations therefore affect the perceptual system more strongly than the motor system.

Simulation-based assessment of pediatric anesthesia skills.

BACKGROUND: Assessment of pediatric anesthesia trainees is complicated by the random nature of adverse patient events and the vagaries of clinical exposure. However, assessment is critical to improve patient safety. In previous studies, a multiple scenario assessment provided reliable and valid measures of the abilities of anesthesia residents. The purpose of this study was to develop a set of relevant simulated pediatric perioperative scenarios and to determine their effectiveness in the assessment of anesthesia residents and pediatric anesthesia fellows. METHODS: Ten simulation scenarios were designed to reflect situations encountered in perioperative pediatric anesthesia care. Anesthesiology residents and fellows consented to participate and were debriefed after each scenario. Two pediatric anesthesiologists scored each scenario by key action checklist. The psychometric properties (reliability, validity) of the scores were studied. RESULTS: Thirty-five anesthesiology residents and pediatric anesthesia fellows participated. The participants with greater experience administering pediatric anesthetics generally outperformed those with less experience. Score variance attributable to raters was low, yielding a high interrater reliability. CONCLUSIONS: A multiple-scenario, simulation-based assessment of pediatric perioperative care was designed and administered to residents and fellows. The scores obtained from the assessment indicated the content was relevant and that raters could reliably score the scenarios. Participants with more training achieved higher scores, but there was a wide range of ability among subjects. This method has the potential to contribute to pediatric anesthesia performance assessment, but additional measures of validity including correlations with more direct measures of clinical performance are needed to establish the utility of this approach.

Simulation-based assessment in anesthesiology: requirements for practical implementation.

Simulations have taken a central role in the education and assessment of medical students, residents, and practicing physicians. The introduction of simulation-based assessments in anesthesiology, especially those used to establish various competencies, has demanded fairly rigorous studies concerning the psychometric properties of the scores. Most important, major efforts have been directed at identifying, and addressing, potential threats to the validity of simulation-based assessment scores. As a result, organizations that wish to incorporate simulation-based assessments into their evaluation practices can access information regarding effective test development practices, the selection of appropriate metrics, the minimization of measurement errors, and test score validation processes. The purpose of this article is to provide a broad overview of the use of simulation for measuring physician skills and competencies. For simulations used in anesthesiology, studies that describe advances in scenario development, the development of scoring rubrics, and the validation of assessment results are synthesized. Based on the summary of relevant research, psychometric requirements for practical implementation of simulation-based assessments in anesthesiology are forwarded. As technology expands, and simulation-based education and evaluation takes on a larger role in patient safety initiatives, the groundbreaking work conducted to date can serve as a model for those individuals and organizations that are responsible for developing, scoring, or validating simulation-based education and assessment programs in anesthesiology.

Management of anesthesia equipment failure: a simulation-based resident skill assessment.

BACKGROUND: Intraoperative anesthesia equipment failures are a cause of anesthetic morbidity. Our purpose in this study was 1) to design a set of simulated scenarios that measure skill in managing intraoperative equipment-related errors and 2) to evaluate the reliability and validity of the measures from this multiple scenario assessment. METHODS: Eight intraoperative scenarios were created to test anesthesia residents' skills in managing a number of equipment-related failures. Fifty-six resident physicians, divided into four groups based on their training year (Resident 1-Resident 4), participated in the individual simulation-based assessment of equipment-related failures. The score for each scenario was generated by a checklist of key actions relevant to each scenario and time to complete these actions. RESULTS: The residents' scores, on average, improved with increased level of training. The more senior residents (R3 and R4) performed better than more junior residents (R1 and R2). Despite similar training background, there was a wide range of skill among the residents within each training year. The summary score on the eight scenario assessments, measured by either the key actions or the time required to manage the events, yielded a reliable estimate of a resident's skill in managing these simulated equipment failures. DISCUSSION: Anesthesia residents' performances could be reliably evaluated using a set of simulated intraoperative equipment problems. This multiple scenario assessment was an effective method to evaluate individual performance. The summary results, by training year, could be used to determine how successful current instructional methods are for acquiring skill.