Is that realistic? The development of a realism assessment questionnaire and its application in appraising three simulators for a gynaecology procedure.

INTRODUCTION: There is no standard approach to determining the realism of a simulator, valuable information when planning simulation training. The aim of this research was to design a generic simulator realism questionnaire and investigate the contributions of different elements of simulator design to a user's impression of simulator realism and performance. METHODS: A questionnaire was designed with procedure-specific and non-procedure-specific (global) questions, grouped in subscales related to simulator structure and function. Three intrauterine contraceptive device (IUCD) simulators were selected for comparison. Participants were doctors of varying experience, who performed an IUCD insertion on each of the three models and used the questionnaire to rate the realism and importance of each aspect of the simulators. The questionnaire was evaluated by correlation between procedure-specific and global items and the correlation of these items to overall realism scores. Realism scores for each simulator were compared by Kruskal-Wallis and subsequent between-simulator comparison by Dunn's test. RESULTS: Global question scores were highly related to procedure-specific scores. Comparison revealed global item subscale scores were significantly different across models on each of the nine subscales (P < 0.001). Function items were rated of higher importance than structure items (mean function item importance 5.36 versus mean structure item importance 5.02; P = 0.009). CONCLUSIONS: The designed questionnaire was able to discriminate between the models for perceived simulator realism. Findings from this study may assist simulator design and inform future development of a generic questionnaire for assessing user perceptions of simulator realism.

Patient acceptance of the optical diagnosis and misdiagnosis of diminutive colorectal polyps.

BACKGROUND AND AIMS: Optical diagnosis allows for real-time endoscopic assessment of colorectal polyp histology and consists of the resect and discard and diagnose and leave paradigms. This survey assessed patient acceptance of optical diagnosis and their responses to a hypothetical doomsday scenario. METHODS: We conducted a 3-month cross-sectional survey of colonoscopy outpatients presenting to an Australian academic endoscopy center. RESULTS: A total of 981 patients completed the survey (76.0% response rate). The 60.8% of patients who supported resect and discard were more likely to be older men who co-supported diagnose and leave. Fewer patients (49.6%) supported diagnose and leave. A family history of missed cancer diagnosis (odds ratio [OR], 0.59; P = .003) was significantly associated with rejection of resect and discard, and a personal or family history of bowel cancer (OR, 0.7; P = .04) was significantly associated with rejection of diagnose and leave. In the hypothetical scenario of a cancerous polyp incorrectly left in situ leading to stage III disease, 208 (21.2%) patients would definitely ask for financial compensation, 584 (59.5%) were unsure, and 189 (19.3%) would definitely not seek compensation. The patient-proposed median value of compensation sought was $760,000 USD ($1,000,000 AUD; $1 AUD = $0.76 USD). Notably, 18.5% would be willing to give optical diagnosis another chance after this error. CONCLUSION: Patient support for optical diagnosis is limited, and those who are not supporters are more likely to seek financial compensation if errors occur.

Assessment of colorectal polyp recognition skill: development and validation of an objective test.

BACKGROUND: The quality of colonoscopy is known to vary. The extent to which colonoscopists can recognize the presence of subtle colorectal lesions by visually distinguishing them from the surrounding mucosa (i.e., polyp recognition skill) may be one of several attributes that influence polyp detection rates. The aim of the present study was to develop and validate the first objective test of polyp recognition skill. METHODS: Validation study. Twenty-eight experienced colonoscopists and eighty novices took a preliminary 280-item computer-based polyp recognition test. Items were genuine endoscopic images which participants assessed for the presence of "likely polyps." Half included clinically identified polyps. Participants clicked on a suspected lesion or a button marked "no likely polyp", and the main outcome measures were accuracy and response latency. The best items were selected for the final 50-item test. RESULTS: In the preliminary test, experienced colonoscopists correctly identified more polyps than novices (P < .0001) and better discriminated between clinically identified polyps and non-polyp features (as measured by d', P < .0001). For polyp items, the experienced group also responded faster (P < .01). Effect sizes were large for accuracy (Cohen's d = 3.22) and d' (Cohen's d = 3.22). The 50 final test items produced comparable results for accuracy, d', and response latency. For both versions of the test, score scale reliability was high for both polyp and non-polyp items (α = .82 to .97). CONCLUSIONS: The observed experienced-novice differences support the construct validity of the performance measures derived from the tests, indicating that polyp recognition skill can be quantified objectively. The final test may potentially be used to assess trainees, but test sensitivity may be insufficient to make fine-grained distinctions between different skill levels among experienced colonoscopists. More sensitive future tests may provide a valuable supplement to clinical detection rates, allowing objective comparisons between skilled colonoscopists.

Simulation training in obstetrics and gynaecology: What's happening on the frontline?

BACKGROUND: Despite evidence supporting simulation training and awareness that trainee exposure to surgery is suboptimal, it is not known how simulation is being incorporated in obstetrics and gynaecology (O&G) training across Australia and New Zealand. AIM: To investigate the current availability and utilisation of simulation training, and the attitudes, perceived barriers and enablers towards simulation in Australia and New Zealand. METHOD: A survey was distributed to O&G trainees and fellows in Australia and New Zealand. The survey recorded demographic data, current exposure to simulation and beliefs about simulation training. RESULTS: The survey returned 624 responses (24.3%). Most trainees had access to at least one type of simulation (87%). Access to simulators was higher for trainees at tertiary hospitals (92% vs 76%). Few trainees had a simulation curriculum, allocated time or supervision for simulation training. 'Limited access' was the highest rated barrier to using simulation. Lack of time, other training priorities and cost were identified as further barriers. More than 80% of respondents believed simulation improves surgical skills, skills transfer to the operating theatre, and the addition of simulation to the RANZCOG curriculum would benefit trainees. However, a minority of respondents believed simulator proficiency should be shown prior to performing surgery. The need for a curriculum and supervision were highlighted as necessary supports for simulation training. CONCLUSIONS: Despite simulator availability, few trainees are supported by simulation training curricula, allocated time or supervision. Participants believed that simulation training benefits trainees and should be supported with a curriculum and teaching.

Validation of a simple classification system for endoscopic diagnosis of small colorectal polyps using narrow-band imaging.

BACKGROUND & AIMS: Almost all colorectal polyps ≤ 5 mm are benign, yet current practice requires costly pathologic analysis. We aimed to develop and evaluate the validity of a simple narrow-band imaging (NBI)-based classification system for differentiating hyperplastic from adenomatous polyps. METHODS: The study was conducted in 4 phases: (1) evaluation of accuracy and reliability of histologic prediction by NBI-experienced colonoscopists; (2) development of a classification based on color, vessels, and surface pattern criteria, using a modified Delphi method; (3) validation of the component criteria by people not experienced in endoscopy or NBI analysis (25 medical students, 19 gastroenterology fellows) using 118 high-definition colorectal polyp images of known histology; and (4) validation of the classification system by NBI-trained gastroenterology fellows, using still images. We performed a pilot evaluation during real-time colonoscopy. RESULTS: We developed a classification system for the endoscopic diagnosis of colorectal polyp histology and established its predictive validity. When all 3 criteria were used, the specificity ranged from 94.9% to 100% and the combined sensitivity ranged from 8.5% to 61.0%. The specificities of the individual criteria were lower although the sensitivities were higher. During real-time colonoscopy, endoscopists made diagnoses with high confidence for 75% of consecutive small colorectal polyps, with 89% accuracy, 98% sensitivity, and 95% negative predictive values. CONCLUSIONS: We developed and established the validity of an NBI classification system that can be used to diagnose colorectal polyps. In preliminary real-time evaluation, the system allowed endoscopic diagnoses of colorectal polyp histology.

Differentiating adenomas from hyperplastic colorectal polyps: narrow-band imaging can be learned in 20 minutes.

BACKGROUND: Colonoscopy with narrow-band imaging can allow real-time determination of polyp histology. OBJECTIVE: To determine whether physicians with varying levels of experience can learn and apply endoscopic criteria to distinguish between adenomas and hyperplastic polyps. DESIGN: Prospective observational study. SETTING: University teaching hospital. PARTICIPANTS: This study involved 37 physicians (medical residents, N = 12; gastroenterology fellows, N = 12; and gastroenterology faculty, N = 13). INTERVENTION: Small-group, 20-minute, didactic teaching sessions in which the endoscopic criteria for determining polyp histology by using narrow-band imaging were described and demonstrated. MAIN OUTCOME MEASUREMENTS: Learning outcomes were evaluated by using written pretests and posttests in which participants scored pathologically verified, high-definition polyp photographs as adenomas or hyperplastic polyps. RESULTS: The mean overall scores increased significantly from 47.6% correct on the pretest to 90.8% correct on the posttest (P = .0001). The overall mean percentage of responses answered don't know was significantly lower on the posttest (0.6%) compared with the pretest (20.5%, P < .0001). After training, the level of agreement was substantial (kappa = 0.69 for all participants, kappa = 0.79 for fellows). LIMITATIONS: Our study did not assess for sustained improvement with time or in vivo accuracy of histological prediction during live colonoscopy. Further validation in a sample of community physicians is required. CONCLUSION: A short, didactic teaching session can achieve high accuracy and good interobserver agreement in the use of narrow-band imaging for determining the histology of colorectal polyps.

Improving colonoscopy quality through health-care payment reform.

Problems with the quality of colonoscopy are well recognized. Variation in colonoscopist performance is compounded by payment structures that reward volume rather than quality. Payment reform has emerged as one strategy to address these and more systemic problems in the quality of health care. Various forms of value-based purchasing might encourage a realignment of incentives, and allow reimbursement to be directly linked with clinically important goals of colonoscopy. This paper proposes criteria for the selection of quality measures, and three candidate indicators to define quality for the purpose of payment reform in colonoscopy: cecal intubation rate, adenoma detection rate, and recommended post-polypectomy surveillance interval. These measures represent valid, credible, and reliable indicators of the quality of colonoscopy for colorectal cancer screening and surveillance. Payment reform should explicitly link public reporting and performance on these quality measures to payment for colonoscopy.