Translating surgical metrics into automated assessments.

In the effort to promote more continuous and quantitative assessment of surgical proficiency, there is an increased need to define and establish common surgical metrics. Furthermore, as various pressures such as limited duty hours and access to educational resources, including materials and expertise, place increased demands on training, the value of quantitative automated assessment becomes increasingly apparent. We present our methods to establish common surgical metrics within the otology and neurotology community and our initial efforts in the subsequent transfer of these metrics into objective automated assessments provided via a simulation environment.

Virtual simulation of mouse anatomy and procedural techniques.

Translational science requires the use of mouse models for the characterization of disease and evaluation of treatment therapies. However, often there is a lack of comprehensive training for scientists in the systemic and regional anatomy of the mouse that limits their ability to perform studies involving complex interventional procedures. We present our methodologies for the development, evaluation, and dissemination of an interactive 3D mouse atlas that includes designs for presenting emulation of procedural technique. We present the novel integration of super-resolution imaging techniques, depth-of-field interactive volume rendering of large data, and the seamless delivery of remote visualization and interaction to thin clients.

Atlas-based segmentation of cochlear microstructures in cone beam CT.

PURPOSE: To develop an automated segmentation approach for cochlear microstructures [scala tympani (ST), scala vestibuli (SV), modiolus (Mod), mid-modiolus (Mid-Mod), and round window membrane (RW)] in clinical cone beam computed tomography (CBCT) images of the temporal bone for use in surgical simulation software and for preoperative surgical evaluation. METHODS: This approach was developed using the publicly available OpenEar (OE) Library that includes temporal bone specimens with spatially registered CBCT and 3D micro-slicing images. Five of these datasets were spatially aligned to our internal OSU atlas. An atlas of cochlear microstructures was created from one of the OE datasets. An affine registration of this atlas to the remaining OE CBCT images was used for automatically segmenting the cochlear microstructures. Quantitative metrics and visual review were used for validating the automatic segmentations. RESULTS: The average DICE metrics were 0.77 and 0.74 for the ST and SV, respectively. The average Hausdorff distance (AVG HD) was 0.11 mm and 0.12 mm for both scalae. The mean distance between the centroids for the round window was 0.32 mm, and the mean AVG HD was 0.09 mm. The mean distance and angular rotation between the mid-modiolar axes were 0.11 mm and 9.8 degrees, respectively. Visually, the segmented structures were accurate and similar to that manually traced by an expert observer. CONCLUSIONS: An atlas-based approach using 3D micro-slicing data and affine spatial registration in the cochlear region was successful in segmenting cochlear microstructures of temporal bone anatomy for use in simulation software and potentially for pre-surgical planning and rehearsal.

Standard Setting of Competency in Mastoidectomy for the Cross-Institutional Mastoidectomy Assessment Tool.

OBJECTIVE: Competency-based surgical training involves progressive autonomy given to the trainee. This requires systematic and evidence-based assessment with well-defined standards of proficiency. The objective of this study is to develop standards for the cross-institutional mastoidectomy assessment tool to inform decisions regarding whether a resident demonstrates sufficient skill to perform a mastoidectomy with or without supervision. METHODS: A panel of fellowship-trained content experts in mastoidectomy was surveyed in relation to the 16 items of the assessment tool to determine the skills needed for supervised and unsupervised surgery. We examined the consensus score to investigate the degree of agreement among respondents for each survey item as well as additional analyses to determine whether the reported skill level required for each survey item was significantly different for the supervised versus unsupervised level. RESULTS: Ten panelists representing different US training programs responded. There was considerable consensus on cut-off scores for each item and trainee level between panelists, with moderate (0.62) to very high (0.95) consensus scores depending on assessment item. Further analyses demonstrated that the difference between supervised and unsupervised skill levels was significantly meaningful for all items. Finally, minimum-passing scores for each item was established. CONCLUSION: We defined performance standards for the cross-institutional mastoidectomy assessment tool using the Angoff method. These cut-off scores that can be used to determine when trainees can progress from performance under supervision to performance without supervision. This can be used to guide training in a competency-based training curriculum.

Enhancing realism of wet surfaces in temporal bone surgical simulation.

We present techniques to improve visual realism in an interactive surgical simulation application: a mastoidectomy simulator that offers a training environment for medical residents as a complement to using a cadaver. As well as displaying the mastoid bone through volume rendering, the simulation allows users to experience haptic feedback and appropriate sound cues while controlling a virtual bone drill and suction/irrigation device. The techniques employed to improve realism consist of a fluid simulator and a shading model. The former allows for deformable boundaries based on volumetric bone data, while the latter gives a wet look to the rendered bone to emulate more closely the appearance of the bone in a surgical environment. The fluid rendering includes bleeding effects, meniscus rendering, and refraction. We incorporate a planar computational fluid dynamics simulation into our three-dimensional rendering to effect realistic blood diffusion. Maintaining real-time performance while drilling away bone in the simulation is critical for engagement with the system.

Atlas-based segmentation of temporal bone surface structures.

PURPOSE: To develop a time-efficient automated segmentation approach that could identify surface structures on the temporal bone for use in surgical simulation software and preoperative surgical training. METHODS: An atlas-based segmentation approach was developed to segment the tegmen, sigmoid sulcus, exterior auditory canal, interior auditory canal, and posterior canal wall in normal temporal bone CT images. This approach was tested in images of 20 cadaver bones (10 left, 10 right). The results of the automated segmentation were compared to manual segmentation using quantitative metrics of similarity, Mahalanobis distance, average Hausdorff distance, and volume similarity. RESULTS: The Mahalanobis distance was less than 0.232 mm for all structures. The average Hausdorff distance was less than 0.464 mm for all structures except the posterior canal wall and external auditory canal for the right bones. Volume similarity was 0.80 or greater for all structures except the sigmoid sulcus that was 0.75 for both left and right bones. Visually, the segmented structures were accurate and similar to that manually traced by an expert observer. CONCLUSIONS: An atlas-based approach using a deformable registration of a Gaussian-smoothed temporal bone image and refinements using surface landmarks was successful in segmenting surface structures of temporal bone anatomy for use in pre-surgical planning and training.

Evaluating elicited anxiety in a simulated environment.

Simulated environments are increasingly being used to evaluate human behavior in a wide variety of situations. These virtual environments are particularly useful in the study of behavior involving dangerous conditions. Quantifying the impact of these environments on the subject is essential to validate the efficacy of the simulation as a virtualization of reality. We have developed a prototypical environment that simulates a common agricultural setting that provides the potential for acute injuries. We report on an exploratory study to evaluate the efficacy of our design to illicit a sense of realism and subsequently evoke a physiological response of anxiety in the subject. We present on the components that comprise the environment, our study design, and preliminary results.

Simulation of punch biopsies: a case study.

As the incidence of skin cancer continues to rise, there is an increasing need for skilled practitioners that are proficient in identifying suspicious lesions and competent in acquiring biopsies that provide for optimal determination of malignancy and staging. We report on the development of a prototype simulation that emulates the basic procedures necessary to acquire a punch biopsy. The objective of this effort is to produce a low-cost, effective method to teach non-specialists, i.e., nurse practitioners, internists, etc., the optimal placement of the punch to obtain a biopsy for pathological analysis. The simulation can be utilized for synchronous sessions with remote experts, as well as asynchronous sessions for deliberate practice. The simulation is designed to allow for easy import of digital images of various lesions to promote use and present the wide pathological variance experienced in the clinic.

Translating human simulation technologies to veterinary surgical training: accelerating adoption.

Through the reduction of live animal use in teaching surgical technique, the opportunities to deliberately study complex regional anatomy and practice surgical technique have decreased. With reduced exposure, there is concern some individuals are graduating without the requisite knowledge and proficiency to perform adequate surgical techniques. Ultimately, animals may unnecessarily suffer due to morbidities from limited or poor surgical competencies. We have translated developments derived from the human surgical simulation field for application to veterinary surgical training. We present our work on intuitive software for learning regional anatomy, surgical simulations, and on several limiting factors that impede the validation and adoption of simulation technologies for use by the veterinarian surgical community.

Cross-Institutional Evaluation of a Mastoidectomy Assessment Instrument.

OBJECTIVE: The objective of this work is to obtain validity evidence for an evaluation instrument used to assess the performance level of a mastoidectomy. The instrument has been previously described and had been formulated by a multi-institutional consortium. DESIGN: Mastoidectomies were performed on a virtual temporal bone system and then rated by experts using a previously described 15 element task-based checklist. Based on the results, a second, similar checklist was created and a second round of rating was performed. SETTING: Twelve otolaryngological surgical training programs in the United States. PARTICIPANTS: In all, 65 mastoidectomy performances were evaluated coming from 37 individuals with a variety of temporal bone dissection experience, from medical students to attending physicians. Raters were attending surgeons from 12 different institutions. RESULTS: Intraclass correlation scores varied greatly between items in the checklist with some being low and some being high. Percentage agreement scores were similar to previous rating instruments. There is strong evidence that a high score on the task-based checklist is necessary for a rater to consider a mastoidectomy to be performed at the level of an expert but a high score is not a sufficient condition. CONCLUSIONS: Rewording of the instrument items to focus on safety does not result in increased reliability of the instrument. The strong result of the Necessary Condition Analysis suggests that going beyond simple correlation measures can give extra insight into grading results. Additionally, we suggest using a multiple point scale instead of a binary pass/fail question combined with descriptive mastery levels.

Multi-Institutional Development of a Mastoidectomy Performance Evaluation Instrument.

OBJECTIVE: A method for rating surgical performance of a mastoidectomy procedure that is shown to apply universally across teaching institutions has not yet been devised. This work describes the development of a rating instrument created from a multi-institutional consortium. DESIGN: Using a participatory design and a modified Delphi approach, a multi-institutional group of expert otologists constructed a 15-element task-based checklist for evaluating mastoidectomy performance. This instrument was further refined into a 14-element checklist focusing on the concept of safety after using it to rate a large and varied population of performances. SETTING: Twelve otolaryngological surgical training programs in the United States. PARTICIPANTS: A total of 14 surgeons from 12 different institutions took part in the construction of the instrument. RESULTS: By using 14 experts from 12 different institutions and a literature review, individual metrics were identified, rated as to the level of importance and operationally defined to create a rating scale for mastoidectomy performance. Initial use of the rating scale showed modest rater agreement. The operational definitions of individual metrics were modified to emphasize "safe" as opposed to "proper" technique. A second rating instrument was developed based on this feedback. CONCLUSIONS: Using a consensus-building approach with multiple rounds of communication between experts is a feasible way to construct a rating instrument for mastoidectomy. Expert opinion alone using a Delphi method provides face and content validity evidence, however, this is not sufficient to develop a universally acceptable rating instrument. A continued process of development and experimentation to demonstrate evidence for reliability and validity making use of a large population of raters and performances is necessary to achieve universal acceptance.

Expert subjective comparison of haptic models for bone-drill interaction.

PURPOSE: A haptic algorithm to simulate the interaction between a surgical drill and bone using a constraint-based algorithm has been previously demonstrated. However, there has been no blinded study to determine whether this algorithm is preferred by professionals who commonly use this type of system METHODS: Fourteen otologic surgeons were presented with a spring-damper model and a constraint-based model of drill-bone interaction rendered on a low-cost haptic device with only linear feedback. The participants were blinded as to what algorithm they were using. They then answered survey questions about their opinions of the models. RESULTS: The surgeons overwhelmingly preferred the constraint-based model. They generally preferred the constraint-based model in the individual questions as well. CONCLUSIONS: Follow-up work can be done to fine-tune the parameters in the model, but this study shows that a sophisticated algorithm can make a significant difference even on a low-fidelity haptic device.

Atlas-Based Segmentation of Temporal Bone Anatomy.

PURPOSE: To develop a time-efficient automated segmentation approach that could identify critical structures in the temporal bone for visual enhancement and use in surgical simulation software. METHODS: An atlas-based segmentation approach was developed to segment the cochlea, ossicles, semicircular canals (SCCs), and facial nerve in normal temporal bone CT images. This approach was tested in images of 26 cadaver bones (13 left, 13 right). The results of the automated segmentation were compared to manual segmentation visually and using DICE metric, average Hausdorff distance, and volume similarity. RESULTS: The DICE metrics were greater than 0.8 for the cochlea, malleus, incus, and the SCCs combined. It was slightly lower for the facial nerve. The average Hausdorff distance was less than one voxel for all structures, and the volume similarity was 0.86 or greater for all structures except the stapes. CONCLUSIONS: The atlas-based approach with rigid body registration of the otic capsule was successful in segmenting critical structures of temporal bone anatomy for use in surgical simulation software.

Integration of high-resolution data for temporal bone surgical simulations.

PURPOSE: To report on the state of the art in obtaining high-resolution 3D data of the microanatomy of the temporal bone and to process that data for integration into a surgical simulator. Specifically, we report on our experience in this area and discuss the issues involved to further the field. DATA SOURCES: Current temporal bone image acquisition and image processing established in the literature as well as in house methodological development. REVIEW METHODS: We reviewed the current English literature for the techniques used in computer-based temporal bone simulation systems to obtain and process anatomical data for use within the simulation. Search terms included "temporal bone simulation, surgical simulation, temporal bone." Articles were chosen and reviewed that directly addressed data acquisition and processing/segmentation and enhancement with emphasis given to computer-based systems. We present the results from this review in relationship to our approach. CONCLUSIONS: High-resolution CT imaging ([Formula: see text] voxel resolution), along with unique image processing and rendering algorithms, and structure-specific enhancement are needed for high-level training and assessment using temporal bone surgical simulators. Higher-resolution clinical scanning and automated processes that run in efficient time frames are needed before these systems can routinely support pre-surgical planning. Additionally, protocols such as that provided in this manuscript need to be disseminated to increase the number and variety of virtual temporal bones available for training and performance assessment.

Use of ultra-high-resolution data for temporal bone dissection simulation.

OBJECTIVES: For the past 5 years, our group has been developing a virtual temporal bone dissection environment for training otologic surgeons. Throughout the course of our development, a recurring challenge is the acquisition of high-resolution, multimodal, and multi-scale data sets that are used for the visual as well as haptic (sense of touch) display. This study presents several new techniques in temporal bone imaging and their use as data for surgical simulation. METHODS: At our institution (OSU), we are fortunate to have a high-field (8 Tesla) magnetic resonance imaging (MRI) research magnet that provides an order of magnitude higher resolution compared to clinical 1.5T MRI scanners. Magnetic resonance imaging has traditionally been superb at delineating soft tissue structure, and certainly, the 8T unit does indeed do this at a resolution of 100-200 microm(3). To delineate the bony structure of the mastoid and middle ear, computed tomography (CT) has traditionally been used because of the high signal-to-noise ratio delineating bone signal from air and soft tissue. We have partnered with researchers at other institutions (CCF) to make use of a "microCT" that provides a resolution of 214 x 214 x 390 micrometers of bony structure. RESULTS: This report provides a description of the 2 methodologies and presentation of the striking image data capable of being generated. See images presented. CONCLUSIONS: Using these 2 new and innovative imaging modalities, we provide an order of magnitude greater resolution to the visual and haptic display in our temporal bone dissection simulation environment.

Emphatic, interactive volume rendering to support variance in user expertise.

Various levels of representation, from abstract to schematic to realistic, have been exploited for millennia to facilitate the transfer of information from one individual to another. Learning complex information, such as that found in biomedicine, proves specifically problematic to many, and requires incremental, step-wise depictions of the information to clarify structural, functional, and procedural relationships.Emerging volume-rendering technique, such as non-photorealistic representation, coupled with advances in computational speeds, especially new graphical processing units, provide unique capabilities to explore the use of various levels of representation in interactive sessions. We have developed a system that produces images that simulate pictorial representations for both scientific and biomedical visualization. The system combines traditional and novel volume illustration techniques. We present examples from our efforts to distill representational techniques for both creative exploration and emphatic presentation for clarity. More specifically, we present our efforts to adapt these techniques for interactive simulation sessions being developed in a concurrent project for resident training in temporal bone dissection simulation. The goal of this effort is to evaluate the use of emphatic rendering to guide the user in an interactive session and to facilitate the learning of complex biomedical information, including structural, functional, and procedural information.

Virtual temporal bone dissection: an interactive surgical simulator.

OBJECTIVE: Our goal was to integrate current and emerging technology in virtual systems to provide a temporal bone dissection simulator that allows the user interactivity and realism similar to the cadaver laboratory. STUDY DESIGN: Iterative design and validation of a virtual environment for simulating temporal bone dissection. SETTING: University otolaryngology training program with interdisciplinary interaction in a high-performance computer facility. RESULTS: The system provides visual, force feedback (haptic), and aural interfaces. Unlike previous "fly through" virtual systems, this environment provides a richer emulation of surgical experience. CONCLUSION: The system provides a high level of functional utility and, through initial evaluations, demonstrates promise in adding to traditional training methods. SIGNIFICANCE: The system provides an environment to learn temporal bone surgery in a way similar to the experience with cadaver material where the subject is able to interact with the data without constraints (nondeterministic). Eventually, it may provide the "front end" to a large repository of various temporal bone pathologies that can be accessed through the Internet.

Update on surgical simulation: the Ohio State University experience.

The authors discuss and review their experience with surgical simulation from early preoperative planning stations, to their involvement with functional endoscopic sinus surgery, to their current project. The future of such endeavors is discussed.

Virtual mastoidectomy performance evaluation through multi-volume analysis.

PURPOSE: Development of a visualization system that provides surgical instructors with a method to compare the results of many virtual surgeries (n > 100). METHODS: A masked distance field models the overlap between expert and resident results. Multiple volume displays are used side-by-side with a 2D point display. RESULTS: Performance characteristics were examined by comparing the results of specific residents with those of experts and the entire class. CONCLUSIONS: The software provides a promising approach for comparing performance between large groups of residents learning mastoidectomy techniques.

Translating the simulation of procedural drilling techniques for interactive neurosurgical training.

BACKGROUND: Through previous efforts we have developed a fully virtual environment to provide procedural training of otologic surgical technique. The virtual environment is based on high-resolution volumetric data of the regional anatomy. These volumetric data help drive an interactive multisensory, ie, visual (stereo), aural (stereo), and tactile, simulation environment. Subsequently, we have extended our efforts to support the training of neurosurgical procedural technique as part of the Congress of Neurological Surgeons simulation initiative. OBJECTIVE: To deliberately study the integration of simulation technologies into the neurosurgical curriculum and to determine their efficacy in teaching minimally invasive cranial and skull base approaches. METHODS: We discuss issues of biofidelity and our methods to provide objective, quantitative and automated assessment for the residents. RESULTS: We conclude with a discussion of our experiences by reporting preliminary formative pilot studies and proposed approaches to take the simulation to the next level through additional validation studies. CONCLUSION: We have presented our efforts to translate an otologic simulation environment for use in the neurosurgical curriculum. We have demonstrated the initial proof of principles and define the steps to integrate and validate the system as an adjuvant to the neurosurgical curriculum.