RESUMO
BACKGROUND: Though there has been considerable effort to implement machine learning (ML) methods for health care, clinical implementation has lagged. Incorporating explainable machine learning (XML) methods through the development of a decision support tool using a design thinking approach is expected to lead to greater uptake of such tools. OBJECTIVE: This work aimed to explore how constant engagement of clinician end users can address the lack of adoption of ML tools in clinical contexts due to their lack of transparency and address challenges related to presenting explainability in a decision support interface. METHODS: We used a design thinking approach augmented with additional theoretical frameworks to provide more robust approaches to different phases of design. In particular, in the problem definition phase, we incorporated the nonadoption, abandonment, scale-up, spread, and sustainability of technology in health care (NASSS) framework to assess these aspects in a health care network. This process helped focus on the development of a prognostic tool that predicted the likelihood of admission to an intensive care ward based on disease severity in chest x-ray images. In the ideate, prototype, and test phases, we incorporated a metric framework to assess physician trust in artificial intelligence (AI) tools. This allowed us to compare physicians' assessments of the domain representation, action ability, and consistency of the tool. RESULTS: Physicians found the design of the prototype elegant, and domain appropriate representation of data was displayed in the tool. They appreciated the simplified explainability overlay, which only displayed the most predictive patches that cumulatively explained 90% of the final admission risk score. Finally, in terms of consistency, physicians unanimously appreciated the capacity to compare multiple x-ray images in the same view. They also appreciated the ability to toggle the explainability overlay so that both options made it easier for them to assess how consistently the tool was identifying elements of the x-ray image they felt would contribute to overall disease severity. CONCLUSIONS: The adopted approach is situated in an evolving space concerned with incorporating XML or AI technologies into health care software. We addressed the alignment of AI as it relates to clinician trust, describing an approach to wire framing and prototyping, which incorporates the use of a theoretical framework for trust in the design process itself. Moreover, we proposed that alignment of AI is dependent upon integration of end users throughout the larger design process. Our work shows the importance and value of engaging end users prior to tool development. We believe that the described approach is a unique and valuable contribution that outlines a direction for ML experts, user experience designers, and clinician end users on how to collaborate in the creation of trustworthy and usable XML-based clinical decision support tools.
RESUMO
BACKGROUND: A virtual reality (VR) neurosurgical simulator with haptic feedback may provide the best model for training and perfecting surgical techniques for transsphenoidal approaches to the sella turcica and cranial base. Currently there are 2 commercially available simulators: NeuroTouch (Cranio and Endo) developed by the National Research Council of Canada in collaboration with surgeons at teaching hospitals in Canada, and the Immersive Touch. Work in progress on other simulators at additional institutions is currently unpublished. OBJECTIVE: This article describes a newly developed application of the NeuroTouch simulator that facilitates the performance and assessment of technical skills for endoscopic endonasal transsphenoidal surgical procedures as well as plans for collecting metrics during its early use. METHODS: The main components of the NeuroTouch-Endo VR neurosurgical simulator are a stereovision system, bimanual haptic tool manipulators, and high-end computers. The software engine continues to evolve, allowing additional surgical tasks to be performed in the VR environment. Device utility for efficient practice and performance metrics continue to be developed by its originators in collaboration with neurosurgeons at several teaching hospitals in the United States. Training tasks are being developed for teaching 1- and 2-nostril endonasal transsphenoidal approaches. Practice sessions benefit from anatomic labeling of normal structures along the surgical approach and inclusion (for avoidance) of critical structures, such as the internal carotid arteries and optic nerves. CONCLUSION: The simulation software for NeuroTouch-Endo VR simulation of transsphenoidal surgery provides an opportunity for beta testing, validation, and evaluation of performance metrics for use in neurosurgical residency training. ABBREVIATIONS: CTA, cognitive task analysisVR, virtual reality.
