Leading in the development, standardised evaluation, and adoption of artificial intelligence in clinical practice: regional anaesthesia as an example.

A recent study by Suissa and colleagues explored the clinical relevance of a medical image segmentation metric (Dice metric) commonly used in the field of artificial intelligence (AI). They showed that pixel-wise agreement for physician identification of structures on ultrasound images is variable, and a relatively low Dice metric (0.34) correlated to a substantial agreement on subjective clinical assessment. We highlight the need to bring structure and clinical perspective to the evaluation of medical AI, which clinicians are best placed to direct.

Artificial intelligence for ultrasound scanning in regional anaesthesia: a scoping review of the evidence from multiple disciplines.

BACKGROUND: Artificial intelligence (AI) for ultrasound scanning in regional anaesthesia is a rapidly developing interdisciplinary field. There is a risk that work could be undertaken in parallel by different elements of the community but with a lack of knowledge transfer between disciplines, leading to repetition and diverging methodologies. This scoping review aimed to identify and map the available literature on the accuracy and utility of AI systems for ultrasound scanning in regional anaesthesia. METHODS: A literature search was conducted using Medline, Embase, CINAHL, IEEE Xplore, and ACM Digital Library. Clinical trial registries, a registry of doctoral theses, regulatory authority databases, and websites of learned societies in the field were searched. Online commercial sources were also reviewed. RESULTS: In total, 13,014 sources were identified; 116 were included for full-text review. A marked change in AI techniques was noted in 2016-17, from which point on the predominant technique used was deep learning. Methods of evaluating accuracy are variable, meaning it is impossible to compare the performance of one model with another. Evaluations of utility are more comparable, but predominantly gained from the simulation setting with limited clinical data on efficacy or safety. Study methodology and reporting lack standardisation. CONCLUSIONS: There is a lack of structure to the evaluation of accuracy and utility of AI for ultrasound scanning in regional anaesthesia, which hinders rigorous appraisal and clinical uptake. A framework for consistent evaluation is needed to inform model evaluation, allow comparison between approaches/models, and facilitate appropriate clinical adoption.

Research priorities in regional anaesthesia: an international Delphi study.

BACKGROUND: Regional anaesthesia use is growing worldwide, and there is an increasing emphasis on research in regional anaesthesia to improve patient outcomes. However, priorities for future study remain unclear. We therefore conducted an international research prioritisation exercise, setting the agenda for future investigators and funding bodies. METHODS: We invited members of specialist regional anaesthesia societies from six continents to propose research questions that they felt were unanswered. These were consolidated into representative indicative questions, and a literature review was undertaken to determine if any indicative questions were already answered by published work. Unanswered indicative questions entered a three-round modified Delphi process, whereby 29 experts in regional anaesthesia (representing all participating specialist societies) rated each indicative question for inclusion on a final high priority shortlist. If ≥75% of participants rated an indicative question as 'definitely' include in any round, it was accepted. Indicative questions rated as 'definitely' or 'probably' by <50% of participants in any round were excluded. Retained indicative questions were further ranked based on the rating score in the final Delphi round. The final research priorities were ratified by the Delphi expert group. RESULTS: There were 1318 responses from 516 people in the initial survey, from which 71 indicative questions were formed, of which 68 entered the modified Delphi process. Eleven 'highest priority' research questions were short listed, covering themes of pain management; training and assessment; clinical practice and efficacy; technology and equipment. CONCLUSIONS: We prioritised unanswered research questions in regional anaesthesia. These will inform a coordinated global research strategy for regional anaesthesia and direct investigators to address high-priority areas.

Evaluation of the impact of assistive artificial intelligence on ultrasound scanning for regional anaesthesia.

BACKGROUND: Ultrasound-guided regional anaesthesia relies on the visualisation of key landmark, target, and safety structures on ultrasound. However, this can be challenging, particularly for inexperienced practitioners. Artificial intelligence (AI) is increasingly being applied to medical image interpretation, including ultrasound. In this exploratory study, we evaluated ultrasound scanning performance by non-experts in ultrasound-guided regional anaesthesia, with and without the use of an assistive AI device. METHODS: Twenty-one anaesthetists, all non-experts in ultrasound-guided regional anaesthesia, underwent a standardised teaching session in ultrasound scanning for six peripheral nerve blocks. All then performed a scan for each block; half of the scans were performed with AI assistance and half without. Experts assessed acquisition of the correct block view and correct identification of sono-anatomical structures on each view. Participants reported scan confidence, experts provided a global rating score of scan performance, and scans were timed. RESULTS: Experts assessed 126 ultrasound scans. Participants acquired the correct block view in 56/62 (90.3%) scans with the device compared with 47/62 (75.1%) without (P=0.031, two data points lost). Correct identification of sono-anatomical structures on the view was 188/212 (88.8%) with the device compared with 161/208 (77.4%) without (P=0.002). There was no significant overall difference in participant confidence, expert global performance score, or scan time. CONCLUSIONS: Use of an assistive AI device was associated with improved ultrasound image acquisition and interpretation. Such technology holds potential to augment performance of ultrasound scanning for regional anaesthesia by non-experts, potentially expanding patient access to these techniques. CLINICAL TRIAL REGISTRATION: NCT05156099.

Assistive artificial intelligence for ultrasound image interpretation in regional anaesthesia: an external validation study.

BACKGROUND: Ultrasonound is used to identify anatomical structures during regional anaesthesia and to guide needle insertion and injection of local anaesthetic. ScanNav Anatomy Peripheral Nerve Block (Intelligent Ultrasound, Cardiff, UK) is an artificial intelligence-based device that produces a colour overlay on real-time B-mode ultrasound to highlight anatomical structures of interest. We evaluated the accuracy of the artificial-intelligence colour overlay and its perceived influence on risk of adverse events or block failure. METHODS: Ultrasound-guided regional anaesthesia experts acquired 720 videos from 40 volunteers (across nine anatomical regions) without using the device. The artificial-intelligence colour overlay was subsequently applied. Three more experts independently reviewed each video (with the original unmodified video) to assess accuracy of the colour overlay in relation to key anatomical structures (true positive/negative and false positive/negative) and the potential for highlighting to modify perceived risk of adverse events (needle trauma to nerves, arteries, pleura, and peritoneum) or block failure. RESULTS: The artificial-intelligence models identified the structure of interest in 93.5% of cases (1519/1624), with a false-negative rate of 3.0% (48/1624) and a false-positive rate of 3.5% (57/1624). Highlighting was judged to reduce the risk of unwanted needle trauma to nerves, arteries, pleura, and peritoneum in 62.9-86.4% of cases (302/480 to 345/400), and to increase the risk in 0.0-1.7% (0/160 to 8/480). Risk of block failure was reported to be reduced in 81.3% of scans (585/720) and to be increased in 1.8% (13/720). CONCLUSIONS: Artificial intelligence-based devices can potentially aid image acquisition and interpretation in ultrasound-guided regional anaesthesia. Further studies are necessary to demonstrate their effectiveness in supporting training and clinical practice. CLINICAL TRIAL REGISTRATION: NCT04906018.

Augmented Reality in Ultrasound-Guided Regional Anaesthesia: An Exploratory Study on Models With Potential Implications for Training.

Introduction Needle tip visualisation is a key skill required for the safe practice of ultrasound-guided regional anaesthesia (UGRA). This exploratory study assesses the utility of a novel augmented reality device, NeedleTrainer™, to differentiate between anaesthetists with varying levels of UGRA experience in a simulated environment. Methods Four groups of five participants were recruited (n = 20): novice, early career, experienced anaesthetists, and UGRA experts. Each participant performed three simulated UGRA blocks using NeedleTrainer™ on healthy volunteers (n = 60). The primary aim was to determine whether there was a difference in needle tip visibility, as calculated by the device, between groups of anaesthetists with differing levels of UGRA experience. Secondary aims included the assessment of simulated block conduct by an expert assessor and subjective participant self-assessment. Results The percentage of time the simulated needle tip was maintained in view was higher in the UGRA expert group (57.1%) versus the other three groups (novice 41.8%, early career 44.5%, and experienced anaesthetists 43.6%), but did not reach statistical significance (p = 0.05). An expert assessor was able to differentiate between participants of different UGRA experience when assessing needle tip visibility (novice 3.3 out of 10, early career 5.1, experienced anaesthetists 5.9, UGRA expert group 8.7; p < 0.01) and final needle tip placement (novice 4.2 out of 10, early career 5.6, experienced anaesthetists 6.8, UGRA expert group 8.9; p < 0.01). Subjective self-assessment by participants did not differentiate UGRA experience when assessing needle tip visibility (p = 0.07) or final needle tip placement (p = 0.07). Discussion An expert assessor was able to differentiate between participants with different levels of UGRA experience in this simulated environment. Objective NeedleTrainer™ and subjective participant assessments did not reach statistical significance. The findings are novel as simulated needling using live human subjects has not been assessed before, and no previous studies have attempted to objectively quantify needle tip visibility during simulated UGRA techniques. Future research should include larger sample sizes to further assess the potential use of such technology.