Robotic scrub nurse to anticipate surgical instruments based on real-time laparoscopic video analysis.

BACKGROUND: Machine learning and robotics technologies are increasingly being used in the healthcare domain to improve the quality and efficiency of surgeries and to address challenges such as staff shortages. Robotic scrub nurses in particular offer great potential to address staff shortages by assuming nursing tasks such as the handover of surgical instruments. METHODS: We introduce a robotic scrub nurse system designed to enhance the quality of surgeries and efficiency of surgical workflows by predicting and delivering the required surgical instruments based on real-time laparoscopic video analysis. We propose a three-stage deep learning architecture consisting of a single frame-, temporal multi frame-, and informed model to anticipate surgical instruments. The anticipation model was trained on a total of 62 laparoscopic cholecystectomies. RESULTS: Here, we show that our prediction system can accurately anticipate 71.54% of the surgical instruments required during laparoscopic cholecystectomies in advance, facilitating a smoother surgical workflow and reducing the need for verbal communication. As the instruments in the left working trocar are changed less frequently and according to a standardized procedure, the prediction system works particularly well for this trocar. CONCLUSIONS: The robotic scrub nurse thus acts as a mind reader and helps to mitigate staff shortages by taking over a great share of the workload during surgeries while additionally enabling an enhanced process standardization.

Staff shortages in healthcare are an emerging problem leading to undersupply of medical experts such as scrub nurses in the operating room. The absence of these scrub nurses, who are responsible for providing surgical instruments, means that surgeries must be postponed or canceled. Robotic technologies and artificial intelligence offer great potential to address staff shortages in the operating room. We developed a robotic scrub nurse system that is able to take over the tasks of a human scrub nurse by delivering the required surgical tools. To maintain the pace of the surgery, our robotic scrub nurse system is also capable of predicting these required surgical tools in advance using artificial intelligence. The system is tested on laparoscopic cholecystectomies, a surgery, where the gallbladder is removed in a minimally invasive technique. We show that our prediction system can predict the majority of surgical instruments for this specific surgery facilitating a smoother surgical workflow and reducing the need for verbal communication. With further development, our system may help to cover the need for surgery while streamlining the surgical process through predictive support, potentially improving patient outcomes.

Model-based individual life-spanning documentation in visceral surgery: a proof of concept.

INTRODUCTION: Surgical documentation has many implications. However, its primary function is to transfer information about surgical procedures to other medical professionals. Thereby, written reports describing procedures in detail are the current standard, impeding comprehensive understanding of patient-individual life-spanning surgical course, especially if surgeries are performed at a timely distance and in diverse facilities. Therefore, we developed a novel model-based approach for documentation of visceral surgeries, denoted as 'Surgical Documentation Markup-Modeling' (SDM-M). MATERIAL AND METHODS: For scientific evaluation, we developed a web-based prototype software allowing for creating hierarchical anatomical models that can be modified by individual surgery-related markup information. Thus, a patient's cumulated 'surgical load' can be displayed on a timeline deploying interactive anatomical 3D models. To evaluate the possible impact on daily clinical routine, we performed an evaluation study with 24 surgeons and advanced medical students, elaborating on simulated complex surgical cases, once with classic written reports and once with our prototypical SDM-M software. RESULTS: Leveraging SDM-M in an experimental environment reduced the time needed for elaborating simulated complex surgical cases from 354 ± 85 s with the classic approach to 277 ± 128 s. (p = 0.00109) The perceived task load measured by the Raw NASA-TLX was reduced significantly (p = 0.00003) with decreased mental (p = 0.00004) and physical (p = 0.01403) demand. Also, time demand (p = 0.00041), performance (p = 0.00161), effort (p = 0.00024), and frustration (p = 0.00031) were improved significantly. DISCUSSION: Model-based approaches for life-spanning surgical documentation could improve the daily clinical elaboration and understanding of complex cases in visceral surgery. Besides reduced workload and time sparing, even a more structured assessment of individual surgical cases could foster improved planning of further surgeries, information transfer, and even scientific evaluation, considering the cumulative 'surgical load.' CONCLUSION: Life-spanning model-based documentation of visceral surgical cases could significantly improve surgery and workload.

The sound of surgery-development of an acoustic trocar system enabling laparoscopic sound analysis.

PURPOSE: Acoustic information can contain viable information in medicine and specifically in surgery. While laparoscopy depends mainly on visual information, our goal is to develop the means to capture and process acoustic information during laparoscopic surgery. METHODS: To achieve this, we iteratively developed three prototypes that will overcome the abdominal wall as a sound barrier and can be used with standard trocars. We evaluated them in terms of clinical applicability and sound transmission quality. Furthermore, the applicability of each prototype for sound classification based on machine learning was evaluated. RESULTS: Our developed prototypes for recording airborne sound from the intraperitoneal cavity represent a promising solution suitable for real-world clinical usage All three prototypes fulfill our set requirements in terms of clinical applicability (i.e., air-tightness, invasiveness, sterility) and show promising results regarding their acoustic characteristics and the associated results on ML-based sound classification. CONCLUSION: In summary, our prototypes for capturing acoustic information during laparoscopic surgeries integrate seamlessly with existing procedures and have the potential to augment the surgeon's perception. This advancement could change how surgeons interact with and understand the surgical field.

Toward telemedical diagnostics-clinical evaluation of a robotic examination system for emergency patients.

Introduction: The SARS-CoV-2 pandemic has affected global public healthcare for several years. Numerous medical professionals have been infected since the outbreak in 2019, resulting in a shortage of healthcare providers. Since traditional personal protective wear was insufficient to eliminate the virus transmission reliably, new strategies to avoid cross-infection were imperative while enabling high-quality medical care. In the project ProteCT, we investigated the potential of robotic-assisted examination in providing medical examination via a telemedical approach. Material and Methods: We constructed a fully functional examination cabin equipped with cameras, microphones, screens and robotic arms to evaluate usability and perception. Therefore, we conducted a preliminary study with 10 healthy volunteers and 10 physicians to gain first insights and optimize the setup. In a second step, we performed telemedical examinations of actual patients from the local emergency department to compare the robotic approach with the classical method of measuring vital signs, auscultation, palpation and percussion. Results: The preliminary study identified basic requirements, such as the need for force-feedback and telemedical training for physicians. In the main study, acceptance was high and most patients indicated they would use a telemedical system again. Our setup enabled the physician to make the same diagnoses as by classic examination in the emergency department in most cases. Discussion: The potential acceptance of a telemedical system such as ProteCT is high. Robotic telemedical approaches could complement future healthcare beyond the Corona pandemic to reach rural areas or even war zones. Moreover, the daily clinical use of robotic telemedicine could improve patients' safety, the quality of perioperative management and the workflow in any medical facility. Conclusion: The development of telemedical and telerobotic systems is a multidisciplinary and complex challenge. However, acceptance of the proposed system was high among patients and physicians, indicating the potential use of similar systems for future healthcare.

Development of an image-based Random Forest classifier for prediction of surgery duration of laparoscopic sigmoid resections.

PURPOSE: Sigmoid diverticulitis is a disease with a high socioeconomic burden, accounting for a high number of left-sided colonic resections worldwide. Modern surgical scheduling relies on accurate prediction of operation times to enhance patient care and optimize healthcare resources. This study aims to develop a predictive model for surgery duration in laparoscopic sigmoid resections, based on preoperative CT biometric and demographic patient data. METHODS: This retrospective single-center cohort study included 85 patients who underwent laparoscopic sigmoid resection for diverticular disease. Potentially relevant procedure-specific anatomical parameters recommended by a surgical expert were measured in preoperative CT imaging. After random split into training and test set (75% / 25%) multiclass logistic regression was performed and a Random Forest classifier was trained on CT imaging parameters, patient age, and sex in the training cohort to predict categorized surgery duration. The models were evaluated in the test cohort using established performance metrics including receiver operating characteristics area under the curve (AUROC). RESULTS: The Random Forest model achieved a good average AUROC of 0.78. It allowed a very good prediction of long (AUROC = 0.89; specificity 0.71; sensitivity 1.0) and short (AUROC = 0.81; specificity 0.77; sensitivity 0.56) procedures. It clearly outperformed the multiclass logistic regression model (AUROC: average = 0.33; short = 0.31; long = 0.22). CONCLUSION: A Random Forest classifier trained on demographic and CT imaging biometric patient data could predict procedure duration outliers of laparoscopic sigmoid resections. Pending validation in a multicenter study, this approach could potentially improve procedure scheduling in visceral surgery and be scaled to other procedures.

The impact of prior endoscopic or surgical therapy on open Zenker's diverticulum surgery: analysis on a large single center cohort : Comparison of primary and revisional open surgery for Zenker's diverticulum.

BACKGROUND: Endoscopic treatment of Zenker's diverticulum is an attractive minimally invasive alternative compared to the classic open approach. However, increased recurrence rate were reported. In case of relapse, endoscopic therapy might be repeated, or alternatively open surgery is performed. This study aimed to identify potential differences in the outcomes between primary or secondary surgical treatment in Zenker's diverticulum. METHODS: From January 2003 to April 2019, 227 subsequent patients underwent surgical diverticulectomy and cervical myotomy at the surgical department of TUM. 41 of 227 patients had received previous therapy, either open or endoscopic. Perioperative parameters in priorly untreated patients were retrospectively compared to those after previous therapy (mostly endoscopic) with special regard to perioperative data and postoperative complications. Univariate and multivariate regression analyses were performed to identify predictors for postoperative complications. RESULTS: We could show that the number of complications (p = 0.047) in pretreated patients is significant higher as well as the severity after Clavien-Dindo (p = 0.025). Stapler line leakage, wound infections, and operative revision rate was higher also pretreated group. Pretreatment and surgery time showed a significant association with postoperative complications in univariate analysis. In multivariate analysis, pretreatment remained a significant independent predictor of complications. CONCLUSION: The present data indicate that endoscopic therapy might represent a risk factor for postoperative complications in case of relapse surgery. Therefore primary open surgery should be debated in patients with an increased high risk of relapse.

Long-term outcome of patients with epiphrenic diverticula: A retrospective single-center analysis over 20 years.

BACKGROUND: Epiphrenic diverticula are extremely rare. Evidence-based treatment recommendations are scarce. The primary study outcome was to examine whether surgical treatment in patients with epiphrenic diverticula leads to improved quality of life by outweighing the perioperative risks compared with conservative treatment. METHODS: All patients with an epiphrenic diverticula at our institution between 2001 and 2021 were retrospectively reviewed and followed-up using a specific questionnaire, including the Eating Assessment Tool, and Gastrointestinal Quality of Life Index quality-of-life scores. RESULTS: Of 51 patients with epiphrenic diverticula, 28 had surgery and 23 had conservative treatment. The most common symptom at presentation was dysphagia. Although 16 patients underwent open surgery, 12 had minimally invasive procedures. A prophylactic stent was applied intraoperatively in 6 patients. The morbidity rate in surgically treated patients was 50% (14/28), with a leakage in 43% (12/28; 33% for prophylactic stenting). Mortality was nil. At a median follow-up of 139 months, patients with surgery had better outcomes than those without (ie, less dysphagia [6/12 vs 11/12; P = .025]), a less likely pathologic Eating Assessment Tool score (4/12 vs 9/12; P = .041), and a nonsignificant better Gastrointestinal Quality of Life Index score (122 vs 112; P = .929). The rate of recurrence/progression of symptoms was significantly higher for conservatively treated patients (11/18 vs 6/27 for any surgery; P = .008), as well as for patients with minimally invasive procedure (5/12 vs 1/15 for open surgery; P = .030). CONCLUSION: Despite the high perioperative morbidity, surgical treatment of epiphrenic diverticula leads to an improved long-term quality of life and lower recurrence rates than conservative treatment.

Robotic Endoscope Control - State of the Art of Voice Control and Other Options for Laparoscopic Camera Robot Guidance.

In video-based surgery (VBS), the surgeon is no longer the master of his or her view. Visualization of the surgical field is mediated by a video camera guided by an assistant. The separation of visualization and surgical activities leads to significant drawbacks. Early in the history of VBS, active camera holders controlled directly by the surgeon were introduced to replace the human camera assistant. However, despite remarkable technological advances over the past 25 years, robotic camera guidance systems (RCGS) still play only a marginal role. This is less due to inadequate mechanical designs or kinematics, and mainly due to inadequate human-machine interaction. In most cases, a simple task-shift is required that increases the surgeon's mental workload instead of reducing it. The current state of the art in robotic camera guidance, as represented by the SOLOASSIST II RCGS (AKTORmed, Neutraubling, Germany), includes a sophisticated combination of direct manual control, joystick navigation and high-quality voice control that provides the most intuitive surgeon-machine interaction currently available. An in-depth analysis of further clinical needs and promising developments in operating room integration suggests that the addition of context sensitivity could actually enhance the assistive effect of RCGS, making them potentially superior to the average human camera guidance.

"Hybrid" scientific conference: lessons learned from the digital annual meeting of the CARS international conference during the Covid-19 pandemic.

OBJECTIVES: Due to the coronavirus disease 2019 (Covid-19) pandemic, all scientific conferences in the year 2020 had to be adapted in their form of presence to accommodate for safety regulations, postponed, or canceled entirely. As organizers of the annual Computer Assisted Radiology & Surgery International Conference & Exhibition (CARS)-Conference 2020, we decided to hold a "hybrid" conference, i.e., a virtual conference with partial presence to mitigate the drawbacks of a purely virtual conference. It is the purpose of this paper to describe the results and experience gained by our first hybrid conference. METHODS: Besides technical necessities like an online conferencing tool, we introduced additional personal namely the technical chairs and communication officers ensuring a smooth flow of presentations. To measure the success of the hybrid conference, we assessed various parameters during the conference (e.g., counting of adverse events, delays, and no-shows) and sent a questionnaire to participants for evaluation after the conference. RESULTS: We offered four types of presentation formats, whereas the majority of speakers presented their pre-produced videos including live discussions. Significant delays in sessions occurred during the morning sessions, which could be reduced during lunch breaks. The analysis of the influence of the distribution of the audience's location/time zone toward the attendance rate showed a high relevance for the American zone and only little influence for the Asian-Pacific region. Based on the questionnaire, 60% of responders considered the hybrid approach as superior and 12% as inferior to purely virtual conferences. CONCLUSIONS: Most scientific associations in 2020 had to struggle with a dramatic change: Regular, traditional meetings with personal communication and exchange, networking, and creation of new visions became obsolete almost instantly. As an alternative, virtual conferences became increasingly popular, and are offering additional advantages (e.g., reduction of cost for travel, lodging, and time on transit). To overcome the drawbacks of purely virtual conferences, we introduced a hybrid concept for the CARS-Congress. While certainly, those with the privilege to take part personally on-site did benefit most from the hybrid format. Facing upcoming waves of the Covid-19 Pandemic, with ongoing changes to the regulations on meetings and transit, hybrid conferences are a viable option for scientific conferences for the future.

[Ethical, legal and social implications in the use of artificial intelligence-based technologies in surgery : Principles, implementation and importance for the user]. / Ethische, legale und soziale Implikationen bei der Anwendung künstliche-Intelligenz-gestützter Technologien in der Chirurgie : Grundlagen, Umsetzung und Bedeutung für den Anwender.

Ethical, legal and social aspects are gaining increasingly more attention in the development and during the initial clinical application of medical devices. The introduction of elements of artificial intelligence (AI) and systems which are using AI makes this already complex topic even more challenging. The introduction of so-called dynamic AI or dynamic machine learning (ML) algorithms in this respect represents a turning point. Unlike conventional medical devices, the development of systems using dynamic AI is not yet complete at the beginning of the clinical application. The aim of a dynamic AI system is to continuously improve through practical use and by the processing of usage data. This continuous evolution, along with the lack of transparency regarding internal work processes, could make it difficult to understand the underlying rationale for the assessments made by the algorithms. This aspect affects the acceptance of the technology both by clinicians and patients and furthermore questions the autonomy of patients and clinicians in the course of the treatment process. A way out of this ethical and regulatory dilemma must urgently be found and will require extreme efforts from all stakeholders. At present, no consensual solution is apparent. What is quite certain, however, is that users, i.e. in concrete terms surgeons, must play a much more active role than they have done in the past when dealing with AI-based medical devices and should prepare themselves to actively accompany the software life cycle of AI technologies.

Surgical data science - from concepts toward clinical translation.

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

Telemedical percussion: objectifying a fundamental clinical examination technique for telemedicine.

PURPOSE: While demand for telemedicine is increasing, patients are currently restricted to tele-consultation for the most part. Fundamental diagnostics like the percussion still require the in person expertize of a physician. To meet today's challenges, a transformation of the manual percussion into a standardized, digital version, ready for telemedical execution is required. METHODS: In conjunction with a comprehensive telemedical diagnostic system, in which patients can get examined by a remote-physician, a series of three robotic end-effectors for mechanical percussion were developed. Comprising a motor, a magnetic and a pneumatic-based version, the devices strike a pleximeter to perform the percussion. Emitted sounds were captured using a microphone-equipped stethoscope. The 84 recordings were further integrated into a survey in order to classify lung and non-lung samples. RESULTS: The study with 21 participants comprised physicians, medical students and non-medical-related raters in equal parts. With 71.4% correctly classified samples, the ventral motorized device prevailed. While the result is significantly better compared to a manual or pneumatic percussion in this very setup, it only has a small edge over the magnetic devices. In addition, for all ventral versions non-lung regions were rather correctly identified than lung regions. CONCLUSION: The overall setup proves the feasibility of a telemedical percussion. Despite the fact, that produced sounds differ compared to today's manual technique, the study shows that a standardized mechanical percussion has the potential to improve the gold standard's accuracy. While further extensive medical evaluation is yet to come, the system paves the way for future uncompromised remote examinations.

New Method for Surgical Diagnostics - a Robotic Telemedical Approach.

Apart from the tremendous increase in the demand for telemedicine during the COVID-19 pandemic, the use of telemedical technology offers many advantages, such as better coverage of rural areas and improved access to specialists. While current telediagnostic possibilities are often limited to a verbal consultation, the field of surgery has already made use of robotics for one of the most challenging areas of medicine: invasive procedures. Since comprehensive diagnostics are a prerequisite for each surgery, we built upon the knowledge gained in telesurgery and developed a telediagnostic system that allows for an extensive perioperative and emergency examination. It is based on a robotic platform consisting of a remote lead robotic arm at the physician's site and a follower robot at the patient's site. Mirroring all movements directly and using force-feedback, both parties can precisely interact, enabling tasks such as auscultation, percussion, and palpation without the need for extensive training. Our overall setup also includes the possibility to measure and monitor all relevant vital parameters and can be used to perform ear and nasopharyngeal inspections as well as an automatic swab to screen for COVID or other contagious diseases prior to hospital admission. In this paper, we focus on the potential of this technology for the surgical community by demonstrating the ease of adding an ultrasound probe to our modular setup to perform a high-quality emergency ultrasound examination. While the system is not yet ready for everyday use in a hospital and drawbacks such as a high cost persist, our setup paves the way for the future use of telediagnostics in surgery.

COVID-19 and beyond: development of a comprehensive telemedical diagnostic framework.

PURPOSE: During the COVID-19 pandemic, a threatening bottleneck of medical staff arose due to a shortage of trained caregivers, who became infected while working with infectious patients. While telemedicine is rapidly evolving in the fields of teleconsultation and telesurgery, proper telediagnostic systems are not yet available, although the demand for contactless patient-doctor interaction is increasing. METHODS: In this project, the current limitations were addressed by developing a comprehensive telediagnostic system. Therefore, medical examinations have been assessed in collaboration with medical experts. Subsequently, a framework was developed, satisfying the relevant constraints of medical-, technical-, and hygienic- aspects in order to transform in-person examinations into a contactless procedure. Diagnostic steps were classified into three groups: assisted procedures carried out by the patient, teleoperated examination methods, and adoptions of conventional methods. RESULTS: The Telemedical Diagnostic Framework was implemented, resulting in a functional proof of concept, where potentially infectious patients could undergo a full medical examination. The system comprises, e.g., a naso-pharyngeal swab, an inspection of the oral cavity, auscultation, percussion, and palpation, based on robotic end-effectors. The physician is thereby connected using a newly developed user-interface and a lead robot, with force feedback control, that enables precise movements with the follower robot on the patient's side. CONCLUSION: Our concept proves the feasibility of a fully telediagnostic system, that consolidates available technology and new developments to an efficient solution enabling safe patient-doctor interaction. Besides infectious situations, this solution can also be applied to remote areas.

Evaluation of improved bi-manual endoscopic resection using a customizable 3D-printed manipulator system designed for use with standard endoscopes: a feasibility study using a porcine ex-vivo model.

Background and study aims A major drawback of endoscopic en-bloc resection technique is its inability to perform bimanual tasks. Although endoscopic platforms that enable bimanual tasks are commercially available, they are neither approved for various locations nor adaptable to specific patients and indications. Methods Based on evolution of an adaptive 3D-printable platform concept, system variants with different characteristic properties were evaluated for ESD scenarios, ex-vivo in two locations in the stomach and colorectum. Results In total 28 ESDs were performed (7 antrum, 7 corpus in inversion, 7 cecum, 7 rectum) in a porcine ex-vivo setup. ESD was feasible in 21 cases. Investigated manipulator variants are differently well suited for performing ESD within the varying interventions scenarios. Dual-arm manipulators allowed autonomous ESD, while single-arm flexible manipulators could be used more universally due to their compact design, especially for lesions difficult to access. Pediatric scopes were too frail to guide the overtube-manipulators in extremely angled positions. Working in the rectum was impaired using long-sized manipulator arms. Conclusions The presented endoscopic platform based on 3D-printable and customizable manipulator structures might be a promising approach for future improvement of ESD procedure. With regard to localization, especially flexible manipulators attached to standard endoscopes appear to be most promising for further application of specific and individualised manipulator systems.

Evaluation of long-term stability of monolithic 3D-printed robotic manipulator structures for minimally invasive surgery.

PURPOSE: In the era of patient-centered medicine, clinical procedures, tools and instruments should be individually adapted to the patient. In this context, the presented 3D-printed Single-Port Overtube Manipulator System follows the aims to provide patient- and task-specific disposable manipulators for minimally invasive surgery. In a first experiment, the robustness of the monolithic flexure hinge structures in use as robotic manipulators will be investigated. METHODS: Customizable monolithic manipulator structures designed by means of an automated design process and manufactured with selective laser sintering were investigated with regard to long-term stability in an endurance test. Therefore, a bare manipulator arm, an arm equipped with a standard instrument and finally loaded with an additional load of 0.5 N were evaluated by continuously following a trajectory within the workspace of the manipulator arms over a period of 90 min. RESULTS: The unloaded manipulator as well as the manipulator arm equipped with a standard instrument showed a sufficient reproducibility (deviation of 1.5 mm and 2.5 mm, respectively, on average) with regard to an application as telemanipulated master-slave surgical robotic system. The 3D-printed manipulators showed no damage and maintained integrity after the experiment. CONCLUSION: It has been shown that 3D-printed manipulators in principle are suitable for use as disposable surgical manipulator systems and offer a long-term stability over at least 90 min. The developed manipulator design shows great potential for the production of patient-, task- and user-specific robot systems. However, the manipulator geometries as well as the control strategies still show room for improvements.

Machine Learning in the OR: A Collaborative Environment for Surgical Interventions in Visceral Medicine.

Modern surgical methods are becoming increasingly sophisticated and the number of technical devices that are used during these interventions is increasing. However, the surgical operating room (OR) remains a mere conglomerate of unconnected medical devices. The increase in the complexity of device functionality, in addition to the demands of surgery, pushes human mental capacity to its limit. Hence, an "intelligent" collaborative support system would be more than welcome. We envision a "human-like" intelligent system, which could support the surgical team as a situation-aware consultant. This so-called "active collaborative support system" (ACSS) is based on four main pillars: real-time data inflow, a comprehensive knowledge-base, access to the Internet of Things (surgical devices), and an understanding of human language through natural language processing. Recent advances in the area of AI are bringing this ambitious goal within reach, but there is still a considerable amount of work to be done, including the establishment of a new way of thinking in the collaboration between surgeons and computer scientists/engineers, and possibly one day with intelligent machines-provided that AI systems can be sufficiently trusted.

Acoustic signal analysis of instrument-tissue interaction for minimally invasive interventions.

PURPOSE: Minimally invasive surgery (MIS) has become the standard for many surgical procedures as it minimizes trauma, reduces infection rates and shortens hospitalization. However, the manipulation of objects in the surgical workspace can be difficult due to the unintuitive handling of instruments and limited range of motion. Apart from the advantages of robot-assisted systems such as augmented view or improved dexterity, both robotic and MIS techniques introduce drawbacks such as limited haptic perception and their major reliance on visual perception. METHODS: In order to address the above-mentioned limitations, a perception study was conducted to investigate whether the transmission of intra-abdominal acoustic signals can potentially improve the perception during MIS. To investigate whether these acoustic signals can be used as a basis for further automated analysis, a large audio data set capturing the application of electrosurgery on different types of porcine tissue was acquired. A sliding window technique was applied to compute log-mel-spectrograms, which were fed to a pre-trained convolutional neural network for feature extraction. A fully connected layer was trained on the intermediate feature representation to classify instrument-tissue interaction. RESULTS: The perception study revealed that acoustic feedback has potential to improve the perception during MIS and to serve as a basis for further automated analysis. The proposed classification pipeline yielded excellent performance for four types of instrument-tissue interaction (muscle, fascia, liver and fatty tissue) and achieved top-1 accuracies of up to 89.9%. Moreover, our model is able to distinguish electrosurgical operation modes with an overall classification accuracy of 86.40%. CONCLUSION: Our proof-of-principle indicates great application potential for guidance systems in MIS, such as controlled tissue resection. Supported by a pilot perception study with surgeons, we believe that utilizing audio signals as an additional information channel has great potential to improve the surgical performance and to partly compensate the loss of haptic feedback.