Vibro-Acoustic Sensing of Instrument Interactions as a Potential Source of Texture-Related Information in Robotic Palpation.

The direct tactile assessment of surface textures during palpation is an essential component of open surgery that is impeded in minimally invasive and robot-assisted surgery. When indirectly palpating with a surgical instrument, the structural vibrations from this interaction contain tactile information that can be extracted and analysed. This study investigates the influence of the parameters contact angle α and velocity v→ on the vibro-acoustic signals from this indirect palpation. A 7-DOF robotic arm, a standard surgical instrument, and a vibration measurement system were used to palpate three different materials with varying α and v→. The signals were processed based on continuous wavelet transformation. They showed material-specific signatures in the time-frequency domain that retained their general characteristic for varying α and v→. Energy-related and statistical features were extracted, and supervised classification was performed, where the testing data comprised only signals acquired with different palpation parameters than for training data. The classifiers support vector machine and k-nearest neighbours provided 99.67% and 96.00% accuracy for the differentiation of the materials. The results indicate the robustness of the features against variations in the palpation parameters. This is a prerequisite for an application in minimally invasive surgery but needs to be confirmed in realistic experiments with biological tissues.

Clustering Methods for Vibro-Acoustic Sensing Features as a Potential Approach to Tissue Characterisation in Robot-Assisted Interventions.

This article provides a comprehensive analysis of the feature extraction methods applied to vibro-acoustic signals (VA signals) in the context of robot-assisted interventions. The primary objective is to extract valuable information from these signals to understand tissue behaviour better and build upon prior research. This study is divided into three key stages: feature extraction using the Cepstrum Transform (CT), Mel-Frequency Cepstral Coefficients (MFCCs), and Fast Chirplet Transform (FCT); dimensionality reduction employing techniques such as Principal Component Analysis (PCA), t-Distributed Stochastic Neighbour Embedding (t-SNE), and Uniform Manifold Approximation and Projection (UMAP); and, finally, classification using a nearest neighbours classifier. The results demonstrate that using feature extraction techniques, especially the combination of CT and MFCC with dimensionality reduction algorithms, yields highly efficient outcomes. The classification metrics (Accuracy, Recall, and F1-score) approach 99%, and the clustering metric is 0.61. The performance of the CT-UMAP combination stands out in the evaluation metrics.

Surgeons' requirements for a surgical support system to improve laparoscopic access.

Creating surgical access is a critical step in laparoscopic surgery. Surgeons have to insert a sharp instrument such as the Veress needle or a trocar into the patient's abdomen until the peritoneal cavity is reached. They solely rely on their experience and distorted tactile feedback in that process, leading to a complication rate as high as 14% of all cases. Recent studies have shown the feasibility of surgical support systems that provide intraoperative feedback regarding the insertion process to improve laparoscopic access outcomes. However, to date, the surgeons' requirements for such support systems remain unclear. This research article presents the results of an explorative study that aimed to acquire data about the information that helps surgeons improve laparoscopic access outcomes. The results indicate that feedback regarding the reaching of the peritoneal cavity is of significant importance and should be presented visually or acoustically. Finally, a solution should be straightforward and intuitive to use, should support or even improve the clinical workflow, but also cheap enough to facilitate its usage rate. While this study was tailored to laparoscopic access, its results also apply to other minimally invasive procedures.

The Advances in Computer Vision That Are Enabling More Autonomous Actions in Surgery: A Systematic Review of the Literature.

This is a review focused on advances and current limitations of computer vision (CV) and how CV can help us obtain to more autonomous actions in surgery. It is a follow-up article to one that we previously published in Sensors entitled, "Artificial Intelligence Surgery: How Do We Get to Autonomous Actions in Surgery?" As opposed to that article that also discussed issues of machine learning, deep learning and natural language processing, this review will delve deeper into the field of CV. Additionally, non-visual forms of data that can aid computerized robots in the performance of more autonomous actions, such as instrument priors and audio haptics, will also be highlighted. Furthermore, the current existential crisis for surgeons, endoscopists and interventional radiologists regarding more autonomy during procedures will be discussed. In summary, this paper will discuss how to harness the power of CV to keep doctors who do interventions in the loop.

Artificial Intelligence Surgery: How Do We Get to Autonomous Actions in Surgery?

Most surgeons are skeptical as to the feasibility of autonomous actions in surgery. Interestingly, many examples of autonomous actions already exist and have been around for years. Since the beginning of this millennium, the field of artificial intelligence (AI) has grown exponentially with the development of machine learning (ML), deep learning (DL), computer vision (CV) and natural language processing (NLP). All of these facets of AI will be fundamental to the development of more autonomous actions in surgery, unfortunately, only a limited number of surgeons have or seek expertise in this rapidly evolving field. As opposed to AI in medicine, AI surgery (AIS) involves autonomous movements. Fortuitously, as the field of robotics in surgery has improved, more surgeons are becoming interested in technology and the potential of autonomous actions in procedures such as interventional radiology, endoscopy and surgery. The lack of haptics, or the sensation of touch, has hindered the wider adoption of robotics by many surgeons; however, now that the true potential of robotics can be comprehended, the embracing of AI by the surgical community is more important than ever before. Although current complete surgical systems are mainly only examples of tele-manipulation, for surgeons to get to more autonomously functioning robots, haptics is perhaps not the most important aspect. If the goal is for robots to ultimately become more and more independent, perhaps research should not focus on the concept of haptics as it is perceived by humans, and the focus should be on haptics as it is perceived by robots/computers. This article will discuss aspects of ML, DL, CV and NLP as they pertain to the modern practice of surgery, with a focus on current AI issues and advances that will enable us to get to more autonomous actions in surgery. Ultimately, there may be a paradigm shift that needs to occur in the surgical community as more surgeons with expertise in AI may be needed to fully unlock the potential of AIS in a safe, efficacious and timely manner.

Deep Convolution Neural Network for Laryngeal Cancer Classification on Contact Endoscopy-Narrow Band Imaging.

(1) Background: Contact Endoscopy (CE) and Narrow Band Imaging (NBI) are optical imaging modalities that can provide enhanced and magnified visualization of the superficial vascular networks in the laryngeal mucosa. The similarity of vascular structures between benign and malignant lesions causes a challenge in the visual assessment of CE-NBI images. The main objective of this study is to use Deep Convolutional Neural Networks (DCNN) for the automatic classification of CE-NBI images into benign and malignant groups with minimal human intervention. (2) Methods: A pretrained Res-Net50 model combined with the cut-off-layer technique was selected as the DCNN architecture. A dataset of 8181 CE-NBI images was used during the fine-tuning process in three experiments where several models were generated and validated. The accuracy, sensitivity, and specificity were calculated as the performance metrics in each validation and testing scenario. (3) Results: Out of a total of 72 trained and tested models in all experiments, Model 5 showed high performance. This model is considerably smaller than the full ResNet50 architecture and achieved the testing accuracy of 0.835 on the unseen data during the last experiment. (4) Conclusion: The proposed fine-tuned ResNet50 model showed a high performance to classify CE-NBI images into the benign and malignant groups and has the potential to be part of an assisted system for automatic laryngeal cancer detection.

Vascular Auscultation of Carotid Artery: Towards Biometric Identification and Verification of Individuals.

BACKGROUND: Biometric sensing is a security method for protecting information and property. State-of-the-art biometric traits are behavioral and physiological in nature. However, they are vulnerable to tampering and forgery. METHODS: The proposed approach uses blood flow sounds in the carotid artery as a source of biometric information. A handheld sensing device and an associated desktop application were built. Between 80 and 160 carotid recordings of 11 s in length were acquired from seven individuals each. Wavelet-based signal analysis was performed to assess the potential for biometric applications. RESULTS: The acquired signals per individual proved to be consistent within one carotid sound recording and between multiple recordings spaced by several weeks. The averaged continuous wavelet transform spectra for all cardiac cycles of one recording showed specific spectral characteristics in the time-frequency domain, allowing for the discrimination of individuals, which could potentially serve as an individual fingerprint of the carotid sound. This is also supported by the quantitative analysis consisting of a small convolutional neural network, which was able to differentiate between different users with over 95% accuracy. CONCLUSION: The proposed approach and processing pipeline appeared promising for the discrimination of individuals. The biometrical recognition could clinically be used to obtain and highlight differences from a previously established personalized audio profile and subsequently could provide information on the source of the deviation as well as on its effects on the individual's health. The limited number of individuals and recordings require a study in a larger population along with an investigation of the long-term spectral stability of carotid sounds to assess its potential as a biometric marker. Nevertheless, the approach opens the perspective for automatic feature extraction and classification.

Laryngeal Lesion Classification Based on Vascular Patterns in Contact Endoscopy and Narrow Band Imaging: Manual Versus Automatic Approach.

Longitudinal and perpendicular changes in the vocal fold's blood vessels are associated with the development of benign and malignant laryngeal lesions. The combination of Contact Endoscopy (CE) and Narrow Band Imaging (NBI) can provide intraoperative real-time visualization of the vascular changes in the laryngeal mucosa. However, the visual evaluation of vascular patterns in CE-NBI images is challenging and highly depends on the clinicians' experience. The current study aims to evaluate and compare the performance of a manual and an automatic approach for laryngeal lesion's classification based on vascular patterns in CE-NBI images. In the manual approach, six observers visually evaluated a series of CE+NBI images that belong to a patient and then classified the patient as benign or malignant. For the automatic classification, an algorithm based on characterizing the level of the vessel's disorder in combination with four supervised classifiers was used to classify CE-NBI images. The results showed that the manual approach's subjective evaluation could be reduced by using a computer-based approach. Moreover, the automatic approach showed the potential to work as an assistant system in case of disagreements among clinicians and to reduce the manual approach's misclassification issue.

Thyroid Nodule Classification for Physician Decision Support Using Machine Learning-Evaluated Geometric and Morphological Features.

The classification of thyroid nodules using ultrasound (US) imaging is done using the Thyroid Imaging Reporting and Data System (TIRADS) guidelines that classify nodules based on visual and textural characteristics. These are composition, shape, size, echogenicity, calcifications, margins, and vascularity. This work aims to reduce subjectivity in the current diagnostic process by using geometric and morphological (G-M) features that represent the visual characteristics of thyroid nodules to provide physicians with decision support. A total of 27 G-M features were extracted from images obtained from an open-access US thyroid nodule image database. 11 significant features in accordance with TIRADS were selected from this global feature set. Each feature was labeled (0 = benign and 1 = malignant) and the performance of the selected features was evaluated using machine learning (ML). G-M features together with ML resulted in the classification of thyroid nodules with a high accuracy, sensitivity and specificity. The results obtained here were compared against state-of the-art methods and perform significantly well in comparison. Furthermore, this method can act as a computer aided diagnostic (CAD) system for physicians by providing them with a validation of the TIRADS visual characteristics used for the classification of thyroid nodules in US images.

Enhancing Veress Needle Entry with Proximal Vibroacoustic Sensing for Automatic Identification of Peritoneum Puncture.

Laparoscopic access, a critical yet challenging step in surgical procedures, often leads to complications. Existing systems, such as improved Veress needles and optical trocars, offer limited safety benefits but come with elevated costs. In this study, a prototype of a novel technology for guiding needle interventions based on vibroacoustic signals is evaluated in porcine cadavers. The prototype consistently detected successful abdominal cavity entry in 100% of cases during 193 insertions across eight porcine cadavers. The high signal quality allowed for the precise identification of all Veress needle insertion phases, including peritoneum puncture. The findings suggest that this vibroacoustic-based guidance technology could enhance surgeons' situational awareness and provide valuable support during laparoscopic access. Unlike existing solutions, this technology does not require sensing elements in the instrument's tip and remains compatible with medical instruments from various manufacturers.

Contact Endoscopy - Narrow Band Imaging (CE-NBI) data set for laryngeal lesion assessment.

The endoscopic examination of subepithelial vascular patterns within the vocal fold is crucial for clinicians seeking to distinguish between benign lesions and laryngeal cancer. Among innovative techniques, Contact Endoscopy combined with Narrow Band Imaging (CE-NBI) offers real-time visualization of these vascular structures. Despite the advent of CE-NBI, concerns have arisen regarding the subjective interpretation of its images. As a result, several computer-based solutions have been developed to address this issue. This study introduces the CE-NBI data set, the first publicly accessible data set that features enhanced and magnified visualizations of subepithelial blood vessels within the vocal fold. This data set encompasses 11144 images from 210 adult patients with pathological vocal fold conditions, where CE-NBI images are annotated using three distinct label categories. The data set has proven invaluable for numerous clinical assessments geared toward diagnosing laryngeal cancer using Optical Biopsy. Furthermore, given its versatility for various image analysis tasks, we have devised and implemented diverse image classification scenarios using Machine Learning (ML) approaches to address critical clinical challenges in assessing laryngeal lesions.

Vibro-acoustic sensing of tissue-instrument-interactions allows a differentiation of biological tissue in computerised palpation.

BACKGROUND: The shift towards minimally invasive surgery is associated with a significant reduction of tactile information available to the surgeon, with compensation strategies ranging from vision-based techniques to the integration of sensing concepts into surgical instruments. Tactile information is vital for palpation tasks such as the differentiation of tissues or the characterisation of surfaces. This work investigates a new sensing approach to derive palpation-related information from vibration signals originating from instrument-tissue-interactions. METHODS: We conducted a feasibility study to differentiate three non-animal and three animal tissue specimens based on palpation of the surface. A sensor configuration was mounted at the proximal end of a standard instrument opposite the tissue-interaction point. Vibro-acoustic signals of 1680 palpation events were acquired, and the time-varying spectrum was computed using Continuous-Wavelet-Transformation. For validation, nine spectral energy-related features were calculated for a subsequent classification using linear Support Vector Machine and k-Nearest-Neighbor. RESULTS: Indicators derived from the vibration signal are highly stable in a set of palpations belonging to the same tissue specimen, regardless of the palpating subject. Differences in the surface texture of the tissue specimens reflect in those indicators and can serve as a basis for differentiation. The classification following a supervised learning approach shows an accuracy of >93.8% for the three-tissue classification tasks and decreases to 78.8% for a combination of all six tissues. CONCLUSIONS: Simple features derived from the vibro-acoustic signals facilitate the differentiation between biological tissues, showing the potential of the presented approach to provide information related to the interacting tissue. The results encourage further investigation of a yet little-exploited source of information in minimally invasive surgery.

Thyroid Nodule Detection and Region Estimation in Ultrasound Images: A Comparison between Physicians and an Automated Decision Support System Approach.

BACKGROUND: Thyroid nodules are very common. In most cases, they are benign, but they can be malignant in a low percentage of cases. The accurate assessment of these nodules is critical to choosing the next diagnostic steps and potential treatment. Ultrasound (US) imaging, the primary modality for assessing these nodules, can lack objectivity due to varying expertise among physicians. This leads to observer variability, potentially affecting patient outcomes. PURPOSE: This study aims to assess the potential of a Decision Support System (DSS) in reducing these variabilities for thyroid nodule detection and region estimation using US images, particularly in lesser experienced physicians. METHODS: Three physicians with varying levels of experience evaluated thyroid nodules on US images, focusing on nodule detection and estimating cystic and solid regions. The outcomes were compared to those obtained from a DSS for comparison. Metrics such as classification match percentage and variance percentage were used to quantify differences. RESULTS: Notable disparities exist between physician evaluations and the DSS assessments: the overall classification match percentage was just 19.2%. Individually, Physicians 1, 2, and 3 had match percentages of 57.6%, 42.3%, and 46.1% with the DSS, respectively. Variances in assessments highlight the subjectivity and observer variability based on physician experience levels. CONCLUSIONS: The evident variability among physician evaluations underscores the need for supplementary decision-making tools. Given its consistency, the CAD offers potential as a reliable "second opinion" tool, minimizing human-induced variabilities in the critical diagnostic process of thyroid nodules using US images. Future integration of such systems could bolster diagnostic precision and improve patient outcomes.

Endoscopic Imaging Technology Today.

One of the most applied imaging methods in medicine is endoscopy. A highly specialized image modality has been developed since the first modern endoscope, the "Lichtleiter" of Bozzini was introduced in the early 19th century. Multiple medical disciplines use endoscopy for diagnostics or to visualize and support therapeutic procedures. Therefore, the shapes, functionalities, handling concepts, and the integrated and surrounding technology of endoscopic systems were adapted to meet these dedicated medical application requirements. This survey gives an overview of modern endoscopic technology's state of the art. Therefore, the portfolio of several manufacturers with commercially available products on the market was screened and summarized. Additionally, some trends for upcoming developments were collected.

Cyclist Effort Features: A Novel Technique for Image Texture Characterization Applied to Larynx Cancer Classification in Contact Endoscopy-Narrow Band Imaging.

BACKGROUND: Feature extraction is an essential part of a Computer-Aided Diagnosis (CAD) system. It is usually preceded by a pre-processing step and followed by image classification. Usually, a large number of features is needed to end up with the desired classification results. In this work, we propose a novel approach for texture feature extraction. This method was tested on larynx Contact Endoscopy (CE)-Narrow Band Imaging (NBI) image classification to provide more objective information for otolaryngologists regarding the stage of the laryngeal cancer. METHODS: The main idea of the proposed methods is to represent an image as a hilly surface, where different paths can be identified between a starting and an ending point. Each of these paths can be thought of as a Tour de France stage profile where a cyclist needs to perform a specific effort to arrive at the finish line. Several paths can be generated in an image where different cyclists produce an average cyclist effort representing important textural characteristics of the image. Energy and power as two Cyclist Effort Features (CyEfF) were extracted using this concept. The performance of the proposed features was evaluated for the classification of 2701 CE-NBI images into benign and malignant lesions using four supervised classifiers and subsequently compared with the performance of 24 Geometrical Features (GF) and 13 Entropy Features (EF). RESULTS: The CyEfF features showed maximum classification accuracy of 0.882 and improved the GF classification accuracy by 3 to 12 percent. Moreover, CyEfF features were ranked as the top 10 features along with some features from GF set in two feature ranking methods. CONCLUSION: The results prove that CyEfF with only two features can describe the textural characterization of CE-NBI images and can be part of the CAD system in combination with GF for laryngeal cancer diagnosis.

Study of needle punctures into soft tissue through audio and force sensing: can audio be a simple alternative for needle guidance?

PURPOSE: Percutaneous needle insertion is one of the most common minimally invasive procedures. The clinician's experience and medical imaging support are essential to the procedure's safety. However, imaging comes with inaccuracies due to artifacts, and therefore sensor-based solutions were proposed to improve accuracy. However, sensors are usually embedded in the needle tip, leading to design limitations. A novel concept was proposed for capturing tip-tissue interaction information through audio sensing, showing promising results for needle guidance. This work demonstrates that this audio approach can provide important puncture information by comparing audio and force signal dynamics during insertion. METHODS: An experimental setup for inserting a needle into soft tissue was prepared. Audio and force signals were synchronously recorded at four different insertion velocities, and a dataset of 200 recordings was acquired. Indicators related to different aspects of the force and audio were compared through signal-to-signal and event-to-event correlation analysis. RESULTS: High signal-to-signal correlations between force and audio indicators regardless of the insertion velocity were obtained. The force curvature indicator obtained the best correlation performances to audio with more than [Formula: see text] of the correlations higher than 0.6. The event-to-event correlation analysis shows that a puncture event in the force is generally identifiable in audio and that their intensities firmly related. CONCLUSIONS: Audio contains valuable information for monitoring needle tip/tissue interaction. Significant dynamics obtained from a well-known sensor as force can also be extracted from audio, regardless of insertion velocities.

Auscultation System for Acquisition of Vascular Sounds - Towards Sound-Based Monitoring of the Carotid Artery.

INTRODUCTION: Atherosclerotic diseases of the carotid are a primary cause of cerebrovascular events such as stroke. For the diagnosis and monitoring angiography, ultrasound- or magnetic resonance-based imaging is used which requires costly hardware. In contrast, the auscultation of carotid sounds and screening for bruits - audible patterns related to turbulent blood flow - is a simple examination with comparably little technical demands. It can indicate atherosclerotic diseases and justify further diagnostics but is currently subjective and examiner dependent. METHODS: We propose an easy-to-use computer-assisted auscultation system for a stable and reproducible acquisition of vascular sounds of the carotid. A dedicated skin-transducer-interface was incorporated into a handheld device. The interface comprises two bell-shaped structures, one with additional acoustic membrane, to ensure defined skin contact and a stable propagation path of the sound. The device is connected wirelessly to a desktop application allowing real-time visualization, assessment of signal quality and input of supplementary information along with storage of recordings in a database. An experimental study with 5 healthy subjects was conducted to evaluate usability and stability of the device. Five recordings per carotid served as data basis for a wavelet-based analysis of the stability of spectral characteristics of the recordings. RESULTS: The energy distribution of the wavelet-based stationary spectra proved stable for measurements of a particular carotid with the majority of the energy located between 3 and 40 Hz. Different spectral properties of the carotids of one individual indicate the presence of sound characteristics linked to the particular vessel. User-dependent parameters such as variations of the applied contact pressure appeared to have minor influence on the general stability. CONCLUSION: The system provides a platform for reproducible carotid auscultation and the creation of a database of pathological vascular sounds, which is a prerequisite to investigate sound-based vascular monitoring.

Evaluation of Vascular Patterns Using Contact Endoscopy and Narrow-Band Imaging (CE-NBI) for the Diagnosis of Vocal Fold Malignancy.

The endoscopic detection of perpendicular vascular changes (PVC) of the vocal folds has been associated with vocal fold cancer, dysplastic lesions, and papillomatosis, according to a classification proposed by the European Laryngological Society (ELS). The combination of contact endoscopy with narrow-band imaging (NBI-CE) allows intraoperatively a highly contrasted, real-time visualization of vascular changes of the vocal folds. Aim of the present study was to determine the association of PVC to specific histological diagnoses, the level of interobserver agreement in the detection of PVC, and their diagnostic effectiveness in diagnosing laryngeal malignancy. The evaluation of our data confirmed the association of PVC to vocal fold cancer, dysplastic lesions, and papillomatosis. The level of agreement between the observers in the identification of PVC was moderate for the less-experienced observers and almost perfect for the experienced observers. The identification of PVC during NBI-CE proved to be a valuable indicator for diagnosing malignant and premalignant lesions.

Novel Ultrasound Texture Based Similarity Metric Using Autoregressive Modelling.

Designing an ultrasound (US) specific similarity metric is essential in integrating advanced techniques like image segmentation and registration to US based interventional procedures. Applying conventional similarity metrics to ultrasound images is hampered by intrinsic noise patterns in an US image. In this work, we propose a texture based similarity metric (TexSimAR) using Autoregressive (AR) modelling. The key idea is to treat an US image as data resulting from a dynamical process which can be parametrically modelled. Using this approach it is possible to compute a parametric spectrum of individual US images and subsequently use it to estimate a similarity value between them. For evaluation, we used thyroid US images and similarity values were calculated between thyroid and non-thyroid regions. A cost function was designed to compare TexSimAR with other conventional similarity metrics. TexSimAR clearly distinguished between thyroid and non-thyroid regions outperfoming the conventional similarity metrics.

Cardiotocograph Data Classification Improvement by Using Empirical Mode Decomposition.

This work proposes to study the fetal heart rate (FHR) signal based on information about its dynamics as a signal resulting from the modulation by the autonomic nervous system. The analysis is performed using the complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) technique. The main idea is to extract a set of signal features based on that technique and also conventional time-domain features proposed in the literature in order to study their performance by using a support vector machine (SVM) as a classifier. As a hypothesis, we postulate that by including CEEMDAN based features, the classification performance should improve compared with the performance achieved by conventional features. The proposed method has been evaluated using real FHR data extracted from the open access CTU-UHB database. Results show that the classification performance improved from 67, 6% using only conventional features, to 71, 7% by incorporating CEEMDAN based features.