A Framework for the Evaluation of Human Machine Interfaces of Robot-Assisted Colonoscopy.

The Human Machine Interface (HMI) of intraluminal robots has a crucial impact on the clinician's performance. It increases or decreases the difficulty of the tasks, and is connected to the users' physical and mental stress. OBJECTIVE: This article presents a framework to compare and evaluate different HMIs for robotic colonoscopy, with the objective of identifying the optimal HMI that minimises the clinician's effort and maximises the clinical outcomes. METHODS: The framework comprises a 1) a virtual simulator (clinically validated), 2) wearable sensors measuring the cognitive load, 3) a data collection unit of metrics correlated to the clinical performance, and 4) questionnaires exploring the users' impressions and perceived stress. The framework was tested with 42 clinicians investigating the optimal device for tele-operated control of robotic colonoscopes. Two control devices were selected and compared: a haptic serial-kinematic device and a standard videogame joypad. RESULTS: The haptic device was preferred by the endoscopists, but the joypad enabled better clinical performance and reduced cognitive and physical load. CONCLUSION: The framework can be used to evaluate different aspects of a HMI, both hardware and software, and determine the optimal HMI that can reduce the burden on clinicians while improving the clinical outcome. SIGNIFICANCE: The findings of this study, and of future studies performed with this framework, can inform the design and development of HMIs for intraluminal robots, leading to improved clinical performance, reduced physical and mental stress for clinicians, and ultimately better patient outcomes.

Endoluminal Procedures and Devices for Esophageal Tract Investigation: A Critical Review.

Diseases of the esophageal tract represent a heterogeneous class of pathological conditions for which diagnostic paradigms continue to emerge. In the last few decades, innovative diagnostic devices have been developed, and several attempts have been made to advance and standardize diagnostic algorithms to be compliant with medical procedures. To the best of our knowledge, a comprehensive review of the procedures and available technologies to investigate the esophageal tract was missing in the literature. Therefore, the proposed review aims to provide a comprehensive analysis of available endoluminal technologies and procedures to investigate esophagus health conditions. The proposed systematic review was performed using PubMed, Scopus, and Web of Science databases. Studies have been divided into categories based on the type of evaluation and measurement that the investigated technology provides. In detail, three main categories have been identified, i.e., endoluminal technologies for the (i) morphological, (ii) bio-mechanical, and (iii) electro-chemical evaluation of the esophagus.

Spherical Wrist Manipulator Local Planner for Redundant Tasks in Collaborative Environments.

Standard industrial robotic manipulators use well-established high performing technologies. However, such manipulators do not guarantee a safe Human-Robot Interaction (HRI), limiting their usage in industrial and medical applications. This paper proposes a novel local path planner for spherical wrist manipulators to control the execution of tasks where the manipulator number of joints is redundant. Such redundancy is used to optimize robot motion and dexterity. We present an intuitive parametrization of the end-effector (EE) angular motion, which decouples the rotation of the third joint of the wrist from the rest of the angular motions. Manipulator EE motion is controlled through a decentralized linear system with closed-loop architecture. The local planner integrates a novel collision avoidance strategy based on a potential repulsive vector applied to the EE. Contrary to classic potential field approaches, the collision avoidance algorithm considers the entire manipulator surface, enhancing human safety. The local path planner is simulated in three generic scenarios: (i) following a periodic reference, (ii) a random sequence of step signal references, and (iii) avoiding instantly introduced obstacles. Time and frequency domain analysis demonstrated that the developed planner, aside from better parametrizing redundant tasks, is capable of successfully executing the simulated paths (max error = 0.25°) and avoiding obstacles.

Sleep Deprivation-Induced Changes in Baseline Brain Activity and Vigilant Attention Performance.

Sleep deprivation (SD) negatively affects several aspects of cognitive performance, and one of the most widely-used tools to evaluate these effects is the Psychomotor Vigilance Test (PVT). The present study investigated the possibility of predicting changes induced by SD in vigilant attention performance by evaluating the baseline electroencephalographic (EEG) activity immediately preceding the PVT stimuli onset. All participants (n = 10) underwent EEG recordings during 10 min of PVT before and after a night of SD. For each participant, the root mean square (RMS) of the baseline EEG signal was evaluated for each 1 s time window, and the respective average value was computed. After SD, participants showed slower (and less accurate) performance in the PVT task. Moreover, a close relationship between the changes in the baseline activity with those in cognitive performance was identified at several electrodes (Fp2, F7, F8, P3, T6, O1, Oz, O2), with the highest predictive power at the occipital derivations. These results indicate that vigilant attention impairments induced by SD can be predicted by the pre-stimulus baseline activity changes.

Decision-Making Algorithm and Predictive Model to Assess the Impact of Infectious Disease Epidemics on the Healthcare System: The COVID-19 Case Study in Italy.

To improve decision-making strategies and prediction based on epidemiological data, so far biased by highly-variable criteria, algorithms using unbiased morbidity parameters, i.e. Intensive Care Units (ICU) and Ordinary Hospitalizations (OH), are proposed. ICU/OH acceleration and velocities are mathematically modeled using available and official data to derive two thresholds, alerting on 30 % ICU and 40 % OH of COVID-19 daily occupancy settled by the Italian Minister of Health, as a case of study. A predictive model is also proposed to estimate the daily occupancy of ICU and OH in hospitals for each region, using a Susceptible-Infected-Recovered-Death (SIRD) epidemic model to further extend occupancy prediction in each regional district. Computed data validated the proposed models in Italy after almost two years of pandemic, obtaining agreements with the Italian Presidential Decree regardless of the different regional trends of epidemic waves. Therefore, the decision-making algorithm and prediction model resulted valuable tools, retrospectively, to be tested prospectively in sustainable strategies to curb the impact of COVID-19, or of any other pandemic threats with any aggregate of data, on local healthcare systems.

Flipping food during grilling tasks, a dataset of utensils kinematics and dynamics, food pose and subject gaze.

This paper presents a multivariate dataset of 2866 food flipping movements, performed by 4 chefs and 5 home cooks, with different grilled food and two utensils (spatula and tweezers). The 3D trajectories of strategic points in the utensils were tracked using optoelectronic motion capture. The pinching force of the tweezers, the bending force and torsion torque of the spatula were also recorded, as well as videos and the subject gaze. These data were collected using a custom experimental setup that allowed the execution of flipping movements with freshly cooked food, without having the sensors near the dangerous cooking area. Complementary, the 2D position of food was computed from the videos. The action of flipping food is, indeed, gaining the attention of both researchers and manufacturers of foodservice technology. The reported dataset contains valuable measurements (1) to characterize and model flipping movements as performed by humans, (2) to develop bio-inspired methods to control a cooking robot, or (3) to study new algorithms for human actions recognition.

From P4 medicine to P5 medicine: transitional times for a more human-centric approach to AI-based tools for hospitals of tomorrow.

Within the debate on shaping future clinical services, where different robotics and artificial intelligence (AI) based technologies are integrated to perform tasks, the authors take the chance to provide an interdisciplinary analysis required to validate a tool aiming at supporting the melanoma cancer diagnosis. In particular, they focus on the ethical-legal and technical requirements needed to address the Assessment List on Trustworthy AI (ALTAI), highlighting some pros and cons of the adopted self-assessment checklist. The dialogue stimulates additionally remarks on the EU regulatory initiatives on AI in the healthcare systems.

Small bowel to closest human body surface distance calculation through a custom-made software using CT-based datasets.

Screening of the gastrointestinal tract is imperative for the detection and treatment of physiological and pathological disorders in humans. Ingestible devices (e.g., magnetic capsule endoscopes) represent an alternative to conventional flexible endoscopy for reducing the invasiveness of the procedure and the related patient's discomforts. However, to properly design localization and navigation strategies for capsule endoscopes, the knowledge of anatomical features is paramount. Therefore, authors developed a semi-automatic software for measuring the distance between the small bowel and the closest human external body surface, using CT colonography images. In this study, volumetric datasets of 30 patients were processed by gastrointestinal endoscopists with the dedicated custom-made software and results showed an average distance of 79.29 ± 23.85 mm.

Automatic Cyclic Alternating Pattern (CAP) analysis: Local and multi-trace approaches.

The Cyclic Alternating Pattern (CAP) is composed of cycles of two different electroencephalographic features: an activation A-phase followed by a B-phase representing the background activity. CAP is considered a physiological marker of sleep instability. Despite its informative nature, the clinical applications remain limited as CAP analysis is a time-consuming activity. In order to overcome this limit, several automatic detection methods were recently developed. In this paper, two new dimensions were investigated in the attempt to optimize novel, efficient and automatic detection algorithms: 1) many electroencephalographic leads were compared to identify the best local performance, and 2) the global contribution of the concurrent detection across several derivations to CAP identification. The developed algorithms were tested on 41 polysomnographic recordings from normal (n = 8) and pathological (n = 33) subjects. In comparison with the visual CAP analysis as the gold standard, the performance of each algorithm was evaluated. Locally, the detection on the F4-C4 derivation showed the best performance in comparison with all other leads, providing practical suggestions of electrode montage when a lean and minimally invasive approach is preferable. A further improvement in the detection was achieved by a multi-trace method, the Global Analysis-Common Events, to be applied when several recording derivations are available. Moreover, CAP time and CAP rate obtained with these algorithms positively correlated with the ones identified by the scorer. These preliminary findings support efficient automated ways for the evaluation of the sleep instability, generalizable to both normal and pathological subjects affected by different sleep disorders.

A Novel Capacitive Measurement Device for Longitudinal Monitoring of Bone Fracture Healing.

The healing process of surgically-stabilised long bone fractures depends on two main factors: (a) the assessment of implant stability, and (b) the knowledge of bone callus stiffness. Currently, X-rays are the main diagnostic tool used for the assessment of bone fractures. However, they are considered unsafe, and the interpretation of the clinical results is highly subjective, depending on the clinician's experience. Hence, there is the need for objective, non-invasive and repeatable methods to allow a longitudinal assessment of implant stability and bone callus stiffness. In this work, we propose a compact and scalable system, based on capacitive sensor technology, able to measure, quantitatively, the relative pins displacements in bone fractures treated with external fixators. The measurement device proved to be easily integrable with the external fixator pins. Smart arrangements of the sensor units were exploited to discriminate relative movements of the external pins in the 3D space with a resolution of 0.5 mm and 0.5°. The proposed capacitive technology was able to detect all of the expected movements of the external pins in the 3D space, providing information on implant stability and bone callus stiffness.

Comparative Analysis of Diagnostic Techniques for Melanoma Detection: A Systematic Review of Diagnostic Test Accuracy Studies and Meta-Analysis.

Melanoma has the highest mortality rate among skin cancers, and early-diagnosis is essential to maximize survival rate. The current procedure for melanoma diagnosis is based on dermoscopy, i.e., a qualitative visual inspection of lesions with intrinsic limited diagnostic reliability and reproducibility. Other non-invasive diagnostic techniques may represent valuable solutions to retrieve additional objective information of a lesion. This review aims to compare the diagnostic performance of non-invasive techniques, alternative to dermoscopy, for melanoma detection in clinical settings. A systematic review of the available literature was performed using PubMed, Scopus and Google scholar databases (2010-September 2020). All human, in-vivo, non-invasive studies using techniques, alternative to dermoscopy, for melanoma diagnosis were included with no restriction on the recruited population. The reference standard was histology but dermoscopy was accepted only in case of benign lesions. Attributes of the analyzed studies were compared, and the quality was evaluated using CASP Checklist. For studies in which the investigated technique was implemented as a diagnostic tool (DTA studies), the QUADAS-2 tool was applied. For DTA studies that implemented a melanoma vs. other skin lesions classification task, a meta-analysis was performed reporting the SROC curves. Sixty-two references were included in the review, of which thirty-eight were analyzed using QUADAS-2. Study designs were: clinical trials (13), retrospective studies (10), prospective studies (8), pilot studies (10), multitiered study (1); the remain studies were proof of concept or had undefined study type. Studies were divided in categories based on the physical principle employed by each diagnostic technique. Twenty-nine out of thirty-eight DTA studies were included in the meta-analysis. Heterogeneity of studies' types, testing strategy, and diagnostic task limited the systematic comparison of the techniques. Based on the SROC curves, spectroscopy achieved the best performance in terms of sensitivity (93%, 95% CI 92.8-93.2%) and specificity (85.2%, 95%CI 84.9-85.5%), even though there was high concern regarding robustness of metrics. Reflectance-confocal-microscopy, instead, demonstrated higher robustness and a good diagnostic performance (sensitivity 88.2%, 80.3-93.1%; specificity 65.2%, 55-74.2%). Best practice recommendations were proposed to reduce bias in future DTA studies. Particular attention should be dedicated to widen the use of alternative techniques to conventional dermoscopy.

Endoluminal Motion Recognition of a Magnetically-Guided Capsule Endoscope Based on Capsule-Tissue Interaction Force.

A magnetically-guided capsule endoscope, embedding flexible force sensors, is designed to measure the capsule-tissue interaction force. The flexible force sensor is composed of eight force-sensitive elements surrounding the internal permanent magnet (IPM). The control of interaction force acting on the intestinal wall can reduce patient's discomfort and maintain the magnetic coupling between the external permanent magnet (EPM) and the IPM during capsule navigation. A flexible force sensor can achieve this control. In particular, by analyzing the signals of the force sensitive elements, we propose a method to recognize the status of the motion of the magnetic capsule, and provide corresponding formulas to evaluate whether the magnetic capsule follows the motion of the external driving magnet. Accuracy of the motion recognition in Ex Vivo tests reached 94% when the EPM was translated along the longitudinal axis. In addition, a method is proposed to realign the EPM and the IPM before the loss of their magnetic coupling. Its translational error, rotational error, and runtime are 7.04 ± 0.71 mm, 3.13 ± 0.47∘, and 11.4 ± 0.39 s, respectively. Finally, a control strategy is proposed to prevent the magnetic capsule endoscope from losing control during the magnetically-guided capsule colonoscopy.

Training Simulators for Gastrointestinal Endoscopy: Current and Future Perspectives.

Gastrointestinal (GI) endoscopy is the gold standard in the detection and treatment of early and advanced GI cancers. However, conventional endoscopic techniques are technically demanding and require visual-spatial skills and significant hands-on experience. GI endoscopy simulators represent a valid solution to allow doctors to practice in a pre-clinical scenario. From the first endoscopy mannequin, developed in 1969, several simulation platforms have been developed, ranging from purely mechanical systems to more complex mechatronic devices and animal-based models. Considering the recent advancement of technologies (e.g., artificial intelligence, augmented reality, robotics), simulation platforms can now reach high levels of realism, representing a valid and smart alternative to standard trainee/mentor learning programs. This is particularly true nowadays, when the current demographic trend and the most recent pandemic demand, more than ever, the ability to cope with many patients. This review offers a broad view of the technology available for GI endoscopy training, including platforms currently in the market and the relevant advancements in this research and application field. Additionally, new training needs and new emerging technologies are discussed to understand where medical education is heading.

Efficient embedded sleep wake classification for open-source actigraphy.

This study presents a thorough analysis of sleep/wake detection algorithms for efficient on-device sleep tracking using wearable accelerometric devices. It develops a novel end-to-end algorithm using convolutional neural network applied to raw accelerometric signals recorded by an open-source wrist-worn actigraph. The aim of the study is to develop an automatic classifier that: (1) is highly generalizable to heterogenous subjects, (2) would not require manual features' extraction, (3) is computationally lightweight, embeddable on a sleep tracking device, and (4) is suitable for a wide assortment of actigraphs. Hereby, authors analyze sleep parameters, such as total sleep time, waking after sleep onset and sleep efficiency, by comparing the outcomes of the proposed algorithm to the gold standard polysomnographic concurrent recordings. The relatively substantial agreement (Cohen's kappa coefficient, median, equal to 0.78 ± 0.07) and the low-computational cost (2727 floating-point operations) make this solution suitable for an on-board sleep-detection approach.

Frontiers of Robotic Gastroscopy: A Comprehensive Review of Robotic Gastroscopes and Technologies.

Upper gastrointestinal (UGI) tract pathology is common worldwide. With recent advancements in robotics, innovative diagnostic and treatment devices have been developed and several translational attempts made. This review paper aims to provide a highly pictorial critical review of robotic gastroscopes, so that clinicians and researchers can obtain a swift and comprehensive overview of key technologies and challenges. Therefore, the paper presents robotic gastroscopes, either commercial or at a progressed technology readiness level. Among them, we show tethered and wireless gastroscopes, as well as devices aimed for UGI surgery. The technological features of these instruments, as well as their clinical adoption and performance, are described and compared. Although the existing endoscopic devices have thus far provided substantial improvements in the effectiveness of diagnosis and treatment, there are certain aspects that represent unwavering predicaments of the current gastroenterology practice. A detailed list includes difficulties and risks, such as transmission of communicable diseases (e.g., COVID-19) due to the doctor-patient proximity, unchanged learning curves, variable detection rates, procedure-related adverse events, endoscopists' and nurses' burnouts, limited human and/or material resources, and patients' preferences to choose non-invasive options that further interfere with the successful implementation and adoption of routine screening. The combination of robotics and artificial intelligence, as well as remote telehealth endoscopy services, are also discussed, as viable solutions to improve existing platforms for diagnosis and treatment are emerging.

Robotic-Assisted Colonoscopy Platform with a Magnetically-Actuated Soft-Tethered Capsule.

Background and Aims: Colorectal cancer (CRC) is a major cause of morbidity and mortality worldwide. Despite offering a prime paradigm for screening, CRC screening is often hampered by invasiveness. Endoo is a potentially painless colonoscopy method with an active locomotion tethered capsule offering diagnostic and therapeutic capabilities. Materials and Methods: The Endoo system comprises a soft-tethered capsule, which embeds a permanent magnet controlled by an external robot equipped with a second permanent magnet. Capsule navigation is achieved via closed-loop interaction between the two magnets. Ex-vivo tests were conducted by endoscopy experts and trainees to evaluate the basic key features, usability, and compliance in comparison with conventional colonoscopy (CC) in feasibility and pilot studies. Results: Endoo showed a 100% success rate in operating channel and target approach tests. Progression of the capsule was feasible and repeatable. The magnetic link was lost an average of 1.28 times per complete procedure but was restored in 100% of cases. The peak value of interaction forces was higher in the CC group than the Endoo group (4.12N vs. 1.17N). The cumulative interaction forces over time were higher in the CC group than the Endoo group between the splenic flexure and mid-transverse colon (16.53Ns vs. 1.67Ns, p < 0.001), as well as between the hepatic flexure and cecum (28.77Ns vs. 2.47Ns, p = 0.005). The polyp detection rates were comparable between groups (9.1 ± 0.9% vs. 8.7 ± 0.9%, CC and Endoo respectively, per procedure). Robotic colonoscopies were completed in 67% of the procedures performed with Endoo (53% experts and 100% trainees). Conclusions: Endoo allows smoother navigation than CC and possesses comparable features. Although further research is needed, magnetic capsule colonoscopy demonstrated promising results compared to CC.

Photometric Stereo-Based Depth Map Reconstruction for Monocular Capsule Endoscopy.

The capsule endoscopy robot can only use monocular vision due to the dimensional limit. To improve the depth perception of the monocular capsule endoscopy robot, this paper proposes a photometric stereo-based depth map reconstruction method. First, based on the characteristics of the capsule endoscopy robot system, a photometric stereo framework is established. Then, by combining the specular property and Lambertian property of the object surface, the depth of the specular highlight point is estimated, and the depth map of the whole object surface is reconstructed by a forward upwind scheme. To evaluate the precision of the depth estimation of the specular highlight region and the depth map reconstruction of the object surface, simulations and experiments are implemented with synthetic images and pig colon tissue, respectively. The results of the simulations and experiments show that the proposed method provides good precision for depth map reconstruction in monocular capsule endoscopy.

Frontiers of Robotic Colonoscopy: A Comprehensive Review of Robotic Colonoscopes and Technologies.

Flexible colonoscopy remains the prime mean of screening for colorectal cancer (CRC) and the gold standard of all population-based screening pathways around the world. Almost 60% of CRC deaths could be prevented with screening. However, colonoscopy attendance rates are affected by discomfort, fear of pain and embarrassment or loss of control during the procedure. Moreover, the emergence and global thread of new communicable diseases might seriously affect the functioning of contemporary centres performing gastrointestinal endoscopy. Innovative solutions are needed: artificial intelligence (AI) and physical robotics will drastically contribute for the future of the healthcare services. The translation of robotic technologies from traditional surgery to minimally invasive endoscopic interventions is an emerging field, mainly challenged by the tough requirements for miniaturization. Pioneering approaches for robotic colonoscopy have been reported in the nineties, with the appearance of inchworm-like devices. Since then, robotic colonoscopes with assistive functionalities have become commercially available. Research prototypes promise enhanced accessibility and flexibility for future therapeutic interventions, even via autonomous or robotic-assisted agents, such as robotic capsules. Furthermore, the pairing of such endoscopic systems with AI-enabled image analysis and recognition methods promises enhanced diagnostic yield. By assembling a multidisciplinary team of engineers and endoscopists, the paper aims to provide a contemporary and highly-pictorial critical review for robotic colonoscopes, hence providing clinicians and researchers with a glimpse of the major changes and challenges that lie ahead.

Visual-Based Defect Detection and Classification Approaches for Industrial Applications-A SURVEY.

This paper reviews automated visual-based defect detection approaches applicable to various materials, such as metals, ceramics and textiles. In the first part of the paper, we present a general taxonomy of the different defects that fall in two classes: visible (e.g., scratches, shape error, etc.) and palpable (e.g., crack, bump, etc.) defects. Then, we describe artificial visual processing techniques that are aimed at understanding of the captured scenery in a mathematical/logical way. We continue with a survey of textural defect detection based on statistical, structural and other approaches. Finally, we report the state of the art for approaching the detection and classification of defects through supervised and non-supervised classifiers and deep learning.

Optical and Electromagnetic Tracking Systems for Biomedical Applications: A Critical Review on Potentialities and Limitations.

Optical and electromagnetic tracking systems represent the two main technologies integrated into commercially-available surgical navigators for computer-assisted image-guided surgery so far. Optical Tracking Systems (OTSs) work within the optical spectrum to track the position and orientation, i.e., pose of target surgical instruments. OTSs are characterized by high accuracy and robustness to environmental conditions. The main limitation of OTSs is the need of a direct line-of-sight between the optical markers and the camera sensor, rigidly fixed into the operating theatre. Electromagnetic Tracking Systems (EMTSs) use electromagnetic field generator to detect the pose of electromagnetic sensors. EMTSs do not require such a direct line-of-sight, however the presence of metal or ferromagnetic sources in the operating workspace can significantly affect the measurement accuracy. The aim of the proposed review is to provide a complete and detailed overview of optical and electromagnetic tracking systems, including working principles, source of error and validation protocols. Moreover, commercial and research-oriented solutions, as well as clinical applications, are described for both technologies. Finally, a critical comparative analysis of the state of the art which highlights the potentialities and the limitations of each tracking system for a medical use is provided.