Closed-loop network of skin-interfaced wireless devices for quantifying vocal fatigue and providing user feedback.

Vocal fatigue is a measurable form of performance fatigue resulting from overuse of the voice and is characterized by negative vocal adaptation. Vocal dose refers to cumulative exposure of the vocal fold tissue to vibration. Professionals with high vocal demands, such as singers and teachers, are especially prone to vocal fatigue. Failure to adjust habits can lead to compensatory lapses in vocal technique and an increased risk of vocal fold injury. Quantifying and recording vocal dose to inform individuals about potential overuse is an important step toward mitigating vocal fatigue. Previous work establishes vocal dosimetry methods, that is, processes to quantify vocal fold vibration dose but with bulky, wired devices that are not amenable to continuous use during natural daily activities; these previously reported systems also provide limited mechanisms for real-time user feedback. This study introduces a soft, wireless, skin-conformal technology that gently mounts on the upper chest to capture vibratory responses associated with vocalization in a manner that is immune to ambient noises. Pairing with a separate, wirelessly linked device supports haptic feedback to the user based on quantitative thresholds in vocal usage. A machine learning-based approach enables precise vocal dosimetry from the recorded data, to support personalized, real-time quantitation and feedback. These systems have strong potential to guide healthy behaviors in vocal use.

Feeling of pulsations in artificial arteries with a real time haptic feedback laparoscopic grasper: a validation study.

INTRODUCTION: Despite the advancements in technology and organized training for surgeons in laparoscopic surgery, the persistent challenge of not being able to feel the resistance and characteristics of the tissue, including pulsations, remains unmet. A recently developed grasper (Optigrip®) with real time haptic feedback, based on photonic technology, aims to address this issue by restoring the tactile sensation for surgeons. The key question is whether pulsations can be detected and at what minimal size level they become clinical significant. METHODS: To simulate arterial conditions during laparoscopic procedures, four different silicone tubes were created, representing the most prevalent arteries. These tubes were connected to a validated pressure system, generating a natural pulse ranging between 80 and 120 mm Hg. One control tube without pressure was added. The surgeons had to grasp these tubes blindly with the conventional grasper or the haptic feedback grasper in a randomized order. They then indicated whether they felt the pressure or not and the percentage of correct answers was calculated. RESULTS: The haptic grasper successfully detected 96% of all pulsations, while the conventional grasper could only detect 6%. When considering the size of the arteries, the Optigrip® identified pulsations in 100% the 4 and 5 mm arteries and 92% of the smallest arteries. The conventional grasper was only able to feel the smallest arteries in 8%. These differences were highly significant (p < 0.0001). CONCLUSION: This study demonstrated that the newly developed haptic feedback grasper enables detection of arterial pulsations during laparoscopy, filling an important absence in tactile perception within laparoscopic surgery.

Field experiment of a telesurgery system using a surgical robot with haptic feedback.

PURPOSE: To verify the usefulness of haptic feedback in telesurgery and improve the safety of telerobotic surgery. METHODS: The surgeon's console was installed at two sites (Fukuoka and Beppu; 140 km apart), and the patient cart was installed in Fukuoka. During the experiment, the surgeon was blinded to the haptic feedback levels and asked to grasp the intestinal tract in an animal model. The surgeon then performed the tasks at each location. RESULTS: No marked differences in task accuracy or average grasping force were observed between the surgeon locations. However, the average task completion time was significantly longer, and the system usability scale (SUS) was significantly lower rating for remote operations than for local ones. No marked differences in task accuracy or task completion time were observed between the haptic feedback levels. However, with haptic feedback, the organ was grasped with a significantly weaker force than that without it. Furthermore, with haptic feedback, experienced surgeons in robotic surgery tended to perform an equivalent task with weaker grasping forces than inexperienced surgeons. CONCLUSION: The haptic feedback function is a tool that allows the surgeon to perform surgery with an appropriate grasping force, both on site and remotely. Improved safety is necessary in telesurgery; haptic feedback will thus be an essential technology in robotic telesurgery going forward.

A Lightweight and Affordable Wearable Haptic Controller for Robot-Assisted Microsurgery.

In robot-assisted microsurgery (RAMS), surgeons often face the challenge of operating with minimal feedback, particularly lacking in haptic feedback. However, most traditional desktop haptic devices have restricted operational areas and limited dexterity. This report describes a novel, lightweight, and low-budget wearable haptic controller for teleoperated microsurgical robotic systems. We designed a wearable haptic interface entirely made using off-the-shelf material-PolyJet Photopolymer, fabricated using liquid and solid hybrid 3D co-printing technology. This interface was designed to resemble human soft tissues and can be wrapped around the fingertips, offering direct contact feedback to the operator. We also demonstrated that the device can be easily integrated with our motion tracking system for remote microsurgery. Two motion tracking methods, marker-based and marker-less, were compared in trajectory-tracking experiments at different depths to find the most effective motion tracking method for our RAMS system. The results indicate that within the 4 to 8 cm tracking range, the marker-based method achieved exceptional detection rates. Furthermore, the performance of three fusion algorithms was compared to establish the unscented Kalman filter as the most accurate and reliable. The effectiveness of the wearable haptic controller was evaluated through user studies focusing on the usefulness of haptic feedback. The results revealed that haptic feedback significantly enhances depth perception for operators during teleoperated RAMS.

Enhancing Tennis Practice: Sensor Fusion and Pose Estimation with a Smart Tennis Ball.

This article demonstrates the integration of sensor fusion for pose estimation and data collection in tennis balls, aiming to create a smaller, less intrusive form factor for use in progressive learning during tennis practice. The study outlines the design and implementation of the Bosch BNO055 smart sensor, which features built-in managed sensor fusion capabilities. The article also discusses deriving additional data using various mathematical and simulation methods to present relevant orientation information from the sensor in Unity. Embedded within a Vermont practice foam tennis ball, the final prototype product communicates with Unity on a laptop via Bluetooth. The Unity interface effectively visualizes the ball's rotation, the resultant acceleration direction, rotations per minute (RPM), and the orientation relative to gravity. The system successfully demonstrates accurate RPM measurement, provides real-time visualization of ball spin and offers a pathway for innovative applications in tennis training technology.

[Virtual reality-brain computer interface hand function enhancement rehabilitation system incorporating multi-sensory stimulation].

Stroke is an acute cerebrovascular disease in which sudden interruption of blood supply to the brain or rupture of cerebral blood vessels cause damage to brain cells and consequently impair the patient's motor and cognitive abilities. A novel rehabilitation training model integrating brain-computer interface (BCI) and virtual reality (VR) not only promotes the functional activation of brain networks, but also provides immersive and interesting contextual feedback for patients. In this paper, we designed a hand rehabilitation training system integrating multi-sensory stimulation feedback, BCI and VR, which guides patients' motor imaginations through the tasks of the virtual scene, acquires patients' motor intentions, and then carries out human-computer interactions under the virtual scene. At the same time, haptic feedback is incorporated to further increase the patients' proprioceptive sensations, so as to realize the hand function rehabilitation training based on the multi-sensory stimulation feedback of vision, hearing, and haptic senses. In this study, we compared and analyzed the differences in power spectral density of different frequency bands within the EEG signal data before and after the incorporation of haptic feedback, and found that the motor brain area was significantly activated after the incorporation of haptic feedback, and the power spectral density of the motor brain area was significantly increased in the high gamma frequency band. The results of this study indicate that the rehabilitation training of patients with the VR-BCI hand function enhancement rehabilitation system incorporating multi-sensory stimulation can accelerate the two-way facilitation of sensory and motor conduction pathways, thus accelerating the rehabilitation process.

Expert laparoscopist performance on virtual reality simulation tasks with and without haptic features.

BACKGROUND: Virtual reality (VR) simulation for laparoscopic training is available with and without haptic feedback features. Currently, there is limited data on haptic feedback's effect on skill development. Our objective is to compare expert laparoscopists' skills characteristics using VR delivered laparoscopic tasks via haptic and nonhaptic laparoscopic surgical interfaces. METHODS: Five expert laparoscopists performed seven skills tasks on two laparoscopic simulators, one with and one without haptic features. Tasks consisted of 2-handed instrument navigation, retraction and exposure, cutting, electrosurgery, and complicated object positioning. Laparoscopists alternated platforms at default difficulty settings. Metrics included time, economy of movement, completed task elements, and errors. Progressive change in performance for the final three iterations were determined by repeated measures ANOVA. Iteration quartile means were determined and compared using paired t-tests. RESULTS: No change in performance was noted in the last three iterations for any metric. There were no significant differences between platforms on the final two quartiles for most metrics except avoidance of over-stretch error for retraction; and cutting task was significantly better with haptics on all iteration quartiles (p < 0.03). Economy of movement was significantly better with haptics for both hands for clip application (p < 0.01) and better for right hand on complex object positioning (p < 0.05). Accuracy was better with haptics for retraction and cutting (p < 0.05) and clip application (p < 0.05). CONCLUSION: Results showed higher performance in accuracy, efficient instrument motion, and avoidance of excessive traction force on selected tasks performed on VR simulator with haptic feedback compared to those performed without haptics feedback. Laparoscopic surgeons interpreted machine-generated haptic cues appropriately and resulted in better performance with VR task requirements. However, our results do not demonstrate an advantage in skills acquisition, which requires additional study.

Research and application of a teaching platform for combined spinal-epidural anesthesia based on virtual reality and haptic feedback technology.

BACKGROUND: Intraspinal anesthesia poses significant teaching challenges and inadequate teaching resources, which ultimately limit students' opportunities for practice. To address this issue, we aimed to develop a virtual platform for combined spinal-epidural anesthesia that merges virtual reality technology with haptic feedback technology, while assessing its educational impact and learning outcomes. METHODS: We utilized MIMICS, 3Ds MAX, and UNITY 3D software to perform 3D reconstruction based on lumbar CT/MRI data from a standard male volunteer. The haptic coefficients were configured on each layer by 20 experienced anesthesiologists in accordance with the Geomagic Touch X force feedback device. A total of 20 anesthesiology interns completed 30 virtual puncture training sessions. Two experienced anesthetists evaluated the efficacy of the platform and the level of mastery achieved using the Global Rating Scale (GRS) and a Checklist score, respectively. Finally, a questionnaire survey was conducted to gather feedback on the virtual platform. RESULTS: After the 10th session, the puncture time stabilized at 2.4 min. As the number of sessions increased, the Global Rating Scale (GRS) score stabilized by the 8th session, and the Checklist scores tended to stabilize by the 10th session. Results from questionnaires indicated that over half of the anesthesiology interns (70%) believed that the platform, which exhibited strong repeatability, improved their anatomical recognition and provided a strong sense of breakthrough in identifying the ligamentum flavum. The majority of them (80%) expressed satisfaction with the virtual platform. CONCLUSIONS: The platform effectively facilitated the acquisition of basic and accurate puncture skills on a virtual patient.

BattleSound: A Game Sound Benchmark for the Sound-Specific Feedback Generation in a Battle Game.

A haptic sensor coupled to a gamepad or headset is frequently used to enhance the sense of immersion for game players. However, providing haptic feedback for appropriate sound effects involves specialized audio engineering techniques to identify target sounds that vary according to the game. We propose a deep learning-based method for sound event detection (SED) to determine the optimal timing of haptic feedback in extremely noisy environments. To accomplish this, we introduce the BattleSound dataset, which contains a large volume of game sound recordings of game effects and other distracting sounds, including voice chats from a PlayerUnknown's Battlegrounds (PUBG) game. Given the highly noisy and distracting nature of war-game environments, we set the annotation interval to 0.5 s, which is significantly shorter than the existing benchmarks for SED, to increase the likelihood that the annotated label contains sound from a single source. As a baseline, we adopt mobile-sized deep learning models to perform two tasks: weapon sound event detection (WSED) and voice chat activity detection (VCAD). The accuracy of the models trained on BattleSound was greater than 90% for both tasks; thus, BattleSound enables real-time game sound recognition in noisy environments via deep learning. In addition, we demonstrated that performance degraded significantly when the annotation interval was greater than 0.5 s, indicating that the BattleSound with short annotation intervals is advantageous for SED applications that demand real-time inferences.

Uneven Terrain Recognition Using Neuromorphic Haptic Feedback.

Recent years have witnessed relevant advancements in the quality of life of persons with lower limb amputations thanks to the technological developments in prosthetics. However, prostheses that provide information about the foot-ground interaction, and in particular about terrain irregularities, are still missing on the market. The lack of tactile feedback from the foot sole might lead subjects to step on uneven terrains, causing an increase in the risk of falling. To address this issue, a biomimetic vibrotactile feedback system that conveys information about gait and terrain features sensed by a dedicated insole has been assessed with intact subjects. After having shortly experienced both even and uneven terrains, the recruited subjects discriminated them with an accuracy of 87.5%, solely relying on the replay of the vibrotactile feedback. With the objective of exploring the human decoding mechanism of the feedback startegy, a KNN classifier was trained to recognize the uneven terrains. The outcome suggested that the subjects achieved such performance with a temporal dynamics of 45 ms. This work is a leap forward to assist lower-limb amputees to appreciate the floor conditions while walking, adapt their gait and promote a more confident use of their artificial limb.

Learning Compliant Box-in-Box Insertion through Haptic-Based Robotic Teleoperation.

In modern logistics, the box-in-box insertion task is representative of a wide range of packaging applications, and automating compliant object insertion is difficult due to challenges in modelling the object deformation during insertion. Using Learning from Demonstration (LfD) paradigms, which are frequently used in robotics to facilitate skill transfer from humans to robots, can be one solution for complex tasks that are difficult to mathematically model. In order to automate the box-in-box insertion task for packaging applications, this study makes use of LfD techniques. The proposed framework has three phases. Firstly, a master-slave teleoperated robot system is used in the initial phase to haptically demonstrate the insertion task. Then, the learning phase involves identifying trends in the demonstrated trajectories using probabilistic methods, in this case, Gaussian Mixture Regression. In the third phase, the insertion task is generalised, and the robot adjusts to any object position using barycentric interpolation. This method is novel because it tackles tight insertion by taking advantage of the boxes' natural compliance, making it possible to complete the task even with a position-controlled robot. To determine whether the strategy is generalisable and repeatable, experimental validation was carried out.

Exploiting the haptic and audio channels to improve orientation and mobility apps for the visually impaired.

Orientation and mobility apps for visually impaired people are well known to be effective in improving the quality of life for this target group. A mobile application that guides a visually impaired person step-by-step through a physical space is a valuable aid, but it does not provide an overview of a complex environment "at a glance," as a traditional hard-copy tactile map does. The aim of this study is to investigate whether a smartphone GPS map, enriched with haptic and audio hints, can facilitate cognitive mapping for visually impaired users. Encouraged by a preliminary study conducted in co-operation with two visually impaired volunteers, we designed and developed an Android prototype for exploration of an urban area. Our goal was to provide an affordable, portable and versatile solution to help users increase awareness of an environment through the positions of its landmarks and points of interest. Vibro-tactile and audio hints were linked to the coordinates on the map via the GeoJSON data format and were issued exploiting the text-to-speech and vibration features of the mobile device, as they were displayed through the operating system's APIs. Test sessions and interviews with visually impaired users produced encouraging results. Results, to be verified by more extensive testing, overall confirm the validity of our approach and are in line with results found in the literature.

The role of feedback in the production of skilled finger sequences.

Actions involving fine control of the hand, for example, grasping an object, rely heavily on sensory information from the fingertips. Although the integration of feedback during the execution of individual movements is well understood, less is known about the use of sensory feedback in the control of skilled movement sequences. To address this gap, we trained participants to produce sequences of finger movements on a keyboard-like device over a 4-day training period. Participants received haptic, visual, and auditory feedback indicating the occurrence of each finger press. We then either transiently delayed or advanced the feedback for a single press by a small amount of time (30 or 60 ms). We observed that participants rapidly adjusted their ongoing finger press by either accelerating or prolonging the ongoing press, in accordance with the direction of the perturbation. Furthermore, we could show that this rapid behavioral modulation was driven by haptic feedback. Although these feedback-driven adjustments reduced in size with practice, they were still clearly present at the end of training. In contrast to the directionally specific effect we observed on the perturbed press, a feedback perturbation resulted in a delayed onset of the subsequent presses irrespective of perturbation direction or feedback modality. This observation is consistent with a hierarchical organization of even very skilled and fast movement sequences, with different levels reacting distinctly to sensory perturbations.NEW & NOTEWORTHY Sensory feedback is important during the execution of a movement. However, little is known about how sensory feedback is used during the production of movement sequences. Here, we show two distinct feedback processes in the execution of fast finger movement sequences. By transiently delaying or advancing the feedback of a single press within a sequence, we observed a directionally specific effect on the perturbed press and a directionally non-specific effect on the subsequent presses.

Robotic-assisted TKA leads to a better prosthesis alignment and a better joint line restoration as compared to conventional TKA: a prospective randomized controlled trial.

PURPOSE: Correct positioning and alignment of the prosthesis is a very important factor for durability of prosthesis and implant survival which is improved with the use of technology in total knee arthroplasty. However, the long-term functional outcomes and survivorship are unclear. For this study, it was hypothesized that mechanical axis alignment of lower limb, post-operative joint line restoration, femoral and tibial component alignment is more accurate with the new handheld semi-active robotic-assisted TKA. METHOD: From April-2019 to March-2020, 60 patients with unilateral knee osteoarthritis who underwent total knee arthroplasties were included in this prospective randomised controlled study. Computer generated randomization was used. Study included 48 female patients and 12 male patients. Pre-operative and post-operative radiographic measurements were done and compared between the two groups. RESULTS: There was a significant difference between two groups with respect to mechanical axis deviation, joint line deviation and coronal alignment of femoral and tibial prosthesis. Mechanical axis deviation > 3° was seen in eight cases (28.5%) in C-TKA group compared to one case (3.1%) in RA-TKA (p 0.019). Joint line deviation of 3.5 mm was noted in C-TKA group as compared to 0.9 mm in RA-TKA group (p < 0.001) which was statistically significant. However, whether this difference of 2.6 mm of joint line elevation between C-TKA and RA-TKA leads to any difference in clinical outcome in terms of knee kinematics and knee flexion needs to be investigated with further studies. Clinically restoring normal joint line is important for improved knee function after primary TKA. No significant difference was noted in femoral component rotation on post-operative computed tomography (CT) scan. CONCLUSION: The novel imageless, handheld semi-autonomous robotic system for TKA is highly accurate with respect to component positioning in coronal plane and mechanical alignment as compared to conventional TKA. Joint line is elevated in conventional TKA but is accurately restored using the robotic-assisted TKA which may lead to better patellofemoral kinematics. LEVEL OF EVIDENCE: I.

Validation of a novel virtual reality simulation system with the focus on training for surgical dissection during laparoscopic sigmoid colectomy.

BACKGROUND: Mastery of technical skills is one of the fundamental goals of surgical training for novices. Meanwhile, performing laparoscopic procedures requires exceptional surgical skills compared to open surgery. However, it is often difficult for trainees to learn through observation and practice only. Virtual reality (VR)-based surgical simulation is expanding and rapidly advancing. A major obstacle for laparoscopic trainees is the difficulty of well-performed dissection. Therefore, we developed a new VR simulation system, Lap-PASS LP-100, which focuses on training to create proper tension on the tissue in laparoscopic sigmoid colectomy dissection. This study aimed to validate this new VR simulation system. METHODS: A total of 50 participants were asked to perform medial dissection of the meso-sigmoid colon on the VR simulator. Forty-four surgeons and six non-medical professionals working in the National Cancer Center Hospital East, Japan, were enrolled in this study. The surgeons were: laparoscopic surgery experts with > 100 laparoscopic surgeries (LS), 21 were novices with experience < 100 LS, and five without previous experience in LS. The participants' surgical performance was evaluated by three blinded raters using Global Operative Assessment of Laparoscopic Skills (GOALS). RESULTS: There were significant differences (P-values < 0.044) in all GOALS items between the non-medical professionals and surgeons. The experts were significantly superior to the novices in one item of GOALS: efficiency ([4(4-5) vs. 4(3-4)], with a 95% confidence interval, p = 0.042). However, both bimanual dexterity and total score in the experts were not statistically different but tended to be higher than in the novices. CONCLUSIONS: Our study demonstrated a full validation of our new system. This could detect the surgeons' ability to perform surgical dissection and suggest that this VR simulator could be an effective training tool. This surgical VR simulator might have tremendous potential to enhance training for surgeons.

A Transparent Teleoperated Robotic Surgical System with Predictive Haptic Feedback and Force Modelling.

In recent years, robotic minimally invasive surgery has transformed many types of surgical procedures and improved their outcomes. Implementing effective haptic feedback into a teleoperated robotic surgical system presents a significant challenge due to the trade-off between transparency and stability caused by system communication time delays. In this paper, these time delays are mitigated by implementing an environment estimation and force prediction methodology into an experimental robotic minimally invasive surgical system. At the slave, an exponentially weighted recursive least squares (EWRLS) algorithm estimates the respective parameters of the Kelvin-Voigt (KV) and Hunt-Crossley (HC) force models. The master then provides force feedback by interacting with a virtual environment via the estimated parameters. Palpation experiments were conducted with the slave in contact with polyurethane foam during human-in-the-loop teleoperation. The experimental results indicated that the prediction RMSE of error between predicted master force feedback and measured slave force was reduced to 0.076 N for the Hunt-Crossley virtual environment, compared to 0.356 N for the Kelvin-Voigt virtual environment and 0.560 N for the direct force feedback methodology. The results also demonstrated that the HC force model is well suited to provide accurate haptic feedback, particularly when there is a delay between the master and slave kinematics. Furthermore, a haptic feedback approach that incorporates environment estimation and force prediction improve transparency during teleoperation. In conclusion, the proposed bilateral master-slave robotic system has the potential to provide transparent and stable haptic feedback to the surgeon in surgical robotics procedures.

Phase Optimization for Multipoint Haptic Feedback Based on Ultrasound Array.

Ultrasound-based haptic feedback is a potential technology for human-computer interaction (HCI) with the advantages of a low cost, low power consumption and a controlled force. In this paper, phase optimization for multipoint haptic feedback based on an ultrasound array was investigated, and the corresponding experimental verification is provided. A mathematical model of acoustic pressure was established for the ultrasound array, and then a phase-optimization model for an ultrasound transducer was constructed. We propose a pseudo-inverse (PINV) algorithm to accurately determine the phase contribution of each transducer in the ultrasound array. By controlling the phase difference of the ultrasound array, the multipoint focusing forces were formed, leading to various shapes such as geometries and letters, which can be visualized. Because the unconstrained PINV solution results in unequal amplitudes for each transducer, a weighted amplitude iterative optimization was deployed to further optimize the phase solution, by which the uniform amplitude distributions of each transducer were obtained. For the purpose of experimental verification, a platform of ultrasound haptic feedback consisting of a Field Programmable Gate Array (FPGA), an electrical circuit and an ultrasound transducer array was prototyped. The haptic performances of a single point, multiple points and dynamic trajectory were verified by controlling the ultrasound force exerted on the liquid surface. The experimental results demonstrate that the proposed phase-optimization model and theoretical results are effective and feasible, and the acoustic pressure distribution is consistent with the simulation results.

Real-Time Scene Monitoring for Deaf-Blind People.

It is estimated that at least 15 million people worldwide live with severe deaf-blindness, with many more experiencing varying degrees of deaf-blindness. The existing options of assistance are mostly limited to walking canes, guide dogs and human care. We propose a wearable device which harnesses a multi-antenna mmWave radar transceiver and a haptic feedback array for real time detection of a person moving within a scene. We present our findings from a series of workshops with participants classed with multi-sensory impairments (MSI), to demonstrate the relative success of this approach and its potential for integration into existing assistance for the MSI of the future.

User-Centered Design Methodologies for the Prototype Development of a Smart Harness and Related System to Provide Haptic Cues to Persons with Parkinson's Disease.

This paper describes the second part of the PASSO (Parkinson smart sensory cues for older users) project, which designs and tests an innovative haptic biofeedback system based on a wireless body sensor network using a smartphone and different smartwatches specifically designed to rehabilitate postural disturbances in persons with Parkinson's disease. According to the scientific literature on the use of smart devices to transmit sensory cues, vibrotactile feedback (particularly on the trunk) seems promising for improving people's gait and posture performance; they have been used in different environments and are well accepted by users. In the PASSO project, we designed and developed a wearable device and a related system to transmit vibrations to a person's body to improve posture and combat impairments like Pisa syndrome and camptocormia. Specifically, this paper describes the methodologies and strategies used to design, develop, and test wearable prototypes and the mHealth system. The results allowed a multidisciplinary comparison among the solutions, which led to prototypes with a high degree of usability, wearability, accessibility, and effectiveness. This mHealth system is now being used in pilot trials with subjects with Parkinson's disease to verify its feasibility among patients.

A Force-Feedback Methodology for Teleoperated Suturing Task in Robotic-Assisted Minimally Invasive Surgery.

With robotic-assisted minimally invasive surgery (RAMIS), patients and surgeons benefit from a reduced incision size and dexterous instruments. However, current robotic surgery platforms lack haptic feedback, which is an essential element of safe operation. Moreover, teleportation control challenges make complex surgical tasks like suturing more time-consuming than those that use manual tools. This paper presents a new force-sensing instrument that semi-automates the suturing task and facilitates teleoperated robotic manipulation. In order to generate the ideal needle insertion trajectory and pass the needle through its curvature, the end-effector mechanism has a rotating degree of freedom. Impedance control was used to provide sensory information about needle-tissue interaction forces to the operator using an indirect force estimation approach based on data-based models. The operator's motion commands were then regulated using a hyperplanar virtual fixture (VF) designed to maintain the desired distance between the end-effector and tissue surface while avoiding unwanted contact. To construct the geometry of the VF, an optoelectronic sensor-based approach was developed. Based on the experimental investigation of the hyperplane VF methodology, improved needle-tissue interaction force, manipulation accuracy, and task completion times were demonstrated. Finally, experimental validation of the trained force estimation models and the perceived interaction forces by the user was conducted using online data, demonstrating the potential of the developed approach in improving task performance.