Head-Mounted Projector for Manual Precision Tasks: Performance Assessment.

The growing interest in augmented reality applications has led to an in-depth look at the performance of head-mounted displays and their testing in numerous domains. Other devices for augmenting the real world with virtual information are presented less frequently and usually focus on the description of the device rather than on its performance analysis. This is the case of projected augmented reality, which, compared to head-worn AR displays, offers the advantages of being simultaneously accessible by multiple users whilst preserving user awareness of the environment and feeling of immersion. This work provides a general evaluation of a custom-made head-mounted projector for the aid of precision manual tasks through an experimental protocol designed for investigating spatial and temporal registration and their combination. The results of the tests show that the accuracy (0.6±0.1 mm of spatial registration error) and motion-to-photon latency (113±12 ms) make the proposed solution suitable for guiding precision tasks.

Semiautonomous Robotic Manipulator for Minimally Invasive Aortic Valve Replacement.

Aortic valve surgery is the preferred procedure for replacing a damaged valve with an artificial one. The ValveTech robotic platform comprises a flexible articulated manipulator and surgical interface supporting the effective delivery of an artificial valve by teleoperation and endoscopic vision. This article presents our recent work on force-perceptive, safe, semiautonomous navigation of the ValveTech platform prior to valve implantation. First, we present a force observer that transfers forces from the manipulator body and tip to a haptic interface. Second, we demonstrate how hybrid forward/inverse mechanics, together with endoscopic visual servoing, lead to autonomous valve positioning. Benchtop experiments and an artificial phantom quantify the performance of the developed robot controller and navigator. Valves can be autonomously delivered with a 2.0±0.5 mm position error and a minimal misalignment of 3.4±0.9°. The hybrid force/shape observer (FSO) algorithm was able to predict distributed external forces on the articulated manipulator body with an average error of 0.09 N. FSO can also estimate loads on the tip with an average accuracy of 3.3%. The presented system can lead to better patient care, delivery outcome, and surgeon comfort during aortic valve surgery, without requiring sensorization of the robot tip, and therefore obviating miniaturization constraints.

ValveTech: A Novel Robotic Approach for Minimally Invasive Aortic Valve Replacement.

OBJECTIVE: Aortic valve disease is the most common heart disease in the elderly calling for replacement with an artificial valve. The presented surgical robot aims to provide a highly controllable instrument for efficient delivery of an artificial valve by the help of integrated endoscopic vision. METHODS: A robot (called ValveTech), intended for minimally invasive surgery (MIS) and consisting of a flexible cable driven manipulator, a passive arm, and a control unit has been designed and prototyped. The flexible manipulator has several features (e.g., stabilizing flaps, tiny cameras, dexterous introducer and custom cartridge) to help the proper valve placement. It provides 5 degrees of freedom for reaching the operative site via mini-thoracotomy; it adjusts the valve and expands it at the optimal position. The robot was evaluated by ten cardiac surgeons following a real surgical scenario in artificial chest simulator with an aortic mockup. Moreover, after each delivery, the expanded valve was evaluated objectively in comparison with the ideal position. RESULTS: The robot performances were evaluated positively by surgeons. The trials resulted in faster delivery and an average misalignment distance of 3.8 mm along the aorta axis; 16.3 degrees rotational angle around aorta axis and 8.8 degrees misalignment of the valve commissure plane to the ideal plane were measured. CONCLUSION: The trials successfully proved the proposed system for valve delivery under endoscopic vision. SIGNIFICANCE: The ValveTech robot can be an alternative solution for minimally invasive aortic valve surgery and improve the quality of the operation both for surgeons and patients.

Monitoring Wound Healing With Contactless Measurements and Augmented Reality.

OBJECTIVE: This work presents a device for non-invasive wound parameters assessment, designed to overcome the drawbacks of traditional methods, which are mostly rough, inaccurate, and painful for the patient. The device estimates the morphological parameters of the wound and provides augmented reality (AR) visual feedback on the wound healing status by projecting the wound border acquired during the last examination, thus improving doctor-patient communication. METHODS: An accurate 3D model of the wound is created by stereophotogrammetry and refined through self-organizing maps. The 3D model is used to estimate physical parameters for wound healing assessment and integrates AR functionalities based on a miniaturized projector. The physical parameter estimation functionalities are evaluated in terms of precision, accuracy, inter-operator variability, and repeatability, whereas AR wound border projection is evaluated in terms of accuracy on the same phantom. RESULTS: The accuracy and precision of the device are respectively 2% and 1.2% for linear parameters, and 1.7% and 1.3% for area and volume. The AR projection shows an error distance <1 mm. No statistical difference was found between the measurements of different operators. CONCLUSION: The device has proven to be an objective and non-operator-dependent tool for assessing the morphological parameters of the wound. Comparison with non-contact devices shows improved accuracy, offering reliable and rigorous measurements. Clinical Impact: Chronic wounds represent a significant health problem with high recurrence rates due to the ageing of the population and diseases such as diabetes and obesity. The device presented in this work provides an easy-to-use non-invasive tool to obtain useful information for treatment.

Interactive serious game for shoulder rehabilitation based on real-time hand tracking.

BACKGROUND: Virtual reality is becoming popular in the rehabilitation field thanks to the several advantages it can offer to patients and physicians. Indeed, serious games can: motivate and engage the patient; offer different levels of challenge and difficulty based on the patient baseline, and integrate objective measures of the patient's performance during each rehabilitation session. OBJECTIVE: We designed and implemented a serious game for shoulder rehabilitation based on real-time hand tracking. The aim was to maintain the medical benefits of traditional rehabilitation, while reducing human resources and costs and facilitating active patient participation. METHODS: Our software application provides the user with a shoulder horizontal adduction exercise. This exercise takes place in a 2D interactive game environment, controlled by hand movements on a desk pad. The hardware includes a standard desktop computer and screen, and the Leap Motion Controller: a hand tracking system. Changing the desk pad material allows the physiotherapist to vary the friction between the user hand and the supporting surface. RESULTS: Fourteen healthy volunteers and six rehabilitation experts tested our serious game. The results showed that the application is attractive, ergonomic and clinically useful. CONCLUSION: Despite promising results, clinical validation is necessary to demonstrate the efficacy of the serious game.

Ambiguity-Free Optical-Inertial Tracking for Augmented Reality Headsets.

The increasing capability of computing power and mobile graphics has made possible the release of self-contained augmented reality (AR) headsets featuring efficient head-anchored tracking solutions. Ego motion estimation based on well-established infrared tracking of markers ensures sufficient accuracy and robustness. Unfortunately, wearable visible-light stereo cameras with short baseline and operating under uncontrolled lighting conditions suffer from tracking failures and ambiguities in pose estimation. To improve the accuracy of optical self-tracking and its resiliency to marker occlusions, degraded camera calibrations, and inconsistent lighting, in this work we propose a sensor fusion approach based on Kalman filtering that integrates optical tracking data with inertial tracking data when computing motion correlation. In order to measure improvements in AR overlay accuracy, experiments are performed with a custom-made AR headset designed for supporting complex manual tasks performed under direct vision. Experimental results show that the proposed solution improves the head-mounted display (HMD) tracking accuracy by one third and improves the robustness by also capturing the orientation of the target scene when some of the markers are occluded and when the optical tracking yields unstable and/or ambiguous results due to the limitations of using head-anchored stereo tracking cameras under uncontrollable lighting conditions.

Bioreactor With Electrically Deformable Curved Membranes for Mechanical Stimulation of Cell Cultures.

Physiologically relevant in vitro models of stretchable biological tissues, such as muscle, lung, cardiac and gastro-intestinal tissues, should mimic the mechanical cues which cells are exposed to in their dynamic microenvironment in vivo. In particular, in order to mimic the mechanical stimulation of tissues in a physiologically relevant manner, cell stretching is often desirable on surfaces with dynamically controllable curvature. Here, we present a device that can deform cell culture membranes without the current need for external pneumatic/fluidic or electrical motors, which typically make the systems bulky and difficult to operate. We describe a modular device that uses elastomeric membranes, which can intrinsically be deformed by electrical means, producing a dynamically tuneable curvature. This approach leads to compact, self-contained, lightweight and versatile bioreactors, not requiring any additional mechanical equipment. This was obtained via a special type of dielectric elastomer actuator. The structure, operation and performance of early prototypes are described, showing preliminary evidence on their ability to induce changes on the spatial arrangement of the cytoskeleton of fibroblasts dynamically stretched for 8 h.

Low-Computational Cost Stitching Method in a Three-Eyed Endoscope.

Aortic valve replacement is the only definitive treatment for aortic stenosis, a highly prevalent condition in elderly population. Minimally invasive surgery brought numerous benefits to this intervention, and robotics recently provided additional improvements in terms of telemanipulation, motion scaling, and smaller incisions. Difficulties in obtaining a clear and wide field of vision is a major challenge in minimally invasive aortic valve surgery: surgeon orientates with difficulty because of lack of direct view and limited spaces. This work focuses on the development of a computer vision methodology, for a three-eyed endoscopic vision system, to ease minimally invasive instrument guidance during aortic valve surgery. Specifically, it presents an efficient image stitching method to improve spatial awareness and overcome the orientation problems which arise when cameras are decentralized with respect to the main axis of the aorta and are nonparallel oriented. The proposed approach was tested for the navigation of an innovative robotic system for minimally invasive valve surgery. Based on the specific geometry of the setup and the intrinsic parameters of the three cameras, we estimate the proper plane-induced homographic transformation that merges the views of the operatory site plane into a single stitched image. To evaluate the deviation from the image correct alignment, we performed quantitative tests by stitching a chessboard pattern. The tests showed a minimum error with respect to the image size of 0.46 ± 0.15% measured at the homography distance of 40 mm and a maximum error of 6.09 ± 0.23% at the maximum offset of 10 mm. Three experienced surgeons in aortic valve replacement by mini-sternotomy and mini-thoracotomy performed experimental tests based on the comparison of navigation and orientation capabilities in a silicone aorta with and without stitched image. The tests showed that the stitched image allows for good orientation and navigation within the aorta, and furthermore, it provides more safety while releasing the valve than driving from the three separate views. The average processing time for the stitching of three views into one image is 12.6 ms, proving that the method is not computationally expensive, thus leaving space for further real-time processing.