Video analysis of optic-haptic-interaction during hydrophobic acrylic intraocular lens implantation using preloaded injectors.

OBJECTIVE: To compare the optic-haptic interaction of different hydrophobic acrylic IOLs after using six preloaded injectors. METHODS: We reviewed the video-recordings of procedures on a total of 388 eyes that underwent phacoemulsification and intraocular lens (IOL) implantation. For six preloaded injectors: multiSert (Hoya Surgical Optics) [System 1], TECNIS Simplicity (Johnson & Johnson Vision) [System 2], TECNIS iTec (Johnson & Johnson Vision) [System 3], AutonoMe (Alcon, Laboratories) [System 4], Bluesert (Carl Zeiss Meditec) [System 5], and Prosert (OphthalmoPro GmbH) [System 6], we noted in each case the time of IOL delivery and made a descriptive observation of IOL insertion and optic-haptic-interaction. RESULTS: We defined standard haptic behavior where the haptics emerged "folded" from the injector and quickly recovered their pre-implantation appearance. The incidence where the leading haptic emerged in a deformed way for System 1 was 20%, System 2: 19%, System 3: 14%, System 4: 56%, System 5: 24% and System 6: 5%. For trailing haptic deformed behavior, the incidence was 36%, 6%, 4%, 8%, 18% and 2%, respectively for Systems 1 to 6. Optic-haptic adhesion occurred in 2% of cases for System 1, 44% for System 2, 52% for System 3, 48% for System 4, and 11% for System 6 (P < 0.05). Adhesion was not found with System 5. CONCLUSIONS: We observed different deformed behavior for leading and trailing haptics in the six preloaded systems, some systems had as much as 52% optic-haptic adhesion.

The "Beam-Me-In Strategy" - remote haptic therapist-patient interaction with two exoskeletons for stroke therapy.

BACKGROUND: We present a robot-assisted telerehabilitation system that allows for haptic interaction between therapist and patient over distance. It consists of two arm therapy robots. Attached to one robot the therapists can feel on their own arm the limitations of the patient's arm which is attached to the other robot. Due to the exoskeleton structure of the robot, movements can be performed in the three-dimensional space. METHODS: Fifteen physical and occupational therapists tested this strategy, named "Beam-Me-In", while using an exoskeleton robot connected to a second exoskeleton robot in the same room used by the study experimenter. Furthermore, the therapists assessed the level of impairment of recorded and simulated arm movements. They quantified four typical impairments of stroke patients: reduced range of motion (active and passive), resistance to passive movement, a lack of ability to fractionate a movement, and disturbed quality of movement. RESULTS: On a Likert Scale (0 to 5 points) therapists rated the "Beam-Me-In" strategy as a very useful medium (mode: 4 points) to evaluate a patient's progress over time. The passive range of motion of the elbow joint was assessed with a mean absolute error of 4.9∘ (absolute precision error: 6.4∘). The active range of motion of the elbow was assessed with a mean absolute error of 4.9∘ (absolute precision error: 6.5∘). The resistance to passive movement (i.e. modified Tardieu Scale) and the lack of ability to fractionate a movement (i.e. quantification of pathological muscle synergies) was assessed with an inter-rater reliability of 0.930 and 0.948, respectively. CONCLUSIONS: The "Beam-Me-In" strategy is a promising approach to complement robot-assisted movement training. It can serve as a platform to assess and identify abnormal movement patterns in patients. This is the first application of remote three-dimensional haptic assessmen t applied to telerehabilitation. Furthermore, the "Beam-Me-In" strategy has a potential to overcome barriers for therapists regarding robot-assisted telerehabilitation.

An Application of Deep Learning to Tactile Data for Object Recognition under Visual Guidance.

Drawing inspiration from haptic exploration of objects by humans, the current work proposes a novel framework for robotic tactile object recognition, where visual information in the form of a set of visually interesting points is employed to guide the process of tactile data acquisition. Neuroscience research confirms the integration of cutaneous data as a response to surface changes sensed by humans with data from joints, muscles, and bones (kinesthetic cues) for object recognition. On the other hand, psychological studies demonstrate that humans tend to follow object contours to perceive their global shape, which leads to object recognition. In compliance with these findings, a series of contours are determined around a set of 24 virtual objects from which bimodal tactile data (kinesthetic and cutaneous) are obtained sequentially and by adaptively changing the size of the sensor surface according to the object geometry for each object. A virtual Force Sensing Resistor array (FSR) is employed to capture cutaneous cues. Two different methods for sequential data classification are then implemented using Convolutional Neural Networks (CNN) and conventional classifiers, including support vector machines and k-nearest neighbors. In the case of conventional classifiers, we exploit contourlet transformation to extract features from tactile images. In the case of CNN, two networks are trained for cutaneous and kinesthetic data and a novel hybrid decision-making strategy is proposed for object recognition. The proposed framework is tested both for contours determined blindly (randomly determined contours of objects) and contours determined using a model of visual attention. Trained classifiers are tested on 4560 new sequential tactile data and the CNN trained over tactile data from object contours selected by the model of visual attention yields an accuracy of 98.97% which is the highest accuracy among other implemented approaches.

Asymmetric Direct Reciprocal Connections Between Primary Visual and Somatosensory Cortices of the Mouse.

Several studies show direct connections between primary sensory cortices involved in multisensory integration. The purpose of this study is to understand the microcircuitry of the reciprocal connections between visual and somatosensory cortices. The laminar distribution of retrogradely labeled cell bodies in V1 and in the somatosensory cortex both in (S1BF) and outside (S1) the barrel field was studied to provide layer indices in order to determine whether the connections are of feedforward, feedback or lateral type. Single axons were reconstructed and the size of their swellings was stereologically sampled. The negative layer indices in S1 and S1BF and the layer index near zero in V1 indicate that the connection from S1BF to V1 is of feedback type while the opposite is of lateral type. The greater incidence of larger axonal swellings in the projection from V1 to S1BF strongly suggests that S1BF receives a stronger driver input from V1 and that S1BF inputs to V1 have a predominant modulatory influence.

Palpation simulator with stable haptic feedback.

BACKGROUND: The main difficulty in constructing palpation simulators is to compute and to generate stable and realistic haptic feedback without vibration. When a user haptically interacts with highly non-homogeneous soft tissues through a palpation simulator, a sudden change of stiffness in target tissues causes unstable interaction with the object. MATERIAL AND METHODS: We propose a model consisting of a virtual adjustable damper and an energy measuring element. The energy measuring element gauges energy which is stored in a palpation simulator and the virtual adjustable damper dissipates the energy to achieve stable haptic interaction. RESULTS: To investigate the haptic behavior of the proposed method, impulse and continuous inputs are provided to target tissues. If a haptic interface point meets with the hardest portion in the target tissues modeled with a conventional method, we observe unstable motion and feedback force. However, when the target tissues are modeled with the proposed method, a palpation simulator provides stable interaction without vibration. CONCLUSION: The proposed method overcomes a problem in conventional haptic palpation simulators where unstable force or vibration can be generated if there is a big discrepancy in material property between an element and its neighboring elements in target tissues.

Perceptually Inspired C0-Continuity Haptic Shape Display with Trichamber Soft Actuators.

Shape display devices composed of actuation pixels enable dynamic rendering of surface morphological features, which have important roles in virtual reality and metaverse applications. The traditional pin-array solution produces sidestep-like structures between neighboring pins and normally relies on high-density pins to obtain curved surfaces. It remains a challenge to achieve continuous curved surfaces using a small number of actuated units. To address the challenge, we resort to the concept of surface continuity in computational geometry and develop a C0-continuity shape display device with trichamber fiber-reinforced soft actuators. Each trichamber unit produces three-dimensional (3D) deformation consisting of elongation, pitch, and yaw rotation, thus ensuring rendered surface continuity using low-resolution actuation units. Inspired by human tactile discrimination threshold on height and angle gradients between adjacent units, we proposed the mathematical criteria of C0-continuity shape display and compared the maximal number of distinguishable shapes using the proposed device in comparison with typical pin-array. We then established a shape control model considering the nonlinearity of soft materials to characterize and control the soft device to display C0-continuity shapes. Experimental results showed that the proposed device with nine trichamber units could render typical sets of distinguishable C0-continuity shape sequence changes. We envision that the concept of C0-continuity shape display with 3D deformation capability could improve the fidelity of the rendered shapes in many metaverse scenarios such as touching human organs in medical palpation simulations.

Multisensory Extended Reality Applications Offer Benefits for Volumetric Biomedical Image Analysis in Research and Medicine.

3D data from high-resolution volumetric imaging is a central resource for diagnosis and treatment in modern medicine. While the fast development of AI enhances imaging and analysis, commonly used visualization methods lag far behind. Recent research used extended reality (XR) for perceiving 3D images with visual depth perception and touch but used restrictive haptic devices. While unrestricted touch benefits volumetric data examination, implementing natural haptic interaction with XR is challenging. The research question is whether a multisensory XR application with intuitive haptic interaction adds value and should be pursued. In a study, 24 experts for biomedical images in research and medicine explored 3D medical shapes with 3 applications: a multisensory virtual reality (VR) prototype using haptic gloves, a simple VR prototype using controllers, and a standard PC application. Results of standardized questionnaires showed no significant differences between all application types regarding usability and no significant difference between both VR applications regarding presence. Participants agreed to statements that VR visualizations provide better depth information, using the hands instead of controllers simplifies data exploration, the multisensory VR prototype allows intuitive data exploration, and it is beneficial over traditional data examination methods. While most participants mentioned manual interaction as the best aspect, they also found it the most improvable. We conclude that a multisensory XR application with improved manual interaction adds value for volumetric biomedical data examination. We will proceed with our open-source research project ISH3DE (Intuitive Stereoptic Haptic 3D Data Exploration) to serve medical education, therapeutic decisions, surgery preparations, or research data analysis.

Dimensionality Reduction in Controlling Articulated Snake Robot for Endoscopy Under Dynamic Active Constraints.

This paper presents a real-time control framework for a snake robot with hyper-kinematic redundancy under dynamic active constraints for minimally invasive surgery. A proximity query (PQ) formulation is proposed to compute the deviation of the robot motion from predefined anatomical constraints. The proposed method is generic and can be applied to any snake robot represented as a set of control vertices. The proposed PQ formulation is implemented on a graphic processing unit, allowing for fast updates over 1 kHz. We also demonstrate that the robot joint space can be characterized into lower dimensional space for smooth articulation. A novel motion parameterization scheme in polar coordinates is proposed to describe the transition of motion, thus allowing for direct manual control of the robot using standard interface devices with limited degrees of freedom. Under the proposed framework, the correct alignment between the visual and motor axes is ensured, and haptic guidance is provided to prevent excessive force applied to the tissue by the robot body. A resistance force is further incorporated to enhance smooth pursuit movement matched to the dynamic response and actuation limit of the robot. To demonstrate the practical value of the proposed platform with enhanced ergonomic control, detailed quantitative performance evaluation was conducted on a group of subjects performing simulated intraluminal and intracavity endoscopic tasks.

Granger Geweke Causality reveals information exchange during physical interaction is modulated by task difficulty.

The haptic sense is an important mode of communication during physical interactions, and it is known to enable humans to estimate key features of their partner's behavior. It is proposed that such estimations are based upon the exchange of information mediated by the interaction forces, resulting in role distribution and coordination between partners. In the present study, we examined whether the information exchange is functionally modified to adapt to the task, or whether it is a fixed process, leaving the adaptation to individual's behaviors. We analyzed the forces during an empirical dyadic interaction task using Granger-Geweke causality analysis, which allowed us to quantify the causal influence of each individual's forces on their partner's. The dynamics of relative phase were also examined. We observed an increase of inter-partner influence with an increase in the spatial accuracy required by the task, demonstrating an adaptation of information flow to the task. This increase of exchange with the spatial accuracy constraint was accompanied by an increase of errors and of the variability of the relative phase between forces. The influence was dominated by participants in a specific role, showing a clear role division as well as task division between the dyad partners. Moreover, the influence occurred in the [2.15-7] Hz frequency band, demonstrating its importance as a frequency band of interest during cooperation involving haptic interaction. Several interpretations are introduced, ranging from sub-division of motion control to phase-amplitude coupling.

Implementation of the Haptic Tele-Weight Device Using a 10 MHz Smart Torch VLC Link.

Considering the prerequisite need for a protected e-commerce platform, absence of haptic interaction in head-mounted displays (HMD), and exploitation of faster communication technology, this research work aims to present an amended version of the tele-weight device that utilizes the 6G visible light communication (VLC) technology, is faster in performance, and deals with a heavier article. The enhanced version of the device is to be called the 'VLC tele-weight device' and the aim for the VLC tele-weight device is to get it affixed over the headset which will allow the user to have the weight-based sensation of the product ordered on the virtual store. The proposed device sending end and receiving end part performs communication over the VLC link. Furthermore, Arduino Nano is used as the microcontroller (MCU) in the project. Sending end circuitry measures the weight using the load cell and HX711 amplifier combination and transmits it via the connected LED. The pre-equalizer circuit is connected between the LED and sending end part to improve the bandwidth. On the receiver side, the post-equalizer circuit improves the shape of the received pulse. The received weight value is then displayed using the motor-gear combination. The sending end device is to be sited at the virtual store, while the receiving end is planned to be positioned over the VR headset. The performance of the device was measured by performing repeated trials and the percentage error was found to be between 0.5-3%. Merging the field of embedded systems, internet of things (IoT), VLC, signal processing, virtual reality (VR), e-commerce, and haptic sensing, the idea proposed in this research work can help introduce the haptic interaction, and sensational realization-based innovation in immersive visualization (IV) and graphical user interface (GUI) domain.