Virtual Inspection System for Pumping Stations with Multimodal Feedback.

Pumping stations have undergone significant modernization and digitalization in recent decades. However, traditional virtual inspections often prioritize the visual experience and fail to effectively represent the haptic physical properties of devices during inspections, resulting in poor immersion and interactivity. This paper presents a novel virtual inspection system for pumping stations, incorporating virtual reality interaction and haptic force feedback technology to enhance immersion and realism. The system leverages a 3D model, crafted in 3Ds Max, to provide immersive visualizations. Multimodal feedback is achieved through a combination of haptic force feedback provided by a haptic device and visual information delivered by a VR headset. The system's data platform integrates with external databases using Unity3D to display relevant information. The system provides immersive 3D visualizations and realistic force feedback during simulated inspections. We compared this system to a traditional virtual inspection method that demonstrated statistically significant improvements in task completion rates and a reduction in failure rates when using the multimodal feedback approach. This innovative approach holds the potential to enhance inspection safety, efficiency, and effectiveness in the pumping station industry.

Detecting Target Objects by Natural Language Instructions Using an RGB-D Camera.

Controlling robots by natural language (NL) is increasingly attracting attention for its versatility, convenience and no need of extensive training for users. Grounding is a crucial challenge of this problem to enable robots to understand NL instructions from humans. This paper mainly explores the object grounding problem and concretely studies how to detect target objects by the NL instructions using an RGB-D camera in robotic manipulation applications. In particular, a simple yet robust vision algorithm is applied to segment objects of interest. With the metric information of all segmented objects, the object attributes and relations between objects are further extracted. The NL instructions that incorporate multiple cues for object specifications are parsed into domain-specific annotations. The annotations from NL and extracted information from the RGB-D camera are matched in a computational state estimation framework to search all possible object grounding states. The final grounding is accomplished by selecting the states which have the maximum probabilities. An RGB-D scene dataset associated with different groups of NL instructions based on different cognition levels of the robot are collected. Quantitative evaluations on the dataset illustrate the advantages of the proposed method. The experiments of NL controlled object manipulation and NL-based task programming using a mobile manipulator show its effectiveness and practicability in robotic applications.

Potential Haptic Perceptual Dimensionality of Rendered Compliance.

Studies have proven that humans perceive haptic textures through different perceptual dimensions, such as rough/smooth and soft/hard, which provide useful guidance in the design of haptic devices. However, few of these have focused on the perception of compliance, which is another important perceptual property in haptic interfaces. This research was conducted to investigate the potential basic perceptual dimensions of the rendered compliance and quantify the effects of the simulation parameters. Two perceptual experiments were designed based on 27 stimuli samples generated by a 3-DOF haptic feedback device. Subjects were asked to describe these stimuli using adjectives, classify the samples, and rate them according to corresponding adjective labels. Multi-dimensional scaling (MDS) methods were then used to project adjective ratings into 2D and 3D perception spaces. According to the results, hardness and viscosity are considered two basic perceptual dimensions of the rendered compliance, while crispness can be regarded as a subsidiary perceptual dimension. Then, the relations between simulation parameters and perceptual feelings were analyzed by the regression analysis. This paper may provide a better understanding of the compliance perception mechanism and useful guidance for the improvement of rendering algorithms and devices for haptic human-computer interaction.