An Efficient Robotic Pushing and Grasping Method in Cluttered Scene.

Pushing and grasping (PG) are crucial skills for intelligent robots. These skills enable robots to perform complex grasping tasks in various scenarios. These PG methods can be categorized into single-stage and multistage approaches. Single-stage methods are faster but less accurate, while multistage methods offer high accuracy at the expense of time efficiency. To address this issue, a novel end-to-end PG method called efficient PG network (EPGNet) is proposed in this article. EPGNet achieves both high accuracy and efficiency simultaneously. To optimize performance with fewer parameters, EfficientNet-B0 is used as the backbone of EPGNet. Additionally, a novel cross-fusion module is introduced to enhance network performance in robotic PG tasks. This module fuses and utilizes local and global features, aiding the network in handling objects of varying sizes in different scenes. EPGNet consists of two branches dedicated to predicting PG actions, respectively. Both branches are trained simultaneously within a Q -learning framework. Training data is collected through trial and error, involving the robot performing PG actions. To bridge the gap between simulation and reality, a unique PG dataset is proposed. Additionally, a YOLACT network is trained on the PG dataset to facilitate object detection and segmentation. A comprehensive set of experiments is conducted in simulated environments and real-world scenarios. The results demonstrate that EPGNet outperforms single-stage methods and offers competitive performance compared to multistage methods, all while utilizing fewer parameters. A video is available at https://youtu.be/HNKJjQH0MPc.

Category-Level 6-D Object Pose Estimation With Shape Deformation for Robotic Grasp Detection.

Category-level 6-D object pose estimation plays a crucial role in achieving reliable robotic grasp detection. However, the disparity between synthetic and real datasets hinders the direct transfer of models trained on synthetic data to real-world scenarios, leading to ineffective results. Additionally, creating large-scale real datasets is a time-consuming and labor-intensive task. To overcome these challenges, we propose CatDeform, a novel category-level object pose estimation network trained on synthetic data but capable of delivering good performance on real datasets. In our approach, we introduce a transformer-based fusion module that enables the network to leverage multiple sources of information and enhance prediction accuracy through feature fusion. To ensure proper deformation of the prior point cloud to align with scene objects, we propose a transformer-based attention module that deforms the prior point cloud from both geometric and feature perspectives. Building upon CatDeform, we design a two-branch network for supervised learning, bridging the gap between synthetic and real datasets and achieving high-precision pose estimation in real-world scenes using predominantly synthetic data supplemented with a small amount of real data. To minimize reliance on large-scale real datasets, we train the network in a self-supervised manner by estimating object poses in real scenes based on the synthetic dataset without manual annotation. We conduct training and testing on CAMERA25 and REAL275 datasets, and our experimental results demonstrate that the proposed method outperforms state-of-the-art (SOTA) techniques in both self-supervised and supervised training paradigms. Finally, we apply CatDeform to object pose estimation and robotic grasp experiments in real-world scenarios, showcasing a higher grasp success rate.

A Novel Robotic Pushing and Grasping Method Based on Vision Transformer and Convolution.

Robotic grasping techniques have been widely studied in recent years. However, it is always a challenging problem for robots to grasp in cluttered scenes. In this issue, objects are placed close to each other, and there is no space around for the robot to place the gripper, making it difficult to find a suitable grasping position. To solve this problem, this article proposes to use the combination of pushing and grasping (PG) actions to help grasp pose detection and robot grasping. We propose a pushing-grasping combined grasping network (GN), PG method based on transformer and convolution (PGTC). For the pushing action, we propose a vision transformer (ViT)-based object position prediction network pushing transformer network (PTNet), which can well capture the global and temporal features and can better predict the position of objects after pushing. To perform the grasping detection, we propose a cross dense fusion network (CDFNet), which can make full use of the RGB image and depth image, and fuse and refine them several times. Compared with previous networks, CDFNet is able to detect the optimal grasping position more accurately. Finally, we use the network for both simulation and actual UR3 robot grasping experiments and achieve SOTA performance. Video and dataset are available at https://youtu.be/Q58YE-Cc250.

Discrete-Time Control Barrier Function: High-Order Case and Adaptive Case.

This article proposes the novel concepts of the high-order discrete-time control barrier function (CBF) and adaptive discrete-time CBF. The high-order discrete-time CBF is used to guarantee forward invariance of a safe set for discrete-time systems of high relative degree. An optimization problem is then established unifying high-order discrete-time CBFs with discrete-time control Lyapunov functions to yield a safe controller. To improve the feasibility of such optimization problems, the adaptive discrete-time CBF is designed, which can relax constraints on system control input through time-varying penalty functions. The effectiveness of the proposed methods in dealing with high relative degree constraints and improving feasibility is verified on the discrete-time system of a three-link manipulator.

Electrically Controlled Aquatic Soft Actuators with Desynchronized Actuation and Light-Mediated Reciprocal Locomotion.

Soft-bodied aquatic invertebrates can overcome hydrodynamic resistance and display diverse locomotion modes in response to environmental cues. Exploring the dynamics of locomotion from bioinspired aquatic actuators will broaden the perspective of underwater manipulation of artificial systems in fluidic environments. Here, we report a multilayer soft actuator design based on a light-driven hydrogel and a laser-induced graphene (LIG) actuator, minimizing the effect of the time delay by a monolithic hydrogel-based system while maintaining shape-morphing functionality. Moreover, different time scales in the response of actuator materials enable a real-time desynchronization of energy inputs, holding great potential for applications requiring desynchronized stimulation. This hybrid design principle is ultimately demonstrated with a high-performance aquatic soft actuator possessing an underwater walking speed of 0.81 body length per minute at a relatively low power consumption of 3 W. When integrated with an optical sensor, the soft actuator can sense the variation in light intensity and achieve mediated reciprocal motion. Our proposed locomotion mechanism could inspire other multilayer soft actuators to achieve underwater functionalities at the same spatiotemporal scale. The underwater actuation platform could be used to study locomotion kinematics and control mechanisms that mimic the motion of soft-bodied aquatic organisms.

An MVMD-CCA Recognition Algorithm in SSVEP-Based BCI and Its Application in Robot Control.

This article proposes a novel recognition algorithm for the steady-state visual evoked potentials (SSVEP)-based brain-computer interface (BCI) system. By combining the advantages of multivariate variational mode decomposition (MVMD) and canonical correlation analysis (CCA), an MVMD-CCA algorithm is investigated to improve the detection ability of SSVEP electroencephalogram (EEG) signals. In comparison with the classical filter bank canonical correlation analysis (FBCCA), the nonlinear and non-stationary EEG signals are decomposed into a fixed number of sub-bands by MVMD, which can enhance the effect of SSVEP-related sub-bands. The experimental results show that MVMD-CCA can effectively reduce the influence of noise and EEG artifacts and improve the performance of SSVEP-based BCI. The offline experiments show that the average accuracies of MVMD-CCA in the training dataset and testing dataset are improved by 3.08% and 1.67%, respectively. In the SSVEP-based online robotic manipulator grasping experiment, the recognition accuracies of the four subjects are 92.5%, 93.33%, 90.83%, and 91.67%, respectively.

A Novel Switching-Based Control Framework for Improved Task Performance in Teleoperation System With Asymmetric Time-Varying Delays.

This paper addresses the adaptive control for task-space teleoperation systems with constrained predefined synchronization error, where a novel switched control framework is investigated. Based on multiple Lyapunov-Krasovskii functionals method, the stability of the resulting closed-loop system is established in the sense of state-independent input-to-output stability. Compared with previous work, the developed method can simultaneously handle the unknown kinematics/dynamics, asymmetric varying time delays, and prescribed performance control in a unified framework. It is shown that the developed controller can guarantee the prescribed transient-state and steady-state synchronization performances between the master and slave robots, which is demonstrated by the simulation study.

Multilateral Telecoordinated Control of Multiple Robots With Uncertain Kinematics.

This paper addresses the telecoordinated control of multiple robots in the simultaneous presence of asymmetric time-varying delays, nonpassive external forces, and uncertain kinematics/dynamics. To achieve the control objective, a neuroadaptive controller with utilizing prescribed performance control and switching control technique is developed, where the basic idea is to employ the concept of motion synchronization in each pair of master-slave robots and among all slave robots. By using the multiple Lyapunov-Krasovskii functionals method, the state-independent input-to-output practical stability of the closed-loop system is established. Compared with the previous approaches, the new design is straightforward and easier to implement and is applicable to a wider area. Simulation results on three pairs of three degrees-of-freedom robots confirm the theoretical findings.

Adaptive Control of Semi-Autonomous Teleoperation System With Asymmetric Time-Varying Delays and Input Uncertainties.

This paper addresses the adaptive task-space bilateral teleoperation for heterogeneous master and slave robots to guarantee stability and tracking performance, where a novel semi-autonomous teleoperation framework is developed to ensure the safety and enhance the efficiency of the robot in remote site. The basic idea is to stabilize the tracking error in task space while enhancing the efficiency of complex teleoperation by using redundant slave robot with subtask control. To unify the study of the asymmetric time-varying delays, passive/nonpassive exogenous forces, dynamic parameter uncertainties and dead-zone input in the same framework, a novel switching technique-based adaptive control scheme is investigated, where a special switched error filter is developed. By replacing the derivatives of position errors with their filtered outputs in the coordinate torque design, and employing the multiple Lyapunov-Krasovskii functionals method, the complete closed-loop master (slave) system is proven to be state-independent input-to-output stable. It is shown that both the position tracking errors in task space and the adaptive parameter estimation errors remain bounded for any bounded exogenous forces. Moreover, by using the redundancy of the slave robot, the proposed teleoperation framework can autonomously achieve additional subtasks in the remote environment. Finally, the obtained results are demonstrated by the simulation.

Neural Network-Based Control of Networked Trilateral Teleoperation With Geometrically Unknown Constraints.

Most studies on bilateral teleoperation assume known system kinematics and only consider dynamical uncertainties. However, many practical applications involve tasks with both kinematics and dynamics uncertainties. In this paper, trilateral teleoperation systems with dual-master-single-slave framework are investigated, where a single robotic manipulator constrained by an unknown geometrical environment is controlled by dual masters. The network delay in the teleoperation system is modeled as Markov chain-based stochastic delay, then asymmetric stochastic time-varying delays, kinematics and dynamics uncertainties are all considered in the force-motion control design. First, a unified dynamical model is introduced by incorporating unknown environmental constraints. Then, by exact identification of constraint Jacobian matrix, adaptive neural network approximation method is employed, and the motion/force synchronization with time delays are achieved without persistency of excitation condition. The neural networks and parameter adaptive mechanism are combined to deal with the system uncertainties and unknown kinematics. It is shown that the system is stable with the strict linear matrix inequality-based controllers. Finally, the extensive simulation experiment studies are provided to demonstrate the performance of the proposed approach.

On Input-to-State Stability of Switched Stochastic Nonlinear Systems Under Extended Asynchronous Switching.

An extended asynchronous switching model is investigated for a class of switched stochastic nonlinear retarded systems in the presence of both detection delay and false alarm, where the extended asynchronous switching is described by two independent and exponentially distributed stochastic processes, and further simplified as Markovian. Based on the Razumikhin-type theorem incorporated with average dwell-time approach, the sufficient criteria for global asymptotic stability in probability and stochastic input-to-state stability are given, whose importance and effectiveness are finally verified by numerical examples.