Optical Upconversion in Mononuclear Lanthanide Co-crystal Assemblies.

In this work, we developed two kinds of co-crystal assemblies systems, consisting of discrete mononuclear Yb3+ and Er3+ and mononuclear Yb3+ and Pr3+, which can achieve Er3+ and Pr3+ upconversion luminescence, respectively, by Yb3+ sensitization under 980 nm excitation. The structure and composition of two co-crystal assemblies were determined by single crystal X-ray diffraction. By investigation of the series of two assemblies, respectively, it is found that the strongest upconversion luminescence is both obtained when the molar ratio of Yb3+ and Ln3+ (Ln = Er or Pr) is 1 : 1. The energy transfer mechanism of Er3+ assemblies is determined as energy transfer upconversion, while that of Pr3+ assemblies is determined as energy transfer upconversion and cooperative sensitization upconversion. This is the first example of Pr3+ upconversion luminescence at the molecular dimension at room temperature, which enriches the research in the field of upconversion luminescence with lanthanide complexes.

Human skill knowledge guided global trajectory policy reinforcement learning method.

Traditional trajectory learning methods based on Imitation Learning (IL) only learn the existing trajectory knowledge from human demonstration. In this way, it can not adapt the trajectory knowledge to the task environment by interacting with the environment and fine-tuning the policy. To address this problem, a global trajectory learning method which combinines IL with Reinforcement Learning (RL) to adapt the knowledge policy to the environment is proposed. In this paper, IL is proposed to acquire basic trajectory skills, and then learns the agent will explore and exploit more policy which is applicable to the current environment by RL. The basic trajectory skills include the knowledge policy and the time stage information in the whole task space to help learn the time series of the trajectory, and are used to guide the subsequent RL process. Notably, neural networks are not used to model the action policy and the Q value of RL during the RL process. Instead, they are sampled and updated in the whole task space and then transferred to the networks after the RL process through Behavior Cloning (BC) to get continuous and smooth global trajectory policy. The feasibility and the effectiveness of the method was validated in a custom Gym environment of a flower drawing task. And then, we executed the learned policy in the real-world robot drawing experiment.

Opsonization Inveigles Macrophages Engulfing Carrier-Free Bilirubin/JPH203 Nanoparticles to Suppress Inflammation for Osteoarthritis Therapy.

Osteoarthritis (OA) is a chronic inflammatory disease characterized by cartilage destruction, synovitis, and osteophyte formation. Disease-modifying treatments for OA are currently lacking. Because inflammation mediated by an imbalance of M1/M2 macrophages in the synovial cavities contributes to OA progression, regulating the M1 to M2 polarization of macrophages can be a potential therapeutic strategy. Basing on the inherent immune mechanism and pathological environment of OA, an immunoglobulin G-conjugated bilirubin/JPH203 self-assembled nanoparticle (IgG/BRJ) is developed, and its therapeutic potential for OA is evaluated. After intra-articular administration, IgG conjugation facilitates the recognition and engulfment of nanoparticles by the M1 macrophages. The internalized nanoparticles disassemble in response to the increased oxidative stress, and the released bilirubin (BR) and JPH203 scavenge reactive oxygen species (ROS), inhibit the nuclear factor kappa-B pathway, and suppress the activated mammalian target of rapamycin pathway, result in the repolarization of macrophages and enhance M2/M1 ratios. Suppression of the inflammatory environment by IgG/BRJ promotes cartilage protection and repair in an OA rat model, thereby improving therapeutic outcomes. This strategy of opsonization involving M1 macrophages to engulf carrier-free BR/JPH203 nanoparticles to suppress inflammation for OA therapy holds great potential for OA intervention and treatment.

A study on the aerodynamic behaviors learned from microscopy imaging of beetle corrugated hindwing.

Beetle hindwings have the unique advantages of lightweight and high strength, which play a key role in flight. In this study, the beetle hindwings were cut along the chordal direction, then the first groove microstructure of different vein cross sections was investigated using the 3D microscope system and the laser scanning confocal microscope. It was found that the position of the first groove relative to the entire chordal cross section of the wing gradually moves backward, which has an effect on the flying aerodynamic behaviors of the beetle. Next, three corrugated airfoils learned from the microscopy imaging of the ladybird beetle hindwing were designed. Then, aerodynamic behaviors were calculated by the ANSYS Fluent software, and it was confirmed that the position of the first groove microstructure affects the aerodynamic performance of the airfoil. For further study, the influence of corrugated structural and motion parameters on the aerodynamic, 2D 'simplified' airfoil models with triangular wave airfoil models (TWA models) was developed and studied. RESEARCH HIGHLIGHTS: The position of the first groove microstructure affects the aerodynamic performance of the airfoil. The pressure difference of different corrugation patterns shows significantly asymmetric during the upstroke and downstroke. The aerodynamic is optimal of 2D-TWA models, when the number of corrugations is five, the corrugation is right angle, and the flapping frequency is 75 Hz.

Microfluidic device featuring micro-constrained channels for multi-parametric assessment of cellular biomechanics and high-precision mechanical phenotyping of gastric cells.

BACKGROUND: Cellular biomechanics plays a significant role in the regulation of cellular physiological and pathological processes. In recent years, multiple methods have been developed to evaluate cellular biomechanics, such as atomic force microscopy (AFM), micropipette aspiration, and magnetic tweezers. However, most of these methods only focus on a single parameter and cannot automate the process at a high-efficiency level. A novel microfluidic method is necessary to achieve the simultaneous multi-parametric measurement of cellular biomechanics and high-precision cellular mechanical phenotyping at high throughput. RESULTS: To tackle the issue concerning the low-throughput and cellular single-parameter evaluation, we designed and fabricated a microfluidic chip featuring multiple micro-constrained channels structure, providing a simultaneous multi-parametric assessment of cellular biomechanics, including elastic modulus, recovery capability, and deformability. We compared the biomechanical properties of normal human gastric mucosal epithelial cells (GES-1) and human gastric cancer cells (AGS and MKN-45) by the chip. Results demonstrated that the elastic modulus of GES-1, AGS, and MKN-45 cells decreased sequentially, which was the opposite of their invasiveness and metastasis potential, suggesting the inverse correlation between cellular elastic modulus and malignancy. Meanwhile, the recovery capability and deformability of GES-1, AGS, and MKN-45 cells increased sequentially, demonstrating the positive correlation between cellular deformability and malignancy. Furthermore, multiple parameters were used to distinguish gastric cancer cells from normal gastric cells via machine learning. An accuracy of over 94.8% for identifying gastric cancer cells was achieved. SIGNIFICANCE: This study provides a deep insight into the biophysical mechanism of gastric cancer metastasis at the single-cell level and possesses great potential to function as a valuable tool for single-cell analysis, thereby facilitating high-precision and high-throughput discrimination of cellular phenotypes that are not easily discernible through single-marker analysis.

Motion planning framework based on dual-agent DDPG method for dual-arm robots guided by human joint angle constraints.

Introduction: Reinforcement learning has been widely used in robot motion planning. However, for multi-step complex tasks of dual-arm robots, the trajectory planning method based on reinforcement learning still has some problems, such as ample exploration space, long training time, and uncontrollable training process. Based on the dual-agent depth deterministic strategy gradient (DADDPG) algorithm, this study proposes a motion planning framework constrained by the human joint angle, simultaneously realizing the humanization of learning content and learning style. It quickly plans the coordinated trajectory of dual-arm for complex multi-step tasks. Methods: The proposed framework mainly includes two parts: one is the modeling of human joint angle constraints. The joint angle is calculated from the human arm motion data measured by the inertial measurement unit (IMU) by establishing a human-robot dual-arm kinematic mapping model. Then, the joint angle range constraints are extracted from multiple groups of demonstration data and expressed as inequalities. Second, the segmented reward function is designed. The human joint angle constraint guides the exploratory learning process of the reinforcement learning method in the form of step reward. Therefore, the exploration space is reduced, the training speed is accelerated, and the learning process is controllable to a certain extent. Results and discussion: The effectiveness of the framework was verified in the gym simulation environment of the Baxter robot's reach-grasp-align task. The results show that in this framework, human experience knowledge has a significant impact on the guidance of learning, and this method can more quickly plan the coordinated trajectory of dual-arm for multi-step tasks.

AI-enhanced biomedical micro/nanorobots in microfluidics.

Human beings encompass sophisticated microcirculation and microenvironments, incorporating a broad spectrum of microfluidic systems that adopt fundamental roles in orchestrating physiological mechanisms. In vitro recapitulation of human microenvironments based on lab-on-a-chip technology represents a critical paradigm to better understand the intricate mechanisms. Moreover, the advent of micro/nanorobotics provides brand new perspectives and dynamic tools for elucidating the complex process in microfluidics. Currently, artificial intelligence (AI) has endowed micro/nanorobots (MNRs) with unprecedented benefits, such as material synthesis, optimal design, fabrication, and swarm behavior. Using advanced AI algorithms, the motion control, environment perception, and swarm intelligence of MNRs in microfluidics are significantly enhanced. This emerging interdisciplinary research trend holds great potential to propel biomedical research to the forefront and make valuable contributions to human health. Herein, we initially introduce the AI algorithms integral to the development of MNRs. We briefly revisit the components, designs, and fabrication techniques adopted by robots in microfluidics with an emphasis on the application of AI. Then, we review the latest research pertinent to AI-enhanced MNRs, focusing on their motion control, sensing abilities, and intricate collective behavior in microfluidics. Furthermore, we spotlight biomedical domains that are already witnessing or will undergo game-changing evolution based on AI-enhanced MNRs. Finally, we identify the current challenges that hinder the practical use of the pioneering interdisciplinary technology.

Novel miniature transendoscopic telerobotic system for endoscopic submucosal dissection (with videos).

BACKGROUND AND AIMS: The lack of tissue traction and instrument dexterity to allow for adequate visualization and effective dissection were the main issues in performing endoscopic submucosal dissection (ESD). Robot-assisted systems may provide advantages. In this study we developed a novel transendoscopic telerobotic system and evaluated its performance in ESD. METHODS: A miniature dual-arm robotic endoscopic assistant for minimally invasive surgery (DREAMS) was developed. The DREAMS system contained the current smallest robotic ESD instruments and was compatible with the commercially available dual-channel endoscope. After the system was established, a prospective randomized controlled study was conducted to validate the performance of the DREAMS-assisted ESD in terms of efficacy, safety, and workload by comparing it with the conventional technique. RESULTS: Two robotic instruments can achieve safe collaboration and provide sufficient visualization and efficient dissection during ESD. Forty ESDs in the stomach and esophagus of 8 pigs were completed by DREAMS-assisted ESD or conventional ESD. Submucosal dissection time was comparable between the 2 techniques, but DREAMS-assisted ESD demonstrated a significantly lower muscular injury rate (15% vs 50%, P = .018) and workload scores (22.30 vs 32.45, P < .001). In the subgroup analysis of esophageal ESD, DREAMS-assisted ESD showed significantly improved submucosal dissection time (6.45 vs 16.37 minutes, P = .002), muscular injury rate (25% vs 87.5%, P = .041), and workload (21.13 vs 40.63, P = .001). CONCLUSIONS: We developed a novel transendoscopic telerobotic system, named DREAMS. The safety profile and technical feasibility of ESD were significantly improved with the assistance of the DREAMS system, especially in the narrower esophageal lumen.

Technologies evolution in robot-assisted fracture reduction systems: a comprehensive review.

Background: Robot-assisted fracture reduction systems can potentially reduce the risk of infection and improve outcomes, leading to significant health and economic benefits. However, these systems are still in the laboratory stage and not yet ready for commercialization due to unresolved difficulties. While previous reviews have focused on individual technologies, system composition, and surgical stages, a comprehensive review is necessary to assist future scholars in selecting appropriate research directions for clinical use. Methods: A literature review using Google Scholar identified articles on robot-assisted fracture reduction systems. A comprehensive search yielded 17,800, 18,100, and 16,700 results for "fracture reduction," "computer-assisted orthopedic surgery," and "robot-assisted fracture reduction," respectively. Approximately 340 articles were selected, and 90 highly relevant articles were chosen for further reading after reviewing the abstracts. Results and Conclusion: Robot-assisted fracture reduction systems offer several benefits, including improved reduction accuracy, reduced physical work and radiation exposure, enhanced preoperative planning and intraoperative visualization, and shortened learning curve for skill acquisition. In the future, these systems will become integrated and practical, with automatic preoperative planning and high intraoperative safety.

Phoenixin-20 ameliorates Sevoflurane inhalation-induced post-operative cognitive dysfunction in rats via activation of the PKA/CREB signaling.

Post-operative cognitive dysfunction (POCD) is a common complication after surgery due to the usage of anesthetics, such as Sevoflurane, which severely impacts the life quality of patients. Currently, the pathogenesis of Sevoflurane-induced POCD has not been fully elucidated but is reportedly involved with oxidative stress (OS) injury and aggravated inflammation. Phoenixin-20 (PNX-20) is a PNX peptide consisting of 20 amino acids with promising inhibitory effects on OS and inflammation. Herein, we proposed to explore the potential protective function of PNX-20 on Sevoflurane inhalation-induced POCD in rats. Sprague-Dawley (SD) rats were treated with 100 ng/g PNX-20 for 7 days with or without pre-inhalation with 2.2% Sevoflurane. Markedly increased escape latency and decreased time in the target quadrant in the Morris water maze (MWM) test, and aggravated pathological changes and apoptosis in the hippocampus tissue were observed in Sevoflurane-treated rats, which were markedly attenuated by PNX-20. Furthermore, the aggravated inflammation and OS in the hippocampus observed in Sevoflurane-treated rats were notably abolished by PNX-20. Moreover, the brain-derived neurotrophic factor (BDNF), protein kinase A (PKA), and phospho-cAMP response element binding protein/cAMP response element binding protein (p-CREB/CREB) levels were markedly decreased in Sevoflurane-treated rats, which were memorably increased by PNX-20. Our results indicated that PNX-20 ameliorated Sevoflurane inhalation-induced POCD in rats via the activation of PKA/CREB signaling, which might supply a new treatment approach for POCD.

Peg-in-hole assembly skill imitation learning method based on ProMPs under task geometric representation.

Introduction: Behavioral Cloning (BC) is a common imitation learning method which utilizes neural networks to approximate the demonstration action samples for task manipulation skill learning. However, in the real world, the demonstration trajectories from human are often sparse and imperfect, which makes it challenging to comprehensively learn directly from the demonstration action samples. Therefore, in this paper, we proposes a streamlined imitation learning method under the terse geometric representation to take good advantage of the demonstration data, and then realize the manipulation skill learning of assembly tasks. Methods: We map the demonstration trajectories into the geometric feature space. Then we align the demonstration trajectories by Dynamic Time Warping (DTW) method to get the unified data sequence so we can segment them into several time stages. The Probability Movement Primitives (ProMPs) of the demonstration trajectories are then extracted, so we can generate a lot of task trajectories to be the global strategy action samples for training the neural networks. Notalby, we regard the current state of the assembly task as the via point of the ProMPs model to get the generated trajectories, while the time point of the via point is calculated according to the probability model of the different time stages. And we get the action of the current state according to the target position of the next time state. Finally, we train the neural network to obtain the global assembly strategy by Behavioral Cloning. Results: We applied the proposed method to the peg-in-hole assembly task in the simulation environment based on Pybullet + Gym to test its task skill learning performance. And the learned assembly strategy was also executed on a real robotic platform to verify the feasibility of the method further. Discussion: According to the result of the experiment, the proposed method achieves higher success rates compared to traditional imitation learning methods while exhibiting reasonable generalization capabilities. It shows that the ProMPs under geometric representation can help the BC method make better use of the demonstration trajectory and thus better learn the task skills.

Understanding Yb3+-sensitized photon avalanche in Pr3+ co-doped nanocrystals: modelling and optimization.

Among different upconversion processes where the emitted photon has higher energy than the one absorbed, photon avalanche (PA) is unique, because the luminescence intensity increases by 2-3 orders of magnitude in response to a tiny increase in excitation intensity. Since its discovery in 1979, PA has been observed in bulk materials but until recently, obtaining it at the nanoscale has been a significant challenge. In the present work, the PA phenomenon in ß-NaYF4 colloidal nanocrystals co-doped with Pr3+ and Yb3+ ions was successfully observed at 482 nm (3P0 → 3H4) and 607 nm (3P0 → 3H6) under excitation at 852 nm. The impact of Pr3+ ion concentration and pump power dependence on PA behavior was investigated, i.e. PA non-linearity slopes of luminescence intensity curves as a function of pump power density as well as PA thresholds. The highest slopes, namely 8.6 and 9.0, and the smallest thresholds equal to 286 kW cm-2 and 281 kW cm-2, observed for emission bands at 607 nm and 482 nm, respectively, were obtained for NaYF4:0.5%Pr3+,15%Yb3+@NaYF4 colloidal nanocrystals. Besides experimental research, simulations of PA behavior in Pr3+, Yb3+ co-doped materials were performed based on differential rate equations describing the phenomena that contribute to the existence of PA. The influence of different processes leading to PA, e.g. the rates of nonradiative and radiative transitions as well as energy transfers, on PA performance was simulated aiming to understand their roles in this complex sensitized system.

Combined three dimensional locomotion and deformation of functional ferrofluidic robots.

Magnetic microrobots possess remarkable potential for targeted applications in the medical field, primarily due to their non-invasive, controllable properties. These unique qualities have garnered increased attention and fascination among researchers. However, these robotic systems do face challenges such as limited deformation capabilities and difficulties navigating confined spaces. Recently, researchers have turned their attention towards magnetic droplet robots, which are notable for their superior deformability, controllability, and potential for a range of applications such as automated virus detection and targeted drug delivery. Despite these advantages, the majority of current research is constrained to two-dimensional deformation and motion, thereby limiting their broader functionality. In response to these limitations, this study proposes innovative strategies for controlling deformation and achieving a three-dimensional (3D) trajectory in ferrofluidic robots. These strategies leverage a custom-designed eight-axis electromagnetic coil and a sliding mode controller. The implementation of these methods exhibits the potential of ferrofluidic robots in diverse applications, including microfluidic pump systems, 3D micromanipulation, and selective vascular occlusion. In essence, this study aims to broaden the capabilities of ferrofluidic robots, thereby enhancing their applicability across a multitude of fields such as medicine, micromanipulation, bioengineering, and more by maximizing the potential of these intricate robotic systems.

Er3+-Sensitized Upconversion/Down-Shifting Luminescence in Metal-Organic Frameworks.

Photoluminescent metal-organic frameworks (MOFs) are used as optical materials with excellent properties, of which the lanthanide-doped MOFs are able to emit in a broad region from visible to near-infrared due to their unique 4f-orbital electronic structure. Herein, Er3+ and Y3+ ions are selected as the metal centers of the MOFs and Er3+ is used as a sensitizer to absorb 980 nm excitation light. At the same time, Er3+ ions also act as activators that emit upconverting visible light and down-shifting near-infrared light. In addition, Tm3+, Ho3+, and Eu3+ ions were individually doped into the Er3+-doped MOFs to investigate the variation of energy-transfer paths in the presence of different lanthanide activators. Finally, the pathway of energy transfer in these Er3+-sensitized luminescent-MOFs was summarized. This work provides new insights for further development of both upconversion and down-shifting luminescence of MOFs.

Learning adaptive reaching and pushing skills using contact information.

In this paper, we propose a deep reinforcement learning-based framework that enables adaptive and continuous control of a robot to push unseen objects from random positions to the target position. Our approach takes into account contact information in the design of the reward function, resulting in improved success rates, generalization for unseen objects, and task efficiency compared to policies that do not consider contact information. Through reinforcement learning using only one object in simulation, we obtain a learned policy for manipulating a single object, which demonstrates good generalization when applied to the task of pushing unseen objects. Finally, we validate the effectiveness of our approach in real-world scenarios.

Ultraviolet B radiation-induced JPH203-loaded keratinocyte extracellular vesicles exert etiological interventions for psoriasis therapy.

Psoriasis is a multifactorial immuno-inflammatory skin disease, characterized by keratinocyte hyperproliferation and aberrant immune activation. Although the pathogenesis is complex, the interactions among inflammation, Th17-mediated immune activation, and keratinocyte hyperplasia are considered to play a crucial role in the occurrence and development of psoriasis. Therefore, pharmacological interventions on the "inflammation-Th17-keratinocyte" vicious cycle may be a potential strategy for psoriasis treatment. In this study, JPH203 (a specific inhibitor of LAT1, which engulfs leucine to activate mTOR signaling)-loaded, ultraviolet B (UVB) radiation-induced, keratinocyte-derived extracellular vesicles (J@EV) were prepared for psoriasis therapy. The EVs led to increased interleukin 1 receptor antagonist (IL-1RA) content due to UVB irradiation, therefore not only acting as a carrier for JPH203 but also functioning through inhibiting the IL-1-mediated inflammation cascade. J@EV effectively restrained the proliferation of inflamed keratinocytes via suppressing mTOR-signaling and NF-κB pathway in vitro. In an imiquimod-induced psoriatic model, J@EV significantly ameliorated the related symptoms as well as suppressed the over-activated immune reaction, evidenced by the decreased keratinocyte hyperplasia, Th17 expansion, and IL17 release. This study shows that J@EV exerts therapeutic efficacy for psoriasis by suppressing LAT1-mTOR involved keratinocyte hyperproliferation and Th17 expansion, as well as inhibiting IL-1-NF-κB mediated inflammation, representing a novel and promising strategy for psoriasis therapy.

A dynamic parallel image acquisition method for slide scanning process.

Automated microscope systems have played an important role in the screening of numerous diseases. However, it is a very time-consuming process to continuously acquire images under the high magnification objective lens. This paper proposes a dynamic parallel image acquisition method, which can greatly improve image acquisition speed. Due to the relative motion between the x-y stage and the camera, some of the captured images have motion blur To this end, we also designed a motor variable speed motion curve to ensure the quality of the collected images. The experimental results show that the traditional image scanning mode needs 47.3 ms to obtain continuous microscopic images, while the dynamic parallel image acquisition method only needs 25.4 ms, which improves the acquisition speed without affecting the clarity of the acquired images.

Upconversion Luminescence in Mononuclear Yb/Sm Co-crystal Assemblies at Room Temperature.

Metal-based upconversion luminescence transforming high-energy photons into low-energy photons is an attractive anti-Stokes shift process for fundamental research and promising applications. In this work, we developed the upconversion luminescence in co-crystal assemblies consisting of discrete mononuclear Yb and Sm complexes. The characteristic visible emissions of Sm3+ were observed under the excitation of absorption band of Yb3+ at 980 nm. A series of co-crystal assemblies were investigated based on mononuclear Yb and Sm complexes, and the strongest luminescence was obtained when the molar concentration between Yb3+ and Sm3+ is equivalent. The crystal structure was fully characterized by the single crystal X-ray diffraction and upconverting energy transfer mechanisms were verified as cooperative sensitization upconversion and energy transfer upconversion. This is the first example of Sm3+ -based upconverting luminescence in discrete lanthanide complexes which present as co-crystal assemblies at room temperature.

Gait parameter fitting and adaptive enhancement based on cerebral blood oxygen information.

Accurate recognition of patients' movement intentions and real-time adjustments are crucial in rehabilitation exoskeleton robots. However, some patients are unable to utilize electromyography (EMG) signals for this purpose due to poor or missing signals in their lower limbs. In order to address this issue, we propose a novel method that fits gait parameters using cerebral blood oxygen signals. Two types of walking experiments were conducted to collect brain blood oxygen signals and gait parameters from volunteers. Time domain, frequency domain, and spatial domain features were extracted from brain hemoglobin. The AutoEncoder-Decoder method is used for feature dimension reduction. A regression model based on the long short-term memory (LSTM) model was established to fit the gait parameters and perform incremental learning for new individual data. Cross-validation was performed on the model to enhance individual adaptivity and reduce the need for individual pre-training. The coefficient of determination (R2) for the gait parameter fit was 71.544%, with a mean square error (RMSE) of less than 3.321%. Following adaptive enhancement, the coefficient of R2 increased by 6.985%, while the RMSE decreased by 0.303%. These preliminary results indicate the feasibility of fitting gait parameters using cerebral blood oxygen information. Our research offers a new perspective on assisted locomotion control for patients who lack effective myoelectricity, thereby expanding the clinical application of rehabilitation exoskeleton robots. This work establishes a foundation for promoting the application of Brain-Computer Interface (BCI) technology in the field of sports rehabilitation.

Adaptive visual-tactile fusion recognition for robotic operation of multi-material system.

The use of robots in various industries is evolving from mechanization to intelligence and precision. These systems often comprise parts made of different materials and thus require accurate and comprehensive target identification. While humans perceive the world through a highly diverse perceptual system and can rapidly identify deformable objects through vision and touch to prevent slipping or excessive deformation during grasping, robot recognition technology mainly relies on visual sensors, which lack critical information such as object material, leading to incomplete cognition. Therefore, multimodal information fusion is believed to be key to the development of robot recognition. Firstly, a method of converting tactile sequences to images is proposed to deal with the obstacles of information exchange between different modalities for vision and touch, which overcomes the problems of the noise and instability of tactile data. Subsequently, a visual-tactile fusion network framework based on an adaptive dropout algorithm is constructed, together with an optimal joint mechanism between visual information and tactile information established, to solve the problem of mutual exclusion or unbalanced fusion in traditional fusion methods. Finally, experiments show that the proposed method effectively improves robot recognition ability, and the classification accuracy is as high as 99.3%.