Avian eye-inspired artificial vision takes a step forward.

Bioinspiration from avian eyes allows development of artificial vision systems with foveated and multispectral imaging.

Microsaccade-inspired event camera for robotics.

Neuromorphic vision sensors or event cameras have made the visual perception of extremely low reaction time possible, opening new avenues for high-dynamic robotics applications. These event cameras' output is dependent on both motion and texture. However, the event camera fails to capture object edges that are parallel to the camera motion. This is a problem intrinsic to the sensor and therefore challenging to solve algorithmically. Human vision deals with perceptual fading using the active mechanism of small involuntary eye movements, the most prominent ones called microsaccades. By moving the eyes constantly and slightly during fixation, microsaccades can substantially maintain texture stability and persistence. Inspired by microsaccades, we designed an event-based perception system capable of simultaneously maintaining low reaction time and stable texture. In this design, a rotating wedge prism was mounted in front of the aperture of an event camera to redirect light and trigger events. The geometrical optics of the rotating wedge prism allows for algorithmic compensation of the additional rotational motion, resulting in a stable texture appearance and high informational output independent of external motion. The hardware device and software solution are integrated into a system, which we call artificial microsaccade-enhanced event camera (AMI-EV). Benchmark comparisons validated the superior data quality of AMI-EV recordings in scenarios where both standard cameras and event cameras fail to deliver. Various real-world experiments demonstrated the potential of the system to facilitate robotics perception both for low-level and high-level vision tasks.

Avian eye-inspired perovskite artificial vision system for foveated and multispectral imaging.

Avian eyes have deep central foveae as a result of extensive evolution. Deep foveae efficiently refract incident light, creating a magnified image of the target object and making it easier to track object motion. These features are essential for detecting and tracking remote objects in dynamic environments. Furthermore, avian eyes respond to a wide spectrum of light, including visible and ultraviolet light, allowing them to efficiently distinguish the target object from complex backgrounds. Despite notable advances in artificial vision systems that mimic animal vision, the exceptional object detection and targeting capabilities of avian eyes via foveated and multispectral imaging remain underexplored. Here, we present an artificial vision system that capitalizes on these aspects of avian vision. We introduce an artificial fovea and vertically stacked perovskite photodetector arrays whose designs were optimized by theoretical simulations for the demonstration of foveated and multispectral imaging. The artificial vision system successfully identifies colored and mixed-color objects and detects remote objects through foveated imaging. The potential for use in uncrewed aerial vehicles that need to detect, track, and recognize distant targets in dynamic environments is also discussed. Our avian eye-inspired perovskite artificial vision system marks a notable advance in bioinspired artificial visions.

Adaptivity of a leaf-inspired wind energy harvester with respect to wind speed and direction.

Environmental wind is a random phenomenon in both speed and direction, though it can be forecasted to some extent. An example of that is a gust which is an abrupt, but short-time change in wind speed and direction. Being a free and clean source for small-scale energy scavenging, attraction of wind is rapidly growing in the world of energy harvesters. In this paper, a leaf-like flapping wind energy harvester is introduced as the base structure in which a short-span airfoil is attached to the free end of a double-deck cantilever beam. A flap mechanism inspired by scales on sharks' skin and a tail mechanism inspired by birds' horizontal tail are proposed for integration to the base harvester to make it adaptive with respect to wind speed and direction, respectively. The use of the flap mechanism increases the leaf flapping frequency by +2.1 to +11.5 Hz at wind speeds of 1.5 to 6.0 m s-1. Therefore, since the output power of a vibrational harvester is a function of vibration frequency, a figure of merit or an efficiency parameter related to the output power will increase, as well. On the other hand, if there is a misalignment between the harvester's heading and wind direction due to change of the latter one, the harvesting performance deteriorates. Although the base harvester can realign in certain ranges of sideslip angle at each wind speed, when the tail mechanism is integrated into that, it broadens the range of realignable sideslip angles at all the investigated wind speeds by up to 80∘.

Legged robots beyond bioinspiration.

Advances in engineering enable wheeled-legged hybrid locomotion, an achievement not feasible in biological systems.

Design of an actuator with bionic claw hook-suction cup hybrid structure for soft robot.

To improve the adaptability of soft robots to the environment and achieve reliable attachment on various surfaces such as smooth and rough, this study draws inspiration from the collaborative attachment strategy of insects, cats, and other biological claw hooks and foot pads, and designs an actuator with a bionic claw hook-suction cup hybrid structure. The rigid biomimetic pop-up claw hook linkage mechanism is combined with a flexible suction cup of a 'foot pad' to achieve a synergistic adhesion effect between claw hook locking and suction cup adhesion through the deformation control of a soft pneumatic actuator. A pop-up claw hook linkage mechanism based on the principle of cat claw movement was designed, and the attachment mechanism of the biological claw hooks and footpads was analysed. An artificial muscle-spring-reinforced flexible pneumatic actuator (SRFPA) was developed and a kinematic model of the SRFPA was established and analysed using Abaqus. Finally, a prototype of the hybrid actuator was fabricated. The kinematic and mechanical performances of the SRFPA and entire actuator were characterised, and the attachment performance of the hybrid actuator to smooth and rough surfaces was tested. The results indicate that the proposed biomimetic claw hook-suction cup hybrid structure actuator is effective for various types of surface adhesion, object grasping, and robot walking. This study provides new insights for the design of highly adaptable robots and biomimetic attachment devices.

Dynamics of seaweed-inspired piezoelectric plates for energy harvesting from oscillatory cross flow.

Inspired by the vibrations of aquatic plants such as seaweed in the unsteady flow fields generated by free-surface waves, we investigate a novel device based on piezoelectric plates to harvest energy from oscillatory cross flows. Towards this end, numerical studies are conducted using a flow-structure-electric interaction model to understand the underlying physical mechanisms involved in the dynamics and energy harvesting performance of one or a pair of piezoelectric plates in an oscillatory cross flow. In a single-plate configuration, both periodic and irregular responses have been observed depending on parameters such as normalized plate stiffness and Keulegan-Carpenter number. Large power harvesting is achieved with the excitation of natural modes. Besides, when the time scale of the motion and the intrinsic time scale of the circuit are close to each other the power extraction is enhanced. In a two-plate configuration with tandem formation, the hydrodynamic interaction between the two plates can induce irregularity in the response. In terms of energy harvesting, two counteracting mechanisms have been identified, shielding and energy recovery. The shielding effect reduces plate motion and energy harvesting, whereas with the energy recovery effect one plate is able to recovery energy from the wake of another for performance enhancement. The competition between these mechanisms leads to constructive or destructive interactions between the two plates. These results suggest that for better performance the system should be excited at its natural period, which should be close to the intrinsic time scale of the circuit. Moreover, using a pair of plates in a tandem formation can further improve the energy harvesting capacity when conditions for constructive interaction are satisfied.

A toe-inspired rigid-flexible coupling wheel design method for improving the terrain adaptability of a sewer robot.

The human toe, characterized by its rigid-flexible structure comprising hard bones and flexible joints, facilitates adaptive and stable movement across varied terrains. In this paper, we utilized a motion capture system to study the adaptive adjustments of toe joints when encountering obstacles. Inspired by the mechanics of toe joints, we proposed a novel design method for a rigid-flexible coupled wheel. The wheel comprises multiple elements: a rigid skeleton, supporting toes, connecting shafts, torsion springs, soft tendons, and damping pads. The torsion springs connect the rigid frame to the supporting toes, enabling them to adapt to uneven terrains and pipes with different diameters. The design was validated through kinematic and dynamic modeling, rigid-flexible coupled dynamics simulation, and stress analysis. Different stiffness coefficients of torsion springs were compared for optimal wheel design. Then, the wheel was applied to a sewer robot, and its performance was evaluated and compared with a pneumatic rubber tire in various experiments, including movement on flat surfaces, overcoming small obstacles, adaptability tests in different terrains, and active driving force tests in dry and wet pipelines. The results prove that the designed wheel showed better stability and anti-slip properties than conventional tires, making it suitable for diverse applications such as pipeline robots, desert vehicles, and lunar rovers.

Nodes for modes: nodal honeycomb metamaterial enables a soft robot with multimodal locomotion.

Soft-bodied animals, such as worms and snakes, use many muscles in different ways to traverse unstructured environments and inspire tools for accessing confined spaces. They demonstrate versatility of locomotion which is essential for adaptation to changing terrain conditions. However, replicating such versatility in untethered soft-bodied robots with multimodal locomotion capabilities have been challenging due to complex fabrication processes and limitations of soft body structures to accommodate hardware such as actuators, batteries and circuit boards. Here, we present MetaCrawler, a 3D printed metamaterial soft robot designed for multimodal and omnidirectional locomotion. Our design approach facilitated an easy fabrication process through a discrete assembly of a modular nodal honeycomb lattice with soft and hard components. A crucial benefit of the nodal honeycomb architecture is the ability of its hard components, nodes, to accommodate a distributed actuation system, comprising servomotors, control circuits, and batteries. Enabled by this distributed actuation, MetaCrawler achieves five locomotion modes: peristalsis, sidewinding, sideways translation, turn-in-place, and anguilliform. Demonstrations showcase MetaCrawler's adaptability in confined channel navigation, vertical traversing, and maze exploration. This soft robotic system holds the potential to offer easy-to-fabricate and accessible solutions for multimodal locomotion in applications such as search and rescue, pipeline inspection, and space missions.

Visualized in-sensor computing.

In artificial nervous systems, conductivity changes indicate synaptic weight updates, but they provide limited information compared to living organisms. We present the pioneering design and production of an electrochromic neuromorphic transistor employing color updates to represent synaptic weight for in-sensor computing. Here, we engineer a specialized mechanism for adaptively regulating ion doping through an ion-exchange membrane, enabling precise control over color-coded synaptic weight, an unprecedented achievement. The electrochromic neuromorphic transistor not only enhances electrochromatic capabilities for hardware coding but also establishes a visualized pattern-recognition network. Integrating the electrochromic neuromorphic transistor with an artificial whisker, we simulate a bionic reflex system inspired by the longicorn beetle, achieving real-time visualization of signal flow within the reflex arc in response to environmental stimuli. This research holds promise in extending the biomimetic coding paradigm and advancing the development of bio-hybrid interfaces, particularly in incorporating color-based expressions.

A Retina-Inspired Optoelectronic Synapse Using Quantum Dots for Neuromorphic Photostimulation of Neurons.

Neuromorphic electronics, inspired by the functions of neurons, have the potential to enable biomimetic communication with cells. Such systems require operation in aqueous environments, generation of sufficient levels of ionic currents for neurostimulation, and plasticity. However, their implementation requires a combination of separate devices, such as sensors, organic synaptic transistors, and stimulation electrodes. Here, a compact neuromorphic synapse that combines photodetection, memory, and neurostimulation functionalities all-in-one is presented. The artificial photoreception is facilitated by a photovoltaic device based on cell-interfacing InP/ZnS quantum dots, which induces photo-faradaic charge-transfer mediated plasticity. The device sends excitatory post-synaptic currents exhibiting paired-pulse facilitation and post-tetanic potentiation to the hippocampal neurons via the biohybrid synapse. The electrophysiological recordings indicate modulation of the probability of action potential firing due to biomimetic temporal summation of excitatory post-synaptic currents. The results pave the way for the development of novel bioinspired neuroprosthetics and soft robotics and highlight the potential of quantum dots for achieving versatile neuromorphic functionality in aqueous environments.

A Retina-Inspired Optoelectronic Synapse Using Quantum Dots for Neuromorphic Photostimulation of Neurons.

Neuromorphic electronics, inspired by the functions of neurons, have the potential to enable biomimetic communication with cells. Such systems require operation in aqueous environments, generation of sufficient levels of ionic currents for neurostimulation, and plasticity. However, their implementation requires a combination of separate devices, such as sensors, organic synaptic transistors, and stimulation electrodes. Here, a compact neuromorphic synapse that combines photodetection, memory, and neurostimulation functionalities all-in-one is presented. The artificial photoreception is facilitated by a photovoltaic device based on cell-interfacing InP/ZnS quantum dots, which induces photo-faradaic charge-transfer mediated plasticity. The device sends excitatory post-synaptic currents exhibiting paired-pulse facilitation and post-tetanic potentiation to the hippocampal neurons via the biohybrid synapse. The electrophysiological recordings indicate modulation of the probability of action potential firing due to biomimetic temporal summation of excitatory post-synaptic currents. These results pave the way for the development of novel bioinspired neuroprosthetics and soft robotics, and highlight the potential of quantum dots for achieving versatile neuromorphic functionality in aqueous environments.

Multi-environment robotic transitions through adaptive morphogenesis.

The current proliferation of mobile robots spans ecological monitoring, warehouse management and extreme environment exploration, to an individual consumer's home1-4. This expanding frontier of applications requires robots to transit multiple environments, a substantial challenge that traditional robot design strategies have not effectively addressed5,6. For example, biomimetic design-copying an animal's morphology, propulsion mechanism and gait-constitutes one approach, but it loses the benefits of engineered materials and mechanisms that can be exploited to surpass animal performance7,8. Other approaches add a unique propulsive mechanism for each environment to the same robot body, which can result in energy-inefficient designs9-11. Overall, predominant robot design strategies favour immutable structures and behaviours, resulting in systems incapable of specializing across environments12,13. Here, to achieve specialized multi-environment locomotion through terrestrial, aquatic and the in-between transition zones, we implemented 'adaptive morphogenesis', a design strategy in which adaptive robot morphology and behaviours are realized through unified structural and actuation systems. Taking inspiration from terrestrial and aquatic turtles, we built a robot that fuses traditional rigid components and soft materials to radically augment the shape of its limbs and shift its gaits for multi-environment locomotion. The interplay of gait, limb shape and the environmental medium revealed vital parameters that govern the robot's cost of transport. The results attest that adaptive morphogenesis is a powerful method to enhance the efficiency of mobile robots encountering unstructured, changing environments.

Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors.

Mixed-signal analog/digital circuits emulate spiking neurons and synapses with extremely high energy efficiency, an approach known as "neuromorphic engineering". However, analog circuits are sensitive to process-induced variation among transistors in a chip ("device mismatch"). For neuromorphic implementation of Spiking Neural Networks (SNNs), mismatch causes parameter variation between identically-configured neurons and synapses. Each chip exhibits a different distribution of neural parameters, causing deployed networks to respond differently between chips. Current solutions to mitigate mismatch based on per-chip calibration or on-chip learning entail increased design complexity, area and cost, making deployment of neuromorphic devices expensive and difficult. Here we present a supervised learning approach that produces SNNs with high robustness to mismatch and other common sources of noise. Our method trains SNNs to perform temporal classification tasks by mimicking a pre-trained dynamical system, using a local learning rule from non-linear control theory. We demonstrate our method on two tasks requiring temporal memory, and measure the robustness of our approach to several forms of noise and mismatch. We show that our approach is more robust than common alternatives for training SNNs. Our method provides robust deployment of pre-trained networks on mixed-signal neuromorphic hardware, without requiring per-device training or calibration.

Low-cost, thermoplastic micro-lens array with a carbon black light screen for bio-mimetic vision.

Micro-lens array is a great example of bio-mimetic technology which was inspired by compound eyes found in insects and is used in lasers, optical communication, and 3D imaging. In this study, a micro-lens array was fabricated from cyclic olefin copolymer using a cost-effective method: compression molding and thermal reflow. Also, a light screen was installed between lenses to reduce the optical interference for clearer individual images. Cyclic olefin copolymer-based micro-lens array showed good optical results under a standard optical microscope. By placing the fabricated micro-lens array directly on an image sensor, it was observed that the light screen shows significant improvement in image quality. Also, the point spread function was analyzed to confirm the optical performance and the effectiveness of the micro-lens array with the light screen installed.

Fabrication of a light screen-aperture integrated flexible thin film micro-lens array for a biomimetic superposition compound eye.

Micro-lens array, an artificial compound eye vision system, provides a wide field of view and multi-perspective view. However, it has not been adopted as a computer vision application due to its limited visible range and high optical interference. In this research, a novel fabrication method for the flexible polydimethylsiloxane micro-lens array with a polytetrafluoroethylene light screen-aperture integrated layer was established by the simple protrusion method. The integrated layer provided longer visible range by one meter while maintaining the wide field-of-view of 100 °. The resulting images were used for obtaining depth information of a target as an example and for analyzing the rectangular and hexagonal arrangements of the micro-lenses for the future applications. With the improved visual range, wide field-of-view and flexibility, the fabricated micro-lens array can be applied to the small and curved CMOS image sensors in the future.

Mixed-Halide Perovskite Film-Based Neuromorphic Phototransistors for Mimicking Experience-History-Dependent Sensory Adaptation.

Sensory adaptation is an essential function for humans to live on the earth. Herein, a hybrid synaptic phototransistor based on the mixed-halide perovskite/organic semiconductor film is reported. This hybrid phototransistor achieves photosensitive performance including a high photoresponsivity over 4 × 103 A/W and an excellent specific detectivity of 2.8 × 1016 Jones. Due to the photoinduced halide-ion segregation of the mixed-halide perovskites and their slow recovery properties, the experience-history-dependent sensory adaptation behavior can be mimicked. Moreover, the light pulse width, intensity, light wavelength, and gate bias can be used to regulate the adaptation processes to improve its adaptability and perceptibility in different environments. The CsPbBrxI3-x/organic semiconductor hybrid films produced by spin coating are beneficial to large-scale fabrication. This study fabricates a novel solution-processable light-stimulated synapse based on inorganic perovskites for mimicking the human sensory adaptation that makes it possible to approach artificial neural sensory systems.

Swarming Aqua Sperm Micromotors for Active Bacterial Biofilms Removal in Confined Spaces.

Microscale self-propelled robots show great promise in the biomedical field and are the focus of many researchers. These tiny devices, which move and navigate by themselves, are typically based on inorganic microstructures that are not biodegradable and potentially toxic, often using toxic fuels or elaborate external energy sources, which limits their real-world applications. One potential solution to these issues is to go back to nature. Here, the authors use high-speed Aqua Sperm micromotors obtained from North African catfish (Clarias gariepinus, B. 1822) to destroy bacterial biofilm. These Aqua Sperm micromotors use water-induced dynein ATPase catalyzed adenosine triphosphate (ATP) degradation as biocompatible fuel to trigger their fast speed and snake-like undulatory locomotion that facilitate biofilm destruction in less than one minute. This efficient biofilm destruction is due to the ultra-fast velocity as well as the head size of Aqua Sperm micromotors being similar to bacteria, which facilitates their entry to and navigation within the biofilm matrix. In addition, the authors demonstrate the real-world application of Aqua Sperm micromotors by destroying biofilms that had colonized medical and laboratory tubing. The implemented system extends the biomedical application of Aqua Sperm micromotors to include hybrid robots for fertilization or cargo tasks.

An Optogenetics-Inspired Flexible van der Waals Optoelectronic Synapse and its Application to a Convolutional Neural Network.

Optogenetics refers to a technique that uses light to modulate neuronal activity with a high spatiotemporal resolution, which enables the manipulation of learning and memory functions in the human brain. This strategy of controlling neuronal activity using light can be applied for the development of intelligent systems, including neuromorphic and in-memory computing systems. Herein, a flexible van der Waals (vdW) optoelectronic synapse is reported, which is a core component of optogenetics-inspired intelligent systems. This synapse is fabricated on 2D vdW layered rhenium disulfide (ReS2 ) that features an inherent photosensitive memory nature derived from the persistent photoconductivity (PPC) effect, successfully mimicking the dynamics of biological synapses. Based on first-principles calculations, the PPC effect is identified to originate from sulfur vacancies in ReS2 that have an inherent tendency to form shallow defect states near the conduction band edges and under optical excitation lead to large lattice relaxation. Finally, the feasibility of applying the synapses in optogenetics-inspired intelligent systems is demonstrated via training and inference tasks for the CIFAR-10 dataset using a convolutional neural network composed of vdW optoelectronic synapse devices.

Biomimetic artificial cells to model the effect of membrane asymmetry on chemoresistance.

We present a microfluidic platform that enables the formation of bespoke asymmetric droplet interface bilayers (DIBs) as artificial cell models from naturally-derived lipids. We use them to perform pharmacokinetic assays to quantify how lipid asymmetry affects the permeability of the chemotherapy drug doxorubicin. Previous attempts to model bilayer asymmetry with DIBs have relied on the use of synthetic lipids to achieve asymmetry. Use of natural lipids serves to increase the biomimetic nature of these artificial cells, showcasing the next step towards forming a true artificial cell membrane in vitro. Here we use our microfluidic platform to form biomimetic, asymmetric and symmetric DIBs, with their asymmetry quantified through their life-mimicking degree of curvature. We subsequently examine permeability of these membranes to doxorubicin, and reveal measurable differences in its pharmacokinetics induced by membrane asymmetry, highlighting another factor that potentially contributes to chemoresistance in some forms of cancer.