Stereoscopic artificial compound eyes for spatiotemporal perception in three-dimensional space.

Arthropods' eyes are effective biological vision systems for object tracking and wide field of view because of their structural uniqueness; however, unlike mammalian eyes, they can hardly acquire the depth information of a static object because of their monocular cues. Therefore, most arthropods rely on motion parallax to track the object in three-dimensional (3D) space. Uniquely, the praying mantis (Mantodea) uses both compound structured eyes and a form of stereopsis and is capable of achieving object recognition in 3D space. Here, by mimicking the vision system of the praying mantis using stereoscopically coupled artificial compound eyes, we demonstrated spatiotemporal object sensing and tracking in 3D space with a wide field of view. Furthermore, to achieve a fast response with minimal latency, data storage/transportation, and power consumption, we processed the visual information at the edge of the system using a synaptic device and a federated split learning algorithm. The designed and fabricated stereoscopic artificial compound eye provides energy-efficient and accurate spatiotemporal object sensing and optical flow tracking. It exhibits a root mean square error of 0.3 centimeter, consuming only approximately 4 millijoules for sensing and tracking. These results are more than 400 times lower than conventional complementary metal-oxide semiconductor-based imaging systems. Our biomimetic imager shows the potential of integrating nature's unique design using hardware and software codesigned technology toward capabilities of edge computing and sensing.

Network of artificial olfactory receptors for spatiotemporal monitoring of toxic gas.

Excessive human exposure to toxic gases can lead to chronic lung and cardiovascular diseases. Thus, precise in situ monitoring of toxic gases in the atmosphere is crucial. Here, we present an artificial olfactory system for spatiotemporal recognition of NO2 gas flow by integrating a network of chemical receptors with a near-sensor computing. The artificial olfactory receptor features nano-islands of metal-based catalysts that cover the graphene surface on the heterostructure of an AlGaN/GaN two-dimensional electron gas (2DEG) channel. Catalytically dissociated NO2 molecules bind to graphene, thereby modulating the conductivity of the 2DEG channel. For the energy/resource-efficient gas flow monitoring, trust-region Bayesian optimization algorithm allocates many sensors optimally in a complex space. Integrated artificial neural networks on a compact microprocessor with a network of sensors provide in situ gas flow predictions. This system enhances protective measures against toxic environments through spatiotemporal monitoring of toxic gases.