Three-Dimensional Encoding Approach for Wearable Tactile Communication Devices.

In this work, we presented a novel encoding method for tactile communication. This approach was based on several tactile sensory characteristics of human skin at different body parts, such as the head and neck, where location coordinates in the three-dimensional (3D) space were clearly mapped in the brain cortex, and gentle stimulations of vibrational touching with varied strengths were received instantly and precisely. For certain applications, such as playing cards or navigating walk paths for blinded people, we demonstrated specifically designed code lists with different patterns of tactile points in varied temporal sequences. By optimizing these codes, we achieved excellent efficiency and accuracy in our test experiments. As this method matched well with the natural habits of tactile sensory, it was easy to learn in a short training period. The results of the present work have offered a silent, efficient and accurate communication solution for visually impaired people or other users.

Locally Oriented Scene Complexity Analysis Real-Time Ocean Ship Detection from Optical Remote Sensing Images.

Due to strong ocean waves, broken clouds, and extensive cloud cover interferences, ocean ship detection performs poorly when using optical remote sensing images. In addition, it is a challenge to detect small ships on medium resolution optical remote sensing that cover a large area. In this paper, in order to balance the requirements of real-time processing and high accuracy detection, we proposed a novel ship detection framework based on locally oriented scene complexity analysis. First, the proposed method can separate a full image into two types of local scenes (i.e., simple or complex local scenes). Next, simple local scenes would utilize the fast saliency model (FSM) to rapidly complete candidate extraction, and for complex local scenes, the ship feature clustering model (SFCM) will be applied to achieve refined detection against severe background interferences. The FSM considers a fusion enhancement image as an input of the pulse response analysis in the frequency domain to achieve rapid ship detection in simple local scenes. Next, the SFCM builds the descriptive model of the ship feature clustering algorithm to ensure the detection performance on complex local scenes. Extensive experiments on SPOT-5 and GF-2 ocean optical remote sensing images show that the proposed ship detection framework has better performance than the state-of-the-art methods, and it addresses the tricky problem of real-time ocean ship detection under strong waves, broken clouds, extensive cloud cover, and ship fleet interferences. Finally, the proposed ocean ship detection framework is demonstrated on an onboard processing hardware.

Extraordinary optical transmission through a rectangular hole filled with extreme uniaxial metamaterials.

This Letter presents a theory of extraordinary optical transmission (EOT) through a rectangular hole filled with the extreme uniaxial metamaterials with infinite longitudinal components of permittivity (ϵz) and permeability (µz). We demonstrate theoretically and numerically that a number of high-order transverse electromagnetic (TEM) modes can be supported by such a structure, and that, more interestingly, their normalized transmittance can be remarkably enhanced due to the Fabry-Perot resonance effect. A set of illustrative examples has been provided to demonstrate that such an EOT property could be explored for the purpose of subwavelength-resolution imaging.

Far-field imaging beyond diffraction limit using single sensor in combination with a resonant aperture.

Far-field imaging beyond the diffraction limit is a long sought-after goal in various imaging applications, which requires usually mechanical scanning or an array of antennas. Here, we propose to solve this challenging problem using a single sensor in combination with a spatio-temporal resonant aperture antenna. We theoretically and numerically demonstrate that such resonant aperture antenna is capable of converting part evanescent waves into propagating waves and delivering them to far fields. The proposed imaging concept provides the unique ability to achieve super resolution for real-time data when illuminated by broadband electromagnetic waves, without the harsh requirements such as near- field scanning, mechanical scanning, or antenna arrays. We expect the imaging methodology to make breakthroughs in super-resolution imaging in microwave, terahertz, optical, and ultrasound regimes.

Computational superoscillation imaging beyond the Rayleigh limit from far-field measurements.

Far-field imaging beyond the Rayleigh limit is one of the most important challenges in optics, microwave, and ultrasonics. We propose a novel sparsity-promoted super-oscillation imaging scheme for reconstructing more universal objects in subwavelength scales, which solves a weighted optimization problem constrained by lp-norm-based sparsity regularization (0≤p≤1). We demonstrate numerically that the proposed imaging technique improves the resolution related to existing approaches remarkably for the case of very high signal-to-noise ratio (SNR), including the traditional super-oscillation imaging and sparsity-based super-resolution imaging. The standard superoscillation based super-resolution imaging approach can be regarded as the first-iteration solution of the proposed scheme. Numerical results for one- and two-dimensional super-resolution imaging are presented for validation.

Feasibility of resonant metalens for the subwavelength imaging using a single sensor in the far field.

This paper investigates the feasibility of the resonant metalens for the imaging beyond the diffraction limit using a single sensor in the far-field. It is shown that the resonant metalens can be related to the super-resonance phenomenon. We demonstrate that the super-resonance supports the enhancement of the information capacity of an imaging system, which is responsible for the subwavelength imaging of the probed objects by using a single sensor in combination with a broadband illumination. Such imaging concept has its unique advantage of producing real-time data when an object is illuminated by broadband waves, without the harsh requirements such as near-field scanning, mechanical scanning, or antenna arrays. The proposed method is expected to find its applications in nanolithography, detection, sensing, and subwavelength imaging in the near future.

Semantic-Electromagnetic Inversion With Pretrained Multimodal Generative Model.

Across diverse domains of science and technology, electromagnetic (EM) inversion problems benefit from the ability to account for multimodal prior information to regularize their inherent ill-posedness. Indeed, besides priors that are formulated mathematically or learned from quantitative data, valuable prior information may be available in the form of text or images. Besides handling semantic multimodality, it is furthermore important to minimize the cost of adapting to a new physical measurement operator and to limit the requirements for costly labeled data. Here, these challenges are tackled with a frugal and multimodal semantic-EM inversion technique. The key ingredient is a multimodal generator of reconstruction results that can be pretrained, being agnostic to the physical measurement operator. The generator is fed by a multimodal foundation model encoding the multimodal semantic prior and a physical adapter encoding the measured data. For a new physical setting, only the lightweight physical adapter is retrained. The authors' architecture also enables a flexible iterative step-by-step solution to the inverse problem where each step can be semantically controlled. The feasibility and benefits of this methodology are demonstrated for three EM inverse problems: a canonical two-dimensional inverse-scattering problem in numerics, as well as three-dimensional and four-dimensional compressive microwave meta-imaging experiments.

Semantic regularization of electromagnetic inverse problems.

Solving ill-posed inverse problems typically requires regularization based on prior knowledge. To date, only prior knowledge that is formulated mathematically (e.g., sparsity of the unknown) or implicitly learned from quantitative data can be used for regularization. Thereby, semantically formulated prior knowledge derived from human reasoning and recognition is excluded. Here, we introduce and demonstrate the concept of semantic regularization based on a pre-trained large language model to overcome this vexing limitation. We study the approach, first, numerically in a prototypical 2D inverse scattering problem, and, second, experimentally in 3D and 4D compressive microwave imaging problems based on programmable metasurfaces. We highlight that semantic regularization enables new forms of highly-sought privacy protection for applications like smart homes, touchless human-machine interaction and security screening: selected subjects in the scene can be concealed, or their actions and postures can be altered in the reconstruction by manipulating the semantic prior with suitable language-based control commands.

Microwave Speech Recognizer Empowered by a Programmable Metasurface.

Speech recognition becomes increasingly important in the modern society, especially for human-machine interactions, but its deployment is still severely thwarted by the struggle of machines to recognize voiced commands in challenging real-life settings: oftentimes, ambient noise drowns the acoustic sound signals, and walls, face masks or other obstacles hide the mouth motion from optical sensors. To address these formidable challenges, an experimental prototype of a microwave speech recognizer empowered by programmable metasurface is presented here that can remotely recognize human voice commands and speaker identities even in noisy environments and if the speaker's mouth is hidden behind a wall or face mask. The programmable metasurface is the pivotal hardware ingredient of the system because its large aperture and huge number of degrees of freedom allows the system to perform a complex sequence of sensing tasks, orchestrated by artificial-intelligence tools. Relying solely on microwave data, the system avoids visual privacy infringements. The developed microwave speech recognizer can enable privacy-respecting voice-commanded human-machine interactions is experimentally demonstrated in many important but to-date inaccessible application scenarios. The presented strategy will unlock new possibilities and have expectations for future smart homes, ambient-assisted health monitoring, as well as intelligent surveillance and security.

Intelligent indoor metasurface robotics.

Intelligent indoor robotics is expected to rapidly gain importance in crucial areas of our modern society such as at-home health care and factories. Yet, existing mobile robots are limited in their ability to perceive and respond to dynamically evolving complex indoor environments because of their inherently limited sensing and computing resources that are, moreover, traded off against their cruise time and payload. To address these formidable challenges, here we propose intelligent indoor metasurface robotics (I2MR), where all sensing and computing are relegated to a centralized robotic brain endowed with microwave perception; and I2MR's limbs (motorized vehicles, airborne drones, etc.) merely execute the wirelessly received instructions from the brain. The key aspect of our concept is the centralized use of a computation-enabled programmable metasurface that can flexibly mold microwave propagation in the indoor wireless environment, including a sensing and localization modality based on configurational diversity and a communication modality to establish a preferential high-capacity wireless link between the I2MR's brain and limbs. The metasurface-enhanced microwave perception is capable of realizing low-latency and high-resolution three-dimensional imaging of humans, even around corners and behind thick concrete walls, which is the basis for action decisions of the I2MR's brain. I2MR is thus endowed with real-time and full-context awareness of its operating indoor environment. We implement, experimentally, a proof-of-principle demonstration at ∼2.4 GHz, in which I2MR provides health-care assistance to a human inhabitant. The presented strategy opens a new avenue for the conception of smart and wirelessly networked indoor robotics.

Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision.

The fifth-generation (5G) wireless communication has an urgent need for target tracking. Digital programmable metasurface (DPM) may offer an intelligent and efficient solution owing to its powerful and flexible controls of electromagnetic waves and advantages of lower cost, less complexity and smaller size than the traditional antenna array. Here, we report an intelligent metasurface system to perform target tracking and wireless communications, in which computer vision integrated with a convolutional neural network (CNN) is used to automatically detect the locations of moving targets, and the dual-polarized DPM integrated with a pre-trained artificial neural network (ANN) serves to realize the smart beam tracking and wireless communications. Three groups of experiments are conducted for demonstrating the intelligent system: detection and identification of moving targets, detection of radio-frequency signals, and real-time wireless communications. The proposed method sets the stage for an integrated implementation of target identification, radio environment tracking, and wireless communications. This strategy opens up an avenue for intelligent wireless networks and self-adaptive systems.

Accurate and broadband manipulations of harmonic amplitudes and phases to reach 256 QAM millimeter-wave wireless communications by time-domain digital coding metasurface.

We propose a theoretical mechanism and new coding strategy to realize extremely accurate manipulations of nonlinear electromagnetic harmonics in ultrawide frequency band based on a time-domain digital coding metasurface (TDCM). Using the proposed mechanism and coding strategy, we design and fabricate a millimeter-wave (mmWave) TDCM, which is composed of reprogrammable meta-atoms embedded with positive-intrinsic-negative diodes. By controlling the duty ratios and time delays of the digital coding sequences loaded on a TDCM, experimental results show that both amplitudes and phases of different harmonics can be engineered at will simultaneously and precisely in broad frequency band from 22 to 33 GHz, even when the coding states are imperfect, which is in good agreement with theoretical calculations. Based on the fabricated high-performance TDCM, we further propose and experimentally realize a large-capacity mmWave wireless communication system, where 256 quadrature amplitude modulation, along with other schemes, is demonstrated. The new wireless communication system has a much simpler architecture than the currently used mmWave wireless systems, and hence can significantly reduce the hardware cost. We believe that the proposed method and system architecture can find vast application in future mmWave and terahertz-wave wireless communication and radar systems.

Metasurface-assisted massive backscatter wireless communication with commodity Wi-Fi signals.

Conventional wireless communication architecture, a backbone of our modern society, relies on actively generated carrier signals to transfer information, leading to important challenges including limited spectral resources and energy consumption. Backscatter communication systems, on the other hand, modulate an antenna's impedance to encode information into already existing waves but suffer from low data rates and a lack of information security. Here, we introduce the concept of massive backscatter communication which modulates the propagation environment of stray ambient waves with a programmable metasurface. The metasurface's large aperture and huge number of degrees of freedom enable unprecedented wave control and thereby secure and high-speed information transfer. Our prototype leveraging existing commodity 2.4 GHz Wi-Fi signals achieves data rates on the order of hundreds of Kbps. Our technique is applicable to all types of wave phenomena and provides a fundamentally new perspective on the role of metasurfaces in future wireless communication.

Information theory of metasurfaces.

We propose a theory to characterize the information and information processing abilities of metasurfaces, and demonstrate the relation between the information of the metasurface and its radiation pattern in the far-field region. By incorporating a general aperture model with uncertainty relation in L 2-space, we propose a theory to predict the upper bound of information contained in the radiation pattern of a metasurface, and reveal the theoretical upper limit of orthogonal radiation states. The proposed theory also provides guidance for inverse design of the metasurface with respect to given functionalities. Through investigation of the information of disordered-phase modulated metasurfaces, we find the information invariance (1-γ, where γ is Euler's constant) of chaotic radiation patterns. That is to say, the information of the disordered-phase modulated radiation patterns is always equal to 1-γ, regardless of variations in size, the number of elements and the phase pattern of metasurface. This value might be the lower bound of radiation-pattern information of the metasurface, which can provide a theoretical limit for information modulation applications, including computational imaging, stealth technologies and wireless communications.

Harmonic information transitions of spatiotemporal metasurfaces.

Facilitated by ultrafast dynamic modulations, spatiotemporal metasurfaces have been identified as a pivotal platform for manipulating electromagnetic waves and creating exotic physical phenomena, such as dispersion cancellation, Lorentz reciprocity breakage, and Doppler illusions. Motivated by emerging information-oriented technologies, we hereby probe the information transition mechanisms induced by spatiotemporal variations and present a general model to characterize the information processing capabilities of the spatiotemporal metasurface. Group theory and abstract number theory are adopted through this investigation, by which the group extension and independent controls of multiple harmonics are proposed and demonstrated as two major tools for information transitions from the spatiotemporal domain to the spectra-wavevector domain. By incorporating Shannon's entropy theory into the proposed model, we further discover the corresponding information transition efficiencies and the upper bound of the channel capacity of the spatiotemporal metasurface. The results of harmonic information transitions show great potential in achieving high-capacity versatile information processing systems with spatiotemporal metasurfaces.

Information Metamaterial Systems.

Metamaterials have great capabilities and flexibilities in controlling electromagnetic (EM) waves because their subwavelength meta-atoms can be designed and tailored in desired ways. However, once the structure-only metamaterials (i.e., passive metamaterials) are fabricated, their functions will be fixed. To control the EM waves dynamically, active devices are integrated into the meta-atoms, yielding active metamaterials. Traditionally, the active metamaterials include tunable metamaterials and reconfigurable metamaterials, which have either small-range tunability or a few numbers of reconfigurability. Recently, a special kind of active metamaterials, digital coding and programmable metamaterials, have been presented, which can realize a large number of distinct functionalities and switch them in real time with the aid of field programmable gate array (FPGA). More importantly, the digital coding representations of metamaterials make it possible to bridge the digital world and physical world using the metamaterial platform and make the metamaterials process digital information directly, resulting in information metamaterials. In this review article, we firstly introduce the evolution of metamaterials and then present the concepts and basic principles of digital coding metamaterials and information metamaterials. With more details, we discuss a series of information metamaterial systems, including the programmable metamaterial systems, software metamaterial systems, intelligent metamaterial systems, and space-time-coding metamaterial systems. Finally, we introduce the current progress and predict the future trends of information metamaterials.

Intelligent Electromagnetic Sensing with Learnable Data Acquisition and Processing.

Electromagnetic (EM) sensing is a widespread contactless examination technique with applications in areas such as health care and the internet of things. Most conventional sensing systems lack intelligence, which not only results in expensive hardware and complicated computational algorithms but also poses important challenges for real-time in situ sensing. To address this shortcoming, we propose the concept of intelligent sensing by designing a programmable metasurface for data-driven learnable data acquisition and integrating it into a data-driven learnable data-processing pipeline. Thereby, a measurement strategy can be learned jointly with a matching data post-processing scheme, optimally tailored to the specific sensing hardware, task, and scene, allowing us to perform high-quality imaging and high-accuracy recognition with a remarkably reduced number of measurements. We report the first experimental demonstration of "learned sensing" applied to microwave imaging and gesture recognition. Our results pave the way for learned EM sensing with low latency and computational burden.

Smart metasurface with self-adaptively reprogrammable functions.

Intelligence at either the material or metamaterial level is a goal that researchers have been pursuing. From passive to active, metasurfaces have been developed to be programmable to dynamically and arbitrarily manipulate electromagnetic (EM) wavefields. However, the programmable metasurfaces require manual control to switch among different functionalities. Here, we put forth a smart metasurface that has self-adaptively reprogrammable functionalities without human participation. The smart metasurface is capable of sensing ambient environments by integrating an additional sensor(s) and can adaptively adjust its EM operational functionality through an unmanned sensing feedback system. As an illustrative example, we experimentally develop a motion-sensitive smart metasurface integrated with a three-axis gyroscope, which can adjust self-adaptively the EM radiation beams via different rotations of the metasurface. We develop an online feedback algorithm as the control software to make the smart metasurface achieve single-beam and multibeam steering and other dynamic reactions adaptively. The proposed metasurface is extendable to other physical sensors to detect the humidity, temperature, illuminating light, and so on. Our strategy will open up a new avenue for future unmanned devices that are consistent with the ambient environment.

Machine-learning reprogrammable metasurface imager.

Conventional microwave imagers usually require either time-consuming data acquisition, or complicated reconstruction algorithms for data post-processing, making them largely ineffective for complex in-situ sensing and monitoring. Here, we experimentally report a real-time digital-metasurface imager that can be trained in-situ to generate the radiation patterns required by machine-learning optimized measurement modes. This imager is electronically reprogrammed in real time to access the optimized solution for an entire data set, realizing storage and transfer of full-resolution raw data in dynamically varying scenes. High-accuracy image coding and recognition are demonstrated in situ for various image sets, including hand-written digits and through-wall body gestures, using a single physical hardware imager, reprogrammed in real time. Our electronically controlled metasurface imager opens new venues for intelligent surveillance, fast data acquisition and processing, imaging at various frequencies, and beyond.

Intelligent metasurface imager and recognizer.

There is an increasing need to remotely monitor people in daily life using radio-frequency probe signals. However, conventional systems can hardly be deployed in real-world settings since they typically require objects to either deliberately cooperate or carry a wireless active device or identification tag. To accomplish complicated successive tasks using a single device in real time, we propose the simultaneous use of a smart metasurface imager and recognizer, empowered by a network of artificial neural networks (ANNs) for adaptively controlling data flow. Here, three ANNs are employed in an integrated hierarchy, transforming measured microwave data into images of the whole human body, classifying specifically designated spots (hand and chest) within the whole image, and recognizing human hand signs instantly at a Wi-Fi frequency of 2.4 GHz. Instantaneous in situ full-scene imaging and adaptive recognition of hand signs and vital signs of multiple non-cooperative people were experimentally demonstrated. We also show that the proposed intelligent metasurface system works well even when it is passively excited by stray Wi-Fi signals that ubiquitously exist in our daily lives. The reported strategy could open up a new avenue for future smart cities, smart homes, human-device interaction interfaces, health monitoring, and safety screening free of visual privacy issues.