Semantic regularization of electromagnetic inverse problems.

Solving ill-posed inverse problems typically requires regularization based on prior knowledge. To date, only prior knowledge that is formulated mathematically (e.g., sparsity of the unknown) or implicitly learned from quantitative data can be used for regularization. Thereby, semantically formulated prior knowledge derived from human reasoning and recognition is excluded. Here, we introduce and demonstrate the concept of semantic regularization based on a pre-trained large language model to overcome this vexing limitation. We study the approach, first, numerically in a prototypical 2D inverse scattering problem, and, second, experimentally in 3D and 4D compressive microwave imaging problems based on programmable metasurfaces. We highlight that semantic regularization enables new forms of highly-sought privacy protection for applications like smart homes, touchless human-machine interaction and security screening: selected subjects in the scene can be concealed, or their actions and postures can be altered in the reconstruction by manipulating the semantic prior with suitable language-based control commands.

Microwave Speech Recognizer Empowered by a Programmable Metasurface.

Speech recognition becomes increasingly important in the modern society, especially for human-machine interactions, but its deployment is still severely thwarted by the struggle of machines to recognize voiced commands in challenging real-life settings: oftentimes, ambient noise drowns the acoustic sound signals, and walls, face masks or other obstacles hide the mouth motion from optical sensors. To address these formidable challenges, an experimental prototype of a microwave speech recognizer empowered by programmable metasurface is presented here that can remotely recognize human voice commands and speaker identities even in noisy environments and if the speaker's mouth is hidden behind a wall or face mask. The programmable metasurface is the pivotal hardware ingredient of the system because its large aperture and huge number of degrees of freedom allows the system to perform a complex sequence of sensing tasks, orchestrated by artificial-intelligence tools. Relying solely on microwave data, the system avoids visual privacy infringements. The developed microwave speech recognizer can enable privacy-respecting voice-commanded human-machine interactions is experimentally demonstrated in many important but to-date inaccessible application scenarios. The presented strategy will unlock new possibilities and have expectations for future smart homes, ambient-assisted health monitoring, as well as intelligent surveillance and security.

Intelligent indoor metasurface robotics.

Intelligent indoor robotics is expected to rapidly gain importance in crucial areas of our modern society such as at-home health care and factories. Yet, existing mobile robots are limited in their ability to perceive and respond to dynamically evolving complex indoor environments because of their inherently limited sensing and computing resources that are, moreover, traded off against their cruise time and payload. To address these formidable challenges, here we propose intelligent indoor metasurface robotics (I2MR), where all sensing and computing are relegated to a centralized robotic brain endowed with microwave perception; and I2MR's limbs (motorized vehicles, airborne drones, etc.) merely execute the wirelessly received instructions from the brain. The key aspect of our concept is the centralized use of a computation-enabled programmable metasurface that can flexibly mold microwave propagation in the indoor wireless environment, including a sensing and localization modality based on configurational diversity and a communication modality to establish a preferential high-capacity wireless link between the I2MR's brain and limbs. The metasurface-enhanced microwave perception is capable of realizing low-latency and high-resolution three-dimensional imaging of humans, even around corners and behind thick concrete walls, which is the basis for action decisions of the I2MR's brain. I2MR is thus endowed with real-time and full-context awareness of its operating indoor environment. We implement, experimentally, a proof-of-principle demonstration at ∼2.4 GHz, in which I2MR provides health-care assistance to a human inhabitant. The presented strategy opens a new avenue for the conception of smart and wirelessly networked indoor robotics.

Intelligent metasurface system for automatic tracking of moving targets and wireless communications based on computer vision.

The fifth-generation (5G) wireless communication has an urgent need for target tracking. Digital programmable metasurface (DPM) may offer an intelligent and efficient solution owing to its powerful and flexible controls of electromagnetic waves and advantages of lower cost, less complexity and smaller size than the traditional antenna array. Here, we report an intelligent metasurface system to perform target tracking and wireless communications, in which computer vision integrated with a convolutional neural network (CNN) is used to automatically detect the locations of moving targets, and the dual-polarized DPM integrated with a pre-trained artificial neural network (ANN) serves to realize the smart beam tracking and wireless communications. Three groups of experiments are conducted for demonstrating the intelligent system: detection and identification of moving targets, detection of radio-frequency signals, and real-time wireless communications. The proposed method sets the stage for an integrated implementation of target identification, radio environment tracking, and wireless communications. This strategy opens up an avenue for intelligent wireless networks and self-adaptive systems.

Three-Dimensional Encoding Approach for Wearable Tactile Communication Devices.

In this work, we presented a novel encoding method for tactile communication. This approach was based on several tactile sensory characteristics of human skin at different body parts, such as the head and neck, where location coordinates in the three-dimensional (3D) space were clearly mapped in the brain cortex, and gentle stimulations of vibrational touching with varied strengths were received instantly and precisely. For certain applications, such as playing cards or navigating walk paths for blinded people, we demonstrated specifically designed code lists with different patterns of tactile points in varied temporal sequences. By optimizing these codes, we achieved excellent efficiency and accuracy in our test experiments. As this method matched well with the natural habits of tactile sensory, it was easy to learn in a short training period. The results of the present work have offered a silent, efficient and accurate communication solution for visually impaired people or other users.

Accurate and broadband manipulations of harmonic amplitudes and phases to reach 256 QAM millimeter-wave wireless communications by time-domain digital coding metasurface.

We propose a theoretical mechanism and new coding strategy to realize extremely accurate manipulations of nonlinear electromagnetic harmonics in ultrawide frequency band based on a time-domain digital coding metasurface (TDCM). Using the proposed mechanism and coding strategy, we design and fabricate a millimeter-wave (mmWave) TDCM, which is composed of reprogrammable meta-atoms embedded with positive-intrinsic-negative diodes. By controlling the duty ratios and time delays of the digital coding sequences loaded on a TDCM, experimental results show that both amplitudes and phases of different harmonics can be engineered at will simultaneously and precisely in broad frequency band from 22 to 33 GHz, even when the coding states are imperfect, which is in good agreement with theoretical calculations. Based on the fabricated high-performance TDCM, we further propose and experimentally realize a large-capacity mmWave wireless communication system, where 256 quadrature amplitude modulation, along with other schemes, is demonstrated. The new wireless communication system has a much simpler architecture than the currently used mmWave wireless systems, and hence can significantly reduce the hardware cost. We believe that the proposed method and system architecture can find vast application in future mmWave and terahertz-wave wireless communication and radar systems.

Harmonic information transitions of spatiotemporal metasurfaces.

Facilitated by ultrafast dynamic modulations, spatiotemporal metasurfaces have been identified as a pivotal platform for manipulating electromagnetic waves and creating exotic physical phenomena, such as dispersion cancellation, Lorentz reciprocity breakage, and Doppler illusions. Motivated by emerging information-oriented technologies, we hereby probe the information transition mechanisms induced by spatiotemporal variations and present a general model to characterize the information processing capabilities of the spatiotemporal metasurface. Group theory and abstract number theory are adopted through this investigation, by which the group extension and independent controls of multiple harmonics are proposed and demonstrated as two major tools for information transitions from the spatiotemporal domain to the spectra-wavevector domain. By incorporating Shannon's entropy theory into the proposed model, we further discover the corresponding information transition efficiencies and the upper bound of the channel capacity of the spatiotemporal metasurface. The results of harmonic information transitions show great potential in achieving high-capacity versatile information processing systems with spatiotemporal metasurfaces.

Intelligent Electromagnetic Sensing with Learnable Data Acquisition and Processing.

Electromagnetic (EM) sensing is a widespread contactless examination technique with applications in areas such as health care and the internet of things. Most conventional sensing systems lack intelligence, which not only results in expensive hardware and complicated computational algorithms but also poses important challenges for real-time in situ sensing. To address this shortcoming, we propose the concept of intelligent sensing by designing a programmable metasurface for data-driven learnable data acquisition and integrating it into a data-driven learnable data-processing pipeline. Thereby, a measurement strategy can be learned jointly with a matching data post-processing scheme, optimally tailored to the specific sensing hardware, task, and scene, allowing us to perform high-quality imaging and high-accuracy recognition with a remarkably reduced number of measurements. We report the first experimental demonstration of "learned sensing" applied to microwave imaging and gesture recognition. Our results pave the way for learned EM sensing with low latency and computational burden.

Metasurface-assisted massive backscatter wireless communication with commodity Wi-Fi signals.

Conventional wireless communication architecture, a backbone of our modern society, relies on actively generated carrier signals to transfer information, leading to important challenges including limited spectral resources and energy consumption. Backscatter communication systems, on the other hand, modulate an antenna's impedance to encode information into already existing waves but suffer from low data rates and a lack of information security. Here, we introduce the concept of massive backscatter communication which modulates the propagation environment of stray ambient waves with a programmable metasurface. The metasurface's large aperture and huge number of degrees of freedom enable unprecedented wave control and thereby secure and high-speed information transfer. Our prototype leveraging existing commodity 2.4 GHz Wi-Fi signals achieves data rates on the order of hundreds of Kbps. Our technique is applicable to all types of wave phenomena and provides a fundamentally new perspective on the role of metasurfaces in future wireless communication.

Information Metamaterial Systems.

Metamaterials have great capabilities and flexibilities in controlling electromagnetic (EM) waves because their subwavelength meta-atoms can be designed and tailored in desired ways. However, once the structure-only metamaterials (i.e., passive metamaterials) are fabricated, their functions will be fixed. To control the EM waves dynamically, active devices are integrated into the meta-atoms, yielding active metamaterials. Traditionally, the active metamaterials include tunable metamaterials and reconfigurable metamaterials, which have either small-range tunability or a few numbers of reconfigurability. Recently, a special kind of active metamaterials, digital coding and programmable metamaterials, have been presented, which can realize a large number of distinct functionalities and switch them in real time with the aid of field programmable gate array (FPGA). More importantly, the digital coding representations of metamaterials make it possible to bridge the digital world and physical world using the metamaterial platform and make the metamaterials process digital information directly, resulting in information metamaterials. In this review article, we firstly introduce the evolution of metamaterials and then present the concepts and basic principles of digital coding metamaterials and information metamaterials. With more details, we discuss a series of information metamaterial systems, including the programmable metamaterial systems, software metamaterial systems, intelligent metamaterial systems, and space-time-coding metamaterial systems. Finally, we introduce the current progress and predict the future trends of information metamaterials.

Information theory of metasurfaces.

We propose a theory to characterize the information and information processing abilities of metasurfaces, and demonstrate the relation between the information of the metasurface and its radiation pattern in the far-field region. By incorporating a general aperture model with uncertainty relation in L 2-space, we propose a theory to predict the upper bound of information contained in the radiation pattern of a metasurface, and reveal the theoretical upper limit of orthogonal radiation states. The proposed theory also provides guidance for inverse design of the metasurface with respect to given functionalities. Through investigation of the information of disordered-phase modulated metasurfaces, we find the information invariance (1-γ, where γ is Euler's constant) of chaotic radiation patterns. That is to say, the information of the disordered-phase modulated radiation patterns is always equal to 1-γ, regardless of variations in size, the number of elements and the phase pattern of metasurface. This value might be the lower bound of radiation-pattern information of the metasurface, which can provide a theoretical limit for information modulation applications, including computational imaging, stealth technologies and wireless communications.

Smart metasurface with self-adaptively reprogrammable functions.

Intelligence at either the material or metamaterial level is a goal that researchers have been pursuing. From passive to active, metasurfaces have been developed to be programmable to dynamically and arbitrarily manipulate electromagnetic (EM) wavefields. However, the programmable metasurfaces require manual control to switch among different functionalities. Here, we put forth a smart metasurface that has self-adaptively reprogrammable functionalities without human participation. The smart metasurface is capable of sensing ambient environments by integrating an additional sensor(s) and can adaptively adjust its EM operational functionality through an unmanned sensing feedback system. As an illustrative example, we experimentally develop a motion-sensitive smart metasurface integrated with a three-axis gyroscope, which can adjust self-adaptively the EM radiation beams via different rotations of the metasurface. We develop an online feedback algorithm as the control software to make the smart metasurface achieve single-beam and multibeam steering and other dynamic reactions adaptively. The proposed metasurface is extendable to other physical sensors to detect the humidity, temperature, illuminating light, and so on. Our strategy will open up a new avenue for future unmanned devices that are consistent with the ambient environment.

Intelligent metasurface imager and recognizer.

There is an increasing need to remotely monitor people in daily life using radio-frequency probe signals. However, conventional systems can hardly be deployed in real-world settings since they typically require objects to either deliberately cooperate or carry a wireless active device or identification tag. To accomplish complicated successive tasks using a single device in real time, we propose the simultaneous use of a smart metasurface imager and recognizer, empowered by a network of artificial neural networks (ANNs) for adaptively controlling data flow. Here, three ANNs are employed in an integrated hierarchy, transforming measured microwave data into images of the whole human body, classifying specifically designated spots (hand and chest) within the whole image, and recognizing human hand signs instantly at a Wi-Fi frequency of 2.4 GHz. Instantaneous in situ full-scene imaging and adaptive recognition of hand signs and vital signs of multiple non-cooperative people were experimentally demonstrated. We also show that the proposed intelligent metasurface system works well even when it is passively excited by stray Wi-Fi signals that ubiquitously exist in our daily lives. The reported strategy could open up a new avenue for future smart cities, smart homes, human-device interaction interfaces, health monitoring, and safety screening free of visual privacy issues.

Machine-learning reprogrammable metasurface imager.

Conventional microwave imagers usually require either time-consuming data acquisition, or complicated reconstruction algorithms for data post-processing, making them largely ineffective for complex in-situ sensing and monitoring. Here, we experimentally report a real-time digital-metasurface imager that can be trained in-situ to generate the radiation patterns required by machine-learning optimized measurement modes. This imager is electronically reprogrammed in real time to access the optimized solution for an entire data set, realizing storage and transfer of full-resolution raw data in dynamically varying scenes. High-accuracy image coding and recognition are demonstrated in situ for various image sets, including hand-written digits and through-wall body gestures, using a single physical hardware imager, reprogrammed in real time. Our electronically controlled metasurface imager opens new venues for intelligent surveillance, fast data acquisition and processing, imaging at various frequencies, and beyond.

Locally Oriented Scene Complexity Analysis Real-Time Ocean Ship Detection from Optical Remote Sensing Images.

Due to strong ocean waves, broken clouds, and extensive cloud cover interferences, ocean ship detection performs poorly when using optical remote sensing images. In addition, it is a challenge to detect small ships on medium resolution optical remote sensing that cover a large area. In this paper, in order to balance the requirements of real-time processing and high accuracy detection, we proposed a novel ship detection framework based on locally oriented scene complexity analysis. First, the proposed method can separate a full image into two types of local scenes (i.e., simple or complex local scenes). Next, simple local scenes would utilize the fast saliency model (FSM) to rapidly complete candidate extraction, and for complex local scenes, the ship feature clustering model (SFCM) will be applied to achieve refined detection against severe background interferences. The FSM considers a fusion enhancement image as an input of the pulse response analysis in the frequency domain to achieve rapid ship detection in simple local scenes. Next, the SFCM builds the descriptive model of the ship feature clustering algorithm to ensure the detection performance on complex local scenes. Extensive experiments on SPOT-5 and GF-2 ocean optical remote sensing images show that the proposed ship detection framework has better performance than the state-of-the-art methods, and it addresses the tricky problem of real-time ocean ship detection under strong waves, broken clouds, extensive cloud cover, and ship fleet interferences. Finally, the proposed ocean ship detection framework is demonstrated on an onboard processing hardware.

Controlling Energy Radiations of Electromagnetic Waves via Frequency Coding Metamaterials.

Metamaterials are artificial structures composed of subwavelength unit cells to control electromagnetic (EM) waves. The spatial coding representation of metamaterial has the ability to describe the material in a digital way. The spatial coding metamaterials are typically constructed by unit cells that have similar shapes with fixed functionality. Here, the concept of frequency coding metamaterial is proposed, which achieves different controls of EM energy radiations with a fixed spatial coding pattern when the frequency changes. In this case, not only different phase responses of the unit cells are considered, but also different phase sensitivities are also required. Due to different frequency sensitivities of unit cells, two units with the same phase response at the initial frequency may have different phase responses at higher frequency. To describe the frequency coding property of unit cell, digitalized frequency sensitivity is proposed, in which the units are encoded with digits "0" and "1" to represent the low and high phase sensitivities, respectively. By this merit, two degrees of freedom, spatial coding and frequency coding, are obtained to control the EM energy radiations by a new class of frequency-spatial coding metamaterials. The above concepts and physical phenomena are confirmed by numerical simulations and experiments.

Spin-Controlled Multiple Pencil Beams and Vortex Beams with Different Polarizations Generated by Pancharatnam-Berry Coding Metasurfaces.

We propose to design coding metasurfaces based on the Pancharatnam-Berry (PB) phase. The proposed PB coding metasurface could control circularly polarized components of incident waves, by encoding geometric phase into the orientation angle of coding particles to generate 1-bit and multibit phase responses. We perform digital convolution operations on scattering patterns of the PB coding metasurface to reach flexible controls of the circularly polarized waves, forming spin-controlled multiple beams with different polarizations in free space, such as pencil beams and vortex beams carrying orbital angular momentum. Both numerical and experimental results demonstrate the excellent performance of the PB coding metasurface, which opens a pathway to novel types of multibeam generations and provides an effective way to expand the beam coverage for wireless communication applications.

Electromagnetic reprogrammable coding-metasurface holograms.

Metasurfaces have enabled a plethora of emerging functions within an ultrathin dimension, paving way towards flat and highly integrated photonic devices. Despite the rapid progress in this area, simultaneous realization of reconfigurability, high efficiency, and full control over the phase and amplitude of scattered light is posing a great challenge. Here, we try to tackle this challenge by introducing the concept of a reprogrammable hologram based on 1-bit coding metasurfaces. The state of each unit cell of the coding metasurface can be switched between '1' and '0' by electrically controlling the loaded diodes. Our proof-of-concept experiments show that multiple desired holographic images can be realized in real time with only a single coding metasurface. The proposed reprogrammable hologram may be a key in enabling future intelligent devices with reconfigurable and programmable functionalities that may lead to advances in a variety of applications such as microscopy, display, security, data storage, and information processing.Realizing metasurfaces with reconfigurability, high efficiency, and control over phase and amplitude is a challenge. Here, Li et al. introduce a reprogrammable hologram based on a 1-bit coding metasurface, where the state of each unit cell of the coding metasurface can be switched electrically.

Extraordinary optical transmission through a rectangular hole filled with extreme uniaxial metamaterials.

This Letter presents a theory of extraordinary optical transmission (EOT) through a rectangular hole filled with the extreme uniaxial metamaterials with infinite longitudinal components of permittivity (ϵz) and permeability (µz). We demonstrate theoretically and numerically that a number of high-order transverse electromagnetic (TEM) modes can be supported by such a structure, and that, more interestingly, their normalized transmittance can be remarkably enhanced due to the Fabry-Perot resonance effect. A set of illustrative examples has been provided to demonstrate that such an EOT property could be explored for the purpose of subwavelength-resolution imaging.

Single-shot and single-sensor high/super-resolution microwave imaging based on metasurface.

Real-time high-resolution (including super-resolution) imaging with low-cost hardware is a long sought-after goal in various imaging applications. Here, we propose broadband single-shot and single-sensor high-/super-resolution imaging by using a spatio-temporal dispersive metasurface and an imaging reconstruction algorithm. The metasurface with spatio-temporal dispersive property ensures the feasibility of the single-shot and single-sensor imager for super- and high-resolution imaging, since it can convert efficiently the detailed spatial information of the probed object into one-dimensional time- or frequency-dependent signal acquired by a single sensor fixed in the far-field region. The imaging quality can be improved by applying a feature-enhanced reconstruction algorithm in post-processing, and the desired imaging resolution is related to the distance between the object and metasurface. When the object is placed in the vicinity of the metasurface, the super-resolution imaging can be realized. The proposed imaging methodology provides a unique means to perform real-time data acquisition, high-/super-resolution images without employing expensive hardware (e.g. mechanical scanner, antenna array, etc.). We expect that this methodology could make potential breakthroughs in the areas of microwave, terahertz, optical, and even ultrasound imaging.