Physics-informed and simulation-driven optimization for binary Fourier single-pixel imaging.

Binary patterns are used in fast Fourier single-pixel imaging (FSI) technology to increase the imaging speed at the expense of spatial resolution or image quality. In this Letter, we propose a method for optimizing the image quality-speed trade-off that is informed by physical principles and driven by data from simulations. To compensate for the quantization error induced by binary dithering, convolution kernels are proposed and optimized for both low and high spatial frequencies. The proposed method has been demonstrated to work in both simulation and experiments. Other single-pixel imaging (SPI) techniques may also benefit from this approach.

Research on the Sensing Characteristics of an Integrated Grid-like Sensor Based on a Triboelectric Nanogenerator.

With the development of the integration and miniaturization of sensing devices, the concept of self-sensing devices has been proposed. A motion state is self-sensed via the structure or integration of an actuator in the construction of a sensing unit. This device is then used to capture the perception and measurement of states such as position, displacement, and speed. A triboelectric nanogenerator converts mechanical energy into electrical energy through the coupling effect of contact generation and electrostatic induction, which represents one of the reliable ways through which to realize integrated sensing. In this world, the power generation technology of the TENG is applied to a sensing device. The sensing characteristics of a grid-like TENG are designed and analyzed in freestanding triboelectric mode. Firstly, a relation model of displacement, velocity, voltage, and charge is established. The charge-transfer increment and current amounts are linearly related to the velocity. The open-circuit voltage has a positive relationship with the displacement. The maximum open-circuit voltage and the maximum charge transfer are fixed values, and they are only related to the inherent parameters of a triboelectric nanogenerator. Next, the sensor model is constructed using COMSOL Multiphysics 6.0. The simulation results show that the relationships between output voltage and charge transfer, as well as those between the increments of charge transfer, velocity, and displacement, are consistent with the results derived from the formula. Finally, a performance test of the designed sensor is carried out, and the results are consistent with the theoretical deduction and simulation. After analysis and processing of the output electrical signal by the host computer, it can feedback the frequency and speed value of the measured object. In addition, the output signal is stable, and there is no large fluctuation or attenuation during the 521-s vibration test. Because the working unit of the sensor is thin filmed, it is small in size, easy to integrate, and has no external power supply; moreover, it can be integrated into a device to realize the self-sensing of a motion state.

Polar coordinate Fourier single-pixel imaging.

Traditional single-pixel imaging uses Fourier patterns to modulate objects in the Cartesian coordinate system. The Cartesian Fourier pattern of single-pixel imaging is inappropriate to display in a circular field of view. However, a circular field of view is a widespread form of display in computed optical imaging. Here, circular patterns are adopted to adapt to the circular visual area. The circular patterns are displayed in polar coordinates and derived from two-dimensional Fourier transform in polar coordinates. The proposed circular patterns have improved imaging efficiency significantly from 63.66% to 100%. The proposed polar coordinate Fourier single-pixel imaging is expected to be applied in circular field-of-view imaging and foveated imaging.

Dynamic 3D Measurement without Motion Artifacts Based on Feature Compensation.

Phase-shift profilometry (PSP) holds great promise for high-precision 3D shape measurements. However, in the case of measuring moving objects, as PSP requires multiple images to calculate the phase, the movement of the object causes artifacts in the measurement, which in turn has a significant impact on the accuracy of the 3D surface measurement. Therefore, we propose a method to reduce motion artifacts using feature information in the image and simulate it using the six-step term shift method as a case study. The simulation results show that the phase of the object is greatly affected when the object is in motion and that the phase shift due to motion can be effectively reduced using this method. Finally, artifact optimization was carried out by way of specific copper tube vibration experiments at a measurement frequency of 320 Hz. The experimental results prove that the method is well implemented.

Ultra-small lipid carriers with adjustable release profiles for synergistic treatment of drug-resistant ovarian cancer.

Multidrug resistance has dramatically compromised the effectiveness of paclitaxel (PTX). The combined application of PTX and tetrandrine (TET) is a promising avenue in drug-resistant cancer therapy. However, poor drug release and limited intracellular drug accumulation greatly impede this combinational antitumor therapy. To address this problem, we successfully developed a tunable controlled release lipid platform (PT@usNLC) for coordinated drug delivery. The drug release rate of PT@usNLC can be tuned by varying the lipid ratio, which has potential to maximize the therapeutic effects of combined drugs. The TET release rate from PT@usNLC was faster than PTX, which could restore the sensitivity of tumor cells to PTX and exert a synergistic antitumor effect. The appropriate size of PT@usNLC could effectively increase the intracellular drug accumulation. Bothin vitroandin vivostudies revealed that PT@usNLC significantly enhanced the therapeutic effect compared to conventional therapies. This study provides a new strategy for resistant ovarian cancer therapy.

Single-pixel imaging of high-temperature objects.

For visual measurement at high temperature, one challenge is how to reduce the radiation emitted by the high-temperature components themselves and the influence of hot gas flow on the image quality, which has a significant impact in fields such as aerospace or automotive manufacturing. Owing to the complicated optical imaging environment at high temperature, a new, to the best of our knowledge, image acquisition method of high-temperature components is proposed in combination with single-pixel imaging in this paper. A series of illumination patterns is emitted to the object, and the light waves measured by the single-pixel detector are used to reconstruct the image of the object. Single-pixel imaging of high-temperature objects at different temperatures and different spectral segments has been studied in this paper. The experiment proves that the method presented in this paper can decrease the strong light interference of the high-temperature object's own radiation light and reduce the halo caused by the high temperature. This study provides a good impetus for the development of single-pixel imaging in the industrial field of high-temperature components by reducing the radiation light.

Reflection removal detection enabled by single-pixel imaging through the semi-reflective medium.

Due to the substantial reflection information of the surrounding environment, it is difficult for a conventional camera to directly capture the distinct image behind without interference from the reflected virtual image through semi-reflective media such as an acrylic plate, glass, or water. Traditional reflective artifact removal methods either demand a major commitment of calculations or constrained photography conditions such as the use of a polarizer, which often degrades the performance of the reflection removal process and imposes a limitation on the application area. A different reflection removal method is investigated, where the interfering light rays can be attenuated effectively based on a differential calculation with a Fourier single-pixel imaging method. Experiments show that this method eliminates the interference caused by reflection from interfering objects and obtains clear images through an acrylic plate (with thicknesses of 1 mm, 2 mm, and 3 mm), glass (5 mm), and even transparent water (100 mm). Another experiment has been carried out to effectively image the target by removing the reflection through the glasses, which have the same thickness (1.1 mm) but different reflectivity (20%, 30%, 40%, and 50%).

Transmissive Single-Pixel Microscopic Imaging through Scattering Media.

Microscopic imaging is of great significance for medical diagnosis. However, due to the strong scattering and absorption of tissue, the implementation of non-invasive microscopic imaging is very difficult. Traditional single-pixel microscopes, based on reflective optical systems, provide an alternative solution for scattering media imaging. Here, the single-pixel microscope with transmissive liquid crystal modulation is proposed. The microscopic ability of the proposed microscope is calibrated. The multi-spectral microscopic imaging of the object is demonstrated. The transmissive imaging of the object behind the scattering media is analyzed. The proposed prototype of the transmissive single-pixel microscope is expected to be applied in microscopic imaging through scattering media and medical imaging.

Target orientation detection based on a neural network with a bionic bee-like compound eye.

The compound eye of insects has many excellent characteristics. Directional navigation is one of the important features of compound eye, which is able to quickly and accurately determine the orientation of an objects. Therefore, bionic curved compound eye have great potential in detecting the orientation of the target. However, there is a serious non-linear relationship between the orientation of the target and the image obtained by the curved compound eye in wide field of view (FOV), and an effective model has not been established to detect the orientation of target. In this paper, a method for detecting the orientation of the target is proposed, which combines a virtual cylinder target with a neural network. To verify the feasibility of the method, a fiber-optic compound eye that is inspired by the structure of the bee's compound eye and that fully utilizes the transmission characteristics and flexibility of optical fibers is developed. A verification experiment shows that the proposed method is able to realize quantitative detection of orientations using a prototype of the fiber-optic compound eye. The average errors between the ground truth and the predicted values of the horizontal and elevation angles of a target are 0.5951 ° and 0.6748°, respectively. This approach has great potential for target tracking, obstacle avoidance by unmanned aerial vehicles, and directional navigation control.

Interface modeling of magnetorheological elastomers subjected to variable working strain.

A magnetorheological elastomer (MRE) is a type of particle-matrix composite material, whose properties depend on the strain to which it is subjected in different applications. This paper proposes an interface model in which the magnetorheological characteristics of an MRE are described in terms of the effect of variable strain on the strength of interfacial bonding between the particles and the matrix. The model can describe the whole process of interface change from a strong interface to a strong-weak mixed interface and then to a weak interface under variable strain. The results indicate that the combined effects of the magnetic flux density, particle content, and strain amplitude are responsible for the magnetorheological performance of the MRE. The maximum value of the shear modulus under large strain is decreased by 0.75 × 105 Pa compared to the value under small strain. This model opens new opportunities for the development of high-performance MREs and MRE-based devices under variable strain conditions.

Multi-camera calibration method based on a multi-plane stereo target.

Machine vision techniques, including camera calibration methods, are of great importance for the development of vision-based measurements. However, in multi-camera calibration methods, rapidly constructing accurate geometric relationships among different coordinates is very difficult. Herein, we present a multi-camera calibration method capable of calibrating the intrinsic and extrinsic parameters of four cameras using only a single captured image per camera. Unlike Zhang's method, which relies on multiple captured images to calibrate the cameras, the method uses a multi-plane stereo target containing multiple fixed planes to which coded patterns are attached. This target greatly reduces the time required for calibration and improves calibration robustness. The proposed method was experimentally compared with traditional camera calibration. The problem affecting the calibration accuracy in single calibration of multiple cameras is that the feature points on the captured images produce occlusion or different degrees of blurring; in the calibration of multiple cameras multiple times, the error accumulation caused by the calibration of two adjacent cameras is solved. This demonstration of a multi-camera calibration method improves camera calibration and provides a new design philosophy, to the best of our knowledge, for machine vision and vision-based measurement.

Dynamic Visual Measurement of Driver Eye Movements.

Vibrations often cause visual fatigue for drivers, and measuring the relative motion between the driver and the display is important for evaluating this visual fatigue. This paper proposes a non-contact videometric measurement method for studying the three-dimensional trajectories of the driver's eyes based on stereo vision. The feasibility of this method is demonstrated by dynamic calibration. A high-speed dual-camera image acquisition system is used to obtain high-definition images of the face, and the relative trajectories between the eyes and the display are obtained by a set of robust algorithms. The trajectories of the eyes in three-dimensional space are then reconstructed during the vehicle driving process. This new approach provides three-dimensional information and is effective for assessing how vibration affects human visual performance.

Catadioptric planar compound eye with large field of view.

The planar compound eye has the advantages of simple structure and no requirement for complex relay optical elements, but the field of view (FOV) is very difficult to expand. Overcoming the limitation of FOV, especially with simple structures, is a great challenge for the development of planar compound eyes. Different from the existing designs that only considering refraction, this article proposes a catadioptric planar compound eye based on the reflection and refraction to expand the FOV. In the proposed design, the incident light from a large angle is reflected into the lenslet array by two rotationally symmetric mirrors whose surface equations are optimized by mathematical and optical softwares. The FOV of the proposed catadioptric planar compound eye theoretically can reach 96.6°, which is much wider than the opening record of 70°. Moreover, no distortion of the imaging system can be obtained theoretically in this design. Simulation results show a linearity of better than 99% for the most of the incident angles. The verification experiments show that the FOV of the proposed device can reach 90.7° while the FOV of the corresponding planar compound eye without mirrors is 41.6°. The proposed catadioptric planar compound eye has the great potential in monitoring, detection and virtual reality since the FOV has been widen significantly.

Vision measurement error analysis for nonlinear light refraction at high temperature.

For vision measurement at high temperature, pixel error from light refraction by high temperature is a problem that cannot be neglected. The refractive index distribution is nonlinear around the high-temperature component, leading to the light deflection. In this paper, the influence of measurement parameters on the deflection of imaging light and the accuracy after binocular reconstruction are systematically analyzed. The heat transfer theory is used to simulate the air temperature distribution near the measured component, and the corresponding refractive index distribution of air is obtained according to the refractive index formula. Then, the imaging measurement model of the nonlinear refractive index air medium for the high-temperature component is established to obtain the light deflection error. The binocular vision system reconstruction theory is applied to evaluate the measurement error between the theoretical reconstruction point and the object point. The influences of error sources, such as the temperature, optical wavelength, and camera parameters, are investigated systematically. It is found that temperature and base distance are the largest error sources on the final measurement error when the measured distance is fixed.

Pulse-Width Multiplexing ϕ-OTDR for Nuisance-Alarm Rate Reduction.

A pulse-width multiplexing method for reducing the nuisance-alarm rate of a phase-sensitive optical time-domain reflectometer ( ϕ -OTDR) is described. In this method, light pulses of different pulse-widths are injected into the sensing fiber; the data acquired at different pulse-widths are regarded as the outputs of different sensors; and these data are then processed by a multisensor data fusion algorithm. In laboratory tests with a sensing fiber on a vibrating table, the effects of pulse-width on the signal-to-noise ratio (SNR) of the ϕ -OTDR data are observed. Furthermore, by utilizing the SNR as the feature in a feature-layer algorithm based on Dempster⁻Shafer evidential theory, a four-pulse-width multiplexing ϕ -OTDR system is constructed, and the nuisance-alarm rate is reduced by about 70%. These experimental results show that the proposed method has great potential for perimeter protection, since the nuisance-alarm rate is significantly reduced by using a simple configuration.

Measurement of Unmanned Aerial Vehicle Attitude Angles Based on a Single Captured Image.

The limited load capacity and power resources of small-scale fixed-wing drones mean that it is difficult to employ internal high-precision inertial navigation devices to assist with the landing procedure. As an alternative, this paper proposes an attitude measurement system based on a monocular camera. The attitude angles are obtained from a single captured image containing five coded landmark points using the radial constraint method and three-dimensional coordinate transformations. The landing procedure is simulated for pitch angles from -15 ∘ to -40 ∘ , roll angles from -15 ∘ to +15 ∘ and yaw angles from -15 ∘ to +15 ∘ . For roll and pitch angles of approximately 0 ∘ and -25 ∘ , respectively, the accuracy of the method reaches 0.01 ∘ and 0.04 ∘ . This UAV attitude measurement system obtains an attitude angle by a single captured image, which has great potential for assisting with the landing of small-scale fixed-wing UAVs.

Camera Calibration Robust to Defocus Using Phase-Shifting Patterns.

Camera parameters can't be estimated accurately using traditional calibration methods if the camera is substantially defocused. To tackle this problem, an improved approach based on three phase-shifting circular grating (PCG) arrays is proposed in this paper. Rather than encoding the feature points into the intensity, the proposed method encodes the feature points into the phase distribution, which can be recovered precisely using phase-shifting methods. The PCG centers are extracted as feature points, which can be located accurately even if the images are severely blurred. Unlike the previous method which just uses a single circle, the proposed method uses a concentric circle to estimate the PCG center, such that the center can be located precisely. This paper also presents a sorting algorithm for the detected feature points automatically. Experiments with both synthetic and real images were carried out to validate the performance of the method. And the results show that the superiority of PCG arrays compared with the concentric circle array even under severe defocus.

Modified Gray-Level Coding Method for Absolute Phase Retrieval.

Fringe projection systems have been widely applied in three-dimensional (3D) shape measurements. One of the important issues is how to retrieve the absolute phase. This paper presents a modified gray-level coding method for absolute phase retrieval. Specifically, two groups of fringe patterns are projected onto the measured objects, including three phase-shift patterns for the wrapped phase, and three n-ary gray-level (nGL) patterns for the fringe order. Compared with the binary gray-level (bGL) method which just uses two intensity values, the nGL method can generate many more unique codewords with multiple intensity values. With assistance from the average intensity and modulation of phase-shift patterns, the intensities of nGL patterns are normalized to deal with ambient light and surface contrast. To reduce the codeword detection errors caused by camera/projector defocus, nGL patterns are designed as n-ary gray-code (nGC) patterns to ensure that at most, one code changes at each point. Experiments verify the robustness and effectiveness of the proposed method to measure isolated objects with complex surfaces.

Quantized phase coding and connected region labeling for absolute phase retrieval.

This paper proposes an absolute phase retrieval method for complex object measurement based on quantized phase-coding and connected region labeling. A specific code sequence is embedded into quantized phase of three coded fringes. Connected regions of different codes are labeled and assigned with 3-digit-codes combining the current period and its neighbors. Wrapped phase, more than 36 periods, can be restored with reference to the code sequence. Experimental results verify the capability of the proposed method to measure multiple isolated objects.

Accurate feature detection for out-of-focus camera calibration.

For conventional camera calibration methods, well-focused images are necessary to detect features accurately. However, this requirement causes practical inconveniences to image acquisition for long- and short-distance photogrammetry. In this study, three active phase-shift circular grating (PCG) arrays are used as calibration patterns. The PCGs' centers are regarded as feature points that can be accurately extracted by ellipse fitting of 2π-phase points even though patterns are substantially blurred. In the experiments, Gaussian filters are utilized to blur pattern images, and different standard deviations are set for different fuzzy degrees. Pattern images with different defocusing degrees are also captured. The period and number of PCGs and noise are considered. Experimental results indicate that our method is accurate, reliable, and insensitive to image defocusing.