High dynamic range 3D shape measurement based on crosstalk characteristics of a color camera.

Fringe projection profilometry (FPP) has been widely used in many fields due to its fast speed, high accuracy and full-field characteristics. However, it is still a challenging problem to deal with high dynamic range (HDR) objects for traditional FPP, which utilizes a single exposure time or a single projection intensity. Overexposure will occur in areas with large reflectivity, which exceeds the maximum capturing capacity of camera sensors, resulting in the failure to obtain the accurate intensity, absolute phase and three-dimensional (3D) data. In this paper, a uniform blue image is projected to divide object surface into three areas with different reflectivity by using different intensity responses of RGB channels of color images. Crosstalk coefficient function is applied to obtain intensity of overexposed areas, and then the optimal exposure time of areas is calculated by the linear photometric response of the camera. Finally, three sets of blue fringe patterns with optimal exposure time are synthesized into the fused HDR images to calculate the absolute phase. Experimental results confirm that the proposed method can accurately measure HDR objects with large variation range of reflectivity.

High-accuracy camera calibration method based on coded concentric ring center extraction.

In the field of three-dimensional (3-D) metrology based on fringe projection profilometry (FPP), accurate camera calibration is an essential task and a primary requirement. In order to improve the accuracy of camera calibration, the calibration board or calibration target needs to be manufactured with high accuracy, and the marker points in calibration image require to be positioned with high accuracy. This paper presents an improved camera calibration method by simultaneously optimizing the camera parameters and target geometry. Specifically, a set of regularly distributed target markers with rich coded concentric ring pattern is first displayed on a liquid crystal display (LCD) screen. Then, the sub-pixel edges of all coded bands radial straight lines are automatically located at several positions of the LCD screen. Finally, the sub-pixel edge point set is mapped into parameter space to form a line set, and the intersection of the lines is defined as the center pixel coordinates of each target point to complete the camera calibration. The simulation and experimental results verify that the proposed camera calibration method is feasible and easy to operate, which can essentially eliminate the perspective transformation error to improve the accuracy of camera parameters and target geometry.

B-spline surface based 3D reconstruction method for deflectometry.

In the field of optical three-dimension (3-D) measurement, reconstruction usually is completed by the integration of a two-dimensional (2-D) gradient data set. Position and posture of camera and shape of the surface under test determine the location of gradient data which usually is on quadrilateral grids. This paper proposes a B-spline surface-based 3D reconstruction method for deflectometry, which reconstructs the surface under test with its 2-D gradient data set. The 2-D gradient data set consists of gradient data and the 2-D location of the gradient data in the camera coordinate system. The 2-D gradient data set is first transferred to the cameras' virtual image plane, so it locates on rectangular grids. Then, based on the properties of the B-spline basis function and characteristics of the camera, linear equations are derived to solve control points along the virtual image plane. The solved control points reconstruct the surface under test in the camera coordinate system. The property of the B-spline basis function determines the relationship between the depth of the surface and its derivative. The characteristic of the camera determines the relationship between the depth of the surface and the 2-D location of the gradient data. Meanwhile, the accuracy of the 2-D location can also be improved by the linear equations. Finally, simulated and actual experiments show that the proposed method is accurate and efficient at reconstructing surfaces in deflectometry.

Stereoscopic deflectometry with a curved screen.

Deflectometry has been widely used in topography measurement of specular surface. In deflectometry with a curved screen, the range of the gradient and height field of the measured specular surface can be effectively expanded compared to deflectometry with a plane screen. As stereo deflectometry measures gradient with high accuracy, the specular surface is reconstructed by integrating the gradient. In this paper, a stereo deflectometry with a curved screen is proposed in the aspect of system calibration and measuring principle. A pair of cameras and deflectometric system are calibrated simultaneously to obtain the camera parameters and relationship between two cameras and the curved screen. Then, pairs of rectified fringe patterns are demodulated to obtain two pairs of rectified absolute phase maps and homologous points are searched along epipolar line with the principle of the same normal direction on specular surface. Finally, simulated and actual experiments are carried out and the results show high accuracy and stability.

Three-dimensional reconstruction from a fringe projection system through a planar transparent medium.

A vision measurement system is placed in a protective cover made of a transparent medium to avoid environmental influences. Due to the deflection of light rays on the front and rear surfaces of the transparent medium, the imaging position of an object on the camera target plane is deviated, which makes the traditional vision detection methods based on the triangulation principle produce large measurement errors. This work introduces a three-dimensional (3D) reconstruction method by fringe projection system through a planar transparent medium. We derive the coordinate transformation relationship between a real-object point and a pseudo-object point caused by light refraction based on Snell's law of flat refraction. Based on the relationship, a modified fringe projection method is proposed for unbiased 3D reconstruction. Two experiments, including 3D shape measurement of a white plate with ring markers and 3D shape measurement of a regular spherical object are conducted. The results demonstrate the effectiveness of the proposed method in such measurement environment.

A High-Accuracy Calibration Method for a Telecentric Structured Light System.

We propose a method for accurately calibrating a telecentric structured light system consisting of a camera attached to a bilateral telecentric lens and a pin-hole projector. The proposed method can be split into two parts: axial calibration and transverse calibration. The first part is used for building the relationship between phase and depth by means of a planar plate with ring markers on its surface at several different positions in the measuring volume. The second part is used for establishing the relationship between transverse coordinates and pixel positions with the depth offered by a translation stage and the extracted ring centers. Compared with existing methods that require projector calibration, the proposed method can avoid a propagation of the correspondence error between the camera imaging plane and projector imaging plane, thus increasing calibration accuracy. The calibrated telecentric structured light system is further used for three-dimensional (3D) reconstructions of a planar, a ruled surface, and complex surfaces. Experimental results demonstrate that the proposed system calibration method can be used for accurate 3D measurement.

Phase Target-Based Calibration of Projector Radial Chromatic Aberration for Color Fringe 3D Measurement Systems.

The camera and projector are indispensable hardware parts of a color fringe projection 3D measurement system. Chromatic aberration between different color channels of the projector and camera has an impact on the measurement accuracy of the color fringe projection 3D profile measurement. There are many studies on camera calibration, but the chromatic aberration of the projector remains a question deserving of further investigation. In view of the complex system architecture and theoretical derivation of the traditional projector radial chromatic aberration method, a phase target based on projector radial chromatic aberration measurement and the correction method are proposed in this paper. This method uses a liquid crystal display with a holographic projection film as the phase target. The liquid crystal display sequentially displays red, green, and blue horizontal and vertical sinusoidal fringe images. The projector projects red, green, and blue horizontal and vertical sinusoidal fringe images to the phase target in turn, and calculates the absolute phases of the display fringes and reflection fringes, respectively. Taking the green channel as the reference channel, a phase coordinate system is established based on the phases of the vertical and horizontal directions displayed on the display screen, using the phase of the reflection fringes on the display screen as the ideal phase value of the phase point. Then, the phase coordinate system of the red and blue channels is transferred to the green phase coordinate system to calculate the chromatic aberration of the red-green channels and the blue-green channels, and pre-compensation is conducted. Experimental results prove that this method can measure and calibrate the radial chromatic aberration of the projector without being affected by the image quality of the camera. The correction effect of this method is that the maximum chromatic aberration of the red-green channel decreases from 1.9591/pixel to 0.5759/pixel, and the average chromatic aberration decreases from 0.2555/pixel to 0.1865/pixel. In addition, blue-green channel maximum chromatic aberration decreased from 1.8906/pixel to 0.5938/pixel, and the average chromatic aberration decreased from 0.2347/pixel to 0.1907/pixel. This method can improve the projection quality for fringe projection 3D profile measurement technology.

Practical Cross-Layer Radio Frequency-Based Authentication Scheme for Internet of Things.

The Internet of Things world is in need of practical solutions for its security. Existing security mechanisms for IoT are mostly not implemented due to complexity, budget, and energy-saving issues. This is especially true for IoT devices that are battery powered, and they should be cost effective to be deployed extensively in the field. In this work, we propose a new cross-layer approach combining existing authentication protocols and existing Physical Layer Radio Frequency Fingerprinting technologies to provide hybrid authentication mechanisms that are practically proved efficient in the field. Even though several Radio Frequency Fingerprinting methods have been proposed so far, as a support for multi-factor authentication or even on their own, practical solutions are still a challenge. The accuracy results achieved with even the best systems using expensive equipment are still not sufficient on real-life systems. Our approach proposes a hybrid protocol that can save energy and computation time on the IoT devices side, proportionally to the accuracy of the Radio Frequency Fingerprinting used, which has a measurable benefit while keeping an acceptable security level. We implemented a full system operating in real time and achieved an accuracy of 99.8% for the additional cost of energy, leading to a decrease of only ~20% in battery life.

Infrared phase measuring deflectometry by using defocused binary fringe.

Three-dimensional surface information acquisition of specular objects plays an important role in the fields of automobile industry, aerospace, cultural relic protection, intelligent robotics, equipment manufacturing, and so on. Most of the existing specular surface measurement methods are based on focused sinusoidal fringe patterns, so there are certain requirements for the range of the depth of field (DOF) of the camera on the focus position. However, for many specular surfaces with a large gradient, the tested objects may not always be in the DOF of the camera, so sinusoidal fringe patterns are defocused to be vulnerable to the noise. In this Letter, a new infrared phase measuring deflectometry (PMD) based on defocused binary fringe is proposed that combines a binary fringe defocusing technique and direct PMD. The measurement principle and the corresponding system calibration method are described. The feasibility and measurement accuracy of fringe defocus in specular measurement are studied in principle. The experimental results on several specular objects show that the proposed method can effectively measure specular surfaces out of the DOF of the camera.

High-accuracy projector calibration method for fringe projection profilometry considering perspective transformation.

Camera and projector are the key components of structured light three-dimensional (3-D) measurements, and Digital Light Processing (DLP) projector has been widely used for projecting digital structured light patterns for the measurement. The light projecting of projectors can be modeled as the inverse procedures of camera imaging, and its high-accuracy calibration is still a remaining challenge. Therefore, this paper proposes a novel projector calibration method to improve the calibration accuracy of DLP projector. By fixing the position of the camera and calibration board, this method essentially eliminates the perspective transformation error and effectively avoids the distortion of the extracted marker points. The proposed projector calibration procedures are given as follows: Firstly, the optical axis of the camera is adjusted parallel to the normal of the hollow ring calibration board, and a texture image is captured by the camera; Secondly, the horizontal and vertical fringe patterns with nine different positions and directions are projected onto the calibration board, and nine sets of projected images are taken; Finally, a one-to-one correspondence between the camera and the projector is established, and the projector is accurately calibrated using the phase equivalence. The experimental results show that the proposed projector calibration method is feasible and easy to operate, which can essentially eliminate the perspective transformation error and ensure the competitive accuracy.

Monocular Depth Estimation with Joint Attention Feature Distillation and Wavelet-Based Loss Function.

Depth estimation is a crucial component in many 3D vision applications. Monocular depth estimation is gaining increasing interest due to flexible use and extremely low system requirements, but inherently ill-posed and ambiguous characteristics still cause unsatisfactory estimation results. This paper proposes a new deep convolutional neural network for monocular depth estimation. The network applies joint attention feature distillation and wavelet-based loss function to recover the depth information of a scene. Two improvements were achieved, compared with previous methods. First, we combined feature distillation and joint attention mechanisms to boost feature modulation discrimination. The network extracts hierarchical features using a progressive feature distillation and refinement strategy and aggregates features using a joint attention operation. Second, we adopted a wavelet-based loss function for network training, which improves loss function effectiveness by obtaining more structural details. The experimental results on challenging indoor and outdoor benchmark datasets verified the proposed method's superiority compared with current state-of-the-art methods.

3D shape measurement of diffused/specular surface by combining fringe projection and direct phase measuring deflectometry.

The three-dimensional (3D) data of object surfaces, like a precision machine part, play an important role in the fields of aerospace, automotive industry, augmented reality, heritage preservation, smart city, etc. The existing fringe projection profilometry and deflectometry can only measure the 3D shape of diffused and specular surfaces, respectively. However, there are many components having both diffused and specular surfaces. This paper presents a novel method for measuring the 3D shape of diffused/specular surfaces by combining fringe projection profilometry and direct phase measuring deflectometry. The principle and calibration method of the proposed method are elaborated. Experimental studies are conducted with an artificial diffused/specular step having diffused/specular surfaces to verify the measurement accuracy. The results on several objects show that the proposed method can measure diffused/specular surfaces effectively with certain accuracy. Error sources are also analyzed to improve the measurement accuracy.

Cost-Effective Wearable Indoor Localization and Motion Analysis via the Integration of UWB and IMU.

Wearable indoor localization can now find applications in a wide spectrum of fields, including the care of children and the elderly, sports motion analysis, rehabilitation medicine, robotics navigation, etc. Conventional inertial measurement unit (IMU)-based position estimation and radio signal indoor localization methods based on WiFi, Bluetooth, ultra-wide band (UWB), and radio frequency identification (RFID) all have their limitations regarding cost, accuracy, or usability, and a combination of the techniques has been considered a promising way to improve the accuracy. This investigation aims to provide a cost-effective wearable sensing solution with data fusion algorithms for indoor localization and real-time motion analysis. The main contributions of this investigation are: (1) the design of a wireless, battery-powered, and light-weight wearable sensing device integrating a low-cost UWB module-DWM1000 and micro-electromechanical system (MEMS) IMU-MPU9250 for synchronized measurement; (2) the implementation of a Mahony complementary filter for noise cancellation and attitude calculation, and quaternions for frame rotation to obtain the continuous attitude for displacement estimation; (3) the development of a data fusion model integrating the IMU and UWB data to enhance the measurement accuracy using Kalman-filter-based time-domain iterative compensations; and (4) evaluation of the developed sensor module by comparing it with UWB- and IMU-only solutions. The test results demonstrate that the average error of the integrated module reached 7.58 cm for an arbitrary walking path, which outperformed the IMU- and UWB-only localization solutions. The module could recognize lateral roll rotations during normal walking, which could be potentially used for abnormal gait recognition.

Measurement of the Three-Dimensional Shape of Discontinuous Specular Objects Using Infrared Phase-Measuring Deflectometry.

Phase-measuring deflectometry (PMD)-based methods have been widely used in the measurement of the three-dimensional (3D) shape of specular objects, and the existing PMD methods utilize visible light. However, specular surfaces are sensitive to ambient light. As a result, the reconstructed 3D shape is affected by the external environment in actual measurements. To overcome this problem, an infrared PMD (IR-PMD) method is proposed to measure specular objects by directly establishing the relationship between absolute phase and depth data for the first time. Moreover, the proposed method can measure discontinuous surfaces. In addition, a new geometric calibration method is proposed by combining fringe projection and fringe reflection. The proposed IR-PMD method uses a projector to project IR sinusoidal fringe patterns onto a ground glass, which can be regarded as an IR digital screen. The IR fringe patterns are reflected by the measured specular surfaces, and the deformed fringe patterns are captured by an IR camera. A multiple-step phase-shifting algorithm and the optimum three-fringe number selection method are applied to the deformed fringe patterns to obtain wrapped and unwrapped phase data, respectively. Then, 3D shape data can be directly calculated by the unwrapped phase data on the screen located in two positions. The results here presented validate the effectiveness and accuracy of the proposed method. It can be used to measure specular components in the application fields of advanced manufacturing, automobile industry, and aerospace industry.

Radio Frequency Identification and Sensing Techniques and Their Applications-A Review of the State-of-the-Art.

Radio Frequency Identification (RFID) sensors, integrating the features of Wireless Information and Power Transfer (WIPT), object identification and energy efficient sensing capabilities, have been considered a new paradigm of sensing and communication for the futuristic information systems. RFID sensor tags featuring contactless sensing, wireless information transfer, wireless powered, light weight, non-line-of-sight transmission, flexible and pasteable are a critical enabling technology for future Internet-of-Things (IoT) applications, such as manufacturing, logistics, healthcare, agriculture and food. They have attracted numerous research efforts due to their innovative potential in the various application fields. However, there has been a gap between the in-lab investigations and the practical IoT application scenarios, which has motivated this survey of this research to identify the promising enabling techniques and the underlying challenges. This study aims to provide an exhaustive review on the state-of-art RFID sensor technologies from the system implementation perspective by focusing on the fundamental RF energy harvesting theories, the recent technical progresses and commercial solutions, innovative applications and some RFID sensor based IoT solutions, identify the underlying technological challenges at the time being, and give the future research trends and promising application fields in the rich sensing applications of the forthcoming IoT era.

Calibration of the Relative Orientation between Multiple Depth Cameras Based on a Three-Dimensional Target.

Depth cameras play a vital role in three-dimensional (3D) shape reconstruction, machine vision, augmented/virtual reality and other visual information-related fields. However, a single depth camera cannot obtain complete information about an object by itself due to the limitation of the camera's field of view. Multiple depth cameras can solve this problem by acquiring depth information from different viewpoints. In order to do so, they need to be calibrated to be able to accurately obtain the complete 3D information. However, traditional chessboard-based planar targets are not well suited for calibrating the relative orientations between multiple depth cameras, because the coordinates of different depth cameras need to be unified into a single coordinate system, and the multiple camera systems with a specific angle have a very small overlapping field of view. In this paper, we propose a 3D target-based multiple depth camera calibration method. Each plane of the 3D target is used to calibrate an independent depth camera. All planes of the 3D target are unified into a single coordinate system, which means the feature points on the calibration plane are also in one unified coordinate system. Using this 3D target, multiple depth cameras can be calibrated simultaneously. In this paper, a method of precise calibration using lidar is proposed. This method is not only applicable to the 3D target designed for the purposes of this paper, but it can also be applied to all 3D calibration objects consisting of planar chessboards. This method can significantly reduce the calibration error compared with traditional camera calibration methods. In addition, in order to reduce the influence of the infrared transmitter of the depth camera and improve its calibration accuracy, the calibration process of the depth camera is optimized. A series of calibration experiments were carried out, and the experimental results demonstrated the reliability and effectiveness of the proposed method.

Single-shot 3D shape measurement of discontinuous objects based on a coaxial fringe projection system.

Fringe projection profilometry has been widely used in high-speed three-dimensional (3D) shape measurement. To improve the speed without loss of accuracy, we present a novel single-shot 3D shape measuring system that utilizes a coaxial fringe projection system and a 2CCD camera. The coaxial fringe projection system, comprising a visible light (red, green, and blue) projector and an infrared (IR) light projector, can simultaneously project red, green, blue, and IR fringe patterns. The 2CCD camera, as the name suggests, has two CCD chips that can acquire visible and IR fringe patterns at the same time. Combining the two-step phase-shifting algorithm, Fourier transform profilometry, and the optimum three-frequency selection method, 3D shape measurement of complex surfaces such as large slopes or discontinuous objects can be obtained from single-shot acquisition. A virtual fringe projection measurement system has been established to generate pre-deformed fringe patterns to correct positional deviations of the coaxial fringe projection system. This method has been applied to simulations and experiments on static and dynamic objects with promising results.

Discovery of new quinazoline derivatives as irreversible dual EGFR/HER2 inhibitors and their anticancer activities - Part 1.

Overexpression of EGFR and HER2 are observed in many breast, ovarian, colon and prostate cancers. The second and third generation irreversible EGFR/HER2 dual kinase inhibitors became popular after the approval of Afatinib by FDA to overcome the mutation related problem. To find efficacious drug candidates, a series of novel quinazoline derivatives were designed, synthesized and evaluated as dual EGFR/HER2 tyrosine kinase (TK) inhibitors. Selected twenty four compounds were reported here with significant inhibitory activities against EGFR/HER2 tyrosine kinases. Several compounds showed nanomolar IC50 values. In vitro studies of quinazoline derivatives were done on NCI-H1975, HCC827, A431, MDA MB-453 cell lines. The compounds 1a, 1d and 1v were found more potent compared to standard drug afatinib. In vivo efficacy study of 1d on nude mice NCI-H1975 tumour xenograft model was discussed.

3D shape measurement of discontinuous specular objects based on advanced PMD with bi-telecentric lens.

This paper presents an advanced phase measuring deflectometry (PMD) method based on a novel mathematical model to obtain three dimensional (3D) shape of discontinuous specular object using a bi-telecentric lens. The proposed method uses an LCD screen, a flat beam splitter, a camera with a bi-telecentric lens, and a translating stage. The LCD screen is used to display sinusoidal fringe patterns and can be moved by the stage to two different positions along the normal direction of a reference plane. The camera captures the deformed fringe patterns reflected by the measured specular surface. The splitter realizes the fringe patterns displaying and imaging from the same direction. Using the proposed advanced PMD method, the depth data can be directly calculated from absolute phase, instead of integrating gradient data. In order to calibrate the relative orientation of the LCD screen and the camera, an auxiliary plane mirror is used to reflect the pattern on the LCD screen three times. After the geometric calibration, 3D shape data of the measured specular objects are calculated from the phase differences between the reference plane and the reflected surface. The experimental results show that 3D shape of discontinuous specular object can be effectively and accurately measured from absolute phase data by the proposed advanced PMD method.

Distance Calibration between Reference Plane and Screen in Direct Phase Measuring Deflectometry.

The recently developed direct phase measuring deflectometry (DPMD) method can directly measure the three-dimensional (3D) shape of specular objects with discontinuous surfaces, but requires a calibrated distance between a reference plane and liquid crystal display screen. Because the plane and screen are different distances from the imaging device, they cannot be clearly captured given the limited depth of field (DOF) of the lens. Therefore, existing machine vision-based methods cannot be used to effectively calibrate a DPMD system. In this paper, a new distance calibration method that uses a mirror with a hollow ring matrix pattern and a mobile stage is presented. The direction of the mobile stage in the camera coordinate system is determined by the mirror's pattern at several positions in the camera's DOF so that the reference position outside of the DOF can be calculated. The screen's position can also be calibrated by displaying patterns at a known scale. Therefore, the required distance is accurately obtained in the camera coordinate system. Evaluation results show that the maximum value of the absolute error is less than 0.031 mm. The experimental results on an artificial stepped mirror and a reflected diamond distribution surface demonstrate the accuracy and practicality of the proposed method.