Photometric-Stereo-Based Defect Detection System for Metal Parts.

Automated inspection technology based on computer vision is now widely used in the manufacturing industry with high speed and accuracy. However, metal parts always appear in high gloss or shadow on the surface, resulting in the overexposure of the captured images. It is necessary to adjust the light direction and view to keep defects out of overexposure and shadow areas. However, it is too tedious to adjust the position of the light direction and view the variety of parts' geometries. To address this problem, we design a photometric-stereo-based defect detection system (PSBDDS), which combines the photometric stereo with defect detection to eliminate the interference of highlights and shadows. Based on the PSBDDS, we introduce a photometric-stereo-based defect detection framework, which takes images captured in multiple directional lights as input and obtains the normal map through the photometric stereo model. Then, the detection model uses the normal map as input to locate and classify defects. Existing learning-based photometric stereo methods and defect detection methods have achieved good performance in their respective fields. However, photometric stereo datasets and defect detection datasets are not sufficient for training and testing photometric-stereo-based defect detection methods, thus we create a photometric stereo defect detection (PSDD) dataset using our PSBDDS to eliminate gaps between learning-based photometric stereo and defect detection methods. Furthermore, experimental results prove the effectiveness of the proposed PSBBD and PSDD dataset.

Computational framework for steady-state NLOS localization under changing ambient illumination conditions.

Non-line-of-sight (NLOS) imaging of hidden objects is a challenging yet vital task, facilitating important applications such as rescue operations, medical imaging, and autonomous driving. In this paper, we attempt to develop a computational steady-state NLOS localization framework that works accurately and robustly under various illumination conditions. For this purpose, we build a physical NLOS image acquisition hardware system and a corresponding virtual setup to obtain real-captured and simulated steady-state NLOS images under different ambient illuminations. Then, we utilize the captured NLOS images to train/fine-tune a multi-task convolutional neural network (CNN) architecture to perform simultaneous background illumination correction and NLOS object localization. Evaluation results on both stimulated and real-captured NLOS images demonstrate that the proposed method can effectively suppress severe disturbance caused by the variation of ambient light, significantly improving the accuracy and stability of steady-state NLOS localization using consumer-grade RGB cameras. The proposed method potentially paves the way to develop practical steady-state NLOS imaging solutions for around-the-clock and all-weather operations.

Data-driven framework for high-accuracy color restoration of RGBN multispectral filter array sensors under extremely low-light conditions.

RGBN multispectral filter array provides a cost-effective and one-shot acquisition solution to capture well-aligned RGB and near-infrared (NIR) images which are useful for various optical applications. However, signal responses of the R, G, B channels are inevitably distorted by the undesirable spectral crosstalk of the NIR bands, thus the captured RGB images are adversely desaturated. In this paper, we present a data-driven framework for effective spectral crosstalk compensation of RGBN multispectral filter array sensors. We set up a multispectral image acquisition system to capture RGB and NIR image pairs under various illuminations which are subsequently utilized to train a multi-task convolutional neural network (CNN) architecture to perform simultaneous noise reduction and color restoration. Moreover, we present a technique for generating high-quality reference images and a task-specific joint loss function to facilitate the training of the proposed CNN model. Experimental results demonstrate the effectiveness of the proposed method, outperforming the state-of-the-art color restoration solutions and achieving more accurate color restoration results for desaturated and noisy RGB images captured under extremely low-light conditions.

Serum Klotho, Cardiovascular Events, and Mortality in Nondiabetic Chronic Kidney Disease.

BACKGROUND: Experimental studies indicate that Klotho deficiency is a pathogenic factor for CKD-related complications, including cardiovascular disease (CVD). However, the association between serum Klotho and clinical outcomes in nondiabetic CKD patients needs to be further clarified. We aimed to determine whether serum Klotho levels are associated with CVD events and mortality in predialysis CKD patients without diabetes. METHODS: A total of 336 CKD stage 2-5 predialysis patients without diabetes were recruited and followed from the end of 2014 to January 2019 for CVD events and overall mortality. Serum Klotho was detected by ELISA and divided into quartiles (lowest, middle, second highest, and highest quartiles) according to their serum Klotho category. RESULTS: After a median follow-up of 3.52 years (IQR 3.34-3.76), Kaplan-Meier analysis showed that, compared to participants with a Klotho level in the highest quartile (the reference category), those in the lowest Klotho quartile were associated with a higher all-cause mortality risk (HR = 7.05; 95% CI 1.59-31.25) and a higher CVD event risk (HR = 3.02; 95% CI 1.45-6.30). In addition, the middle Klotho quartile was also associated with CVD event risk (HR = 2.56; 95% CI 1.21-5.41). Moreover, in the multivariate-adjusted model, the lowest Klotho quartile remained significantly associated with all-cause mortality (HR = 5.17; 95% CI 1.07-24.96), and the middle Klotho quartile maintained a significant association with CVD event risk (HR = 2.32; 95% CI 1.03-5.21). CONCLUSION: These results suggest that lower serum Klotho levels are independently associated with overall mortality and CVD events in nondiabetic predialysis CKD patients.

Multi-channel residual network model for accurate estimation of spatially-varying and depth-dependent defocus kernels.

Digital projectors have been increasingly utilized in various commercial and scientific applications. However, they are prone to the out-of-focus blurring problem since their depth-of-fields are typically limited. In this paper, we explore the feasibility of utilizing a deep learning-based approach to analyze the spatially-varying and depth-dependent defocus properties of digital projectors. A multimodal displaying/imaging system is built for capturing images projected at various depths. Based on the constructed dataset containing well-aligned in-focus, out-of-focus, and depth images, we propose a novel multi-channel residual deep network model to learn the end-to-end mapping function between the in-focus and out-of-focus image patches captured at different spatial locations and depths. To the best of our knowledge, it is the first research work revealing that the complex spatially-varying and depth-dependent blurring effects can be accurately learned from a number of real-captured image pairs instead of being hand-crafted as before. Experimental results demonstrate that our proposed deep learning-based method significantly outperforms the state-of-the-art defocus kernel estimation techniques and thus leads to better out-of-focus compensation for extending the dynamic ranges of digital projectors.

On-the-fly extrinsic calibration of multimodal sensing system for fast 3D thermographic scanning.

The fusion of three-dimensional (3D) geometrical and two-dimensional (2D) thermal information provides a promising method for characterizing temperature distribution of 3D objects, extending infrared imaging from 2D to 3D to support various thermal inspection applications. In this paper, we present an effective on-the-fly calibration approach for accurate alignment of depth and thermal data to facilitate dynamic and fast-speed 3D thermal scanning tasks. For each pair of depth and thermal frames, we estimate their relative pose by minimizing the objective function that measures the temperature consistency between a 2D infrared image and the reference 3D thermographic model. Our proposed frame-to-model mapping scheme can be seamlessly integrated into a generic 3D thermographic reconstruction framework. Through graphics-processing-unit-based acceleration, our method requires less than 10 ms to generate a pair of well-aligned depth and thermal images without hardware synchronization and improves the robustness of the system against significant camera motion.

Exploiting fusion architectures for multispectral pedestrian detection and segmentation.

Recent research has demonstrated that the fusion of complementary information captured by multi-modal sensors (visible and infrared cameras) enables robust pedestrian detection under various surveillance situations (e.g., daytime and nighttime). In this paper, we investigate a number of fusion architectures in an attempt to identify the optimal way of incorporating multispectral information for joint semantic segmentation and pedestrian detection. We made two important findings: (1) the sum fusion strategy, which computes the sum of two feature maps at the same spatial locations, delivers the best performance of multispectral detection, while the most commonly used concatenation fusion surprisingly performs the worst; and (2) two-stream semantic segmentation without multispectral fusion is the most effective scheme to infuse semantic information as supervision for learning human-related features. Based on these studies, we present a unified multispectral fusion framework for joint training of semantic segmentation and target detection that outperforms state-of-the-art multispectral pedestrian detectors by a large margin on the KAIST benchmark dataset.

Single-image-based nonuniformity correction of uncooled long-wave infrared detectors: a deep-learning approach.

Fixed-pattern noise (FPN), which is caused by the nonuniform opto-electronic responses of microbolometer focal-plane-array (FPA) optoelectronics, imposes a challenging problem in infrared imaging systems. In this paper, we successfully demonstrate that a better single-image-based non-uniformity correction (NUC) operator can be directly learned from a large number of simulated training images instead of being handcrafted as before. Our proposed training scheme, which is based on convolutional neural networks (CNNs) and a column FPN simulation module, gives rise to a powerful technique to reconstruct the noise-free infrared image from its corresponding noisy observation. Specifically, a comprehensive column FPN model is utilized to depict the nonlinear characteristics of column amplifiers in the readout circuit of FPA. A large number of high-fidelity training images are simulated based on this model and the end-to-end residual deep network is capable of learning the intrinsic difference between undesirable FPN and original image details. Therefore, column FPN can be accurately estimated and further subtracted from the raw infrared images to obtain NUC results. Comparative results with state-of-the-art single-image-based NUC methods, using real-captured noisy infrared images, demonstrate that our proposed deep-learning-based approach delivers better performances of FPN removal, detail preservation, and artifact suppression.

Depth and thermal sensor fusion to enhance 3D thermographic reconstruction.

Three-dimensional geometrical models with incorporated surface temperature data provide important information for various applications such as medical imaging, energy auditing, and intelligent robots. In this paper we present a robust method for mobile and real-time 3D thermographic reconstruction through depth and thermal sensor fusion. A multimodal imaging device consisting of a thermal camera and a RGB-D sensor is calibrated geometrically and used for data capturing. Based on the underlying principle that temperature information remains robust against illumination and viewpoint changes, we present a Thermal-guided Iterative Closest Point (T-ICP) methodology to facilitate reliable 3D thermal scanning applications. The pose of sensing device is initially estimated using correspondences found through maximizing the thermal consistency between consecutive infrared images. The coarse pose estimate is further refined by finding the motion parameters that minimize a combined geometric and thermographic loss function. Experimental results demonstrate that complimentary information captured by multimodal sensors can be utilized to improve performance of 3D thermographic reconstruction. Through effective fusion of thermal and depth data, the proposed approach generates more accurate 3D thermal models using significantly less scanning data.

Mathematical model of cylindrical form tolerance.

Tolerance is essential for integration of CAD and CAM. Unfortunately, the meaning of tolerances in the national standard is expressed in graphical and language forms and is not adaptable for expression, processing and data transferring with computers. How to interpret its semantics is becoming a focus of relevant studies. This work based on the mathematical definition of form tolerance in ANSI Y14.5.1M-1994, established the mathematical model of form tolerance for cylindrical feature. First, each tolerance in the national standard was established by vector equation. Then on the foundation of tolerance's mathematical definition theory, each tolerance zone's mathematical model was established by inequality based on degrees of feature. At last the variance area of each tolerance zone is derived. This model can interpret the semantics of form tolerance exactly and completely.

A robust tolerance design method based on process capability.

This paper presents a method for robust tolerance design in terms of Process Capability Indices (PCI). The component tolerance and the suitable manufacturing processes can be selected based on the real manufacturing context. The robustness of design feasibility under the effect of uncertainties is also discussed. A comparison between the results obtained by the proposed model and other methods indicates that robust and reliable tolerance can be obtained.