SEpi-3D: soft epipolar 3D shape measurement with an event camera for multipath elimination.

Multipath in 3D imaging happens when one pixel receives light from multiple reflections, which causes errors in the measured point cloud. In this paper, we propose the soft epipolar 3D(SEpi-3D) method to eliminate multipath in temporal space with an event camera and a laser projector. Specifically, we align the projector and event camera row onto the same epipolar plane with stereo rectification; we capture event flow synchronized with the projector frame to construct a mapping relationship between event timestamp and projector pixel; we develop a multipath eliminating method that utilizes the temporal information from the event data together with the epipolar geometry. Experiments show that the RMSE decreases by 6.55mm on average in the tested multipath scenes, and the percentage of error points decreases by 7.04%.

A Feature-Trajectory-Smoothed High-Speed Model for Video Anomaly Detection.

High-speed detection of abnormal frames in surveillance videos is essential for security. This paper proposes a new video anomaly-detection model, namely, feature trajectory-smoothed long short-term memory (FTS-LSTM). This model trains an LSTM autoencoder network to generate future frames on normal video streams, and uses the FTS detector and generation error (GE) detector to detect anomalies on testing video streams. FTS loss is a new indicator in the anomaly-detection area. In the training stage, the model applies a feature trajectory smoothness (FTS) loss to constrain the LSTM layer. This loss enables the LSTM layer to learn the temporal regularity of video streams more precisely. In the detection stage, the model utilizes the FTS loss and the GE loss as two detectors to detect anomalies. By cascading the FTS detector and the GE detector to detect anomalies, the model achieves a high speed and competitive anomaly-detection performance on multiple datasets.

Hierarchical Vision Navigation System for Quadruped Robots with Foothold Adaptation Learning.

Legged robots can travel through complex scenes via dynamic foothold adaptation. However, it remains a challenging task to efficiently utilize the dynamics of robots in cluttered environments and to achieve efficient navigation. We present a novel hierarchical vision navigation system combining foothold adaptation policy with locomotion control of the quadruped robots. The high-level policy trains an end-to-end navigation policy, generating an optimal path to approach the target with obstacle avoidance. Meanwhile, the low-level policy trains the foothold adaptation network through auto-annotated supervised learning to adjust the locomotion controller and to provide more feasible foot placement. Extensive experiments in both simulation and the real world show that the system achieves efficient navigation against challenges in dynamic and cluttered environments without prior information.

OMC-SLIO: Online Multiple Calibrations Spinning LiDAR Inertial Odometry.

Light detection and ranging (LiDAR) is often combined with an inertial measurement unit (IMU) to get the LiDAR inertial odometry (LIO) for robot localization and mapping. In order to apply LIO efficiently and non-specialistically, self-calibration LIO is a hot research topic in the related community. Spinning LiDAR (SLiDAR), which uses an additional rotating mechanism to spin a common LiDAR and scan the surrounding environment, achieves a large field of view (FoV) with low cost. Unlike common LiDAR, in addition to the calibration between the IMU and the LiDAR, the self-calibration odometer for SLiDAR must also consider the mechanism calibration between the rotating mechanism and the LiDAR. However, existing self-calibration LIO methods require the LiDAR to be rigidly attached to the IMU and do not take the mechanism calibration into account, which cannot be applied to the SLiDAR. In this paper, we propose firstly a novel self-calibration odometry scheme for SLiDAR, named the online multiple calibration inertial odometer (OMC-SLIO) method, which allows online estimation of multiple extrinsic parameters among the LiDAR, rotating mechanism and IMU, as well as the odometer state. Specially, considering that the rotating and static parts of the motor encoder inside the SLiDAR are rigidly connected to the LiDAR and IMU respectively, we formulate the calibration within the SLiDAR as two separate sets of calibrations: the mechanism calibration between the LiDAR and the rotating part of the motor encoder and the sensor calibration between the static part of the motor encoder and the IMU. Based on such a SLiDAR calibration formulation, we can construct a well-defined kinematic model from the LiDAR to the IMU with the angular information from the motor encoder. Based on the kinematic model, a two-stage motion compensation method is presented to eliminate the point cloud distortion resulting from LiDAR spinning and platform motion. Furthermore, the mechanism and sensor calibration as well as the odometer state are wrapped in a measurement model and estimated via an error-state iterative extended Kalman filter (ESIEKF). Experimental results show that our OMC-SLIO is effective and attains excellent performance.

The association between ECG criteria and Echo criteria for left ventricular hypertrophy in a general Chinese population.

BACKGROUND: Several ECG criteria have been widely used for diagnosis of left ventricular hypertrophy (LVH) in clinical practice. However, their performance in a general Chinese population is limited. METHODS AND RESULTS: A multi-stage, stratified cluster sampling across China was performed and 7415 representative Chinese adults aged 18-85 years were analyzed. ECG was collected by using GE MAC 5500 machine. The association between five ECG-LVH criteria (i.e., Peguero-Lo Presti, Cornell, Cornell product, Sokolow-Lyon and Sokolow-Lyon product) and echocardiographic LVH (Echo-LVH) was assessed by Pearson's correlation, diagnostic statistics like predictive values, and receiver operating characteristics (ROC) curve. We found that the prevalence of the Echo-LVH was 11% while ECG-LVH ranged from 3% to 27%. All ECG-LVH criteria had high negative predictive value (NPV) (89%) and specificity (73-96%) but low positive predictive value (PPV) (12-24%) and sensitivity (4-29%). The newly Peguero-Lo Presti criteria had higher sensitivity (29%) but lower specificity (73%) and accuracy (68%) compared with other criteria. Cornell product had the best diagnostic performance (AUC: 0.59), as well as the highest specificity (96%) and accuracy (86%) but lowest sensitivity (4%). Among single-lead components of ECG criteria, RaVL voltage and QRS duration performed relatively better than others. Hypertensive and older individuals had higher sensitivity but lower specificity and accuracy than their counterparts. CONCLUSION: ECG-LVH criteria had high NPV to detect Echo-LVH. Though with higher sensitivity, Peguero-Lo Presti criteria did not have better diagnostic performance to detect Echo-LVH. RaVL and QRS duration had stronger association with Echo-LVH among all single-lead components.

Large depth-of-field three-dimensional shape measurement with the focal sweep technique.

Three-dimensional (3D) shape measurement based on the fringe projection technique has been extensively used for scientific discoveries and industrial practices. Yet, one of the most challenging issues is its limited depth of field (DOF). This paper presents a method to drastically increase DOF of 3D shape measurement technique by employing the focal sweep method. The proposed method employs an electrically tunable lens (ETL) to rapidly sweep the focal plane during image integration and the post deconvolution algorithm to reconstruct focused images for 3D reconstruction. Experimental results demonstrated that our proposed method can achieve high-resolution and high-accuracy 3D shape measurement with greatly improved DOF in real time.

Large depth-of-field 3D shape measurement using an electrically tunable lens.

The state-of-the-art 3D shape measurement system has rather shallow working volume due to the limited depth-of-field (DOF) of conventional lens. In this paper, we propose to use the electrically tunable lens to substantially enlarge the DOF. Specifically, we capture always in-focus phase-shifted fringe patterns by precisely synchronizing the tunable lens attached to the camera with the image acquisition and the pattern projection; we develop a phase unwrapping framework that fully utilizes the geometric constraint from the camera focal length setting; and we pre-calibrate the system under different focal distance to reconstruct 3D shape from unwrapped phase map. To validate the proposed idea, we developed a prototype system that can perform high-quality measurement for the depth range of approximately 1,000 mm (400 mm - 1400 mm) with the measurement error of 0.05%. Furthermore, we demonstrated that such a technique can be used for real-time 3D shape measurement by experimentally measuring moving objects.

Guided, Fusion-Based, Large Depth-of-field 3D Imaging Using a Focal Stack.

Three dimensional (3D) imaging technology has been widely used for many applications, such as human-computer interactions, making industrial measurements, and dealing with cultural relics. However, existing active methods often require both large apertures of projector and camera to maximize light throughput, resulting in a shallow working volume in which projector and camera are simultaneously in focus. In this paper, we propose a novel method to extend the working range of the structured light 3D imaging system based on the focal stack. Specifically in the case of large depth variation scenes, we first adopted the gray code method for local, 3D shape measurement with multiple focal distance settings. Then we extracted the texture map of each focus position into a focal stack to generate a global coarse depth map. Under the guidance of the global coarse depth map, the high-quality 3D shape measurement of the overall scene was obtained by local, 3D shape-measurement fusion. To validate the method, we developed a prototype system that can perform high-quality measurements in the depth range of 400 mm with a measurement error of 0.08%.

MFA-Net: Motion Feature Augmented Network for Dynamic Hand Gesture Recognition from Skeletal Data.

Dynamic hand gesture recognition has attracted increasing attention because of its importance for human⁻computer interaction. In this paper, we propose a novel motion feature augmented network (MFA-Net) for dynamic hand gesture recognition from skeletal data. MFA-Net exploits motion features of finger and global movements to augment features of deep network for gesture recognition. To describe finger articulated movements, finger motion features are extracted from the hand skeleton sequence via a variational autoencoder. Global motion features are utilized to represent the global movements of hand skeleton. These motion features along with the skeleton sequence are then fed into three branches of a recurrent neural network (RNN), which augment the motion features for RNN and improve the classification performance. The proposed MFA-Net is evaluated on two challenging skeleton-based dynamic hand gesture datasets, including DHG-14/28 dataset and SHREC'17 dataset. Experimental results demonstrate that our proposed method achieves comparable performance on DHG-14/28 dataset and better performance on SHREC'17 dataset when compared with start-of-the-art methods.

High-performance color sequence particle streak velocimetry for 3D airflow measurement.

In this paper, a high-performance color sequence particle streak velocimetry (CSPSV) technique is proposed to measure the air velocity in a large three-dimensional (3D) space. Based on the basic principle of CSPSV, a new color sequence illumination pattern is designed to mark seeding bubbles for better imaging performance. Synchronized with the illumination system, cameras record the targets' paths at the start, middle, and end points with different color information more clearly. Thus, a rectification-based stereo corresponding algorithm is presented to reconstruct the 3D trajectory of the bubbles. The accuracy of this system is verified and shows good consistency with a hot-wire anemometer (the principal research tool for turbulent-flow studies). The vortex test experiments also indicate its capability for complex airflow. Our high-performance CSPSV can extend the 3D measurement zone from several cubic centimeters to several cubic meters with regular, off-the-shelf cameras.

Large aperture focus stacking with max-gradient flow by anchored rolling filtering.

Focus stacking is a computational technique to extend the depth of field through combining multiple images taken at various focus distances. However, in the large aperture case, there are always defects caused by the large blur scale, which, to the best of our knowledge, has not been well studied. In our work, we propose a max-gradient flow-based method to reduce artifacts and obtain a high-quality all-in-focus image by anchored rolling filtering. First, we define a max-gradient flow to describe the gradient propagation in the stack. The points are divided into trivial and source points with this flow. The source points are extracted as true edge points and are utilized as anchors to refine the depth map and the composited all-in-focus image iteratively. The experiments show that our method can effectively suppress the incorrect depth estimations and give a high-quality all-in-focus image.

Iteratively reconstructing 4D light fields from focal stacks.

Acquiring and representing the 4D space of rays in the world (the light field) is important for many computer vision and graphics applications. In this paper, we propose an iterative method to acquire the 4D light field from a focal stack. First, a discrete refocusing equation is derived from integral imaging principles. With this equation, a linear projection system is formulated to model the focal stack imaging process. Then we reconstruct the 4D light field from the focal stack through solving the inverse problem with a filtering-based iterative method. The experimental results show that our approach is effective and outperforms state-of-the-art methods in reconstruction accuracy, reduced sampling, and occluded boundaries.

DeGCN: Deformable Graph Convolutional Networks for Skeleton-Based Action Recognition.

Graph convolutional networks (GCN) have recently been studied to exploit the graph topology of the human body for skeleton-based action recognition. However, most of these methods unfortunately aggregate messages via an inflexible pattern for various action samples, lacking the awareness of intra-class variety and the suitableness for skeleton sequences, which often contain redundant or even detrimental connections. In this paper, we propose a novel Deformable Graph Convolutional Network (DeGCN) to adaptively capture the most informative joints. The proposed DeGCN learns the deformable sampling locations on both spatial and temporal graphs, enabling the model to perceive discriminative receptive fields. Notably, considering human action is inherently continuous, the corresponding temporal features are defined in a continuous latent space. Furthermore, we design an innovative multi-branch framework, which not only strikes a better trade-off between accuracy and model size, but also elevates the effect of ensemble between the joint and bone modalities remarkably. Extensive experiments show that our proposed method achieves state-of-the-art performances on three widely used datasets, NTU RGB+D, NTU RGB+D 120, and NW-UCLA.

Depth estimation for speckle projection system using progressive reliable points growing matching.

In this paper, we propose a progressive reliable points growing matching scheme to estimate the depth from the speckle projection image. First a self-adapting binarization is introduced to reduce the influence of inconsistent intensity. Then we apply local window-based correlation matching to get the initial disparity map. After the initialization, we formulate a progressive updating scheme to update the disparity estimation. There are two main steps in each round of updation. At first new reliable points are progressively selected based on three aspects of criterion including matching degree, confidence, and left-right consistency; then prediction-based growing matching is adopted to recalculate the disparity map from the reliable points. Finally, the more accurate depth map can be obtained by subpixel interpolation and transformation. The experimental results well demonstrate the effectiveness and low computational cost of our scheme.

A Dual-Scale Lead-Separated Transformer for ECG Classification.

This work proposes a novel dual-scale lead-separated transformer for the auxiliary diagnosis of 12-lead electrocardiograms (ECGs). We added a new structure design on the basis of traditional ECG signal processing, which led to our model with only 2.6M parameters. The output of the system is the classification results. The fixed 0.5 second ECG segments of each lead are interpreted as independent patches. Together with the reduced dimension signal, patches form a dual-scale representation. As a method to reduce interference from segments with low correlation, a lead-orthogonal attention module is proposed. Experimental results show the effectiveness and scalability of our model.Clinical relevance- Our method improves the scores of clinical 12-lead ECG classification and shows generalization ability. Our model is suitable for single-label and multi-label classification tasks on clinical 12-lead ECG and is compatible with single lead classification. The integration of clinical information can further improve the effectiveness of the model.

MVKT-ECG: Efficient single-lead ECG classification for multi-label arrhythmia by multi-view knowledge transferring.

Electrocardiogram (ECG) is a widely used technique for diagnosing cardiovascular disease. The widespread emergence of smart ECG devices has sparked the demand for intelligent single-lead ECG-based diagnostic systems. However, it is challenging to develop a single-lead-based ECG interpretation model for multiple disease diagnosis due to the lack of some key disease information. We aim to improve the diagnostic capabilities of single-lead ECG for multi-label disease classification in a new teacher-student manner, where the teacher trained by multi-lead ECG educates a student who observes only single-lead ECG We present a new disease-aware Contrastive Lead-information Transferring (CLT) to improve the mutual disease information between the single-lead-based ECG interpretation model and multi-lead-based ECG interpretation model. Moreover, We modify the traditional Knowledge Distillation into Multi-label disease Knowledge Distillation (MKD) to make it applicable for multi-label disease diagnosis. The whole knowledge transferring process is inter-lead Multi-View Knowledge Transferring of ECG (MVKT-ECG). By employing the training strategy, we can effectively transfer comprehensive disease knowledge from various views of ECG, such as the 12-lead ECG, to a single-lead-based ECG interpretation model. This enables the model to extract intricate details from single-lead ECG signals and enhances the model's capability of diagnosing and identifying single-lead signals. Extensive experiments on two commonly used public multi-label datasets, ICBEB2018 and PTB-XL demonstrate that our MVKT-ECG yields exceptional diagnostic performance improvements for single-lead ECG. The student outperforms its baseline observably on the PTB-XL dataset (1.3 % on PTB.super, and 1.4 % on PTB.sub), and on ICBEB2018 dataset (3.2 %).

A community effort to assess and improve computerized interpretation of 12-lead resting electrocardiogram.

Computerized interpretation of electrocardiogram plays an important role in daily cardiovascular healthcare. However, inaccurate interpretations lead to misdiagnoses and delay proper treatments. In this work, we built a high-quality Chinese 12-lead resting electrocardiogram dataset with 15,357 records, and called for a community effort to improve the performances of CIE through the China ECG AI Contest 2019. This dataset covers most types of ECG interpretations, including the normal type, 8 common abnormal types, and the other type which includes both uncommon abnormal and noise signals. Based on the Contest, we systematically assessed and analyzed a set of top-performing methods, most of which are deep neural networks, with both their commonalities and characteristics. This study establishes the benchmarks for computerized interpretation of 12-lead resting electrocardiogram and provides insights for the development of new methods. Graphical Abstract A community effort to assess and improve computerized interpretation of 12-lead resting electrocardiogram.

Improving Automatic Detection of ECG Abnormality with Less Manual Annotations using Siamese Network.

Electrocardiography is a very common, non-invasive diagnostic procedure and its interpretation is increasingly supported by automatic interpretation algorithms. Recently many works also focused on the design of automatic ECG abnormality detection algorithms. However, clinical electrocardiogram datasets often suffer from their heavy needs for expert annotations, which are often expensive and hard to obtain. In this work, we proposed a weakly supervised pretraining method based on the Siamese neural network, which utilizes the original diagnostic information written by physicians to produce useful feature representations of the ECG signal which improves performance of ECG abnormality detection algorithms with fewer expert annotations. The experiment showed that with the proposed weekly supervised pretraining, the performance of ECG abnormality detection algorithms that was trained with only 1/8 annotated ECG data outperforms classical models that was trained with fully annotated ECG data, which implies a large proportion of annotation resource could be saved. The proposed technique could be easily extended to other tasks beside abnormality detection provided that the text similarity metric is specifically designed for the given task.Clinical Relevance-This work proposes a novel framework for the automatic detection of cardiovascular disease based on electrocardiogram.

Segment Origin Prediction: A Self-supervised Learning Method for Electrocardiogram Arrhythmia Classification.

The automatic arrhythmia classification system has made a significant contribution to reducing the mortality rate of cardiovascular diseases. Although the current deep-learning-based models have achieved ideal effects in arrhythmia classification, their performance still needs to be further improved due to the small scale of the dataset. In this paper, we propose a novel self-supervised pre-training method called Segment Origin Prediction (SOP) to improve the model's arrhythmia classification performance. We design a data reorganization module, which allows the model to learn ECG features by predicting whether two segments are from the same original signal without using annotations. Further, by adding a feed-forward layer to the pre-training stage, the model can achieve better performance when using labeled data for arrhythmia classification in the downstream stage. We apply the proposed SOP method to six representative models and evaluate the performances on the PhysioNet Challenge 2017 dataset. After using the SOP pre-training method, all baseline models gain significant improvement. The experimental results verify the effectiveness of the proposed SOP method.

Class-Variant Margin Normalized Softmax Loss for Deep Face Recognition.

In deep face recognition, the commonly used softmax loss and its newly proposed variations are not yet sufficiently effective to handle the class imbalance and softmax saturation issues during the training process while extracting discriminative features. In this brief, to address both issues, we propose a class-variant margin (CVM) normalized softmax loss, by introducing a true-class margin and a false-class margin into the cosine space of the angle between the feature vector and the class-weight vector. The true-class margin alleviates the class imbalance problem, and the false-class margin postpones the early individual saturation of softmax. With negligible computational complexity increment during training, the new loss function is easy to implement in the common deep learning frameworks. Comprehensive experiments on the LFW, YTF, and MegaFace protocols demonstrate the effectiveness of the proposed CVM loss function.