Mask-Pyramid Network: A Novel Panoptic Segmentation Method.

In this paper, we introduce a novel panoptic segmentation method called the Mask-Pyramid Network. Existing Mask RCNN-based methods first generate a large number of box proposals and then filter them at each feature level, which requires a lot of computational resources, while most of the box proposals are suppressed and discarded in the Non-Maximum Suppression process. Additionally, for panoptic segmentation, it is a problem to properly fuse the semantic segmentation results with the Mask RCNN-produced instance segmentation results. To address these issues, we propose a new mask pyramid mechanism to distinguish objects and generate much fewer proposals by referring to existing segmented masks, so as to reduce computing resource consumption. The Mask-Pyramid Network generates object proposals and predicts masks from larger to smaller sizes. It records the pixel area occupied by the larger object masks, and then only generates proposals on the unoccupied areas. Each object mask is represented as a H × W × 1 logit, which fits well in format with the semantic segmentation logits. By applying SoftMax to the concatenated semantic and instance segmentation logits, it is easy and natural to fuse both segmentation results. We empirically demonstrate that the proposed Mask-Pyramid Network achieves comparable accuracy performance on the Cityscapes and COCO datasets. Furthermore, we demonstrate the computational efficiency of the proposed method and obtain competitive results.

Influence of Differences in the Density of Seawater on the Measurement of the Underwater Gravity Gradient.

In preparing gravity gradient reference maps for navigation purposes, researchers have tended to use a constant value for the density of seawater. However, the actual seawater density at a particular location may vary due to the effects of longitude, latitude and bathymetry. In this study, the right rectangular prism method was used to calculate the disturbing gravity gradient caused by the mass deficiency of seawater for three different seawater profiles in an area east of Taiwan. For this purpose, two seawater density models were used as alternatives to the constant seawater density model, and the alteration in the gravity gradient was calculated to quantify the error in the gravity gradient as a result of using a constant seawater density. The results demonstrated that the error in the gravity gradient can reach 1E for water at large depths. Moreover, the difference between the amplitude of the error of the corrected thermocline and that for the uncorrected seawater density model was found to be quite small. If a gravity gradient reference map with accuracy better than 1E is to be realized, the seawater density cannot be taken as constant during forward modeling.

Influence of Sea Level Anomaly on Underwater Gravity Gradient Measurements.

Considering the theoretical research needs of gravity gradient detection and navigation, this study uses the right rectangular prism method to calculate the disturbing gravity gradient from sea level anomalies in the range of 5° × 5° in the Kuroshio extension area of the western Pacific with large sea level anomalies. The disturbing gravity gradient is obtained in different directions within a depth of 50 m below the mean sea level based on the principle of the disturbing gravity gradient. The calculation results show that the sea level anomalies at local positions significantly impact the underwater gravity gradient measurements, with the maximum contribution exceeding 10 E and the maximum difference between different locations exceeding 20 E. The change of the sea level anomaly over time also significantly impacts the measurement of the underwater gravity gradient, with the maximum change value exceeding 20 E. The impact will have a corresponding change with the seasonal change of the sea level anomaly. Therefore, the underwater carrier needs to consider the disturbing gravity gradient caused by sea level anomalies when using the gravity gradient for underwater detection and navigation.

Edge-Sensitive Left Ventricle Segmentation Using Deep Reinforcement Learning.

Deep reinforcement learning (DRL) has been utilized in numerous computer vision tasks, such as object detection, autonomous driving, etc. However, relatively few DRL methods have been proposed in the area of image segmentation, particularly in left ventricle segmentation. Reinforcement learning-based methods in earlier works often rely on learning proper thresholds to perform segmentation, and the segmentation results are inaccurate due to the sensitivity of the threshold. To tackle this problem, a novel DRL agent is designed to imitate the human process to perform LV segmentation. For this purpose, we formulate the segmentation problem as a Markov decision process and innovatively optimize it through DRL. The proposed DRL agent consists of two neural networks, i.e., First-P-Net and Next-P-Net. The First-P-Net locates the initial edge point, and the Next-P-Net locates the remaining edge points successively and ultimately obtains a closed segmentation result. The experimental results show that the proposed model has outperformed the previous reinforcement learning methods and achieved comparable performances compared with deep learning baselines on two widely used LV endocardium segmentation datasets, namely Automated Cardiac Diagnosis Challenge (ACDC) 2017 dataset, and Sunnybrook 2009 dataset. Moreover, the proposed model achieves higher F-measure accuracy compared with deep learning methods when training with a very limited number of samples.

HSGAN: Hyperspectral Reconstruction From RGB Images With Generative Adversarial Network.

Hyperspectral (HS) reconstruction from RGB images denotes the recovery of whole-scene HS information, which has attracted much attention recently. State-of-the-art approaches often adopt convolutional neural networks to learn the mapping for HS reconstruction from RGB images. However, they often do not achieve high HS reconstruction performance across different scenes consistently. In addition, their performance in recovering HS images from clean and real-world noisy RGB images is not consistent. To improve the HS reconstruction accuracy and robustness across different scenes and from different input images, we present an effective HSGAN framework with a two-stage adversarial training strategy. The generator is a four-level top-down architecture that extracts and combines features on multiple scales. To generalize well to real-world noisy images, we further propose a spatial-spectral attention block (SSAB) to learn both spatial-wise and channel-wise relations. We conduct the HS reconstruction experiments from both clean and real-world noisy RGB images on five well-known HS datasets. The results demonstrate that HSGAN achieves superior performance to existing methods. Please visit https://github.com/zhaoyuzhi/HSGAN to try our codes.

SVCNet: Scribble-Based Video Colorization Network With Temporal Aggregation.

In this paper, we propose a scribble-based video colorization network with temporal aggregation called SVCNet. It can colorize monochrome videos based on different user-given color scribbles. It addresses three common issues in the scribble-based video colorization area: colorization vividness, temporal consistency, and color bleeding. To improve the colorization quality and strengthen the temporal consistency, we adopt two sequential sub-networks in SVCNet for precise colorization and temporal smoothing, respectively. The first stage includes a pyramid feature encoder to incorporate color scribbles with a grayscale frame, and a semantic feature encoder to extract semantics. The second stage finetunes the output from the first stage by aggregating the information of neighboring colorized frames (as short-range connections) and the first colorized frame (as a long-range connection). To alleviate the color bleeding artifacts, we learn video colorization and segmentation simultaneously. Furthermore, we set the majority of operations on a fixed small image resolution and use a Super-resolution Module at the tail of SVCNet to recover original sizes. It allows the SVCNet to fit different image resolutions at the inference. Finally, we evaluate the proposed SVCNet on DAVIS and Videvo benchmarks. The experimental results demonstrate that SVCNet produces both higher-quality and more temporally consistent videos than other well-known video colorization approaches. The codes and models can be found at https://github.com/zhaoyuzhi/SVCNet.

ShaTure: Shape and Texture Deformation for Human Pose and Attribute Transfer.

In this paper, we present a novel end-to-end pose transfer framework to transform a source person image to an arbitrary pose with controllable attributes. Due to the spatial misalignment caused by occlusions and multi-viewpoints, maintaining high-quality shape and texture appearance is still a challenging problem for pose-guided person image synthesis. Without considering the deformation of shape and texture, existing solutions on controllable pose transfer still cannot generate high-fidelity texture for the target image. To solve this problem, we design a new image reconstruction decoder - ShaTure which formulates shape and texture in a braiding manner. It can interchange discriminative features in both feature-level space and pixel-level space so that the shape and texture can be mutually fine-tuned. In addition, we develop a new bottleneck module - Adaptive Style Selector (AdaSS) Module which can enhance the multi-scale feature extraction capability by self-recalibration of the feature map through channel-wise attention. Both quantitative and qualitative results show that the proposed framework has superiority compared with the state-of-the-art human pose and attribute transfer methods. Detailed ablation studies report the effectiveness of each contribution, which proves the robustness and efficacy of the proposed framework.

Temperature sensing performance based on up-conversion luminescence in hydrothermally synthesized Yb3+/Er3+ co-doped NaScF4 phosphors.

NaScF4:Yb3+/Er3+ crystals were successfully hydrothermally synthesized using distilled water as a single solvent. The crystal phase and morphology were characterized by XRD and SEM. Compared with organic solvents such as ethanol, ethylene glycol and ether, the molar ratio of Na/F/Sc plays an important role in the synthesis of NaScF4 crystals. Morphological control was achieved by changing the types of additives, and the relationship between morphology and luminescence properties was explored. The NaScF4:Yb3+/Er3+ phosphors exhibit strong green and red UC emissions under the excitation of 980 nm NIR. Optimal concentrations of Yb3+ and Er3+ for up-conversion luminescence performance were identified as 10% and 2%, respectively. Without any subsequent heat treatment process, the obtained NaScF4:10%Yb3+/2%Er3+ showed good temperature sensitivity. The temperature sensing ability was investigated by employing the dependence of the fluorescence intensity ratio (FIR) of the two thermal coupling energy levels of Er3+ (2H11/2→4I15/2 and 4S3/2→4I15/2) on temperature; the maximum sensitivity SA/SR was 0.00256 K-1/0.00317 K-1 at 548 K, and it increased to 0.00328 K-1/0.00413 K-1 after adding EDTA. In addition, an evaluation of temperature uncertainty during temperature measurement was performed, and was found to be 0.073 K and 0.095 K in the presence and absence of EDTA, respectively. Compared with some other reported materials, the obtained material shows a relatively superior temperature sensitivity, which provides new ideas for the improvement of temperature-sensitive materials.