Virtual-scanning light-field microscopy for robust snapshot high-resolution volumetric imaging.

High-speed three-dimensional (3D) intravital imaging in animals is useful for studying transient subcellular interactions and functions in health and disease. Light-field microscopy (LFM) provides a computational solution for snapshot 3D imaging with low phototoxicity but is restricted by low resolution and reconstruction artifacts induced by optical aberrations, motion and noise. Here, we propose virtual-scanning LFM (VsLFM), a physics-based deep learning framework to increase the resolution of LFM up to the diffraction limit within a snapshot. By constructing a 40 GB high-resolution scanning LFM dataset across different species, we exploit physical priors between phase-correlated angular views to address the frequency aliasing problem. This enables us to bypass hardware scanning and associated motion artifacts. Here, we show that VsLFM achieves ultrafast 3D imaging of diverse processes such as the beating heart in embryonic zebrafish, voltage activity in Drosophila brains and neutrophil migration in the mouse liver at up to 500 volumes per second.

Learning-based denoising for polarimetric images.

Based on measuring the polarimetric parameters which contain specific physical information, polarimetric imaging has been widely applied to various fields. However, in practice, the noise during image acquisition could lead to the output of noisy polarimetric images. In this paper, we propose, for the first time to our knowledge, a learning-based method for polarimetric image denoising. This method is based on the residual dense network and can significantly suppress the noise in polarimetric images. The experimental results show that the proposed method has an evident performance on the noise suppression and outperforms other existing methods. Especially for the images of the degree of polarization and the angle of polarization, which are quite sensitive to the noise, the proposed learning-based method can well reconstruct the details flooded in strong noise.

Resonance Character of Copper/Silver/Gold Bonding in Small Molecule⋠⋠⋠M-X (X=F, Cl, Br, CH3, CF3) Complexes.

The resonance character of Cu/Ag/Au bonding is investigated in B⋅⋅⋅M-X (M=Cu, Ag, Au; X=F, Cl, Br, CH3, CF3; B=CO, H2O, H2S, C2H2, C2H4) complexes. The natural bond orbital/natural resonance theory results strongly support the general resonance-type three-center/four-electron (3c/4e) picture of Cu/Ag/Au bonding, B:M-X↔B(+) -M:X(-) , which mainly arises from hyperconjugation interactions. On the basis of such resonance-type bonding mechanisms, the ligand effects in the more strongly bound OC⋅⋅⋅M-X series are analyzed, and distinct competition between CO and the axial ligand X is observed. This competitive bonding picture directly explains why CO in OC⋅⋅⋅Au-CF3 can be readily replaced by a number of other ligands. Additionally, conservation of the bond order indicates that the idealized relationship bB⋅⋅⋅M +bMX =1 should be suitably generalized for intermolecular bonding, especially if there is additional partial multiple bonding at one end of the 3c/4e hyperbonded triad.

Learning to Reconstruct and Understand Indoor Scenes From Sparse Views.

This paper proposes a new method for simultaneous 3D reconstruction and semantic segmentation for indoor scenes. Unlike existing methods that require recording a video using a color camera and/or a depth camera, our method only needs a small number of (e.g., 3~5) color images from uncalibrated sparse views, which significantly simplifies data acquisition and broadens applicable scenarios. To achieve promising 3D reconstruction from sparse views with limited overlap, our method first recovers the depth map and semantic information for each view, and then fuses the depth maps into a 3D scene. To this end, we design an iterative deep architecture, named IterNet, to estimate the depth map and semantic segmentation alternately. To obtain accurate alignment between views with limited overlap, we further propose a joint global and local registration method to reconstruct a 3D scene with semantic information. We also make available a new indoor synthetic dataset, containing photorealistic high-resolution RGB images, accurate depth maps and pixel-level semantic labels for thousands of complex layouts. Experimental results on public datasets and our dataset demonstrate that our method achieves more accurate depth estimation, smaller semantic segmentation errors, and better 3D reconstruction results over state-of-the-art methods.

High ISO JPEG Image Denoising by Deep Fusion of Collaborative and Convolutional Filtering.

Capturing images at high ISO modes will introduce much realistic noise, which is difficult to be removed by traditional denoising methods. In this paper, we propose a novel denoising method for high ISO JPEG images via deep fusion of collaborative and convolutional filtering. Collaborative filtering explores the non-local similarity of natural images, while convolutional filtering takes advantage of the large capacity of convolutional neural networks (CNNs) to infer noise from noisy images. We observe that the noise variance map of a high ISO JPEG image is spatial-dependent and has a Bayer-like pattern. Therefore, we introduce the Bayer pattern prior in our noise estimation and collaborative filtering stages. Since collaborative filtering is good at recovering repeatable structures and convolutional filtering is good at recovering irregular patterns and removing noise in flat regions, we propose to fuse the strengths of the two methods via deep CNN. The experimental results demonstrate that our method outperforms the state-of-the-art realistic noise removal methods for a wide variety of testing images in both subjective and objective measurements. In addition, we construct a dataset with noisy and clean image pairs for high ISO JPEG images to facilitate research on this topic.

Depth Super-resolution from RGB-D Pairs with Transform and Spatial Domain Regularization.

This paper proposes a depth super-resolution method with both transform and spatial domain regularization. In the transform domain regularization, nonlocal correlations are exploited via an auto-regressive model, where each patch is further sparsified with a locally-trained transform to consider intra-patch correlations. In the spatial domain regularization, we propose a multi-directional total variation (MTV) prior to characterize the geometrical structures spatially orientated at arbitrary directions in depth maps. To achieve adaptive regularization, the MTV is weighted for each directional finite difference considering local characteristics of RGB-D data. We develop an accelerated proximal gradient algorithm to solve the proposed model. Quantitative and qualitative evaluations compared with state-of-the-art methods demonstrate that the proposed method achieves superior depth super-resolution performance for various configurations of magnification factors and datasets.

Depth Map Super-Resolution Considering View Synthesis Quality.

Accurate and high-quality depth maps are required in lots of 3D applications, such as multi-view rendering, 3D reconstruction and 3DTV. However, the resolution of captured depth image is much lower than that of its corresponding color image, which affects its application performance. In this paper, we propose a novel depth map super-resolution (SR) method by taking view synthesis quality into account. The proposed approach mainly includes two technical contributions. First, since the captured low-resolution (LR) depth map may be corrupted by noise and occlusion, we propose a credibility based multi-view depth maps fusion strategy, which considers the view synthesis quality and interview correlation, to refine the LR depth map. Second, we propose a view synthesis quality based trilateral depth-map up-sampling method, which considers depth smoothness, texture similarity and view synthesis quality in the up-sampling filter. Experimental results demonstrate that the proposed method outperforms state-of-the-art depth SR methods for both super-resolved depth maps and synthesized views. Furthermore, the proposed method is robust to noise and achieves promising results under noise-corruption conditions.

Image denoising by exploring external and internal correlations.

Single image denoising suffers from limited data collection within a noisy image. In this paper, we propose a novel image denoising scheme, which explores both internal and external correlations with the help of web images. For each noisy patch, we build internal and external data cubes by finding similar patches from the noisy and web images, respectively. We then propose reducing noise by a two-stage strategy using different filtering approaches. In the first stage, since the noisy patch may lead to inaccurate patch selection, we propose a graph based optimization method to improve patch matching accuracy in external denoising. The internal denoising is frequency truncation on internal cubes. By combining the internal and external denoising patches, we obtain a preliminary denoising result. In the second stage, we propose reducing noise by filtering of external and internal cubes, respectively, on transform domain. In this stage, the preliminary denoising result not only enhances the patch matching accuracy but also provides reliable estimates of filtering parameters. The final denoising image is obtained by fusing the external and internal filtering results. Experimental results show that our method constantly outperforms state-of-the-art denoising schemes in both subjective and objective quality measurements, e.g., it achieves >2 dB gain compared with BM3D at a wide range of noise levels.

Ligand effects due to resonance character in LAuCCH(-) (L = F, Cl, Br, I, CCH) complexes: an NBO/NRT analysis.

The organogold complexes of LAuCCH(-) (L = F, Cl, Br, I, CCH) were investigated using natural bond orbital/natural resonance theory (NBO/NRT) methods. The NBO/NRT results strongly support the general resonance-type three-center-four-electron (3c/4e) picture of LAuCCH: L(-): Au-CCH ↔ L-Au :CCH(-), arising from hyperconjugation interactions. The sums of ionic and covalent contributions to both L-Au and Au-CCH bonds are all slightly larger than that due to the additional π-back bonding within the 3c/4e hyperbonded triad. This complementary relationship between L-Au and Au-CCH bond orders implies a competing relationship between the ancillary ligand and CCH around the gold atom. We discuss the ligand effects in the LAuCCH(-) series on the basis of this competing relationship.

Landmark image super-resolution by retrieving web images.

This paper proposes a new super-resolution (SR) scheme for landmark images by retrieving correlated web images. Using correlated web images significantly improves the exemplar-based SR. Given a low-resolution (LR) image, we extract local descriptors from its up-sampled version and bundle the descriptors according to their spatial relationship to retrieve correlated high-resolution (HR) images from the web. Though similar in content, the retrieved images are usually taken with different illumination, focal lengths, and shot perspectives, resulting in uncertainty for the HR detail approximation. To solve this problem, we first propose aligning these images to the up-sampled LR image through a global registration, which identifies the corresponding regions in these images and reduces the mismatching. Second, we propose a structure-aware matching criterion and adaptive block sizes to improve the mapping accuracy between LR and HR patches. Finally, these matched HR patches are blended together by solving an energy minimization problem to recover the desired HR image. Experimental results demonstrate that our SR scheme achieves significant improvement compared with four state-of-the-art schemes in terms of both subjective and objective qualities.