PSDF: Prior-Driven Neural Implicit Surface Learning for Multi-view Reconstruction.

Surface reconstruction has traditionally relied on the Multi-View Stereo (MVS)-based pipeline, which often suffers from noisy and incomplete geometry. This is due to that although MVS has been proven to be an effective way to recover the geometry of the scenes, especially for locally detailed areas with rich textures, it struggles to deal with areas with low texture and large variations of illumination where the photometric consistency is unreliable. Recently, Neural Implicit Surface Reconstruction (NISR) combines surface rendering and volume rendering techniques and bypasses the MVS as an intermediate step, which has emerged as a promising alternative to overcome the limitations of traditional pipelines. While NISR has shown impressive results on simple scenes, it remains challenging to recover delicate geometry from uncontrolled real-world scenes which is caused by its underconstrained optimization. To this end, the framework PSDF is proposed which resorts to external geometric priors from a pretrained MVS network and internal geometric priors inherent in the NISR model to facilitate high-quality neural implicit surface learning. Specifically, the visibility-aware feature consistency loss and depth prior-assisted sampling based on external geometric priors are introduced. These proposals provide powerfully geometric consistency constraints and aid in locating surface intersection points, thereby significantly improving the accuracy and delicate reconstruction of NISR. Meanwhile, the internal prior-guided importance rendering is presented to enhance the fidelity of the reconstructed surface mesh by mitigating the biased rendering issue in NISR. Extensive experiments on Tanks and Temples datasets show that PSDF achieves state-of-the-art performance on complex uncontrolled scenes.

InstaHMR: Instance-aware One-stage Multi-person Human Mesh Recovery.

Human mesh recovery aims to estimate all human meshes within a given image. In this paper, we propose an Instance-aware Multi-person 3D Human Mesh Recovery (InstaHMR) network based on the one-stage framework. Compared to former one-stage methods, instance-aware single person feature is exploited to represent more accurate human mesh. Specifically, we propose the Contextual Instance Guidance (CIG) module which generates instance-aware single person feature by leveraging spatial and channel attention operations. In this way, it preserves more instance-specific information compared to the pixel-level feature used in some existing one-stage methods. Besides, we further introduce two auxiliary losses for better mesh recovery, namely the Human Triplet Planes (HTP) loss and the T-pose Shape (TS) loss. The HTP loss encourages the model to capture subtle differences in human joint positions, while the TS loss facilitates the learning of abstract shape parameters. By incorporating these advancements, our model achieves state-of-the-art results on four multi-person datasets.

Crystallization and preliminary X-ray analysis of the CRP-cAMP-DNA (full length) complex.

The Escherichia coli cyclic AMP receptor protein (CRP) is a well known transcription activator protein. In this study, CRP was overexpressed, purified and cocrystallized with cAMP and a 38 bp full-length double-stranded DNA fragment. The full-length segment differed from the half-site fragments used in previous crystallization experiments and is more similar to the environment in vivo. CRP-cAMP-DNA crystals were obtained and diffracted to 2.9 Šresolution. The crystals belonged to space group P3121, with unit-cell parameters a = b = 76.03, c = 144.00 Å. The asymmetric unit was found to contain one protein molecule and half a 38 bp full-length double-stranded DNA fragment, with a Matthews coefficient of 2.62 Å(3) Da(-1) and a solvent content of 53.14%.

Multi-Scale Geometric Consistency Guided and Planar Prior Assisted Multi-View Stereo.

In this paper, we propose some efficient multi-view stereo methods for accurate and complete depth map estimation. We first present our basic methods with Adaptive Checkerboard sampling and Multi-Hypothesis joint view selection (ACMH & ACMH+). Based on our basic models, we develop two frameworks to deal with the depth estimation of ambiguous regions (especially low-textured areas) from two different perspectives: multi-scale information fusion and planar geometric clue assistance. For the former one, we propose a multi-scale geometric consistency guidance framework (ACMM) to obtain the reliable depth estimates for low-textured areas at coarser scales and guarantee that they can be propagated to finer scales. For the latter one, we propose a planar prior assisted framework (ACMP). We utilize a probabilistic graphical model to contribute a novel multi-view aggregated matching cost. At last, by taking advantage of the above frameworks, we further design a multi-scale geometric consistency guided and planar prior assisted multi-view stereo (ACMMP). This greatly enhances the discrimination of ambiguous regions and helps their depth sensing. Experiments on extensive datasets show our methods achieve state-of-the-art performance, recovering the depth estimation not only in low-textured areas but also in details. Related codes are available at https://github.com/GhiXu.

SSRNet: Scalable 3D Surface Reconstruction Network.

Learning-based surface reconstruction methods have received considerable attention in recent years due to their excellent expressiveness. However, existing learning-based methods lack scalability in processing large-scale point clouds. This paper proposes a novel scalable learning-based 3D surface reconstruction method based on octree, called SSRNet. SSRNet works in a scalable reconstruction pipeline, which divides oriented point clouds into different local parts and then processes them in parallel. Accommodating this scalable design pattern, SSRNet constructs local geometric features for octree vertices. Such features comprise the relation between the vertices and the implicit surface, ensuring geometric perception. Focusing on local geometric information also enables the network to avoid the overfitting problem and generalize well on different datasets. Finally, as a learning-based method, SSRNet can process large-scale point clouds in a short time. And to further solve the efficiency problem, we provide a lightweight and efficient version that is about five times faster while maintaining reconstruction performance. Experiments show that our methods achieve state-of-the-art performance with outstanding efficiency.

SC 2-PCR++: Rethinking the Generation and Selection for Efficient and Robust Point Cloud Registration.

Outlier removal is a critical part of feature-based point cloud registration. In this article, we revisit the model generation and selection of the classic RANSAC approach for fast and robust point cloud registration. For the model generation, we propose a second-order spatial compatibility (SC 2) measure to compute the similarity between correspondences. It takes into account global compatibility instead of local consistency, allowing for more distinctive clustering between inliers and outliers at an early stage. The proposed measure promises to find a certain number of outlier-free consensus sets using fewer samplings, making the model generation more efficient. For the model selection, we propose a new Feature and Spatial consistency constrained Truncated Chamfer Distance (FS-TCD) metric for evaluating the generated models. It considers the alignment quality, the feature matching properness, and the spatial consistency constraint simultaneously, enabling the correct model to be selected even when the inlier rate of the putative correspondence set is extremely low. Extensive experiments are carried out to investigate the performance of our method. In addition, we also experimentally prove that the proposed SC 2 measure and the FS-TCD metric are general and can be easily plugged into deep learning based frameworks.

Crystallization and preliminary X-ray analysis of the ligand-binding domain of cAMP receptor protein.

The cyclic AMP receptor protein (CRP) from Escherichia coli regulates the expression of a large number of genes. In this work, CRP has been overexpressed, purified and digested by subtilisin and chymotrypsin. The fragments S-CRP (digested by subtilisin) and CH-CRP (digested by chymotrypsin) have been purified and crystallized. Crystals of S-CRP diffracted to 2.0 A resolution and belonged to space group P2(1), with unit-cell parameters a = 59.7, b = 75.1, c = 128.3 A, beta = 91.5 degrees . Crystals of CH-CRP diffracted to 2.8 A resolution and belonged to space group P222, with unit-cell parameters a = 45.8, b = 60.9, c = 205.6 A.

The role of loops 3 and 4 in the interdomains and intersubunits communication of E. coli cAMP receptor protein.

Cyclic AMP serves as an intracellular messenger in cells and regulates a variety of biological functions by transmitting information through proteins. These proteins of different functions all consist of a cAMP-binding motif, and the structure of this motif is highly conserved with an exception of the loop 3 and 4. In current study, cAMP receptor protein was employed as a model system to investigate the function of the two loops. The results indicated that the loop 3 involves in the intersubunits communication of CRP, whereas the loop 4 involves in cAMP binding and interdomains communication.

Convolutional Regression for Visual Tracking.

Recently, discriminatively learned correlation filters (DCF) has attracted much attention in visual object tracking community. The success of DCF is potentially attributed to the fact that a large number of samples are utilized to train the ridge regression model and predict the location of an object. To solve the regression problem in an efficient way, these samples are all generated by circularly shifting from a searching patch. However, these synthetic samples also induce some negative effects which weaken the robustness of DCF based trackers. In this paper, we propose a new approach to learn the regression model for visual tracking with single convolutional layer. Instead of learning the linear regression model in a closed form, we try to solve the regression problem by optimizing a onechannel- output convolution layer with gradient descent (GD). In particular, the kernel size of the convolution layer is set to the size of the object. Contrary to DCF, it is possible to incorporate all "real" samples clipped from the whole image. A critical issue of the GD approach is that most of the convolutional samples are negative and the contribution of positive samples will be suppressed. To address this problem, we propose a novel objective function to eliminate easy negatives and enhance positives. We perform extensive experiments on four widely-used datasets: OTB-100, OTB-50, TempleColor, and VOT-2016. The results show that the proposed algorithm achieves outstanding performance and outperforms most of the existing DCF based algorithms.

Color image segmentation based on mean shift and normalized cuts.

In this correspondence, we develop a novel approach that provides effective and robust segmentation of color images. By incorporating the advantages of the mean shift (MS) segmentation and the normalized cut (Ncut) partitioning methods, the proposed method requires low computational complexity and is therefore very feasible for real-time image segmentation processing. It preprocesses an image by using the MS algorithm to form segmented regions that preserve the desirable discontinuity characteristics of the image. The segmented regions are then represented by using the graph structures, and the Ncut method is applied to perform globally optimized clustering. Because the number of the segmented regions is much smaller than that of the image pixels, the proposed method allows a low-dimensional image clustering with significant reduction of the complexity compared to conventional graph-partitioning methods that are directly applied to the image pixels. In addition, the image clustering using the segmented regions, instead of the image pixels, also reduces the sensitivity to noise and results in enhanced image segmentation performance. Furthermore, to avoid some inappropriate partitioning when considering every region as only one graph node, we develop an improved segmentation strategy using multiple child nodes for each region. The superiority of the proposed method is examined and demonstrated through a large number of experiments using color natural scene images.

Robust Point Sets Matching by Fusing Feature and Spatial Information Using Nonuniform Gaussian Mixture Models.

Most of the traditional methods that handle the point sets matching between two images are based on local feature descriptors and the succedent mismatch eliminating strategies, which usually suffers from the sparsity of the initial match set because some correct ambiguous associations are easily filtered out by the ratio test of SIFT matching due to their second ranking in feature similarity. In this paper, we propose a nonuniform Gaussian mixture model (NGMM) for point sets matching between a pair of images which combines feature with position information of the local feature points extracted from the image pair to achieve point sets matching in a GMM framework. The proposed point set matching using an NGMM is able to change the correspondence assignments throughout the matching process and has the potential to match up even ambiguous matches correctly. The proposed NGMM framework can be either used to directly find matches between two point sets obtained from two images or applied to remove outliers in a match set. When finding matches, NGMM tries to learn a nonrigid transformation between the two point sets and provide a probability for every found match to measure the reliability of the match. Then, a probability threshold can be used to get the final robust match set. When removing outliers, NGMM requires that the vector field formed by the correct matches to be coherent and the matches contradicting the coherent vector field will be regarded as mismatches to be removed. A number of comparison and evaluation experiments reveal the good performance of the proposed NGMM framework in both finding matches and discarding mismatches.

SaCoseg: object cosegmentation by shape conformability.

In this paper, an object cosegmentation method based on shape conformability is proposed. Different from the previous object cosegmentation methods which are based on the region feature similarity of the common objects in image set, our proposed SaCoseg cosegmentation algorithm focuses on the shape consistency of the foreground objects in image set. In the proposed method, given an image set where the implied foreground objects may be varied in appearance but share similar shape structures, the implied common shape pattern in the image set can be automatically mined and regarded as the shape prior of those unsatisfactorily segmented images. The SaCoseg algorithm mainly consists of four steps: 1) the initial Grabcut segmentation; 2) the shape mapping by coherent point drift registration; 3) the common shape pattern discovery by affinity propagation clustering; and 4) the refinement by Grabcut with common shape constraint. To testify our proposed algorithm and establish a benchmark for future work, we built the CoShape data set to evaluate the shape-based cosegmentation. The experiments on CoShape data set and the comparison with some related cosegmentation algorithms demonstrate the good performance of the proposed SaCoseg algorithm.

An ordered-patch-based image classification approach on the image Grassmannian manifold.

This paper presents an ordered-patch-based image classification framework integrating the image Grassmannian manifold to address handwritten digit recognition, face recognition, and scene recognition problems. Typical image classification methods explore image appearances without considering the spatial causality among distinctive domains in an image. To address the issue, we introduce an ordered-patch-based image representation and use the autoregressive moving average (ARMA) model to characterize the representation. First, each image is encoded as a sequence of ordered patches, integrating both the local appearance information and spatial relationships of the image. Second, the sequence of these ordered patches is described by an ARMA model, which can be further identified as a point on the image Grassmannian manifold. Then, image classification can be conducted on such a manifold under this manifold representation. Furthermore, an appropriate Grassmannian kernel for support vector machine classification is developed based on a distance metric of the image Grassmannian manifold. Finally, the experiments are conducted on several image data sets to demonstrate that the proposed algorithm outperforms other existing image classification methods.

Iterative narrowband-based graph cuts optimization for geodesic active contours with region forces (GACWRF).

In this paper, an iterative narrow-band-based graph cuts (INBBGC) method is proposed to optimize the geodesic active contours with region forces (GACWRF) model for interactive object segmentation. Based on cut metric on graphs proposed by Boykov and Kolmogorov, an NBBGC method is devised to compute the local minimization of GAC. An extension to an iterative manner, namely, INBBGC, is developed for less sensitivity to the initial curve. The INBBGC method is similar to graph-cuts-based active contour (GCBAC) presented by Xu , and their differences have been analyzed and discussed. We then integrate the region force into GAC. An improved INBBGC (IINBBGC) method is proposed to optimize the GACWRF model, thus can effectively deal with the concave region and complicated real-world images segmentation. Two region force models such as mean and probability models are studied. Therefore, the GCBAC method can be regarded as the special case of our proposed IINBBGC method without region force. Our proposed algorithm has been also analyzed to be similar to the Grabcut method when the Gaussian mixture model region force is adopted, and the band region is extended to the whole image. Thus, our proposed IINBBGC method can be regarded as narrow-band-based Grabcut method or GCBAC with region force method. We apply our proposed IINBBGC algorithm on synthetic and real-world images to emphasize its performance, compared with other segmentation methods, such as GCBAC and Grabcut methods.

The 1.6Å resolution structure of activated D138L mutant of catabolite gene activator protein with two cAMP bound in each monomer.

The X-ray crystal structure of the cAMP-liganded D138L mutant of Escherichia coli catabolite gene activator protein (CAP) was determined at a resolution of 1.66Å. This high resolution crystal structure reveals four cAMP binding sites in the homodimer. Two anti conformations of cAMPs (anti-cAMP) locate between the ß-barrel and the C-helix of each subunit; two syn conformations of cAMPs (syn-cAMP) bind on the surface of the C-terminal domain. With two syn-cAMP molecules bound, the D138L CAP is highly symmetrical with both subunits assuming a "closed" conformation. These differences make the hinge region of the mutant more flexible. Protease susceptibility measurements indicate that D138L is more susceptible to proteases than that of wild type (WT) CAP. The results of protein dynamic experiments (H/D exchange measurements) indicate that the structure of D138L mutant is more dynamic than that of WT CAP, which may impact the recognition of specific DNA sequences.

The conformational change of rabbit muscle pyruvate kinase induced by activating cations and its substrates.

Catalysis by rabbit muscle pyruvate kinase involves domain movements and conformational changes induced by activating cations and its substrates. Fluorescence acrylamide quenching analyses reveal that interactions with Mg(2+) and K(+) lead to a more exposed active site of PK while interactions with PEP and ADP decrease solvent accessibility of the active site.

Image segmentation based on GrabCut framework integrating multiscale nonlinear structure tensor.

In this paper, we propose an interactive color natural image segmentation method. The method integrates color feature with multiscale nonlinear structure tensor texture (MSNST) feature and then uses GrabCut method to obtain the segmentations. The MSNST feature is used to describe the texture feature of an image and integrated into GrabCut framework to overcome the problem of the scale difference of textured images. In addition, we extend the Gaussian Mixture Model (GMM) to MSNST feature and GMM based on MSNST is constructed to describe the energy function so that the texture feature can be suitably integrated into GrabCut framework and fused with the color feature to achieve the more superior image segmentation performance than the original GrabCut method. For easier implementation and more efficient computation, the symmetric KL divergence is chosen to produce the estimates of the tensor statistics instead of the Riemannian structure of the space of tensor. The Conjugate norm was employed using Locality Preserving Projections (LPP) technique as the distance measure in the color space for more discriminating power. An adaptive fusing strategy is presented to effectively adjust the mixing factor so that the color and MSNST texture features are efficiently integrated to achieve more robust segmentation performance. Last, an iteration convergence criterion is proposed to reduce the time of the iteration of GrabCut algorithm dramatically with satisfied segmentation accuracy. Experiments using synthesis texture images and real natural scene images demonstrate the superior performance of our proposed method.

The secondary structure of calcineurin regulatory region and conformational change induced by calcium/calmodulin binding.

The protein serine/threonine phosphatase calcineurin (CN) is activated by calmodulin (CaM) in response to intracellular calcium mobilization. A widely accepted model for CN activation involves displacement of the CN autoinhibitory peptide (CN(467-486)) from the active site upon binding of CaM. However, CN activation requires calcium binding both to the low affinity sites of CNB and to CaM, and previous studies did not dissect the individual contributions of CNB and CaM to displacement of the autoinhibitory peptide from the active site. In this work we have produced separate CN fragments corresponding to the CNA regulatory region (CNRR(381-521), residues 381-521), the CNA catalytic domain truncated at residue 341, and the CNA-CNB heterodimer with CNA truncated at residue 380 immediately after the CNB binding helix. We show that the separately expressed regulatory region retains its ability to inhibit CN phosphatase activity of the truncated CN341 and CN380 and that the inhibition can be reversed by calcium/CaM binding. Tryptophan fluorescence quenching measurements further indicate that the isolated regulatory region inhibits CN activity by occluding the catalytic site and that CaM binding exposes the catalytic site. The results provide new support for a model in which calcium binding to CNB enables CaM binding to the CNA regulatory region, and CaM binding then instructs an activating conformational change of the regulatory region that does not depend further on CNB. Moreover, the secondary structural content of the CNRR(381-521) was tentatively addressed by Fourier transform infrared spectroscopy. The results indicate that the secondary structure of CNRR(381-521) fragment is predominantly random coil, but with significant amount of beta-strand and alpha-helix structures.