Self-Paced Multi-Grained Cross-Modal Interaction Modeling for Referring Expression Comprehension.

As an important and challenging problem in vision-language tasks, referring expression comprehension (REC) generally requires a large amount of multi-grained information of visual and linguistic modalities to realize accurate reasoning. In addition, due to the diversity of visual scenes and the variation of linguistic expressions, some hard examples have much more abundant multi-grained information than others. How to aggregate multi-grained information from different modalities and extract abundant knowledge from hard examples is crucial in the REC task. To address aforementioned challenges, in this paper, we propose a Self-paced Multi-grained Cross-modal Interaction Modeling framework, which improves the language-to-vision localization ability through innovations in network structure and learning mechanism. Concretely, we design a transformer-based multi-grained cross-modal attention, which effectively utilizes the inherent multi-grained information in visual and linguistic encoders. Furthermore, considering the large variance of samples, we propose a self-paced sample informativeness learning to adaptively enhance the network learning for samples containing abundant multi-grained information. The proposed framework significantly outperforms state-of-the-art methods on widely used datasets, such as RefCOCO, RefCOCO+, RefCOCOg, and ReferItGame datasets, demonstrating the effectiveness of our method.

Deep learning-enabled 3D multimodal fusion of cone-beam CT and intraoral mesh scans for clinically applicable tooth-bone reconstruction.

High-fidelity three-dimensional (3D) models of tooth-bone structures are valuable for virtual dental treatment planning; however, they require integrating data from cone-beam computed tomography (CBCT) and intraoral scans (IOS) using methods that are either error-prone or time-consuming. Hence, this study presents Deep Dental Multimodal Fusion (DDMF), an automatic multimodal framework that reconstructs 3D tooth-bone structures using CBCT and IOS. Specifically, the DDMF framework comprises CBCT and IOS segmentation modules as well as a multimodal reconstruction module with novel pixel representation learning architectures, prior knowledge-guided losses, and geometry-based 3D fusion techniques. Experiments on real-world large-scale datasets revealed that DDMF achieved superior segmentation performance on CBCT and IOS, achieving a 0.17 mm average symmetric surface distance (ASSD) for 3D fusion with a substantial processing time reduction. Additionally, clinical applicability studies have demonstrated DDMF's potential for accurately simulating tooth-bone structures throughout the orthodontic treatment process.

Topology-Based and Conformation-Based Decoys Database: An Unbiased Online Database for Training and Benchmarking Machine-Learning Scoring Functions.

Machine-learning-based scoring functions (MLSFs) have gained attention for their potential to improve accuracy in binding affinity prediction and structure-based virtual screening (SBVS) compared to classical SFs. Developing accurate MLSFs for SBVS requires a large and unbiased dataset that includes structurally diverse actives and decoys. Unfortunately, most datasets suffer from hidden biases and data insufficiency. Here, we developed topology-based and conformation-based decoys database (ToCoDDB). The biological targets and active ligands in ToCoDDB were collected from scientific literature and established datasets. The decoys were generated and debiased by using conditional recurrent neural networks and molecular docking. ToCoDDB is presently the largest unbiased database with 2.4 million decoys encompassing 155 targets. The detailed information and performance benchmark for each target are provided, which are beneficial for training and evaluating MLSFs. Moreover, the online decoys generation function of ToCoDDB further expands its application range to any target. ToCoDDB is freely available at http://cadd.zju.edu.cn/tocodecoy/.

Hierarchical Self-Supervised Learning for 3D Tooth Segmentation in Intra-Oral Mesh Scans.

Accurately delineating individual teeth and the gingiva in the three-dimension (3D) intraoral scanned (IOS) mesh data plays a pivotal role in many digital dental applications, e.g., orthodontics. Recent research shows that deep learning based methods can achieve promising results for 3D tooth segmentation, however, most of them rely on high-quality labeled dataset which is usually of small scales as annotating IOS meshes requires intensive human efforts. In this paper, we propose a novel self-supervised learning framework, named STSNet, to boost the performance of 3D tooth segmentation leveraging on large-scale unlabeled IOS data. The framework follows two-stage training, i.e., pre-training and fine-tuning. In pre-training, three hierarchical-level, i.e., point-level, region-level, cross-level, contrastive losses are proposed for unsupervised representation learning on a set of predefined matched points from different augmented views. The pretrained segmentation backbone is further fine-tuned in a supervised manner with a small number of labeled IOS meshes. With the same amount of annotated samples, our method can achieve an mIoU of 89.88%, significantly outperforming the supervised counterparts. The performance gain becomes more remarkable when only a small amount of labeled samples are available. Furthermore, STSNet can achieve better performance with only 40% of the annotated samples as compared to the fully supervised baselines. To the best of our knowledge, we present the first attempt of unsupervised pre-training for 3D tooth segmentation, demonstrating its strong potential in reducing human efforts for annotation and verification.

VGSC-DB: an online database of voltage-gated sodium channels.

As an important member of ion channels family, the voltage-gated sodium channel (VGSC/Nav) is associated with a variety of diseases, including epilepsy, migraine, ataxia, etc., and has always been a hot target for drug design and discovery. Many subtype-selective modulators targeting VGSCs have been reported, and some of them have been approved for clinical applications. However, the drug design resources related to VGSCs are insufficient, especially the lack of accurate and extensive compound data toward VGSCs. To fulfill this demand, we develop the Voltage-gated Sodium Channels Database (VGSC-DB). VGSC-DB is the first open-source database for VGSCs, which provides open access to 6055 data records, including 3396 compounds from 173 references toward nine subtypes of Navs (Nav1.1 ~ Nav1.9). A total of 28 items of information is included in each data record, including the chemical structure, biological activity (IC50/EC50), target, binding site, organism, chemical and physical properties, etc. VGSC-DB collects the data from small-molecule compounds, toxins and various derivatives. Users can search the information of compounds by text or structure, and the advanced search function is also supported to realize batch query. VGSC-DB is freely accessible at http://cadd.zju.edu.cn/vgsc/ , and all the data can be downloaded in XLSX/SDF file formats.

MAP-SNN: Mapping spike activities with multiplicity, adaptability, and plasticity into bio-plausible spiking neural networks.

Spiking Neural Networks (SNNs) are considered more biologically realistic and power-efficient as they imitate the fundamental mechanism of the human brain. Backpropagation (BP) based SNN learning algorithms that utilize deep learning frameworks have achieved good performance. However, those BP-based algorithms partially ignore bio-interpretability. In modeling spike activity for biological plausible BP-based SNNs, we examine three properties: multiplicity, adaptability, and plasticity (MAP). Regarding multiplicity, we propose a Multiple-Spike Pattern (MSP) with multiple-spike transmission to improve model robustness in discrete time iterations. To realize adaptability, we adopt Spike Frequency Adaption (SFA) under MSP to reduce spike activities for enhanced efficiency. For plasticity, we propose a trainable state-free synapse that models spike response current to increase the diversity of spiking neurons for temporal feature extraction. The proposed SNN model achieves competitive performances on the N-MNIST and SHD neuromorphic datasets. In addition, experimental results demonstrate that the proposed three aspects are significant to iterative robustness, spike efficiency, and the capacity to extract spikes' temporal features. In summary, this study presents a realistic approach for bio-inspired spike activity with MAP, presenting a novel neuromorphic perspective for incorporating biological properties into spiking neural networks.

pcnaDeep: a fast and robust single-cell tracking method using deep-learning mediated cell cycle profiling.

SUMMARY: Computational methods that track single cells and quantify fluorescent biosensors in time-lapse microscopy images have revolutionized our approach in studying the molecular control of cellular decisions. One barrier that limits the adoption of single-cell analysis in biomedical research is the lack of efficient methods to robustly track single cells over cell division events. Here, we developed an application that automatically tracks and assigns mother-daughter relationships of single cells. By incorporating cell cycle information from a well-established fluorescent cell cycle reporter, we associate mitosis relationships enabling high fidelity long-term single-cell tracking. This was achieved by integrating a deep-learning-based fluorescent proliferative cell nuclear antigen signal instance segmentation module with a cell tracking and cell cycle resolving pipeline. The application offers a user-friendly interface and extensible APIs for customized cell cycle analysis and manual correction for various imaging configurations. AVAILABILITY AND IMPLEMENTATION: pcnaDeep is an open-source Python application under the Apache 2.0 licence. The source code, documentation and tutorials are available at https://github.com/chan-labsite/PCNAdeep. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

STSC-SNN: Spatio-Temporal Synaptic Connection with temporal convolution and attention for spiking neural networks.

Spiking neural networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal information processing capability, low power consumption, and high biological plausibility. The potential to efficiently extract spatio-temporal features makes it suitable for processing event streams. However, existing synaptic structures in SNNs are almost full-connections or spatial 2D convolution, neither of which can extract temporal dependencies adequately. In this work, we take inspiration from biological synapses and propose a Spatio-Temporal Synaptic Connection SNN (STSC-SNN) model to enhance the spatio-temporal receptive fields of synaptic connections, thereby establishing temporal dependencies across layers. Specifically, we incorporate temporal convolution and attention mechanisms to implement synaptic filtering and gating functions. We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks. In addition, we investigate the impact of performance via varied spatial-temporal receptive fields and reevaluate the temporal modules in SNNs. Our approach is tested on neuromorphic datasets, including DVS128 Gesture (gesture recognition), N-MNIST, CIFAR10-DVS (image classification), and SHD (speech digit recognition). The results show that the proposed model outperforms the state-of-the-art accuracy on nearly all datasets.

ASFP (Artificial Intelligence based Scoring Function Platform): a web server for the development of customized scoring functions.

Virtual screening (VS) based on molecular docking has emerged as one of the mainstream technologies of drug discovery due to its low cost and high efficiency. However, the scoring functions (SFs) implemented in most docking programs are not always accurate enough and how to improve their prediction accuracy is still a big challenge. Here, we propose an integrated platform called ASFP, a web server for the development of customized SFs for structure-based VS. There are three main modules in ASFP: (1) the descriptor generation module that can generate up to 3437 descriptors for the modelling of protein-ligand interactions; (2) the AI-based SF construction module that can establish target-specific SFs based on the pre-generated descriptors through three machine learning (ML) techniques; (3) the online prediction module that provides some well-constructed target-specific SFs for VS and an additional generic SF for binding affinity prediction. Our methodology has been validated on several benchmark datasets. The target-specific SFs can achieve an average ROC AUC of 0.973 towards 32 targets and the generic SF can achieve the Pearson correlation coefficient of 0.81 on the PDBbind version 2016 core set. To sum up, the ASFP server is a powerful tool for structure-based VS.

Beware of the generic machine learning-based scoring functions in structure-based virtual screening.

Machine learning-based scoring functions (MLSFs) have attracted extensive attention recently and are expected to be potential rescoring tools for structure-based virtual screening (SBVS). However, a major concern nowadays is whether MLSFs trained for generic uses rather than a given target can consistently be applicable for VS. In this study, a systematic assessment was carried out to re-evaluate the effectiveness of 14 reported MLSFs in VS. Overall, most of these MLSFs could hardly achieve satisfactory results for any dataset, and they could even not outperform the baseline of classical SFs such as Glide SP. An exception was observed for RFscore-VS trained on the Directory of Useful Decoys-Enhanced dataset, which showed its superiority for most targets. However, in most cases, it clearly illustrated rather limited performance on the targets that were dissimilar to the proteins in the corresponding training sets. We also used the top three docking poses rather than the top one for rescoring and retrained the models with the updated versions of the training set, but only minor improvements were observed. Taken together, generic MLSFs may have poor generalization capabilities to be applicable for the real VS campaigns. Therefore, it should be quite cautious to use this type of methods for VS.

Can machine learning consistently improve the scoring power of classical scoring functions? Insights into the role of machine learning in scoring functions.

How to accurately estimate protein-ligand binding affinity remains a key challenge in computer-aided drug design (CADD). In many cases, it has been shown that the binding affinities predicted by classical scoring functions (SFs) cannot correlate well with experimentally measured biological activities. In the past few years, machine learning (ML)-based SFs have gradually emerged as potential alternatives and outperformed classical SFs in a series of studies. In this study, to better recognize the potential of classical SFs, we have conducted a comparative assessment of 25 commonly used SFs. Accordingly, the scoring power was systematically estimated by using the state-of-the-art ML methods that replaced the original multiple linear regression method to refit individual energy terms. The results show that the newly-developed ML-based SFs consistently performed better than classical ones. In particular, gradient boosting decision tree (GBDT) and random forest (RF) achieved the best predictions in most cases. The newly-developed ML-based SFs were also tested on another benchmark modified from PDBbind v2007, and the impacts of structural and sequence similarities were evaluated. The results indicated that the superiority of the ML-based SFs could be fully guaranteed when sufficient similar targets were contained in the training set. Moreover, the effect of the combinations of features from multiple SFs was explored, and the results indicated that combining NNscore2.0 with one to four other classical SFs could yield the best scoring power. However, it was not applicable to derive a generic target-specific SF or SF combination.

Uncertainty-Based Active Learning via Sparse Modeling for Image Classification.

Uncertainty sampling-based active learning has been well studied for selecting informative samples to improve the performance of a classifier. In batch-mode active learning, a batch of samples are selected for a query at the same time. The samples with top uncertainty are encouraged to be selected. However, this selection strategy ignores the relations among the samples, because the selected samples may have much redundant information with each other. This paper addresses this problem by proposing a novel method that combines uncertainty, diversity, and density via sparse modeling in the sample selection. We use sparse linear combination to represent the uncertainty of unlabeled pool data with Gaussian kernels, in which the diversity and density are well incorporated. The selective sampling method is proposed before optimization to reduce the representation error. To deal with ${l}_{0}$ norm constraint in the sparse problem, two approximated approaches are adopted for efficient optimization. Four image classification data sets are used for evaluation. Extensive experiments related to batch size, feature space, seed size, significant analysis, data transform, and time efficiency demonstrate the advantages of the proposed method.