Reliable deep learning in anomalous diffusion against out-of-distribution dynamics.

Anomalous diffusion plays a crucial rule in understanding molecular-level dynamics by offering valuable insights into molecular interactions, mobility states and the physical properties of systems across both biological and materials sciences. Deep-learning techniques have recently outperformed conventional statistical methods in anomalous diffusion recognition. However, deep-learning networks are typically trained by data with limited distribution, which inevitably fail to recognize unknown diffusion models and misinterpret dynamics when confronted with out-of-distribution (OOD) scenarios. In this work, we present a general framework for evaluating deep-learning-based OOD dynamics-detection methods. We further develop a baseline approach that achieves robust OOD dynamics detection as well as accurate recognition of in-distribution anomalous diffusion. We demonstrate that this method enables a reliable characterization of complex behaviors across a wide range of experimentally diverse systems, including nicotinic acetylcholine receptors in membranes, fluorescent beads in dextran solutions and silver nanoparticles undergoing active endocytosis.

Enhancing Few-Shot CLIP With Semantic-Aware Fine-Tuning.

Learning generalized representations from limited training samples is crucial for applying deep neural networks in low-resource scenarios. Recently, methods based on contrastive language-image pretraining (CLIP) have exhibited promising performance in few-shot adaptation tasks. To avoid catastrophic forgetting and overfitting caused by few-shot fine-tuning, existing works usually freeze the parameters of CLIP pretrained on large-scale datasets, overlooking the possibility that some parameters might not be suitable for downstream tasks. To this end, we revisit CLIP's visual encoder with a specific focus on its distinctive attention pooling layer, which performs a spatial weighted-sum of the dense feature maps. Given that dense feature maps contain meaningful semantic information, and different semantics hold varying importance for diverse downstream tasks (such as prioritizing semantics like ears and eyes in pet classification tasks rather than side mirrors), using the same weighted-sum operation for dense features across different few-shot tasks might not be appropriate. Hence, we propose fine-tuning the parameters of the attention pooling layer during the training process to encourage the model to focus on task-specific semantics. In the inference process, we perform residual blending between the features pooled by the fine-tuned and the original attention pooling layers to incorporate both the few-shot knowledge and the pretrained CLIP's prior knowledge. We term this method as semantic-aware fine-tuning (). is effective in enhancing the conventional few-shot CLIP and is compatible with the existing adapter approach (termed ). Extensive experiments on 11 benchmarks demonstrate that both and significantly outperform the second-best method by + 1.51 % and + 2.38 % in the one-shot setting and by + 0.48 % and + 1.37 % in the four-shot setting, respectively.

Unlocking comprehensive molecular design across all scenarios with large language model and unordered chemical language.

Molecular generation stands at the forefront of AI-driven technologies, playing a crucial role in accelerating the development of small molecule drugs. The intricate nature of practical drug discovery necessitates the development of a versatile molecular generation framework that can tackle diverse drug design challenges. However, existing methodologies often struggle to encompass all aspects of small molecule drug design, particularly those rooted in language models, especially in tasks like linker design, due to the autoregressive nature of large language model-based approaches. To empower a language model for a wider range of molecular design tasks, we introduce an unordered simplified molecular-input line-entry system based on fragments (FU-SMILES). Building upon this foundation, we propose FragGPT, a universal fragment-based molecular generation model. Initially pretrained on extensive molecular datasets, FragGPT utilizes FU-SMILES to facilitate efficient generation across various practical applications, such as de novo molecule design, linker design, R-group exploration, scaffold hopping, and side chain optimization. Furthermore, we integrate conditional generation and reinforcement learning (RL) methodologies to ensure that the generated molecules possess multiple desired biological and physicochemical properties. Experimental results across diverse scenarios validate FragGPT's superiority in generating molecules with enhanced properties and novel structures, outperforming existing state-of-the-art models. Moreover, its robust drug design capability is further corroborated through real-world drug design cases.

Explicit Margin Equilibrium for Few-Shot Object Detection.

Under low data regimes, few-shot object detection (FSOD) transfers related knowledge from base classes with sufficient annotations to novel classes with limited samples in a two-step paradigm, including base training and balanced fine-tuning. In base training, the learned embedding space needs to be dispersed with large class margins to facilitate novel class accommodation and avoid feature aliasing while in balanced fine-tuning properly concentrating with small margins to represent novel classes precisely. Although obsession with the discrimination and representation dilemma has stimulated substantial progress, explorations for the equilibrium of class margins within the embedding space are still in full swing. In this study, we propose a class margin optimization scheme, termed explicit margin equilibrium (EME), by explicitly leveraging the quantified relationship between base and novel classes. EME first maximizes base-class margins to reserve adequate space to prepare for novel class adaptation. During fine-tuning, it quantifies the interclass semantic relationships by calculating the equilibrium coefficients based on the assumption that novel instances can be represented by linear combinations of base-class prototypes. EME finally reweights margin loss using equilibrium coefficients to adapt base knowledge for novel instance learning with the help of instance disturbance (ID) augmentation. As a plug-and-play module, EME can also be applied to few-shot classification. Consistent performance gains upon various baseline methods and benchmarks validate the generality and efficacy of EME. The code is available at github.com/Bohao-Lee/EME.

Association between gait video information and general cardiovascular diseases: a prospective cross-sectional study.

Aims: Cardiovascular disease (CVD) may not be detected in time with conventional clinical approaches. Abnormal gait patterns have been associated with pathological conditions and can be monitored continuously by gait video. We aim to test the association between non-contact, video-based gait information and general CVD status. Methods and results: Individuals undergoing confirmatory CVD evaluation were included in a prospective, cross-sectional study. Gait videos were recorded with a Kinect camera. Gait features were extracted from gait videos to correlate with the composite and individual components of CVD, including coronary artery disease, peripheral artery disease, heart failure, and cerebrovascular events. The incremental value of incorporating gait information with traditional CVD clinical variables was also evaluated. Three hundred fifty-two participants were included in the final analysis [mean (standard deviation) age, 59.4 (9.8) years; 25.3% were female]. Compared with the baseline clinical variable model [area under the receiver operating curve (AUC) 0.717, (0.690-0.743)], the gait feature model demonstrated statistically better performance [AUC 0.753, (0.726-0.780)] in predicting the composite CVD, with further incremental value when incorporated with the clinical variables [AUC 0.764, (0.741-0.786)]. Notably, gait features exhibited varied association with different CVD component conditions, especially for peripheral artery disease [AUC 0.752, (0.728-0.775)] and heart failure [0.733, (0.707-0.758)]. Additional analyses also revealed association of gait information with CVD risk factors and the established CVD risk score. Conclusion: We demonstrated the association and predictive value of non-contact, video-based gait information for general CVD status. Further studies for gait video-based daily living CVD monitoring are promising.

Prediction of coronary artery disease based on facial temperature information captured by non-contact infrared thermography.

BACKGROUND: Current approaches for initial coronary artery disease (CAD) assessment rely on pretest probability (PTP) based on risk factors and presentations, with limited performance. Infrared thermography (IRT), a non-contact technology that detects surface temperature, has shown potential in assessing atherosclerosis-related conditions, particularly when measured from body regions such as faces. We aim to assess the feasibility of using facial IRT temperature information with machine learning for the prediction of CAD. METHODS: Individuals referred for invasive coronary angiography or coronary CT angiography (CCTA) were enrolled. Facial IRT images captured before confirmatory CAD examinations were used to develop and validate a deep-learning IRT image model for detecting CAD. We compared the performance of the IRT image model with the guideline-recommended PTP model on the area under the curve (AUC). In addition, interpretable IRT tabular features were extracted from IRT images to further validate the predictive value of IRT information. RESULTS: A total of 460 eligible participants (mean (SD) age, 58.4 (10.4) years; 126 (27.4%) female) were included. The IRT image model demonstrated outstanding performance (AUC 0.804, 95% CI 0.785 to 0.823) compared with the PTP models (AUC 0.713, 95% CI 0.691 to 0.734). A consistent level of superior performance (AUC 0.796, 95% CI 0.782 to 0.811), achieved with comprehensive interpretable IRT features, further validated the predictive value of IRT information. Notably, even with only traditional temperature features, a satisfactory performance (AUC 0.786, 95% CI 0.769 to 0.803) was still upheld. CONCLUSION: In this prospective study, we demonstrated the feasibility of using non-contact facial IRT information for CAD prediction.

Cell-free biosynthesis and engineering of ribosomally synthesized lanthipeptides.

Ribosomally synthesized and post-translationally modified peptides (RiPPs) are a major class of natural products with diverse chemical structures and potent biological activities. A vast majority of RiPP gene clusters remain unexplored in microbial genomes, which is partially due to the lack of rapid and efficient heterologous expression systems for RiPP characterization and biosynthesis. Here, we report a unified biocatalysis (UniBioCat) system based on cell-free gene expression for rapid biosynthesis and engineering of RiPPs. We demonstrate UniBioCat by reconstituting a full biosynthetic pathway for de novo biosynthesis of salivaricin B, a lanthipeptide RiPP. Next, we delete several protease/peptidase genes from the source strain to enhance the performance of UniBioCat, which then can synthesize and screen salivaricin B variants with enhanced antimicrobial activity. Finally, we show that UniBioCat is generalizable by synthesizing and evaluating the bioactivity of ten uncharacterized lanthipeptides. We expect UniBioCat to accelerate the discovery, characterization, and synthesis of RiPPs.

Deep Lossy Plus Residual Coding for Lossless and Near-Lossless Image Compression.

Lossless and near-lossless image compression is of paramount importance to professional users in many technical fields, such as medicine, remote sensing, precision engineering and scientific research. But despite rapidly growing research interests in learning-based image compression, no published method offers both lossless and near-lossless modes. In this paper, we propose a unified and powerful deep lossy plus residual (DLPR) coding framework for both lossless and near-lossless image compression. In the lossless mode, the DLPR coding system first performs lossy compression and then lossless coding of residuals. We solve the joint lossy and residual compression problem in the approach of VAEs, and add autoregressive context modeling of the residuals to enhance lossless compression performance. In the near-lossless mode, we quantize the original residuals to satisfy a given ℓ∞ error bound, and propose a scalable near-lossless compression scheme that works for variable ℓ∞ bounds instead of training multiple networks. To expedite the DLPR coding, we increase the degree of algorithm parallelization by a novel design of coding context, and accelerate the entropy coding with adaptive residual interval. Experimental results demonstrate that the DLPR coding system achieves both the state-of-the-art lossless and near-lossless image compression performance with competitive coding speed.

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle.

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

CarsiDock: a deep learning paradigm for accurate protein-ligand docking and screening based on large-scale pre-training.

The expertise accumulated in deep neural network-based structure prediction has been widely transferred to the field of protein-ligand binding pose prediction, thus leading to the emergence of a variety of deep learning-guided docking models for predicting protein-ligand binding poses without relying on heavy sampling. However, their prediction accuracy and applicability are still far from satisfactory, partially due to the lack of protein-ligand binding complex data. To this end, we create a large-scale complex dataset containing ∼9 M protein-ligand docking complexes for pre-training, and propose CarsiDock, the first deep learning-guided docking approach that leverages pre-training of millions of predicted protein-ligand complexes. CarsiDock contains two main stages, i.e., a deep learning model for the prediction of protein-ligand atomic distance matrices, and a translation, rotation and torsion-guided geometry optimization procedure to reconstruct the matrices into a credible binding pose. The pre-training and multiple innovative architectural designs facilitate the dramatically improved docking accuracy of our approach over the baselines in terms of multiple docking scenarios, thereby contributing to its outstanding early recognition performance in several retrospective virtual screening campaigns. Further explorations demonstrate that CarsiDock can not only guarantee the topological reliability of the binding poses but also successfully reproduce the crucial interactions in crystalized structures, highlighting its superior applicability.

Expanding the toolbox of probiotic Escherichia coli Nissle 1917 for synthetic biology.

Escherichia coli Nissle 1917 (EcN) is a probiotic microbe that has the potential to be developed as a promising chassis for synthetic biology applications. However, the molecular tools and techniques for utilizing EcN remain to be further explored. To address this opportunity, the EcN-based toolbox was systematically expanded, enabling EcN as a powerful platform for more applications. First, two EcN cryptic plasmids and other compatible plasmids were genetically engineered to enrich the manipulable plasmid toolbox for multiple gene coexpression. Next, two EcN-based technologies were developed, including the conjugation strategy for DNA transfer, and quantification of protein expression capability. Finally, the EcN-based applications were further expanded by developing EcN native integrase-mediated genetic engineering and establishing an in vitro cell-free protein synthesis (CFPS) system. Overall, this study expanded the toolbox for manipulating and making full use of EcN as a commonly used probiotic chassis, providing several simplified, dependable, and predictable strategies for researchers working in synthetic biology fields.

Proposal Distribution Calibration for Few-Shot Object Detection.

Adapting object detectors learned with sufficient supervision to novel classes under low data regimes is charming yet challenging. In few-shot object detection (FSOD), the two-step training paradigm is widely adopted to mitigate the severe sample imbalance, i.e., holistic pre-training on base classes, then partial fine-tuning in a balanced setting with all classes. Since unlabeled instances are suppressed as backgrounds in the base training phase, the learned region proposal network (RPN) is prone to produce biased proposals for novel instances, resulting in dramatic performance degradation. Unfortunately, the extreme data scarcity aggravates the proposal distribution bias, hindering the region of interest (RoI) head from evolving toward novel classes. In this brief, we introduce a simple yet effective proposal distribution calibration (PDC) approach to neatly enhance the localization and classification abilities of the RoI head by recycling its localization ability endowed in base training and enriching high-quality positive samples for semantic fine-tuning. Specifically, we sample proposals based on the base proposal statistics to calibrate the distribution bias and impose additional localization and classification losses upon the sampled proposals for fast expanding the base detector to novel classes. Experiments on the commonly used Pascal VOC and MS COCO datasets with explicit state-of-the-art performances justify the efficacy of our PDC for FSOD. Code is available at github.com/Bohao-Lee/PDC.

MagConv: Mask-Guided Convolution for Image Inpainting.

Standard convolution applied to image inpainting would lead to color discrepancy and blurriness for treating valid and invalid/hole regions without difference, which was partially amended by partial convolution (PConv). In PConv, a binary/hard mask was maintained as an indicator of valid and invalid pixels, where valid pixels and invalid pixels were treated differently. However, it can not describe validity degree of an impaired pixel. In addition, mask and image paths were separated, without sharing convolution kernel and exchanging information mutually, reducing data utilization efficiency. In this paper, a mask-guided convolution (MagConv) is proposed for image inpainting. In MagConv, mask and image paths share a convolution kernel to interact with each other and form a joint optimization scheme. In addition, a learnable piecewise activation function is raised to replace the reciprocal function of PConv, providing more flexible and adaptable compensation to convolution contaminated by invalid pixels. It also results in a soft mask of floating-point coefficients from 0 to 1 capable of indicating the validity degree of each pixel. Last but not least, MagConv splits the convolution kernel into positive and negative weights so that they can evaluate the validity of each pixel faithfully. Qualitative and quantitative experiments on the CelebA, Paris StreetView and Places2 datasets demonstrate that our method achieves favorable visual quality against state-of-the-art approaches.

Self-Supervised Arbitrary-Scale Implicit Point Clouds Upsampling.

Point clouds upsampling (PCU), which aims to generate dense and uniform point clouds from the captured sparse input of 3D sensor such as LiDAR, is a practical yet challenging task. It has potential applications in many real-world scenarios, such as autonomous driving, robotics, AR/VR, etc. Deep neural network based methods achieve remarkable success in PCU. However, most existing deep PCU methods either take the end-to-end supervised training, where large amounts of pairs of sparse input and dense ground-truth are required to serve as the supervision; or treat up-scaling of different factors as independent tasks, where multiple networks are required for different scaling factors, leading to significantly increased model complexity and training time. In this article, we propose a novel method that achieves self-supervised and magnification-flexible PCU simultaneously. No longer explicitly learning the mapping between sparse and dense point clouds, we formulate PCU as the task of seeking nearest projection points on the implicit surface for seed points. We then define two implicit neural functions to estimate projection direction and distance respectively, which can be trained by the pretext learning tasks. Moreover, the projection rectification strategy is tailored to remove outliers so as to keep the shape of object clear and sharp. Experimental results demonstrate that our self-supervised learning based scheme achieves competitive or even better performance than state-of-the-art supervised methods.

Efficient framework of solving time-gated reflection matrix for imaging through turbid medium.

Imaging through turbid medium is a long pursuit in many research fields, such as biomedicine, astronomy and automatic vehicle, in which the reflection matrix-based method is a promising solution. However, the epi-detection geometry suffers from round-trip distortion and it is challenging to isolate the input and output aberrations in non-ideal cases due to system imperfections and measurement noises. Here, we present an efficient framework based on single scattering accumulation together with phase unwrapping that can accurately separate input and output aberrations from the noise-affected reflection matrix. We propose to only correct the output aberration while suppressing the input aberration by incoherent averaging. The proposed method is faster in convergence and more robust against noise, avoiding precise and tedious system adjustments. In both simulations and experiments, we demonstrate the diffraction-limited resolution capability under optical thickness beyond 10 scattering mean free paths, showing the potential of applications in neuroscience and dermatology.

Multiple Instance Differentiation Learning for Active Object Detection.

Despite the substantial progress of active learning for image recognition, there lacks a systematic investigation of instance-level active learning for object detection. In this paper, we propose to unify instance uncertainty calculation with image uncertainty estimation for informative image selection, creating a multiple instance differentiation learning (MIDL) method for instance-level active learning. MIDL consists of a classifier prediction differentiation module and a multiple instance differentiation module. The former leverages two adversarial instance classifiers trained on the labeled and unlabeled sets to estimate instance uncertainty of the unlabeled set. The latter treats unlabeled images as instance bags and re-estimates image-instance uncertainty using the instance classification model in a multiple instance learning fashion. Through weighting the instance uncertainty using instance class probability and instance objectness probability under the total probability formula, MIDL unifies the image uncertainty with instance uncertainty in the Bayesian theory framework. Extensive experiments validate that MIDL sets a solid baseline for instance-level active learning. On commonly used object detection datasets, it outperforms other state-of-the-art methods by significant margins, particularly when the labeled sets are small.

Perspectively Equivariant Keypoint Learning for Omnidirectional Images.

Robust keypoint detection on omnidirectional images against large perspective variations, is a key problem in many computer vision tasks. In this paper, we propose a perspectively equivariant keypoint learning framework named OmniKL for addressing this problem. Specifically, the framework is composed of a perspective module and a spherical module, each one including a keypoint detector specific to the type of the input image and a shared descriptor providing uniform description for omnidirectional and perspective images. In these detectors, we propose a differentiable candidate position sorting operation for localizing keypoints, which directly sorts the scores of the candidate positions in a differentiable manner and returns the globally top-K keypoints on the image. This approach does not break the differentiability of the two modules, thus they are end-to-end trainable. Moreover, we design a novel training strategy combining the self-supervised and co-supervised methods to train the framework without any labeled data. Extensive experiments on synthetic and real-world 360° image datasets demonstrate the effectiveness of OmniKL in detecting perspectively equivariant keypoints on omnidirectional images. Our source code are available online at https://github.com/vandeppce/sphkpt.

Deep Attentional Guided Image Filtering.

Guided filter is a fundamental tool in computer vision and computer graphics, which aims to transfer structure information from the guide image to the target image. Most existing methods construct filter kernels from the guidance itself without considering the mutual dependency between the guidance and the target. However, since there typically exist significantly different edges in two images, simply transferring all structural information from the guide to the target would result in various artifacts. To cope with this problem, we propose an effective framework named deep attentional guided image filtering, the filtering process of which can fully integrate the complementary information contained in both images. Specifically, we propose an attentional kernel learning module to generate dual sets of filter kernels from the guidance and the target and then adaptively combine them by modeling the pixelwise dependency between the two images. Meanwhile, we propose a multiscale guided image filtering module to progressively generate the filtering result with the constructed kernels in a coarse-to-fine manner. Correspondingly, a multiscale fusion strategy is introduced to reuse the intermediate results in the coarse-to-fine process. Extensive experiments show that the proposed framework compares favorably with the state-of-the-art methods in a wide range of guided image filtering applications, such as guided super-resolution (SR), cross-modality restoration, and semantic segmentation. Moreover, our scheme achieved the first place in the real depth map SR challenge held in ACM ICMR 2021. The codes can be found at https://github.com/zhwzhong/DAGF.

Asymmetric Loss Functions for Noise-Tolerant Learning: Theory and Applications.

Supervised deep learning has achieved tremendous success in many computer vision tasks, which however is prone to overfit noisy labels. To mitigate the undesirable influence of noisy labels, robust loss functions offer a feasible approach to achieve noise-tolerant learning. In this work, we systematically study the problem of noise-tolerant learning with respect to both classification and regression. Specifically, we propose a new class of loss function, namely asymmetric loss functions (ALFs), which are tailored to satisfy the Bayes-optimal condition and thus are robust to noisy labels. For classification, we investigate general theoretical properties of ALFs on categorical noisy labels, and introduce the asymmetry ratio to measure the asymmetry of a loss function. We extend several commonly-used loss functions, and establish the necessary and sufficient conditions to make them asymmetric and thus noise-tolerant. For regression, we extend the concept of noise-tolerant learning for image restoration with continuous noisy labels. We theoretically prove that lp loss ( ) is noise-tolerant for targets with the additive white Gaussian noise. For targets with general noise, we introduce two losses as surrogates of l0 loss that seeks the mode when clean pixels keep dominant. Experimental results demonstrate that ALFs can achieve better or comparative performance compared with the state-of-the-arts. The source code of our method is available at: https://github.com/hitcszx/ALFs.

Engineering Escherichia coli to Utilize Erythritol as Sole Carbon Source.

Erythritol, one of the natural sugar alcohols, is widely used as a sugar substitute sweetener in food industries. Humans themselves are not able to catabolize erythritol and their gut microbes lack related catabolic pathways either to metabolize erythritol. Here, Escherichia coli (E. coli) is engineered to utilize erythritol as sole carbon source aiming for defined applications. First, the erythritol metabolic gene cluster is isolated and the erythritol-binding transcriptional repressor and its DNA-binding site are experimentally characterized. Transcriptome analysis suggests that carbohydrate metabolism-related genes in the engineered E. coli are overall upregulated. In particular, the enzymes of transaldolase (talA and talB) and transketolase (tktA and tktB) are notably overexpressed (e.g., the expression of tktB is improved by nearly sixfold). By overexpression of the four genes, cell growth can be increased as high as three times compared to the cell cultivation without overexpression. Finally, engineered E. coli strains can be used as a living detector to distinguish erythritol-containing soda soft drinks and can grow in the simulated intestinal fluid supplemented with erythritol. This work is expected to inspire the engineering of more hosts to respond and utilize erythritol for broad applications in metabolic engineering, synthetic biology, and biomedical engineering.