Conditional Molecular Generation Net Enables Automated Structure Elucidation Based on 13C NMR Spectra and Prior Knowledge.

Structure elucidation of unknown compounds based on nuclear magnetic resonance (NMR) remains a challenging problem in both synthetic organic and natural product chemistry. Library matching has been an efficient method to assist structure elucidation. However, it is limited by the coverage of libraries. In addition, prior knowledge such as molecular fragments is neglected. To solve the problem, we propose a conditional molecular generation net (CMGNet) to allow input of multiple sources of information. CMGNet not only uses 13C NMR spectrum data as input but molecular formulas and fragments of molecules are also employed as input conditions. Our model applies large-scale pretraining for molecular understanding and fine-tuning on two NMR spectral data sets of different granularity levels to accommodate structure elucidation tasks. CMGNet generates structures based on 13C NMR data, molecular formula, and fragment information, with a recovery rate of 94.17% in the top 10 recommendations. In addition, the generative model performed well in the generation of various classes of compounds and in the structural revision task. CMGNet has a deep understanding of molecular connectivities from 13C NMR, molecular formula, and fragments, paving the way for a new paradigm of deep learning-assisted inverse problem-solving.

Efficient framework of solving time-gated reflection matrix for imaging through turbid medium.

Imaging through turbid medium is a long pursuit in many research fields, such as biomedicine, astronomy and automatic vehicle, in which the reflection matrix-based method is a promising solution. However, the epi-detection geometry suffers from round-trip distortion and it is challenging to isolate the input and output aberrations in non-ideal cases due to system imperfections and measurement noises. Here, we present an efficient framework based on single scattering accumulation together with phase unwrapping that can accurately separate input and output aberrations from the noise-affected reflection matrix. We propose to only correct the output aberration while suppressing the input aberration by incoherent averaging. The proposed method is faster in convergence and more robust against noise, avoiding precise and tedious system adjustments. In both simulations and experiments, we demonstrate the diffraction-limited resolution capability under optical thickness beyond 10 scattering mean free paths, showing the potential of applications in neuroscience and dermatology.

Learning to autofocus in whole slide imaging via physics-guided deep cascade networks.

Whole slide imaging (WSI), is an essential technology for digital pathology, the performance of which is primarily affected by the autofocusing process. Conventional autofocusing methods either are time-consuming or require additional hardware and thus are not compatible with the current WSI systems. In this paper, we propose an effective learning-based method for autofocusing in WSI, which can realize accurate autofocusing at high speed as well as without any optical hardware modifications. Our method is inspired by an observation that sample images captured by WSI have distinctive characteristics with respect to positive / negative defocus offsets, due to the asymmetry effect of optical aberrations. Based on this physical knowledge, we develop novel deep cascade networks to enhance autofocusing quality. Specifically, to handle the effect of optical aberrations, a binary classification network is tailored to distinguish sample images with positive / negative defocus. As such, samples within the same category share similar characteristics. It facilitates the followed refocusing network, which is designed to learn the mapping between the defocus image and defocus distance. Experimental results demonstrate that our method achieves superior autofocusing performance to other related methods.

Coexistence of light emission and detection in a III-nitride quantum well diode.

The demand for on-chip multifunctional optoelectronic systems is increasing in today's Internet of Things era. III-nitride quantum well diodes (QWDs) can transmit and receive information through visible light and can be used as both light-emitting diodes (LEDs) and photodetectors (PDs). Spectral emission-detection overlap gives the III-nitride QWD an intriguing capability to detect and modulate light emitted by itself. In this paper, the coexistence of light emission and detection in a III-nitride QWD is experimentally demonstrated, and a wireless video communication system through light is established. When approximately biasing and illuminating at the same time, the III-nitride QWD can achieve light emission and detection simultaneously. This work provides a foundation for the development of multifunctional III-nitride QWDs and the realization of device-to-device data communication.

Full-duplex visible light communication system using a single channel.

Multiple quantum well (MQW) III-nitride diodes can emit light and detect light at the same time. In particular, given the overlapping region between the emission spectrum and the detection spectrum, the III-nitride diode can absorb photons of shorter wavelengths generated from another III-nitride diode with the same MQW structure. In this study, a wireless visible light communication system was established using two pairs of identical III-nitride diodes with different wavelengths. In this system, two green light diode chips were used to transmit and receive green light signals on both sides. We have integrated two blue light chips with optical filtering in the middle of the optical link to carry out blue light communication, with one end transmitting and one end receiving. Simultaneously, green light was allowed to pass through two blue light chips for optical communication. Combined with a distributed Bragg reflection (DBR) coating, we proposed using four chips in one optical path to carry out optical communication between chips with the same wavelength and used the coating principle to gate the optical wavelength to filter the clutter of green light chips on both sides to make the channel purer and the symbols easier to demodulate. Based on this multifunctional equipment, advanced single-optical path, III-nitride, full-duplex optical communication links can be developed for the deployment of the Internet of Things.

Cross-Modal Retrieval between 13C NMR Spectra and Structures for Compound Identification Using Deep Contrastive Learning.

Library matching using carbon-13 nuclear magnetic resonance (13C NMR) spectra has been a popular method adopted in compound identification systems. However, the usability of existing approaches has been restricted as enlarging a library containing both a chemical structure and spectrum is a costly and time-consuming process. Therefore, we propose a fundamentally different, novel approach to match 13C NMR spectra directly against a molecular structure library. We develop a cross-modal retrieval between spectrum and structure (CReSS) system using deep contrastive learning, which allows us to search a molecular structure library using the 13C NMR spectrum of a compound. In the test of searching 41,494 13C NMR spectra against a reference structure library containing 10.4 million compounds, CReSS reached a recall@10 accuracy of 91.64% and a processing speed of 0.114 s per query spectrum. When further incorporating a filter with a molecular weight tolerance of 5 Da, CReSS achieved a new remarkable recall@10 of 98.39%. Furthermore, CReSS has potential in detecting scaffolds of novel structures and demonstrates great performance for the task of structural revision. CReSS is built and developed to bridge the gap between 13C NMR spectra and structures and could be generally applicable in compound identification.

Ptychography-based high-throughput lensless on-chip microscopy via incremental proximal algorithms.

Ptychography-based lensless on-chip microscopy enables high-throughput imaging by retrieving the missing phase information from intensity measurements. Numerous reconstruction algorithms for ptychography have been proposed, yet only a few incremental algorithms can be extended to lensless on-chip microscopy because of large-scale datasets but limited computational efficiency. In this paper, we propose the use of accelerated proximal gradient methods for blind ptychographic phase retrieval in lensless on-chip microscopy. Incremental gradient approaches are adopted in the reconstruction routine. Our algorithms divide the phase retrieval problem into sub-problems involving the evaluation of proximal operator, stochastic gradient descent, and Wirtinger derivatives. We benchmark the performances of accelerated proximal gradient, extended ptychographic iterative engine, and alternating direction method of multipliers, and discuss their convergence and accuracy in both noisy and noiseless cases. We also validate our algorithms using experimental datasets, where full field of view measurements are captured to recover the high-resolution complex samples. Among these algorithms, accelerated proximal gradient presents the overall best performance regarding accuracy and convergence rate. The proposed methods may find applications in ptychographic reconstruction, especially for cases where a wide field of view and high resolution are desired at the same time.

Tailoring atomic 1T phase CrTe2forin situfabrication.

Controllable tailoring and understanding the phase-structure relationship of the 1T phase two-dimensional (2D) materials are critical for their applications in nanodevices. Thein situtransmission electron microscope (TEM) could regulate and monitor the evolution process of the nanostructure of 2D material with atomic resolution. In this work, a controllably tailoring 1T-CrTe2nanopore is carried out by thein situTEM. A preferred formation of the 1T-CrTe2border structure and nanopore healing process are studied at the atomic scale. The controllable tailoring of the 1T phase nanopore could be achieved by regulating the transformation of two types of low indices of crystal faces {101¯0} and {112¯0} at the nanopore border. Machine learning is applied to automatically process the TEM images with high efficiency. By adopting the deep-learning-based image segmentation method and augmenting the TEM images specifically, the nanopore of the TEM image could be automatically identified and the evaluation result of DICE metric reaches 93.17% on test set. This work presents the unique structure evolution of 1T phase 2D material and the computer aided high efficiency TEM data analysis based on deep learning. The techniques applied in this work could be generalized to other materials for controlled nanostructure regulation and automatic TEM image analyzation.

Feasibility of using deep learning to detect coronary artery disease based on facial photo.

AIMS: Facial features were associated with increased risk of coronary artery disease (CAD). We developed and validated a deep learning algorithm for detecting CAD based on facial photos. METHODS AND RESULTS: We conducted a multicentre cross-sectional study of patients undergoing coronary angiography or computed tomography angiography at nine Chinese sites to train and validate a deep convolutional neural network for the detection of CAD (at least one ≥50% stenosis) from patient facial photos. Between July 2017 and March 2019, 5796 patients from eight sites were consecutively enrolled and randomly divided into training (90%, n = 5216) and validation (10%, n = 580) groups for algorithm development. Between April 2019 and July 2019, 1013 patients from nine sites were enrolled in test group for algorithm test. Sensitivity, specificity, and area under the receiver operating characteristic curve (AUC) were calculated using radiologist diagnosis as the reference standard. Using an operating cut point with high sensitivity, the CAD detection algorithm had sensitivity of 0.80 and specificity of 0.54 in the test group; the AUC was 0.730 (95% confidence interval, 0.699-0.761). The AUC for the algorithm was higher than that for the Diamond-Forrester model (0.730 vs. 0.623, P < 0.001) and the CAD consortium clinical score (0.730 vs. 0.652, P < 0.001). CONCLUSION: Our results suggested that a deep learning algorithm based on facial photos can assist in CAD detection in this Chinese cohort. This technique may hold promise for pre-test CAD probability assessment in outpatient clinics or CAD screening in community. Further studies to develop a clinical available tool are warranted.

Prior-information-free single-shot scattering imaging beyond the memory effect.

Imaging beyond the memory effect (ME) is critical to seeing through the scattering media. Methods proposed before have suffered from invasive point spread function measurement or the availability of prior information of the imaging targets. In this Letter, we propose a prior-information-free single-shot scattering imaging method to exceed the ME range. The autocorrelation of each imaging target is separated blindly from the autocorrelation of the recorded dual-target speckle via Fourier spectrum guessing and iterative energy constrained compensation. Working together with phase retrieval, dual targets exceeding the ME range can be reconstructed via a single shot. The effectiveness of the algorithm is verified by simulated experiments and a real imaging system.

Information transduction capacity reduces the uncertainties in annotation-free isoform discovery and quantification.

The automated transcript discovery and quantification of high-throughput RNA sequencing (RNA-seq) data are important tasks of next-generation sequencing (NGS) research. However, these tasks are challenging due to the uncertainties that arise in the inference of complete splicing isoform variants from partially observed short reads. Here, we address this problem by explicitly reducing the inherent uncertainties in a biological system caused by missing information. In our approach, the RNA-seq procedure for transforming transcripts into short reads is considered an information transmission process. Consequently, the data uncertainties are substantially reduced by exploiting the information transduction capacity of information theory. The experimental results obtained from the analyses of simulated datasets and RNA-seq datasets from cell lines and tissues demonstrate the advantages of our method over state-of-the-art competitors. Our algorithm is an open-source implementation of MaxInfo.

Deep Lossy Plus Residual Coding for Lossless and Near-Lossless Image Compression.

Lossless and near-lossless image compression is of paramount importance to professional users in many technical fields, such as medicine, remote sensing, precision engineering and scientific research. But despite rapidly growing research interests in learning-based image compression, no published method offers both lossless and near-lossless modes. In this paper, we propose a unified and powerful deep lossy plus residual (DLPR) coding framework for both lossless and near-lossless image compression. In the lossless mode, the DLPR coding system first performs lossy compression and then lossless coding of residuals. We solve the joint lossy and residual compression problem in the approach of VAEs, and add autoregressive context modeling of the residuals to enhance lossless compression performance. In the near-lossless mode, we quantize the original residuals to satisfy a given ℓ∞ error bound, and propose a scalable near-lossless compression scheme that works for variable ℓ∞ bounds instead of training multiple networks. To expedite the DLPR coding, we increase the degree of algorithm parallelization by a novel design of coding context, and accelerate the entropy coding with adaptive residual interval. Experimental results demonstrate that the DLPR coding system achieves both the state-of-the-art lossless and near-lossless image compression performance with competitive coding speed.

Multispectral Depolarization Based Living-Skin Detection: A New Measurement Principle.

Camera-based photoplethysmographic imaging enabled the segmentation of living-skin tissues in a video, but it has inherent limitations to be used in real-life applications such as video health monitoring and face anti-spoofing. Inspired by the use of polarization for improving vital signs monitoring (i.e. specular reflection removal), we observed that skin tissues have an attractive property of wavelength-dependent depolarization due to its multi-layer structure containing different absorbing chromophores, i.e. polarized light photons with longer wavelengths (R) have deeper skin penetrability and thus experience thorougher depolarization than those with shorter wavelengths (G and B). Thus we proposed a novel dual-polarization setup and an elegant algorithm (named "MSD") that exploits the nature of multispectral depolarization of skin tissues to detect living-skin pixels, which only requires two images sampled at the parallel and cross polarizations to estimate the characteristic chromaticity changes (R/G) caused by tissue depolarization. Our proposal was verified in both the laboratory and hospital settings (ICU and NICU) focused on anti-spoofing and patient skin segmentation. The clinical experiments in ICU also indicate the potential of MSD for skin perfusion analysis, which may lead to a new diagnostic imaging approach in the future.

Expanding the toolbox of probiotic Escherichia coli Nissle 1917 for synthetic biology.

Escherichia coli Nissle 1917 (EcN) is a probiotic microbe that has the potential to be developed as a promising chassis for synthetic biology applications. However, the molecular tools and techniques for utilizing EcN remain to be further explored. To address this opportunity, the EcN-based toolbox was systematically expanded, enabling EcN as a powerful platform for more applications. First, two EcN cryptic plasmids and other compatible plasmids were genetically engineered to enrich the manipulable plasmid toolbox for multiple gene coexpression. Next, two EcN-based technologies were developed, including the conjugation strategy for DNA transfer, and quantification of protein expression capability. Finally, the EcN-based applications were further expanded by developing EcN native integrase-mediated genetic engineering and establishing an in vitro cell-free protein synthesis (CFPS) system. Overall, this study expanded the toolbox for manipulating and making full use of EcN as a commonly used probiotic chassis, providing several simplified, dependable, and predictable strategies for researchers working in synthetic biology fields.

Cell-free biosynthesis and engineering of ribosomally synthesized lanthipeptides.

Ribosomally synthesized and post-translationally modified peptides (RiPPs) are a major class of natural products with diverse chemical structures and potent biological activities. A vast majority of RiPP gene clusters remain unexplored in microbial genomes, which is partially due to the lack of rapid and efficient heterologous expression systems for RiPP characterization and biosynthesis. Here, we report a unified biocatalysis (UniBioCat) system based on cell-free gene expression for rapid biosynthesis and engineering of RiPPs. We demonstrate UniBioCat by reconstituting a full biosynthetic pathway for de novo biosynthesis of salivaricin B, a lanthipeptide RiPP. Next, we delete several protease/peptidase genes from the source strain to enhance the performance of UniBioCat, which then can synthesize and screen salivaricin B variants with enhanced antimicrobial activity. Finally, we show that UniBioCat is generalizable by synthesizing and evaluating the bioactivity of ten uncharacterized lanthipeptides. We expect UniBioCat to accelerate the discovery, characterization, and synthesis of RiPPs.

Prediction of coronary artery disease based on facial temperature information captured by non-contact infrared thermography.

BACKGROUND: Current approaches for initial coronary artery disease (CAD) assessment rely on pretest probability (PTP) based on risk factors and presentations, with limited performance. Infrared thermography (IRT), a non-contact technology that detects surface temperature, has shown potential in assessing atherosclerosis-related conditions, particularly when measured from body regions such as faces. We aim to assess the feasibility of using facial IRT temperature information with machine learning for the prediction of CAD. METHODS: Individuals referred for invasive coronary angiography or coronary CT angiography (CCTA) were enrolled. Facial IRT images captured before confirmatory CAD examinations were used to develop and validate a deep-learning IRT image model for detecting CAD. We compared the performance of the IRT image model with the guideline-recommended PTP model on the area under the curve (AUC). In addition, interpretable IRT tabular features were extracted from IRT images to further validate the predictive value of IRT information. RESULTS: A total of 460 eligible participants (mean (SD) age, 58.4 (10.4) years; 126 (27.4%) female) were included. The IRT image model demonstrated outstanding performance (AUC 0.804, 95% CI 0.785 to 0.823) compared with the PTP models (AUC 0.713, 95% CI 0.691 to 0.734). A consistent level of superior performance (AUC 0.796, 95% CI 0.782 to 0.811), achieved with comprehensive interpretable IRT features, further validated the predictive value of IRT information. Notably, even with only traditional temperature features, a satisfactory performance (AUC 0.786, 95% CI 0.769 to 0.803) was still upheld. CONCLUSION: In this prospective study, we demonstrated the feasibility of using non-contact facial IRT information for CAD prediction.

CarsiDock: a deep learning paradigm for accurate protein-ligand docking and screening based on large-scale pre-training.

The expertise accumulated in deep neural network-based structure prediction has been widely transferred to the field of protein-ligand binding pose prediction, thus leading to the emergence of a variety of deep learning-guided docking models for predicting protein-ligand binding poses without relying on heavy sampling. However, their prediction accuracy and applicability are still far from satisfactory, partially due to the lack of protein-ligand binding complex data. To this end, we create a large-scale complex dataset containing ∼9 M protein-ligand docking complexes for pre-training, and propose CarsiDock, the first deep learning-guided docking approach that leverages pre-training of millions of predicted protein-ligand complexes. CarsiDock contains two main stages, i.e., a deep learning model for the prediction of protein-ligand atomic distance matrices, and a translation, rotation and torsion-guided geometry optimization procedure to reconstruct the matrices into a credible binding pose. The pre-training and multiple innovative architectural designs facilitate the dramatically improved docking accuracy of our approach over the baselines in terms of multiple docking scenarios, thereby contributing to its outstanding early recognition performance in several retrospective virtual screening campaigns. Further explorations demonstrate that CarsiDock can not only guarantee the topological reliability of the binding poses but also successfully reproduce the crucial interactions in crystalized structures, highlighting its superior applicability.

MagConv: Mask-Guided Convolution for Image Inpainting.

Standard convolution applied to image inpainting would lead to color discrepancy and blurriness for treating valid and invalid/hole regions without difference, which was partially amended by partial convolution (PConv). In PConv, a binary/hard mask was maintained as an indicator of valid and invalid pixels, where valid pixels and invalid pixels were treated differently. However, it can not describe validity degree of an impaired pixel. In addition, mask and image paths were separated, without sharing convolution kernel and exchanging information mutually, reducing data utilization efficiency. In this paper, a mask-guided convolution (MagConv) is proposed for image inpainting. In MagConv, mask and image paths share a convolution kernel to interact with each other and form a joint optimization scheme. In addition, a learnable piecewise activation function is raised to replace the reciprocal function of PConv, providing more flexible and adaptable compensation to convolution contaminated by invalid pixels. It also results in a soft mask of floating-point coefficients from 0 to 1 capable of indicating the validity degree of each pixel. Last but not least, MagConv splits the convolution kernel into positive and negative weights so that they can evaluate the validity of each pixel faithfully. Qualitative and quantitative experiments on the CelebA, Paris StreetView and Places2 datasets demonstrate that our method achieves favorable visual quality against state-of-the-art approaches.

Self-Supervised Arbitrary-Scale Implicit Point Clouds Upsampling.

Point clouds upsampling (PCU), which aims to generate dense and uniform point clouds from the captured sparse input of 3D sensor such as LiDAR, is a practical yet challenging task. It has potential applications in many real-world scenarios, such as autonomous driving, robotics, AR/VR, etc. Deep neural network based methods achieve remarkable success in PCU. However, most existing deep PCU methods either take the end-to-end supervised training, where large amounts of pairs of sparse input and dense ground-truth are required to serve as the supervision; or treat up-scaling of different factors as independent tasks, where multiple networks are required for different scaling factors, leading to significantly increased model complexity and training time. In this article, we propose a novel method that achieves self-supervised and magnification-flexible PCU simultaneously. No longer explicitly learning the mapping between sparse and dense point clouds, we formulate PCU as the task of seeking nearest projection points on the implicit surface for seed points. We then define two implicit neural functions to estimate projection direction and distance respectively, which can be trained by the pretext learning tasks. Moreover, the projection rectification strategy is tailored to remove outliers so as to keep the shape of object clear and sharp. Experimental results demonstrate that our self-supervised learning based scheme achieves competitive or even better performance than state-of-the-art supervised methods.

Deep Attentional Guided Image Filtering.

Guided filter is a fundamental tool in computer vision and computer graphics, which aims to transfer structure information from the guide image to the target image. Most existing methods construct filter kernels from the guidance itself without considering the mutual dependency between the guidance and the target. However, since there typically exist significantly different edges in two images, simply transferring all structural information from the guide to the target would result in various artifacts. To cope with this problem, we propose an effective framework named deep attentional guided image filtering, the filtering process of which can fully integrate the complementary information contained in both images. Specifically, we propose an attentional kernel learning module to generate dual sets of filter kernels from the guidance and the target and then adaptively combine them by modeling the pixelwise dependency between the two images. Meanwhile, we propose a multiscale guided image filtering module to progressively generate the filtering result with the constructed kernels in a coarse-to-fine manner. Correspondingly, a multiscale fusion strategy is introduced to reuse the intermediate results in the coarse-to-fine process. Extensive experiments show that the proposed framework compares favorably with the state-of-the-art methods in a wide range of guided image filtering applications, such as guided super-resolution (SR), cross-modality restoration, and semantic segmentation. Moreover, our scheme achieved the first place in the real depth map SR challenge held in ACM ICMR 2021. The codes can be found at https://github.com/zhwzhong/DAGF.