Test-Time Training for Deep MS/MS Spectrum Prediction Improves Peptide Identification.

In bottom-up proteomics, peptide-spectrum matching is critical for peptide and protein identification. Recently, deep learning models have been used to predict tandem mass spectra of peptides, enabling the calculation of similarity scores between the predicted and experimental spectra for peptide-spectrum matching. These models follow the supervised learning paradigm, which trains a general model using paired peptides and spectra from standard data sets and directly employs the model on experimental data. However, this approach can lead to inaccurate predictions due to differences between the training data and the experimental data, such as sample types, enzyme specificity, and instrument calibration. To tackle this problem, we developed a test-time training paradigm that adapts the pretrained model to generate experimental data-specific models, namely, PepT3. PepT3 yields a 10-40% increase in peptide identification depending on the variability in training and experimental data. Intriguingly, when applied to a patient-derived immunopeptidomic sample, PepT3 increases the identification of tumor-specific immunopeptide candidates by 60%. Two-thirds of the newly identified candidates are predicted to bind to the patient's human leukocyte antigen isoforms. To facilitate access of the model and all the results, we have archived all the intermediate files in Zenodo.org with identifier 8231084.

A volatile microemulsion method of preparing water-soluble photo-absorbers for 3D printing of high-resolution, high-water-content hydrogel structures.

Digital light processing (DLP)-based three-dimensional (3D) printing is an ideal tool to manufacture hydrogel structures in complex 3D forms. Using DLP to print hydrogel structures with high resolution requires the addition of water-soluble photo-absorbers to absorb excess light and confine photopolymerization to the desired area. However, the current photo-absorbers for hydrogel printing are neither efficient to absorb the excess light nor water-soluble. Herein, we report a volatile microemulsion template method that converts a wide range of commercial non-water-soluble photo-absorbers including Sudan orange G, quercetin, and many others to water-soluble nanoparticles with solubility above 1.0 g mL-1. After using these water-soluble photo-absorber nanoparticles, the highest lateral and vertical resolutions of printing high-water-content (70-80 wt%) hydrogels can be improved to 5 µm and 20 µm, respectively. Moreover, the quercetin nanoparticle can be easily washed out so that we achieve colorless and transparent printed hydrogel structures with excellent mechanical deformability and biocompatibility as well as thermally controllable variations on transparency and actuation. The proposed methods pave a new efficient way to develop water-soluble photo-absorbers, which helps to greatly improve the printing resolution of the high-water-content hydrogel structure and would be beneficial to extend the application scope of hydrogels.

Neonatal CX26 removal impairs neocortical development and leads to elevated anxiety.

Electrical coupling between excitatory neurons in the neocortex is developmentally regulated. It is initially prominent but eliminated at later developmental stages when chemical synapses emerge. However, it remains largely unclear whether early electrical coupling networks broadly contribute to neocortical circuit formation and animal behavior. Here, we report that neonatal electrical coupling between neocortical excitatory neurons is critical for proper neuronal development, synapse formation, and animal behavior. Conditional deletion of Connexin 26 (CX26) in the superficial layer excitatory neurons of the mouse neocortex around birth significantly reduces spontaneous firing activity and the frequency and size of spontaneous network oscillations at postnatal day 5-6. Moreover, CX26-conditional knockout (CX26-cKO) neurons tend to have simpler dendritic trees and lower spine density compared with wild-type neurons. Importantly, early, but not late, postnatal deletion of CX26, decreases the frequency of miniature excitatory postsynaptic currents (mEPSCs) in both young and adult mice, whereas miniature inhibitory postsynaptic currents (mIPSCs) were unaffected. Furthermore, CX26-cKO mice exhibit increased anxiety-related behavior. These results suggest that electrical coupling between excitatory neurons at early postnatal stages is a critical step for neocortical development and function.

A Truthful Incentive Mechanism for Online Recruitment in Mobile Crowd Sensing System.

We investigate emerging mobile crowd sensing (MCS) systems, in which new cloud-based platforms sequentially allocate homogenous sensing jobs to dynamically-arriving users with uncertain service qualities. Given that human beings are selfish in nature, it is crucial yet challenging to design an efficient and truthful incentive mechanism to encourage users to participate. To address the challenge, we propose a novel truthful online auction mechanism that can efficiently learn to make irreversible online decisions on winner selections for new MCS systems without requiring previous knowledge of users. Moreover, we theoretically prove that our incentive possesses truthfulness, individual rationality and computational efficiency. Extensive simulation results under both real and synthetic traces demonstrate that our incentive mechanism can reduce the payment of the platform, increase the utility of the platform and social welfare.

Feature Mixture on Pre-trained Model for Few-shot Learning.

Few-shot learning (FSL) aims at recognizing a novel object under limited training samples. A robust feature extractor (backbone) can significantly improve the recognition performance of the FSL model. However, training an effective backbone is a challenging issue since (1) designing and validating structures of backbones are time-consuming and expensive processes, and (2) a backbone trained on the known (base) categories is more inclined to focus on the textures of the objects it learns, which is hard to describe the novel samples. To solve these problems, we propose a feature mixture operation on the pre-trained (fixed) features: (1) We replace a part of the values of the feature map from a novel category with the content of other feature maps to increase the generalizability and diversity of training samples, which avoids retraining a complex backbone with high computational costs. (2) We use the similarities between the features to constrain the mixture operation, which helps the classifier focus on the representations of the novel object where these representations are hidden in the features from the pre-trained backbone with biased training. Experimental studies on five benchmark datasets in both inductive and transductive settings demonstrate the effectiveness of our feature mixture (FM). Specifically, compared with the baseline on the Mini-ImageNet dataset, it achieves 3.8% and 4.2% accuracy improvements for 1 and 5 training samples, respectively. Additionally, the proposed mixture operation can be used to improve other existing FSL methods based on backbone training.

Learning to Double-Check Model Prediction From a Causal Perspective.

The present machine learning schema typically uses a one-pass model inference (e.g., forward propagation) to make predictions in the testing phase. It is inherently different from human students who double-check the answer during examinations especially when the confidence is low. To bridge this gap, we propose a learning to double-check (L2D) framework, which formulates double check as a learnable procedure with two core operations: recognizing unreliable predictions and revising predictions. To judge the correctness of a prediction, we resort to counterfactual faithfulness in causal theory and design a contrastive faithfulness measure. In particular, L2D generates counterfactual features by imagining: "what would the sample features be if its label was the predicted class" and judges the prediction by the faithfulness of the counterfactual features. Furthermore, we design a simple and effective revision module to revise the original model prediction according to the faithfulness. We apply the L2D framework to three classification models and conduct experiments on two public datasets for image classification, validating the effectiveness of L2D in prediction correctness judgment and revision.

Direct 4D printing of ceramics driven by hydrogel dehydration.

4D printing technology combines 3D printing and stimulus-responsive materials, enabling construction of complex 3D objects efficiently. However, unlike smart soft materials, 4D printing of ceramics is a great challenge due to the extremely weak deformability of ceramics. Here, we report a feasible and efficient manufacturing and design approach to realize direct 4D printing of ceramics. Photocurable ceramic elastomer slurry and hydrogel precursor are developed for the fabrication of hydrogel-ceramic laminates via multimaterial digital light processing 3D printing. Flat patterned laminates evolve into complex 3D structures driven by hydrogel dehydration, and then turn into pure ceramics after sintering. Considering the dehydration-induced deformation and sintering-induced shape retraction, we develop a theoretical model to calculate the curvatures of bent laminate and sintered ceramic part. Then, we build a design flow for direct 4D printing of various complex ceramic objects. This approach opens a new avenue for the development of ceramic 4D printing technology.

Reconfigurable 4D printing via mechanically robust covalent adaptable network shape memory polymer.

4D printing enables 3D printed structures to change shape over "time" in response to environmental stimulus. Because of relatively high modulus, shape memory polymers (SMPs) have been widely used for 4D printing. However, most SMPs for 4D printing are thermosets, which only have one permanent shape. Despite the efforts that implement covalent adaptable networks (CANs) into SMPs to achieve shape reconfigurability, weak thermomechanical properties of the current CAN-SMPs exclude them from practical applications. Here, we report reconfigurable 4D printing via mechanically robust CAN-SMPs (MRC-SMPs), which have high deformability at both programming and reconfiguration temperatures (>1400%), high Tg (75°C), and high room temperature modulus (1.06 GPa). The high printability for DLP high-resolution 3D printing allows MRC-SMPs to create highly complex SMP 3D structures that can be reconfigured multiple times under large deformation. The demonstrations show that the reconfigurable 4D printing allows one printed SMP structure to fulfill multiple tasks.

GCRec: Graph-Augmented Capsule Network for Next-Item Recommendation.

Next-item recommendation has been a hot research, which aims at predicting the next action by modeling users' behavior sequences. While previous efforts toward this task have been made in capturing complex item transition patterns, we argue that they still suffer from three limitations: 1) they have difficulty in explicitly capturing the impact of inherent order of item transition patterns; 2) only a simple and crude embedding is insufficient to yield satisfactory long-term users' representations from limited training sequences; and 3) they are incapable of dynamically integrating long-term and short-term user interest modeling. In this work, we propose a novel solution named graph-augmented capsule network (GCRec), which exploits sequential user behaviors in a more fine-grained manner. Specifically, we employ a linear graph convolution module to learn informative long-term representations of users. Furthermore, we devise a user-specific capsule module and a position-aware gating module, which are sensitive to the relative sequential order of the recently interacted items, to capture sequential patterns at union-level and point-level. To aggregate the long-term and short-term user interests as a representative vector, we design a dual-gating mechanism, which could decide the contribution ratio of each module given different contextual information. Through extensive experiments on four benchmarks, we validate the rationality and effectiveness of GCRec on the next-item recommendation task.

Reinforced Causal Explainer for Graph Neural Networks.

Explainability is crucial for probing graph neural networks (GNNs), answering questions like "Why the GNN model makes a certain prediction?". Feature attribution is a prevalent technique of highlighting the explanatory subgraph in the input graph, which plausibly leads the GNN model to make its prediction. Various attribution methods have been proposed to exploit gradient-like or attention scores as the attributions of edges, then select the salient edges with top attribution scores as the explanation. However, most of these works make an untenable assumption - the selected edges are linearly independent - thus leaving the dependencies among edges largely unexplored, especially their coalition effect. We demonstrate unambiguous drawbacks of this assumption - making the explanatory subgraph unfaithful and verbose. To address this challenge, we propose a reinforcement learning agent, Reinforced Causal Explainer (RC-Explainer). It frames the explanation task as a sequential decision process - an explanatory subgraph is successively constructed by adding a salient edge to connect the previously selected subgraph. Technically, its policy network predicts the action of edge addition, and gets a reward that quantifies the action's causal effect on the prediction. Such reward accounts for the dependency of the newly-added edge and the previously-added edges, thus reflecting whether they collaborate together and form a coalition to pursue better explanations. It is trained via policy gradient to optimize the reward stream of edge sequences. As such, RC-Explainer is able to generate faithful and concise explanations, and has a better generalization power to unseen graphs. When explaining different GNNs on three graph classification datasets, RC-Explainer achieves better or comparable performance to state-of-the-art approaches w.r.t. two quantitative metrics: predictive accuracy, contrastivity, and safely passes sanity checks and visual inspections. Codes and datasets are available at https://github.com/xiangwang1223/reinforced_causal_explainer.

Polyelectrolyte elastomer-based ionotronic sensors with multi-mode sensing capabilities via multi-material 3D printing.

Stretchable ionotronics have drawn increasing attention during the past decade, enabling myriad applications in engineering and biomedicine. However, existing ionotronic sensors suffer from limited sensing capabilities due to simple device structures and poor stability due to the leakage of ingredients. In this study, we rationally design and fabricate a plethora of architected leakage-free ionotronic sensors with multi-mode sensing capabilities, using DLP-based 3D printing and a polyelectrolyte elastomer. We synthesize a photo-polymerizable ionic monomer for the polyelectrolyte elastomer, which is stretchable, transparent, ionically conductive, thermally stable, and leakage-resistant. The printed sensors possess robust interfaces and extraordinary long-term stability. The multi-material 3D printing allows high flexibility in structural design, enabling the sensing of tension, compression, shear, and torsion, with on-demand tailorable sensitivities through elaborate programming of device architectures. Furthermore, we fabricate integrated ionotronic sensors that can perceive different mechanical stimuli simultaneously without mutual signal interferences. We demonstrate a sensing kit consisting of four shear sensors and one compressive sensor, and connect it to a remote-control system that is programmed to wirelessly control the flight of a drone. Multi-material 3D printing of leakage-free polyelectrolyte elastomers paves new avenues for manufacturing stretchable ionotronics by resolving the deficiencies of stability and functionalities simultaneously.

Multimaterial Three-Dimensional Printing of Ultraviolet-Curable Ionic Conductive Elastomers with Diverse Polymers for Multifunctional Flexible Electronics.

Ionic conductive elastomers (ICEs) are emerging stretchable and ionic conductive materials that are solvent-free and thus demonstrate excellent thermal stability. Three-dimensional (3D) printing that creates complex 3D structures in free forms is considered as an ideal approach to manufacture sophisticated ICE-based devices. However, the current technologies constrain 3D printed ICE structures in a single material, which greatly limits functionality and performance of ICE-based devices and machines. Here, we report a digital light processing (DLP)-based multimaterial 3D printing capability to seemly integrate ultraviolet-curable ICE (UV-ICE) with nonconductive materials to create ionic flexible electronic devices in 3D forms with enhanced performance. This unique capability allows us to readily manufacture various 3D flexible electronic devices. To demonstrate this, we printed UV-ICE circuits into polymer substrates with different mechanical properties to create resistive strain and force sensors; we printed flexible capacitive sensors with high sensitivity (2 kPa-1) and a wide range of measured pressures (from 5 Pa to 550 kPa) by creating a complex microstructure in the dielectric layer; we even realized ionic conductor-activated four-dimensional (4D) printing by printing a UV-ICE circuit into a shape memory polymer substrate. The proposed approach paves a new efficient way to realize multifunctional flexible devices and machines by bonding ICEs with other polymers in 3D forms.

Shape-Tunable 4D Printing of LCEs via Cooling Rate Modulation: Stimulus-Free Locking of Actuated State at Room Temperature.

Liquid crystal elastomers (LCEs) have garnered considerable attention in the field of four-dimensional (4D) printing due to their large, reversible, and anisotropic shape-morphing capabilities. By utilizing direct ink writing, intricate LCE structures with programmable shape morphing can be achieved. However, the maintenance of the actuated state for LCEs requires continuous and substantial external stimuli, presenting challenges for practical applications, particularly under ambient conditions. This study reports a straightforward and effective physical approach to lock the actuated state of LCEs through rapid cooling while preserving their reversible performance. Rapid cooling significantly reduces the mobility of the lightly cross-linked network in LCEs, resulting in a notably slow recovery of mesogen alignment. As a result, the locked LCE structures retain their actuated state even at room temperature. Moreover, we demonstrate the ability to achieve tunable shapes between the original and actuated states by modulating the cooling rate, i.e., varying the temperature and type of cooling medium. The proposed method opens up new possibilities to achieve stable and tunable shape locking of soft devices for engineering applications.

Learning to Compose and Reason with Language Tree Structures for Visual Grounding.

Grounding natural language in images, such as localizing "the black dog on the left of the tree", is one of the core problems in artificial intelligence, as it needs to comprehend the fine-grained language compositions. However, existing solutions merely rely on the association between the holistic language features and visual features, while neglect the nature of composite reasoning implied in the language. In this paper, we propose a natural language grounding model that can automatically compose a binary tree structure for parsing the language and then perform visual reasoning along the tree in a bottom-up fashion. We call our model RvG-Tree: Recursive Grounding Tree, which is inspired by the intuition that any language expression can be recursively decomposed into two constituent parts, and the grounding confidence score can be recursively accumulated by calculating their grounding scores returned by the two sub-trees.RvG-Tree can be trained end-to-end by using the Straight-Through Gumbel-Softmax estimator that allows the gradients from the continuous score functions passing through the discrete tree construction. Experiments on several benchmarks show that our model achieves the state-of-the-art performance with more explainable reasoning.

Causal Incremental Graph Convolution for Recommender System Retraining.

The real-world recommender system needs to be regularly retrained to keep with the new data. In this work, we consider how to efficiently retrain graph convolution network (GCN)-based recommender models that are state-of-the-art techniques for the collaborative recommendation. To pursue high efficiency, we set the target as using only new data for model updating, meanwhile not sacrificing the recommendation accuracy compared with full model retraining. This is nontrivial to achieve since the interaction data participates in both the graph structure for model construction and the loss function for model learning, whereas the old graph structure is not allowed to use in model updating. Toward the goal, we propose a causal incremental graph convolution (IGC) approach, which consists of two new operators named IGC and colliding effect distillation (CED) to estimate the output of full graph convolution. In particular, we devise simple and effective modules for IGC to ingeniously combine the old representations and the incremental graph and effectively fuse the long- and short-term preference signals. CED aims to avoid the out-of-date issue of inactive nodes that are not in the incremental graph, which connects the new data with inactive nodes through causal inference. In particular, CED estimates the causal effect of new data on the representation of inactive nodes through the control of their collider. Extensive experiments on three real-world datasets demonstrate both accuracy gains and significant speed-ups over the existing retraining mechanism.

Spatio-Temporal Collaborative Module for Efficient Action Recognition.

Efficient action recognition aims to classify a video clip into a specific action category with a low computational cost. It is challenging since the integrated spatial-temporal calculation (e. g., 3D convolution) introduces intensive operations and increases complexity. This paper explores the feasibility of the integration of channel splitting and filter decoupling for efficient architecture design and feature refinement by proposing a novel spatio-temporal collaborative (STC) module. STC splits the video feature channels into two groups and separately learns spatio-temporal representations in parallel with decoupled convolutional operators. Particularly, STC consists of two computation-efficient blocks, i.e., [Formula: see text] and [Formula: see text], where they extract either spatial ( S· ) or temporal ( T· ) features and further refine their features with either temporal ( ·T ) or spatial ( ·S ) contexts globally. The spatial/temporal context refers to information dynamics aggregated from temporal/spatial axis. To thoroughly examine our method's performance in video action recognition tasks, we conduct extensive experiments using five video benchmark datasets requiring temporal reasoning. Experimental results show that the proposed STC networks achieve a competitive trade-off between model efficiency and effectiveness.

A KG-Enhanced Multi-Graph Neural Network for Attentive Herb Recommendation.

Traditional Chinese Medicine (TCM) has the longest clinical history in Asia and contributes a lot to health maintenance worldwide. An essential step during the TCM diagnostic process is syndrome induction, which comprehensively analyzes the symptoms and generates an overall summary of the symptoms. Given a set of symptoms, the existing herb recommenders aim to generate the corresponding herbs as a treatment by inducing the implicit syndrome representations based on TCM prescriptions. As different symptoms have various importance during the comprehensive consideration, we argue that treating the co-occurred symptoms equally to do syndrome induction in the previous studies will lead to the coarse-grained syndrome representation. In this paper, we bring the attention mechanism to model the syndrome induction process. Given a set of symptoms, we leverage an attention network to discriminate the symptom importance and adaptively fuse the symptom embeddings. Besides, we introduce a TCM knowledge graph to enrich the input corpus and improve the quality of representation learning. Further, we build a KG-enhanced Multi-Graph Neural Network architecture, which performs the attentive propagation to combine node feature and graph structural information. Extensive experimental results on two TCM data sets show that our proposed model has the outstanding performance over the state-of-the-arts.

Centrifugal multimaterial 3D printing of multifunctional heterogeneous objects.

There are growing demands for multimaterial three-dimensional (3D) printing to manufacture 3D object where voxels with different properties and functions are precisely arranged. Digital light processing (DLP) is a high-resolution fast-speed 3D printing technology suitable for various materials. However, multimaterial 3D printing is challenging for DLP as the current multimaterial switching methods require direct contact onto the printed part to remove residual resin. Here we report a DLP-based centrifugal multimaterial (CM) 3D printing method to generate large-volume heterogeneous 3D objects where composition, property and function are programmable at voxel scale. Centrifugal force enables non-contact, high-efficiency multimaterial switching, so that the CM 3D printer can print heterogenous 3D structures in large area (up to 180 mm × 130 mm) made of materials ranging from hydrogels to functional polymers, and even ceramics. Our CM 3D printing method exhibits excellent capability of fabricating digital materials, soft robots, and ceramic devices.

Raman spectroscopy based on a single-crystal sapphire fiber.

We demonstrate the advantages of using a single-crystal sapphire fiber to perform Raman spectroscopy over using a standard glass fiber. The narrow Raman peaks and the low background signal of a single-crystal sapphire fiber allow for the detection of weaker Raman signals or Raman signals much closer to the incident laser wavelength than with a standard glass fiber, which has a strong broadband Raman spectrum of its own. The Raman spectra of calcite, single-walled carbon nanotubes, and aqueous sodium carbonate solution were collected using a 50 cm long, 200 µm diameter sapphire fiber, and then they were compared with the results using a 50 cm long, 100 µm diameter glass fiber. The Raman spectrum of each tested material becomes indistinguishable due to the background Raman signal of the glass fiber but can easily be seen using the sapphire fiber, which has only a few narrow Raman peaks of its own.

Generating Face Images With Attributes for Free.

With superhuman-level performance of face recognition, we are more concerned about the recognition of fine-grained attributes, such as emotion, age, and gender. However, given that the label space is extremely large and follows a long-tail distribution, it is quite expensive to collect sufficient samples for fine-grained attributes. This results in imbalanced training samples and inferior attribute recognition models. To this end, we propose the use of arbitrary attribute combinations, without human effort, to synthesize face images. In particular, to bridge the semantic gap between high-level attribute label space and low-level face image, we propose a novel neural-network-based approach that maps the target attribute labels to an embedding vector, which can be fed into a pretrained image decoder to synthesize a new face image. Furthermore, to regularize the attribute for image synthesis, we propose to use a perceptual loss to make the new image explicitly faithful to target attributes. Experimental results show that our approach can generate photorealistic face images from attribute labels, and more importantly, by serving as augmented training samples, these images can significantly boost the performance of attribute recognition model. The code is open-sourced at this link.