Learning a robust foundation model against clean-label data poisoning attacks at downstream tasks.

In the transfer learning paradigm, models that are pre-trained on large datasets are used as the foundation models for various downstream tasks. However, this paradigm exposes downstream practitioners to data poisoning threats, as attackers can inject malicious samples into the re-training datasets to manipulate the behavior of models in downstream tasks. In this work, we propose a defense strategy that significantly reduces the success rate of various data poisoning attacks in downstream tasks. Our defense aims to pre-train a robust foundation model by reducing adversarial feature distance and increasing inter-class feature distance. Experiments demonstrate the excellent defense performance of the proposed strategy towards state-of-the-art clean-label poisoning attacks in the transfer learning scenario.

Adversarial Domain Adaptation With Prototype-Based Normalized Output Conditioner.

Domain adversarial training has become a prevailing and effective paradigm for unsupervised domain adaptation (UDA). To successfully align the multi-modal data structures across domains, the following works exploit discriminative information in the adversarial training process, e.g., using multiple class-wise discriminators and involving conditional information in the input or output of the domain discriminator. However, these methods either require non-trivial model designs or are inefficient for UDA tasks. In this work, we attempt to address this dilemma by devising simple and compact conditional domain adversarial training methods. We first revisit the simple concatenation conditioning strategy where features are concatenated with output predictions as the input of the discriminator. We find the concatenation strategy suffers from the weak conditioning strength. We further demonstrate that enlarging the norm of concatenated predictions can effectively energize the conditional domain alignment. Thus we improve concatenation conditioning by normalizing the output predictions to have the same norm of features, and term the derived method as Normalized OutpUt coNditioner (NOUN). However, conditioning on raw output predictions for domain alignment, NOUN suffers from inaccurate predictions of the target domain. To this end, we propose to condition the cross-domain feature alignment in the prototype space rather than in the output space. Combining the novel prototype-based conditioning with NOUN, we term the enhanced method as PROtotype-based Normalized OutpUt coNditioner (PRONOUN). Experiments on both object recognition and semantic segmentation show that NOUN can effectively align the multi-modal structures across domains and even outperform state-of-the-art domain adversarial training methods. Together with prototype-based conditioning, PRONOUN further improves the adaptation performance over NOUN on multiple object recognition benchmarks for UDA. Code is available at https://github.com/tim-learn/NOUN.

Deep Learning-Guided Fiberoptic Raman Spectroscopy Enables Real-Time In Vivo Diagnosis and Assessment of Nasopharyngeal Carcinoma and Post-treatment Efficacy during Endoscopy.

In this work, we develop a deep learning-guided fiberoptic Raman diagnostic platform to assess its ability of real-time in vivo nasopharyngeal carcinoma (NPC) diagnosis and post-treatment follow-up of NPC patients. The robust Raman diagnostic platform is established using innovative multi-layer Raman-specified convolutional neural networks (RS-CNN) together with simultaneous fingerprint and high-wavenumber spectra acquired within sub-seconds using a fiberoptic Raman endoscopy system. We have acquired a total of 15,354 FP/HW in vivo Raman spectra (control: 1761; NPC: 4147; and post-treatment (PT): 9446) from 888 tissue sites of 418 subjects (healthy control: 85; NPC: 82; and PT: 251) during endoscopic examination. The optimized RS-CNN model provides an overall diagnostic accuracy of 82.09% (sensitivity of 92.18% and specificity of 73.99%) for identifying NPC from control and post-treatment patients, which is superior to the best diagnosis performance (accuracy of 73.57%; sensitivity of 89.74%; and specificity of 58.10%) using partial-least-squares linear-discriminate-analysis, proving the robustness and high spectral information sensitiveness of the RS-CNN model developed. We further investigate the saliency map of the best RS-CNN models using the correctly predicted Raman spectra. The specific Raman signatures that are related to the cancer-associated biomolecular variations (e.g., collagens, lipids, and nucleic acids) are uncovered in the map, validating the diagnostic capability of RS-CNN models to correlate with biomolecular signatures. Deep learning-based Raman spectroscopy is a powerful diagnostic tool for rapid screening and surveillance of NPC patients and can also be deployed for longitudinal follow-up monitoring of post-treatment NPC patients to detect early cancer recurrences in the head and neck.