Prompt-and-Transfer: Dynamic Class-aware Enhancement for Few-shot Segmentation.

For more efficient generalization to unseen domains (classes), most Few-shot Segmentation (FSS) would directly exploit pre-trained encoders and only fine-tune the decoder, especially in the current era of large models. However, such fixed feature encoders tend to be class-agnostic, inevitably activating objects that are irrelevant to the target class. In contrast, humans can effortlessly focus on specific objects in the line of sight. This paper mimics the visual perception pattern of human beings and proposes a novel and powerful prompt-driven scheme, called "Prompt and Transfer" (PAT), which constructs a dynamic class-aware prompting paradigm to tune the encoder for focusing on the interested object (target class) in the current task. Three key points are elaborated to enhance the prompting: 1) Cross-modal linguistic information is introduced to initialize prompts for each task. 2) Semantic Prompt Transfer (SPT) that precisely transfers the class-specific semantics within the images to prompts. 3) Part Mask Generator (PMG) that works in conjunction with SPT to adaptively generate different but complementary part prompts for different individuals. Surprisingly, PAT achieves competitive performance on 4 different tasks including standard FSS, Cross-domain FSS (e.g., CV, medical, and remote sensing domains), Weak-label FSS, and Zero-shot Segmentation, setting new state-of-the-arts on 11 benchmarks.

Remote Sensing Change Detection With Bitemporal and Differential Feature Interactive Perception.

Recently, the transformer has achieved notable success in remote sensing (RS) change detection (CD). Its outstanding long-distance modeling ability can effectively recognize the change of interest (CoI). However, in order to obtain the precise pixel-level change regions, many methods directly integrate the stacked transformer blocks into the UNet-style structure, which causes the high computation costs. Besides, the existing methods generally consider bitemporal or differential features separately, which makes the utilization of ground semantic information still insufficient. In this paper, we propose the multiscale dual-space interactive perception network (MDIPNet) to fill these two gaps. On the one hand, we simplify the stacked multi-head transformer blocks into the single-layer single-head attention module and further introduce the lightweight parallel fusion module (LPFM) to perform the efficient information integration. On the other hand, based on the simplified attention mechanism, we propose the cross-space perception module (CSPM) to connect the bitemporal and differential feature spaces, which can help our model suppress the pseudo changes and mine the more abundant semantic consistency of CoI. Extensive experiment results on three challenging datasets and one urban expansion scene indicate that compared with the mainstream CD methods, our MDIPNet obtains the state-of-the-art (SOTA) performance while further controlling the computation costs.

Multi-Modal Remote Sensing Image Segmentation with Intuition-Inspired Hypergraph Modeling.

Multi-modal remote sensing (RS) image segmentation aims to comprehensively utilize multiple RS modalities to assign pixel-level semantics to the studied scenes, which can provide a new perspective for global city understanding. Multi-modal segmentation inevitably encounters the challenge of modeling intra- and inter-modal relationships, i.e., object diversity and modal gaps. However, the previous methods are usually designed for a single RS modality, limited by the noisy collection environment and poor discrimination information. Neuropsychology and neuroanatomy confirm that the human brain performs the guiding perception and integrative cognition of multi-modal semantics through intuitive reasoning. Therefore, establishing a semantic understanding framework inspired by intuition to realize multi-modal RS segmentation becomes the main motivation of this work. Drived by the superiority of hypergraphs in modeling high-order relationships, we propose an intuition-inspired hypergraph network (I2HN) for multi-modal RS segmentation. Specifically, we present a hypergraph parser to imitate guiding perception to learn intra-modal object-wise relationships. It parses the input modality into irregular hypergraphs to mine semantic clues and generate robust mono-modal representations. In addition, we also design a hypergraph matcher to dynamically update the hypergraph structure from the explicit correspondence of visual concepts, similar to integrative cognition, to improve cross-modal compatibility when fusing multi-modal features. Extensive experiments on two multi-modal RS datasets show that the proposed I2HN outperforms the state-of-the-art models, achieving F1/mIoU accuracy 91.4%/82.9% on the ISPRS Vaihingen dataset, and 92.1%/84.2% on the MSAW dataset. The complete algorithm and benchmark results will be available online.

Double Similarity Distillation for Semantic Image Segmentation.

The balance between high accuracy and high speed has always been a challenging task in semantic image segmentation. Compact segmentation networks are more widely used in the case of limited resources, while their performances are constrained. In this paper, motivated by the residual learning and global aggregation, we propose a simple yet general and effective knowledge distillation framework called double similarity distillation (DSD) to improve the classification accuracy of all existing compact networks by capturing the similarity knowledge in pixel and category dimensions, respectively. Specifically, we propose a pixel-wise similarity distillation (PSD) module that utilizes residual attention maps to capture more detailed spatial dependencies across multiple layers. Compared with exiting methods, the PSD module greatly reduces the amount of calculation and is easy to expand. Furthermore, considering the differences in characteristics between semantic segmentation task and other computer vision tasks, we propose a category-wise similarity distillation (CSD) module, which can help the compact segmentation network strengthen the global category correlation by constructing the correlation matrix. Combining these two modules, DSD framework has no extra parameters and only a minimal increase in FLOPs. Extensive experiments on four challenging datasets, including Cityscapes, CamVid, ADE20K, and Pascal VOC 2012, show that DSD outperforms current state-of-the-art methods, proving its effectiveness and generality. The code and models will be publicly available.

Opposite Effects of Flexible Single-Stranded DNA Regions and Rigid Loops in DNAzyme on Colloidal Nanoparticle Stability for "Turn-On" Plasmonic Detection of Lead Ions.

Strains in biomolecules greatly restrict their structural flexibility. The effects of DNA's structural flexibility on nanoparticle stability have remained less explored in the field of plasmonic biosensors. In the present study, we discover the opposite effects of a rigid loop and a flexible single-stranded DNA (ssDNA) region in DNAzyme on the colloidal stability of gold nanoparticles (AuNPs), which afford "turn-on" plasmonic detection of Pb2+. In specific, DNAzyme-functionalized AuNPs undergo spontaneous assembly at high ionic strength upon hybridization to their substrate sequence because of a DNA base stacking interaction. In the presence of Pb2+, however, the DNAzyme grafted on the AuNP cleaves the substrate and forms an ssDNA region in the middle of the rigid loop. The induced structural flexibility of the surface-grafted DNAzyme by the ssDNA region in the middle helps elevate interparticle entropic repulsion, thereby bringing AuNP assemblies back to dispersion. We discover that this process can afford a dramatic increase of the AuNPs' plasmon resonance for determination of Pb2+ concentration. Under optimized conditions, a detection limit of 8.0 nM can be achieved for Pb2+ by this method with high selectivity. Its applicability to Pb2+ analysis in tap water samples has also been demonstrated.