RESUMO
High-speed three-dimensional (3D) intravital imaging in animals is useful for studying transient subcellular interactions and functions in health and disease. Light-field microscopy (LFM) provides a computational solution for snapshot 3D imaging with low phototoxicity but is restricted by low resolution and reconstruction artifacts induced by optical aberrations, motion and noise. Here, we propose virtual-scanning LFM (VsLFM), a physics-based deep learning framework to increase the resolution of LFM up to the diffraction limit within a snapshot. By constructing a 40 GB high-resolution scanning LFM dataset across different species, we exploit physical priors between phase-correlated angular views to address the frequency aliasing problem. This enables us to bypass hardware scanning and associated motion artifacts. Here, we show that VsLFM achieves ultrafast 3D imaging of diverse processes such as the beating heart in embryonic zebrafish, voltage activity in Drosophila brains and neutrophil migration in the mouse liver at up to 500 volumes per second.
Assuntos
Microscopia , Peixe-Zebra , Animais , Camundongos , Imageamento Tridimensional/métodosRESUMO
The combination of surface coils and metamaterials remarkably enhance magnetic resonance imaging (MRI) performance for significant local staging flexibility. However, due to the coupling in between, impeded signal-to-noise ratio (SNR) and low-contrast resolution, further hamper the future growth in clinical MRI. In this paper, we propose a high-Q metasurface decoupling isolator fueled by topological LC loops for 1.5T surface coil MRI system, increasing the magnetic field up to fivefold at 63.8 MHz. We have employed a polarization conversion mechanism to effectively eliminate the coupling between the MRI metamaterial and the radio frequency (RF) surface transmitter-receiver coils. Furthermore, a high-Q metasurface isolator was achieved by taking advantage of bound states in the continuum (BIC) for extremely high-resolution MRI and spectroscopy. An equivalent physical model of the miniaturized metasurface design was put forward through LC circuit analysis. This study opens up a promising route for the easy-to-use and portable surface coil MRI scanners.
RESUMO
PURPOSE: To introduce a dual-domain reconstruction network with V-Net and K-Net for accurate MR image reconstruction from undersampled k-space data. METHODS: Most state-of-the-art reconstruction methods apply U-Net or cascaded U-Nets in the image domain and/or k-space domain. Nevertheless, these methods have the following problems: (1) directly applying U-Net in the k-space domain is not optimal for extracting features; (2) classical image-domain-oriented U-Net is heavyweighted and hence inefficient when cascaded many times to yield good reconstruction accuracy; (3) classical image-domain-oriented U-Net does not make full use of information of the encoder network for extracting features in the decoder network; and (4) existing methods are ineffective in simultaneously extracting and fusing features in the image domain and its dual k-space domain. To tackle these problems, we present 3 different methods: (1) V-Net, an image-domain encoder-decoder subnetwork that is more lightweight for cascading and effective in fully utilizing features in the encoder for decoding; (2) K-Net, a k-space domain subnetwork that is more suitable for extracting hierarchical features in the k-space domain, and (3) KV-Net, a dual-domain reconstruction network in which V-Nets and K-Nets are effectively combined and cascaded. RESULTS: Extensive experimental results on the fastMRI dataset demonstrate that the proposed KV-Net can reconstruct high-quality images and outperform state-of-the-art approaches with fewer parameters. CONCLUSIONS: To reconstruct images effectively and efficiently from incomplete k-space data, we have presented a dual-domain KV-Net to combine K-Nets and V-Nets. The KV-Net achieves better results with 9% and 5% parameters than comparable methods (XPD-Net and i-RIM).
Assuntos
Processamento de Imagem Assistida por Computador , Imageamento por Ressonância Magnética , Processamento de Imagem Assistida por Computador/métodos , Imageamento por Ressonância Magnética/métodosRESUMO
Biologically plausible learning with neuronal dendrites is a promising perspective to improve the spike-driven learning capability by introducing dendritic processing as an additional hyperparameter. Neuromorphic computing is an effective and essential solution towards spike-based machine intelligence and neural learning systems. However, on-line learning capability for neuromorphic models is still an open challenge. In this study a novel neuromorphic architecture with dendritic on-line learning (NADOL) is presented, which is a novel efficient methodology for brain-inspired intelligence on embedded hardware. With the feature of distributed processing using spiking neural network, NADOL can cut down the power consumption and enhance the learning efficiency and convergence speed. A detailed analysis for NADOL is presented, which demonstrates the effects of different conditions on learning capabilities, including neuron number in hidden layer, dendritic segregation parameters, feedback connection, and connection sparseness with various levels of amplification. Piecewise linear approximation approach is used to cut down the computational resource cost. The experimental results demonstrate a remarkable learning capability that surpasses other solutions, with NADOL exhibiting superior performance over the GPU platform in dendritic learning. This study's applicability extends across diverse domains, including the Internet of Things, robotic control, and brain-machine interfaces. Moreover, it signifies a pivotal step in bridging the gap between artificial intelligence and neuroscience through the introduction of an innovative neuromorphic paradigm.
Assuntos
Inteligência Artificial , Educação a Distância , Redes Neurais de Computação , Computadores , DendritosRESUMO
Few-Shot Class-Incremental Learning (FSCIL) aims at incrementally learning new knowledge from limited training examples without forgetting previous knowledge. However, we observe that existing methods face a challenge known as supervision collapse, where the model disproportionately emphasizes class-specific features of base classes at the detriment of novel class representations, leading to restricted cognitive capabilities. To alleviate this issue, we propose a new framework, Model aTtention Expansion for Few-Shot Class-Incremental Learning (MTE-FSCIL), aimed at expanding the model attention fields to improve transferability without compromising the discriminative capability for base classes. Specifically, the framework adopts a dual-stage training strategy, comprising pre-training and meta-training stages. In the pre-training stage, we present a new regularization technique, named the Reserver (RS) loss, to expand the global perception and reduce over-reliance on class-specific features by amplifying feature map activations. During the meta-training stage, we introduce the Repeller (RP) loss, a novel pair-based loss that promotes variation in representations and improves the model's recognition of sample uniqueness by scattering intra-class samples within the embedding space. Furthermore, we propose a Transformational Adaptation (TA) strategy to enable continuous incorporation of new knowledge from downstream tasks, thus facilitating cross-task knowledge transfer. Extensive experimental results on mini-ImageNet, CIFAR100, and CUB200 datasets demonstrate that our proposed framework consistently outperforms the state-of-the-art methods.
RESUMO
Continual learning (CL) aims at studying how to learn new knowledge continuously from data streams without catastrophically forgetting the previous knowledge. One of the key problems is catastrophic forgetting, that is, the performance of the model on previous tasks declines significantly after learning the subsequent task. Several studies addressed it by replaying samples stored in the buffer when training new tasks. However, the data imbalance between old and new task samples results in two serious problems: information suppression and weak feature discriminability. The former refers to the information in the sufficient new task samples suppressing that in the old task samples, which is harmful to maintaining the knowledge since the biased output worsens the consistency of the same sample's output at different moments. The latter refers to the feature representation being biased to the new task, which lacks discrimination to distinguish both old and new tasks. To this end, we build an imbalance mitigation for CL (IMCL) framework that incorporates a decoupled knowledge distillation (DKD) approach and a dual enhanced contrastive learning (DECL) approach to tackle both problems. Specifically, the DKD approach alleviates the suppression of the new task on the old tasks by decoupling the model output probability during the replay stage, which better maintains the knowledge of old tasks. The DECL approach enhances both low-and high-level features and fuses the enhanced features to construct contrastive loss to effectively distinguish different tasks. Extensive experiments on three popular datasets show that our method achieves promising performance under task incremental learning (Task-IL), class incremental learning (Class-IL), and domain incremental learning (Domain-IL) settings.
RESUMO
As a fundamental and challenging task in bridging language and vision domains, Image-Text Retrieval (ITR) aims at searching for the target instances that are semantically relevant to the given query from the other modality, and its key challenge is to measure the semantic similarity across different modalities. Although significant progress has been achieved, existing approaches typically suffer from two major limitations: (1) It hurts the accuracy of the representation by directly exploiting the bottom-up attention based region-level features where each region is equally treated. (2) It limits the scale of negative sample pairs by employing the mini-batch based end-to-end training mechanism. To address these limitations, we propose a Unified Semantic Enhancement Momentum Contrastive Learning (USER) method for ITR. Specifically, we delicately design two simple but effective Global representation based Semantic Enhancement (GSE) modules. One learns the global representation via the self-attention algorithm, noted as Self-Guided Enhancement (SGE) module. The other module benefits from the pre-trained CLIP module, which provides a novel scheme to exploit and transfer the knowledge from an off-the-shelf model, noted as CLIP-Guided Enhancement (CGE) module. Moreover, we incorporate the training mechanism of MoCo into ITR, in which two dynamic queues are employed to enrich and enlarge the scale of negative sample pairs. Meanwhile, a Unified Training Objective (UTO) is developed to learn from mini-batch based and dynamic queue based samples. Extensive experiments on the benchmark MSCOCO and Flickr30K datasets demonstrate the superiority of both retrieval accuracy and inference efficiency. For instance, compared with the existing best method NAAF, the metric R@1 of our USER on the MSCOCO 5K Testing set is improved by 5% and 2.4% on caption retrieval and image retrieval without any external knowledge or pre-trained model while enjoying over 60 times faster inference speed. Our source code will be released at https://github.com/zhangy0822/USER.
RESUMO
Deep cascaded networks have been extensively studied and applied to accelerate Magnetic Resonance Imaging (MRI) and have shown promising results. Most existing works employ a large cascading number for the sake of superior performances. However, due to the lack of proper guidance, the reconstruction performance can easily reach a plateau and even face degradation if simply increasing the cascading number. In this paper, we aim to boost the reconstruction performance from a novel perspective by proposing a parallel architecture called DIRECTION that fully exploits the guiding value of the reconstruction residual of each subnetwork. Specifically, we introduce a novel Reconstruction Residual-Based Feature Modulation Mechanism (RRFMM) which utilizes the reconstruction residual of the previous subnetwork to guide the next subnetwork at the feature level. To achieve this, a Residual Attention Modulation Block (RAMB) is proposed to generate attention maps using multi-scale residual features to modulate the image features of the corresponding scales. Equipped with this strategy, each subnetwork within the cascaded network possesses its unique optimization objective and emphasis rather than blindly updating its parameters. To further boost the performance, we introduce the Cross-Stage Feature Reuse Connection (CSFRC) and the Reconstruction Dense Connection (RDC), which can reduce information loss and enhance representative ability. We conduct sufficient experiments and evaluate our method on the fastMRI knee dataset using multiple subsampling masks. Comprehensive experimental results show that our method can markedly boost the performance of cascaded networks and significantly outperforms other compared state-of-the-art methods quantitatively and qualitatively.
Assuntos
Algoritmos , Processamento de Imagem Assistida por Computador , Imageamento por Ressonância Magnética , Imageamento por Ressonância Magnética/métodos , Humanos , Processamento de Imagem Assistida por Computador/métodos , Encéfalo/diagnóstico por imagem , Redes Neurais de Computação , Aprendizado ProfundoRESUMO
Accurate acquisition of physiological and physical information from human tissue is essential for health monitoring, disease prevention and treatment. The existing antennas with traditional rigid or flexible substrates are susceptible to motion artifacts in wearable applications due to the miniaturization limitation and lack of proper adhesion and conformal interfaces with the skin. Recent advances in wearable radio frequency (RF) bioelectronics directly drawn on the skin are a promising solution for future skin-interfaced devices. Herein, we present a first-of-its kind epidermal antenna architecture with skin as the antenna substrate, which is ultra-low profile, ultra-conformal, ultra-compact, and simple fabrication without specialized equipment. The radiation unit and ground of antenna are drawn directly on the skin with the strong adhesion and ultra conformality. Therefore, this RF device is highly adaptable to motion. As a proof-of- feasibility, epidermal antenna can be freely drawn on demand at different locations on the skin for the development of temperature sensor, skin hydration sensor, strain sensor, glucose sensor and other devices. An epidermal antenna-based temperature sensor can offer accurate and real-time monitoring of human body temperature changes in the ultra-wideband (UWB) range. The results during the monitoring of hydration level with and without stretching show that the epidermal antenna drawn on the skin is motion artifact-free. We also designed an epidermal antenna array employing a horseshoe-shaped configuration for the precise identification of various gestures. In addition, the non-invasive blood glucose level (BGL) monitoring results during the in-vivo experiments report high correlation between the epidermal antenna responses and BGLs, without any time hysteresis. After the prediction of BGL by BP network, all the predicted BGL values are fallen 100% into the clinically acceptable zones. Together, these results show that epidermal antenna offers a promising new approach for biosensing platform.
Assuntos
Artefatos , Técnicas Biossensoriais , Humanos , Sistemas Automatizados de Assistência Junto ao Leito , Movimento (Física) , EpidermeRESUMO
As a promising field, Multi-Query Image Retrieval (MQIR) aims at searching for the semantically relevant image given multiple region-specific text queries. Existing works mainly focus on a single-level similarity between image regions and text queries, which neglect the hierarchical guidance of multi-level similarities and result in incomplete alignments. Besides, the high-level semantic correlations that intrinsically connect different region-query pairs are rarely considered. To address above limitations, we propose a novel Hierarchical Matching and Reasoning Network (HMRN) for MQIR. It disentangles MQIR into three hierarchical semantic representations, which is responsible to capture fine-grained local details, contextual global scopes, and high-level inherent correlations. HMRN consists of two modules: Scalar-based Matching (SM) module and Vector-based Reasoning (VR) module. Specifically, the SM module characterizes the multi-level alignment similarity, which consists of a fine-grained local-level similarity and a context-aware global-level similarity. Afterwards, the VR module is developed to excavate the potential semantic correlations among multiple region-query pairs, which further explores the high-level reasoning similarity. Finally, these three-level similarities are aggregated into a joint similarity space to form the ultimate similarity. Extensive experiments on the benchmark dataset demonstrate that our HMRN substantially surpasses the current state-of-the-art methods. For instance, compared with the existing best method Drill-down, the metric R@1 in the last round is improved by 23.4%. Our source codes will be released at https://github.com/LZH-053/HMRN.
Assuntos
Benchmarking , Resolução de Problemas , Semântica , SoftwareRESUMO
Deep learning methods for fast MRI have shown promise in reconstructing high-quality images from undersampled multi-coil k-space data, leading to reduced scan duration. However, existing methods encounter challenges related to limited receptive fields in dual-domain (k-space and image domains) reconstruction networks, rigid data consistency operations, and suboptimal refinement structures, which collectively restrict overall reconstruction performance. This study introduces a comprehensive framework that addresses these challenges and enhances MR image reconstruction quality. Firstly, we propose Faster Inverse Fourier Convolution (FasterIFC), a frequency domain convolutional operator that significantly expands the receptive field of k-space domain reconstruction networks. Expanding the information extraction range to the entire frequency spectrum according to the spectral convolution theorem in Fourier theory enables the network to easily utilize richer redundant long-range information from adjacent, symmetrical, and diagonal locations of multi-coil k-space data. Secondly, we introduce a novel softer Data Consistency (softerDC) layer, which achieves an enhanced balance between data consistency and smoothness. This layer facilitates the implementation of diverse data consistency strategies across distinct frequency positions, addressing the inflexibility observed in current methods. Finally, we present the Dual-Domain Faster Fourier Convolution Based Network (D2F2), which features a centrosymmetric dual-domain parallel structure based on FasterIFC. This architecture optimally leverages dual-domain data characteristics while substantially expanding the receptive field in both domains. Coupled with the softerDC layer, D2F2 demonstrates superior performance on the NYU fastMRI dataset at multiple acceleration factors, surpassing state-of-the-art methods in both quantitative and qualitative evaluations.
Assuntos
Análise de Fourier , Processamento de Imagem Assistida por Computador , Imageamento por Ressonância Magnética , Imageamento por Ressonância Magnética/métodos , Humanos , Processamento de Imagem Assistida por Computador/métodos , Aprendizado Profundo , Encéfalo/diagnóstico por imagem , AlgoritmosRESUMO
High quality image reconstruction from undersampled k -space data is key to accelerating MR scanning. Current deep learning methods are limited by the small receptive fields in reconstruction networks, which restrict the exploitation of long-range information, and impede the mitigation of full-image artifacts, particularly in 3D reconstruction tasks. Additionally, the substantial computational demands of 3D reconstruction considerably hinder advancements in related fields. To tackle these challenges, we propose the following: 1) A novel convolution operator named Faster Fourier Convolution (FasterFC), aims at providing an adaptable broad receptive field for spatial domain reconstruction networks with fast computational speed. 2) A split-slice strategy that substantially reduces the computational load of 3D reconstruction, enabling high-resolution, multi-coil, 3D MR image reconstruction while fully utilizing inter-layer and intra-layer information. 3) A single-to-group algorithm that efficiently utilizes scan-specific and data-driven priors to enhance k -space interpolation effects. 4) A multi-stage, multi-coil, 3D fast MRI method, called the faster Fourier convolution based single-to-group network (FAS-Net), comprising a single-to-group k -space interpolation algorithm and a FasterFC-based image domain reconstruction module, significantly minimizes the computational demands of 3D reconstruction through split-slice strategy. Experimental evaluations conducted on the NYU fastMRI and Stanford MRI Data datasets reveal that the FasterFC significantly enhances the quality of both 2D and 3D reconstruction results. Moreover, FAS-Net, characterized as a method that can achieve high-resolution (320, 320, 256), multi-coil, (8 coils), 3D fast MRI, exhibits superior reconstruction performance compared to other state-of-the-art 2D and 3D methods.
RESUMO
Two new piperidine alkaloids, streptazones E (1) and F (2), were isolated from Streptomyces sp. NEAU-Z4. Their structures were elucidated on the basis of MS and NMR analysis.
Assuntos
Alcaloides/isolamento & purificação , Antineoplásicos/isolamento & purificação , Piperidinas/isolamento & purificação , Streptomyces/química , Alcaloides/química , Alcaloides/farmacologia , Antineoplásicos/química , Antineoplásicos/farmacologia , Ensaios de Seleção de Medicamentos Antitumorais , Células HCT116 , Células Hep G2 , Humanos , Estrutura Molecular , Ressonância Magnética Nuclear Biomolecular , Piperidinas/química , Piperidinas/farmacologiaRESUMO
Reconstructing zero-filled MR images (ZF) from partial k-space by convolutional neural networks (CNN) is an important way to accelerate MRI. However, due to the lack of attention to different components in ZF, it is challenging to learn the mapping from ZF to targets effectively. To ameliorate this issue, we propose a Detail and Structure Mutually Enhancing Network (DSMENet), which benefits from the complementary of the Structure Reconstruction UNet (SRUN) and the Detail Feature Refinement Module (DFRM). The SRUN learns structure-dominated information at multiple scales. And the DRFM enriches detail-dominated information from coarse to fine. The bidirectional alternate connections then exchange information between them. Moreover, the Detail Representation Construction Module (DRCM) extracts valuable initial detail representation for DFRM. And the Detail Guided Fusion Module (DGFM) facilitates the deep fusion of these complementary information. With the help of them, various components in ZF can be applied with discriminative attentions and mutually enhanced. In addition, the performance can be further improved by the Deep Enhanced Restoration (DER), a strategy based on recursion and constrain. Extensive experiments on fastMRI and CC-359 datasets demonstrate that DSMENet has robustness in terms of various body parts, under-sampling rates, and masks. Furthermore, DSMENet can achieve promising performance on qualitative and quantitative results, especially the competitive NMSE of 0.0268, PSNE of 33.7, and SSIM of 0.7808 on fastMRI 4 × single-coil knee leaderboard.
Assuntos
Processamento de Imagem Assistida por Computador , Redes Neurais de Computação , Humanos , Processamento de Imagem Assistida por Computador/métodos , Joelho , Articulação do Joelho , Imageamento por Ressonância Magnética/métodosRESUMO
Domain Adaptive Few-Shot Learning (DA-FSL) aims at accomplishing few-shot classification tasks on a novel domain with the aid of a large number of source-style samples and several target-style samples. It is essential for DA-FSL to transfer task knowledge from the source domain to the target domain and overcome the asymmetry amount of labeled data in both domains. To this end, we propose Dual Distillation Discriminator Networks (D3Net) from the perspective of the lack of labeled target domain style samples in DA-FSL. Specifically, we employ the idea of distillation discrimination to avoid the over-fitting caused by the unequal number of samples in the target and source domains, which trains the student discriminator by the soft labels from the teacher discriminator. Meanwhile, we design the task propagation stage and the mixed domain stage respectively from the level of feature space and instances to generate more target-style samples, which apply the task distributions and the sample diversity of the source domain to enhance the target domain. Our D3Net realizes the distribution alignment between the source domain and the target domain and constraints the FSL task distribution by prototype distributions on the mixed domain. Extensive experiments on three DA-FSL benchmark datasets, i.e., mini-ImageNet, tiered-ImageNet, and DomainNet, demonstrate that our D3Net achieves competitive performance.
Assuntos
Destilação , Aprendizagem , Humanos , Estudantes , Benchmarking , ConhecimentoRESUMO
Accurate object detection requires correct classification and high-quality localization. Currently, most of the single shot detectors (SSDs) conduct simultaneous classification and regression using a fully convolutional network. Despite high efficiency, this structure has some inappropriate designs for accurate object detection. The first one is the mismatch of bounding box classification, where the classification results of the default bounding boxes are improperly treated as the results of the regressed bounding boxes during the inference. The second one is that only one-time regression is not good enough for high-quality object localization. To solve the problem of classification mismatch, we propose a novel reg-offset-cls (ROC) module including three hierarchical steps: the regression of the default bounding box, the prediction of new feature sampling locations, and the classification of the regressed bounding box with more accurate features. For high-quality localization, we stack two ROC modules together. The input of the second ROC module is the output of the first ROC module. In addition, we inject a feature enhanced (FE) module between two stacked ROC modules to extract more contextual information. The experiments on three different datasets (i.e., MS COCO, PASCAL VOC, and UAVDT) are performed to demonstrate the effectiveness and superiority of our method. Without any bells or whistles, our proposed method outperforms state-of-the-art one-stage methods at a real-time speed. The source code is available at https://github.com/JialeCao001/HSD.
RESUMO
Recently stereo image deraining has attracted lots of attention due to its superiority of abundant information from cross views. Exploring interaction information across stereo views is the key to improving the performance of stereo image deraining. In this paper, we design a general coarse-to-fine deraining framework for stereo rain streak and raindrop removal, called CDINet, comprising a stereo rain removal subnet and a stereo detail recovery subnet to restore images progressively. Two types of interaction modules are devised to explore interaction information for rain removal and detail recovery, respectively. Specifically, a global context interaction module is proposed to learn long-range dependencies of stereo images and remove rain by utilizing stereo structural information. A local detail interaction module is designed to model local contextual correlation, which aims at restoring the detail information by using neighborhood information from cross views. Extensive experiments are conducted on the two datasets including a synthetic rain streak removal dataset (RainKITTI) and a real raindrop removal dataset (Stereo Waterdrop), which demonstrates that our method sets new state-of-the-art deraining performance in terms of both quantitative and qualitative metrics with faster speed.
Assuntos
Benchmarking , Aprendizagem , ChuvaRESUMO
Few-shot Class-Incremental Learning (FSCIL) aims at learning new concepts continually with only a few samples, which is prone to suffer the catastrophic forgetting and overfitting problems. The inaccessibility of old classes and the scarcity of the novel samples make it formidable to realize the trade-off between retaining old knowledge and learning novel concepts. Inspired by that different models memorize different knowledge when learning novel concepts, we propose a Memorizing Complementation Network (MCNet) to ensemble multiple models that complements the different memorized knowledge with each other in novel tasks. Additionally, to update the model with few novel samples, we develop a Prototype Smoothing Hard-mining Triplet (PSHT) loss to push the novel samples away from not only each other in current task but also the old distribution. Extensive experiments on three benchmark datasets, e.g., CIFAR100, miniImageNet and CUB200, have demonstrated the superiority of our proposed method.
RESUMO
Recognizing human-object interaction (HOI) aims at inferring various relationships between actions and objects. Although great progress in HOI has been made, the long-tail problem and combinatorial explosion problem are still practical challenges. To this end, we formulate HOI as a few-shot task to tackle both challenges and design a novel dynamic generation method to address this task. The proposed approach is called semantic-aware dynamic generation networks (SADG-Nets). Specifically, SADG-Net first assigns semantic-aware task representations for different batches of data, which further generates dynamic parameters. It obtains the features that highlight intercategory discriminability and intracategory commonality adaptively. In addition, we also design a dual semantic-aware encoder module (DSAE-Module), that is, verb-aware and noun-aware branches, to yield both action and object prototypes of HOI for each task space, which generalizes to novel combinations by transferring similarities among interactions. Extensive experimental results on two benchmark datasets, that is, humans interacting with common objects (HICO)-FS and trento universal HOI (TUHOI)-FS, illustrate that our SADG-Net achieves superior performance over state-of-the-art approaches, which proves its impressive effectiveness on few-shot HOI recognition.
RESUMO
We propose a fast single-stage method for both image and video instance segmentation, called SipMask, that preserves the instance spatial information by performing multiple sub-region mask predictions. The main module in our method is a light-weight spatial preservation (SP) module that generates a separate set of spatial coefficients for the sub-regions within a bounding-box, enabling a better delineation of spatially adjacent instances. To better correlate mask prediction with object detection, we further propose a mask alignment weighting loss and a feature alignment scheme. In addition, we identify two issues that impede the performance of single-stage instance segmentation and introduce two modules, including a sample selection scheme and an instance refinement module, to address these two issues. Experiments are performed on both image instance segmentation dataset MS COCO and video instance segmentation dataset YouTube-VIS. On MS COCO test-dev set, our method achieves a state-of-the-art performance. In terms of real-time capabilities, it outperforms YOLACT by a gain of 3.0% (mask AP) under the similar settings, while operating at a comparable speed. On YouTube-VIS validation set, our method also achieves promising results. The source code is available at https://github.com/JialeCao001/SipMask.