RESUMEN
Exosome-derived microRNAs (miRNAs) are emerging as pivotal players in the pathophysiology of sepsis, representing a new frontier in both the diagnosis and treatment of this complex condition. Sepsis, a severe systemic response to infection, involves intricate immune and nonimmune mechanisms, where exosome-mediated communication can significantly influence disease progression and outcomes. During the progress of sepsis, the miRNA profile of exosomes undergoes notable alterations, is reflecting, and may affect the progression of the disease. This review comprehensively explores the biology of exosome-derived miRNAs, which originate from both immune cells (such as macrophages and dendritic cells) and nonimmune cells (such as endothelial and epithelial cells) and play a dynamic role in modulating pathways that affect the course of sepsis, including those related to inflammation, immune response, cell survival, and apoptosis. Taking into account these dynamic changes, we further discuss the potential of exosome-derived miRNAs as biomarkers for the early detection and prognosis of sepsis and advantages over traditional biomarkers due to their stability and specificity. Furthermore, this review evaluates exosome-based therapeutic miRNA delivery systems in sepsis, which may pave the way for targeted modulation of the septic response and personalized treatment options.
Asunto(s)
Biomarcadores , Exosomas , MicroARNs , Sepsis , Humanos , Exosomas/metabolismo , Sepsis/diagnóstico , Sepsis/terapia , Sepsis/genética , Sepsis/inmunología , MicroARNs/genética , Animales , Pronóstico , Macrófagos/inmunología , Macrófagos/metabolismoRESUMEN
Recent unsupervised domain adaptation (UDA) methods in medical image segmentation commonly utilize Generative Adversarial Networks (GANs) for domain translation. However, the translated images often exhibit a distribution deviation from the ideal due to the inherent instability of GANs, leading to challenges such as visual inconsistency and incorrect style, consequently causing the segmentation model to fall into the fixed wrong pattern. To address this problem, we propose a novel UDA framework known as Dual Domain Distribution Disruption with Semantics Preservation (DDSP). Departing from the idea of generating images conforming to the target domain distribution in GAN-based UDA methods, we make the model domain-agnostic and focus on anatomical structural information by leveraging semantic information as constraints to guide the model to adapt to images with disrupted distributions in both source and target domains. Furthermore, we introduce the inter-channel similarity feature alignment based on the domain-invariant structural prior information, which facilitates the shared pixel-wise classifier to achieve robust performance on target domain features by aligning the source and target domain features across channels. Without any exaggeration, our method significantly outperforms existing state-of-the-art UDA methods on three public datasets (i.e., the heart dataset, the brain dataset, and the prostate dataset). The code is available at https://github.com/MIXAILAB/DDSPSeg.
RESUMEN
West syndrome is an epileptic disease that seriously affects the normal growth and development of infants in early childhood. Based on the methods of brain topological network and graph theory, this article focuses on three clinical states of patients before and after treatment. In addition to discussing bidirectional and unidirectional global networks from the perspective of computational principles, a more in-depth analysis of local intra-network and inter-network characteristics of multi-partitioned networks is also performed. The spatial feature distribution based on feature path length is introduced for the first time. The results show that the bidirectional network has better significant differentiation. The rhythmic feature change trend and spatial characteristic distribution of this network can be used as a measure of the impact on global information processing in the brain after treatment. And localized brain regions variability in features and differences in the ability to interact with information between brain regions have potential as biomarkers for medication assessment in WEST syndrome. The above shows specific conclusions on the interaction relationship and consistency of macro-network and micro-network, which may have a positive effect on patients' treatment and prognosis management.
RESUMEN
Pathology image are essential for accurately interpreting lesion cells in cytopathology screening, but acquiring high-resolution digital slides requires specialized equipment and long scanning times. Though super-resolution (SR) techniques can alleviate this problem, existing deep learning models recover pathology image in a black-box manner, which can lead to untruthful biological details and misdiagnosis. Additionally, current methods allocate the same computational resources to recover each pixel of pathology image, leading to the sub-optimal recovery issue due to the large variation of pathology image. In this paper, we propose the first hierarchical reinforcement learning framework named Spatial-Temporal hierARchical Reinforcement Learning (STAR-RL), mainly for addressing the aforementioned issues in pathology image super-resolution problem. We reformulate the SR problem as a Markov decision process of interpretable operations and adopt the hierarchical recovery mechanism in patch level, to avoid sub-optimal recovery. Specifically, the higher-level spatial manager is proposed to pick out the most corrupted patch for the lower-level patch worker. Moreover, the higher-level temporal manager is advanced to evaluate the selected patch and determine whether the optimization should be stopped earlier, thereby avoiding the over-processed problem. Under the guidance of spatial-temporal managers, the lower-level patch worker processes the selected patch with pixel-wise interpretable actions at each time step. Experimental results on medical images degraded by different kernels show the effectiveness of STAR-RL. Furthermore, STAR-RL validates the promotion in tumor diagnosis with a large margin and shows generalizability under various degradation. The source code is to be released.
RESUMEN
The advancement of artificial intelligence (AI) for organ segmentation and tumor detection is propelled by the growing availability of computed tomography (CT) datasets with detailed, per-voxel annotations. However, these AI models often struggle with flexibility for partially annotated datasets and extensibility for new classes due to limitations in the one-hot encoding, architectural design, and learning scheme. To overcome these limitations, we propose a universal, extensible framework enabling a single model, termed Universal Model, to deal with multiple public datasets and adapt to new classes (e.g., organs/tumors). Firstly, we introduce a novel language-driven parameter generator that leverages language embeddings from large language models, enriching semantic encoding compared with one-hot encoding. Secondly, the conventional output layers are replaced with lightweight, class-specific heads, allowing Universal Model to simultaneously segment 25 organs and six types of tumors and ease the addition of new classes. We train our Universal Model on 3410 CT volumes assembled from 14 publicly available datasets and then test it on 6173 CT volumes from four external datasets. Universal Model achieves first place on six CT tasks in the Medical Segmentation Decathlon (MSD) public leaderboard and leading performance on the Beyond The Cranial Vault (BTCV) dataset. In summary, Universal Model exhibits remarkable computational efficiency (6× faster than other dataset-specific models), demonstrates strong generalization across different hospitals, transfers well to numerous downstream tasks, and more importantly, facilitates the extensibility to new classes while alleviating the catastrophic forgetting of previously learned classes. Codes, models, and datasets are available at https://github.com/ljwztc/CLIP-Driven-Universal-Model.
RESUMEN
Multi-modal prompt learning is a high-performance and cost-effective learning paradigm, which learns text as well as image prompts to tune pre-trained vision-language (V-L) models like CLIP for adapting multiple downstream tasks. However, recent methods typically treat text and image prompts as independent components without considering the dependency between prompts. Moreover, extending multi-modal prompt learning into the medical field poses challenges due to a significant gap between general- and medical-domain data. To this end, we propose a Multi-modal Collaborative Prompt Learning (MCPL) pipeline to tune a frozen V-L model for aligning medical text-image representations, thereby achieving medical downstream tasks. We first construct the anatomy-pathology (AP) prompt for multi-modal prompting jointly with text and image prompts. The AP prompt introduces instance-level anatomy and pathology information, thereby making a V-L model better comprehend medical reports and images. Next, we propose graph-guided prompt collaboration module (GPCM), which explicitly establishes multi-way couplings between the AP, text, and image prompts, enabling collaborative multi-modal prompt producing and updating for more effective prompting. Finally, we develop a novel prompt configuration scheme, which attaches the AP prompt to the query and key, and the text/image prompt to the value in self-attention layers for improving the interpretability of multi-modal prompts. Extensive experiments on numerous medical classification and object detection datasets show that the proposed pipeline achieves excellent effectiveness and generalization. Compared with state-of-the-art prompt learning methods, MCPL provides a more reliable multi-modal prompt paradigm for reducing tuning costs of V-L models on medical downstream tasks. Our code: https://github.com/CUHK-AIM-Group/MCPL.
RESUMEN
Multi-phase enhanced computed tomography (MPECT) translation from plain CT can help doctors to detect the liver lesion and prevent patients from the allergy during MPECT examination. Existing CT translation methods directly learn an end-to-end mapping from plain CT to MPECT, ignoring the crucial clinical domain knowledge. As clinicians subtract the plain CT from MPECT images as subtraction image to highlight the contrast-enhanced regions and further to facilitate liver disease diagnosis in the clinical diagnosis, we aim to exploit this domain knowledge for automatic CT translation. To this end, we propose a Mask-Aware Transformer (MAFormer) with structure invariant loss for CT translation, which presents the first effort to exploit this domain knowledge for CT translation. Specifically, the proposed MAFormer introduces a mask estimator to predict the subtraction image from the plain CT image. To integrate the subtraction image into the network, the MAFormer devises a Mask-Aware Transformer based Normalization (MATNorm) as normalization layer to highlight the contrast-enhanced regions and capture the long-range dependencies among these regions. Moreover, aiming to preserve the biological structure of CT slices, a structure invariant loss is designed to extract the structural information and minimize the structural similarity between the plain and synthetic CT images to ensure the structure invariant. Extensive experiments have proven the effectiveness of the proposed method and its superiority to the state-of-the-art CT translation methods. Source code is to be released.
Asunto(s)
Tomografía Computarizada por Rayos X , Humanos , Tomografía Computarizada por Rayos X/métodos , Algoritmos , Técnica de Sustracción , Interpretación de Imagen Radiográfica Asistida por Computador/métodosRESUMEN
Fusing multi-modal radiology and pathology data with complementary information can improve the accuracy of tumor typing. However, collecting pathology data is difficult since it is high-cost and sometimes only obtainable after the surgery, which limits the application of multi-modal methods in diagnosis. To address this problem, we propose comprehensively learning multi-modal radiology-pathology data in training, and only using uni-modal radiology data in testing. Concretely, a Memory-aware Hetero-modal Distillation Network (MHD-Net) is proposed, which can distill well-learned multi-modal knowledge with the assistance of memory from the teacher to the student. In the teacher, to tackle the challenge in hetero-modal feature fusion, we propose a novel spatial-differentiated hetero-modal fusion module (SHFM) that models spatial-specific tumor information correlations across modalities. As only radiology data is accessible to the student, we store pathology features in the proposed contrast-boosted typing memory module (CTMM) that achieves type-wise memory updating and stage-wise contrastive memory boosting to ensure the effectiveness and generalization of memory items. In the student, to improve the cross-modal distillation, we propose a multi-stage memory-aware distillation (MMD) scheme that reads memory-aware pathology features from CTMM to remedy missing modal-specific information. Furthermore, we construct a Radiology-Pathology Thymic Epithelial Tumor (RPTET) dataset containing paired CT and WSI images with annotations. Experiments on the RPTET and CPTAC-LUAD datasets demonstrate that MHD-Net significantly improves tumor typing and outperforms existing multi-modal methods on missing modality situations.
Asunto(s)
Neoplasias Glandulares y Epiteliales , Neoplasias del Timo , Humanos , Neoplasias del Timo/diagnóstico por imagen , Neoplasias Glandulares y Epiteliales/diagnóstico por imagen , Interpretación de Imagen Asistida por Computador/métodos , Tomografía Computarizada por Rayos X/métodos , Algoritmos , Redes Neurales de la Computación , Aprendizaje Profundo , Imagen Multimodal/métodosRESUMEN
Polyurethanes (PUs) are a highly adaptable class of biomaterials that are among some of the most researched materials for various biomedical applications. However, engineered tissue scaffolds composed of PU have not found their way into clinical application, mainly due to the difficulty of balancing the control of material properties with the desired cellular response. A simple method for the synthesis of tunable bioactive poly(ethylene glycol) diacrylate (PEGDA) hydrogels containing photocurable PU is described. These hydrogels may be modified with PEGylated peptides or proteins to impart variable biological functions, and the mechanical properties of the hydrogels can be tuned based on the ratios of PU and PEGDA. Studies with human cells revealed that PU-PEG blended hydrogels support cell adhesion and viability when cell adhesion peptides are crosslinked within the hydrogel matrix. These hydrogels represent a unique and highly tailorable system for synthesizing PU-based synthetic extracellular matrices for tissue engineering applications.
RESUMEN
Integrated microwave photonics (MWP) is an intriguing technology for the generation, transmission and manipulation of microwave signals in chip-scale optical systems1,2. In particular, ultrafast processing of analogue signals in the optical domain with high fidelity and low latency could enable a variety of applications such as MWP filters3-5, microwave signal processing6-9 and image recognition10,11. An ideal integrated MWP processing platform should have both an efficient and high-speed electro-optic modulation block to faithfully perform microwave-optic conversion at low power and also a low-loss functional photonic network to implement various signal-processing tasks. Moreover, large-scale, low-cost manufacturability is required to monolithically integrate the two building blocks on the same chip. Here we demonstrate such an integrated MWP processing engine based on a 4 inch wafer-scale thin-film lithium niobate platform. It can perform multipurpose tasks with processing bandwidths of up to 67 GHz at complementary metal-oxide-semiconductor (CMOS)-compatible voltages. We achieve ultrafast analogue computation, namely temporal integration and differentiation, at sampling rates of up to 256 giga samples per second, and deploy these functions to showcase three proof-of-concept applications: solving ordinary differential equations, generating ultra-wideband signals and detecting edges in images. We further leverage the image edge detector to realize a photonic-assisted image segmentation model that can effectively outline the boundaries of melanoma lesion in medical diagnostic images. Our ultrafast lithium niobate MWP engine could provide compact, low-latency and cost-effective solutions for future wireless communications, high-resolution radar and photonic artificial intelligence.
Asunto(s)
Microondas , Niobio , Óptica y Fotónica , Óxidos , Fotones , Inteligencia Artificial , Diagnóstico por Imagen/instrumentación , Diagnóstico por Imagen/métodos , Melanoma/diagnóstico por imagen , Melanoma/patología , Óptica y Fotónica/instrumentación , Óptica y Fotónica/métodos , Radar , Tecnología Inalámbrica , HumanosRESUMEN
Domain generalization-based fault diagnosis (DGFD) presents significant prospects for recognizing faults without the accessibility of the target domain. Previous DGFD methods have achieved significant progress; however, there are some limitations. First, most DGFG methods statistically model the dependence between time-series data and labels, and they are superficial descriptions to the actual data-generating process. Second, most of the existing DGFD methods are only verified on vibrational time-series datasets, which is insufficient to show the potential of domain generalization in the fault diagnosis area. In response to the above issues, this paper first proposes a DGFD method named Causal Disentanglement Domain Generalization (CDDG), which can reestablish the data-generating process by disentangling time-series data into the causal factors (fault-related representation) and no-casual factors (domain-related representation) with a structural causal model. Specifically, in CDDG, causal aggregation loss is designed to separate the unobservable causal and non-causal factors. Meanwhile, the reconstruction loss is proposed to ensure the information completeness of the disentangled factors. We also introduce a redundancy reduction loss to learn efficient features. The proposed CDDG is verified on five cross-machine vibrational fault diagnosis cases and three cross-environment acoustical anomaly detection cases by comparing it with eight state-of-the-art (SOTA) DGFD methods. We publicize the open-source time-series DGFD Benchmark containing CDDG and the eight SOTA methods. The code repository will be available at https://github.com/ShaneSpace/DGFDBenchmark.
Asunto(s)
Generalización Psicológica , Aprendizaje , Acústica , Benchmarking , CausalidadRESUMEN
Taking advantage of multi-modal radiology-pathology data with complementary clinical information for cancer grading is helpful for doctors to improve diagnosis efficiency and accuracy. However, radiology and pathology data have distinct acquisition difficulties and costs, which leads to incomplete-modality data being common in applications. In this work, we propose a Memory- and Gradient-guided Incomplete Modal-modal Learning (MGIML) framework for cancer grading with incomplete radiology-pathology data. Firstly, to remedy missing-modality information, we propose a Memory-driven Hetero-modality Complement (MH-Complete) scheme, which constructs modal-specific memory banks constrained by a coarse-grained memory boosting (CMB) loss to record generic radiology and pathology feature patterns, and develops a cross-modal memory reading strategy enhanced by a fine-grained memory consistency (FMC) loss to take missing-modality information from well-stored memories. Secondly, as gradient conflicts exist between missing-modality situations, we propose a Rotation-driven Gradient Homogenization (RG-Homogenize) scheme, which estimates instance-specific rotation matrices to smoothly change the feature-level gradient directions, and computes confidence-guided homogenization weights to dynamically balance gradient magnitudes. By simultaneously mitigating gradient direction and magnitude conflicts, this scheme well avoids the negative transfer and optimization imbalance problems. Extensive experiments on CPTAC-UCEC and CPTAC-PDA datasets show that the proposed MGIML framework performs favorably against state-of-the-art multi-modal methods on missing-modality situations.
Asunto(s)
Algoritmos , Clasificación del Tumor , Humanos , Clasificación del Tumor/métodos , Interpretación de Imagen Asistida por Computador/métodos , Aprendizaje Automático , Neoplasias/diagnóstico por imagenRESUMEN
The computer-aided diagnosis (CAD) for rare diseases using medical imaging poses a significant challenge due to the requirement of large volumes of labeled training data, which is particularly difficult to collect for rare diseases. Although Few-shot learning (FSL) methods have been developed for this task, these methods focus solely on rare disease diagnosis, failing to preserve the performance in common disease diagnosis. To address this issue, we propose the Disentangle then Calibrate with Gradient Guidance (DCGG) framework under the setting of generalized few-shot learning, i.e., using one model to diagnose both common and rare diseases. The DCGG framework consists of a network backbone, a gradient-guided network disentanglement (GND) module, and a gradient-induced feature calibration (GFC) module. The GND module disentangles the network into a disease-shared component and a disease-specific component based on gradient guidance, and devises independent optimization strategies for both components, respectively, when learning from rare diseases. The GFC module transfers only the disease-shared channels of common-disease features to rare diseases, and incorporates the optimal transport theory to identify the best transport scheme based on the semantic relationship among different diseases. Based on the best transport scheme, the GFC module calibrates the distribution of rare-disease features at the disease-shared channels, deriving more informative rare-disease features for better diagnosis. The proposed DCGG framework has been evaluated on three public medical image classification datasets. Our results suggest that the DCGG framework achieves state-of-the-art performance in diagnosing both common and rare diseases.
Asunto(s)
Algoritmos , Interpretación de Imagen Asistida por Computador , Enfermedades Raras , Humanos , Enfermedades Raras/diagnóstico por imagen , Interpretación de Imagen Asistida por Computador/métodos , Bases de Datos Factuales , Imagen por Resonancia Magnética/métodos , Aprendizaje AutomáticoRESUMEN
Open set recognition (OSR) aims to accurately classify known diseases and recognize unseen diseases as the unknown class in medical scenarios. However, in existing OSR approaches, gathering data from distributed sites to construct large-scale centralized training datasets usually leads to high privacy and security risk, which could be alleviated elegantly via the popular cross-site training paradigm, federated learning (FL). To this end, we represent the first effort to formulate federated open set recognition (FedOSR), and meanwhile propose a novel Federated Open Set Synthesis (FedOSS) framework to address the core challenge of FedOSR: the unavailability of unknown samples for all anticipated clients during the training phase. The proposed FedOSS framework mainly leverages two modules, i.e., Discrete Unknown Sample Synthesis (DUSS) and Federated Open Space Sampling (FOSS), to generate virtual unknown samples for learning decision boundaries between known and unknown classes. Specifically, DUSS exploits inter-client knowledge inconsistency to recognize known samples near decision boundaries and then pushes them beyond decision boundaries to synthesize discrete virtual unknown samples. FOSS unites these generated unknown samples from different clients to estimate the class-conditional distributions of open data space near decision boundaries and further samples open data, thereby improving the diversity of virtual unknown samples. Additionally, we conduct comprehensive ablation experiments to verify the effectiveness of DUSS and FOSS. FedOSS shows superior performance on public medical datasets in comparison with state-of-the-art approaches. The source code is available at https://github.com/CityU-AIM-Group/FedOSS.
Asunto(s)
Aprendizaje Automático , Programas Informáticos , Humanos , EnfermedadRESUMEN
The fusion of multi-modal data, e.g., pathology slides and genomic profiles, can provide complementary information and benefit glioma grading. However, genomic profiles are difficult to obtain due to the high costs and technical challenges, thus limiting the clinical applications of multi-modal diagnosis. In this work, we investigate the realistic problem where paired pathology-genomic data are available during training, while only pathology slides are accessible for inference. To solve this problem, a comprehensive learning and adaptive teaching framework is proposed to improve the performance of pathological grading models by transferring the privileged knowledge from the multi-modal teacher to the pathology student. For comprehensive learning of the multi-modal teacher, we propose a novel Saliency-Aware Masking (SA-Mask) strategy to explore richer disease-related features from both modalities by masking the most salient features. For adaptive teaching of the pathology student, we first devise a Local Topology Preserving and Discrepancy Eliminating Contrastive Distillation (TDC-Distill) module to align the feature distributions of the teacher and student models. Furthermore, considering the multi-modal teacher may include incorrect information, we propose a Gradient-guided Knowledge Refinement (GK-Refine) module that builds a knowledge bank and adaptively absorbs the reliable knowledge according to their agreement in the gradient space. Experiments on the TCGA GBM-LGG dataset show that our proposed distillation framework improves the pathological glioma grading and outperforms other KD methods. Notably, with the sole pathology slides, our method achieves comparable performance with existing multi-modal methods. The code is available at https://github.com/CUHK-AIM-Group/MultiModal-learning.
Asunto(s)
Glioma , Aprendizaje , HumanosRESUMEN
In terms of increasing privacy issues, Federated Learning (FL) has received extensive attention in medical imaging. Through collaborative training, FL can produce superior diagnostic models with global knowledge, while preserving private data locally. In practice, medical diagnosis suffers from intra-/inter-observer variability, thus label noise is inevitable in dataset preparation. Different from existing studies on centralized datasets, the label noise problem in FL scenarios confronts more challenges, due to data inaccessibility and even noise heterogeneity. In this work, we propose a federated framework with joint Graph Purification (FedGP) to address the label noise in FL through server and clients collaboration. Specifically, to overcome the impact of label noise on local training, we first devise a noisy graph purification on the client side to generate reliable pseudo labels by progressively expanding the purified graph with topological knowledge. Then, we further propose a graph-guided negative ensemble loss to exploit the topology of the client-side purified graph with robust complementary supervision against label noise. Moreover, to address the FL label noise with data silos, we propose a global centroid aggregation on the server side to produce a robust classifier with global knowledge, which can be optimized collaboratively in the FL framework. Extensive experiments are conducted on endoscopic and pathological images with the comparison under the homogeneous, heterogeneous, and real-world label noise for medical FL. Among these diverse noisy FL settings, our FedGP framework significantly outperforms denoising and noisy FL state-of-the-arts by a large margin. The source code is available at https://github.com/CUHK-AIM-Group/FedGP.
Asunto(s)
Aprendizaje , Programas Informáticos , HumanosRESUMEN
This paper studies a practical domain adaptive (DA) semantic segmentation problem where only pseudo-labeled target data is accessible through a black-box model. Due to the domain gap and label shift between two domains, pseudo-labeled target data contains mixed closed-set and open-set label noises. In this paper, we propose a simplex noise transition matrix (SimT) to model the mixed noise distributions in DA semantic segmentation, and leverage SimT to handle open-set label noise and enable novel target recognition. When handling open-set noises, we formulate the problem as estimation of SimT. By exploiting computational geometry analysis and properties of segmentation, we design four complementary regularizers, i.e., volume regularization, anchor guidance, convex guarantee, and semantic constraint, to approximate the true SimT. Specifically, volume regularization minimizes the volume of simplex formed by rows of the non-square SimT, ensuring outputs of model to fit into the ground truth label distribution. To compensate for the lack of open-set knowledge, anchor guidance, convex guarantee, and semantic constraint are devised to enable the modeling of open-set noise distribution. The estimated SimT is utilized to correct noise issues in pseudo labels and promote the generalization ability of segmentation model on target domain data. In the task of novel target recognition, we first propose closed-to-open label correction (C2OLC) to explicitly derive the supervision signal for open-set classes by exploiting the estimated SimT, and then advance a semantic relation (SR) loss that harnesses the inter-class relation to facilitate the open-set class sample recognition in target domain. Extensive experimental results demonstrate that the proposed SimT can be flexibly plugged into existing DA methods to boost both closed-set and open-set class performance.
RESUMEN
Annotated images for rare disease diagnosis are extremely hard to collect. Therefore, identifying rare diseases under a few-shot learning (FSL) setting is significant. Existing FSL methods transfer useful and global knowledge from base classes with abundant training samples to enrich features of novel classes with few training samples, but still face difficulties when being applied to medical images due to the complex lesion characteristics and large intra-class variance. In this paper, we propose a dynamic feature splicing (DNFS) framework for few-shot rare disease diagnosis. Under DNFS, both low-level features (i.e., the output of three convolutional blocks) and high-level features (i.e., the output of the last fully connected layer) of novel classes are dynamically enriched. We construct the position coherent DNFS (P-DNFS) module to perform low-level feature splicing, where a lesion-oriented Transformer is designed to detect lesion regions. Thus, novel-class channels are replaced by similar base-class channels within the detected lesion regions to achieve disease-related feature enrichment. We also devise a semantic coherent DNFS (S-DNFS) module to perform high-level feature splicing. It explores cross-image channel relations and selects base-class channels with semantic consistency for explicit knowledge transfer. Both low-level and high-level feature splicings are performed dynamically and iteratively. Consequently, abundant spliced features are generated for disease diagnosis, leading to more accurate decision boundary and improved diagnosis performance. Extensive experiments have been conducted on three medical image classification datasets. Our results suggest that the proposed DNFS achieves superior performance against state-of-the-art approaches.
RESUMEN
One-shot organ segmentation (OS2) aims at segmenting the desired organ regions from the input medical imaging data with only one pre-annotated example as the reference. By using the minimal annotation data to facilitate organ segmentation, OS2 receives great attention in the medical image analysis community due to its weak requirement on human annotation. In OS2, one core issue is to explore the mutual information between the support (reference slice) and the query (test slice). Existing methods rely heavily on the similarity between slices, and additional slice allocation mechanisms need to be designed to reduce the impact of the similarity between slices on the segmentation performance. To address this issue, we build a novel support-query interactive embedding (SQIE) module, which is equipped with the channel-wise co-attention, spatial-wise co-attention, and spatial bias transformation blocks to identify "what to look", "where to look", and "how to look" in the input test slice. By combining the three mechanisms, we can mine the interactive information of the intersection area and the disputed area between slices, and establish the feature connection between the target in slices with low similarity. We also propose a self-supervised contrastive learning framework, which transforms knowledge from the physical position to the embedding space to facilitate the self-supervised interactive embedding of the query and support slices. Comprehensive experiments on two large benchmarks demonstrate the superior capacity of the proposed approach when compared with the current alternatives and baseline models.
RESUMEN
Multi-modality medical data provide complementary information, and hence have been widely explored for computer-aided AD diagnosis. However, the research is hindered by the unavoidable missing-data problem, i.e., one data modality was not acquired on some subjects due to various reasons. Although the missing data can be imputed using generative models, the imputation process may introduce unrealistic information to the classification process, leading to poor performance. In this paper, we propose the Disentangle First, Then Distill (DFTD) framework for AD diagnosis using incomplete multi-modality medical images. First, we design a region-aware disentanglement module to disentangle each image into inter-modality relevant representation and intra-modality specific representation with emphasis on disease-related regions. To progressively integrate multi-modality knowledge, we then construct an imputation-induced distillation module, in which a lateral inter-modality transition unit is created to impute representation of the missing modality. The proposed DFTD framework has been evaluated against six existing methods on an ADNI dataset with 1248 subjects. The results show that our method has superior performance in both AD-CN classification and MCI-to-AD prediction tasks, substantially over-performing all competing methods.