RESUMO
Human anatomy is the foundation of medical imaging and boasts one striking characteristic: its hierarchy in nature, exhibiting two intrinsic properties: (1) locality: each anatomical structure is morphologically distinct from the others; and (2) compositionality: each anatomical structure is an integrated part of a larger whole. We envision a foundation model for medical imaging that is consciously and purposefully developed upon this foundation to gain the capability of "understanding" human anatomy and to possess the fundamental properties of medical imaging. As our first step in realizing this vision towards foundation models in medical imaging, we devise a novel self-supervised learning (SSL) strategy that exploits the hierarchical nature of human anatomy. Our extensive experiments demonstrate that the SSL pretrained model, derived from our training strategy, not only outperforms state-of-the-art (SOTA) fully/self-supervised baselines but also enhances annotation efficiency, offering potential few-shot segmentation capabilities with performance improvements ranging from 9% to 30% for segmentation tasks compared to SSL baselines. This performance is attributed to the significance of anatomy comprehension via our learning strategy, which encapsulates the intrinsic attributes of anatomical structures-locality and compositionality-within the embedding space, yet overlooked in existing SSL methods. All code and pretrained models are available at GitHub.com/JLiangLab/Eden.
RESUMO
Recent studies have shown that methane emissions are underestimated by inventories in many US urban areas. This has important implications for climate change mitigation policy at the city, state, and national levels. Uncertainty in both the spatial distribution and sectoral allocation of urban emissions can limit the ability of policy makers to develop appropriately focused emission reduction strategies. Top-down emission estimates based on atmospheric greenhouse gas measurements can help to improve inventories and inform policy decisions. This study presents a new high-resolution (0.02 × 0.02°) methane emission inventory for New York City and its surrounding area, constructed using the latest activity data, emission factors, and spatial proxies. The new high-resolution inventory estimates of methane emissions for the New York-Newark urban area are 1.3 times larger than those for the gridded Environmental Protection Agency inventory. We used aircraft mole fraction measurements from nine research flights to optimize the high-resolution inventory emissions within a Bayesian inversion. These sectorally optimized emissions show that the high-resolution inventory still significantly underestimates methane emissions within the New York-Newark urban area, primarily because it underestimates emissions from thermogenic sources (by a factor of 2.3). This suggests that there remains a gap in our process-based understanding of urban methane emissions.
Assuntos
Metano , Cidade de Nova Iorque , Metano/análise , Monitoramento Ambiental , Poluentes Atmosféricos/análise , Teorema de BayesRESUMO
We have developed a United framework that integrates three self-supervised learning (SSL) ingredients (discriminative, restorative, and adversarial learning), enabling collaborative learning among the three learning ingredients and yielding three transferable components: a discriminative encoder, a restorative decoder, and an adversary encoder. To leverage this collaboration, we redesigned nine prominent self-supervised methods, including Rotation, Jigsaw, Rubik's Cube, Deep Clustering, TransVW, MoCo, BYOL, PCRL, and Swin UNETR, and augmented each with its missing components in a United framework for 3D medical imaging. However, such a United framework increases model complexity, making 3D pretraining difficult. To overcome this difficulty, we propose stepwise incremental pretraining, a strategy that unifies the pretraining, in which a discriminative encoder is first trained via discriminative learning, the pretrained discriminative encoder is then attached to a restorative decoder, forming a skip-connected encoder-decoder, for further joint discriminative and restorative learning. Last, the pretrained encoder-decoder is associated with an adversarial encoder for final full discriminative, restorative, and adversarial learning. Our extensive experiments demonstrate that the stepwise incremental pretraining stabilizes United models pretraining, resulting in significant performance gains and annotation cost reduction via transfer learning in six target tasks, ranging from classification to segmentation, across diseases, organs, datasets, and modalities. This performance improvement is attributed to the synergy of the three SSL ingredients in our United framework unleashed through stepwise incremental pretraining. Our codes and pretrained models are available at GitHub.com/JLiangLab/StepwisePretraining.
Assuntos
Imageamento Tridimensional , Aprendizado de Máquina Supervisionado , Humanos , Imageamento Tridimensional/métodos , AlgoritmosRESUMO
Discriminative, restorative, and adversarial learning have proven beneficial for self-supervised learning schemes in computer vision and medical imaging. Existing efforts, however, fail to capitalize on the potentially synergistic effects these methods may offer in a ternary setup, which, we envision can significantly benefit deep semantic representation learning. Towards this end, we developed DiRA, the first framework that unites discriminative, restorative, and adversarial learning in a unified manner to collaboratively glean complementary visual information from unlabeled medical images for fine-grained semantic representation learning. Our extensive experiments demonstrate that DiRA: (1) encourages collaborative learning among three learning ingredients, resulting in more generalizable representation across organs, diseases, and modalities; (2) outperforms fully supervised ImageNet models and increases robustness in small data regimes, reducing annotation cost across multiple medical imaging applications; (3) learns fine-grained semantic representation, facilitating accurate lesion localization with only image-level annotation; (4) improves reusability of low/mid-level features; and (5) enhances restorative self-supervised approaches, revealing that DiRA is a general framework for united representation learning. Code and pretrained models are available at https://github.com/JLiangLab/DiRA.
Assuntos
Doenças Hereditárias Autoinflamatórias , Humanos , Semântica , Aprendizado de Máquina Supervisionado , Proteína Antagonista do Receptor de Interleucina 1RESUMO
Background: Most solid tumors are not diagnosed and treated until the advanced stage, in which tumors have shaped mature self-protective power, leading to off-target drugs and nanomedicines. In the present studies, we established a more realistic large tumor model to test the antitumor activity of a multifunctional ginsenoside Rh2-based liposome system (Rh2-lipo) on advanced breast cancer. Methods: Both cholesterol and PEG were substituted by Rh2 to prepare the Rh2-lipo using ethanol-water system and characterized. The effects of Rh2-lipo on cell uptake, penetration of the tumor spheroid, cytotoxicity assay was investigated with 4T1 breast cancer cells and L929 fibroblast cells. The 4T1 orthotopic-bearing large tumor model was established to study the targeting effect of Rh2-lipo and inhibitory effect of paclitaxel loaded Rh2-lipo (PTX-Rh2-lipo) on advanced breast tumors. Results: Rh2-lipo exhibit many advantages that address the limitations of current liposome formulations against large tumors, such as enhanced uptake in TAFs and tumor cells, high targeting and penetration capacity, cytotoxicity against TAFs, normalization of the vessel network, and depletion of stromal collagen. In in vivo study, PTX-Rh2-lipo effectively inhibiting the growth of advanced breast tumors and outperformed most reported PTX formulations, including Lipusu® and Abraxane®. Conclusion: Rh2-lipo have improved drug delivery efficiency and antitumor efficacy in advanced breast cancer, which offers a novel promising platform for advanced tumor therapy.
Assuntos
Neoplasias da Mama , Ginsenosídeos , Lipossomos , Humanos , Feminino , Lipossomos/uso terapêutico , Neoplasias da Mama/tratamento farmacológico , Sistemas de Liberação de Medicamentos , Paclitaxel/farmacologia , Paclitaxel/uso terapêutico , Linhagem Celular TumoralRESUMO
BACKGROUND: Based on understanding of placental pathological features and safe medication in pregnancy-associated malaria (PAM), establishment of a stable pregnant mouse infection model with Plasmodium was urgently needed. METHODS: ICR mice with vaginal plugs detected were randomly divided into post-pregnancy infection (Malaria+) and uninfected pregnancy (Malaria-) cohorts. Age-matched mice that had not been mated were infected as pre-pregnancy infection group (Virgin control), which were subsequently mated with ICR males. All mice were inoculated with 1 × 106Plasmodium berghei ANKA-infected RBCs by intraperitoneal injection, and the same amount of saline was given to Malaria- group. We recorded the incidence of adverse pregnancy outcomes and the amounts of offspring in each group. RESULTS: The Virgin group mice were unable to conceive normally, and vaginal bleeding, abortion, or stillbirth appeared in the Malaria+ group. The incidence of adverse pregnancy outcomes was extremely high and statistically significant compared with the control (Malaria-) group (P < 0.05), of which placenta exhibited pathological features associated with human gestational malaria. CONCLUSIONS: The intraperitoneal injection of 1 × 106Plasmodium berghei ANKA-infected RBCs could establish a model of pregnancy-associated malaria in ICR mouse.
Assuntos
Malária , Resultado da Gravidez , Masculino , Gravidez , Feminino , Camundongos , Animais , Humanos , Camundongos Endogâmicos ICR , Placenta/patologia , Malária/tratamento farmacológico , Plasmodium bergheiRESUMO
Pulmonary Embolism (PE) represents a thrombus ("blood clot"), usually originating from a lower extremity vein, that travels to the blood vessels in the lung, causing vascular obstruction and in some patients death. This disorder is commonly diagnosed using Computed Tomography Pulmonary Angiography (CTPA). Deep learning holds great promise for the Computer-aided Diagnosis (CAD) of PE. However, numerous deep learning methods, such as Convolutional Neural Networks (CNN) and Transformer-based models, exist for a given task, causing great confusion regarding the development of CAD systems for PE. To address this confusion, we present a comprehensive analysis of competing deep learning methods applicable to PE diagnosis based on four datasets. First, we use the RSNA PE dataset, which includes (weak) slice-level and exam-level labels, for PE classification and diagnosis, respectively. At the slice level, we compare CNNs with the Vision Transformer (ViT) and the Swin Transformer. We also investigate the impact of self-supervised versus (fully) supervised ImageNet pre-training, and transfer learning over training models from scratch. Additionally, at the exam level, we compare sequence model learning with our proposed transformer-based architecture, Embedding-based ViT (E-ViT). For the second and third datasets, we utilize the CAD-PE Challenge Dataset and Ferdowsi University of Mashad's PE Dataset, where we convert (strong) clot-level masks into slice-level annotations to evaluate the optimal CNN model for slice-level PE classification. Finally, we use our in-house PE-CAD dataset, which contains (strong) clot-level masks. Here, we investigate the impact of our vessel-oriented image representations and self-supervised pre-training on PE false positive reduction at the clot level across image dimensions (2D, 2.5D, and 3D). Our experiments show that (1) transfer learning boosts performance despite differences between photographic images and CTPA scans; (2) self-supervised pre-training can surpass (fully) supervised pre-training; (3) transformer-based models demonstrate comparable performance but slower convergence compared with CNNs for slice-level PE classification; (4) model trained on the RSNA PE dataset demonstrates promising performance when tested on unseen datasets for slice-level PE classification; (5) our E-ViT framework excels in handling variable numbers of slices and outperforms sequence model learning for exam-level diagnosis; and (6) vessel-oriented image representation and self-supervised pre-training both enhance performance for PE false positive reduction across image dimensions. Our optimal approach surpasses state-of-the-art results on the RSNA PE dataset, enhancing AUC by 0.62% (slice-level) and 2.22% (exam-level). On our in-house PE-CAD dataset, 3D vessel-oriented images improve performance from 80.07% to 91.35%, a remarkable 11% gain. Codes are available at GitHub.com/JLiangLab/CAD_PE.
Assuntos
Diagnóstico por Computador , Embolia Pulmonar , Humanos , Diagnóstico por Computador/métodos , Redes Neurais de Computação , Imageamento Tridimensional , Embolia Pulmonar/diagnóstico por imagem , ComputadoresRESUMO
Recently, artemisinin and derivatives have been revealed to possess encouraging antitumor activity. Herein, we integrated the antitumor advantages of artesunate and platinum drugs to construct novel PtIV-artesunate dual-action and triple-action complexes. Most derivatives, especially 10f, displayed broad-spectrum and potent in vitro antitumor activities against a number of cancer cell lines. Compound 10f displayed potent antimetastasis and anticlonogenic activities, efficiently induced autophagic cell death and apoptosis, and arrested the cell cycle at both S and G2/M phases. More importantly, it displayed remarkable in vivo antitumor efficacy in the A549 xenograft model (TGI = 53.4%; 6 µmol/kg) with low toxicity. In addition to the antitumor application, 10f showed potent in vivo antimalarial activity in malarial-infected mice model and obviously alleviated malarial-related multiorgan injury. This conjugation greatly improved safety, especially reducing the platinum drugs' nephrotoxicity. Taken together, this study highlighted the therapeutic potential of PtIV-artesunate complexes as antitumor and antimalarial agents.
Assuntos
Antimaláricos , Antineoplásicos , Pró-Fármacos , Camundongos , Animais , Humanos , Platina/farmacologia , Antineoplásicos/farmacologia , Antineoplásicos/uso terapêutico , Antimaláricos/farmacologia , Antimaláricos/uso terapêutico , Artesunato/farmacologia , Artesunato/uso terapêutico , Pró-Fármacos/farmacologia , Pró-Fármacos/uso terapêutico , Linhagem Celular Tumoral , ApoptoseRESUMO
BACKGROUND: Malaria remains a serious threat to global public health. With poor efficacies of vaccines and the emergence of drug resistance, novel strategies to control malaria are urgently needed. RESULTS: We developed erythrocyte membrane-camouflaged nanoparticles loaded with artemether based on the growth characteristics of Plasmodium. The nanoparticles could capture the merozoites to inhibit them from repeatedly infecting normal erythrocytes, owing to the interactions between merozoites and heparin-like molecules on the erythrocyte membrane. Modification with a phosphatidylserine-targeting peptide (CLIPPKF) improved the drug accumulation in infected red blood cells (iRBCs) from the externalized phosphatidylserine induced by Plasmodium infection. In Plasmodium berghei ANKA strain (pbANKA)-infected C57BL/6 mice, the nanoparticles significantly attenuated Plasmodium-induced inflammation, apoptosis, and anemia. We observed reduced weight variation and prolonged survival time in pbANKA-challenged mice, and the nanoparticles showed good biocompatibility and negligible cytotoxicity. CONCLUSION: Erythrocyte membrane-camouflaged nanoparticles loaded with artemether were shown to provide safe and effective protection against Plasmodium infection.
Assuntos
Malária , Merozoítos , Animais , Camundongos , Membrana Eritrocítica , Fosfatidilserinas , Biomimética , Camundongos Endogâmicos C57BL , Malária/tratamento farmacológico , Malária/prevenção & controle , Eritrócitos , Artemeter/farmacologia , Plasmodium berghei , Plasmodium falciparumRESUMO
PURPOSE: Nicotinamide adenine dinucleotide (NAD+) is closely related to the pathogenesis of tumors. However, the effect of NAD+ metabolism of gastric cancer (GC) cells on immune cells remains unexplained. We targeted nicotinamide phosphoribosyltransferase (NAMPT), a rate-limiting enzyme in the NAD+ synthesis salvage pathway, to observe its effect in the immune microenvironment. METHODS: NAMPT of GC cell lines was inhibited by using the small molecule inhibitor (FK866) and short hairpin RNA (shRNA). CCK-8 test and flow cytometry were performed to detect cell viability and apoptosis. Immunofluorescence was used to observe changes in mitochondrial membrane potential (MMP).The transfected GC cells (AGS) and patient-derived organoids (PDOs) were cocultured with activated PBMCs, followed by flow cytometric analysis (FCA) for cytokines and inhibitory marker. The level of NAD and ATP of GC cells (AGS & MKN45) was tested combined with NMN and CD39 inhibitor. RESULTS: Targeting NAD+ by FK866 obviously reduced MMP, which ultimately inhibited proliferation and increased the apoptosis of GC cells. NAMPT silencing reduced intracellular NAD and ATPï¼further decreased extracellular adenosine. Meawhile, the cytokines of CD8+T cells were significantly increased after cocultured with transfected AGS, and the expression of PD-1 was distinctly decreased. NMN reversed the effect of shNAMPT and enhanced the immunosuppression. Consistent results were obtained by coculturing PBMCs with PDOs. CONCLUSION: Restraining the function of NAMPT resulted in the functional improvement of effector CD8+ T cells by decreasing extracellular adenosine levels and inducing apoptosis of GC cells simultaneously. Therefore, this study demonstrates that NAMPT can be an effective target for gastric cancer immunotherapy.
Assuntos
NAD , Neoplasias Gástricas , Humanos , NAD/metabolismo , Adenosina/farmacologia , Neoplasias Gástricas/tratamento farmacológico , Linhagem Celular Tumoral , Microambiente Tumoral , Citocinas/metabolismo , Nicotinamida Fosforribosiltransferase/genética , Nicotinamida Fosforribosiltransferase/metabolismo , Trifosfato de Adenosina/metabolismo , Linfócitos T CD8-Positivos/metabolismoAssuntos
Higiene das Mãos , Humanos , Desinfecção das Mãos , Fidelidade a Diretrizes , Corpo ClínicoRESUMO
Deep learning nowadays offers expert-level and sometimes even super-expert-level performance, but achieving such performance demands massive annotated data for training (e.g., Google's proprietary CXR Foundation Model (CXR-FM) was trained on 821,544 labeled and mostly private chest X-rays (CXRs)). Numerous datasets are publicly available in medical imaging but individually small and heterogeneous in expert labels. We envision a powerful and robust foundation model that can be trained by aggregating numerous small public datasets. To realize this vision, we have developed Ark, a framework that accrues and reuses knowledge from heterogeneous expert annotations in various datasets. As a proof of concept, we have trained two Ark models on 335,484 and 704,363 CXRs, respectively, by merging several datasets including ChestX-ray14, CheXpert, MIMIC-II, and VinDr-CXR, evaluated them on a wide range of imaging tasks covering both classification and segmentation via fine-tuning, linear-probing, and gender-bias analysis, and demonstrated our Ark's superior and robust performance over the state-of-the-art (SOTA) fully/self-supervised baselines and Google's proprietary CXR-FM. This enhanced performance is attributed to our simple yet powerful observation that aggregating numerous public datasets diversifies patient populations and accrues knowledge from diverse experts, yielding unprecedented performance yet saving annotation cost. With all codes and pretrained models released at GitHub.com/JLiangLab/Ark, we hope that Ark exerts an important impact on open science, as accruing and reusing knowledge from expert annotations in public datasets can potentially surpass the performance of proprietary models trained on unusually large data, inspiring many more researchers worldwide to share codes and datasets to build open foundation models, accelerate open science, and democratize deep learning for medical imaging.
RESUMO
Self-supervised learning (SSL) approaches have recently shown substantial success in learning visual representations from unannotated images. Compared with photographic images, medical images acquired with the same imaging protocol exhibit high consistency in anatomy. To exploit this anatomical consistency, this paper introduces a novel SSL approach, called PEAC (patch embedding of anatomical consistency), for medical image analysis. Specifically, in this paper, we propose to learn global and local consistencies via stable grid-based matching, transfer pre-trained PEAC models to diverse downstream tasks, and extensively demonstrate that (1) PEAC achieves significantly better performance than the existing state-of-the-art fully/self-supervised methods, and (2) PEAC captures the anatomical structure consistency across views of the same patient and across patients of different genders, weights, and healthy statuses, which enhances the interpretability of our method for medical image analysis. All code and pretrained models are available at GitHub.com/JLiangLab/PEAC.
RESUMO
Vision transformer-based self-supervised learning (SSL) approaches have recently shown substantial success in learning visual representations from unannotated photographic images. However, their acceptance in medical imaging is still lukewarm, due to the significant discrepancy between medical and photographic images. Consequently, we propose POPAR (patch order prediction and appearance recovery), a novel vision transformer-based self-supervised learning framework for chest X-ray images. POPAR leverages the benefits of vision transformers and unique properties of medical imaging, aiming to simultaneously learn patch-wise high-level contextual features by correcting shuffled patch orders and fine-grained features by recovering patch appearance. We transfer POPAR pretrained models to diverse downstream tasks. The experiment results suggest that (1) POPAR outperforms state-of-the-art (SoTA) self-supervised models with vision transformer backbone; (2) POPAR achieves significantly better performance over all three SoTA contrastive learning methods; and (3) POPAR also outperforms fully-supervised pretrained models across architectures. In addition, our ablation study suggests that to achieve better performance on medical imaging tasks, both fine-grained and global contextual features are preferred. All code and models are available at GitHub.com/JLiangLab/POPAR.
RESUMO
Uniting three self-supervised learning (SSL) ingredients (discriminative, restorative, and adversarial learning) enables collaborative representation learning and yields three transferable components: a discriminative encoder, a restorative decoder, and an adversary encoder. To leverage this advantage, we have redesigned five prominent SSL methods, including Rotation, Jigsaw, Rubik's Cube, Deep Clustering, and TransVW, and formulated each in a United framework for 3D medical imaging. However, such a United framework increases model complexity and pretraining difficulty. To overcome this difficulty, we develop a stepwise incremental pretraining strategy, in which a discriminative encoder is first trained via discriminative learning, the pretrained discriminative encoder is then attached to a restorative decoder, forming a skip-connected encoder-decoder, for further joint discriminative and restorative learning, and finally, the pretrained encoder-decoder is associated with an adversarial encoder for final full discriminative, restorative, and adversarial learning. Our extensive experiments demonstrate that the stepwise incremental pretraining stabilizes United models training, resulting in significant performance gains and annotation cost reduction via transfer learning for five target tasks, encompassing both classification and segmentation, across diseases, organs, datasets, and modalities. This performance is attributed to the synergy of the three SSL ingredients in our United framework unleashed via stepwise incremental pretraining. All codes and pretrained models are available at GitHub.com/JLiangLab/StepwisePretraining.
RESUMO
Recently, self-supervised instance discrimination methods have achieved significant success in learning visual representations from unlabeled photographic images. However, given the marked differences between photographic and medical images, the efficacy of instance-based objectives, focusing on learning the most discriminative global features in the image (i.e., wheels in bicycle), remains unknown in medical imaging. Our preliminary analysis showed that high global similarity of medical images in terms of anatomy hampers instance discrimination methods for capturing a set of distinct features, negatively impacting their performance on medical downstream tasks. To alleviate this limitation, we have developed a simple yet effective self-supervised framework, called Context-Aware instance Discrimination (CAiD). CAiD aims to improve instance discrimination learning by providing finer and more discriminative information encoded from a diverse local context of unlabeled medical images. We conduct a systematic analysis to investigate the utility of the learned features from a three-pronged perspective: (i) generalizability and transferability, (ii) separability in the embedding space, and (iii) reusability. Our extensive experiments demonstrate that CAiD (1) enriches representations learned from existing instance discrimination methods; (2) delivers more discriminative features by adequately capturing finer contextual information from individual medial images; and (3) improves reusability of low/mid-level features compared to standard instance discriminative methods. As open science, all codes and pre-trained models are available on our GitHub page: https://github.com/JLiangLab/CAiD.
RESUMO
Purpose: Osteosarcoma (OS) is the most common bone cancer with a high risk of metastasis, high growth rate, and poor prognosis. Honokiol (HNK) is a general ingredient of traditional Chinese medicine, with a potential anti-tumor effect. However, HNK is insoluble in water and lacks drug targeting, which limits its clinical application. To improve the OS therapeutic effect of HNK, we used HNK-loaded liposomes modified with hyaluronic acid-phospholipid conjugates (HA-DOPE) to treat OS based on the HA interaction with CD44. Methods: The HNK-loaded liposomes were prepared via thin-film hydration and sonication. HA-DOPE was used to combine the HNK-loaded liposomes (HA-DOPE@Lips/HNK) via sonication and co-extrusion. HA-DOPE@Lips/HNK were characterized with respect to size, zeta potential, polymer dispersity index (PDI), and stability, and transmission electron microscopy was performed. Cellular uptake, cell viability, cell apoptosis, cell cycle, and mitochondrial activity were utilized to evaluate the antitumor effect in vitro. The biodistribution, xenograft tumor growth inhibition, and safety of HA-DOPE@Lips/HNK were evaluated in 143B OS xenograft mice in vivo. Results: The particle size, PDI, and zeta potential of HA-DOPE@Lips/HNK were 146.20±0.26 nm, 0.20±0.01, and -38.45±0.98 mV, respectively. The encapsulation rate and drug loading were 80.14±0.32% and 3.78±0.09%, respectively. HA-DOPE@Lips/HNK could inhibit cell proliferation, cause apoptosis, block the cell cycle and disrupt mitochondrial activity. HA-DOPE@Lips/HNK specially delivered the drug into the tumor and inhibited tumor growth, and showed no obvious toxicity to normal tissues. Conclusion: HA-DOPE@Lips/HNK could deliver HNK into the tumor site and had a good antitumor ability in vitro and in vivo. In addition, HA-DOPE@Lips/HNK increased the antitumor effects of HNK. Thus, it provides a promising nanocarrier to improve drug delivery in OS therapy.
Assuntos
Neoplasias Ósseas , Osteossarcoma , Humanos , Camundongos , Animais , Lipossomos/uso terapêutico , Distribuição Tecidual , Linhagem Celular Tumoral , Osteossarcoma/patologia , Neoplasias Ósseas/patologia , Ácido Hialurônico , Polímeros/metabolismoRESUMO
Visual transformers have recently gained popularity in the computer vision community as they began to outrank convolutional neural networks (CNNs) in one representative visual benchmark after another. However, the competition between visual transformers and CNNs in medical imaging is rarely studied, leaving many important questions unanswered. As the first step, we benchmark how well existing transformer variants that use various (supervised and self-supervised) pre-training methods perform against CNNs on a variety of medical classification tasks. Furthermore, given the data-hungry nature of transformers and the annotation-deficiency challenge of medical imaging, we present a practical approach for bridging the domain gap between photographic and medical images by utilizing unlabeled large-scale in-domain data. Our extensive empirical evaluations reveal the following insights in medical imaging: (1) good initialization is more crucial for transformer-based models than for CNNs, (2) self-supervised learning based on masked image modeling captures more generalizable representations than supervised models, and (3) assembling a larger-scale domain-specific dataset can better bridge the domain gap between photographic and medical images via self-supervised continuous pre-training. We hope this benchmark study can direct future research on applying transformers to medical imaging analysis. All codes and pre-trained models are available on our GitHub page https://github.com/JLiangLab/BenchmarkTransformers.
RESUMO
Discriminative learning, restorative learning, and adversarial learning have proven beneficial for self-supervised learning schemes in computer vision and medical imaging. Existing efforts, however, omit their synergistic effects on each other in a ternary setup, which, we envision, can significantly benefit deep semantic representation learning. To realize this vision, we have developed DiRA, the first framework that unites discriminative, restorative, and adversarial learning in a unified manner to collaboratively glean complementary visual information from unlabeled medical images for fine-grained semantic representation learning. Our extensive experiments demonstrate that DiRA (1) encourages collaborative learning among three learning ingredients, resulting in more generalizable representation across organs, diseases, and modalities; (2) outperforms fully supervised ImageNet models and increases robustness in small data regimes, reducing annotation cost across multiple medical imaging applications; (3) learns fine-grained semantic representation, facilitating accurate lesion localization with only image-level annotation; and (4) enhances state-of-the-art restorative approaches, revealing that DiRA is a general mechanism for united representation learning. All code and pretrained models are available at https://github.com/JLiangLab/DiRA.
RESUMO
The Schlafen (SLFN) gene family plays an important role in immune cell differentiation and immune regulation. Previous studies have found that the increased SLFN5 expression in patients with intestinal metaplasia correlates with gastric cancer (GC) progression. However, no investigation has been conducted on the SLFN family in GC. Therefore, we systematically explore the expression and prognostic value of SLFN family members in patients with GC, elucidating their possible biological function and its correlation with tumor immune cells infiltration. TCGA database results indicated that the SLFN5, SLFN11, SLFN12, SLFN12L, and SLFN13 expression was significantly higher in GC. The UALCAN and KM plotter databases indicated that enhanced the SLFN family expression was associated with lymph node metastasis, tumor stage, and tumor grade and predicted an adverse prognosis. cBioportal database revealed that the SLFN family had a high frequency of genetic alterations in GC (about 12%), including mutations and amplification. The GeneMANIA and STRING databases identified 20 interacting genes and 16 interacting proteins that act as potential targets of the SLFN family. SLFN5, SLFN11, SLFN12, SLFN12L, and SLFN14 may be implicated in the immunological response, according to Gene Set Enrichment Analysis (GSEA) and Kyoto Encyclopedia of Genes and Genomes (KEGG). Additionally, Timer and TISIDB databases indicate that SLFN5, SLFN11, SLFN12, SLFN12L, and SLFN14 are involved in the immune response. Furthermore, Timer, TCGA, and TISIDB databases suggested that the SLFN5, SLFN11, SLFN12, SLFN12L, and SLFN14 expression in GC is highly linked with immune cell infiltration levels, immune checkpoint, and the many immune cell marker sets expression. We isolated three samples of peripheral blood mononuclear cell (PBMC) and activated T cells; the results showed the expression of SLFN family members decreased significantly when T cell active. In conclusion, the SLFN family of proteins may act as a prognostic indicator of GC and is associated with immune cell infiltration and immune checkpoint expression in GC. Additionally, it may be involved in tumor immune evasion by regulating T cell activation.