Your browser doesn't support javascript.
loading
Show: 20 | 50 | 100
Results 1 - 20 de 133
Filter
1.
JMIR Med Inform ; 12: e57674, 2024 Jun 28.
Article in English | MEDLINE | ID: mdl-38952020

ABSTRACT

Background: Large language models (LLMs) have achieved great progress in natural language processing tasks and demonstrated the potential for use in clinical applications. Despite their capabilities, LLMs in the medical domain are prone to generating hallucinations (not fully reliable responses). Hallucinations in LLMs' responses create substantial risks, potentially threatening patients' physical safety. Thus, to perceive and prevent this safety risk, it is essential to evaluate LLMs in the medical domain and build a systematic evaluation. Objective: We developed a comprehensive evaluation system, MedGPTEval, composed of criteria, medical data sets in Chinese, and publicly available benchmarks. Methods: First, a set of evaluation criteria was designed based on a comprehensive literature review. Second, existing candidate criteria were optimized by using a Delphi method with 5 experts in medicine and engineering. Third, 3 clinical experts designed medical data sets to interact with LLMs. Finally, benchmarking experiments were conducted on the data sets. The responses generated by chatbots based on LLMs were recorded for blind evaluations by 5 licensed medical experts. The evaluation criteria that were obtained covered medical professional capabilities, social comprehensive capabilities, contextual capabilities, and computational robustness, with 16 detailed indicators. The medical data sets include 27 medical dialogues and 7 case reports in Chinese. Three chatbots were evaluated: ChatGPT by OpenAI; ERNIE Bot by Baidu, Inc; and Doctor PuJiang (Dr PJ) by Shanghai Artificial Intelligence Laboratory. Results: Dr PJ outperformed ChatGPT and ERNIE Bot in the multiple-turn medical dialogues and case report scenarios. Dr PJ also outperformed ChatGPT in the semantic consistency rate and complete error rate category, indicating better robustness. However, Dr PJ had slightly lower scores in medical professional capabilities compared with ChatGPT in the multiple-turn dialogue scenario. Conclusions: MedGPTEval provides comprehensive criteria to evaluate chatbots by LLMs in the medical domain, open-source data sets, and benchmarks assessing 3 LLMs. Experimental results demonstrate that Dr PJ outperforms ChatGPT and ERNIE Bot in social and professional contexts. Therefore, such an assessment system can be easily adopted by researchers in this community to augment an open-source data set.

2.
Comput Med Imaging Graph ; 116: 102416, 2024 Jul 09.
Article in English | MEDLINE | ID: mdl-39018640

ABSTRACT

Despite that deep learning has achieved state-of-the-art performance for automatic medical image segmentation, it often requires a large amount of pixel-level manual annotations for training. Obtaining these high-quality annotations is time-consuming and requires specialized knowledge, which hinders the widespread application that relies on such annotations to train a model with good segmentation performance. Using scribble annotations can substantially reduce the annotation cost, but often leads to poor segmentation performance due to insufficient supervision. In this work, we propose a novel framework named as ScribSD+ that is based on multi-scale knowledge distillation and class-wise contrastive regularization for learning from scribble annotations. For a student network supervised by scribbles and the teacher based on Exponential Moving Average (EMA), we first introduce multi-scale prediction-level Knowledge Distillation (KD) that leverages soft predictions of the teacher network to supervise the student at multiple scales, and then propose class-wise contrastive regularization which encourages feature similarity within the same class and dissimilarity across different classes, thereby effectively improving the segmentation performance of the student network. Experimental results on the ACDC dataset for heart structure segmentation and a fetal MRI dataset for placenta and fetal brain segmentation demonstrate that our method significantly improves the student's performance and outperforms five state-of-the-art scribble-supervised learning methods. Consequently, the method has a potential for reducing the annotation cost in developing deep learning models for clinical diagnosis.

3.
Cell Biosci ; 14(1): 87, 2024 Jun 29.
Article in English | MEDLINE | ID: mdl-38951864

ABSTRACT

BACKGROUND: Zinc finger SWIM-type containing 4 (ZSWIM4) is a zinc finger protein with its function largely uncharacterized. In this study, we aimed to investigate the role of ZSWIM4 in gastrointestinal stromal tumors (GISTs). RESULTS: We found that ZSWIM4 expression is inhibited by the predominantly mutated protein KIT in GISTs, while conversely, ZSWIM4 inhibits KIT expression and downstream signaling. Consistent with the observation, ZSWIM4 inhibited GIST cell survival and proliferation in vitro. RNA sequencing of GISTs from KITV558A/WT mice and KITV558A/WT/ZSWIM4-/- mice showed that loss of ZSWIM4 expression increases the expression of circadian clock pathway member BMAL1 which contributes to GIST cell survival and proliferation. In addition, we found that KIT signaling increases the distribution of ZSWIM4 in the nucleus of GIST cells, and which is important for its inhibition of KIT and BMAL1. In agreement with the results in vitro, the in vivo studies showed that ZSWIM4 deficiency increases the tumorigenesis of GISTs in KITV558A/WT mice. CONCLUSIONS: Taken together, our results revealed that the entry of ZSWIM4 to the nucleus is important for its inhibition of KIT and BMAL1, ultimately attenuating GIST tumorigenesis. The results provide a novel insight in the understanding of signal transduction in GISTs and lay strong theoretical basis for the advancement of GIST treatment.

4.
Abdom Radiol (NY) ; 2024 Jun 06.
Article in English | MEDLINE | ID: mdl-38842727

ABSTRACT

PURPOSE: This study aimed to develop and validate a computed tomography-based nomogram assessing visceral and subcutaneous adiposity for predicting outcomes in localized clear cell renal cell carcinoma (ccRCC). METHODS: A cohort of 364 patients with pathologically confirmed ccRCC participated in this retrospective study, with 254 patients assigned to the training set and 110 to the validation set (a 7:3 distribution ratio). The adipose score (AS) was generated using the least absolute shrinkage and selection operator Cox regression. Subsequently, a nomogram was constructed by integrating the clinical independent predictor with the AS to predict disease-free survival (DFS) in localized ccRCC after surgery. The performance of the nomogram was compared with the University of California, Los Angeles, Integrated Staging System (UISS), and the Stage, Size, Grade, and Necrosis (SSIGN) score. RESULTS: In both the training and validation cohorts, the nomogram exhibited superior discrimination compared to SSIGN and UISS (C-index: 0.897 vs. 0.781 vs. 0.776 in the training cohort, and 0.752 vs. 0.596 vs. 0.686 in the validation cohort; 5 year AUC: 0.907 vs. 0.805 vs. 0.820 in the training cohort, and 0.832 vs. 0.577 vs. 0.726 in the validation cohort). Decision curve analysis (DCA) revealed a superior net benefit across a wider range of threshold probabilities for predicting 5 year DFS compared to UISS and SSIGN scores. CONCLUSIONS: The developed prognostic nomogram demonstrated high accuracy and overall superior performance compared to existing prognostic models.

5.
Genome Biol ; 25(1): 149, 2024 06 06.
Article in English | MEDLINE | ID: mdl-38845006

ABSTRACT

Cancer is a complex disease composing systemic alterations in multiple scales. In this study, we develop the Tumor Multi-Omics pre-trained Network (TMO-Net) that integrates multi-omics pan-cancer datasets for model pre-training, facilitating cross-omics interactions and enabling joint representation learning and incomplete omics inference. This model enhances multi-omics sample representation and empowers various downstream oncology tasks with incomplete multi-omics datasets. By employing interpretable learning, we characterize the contributions of distinct omics features to clinical outcomes. The TMO-Net model serves as a versatile framework for cross-modal multi-omics learning in oncology, paving the way for tumor omics-specific foundation models.


Subject(s)
Neoplasms , Humans , Neoplasms/genetics , Genomics , Medical Oncology , Machine Learning , Multiomics
6.
J Med Internet Res ; 26: e58158, 2024 Jul 22.
Article in English | MEDLINE | ID: mdl-38833165

ABSTRACT

BACKGROUND: The efficacy of large language models (LLMs) in domain-specific medicine, particularly for managing complex diseases such as osteoarthritis (OA), remains largely unexplored. OBJECTIVE: This study focused on evaluating and enhancing the clinical capabilities and explainability of LLMs in specific domains, using OA management as a case study. METHODS: A domain-specific benchmark framework was developed to evaluate LLMs across a spectrum from domain-specific knowledge to clinical applications in real-world clinical scenarios. DocOA, a specialized LLM designed for OA management integrating retrieval-augmented generation and instructional prompts, was developed. It can identify the clinical evidence upon which its answers are based through retrieval-augmented generation, thereby demonstrating the explainability of those answers. The study compared the performance of GPT-3.5, GPT-4, and a specialized assistant, DocOA, using objective and human evaluations. RESULTS: Results showed that general LLMs such as GPT-3.5 and GPT-4 were less effective in the specialized domain of OA management, particularly in providing personalized treatment recommendations. However, DocOA showed significant improvements. CONCLUSIONS: This study introduces a novel benchmark framework that assesses the domain-specific abilities of LLMs in multiple aspects, highlights the limitations of generalized LLMs in clinical contexts, and demonstrates the potential of tailored approaches for developing domain-specific medical LLMs.


Subject(s)
Osteoarthritis , Humans , Osteoarthritis/therapy
7.
Med Image Anal ; 95: 103199, 2024 Jul.
Article in English | MEDLINE | ID: mdl-38759258

ABSTRACT

The accurate diagnosis on pathological subtypes for lung cancer is of significant importance for the follow-up treatments and prognosis managements. In this paper, we propose self-generating hybrid feature network (SGHF-Net) for accurately classifying lung cancer subtypes on computed tomography (CT) images. Inspired by studies stating that cross-scale associations exist in the image patterns between the same case's CT images and its pathological images, we innovatively developed a pathological feature synthetic module (PFSM), which quantitatively maps cross-modality associations through deep neural networks, to derive the "gold standard" information contained in the corresponding pathological images from CT images. Additionally, we designed a radiological feature extraction module (RFEM) to directly acquire CT image information and integrated it with the pathological priors under an effective feature fusion framework, enabling the entire classification model to generate more indicative and specific pathologically related features and eventually output more accurate predictions. The superiority of the proposed model lies in its ability to self-generate hybrid features that contain multi-modality image information based on a single-modality input. To evaluate the effectiveness, adaptability, and generalization ability of our model, we performed extensive experiments on a large-scale multi-center dataset (i.e., 829 cases from three hospitals) to compare our model and a series of state-of-the-art (SOTA) classification models. The experimental results demonstrated the superiority of our model for lung cancer subtypes classification with significant accuracy improvements in terms of accuracy (ACC), area under the curve (AUC), positive predictive value (PPV) and F1-score.


Subject(s)
Lung Neoplasms , Tomography, X-Ray Computed , Humans , Lung Neoplasms/diagnostic imaging , Lung Neoplasms/classification , Tomography, X-Ray Computed/methods , Neural Networks, Computer , Radiographic Image Interpretation, Computer-Assisted/methods , Algorithms
8.
Oncogene ; 43(27): 2078-2091, 2024 Jun.
Article in English | MEDLINE | ID: mdl-38760447

ABSTRACT

The aberrant activation of RAS/RAF/MEK/ERK signaling is important for KIT mutation-mediated tumorigenesis of gastrointestinal stromal tumor (GIST). In this study, we found that inhibition of RAF1 suppresses the activation of both wild-type KIT and primary KIT mutations in GIST, with primary KIT mutations showing greater sensitivity. This suggests a positive feedback loop between KIT and RAF1, wherein RAF1 facilitates KIT signaling. We further demonstrated that RAF1 associates with KIT and the kinase activity of RAF1 is necessary for its contribution to KIT activation. Accordingly, inhibition of RAF1 suppressed cell survival, proliferation, and cell cycle progression in vitro mediated by both wild-type KIT and primary KIT mutations. Inhibition of RAF1 in vivo suppressed GIST growth in a transgenic mouse model carrying germline KIT/V558A mutation, showing a similar treatment efficiency as imatinib, the first-line targeted therapeutic drug of GIST, while the combination use of imatinib and RAF1 inhibitor further suppressed tumor growth. Acquisition of drug-resistant secondary mutation of KIT is a major cause of treatment failure of GIST following targeted therapy. Like wild-type KIT and primary KIT mutations, inhibition of RAF1 suppressed the activation of secondary KIT mutation, and the cell survival, proliferation, cell cycle progression in vitro, and tumor growth in vivo mediated by secondary KIT mutation. However, the activation of secondary KIT mutation is less dependent on RAF1 compared with that of primary KIT mutations. Taken together, our results revealed that RAF1 facilitates KIT signaling and KIT mutation-mediated tumorigenesis of GIST, providing a rationale for further investigation into the use of RAF1 inhibitors alone or in combination with KIT inhibitor in the treatment of GIST, particularly in cases resistant to KIT inhibitors.


Subject(s)
Gastrointestinal Stromal Tumors , Proto-Oncogene Proteins c-kit , Proto-Oncogene Proteins c-raf , Signal Transduction , Gastrointestinal Stromal Tumors/genetics , Gastrointestinal Stromal Tumors/drug therapy , Gastrointestinal Stromal Tumors/pathology , Gastrointestinal Stromal Tumors/metabolism , Proto-Oncogene Proteins c-kit/genetics , Proto-Oncogene Proteins c-kit/metabolism , Animals , Proto-Oncogene Proteins c-raf/metabolism , Proto-Oncogene Proteins c-raf/genetics , Humans , Mice , Mice, Transgenic , Cell Proliferation , Cell Line, Tumor , Mutation , Imatinib Mesylate/pharmacology , Imatinib Mesylate/therapeutic use , Gastrointestinal Neoplasms/pathology , Gastrointestinal Neoplasms/drug therapy , Gastrointestinal Neoplasms/genetics , Gastrointestinal Neoplasms/metabolism
9.
Org Lett ; 26(22): 4773-4778, 2024 Jun 07.
Article in English | MEDLINE | ID: mdl-38780223

ABSTRACT

Gold-catalyzed cascade cyclization of diynes for the synthesis of previously unexplored C-N axially chiral N-arylbenzo[g]indoles was described. The transformation was achieved via a central-to-axial chirality conversion strategy. The chiral conversion exhibited high efficiency. Besides single C-N chiral axis, N-arylbenzo[g]indoles bearing both C-N and C-C chiral axes were also afforded. The title compound derived monophosphine ligand was prepared and was evaluated in Pd-catalyzed asymmetric allylic substitutions, showing excellent chiral induction ability.

10.
Mol Carcinog ; 63(7): 1334-1348, 2024 Jul.
Article in English | MEDLINE | ID: mdl-38629424

ABSTRACT

Gastrointestinal stromal tumors (GISTs) are predominately induced by KIT mutants. In this study, we found that four and a half LIM domains 2 (FHL2) was highly expressed in GISTs and KIT signaling dramatically increased FHL2 transcription while FHL2 inhibited KIT transcription. In addition, our results showed that FHL2 associated with KIT and increased the ubiquitination of both wild-type KIT and primary KIT mutants in GISTs, leading to decreased expression and activation of KIT although primary KIT mutants were less inhibited by FHL2 than wild-type KIT. In the animal experiments, loss of FHL2 expression in mice carrying germline KIT/V558A mutation which can develop GISTs resulted in increased tumor growth, but increased sensitivity of GISTs to imatinib treatment which is used as the first-line targeted therapy of GISTs, suggesting that FHL2 plays a role in the response of GISTs to KIT inhibitor. Unlike wild-type KIT and primary KIT mutants, we further found that FHL2 didn't alter the expression and activation of drug-resistant secondary KIT mutants. Taken together, our results indicated that FHL2 acts as the negative feedback of KIT signaling in GISTs while primary KIT mutants are less sensitive and secondary KIT mutants are resistant to the inhibition of FHL2.


Subject(s)
Gastrointestinal Stromal Tumors , LIM-Homeodomain Proteins , Muscle Proteins , Proto-Oncogene Proteins c-kit , Signal Transduction , Transcription Factors , Gastrointestinal Stromal Tumors/genetics , Gastrointestinal Stromal Tumors/pathology , Gastrointestinal Stromal Tumors/metabolism , Animals , Proto-Oncogene Proteins c-kit/genetics , Proto-Oncogene Proteins c-kit/metabolism , LIM-Homeodomain Proteins/genetics , LIM-Homeodomain Proteins/metabolism , Humans , Muscle Proteins/genetics , Muscle Proteins/metabolism , Mice , Transcription Factors/genetics , Transcription Factors/metabolism , Mutation , Carcinogenesis/genetics , Gene Expression Regulation, Neoplastic , Imatinib Mesylate/pharmacology , Gastrointestinal Neoplasms/genetics , Gastrointestinal Neoplasms/pathology , Gastrointestinal Neoplasms/metabolism , Cell Line, Tumor , Ubiquitination
11.
IEEE Trans Med Imaging ; PP2024 Apr 11.
Article in English | MEDLINE | ID: mdl-38602852

ABSTRACT

Adapting a medical image segmentation model to a new domain is important for improving its cross-domain transferability, and due to the expensive annotation process, Unsupervised Domain Adaptation (UDA) is appealing where only unlabeled images are needed for the adaptation. Existing UDA methods are mainly based on image or feature alignment with adversarial training for regularization, and they are limited by insufficient supervision in the target domain. In this paper, we propose an enhanced Filtered Pseudo Label (FPL+)-based UDA method for 3D medical image segmentation. It first uses cross-domain data augmentation to translate labeled images in the source domain to a dual-domain training set consisting of a pseudo source-domain set and a pseudo target-domain set. To leverage the dual-domain augmented images to train a pseudo label generator, domain-specific batch normalization layers are used to deal with the domain shift while learn the domain-invariant structure features, generating high-quality pseudo labels for target-domain images. We then combine labeled source-domain images and target-domain images with pseudo labels to train a final segmentor, where image-level weighting based on uncertainty estimation and pixel-level weighting based on dual-domain consensus are proposed to mitigate the adverse effect of noisy pseudo labels. Experiments on three public multi-modal datasets for Vestibular Schwannoma, brain tumor and whole heart segmentation show that our method surpassed ten state-of-the-art UDA methods, and it even achieved better results than fully supervised learning in the target domain in some cases.

12.
NPJ Precis Oncol ; 8(1): 76, 2024 Mar 27.
Article in English | MEDLINE | ID: mdl-38538739

ABSTRACT

Diffuse large B cell lymphoma (DLBCL) is an aggressive blood cancer known for its rapid progression and high incidence. The growing use of immunohistochemistry (IHC) has significantly contributed to the detailed cell characterization, thereby playing a crucial role in guiding treatment strategies for DLBCL. In this study, we developed an AI-based image analysis approach for assessing PD-L1 expression in DLBCL patients. PD-L1 expression represents as a major biomarker for screening patients who can benefit from targeted immunotherapy interventions. In particular, we performed large-scale cell annotations in IHC slides, encompassing over 5101 tissue regions and 146,439 live cells. Extensive experiments in primary and validation cohorts demonstrated the defined quantitative rule helped overcome the difficulty of identifying specific cell types. In assessing data obtained from fine needle biopsies, experiments revealed that there was a higher level of agreement in the quantitative results between Artificial Intelligence (AI) algorithms and pathologists, as well as among pathologists themselves, in comparison to the data obtained from surgical specimens. We highlight that the AI-enabled analytics enhance the objectivity and interpretability of PD-L1 quantification to improve the targeted immunotherapy development in DLBCL patients.

13.
BMJ Open ; 14(1): e073024, 2024 01 04.
Article in English | MEDLINE | ID: mdl-38176870

ABSTRACT

INTRODUCTION: It is encouraged to estimate the effectiveness of components within the enhanced recovery after surgery (ERAS) protocol through patient-reported outcomes, alongside doctor-reported outcomes and length of hospital stay. At present, studies on the contributions of optimal anaesthetic drugs within the ERAS protocol to patient-reported and doctor-reported outcomes are limited. Therefore, this study aims to pragmatically evaluate the effectiveness and safety of general anaesthesia (GA) with remimazolam tosilate within the ERAS protocol on intraoperative haemodynamics and postoperative recovery in adults undergoing elective surgeries, compared with propofol. METHODS AND ANALYSIS: This study is a single-centre, randomised, blinded, positive-controlled, pragmatic clinical trial. A total of 900 patients, aged ≥18 years old, scheduled for an elective surgical procedure under GA will be included. Patients will be randomised in a 1:1 ratio to the remimazolam group (the GA with remimazolam tosilate within the ERAS protocol group) or propofol group (the GA with propofol within the ERAS protocol group), stratified by general surgery, thoracic surgery and other surgeries (including urological surgery and otolaryngology surgery). The primary outcomes include the 24-hour postoperative quality of recovery-40 score and the rate of intraoperative hypotension. Secondary endpoints include the rate of sedative hypotension requiring treatment, the haemodynamic profiles, the 72-hour postoperative quality of recovery-40 score, the functional anaesthetic capability, adverse events and complications, quality of life within 3 months as well as economic health outcomes. ETHICS AND DISSEMINATION: This study protocol has been approved by the ethics committee of Guangdong Provincial People's Hospital (KY-H-2022-005-03-08). Dissemination plans will be presented at scientific meetings and in scientific publications. TRIAL REGISTRATION NUMBER: ChiCTR2200062520.


Subject(s)
Anesthetics , Hypotension , Propofol , Adolescent , Adult , Humans , Anesthesia, General/adverse effects , Hemodynamics , Hypotension/etiology , Postoperative Complications/prevention & control , Postoperative Complications/etiology , Quality of Life , Randomized Controlled Trials as Topic , Pragmatic Clinical Trials as Topic
14.
Mol Biol Rep ; 51(1): 98, 2024 Jan 11.
Article in English | MEDLINE | ID: mdl-38206538

ABSTRACT

BACKGROUND: Mutations in the receptor tyrosine kinase KIT are the main cause of gastrointestinal stromal tumor (GIST), and the KIT mutants mediated PI3 kinase activation plays a key role in the tumorigenesis of GIST. In this study, we aimed to block PI3 kinase activation by cell-permeable peptide and investigate its possible application in the treatment of GIST. METHODS AND RESULTS: We designed cell-permeable peptides based on the binding domain of PI3 kinase subunit p85 to KIT or PI3 kinase subunit p110, respectively, in order to compete for the binding between p85 and KIT or p110 and therefore inhibit the activation of PI3 kinases mediated by KIT. The results showed that the peptide can penetrate the cells, and inhibit the activation of PI3 kinases, leading to reduced cell survival and cell proliferation mediated by KIT mutants in vitro. Treatment of mice carrying germline KIT/V558A mutation, which can develop GIST, with the peptide that can compete for the binding between p85 and p110, led to reduced tumorigenesis of GIST. The peptide can further enhance the inhibition of the tumor growth by imatinib which is used as the first line targeted therapy of GIST. CONCLUSIONS: Our results showed that cell-permeable PI3 kinase competitive peptide can inhibit KIT-mediated PI3 kinase activation and tumorigenesis of GIST, providing a rationale to further test the peptide in the treatment of GIST and even other tumors with over-activation of PI3 kinases.


Subject(s)
Gastrointestinal Stromal Tumors , Phosphatidylinositol 3-Kinases , Animals , Mice , Phosphatidylinositol 3-Kinases/genetics , Gastrointestinal Stromal Tumors/drug therapy , Gastrointestinal Stromal Tumors/genetics , Carcinogenesis/genetics , Cell Transformation, Neoplastic , Phosphatidylinositol 3-Kinase , Peptides/pharmacology
15.
IEEE Trans Med Imaging ; 43(1): 416-426, 2024 Jan.
Article in English | MEDLINE | ID: mdl-37651492

ABSTRACT

Deep learning methods are often hampered by issues such as data imbalance and data-hungry. In medical imaging, malignant or rare diseases are frequently of minority classes in the dataset, featured by diversified distribution. Besides that, insufficient labels and unseen cases also present conundrums for training on the minority classes. To confront the stated problems, we propose a novel Hierarchical-instance Contrastive Learning (HCLe) method for minority detection by only involving data from the majority class in the training stage. To tackle inconsistent intra-class distribution in majority classes, our method introduces two branches, where the first branch employs an auto-encoder network augmented with three constraint functions to effectively extract image-level features, and the second branch designs a novel contrastive learning network by taking into account the consistency of features among hierarchical samples from majority classes. The proposed method is further refined with a diverse mini-batch strategy, enabling the identification of minority classes under multiple conditions. Extensive experiments have been conducted to evaluate the proposed method on three datasets of different diseases and modalities. The experimental results show that the proposed method outperforms the state-of-the-art methods.

16.
IEEE Trans Med Imaging ; 43(1): 175-189, 2024 Jan.
Article in English | MEDLINE | ID: mdl-37440388

ABSTRACT

Deep neural networks typically require accurate and a large number of annotations to achieve outstanding performance in medical image segmentation. One-shot and weakly-supervised learning are promising research directions that reduce labeling effort by learning a new class from only one annotated image and using coarse labels instead, respectively. In this work, we present an innovative framework for 3D medical image segmentation with one-shot and weakly-supervised settings. Firstly a propagation-reconstruction network is proposed to propagate scribbles from one annotated volume to unlabeled 3D images based on the assumption that anatomical patterns in different human bodies are similar. Then a multi-level similarity denoising module is designed to refine the scribbles based on embeddings from anatomical- to pixel-level. After expanding the scribbles to pseudo masks, we observe the miss-classified voxels mainly occur at the border region and propose to extract self-support prototypes for the specific refinement. Based on these weakly-supervised segmentation results, we further train a segmentation model for the new class with the noisy label training strategy. Experiments on three CT and one MRI datasets show the proposed method obtains significant improvement over the state-of-the-art methods and performs robustly even under severe class imbalance and low contrast. Code is publicly available at https://github.com/LWHYC/OneShot_WeaklySeg.


Subject(s)
Image Processing, Computer-Assisted , Neural Networks, Computer , Humans , Supervised Machine Learning
17.
IEEE Trans Med Imaging ; 43(1): 405-415, 2024 Jan.
Article in English | MEDLINE | ID: mdl-37594875

ABSTRACT

This paper investigates how to effectively mine contextual information among sequential images and jointly model them in medical imaging tasks. Different from state-of-the-art methods that model sequential correlations via point-wise token encoding, this paper develops a novel hierarchical pattern-aware tokenization strategy. It handles distinct visual patterns independently and hierarchically, which not only ensures the full flexibility of attention aggregation under different pattern representations but also preserves both local and global information simultaneously. Based on this strategy, we propose a Pattern-Aware Transformer (PATrans) featuring a global-local dual-path pattern-aware cross-attention mechanism to achieve hierarchical pattern matching and propagation among sequential images. Furthermore, PATrans is plug-and-play and can be seamlessly integrated into various backbone networks for diverse downstream sequence modeling tasks. We demonstrate its general application paradigm across four domains and five benchmarks in video object detection and 3D volumetric semantic segmentation tasks, respectively. Impressively, PATrans sets new state-of-the-art across all these benchmarks, i.e., CVC-Video (92.3% detection F1), ASU-Mayo (99.1% localization F1), Lung Tumor (78.59% DSC), Nasopharynx Tumor (75.50% DSC), and Kidney Tumor (87.53% DSC). Codes and models are available at https://github.com/GGaoxiang/PATrans.


Subject(s)
Lung Neoplasms , Humans , Semantics
18.
Med Image Anal ; 91: 102996, 2024 Jan.
Article in English | MEDLINE | ID: mdl-37857067

ABSTRACT

This article discusses the opportunities, applications and future directions of large-scale pretrained models, i.e., foundation models, which promise to significantly improve the analysis of medical images. Medical foundation models have immense potential in solving a wide range of downstream tasks, as they can help to accelerate the development of accurate and robust models, reduce the dependence on large amounts of labeled data, preserve the privacy and confidentiality of patient data. Specifically, we illustrate the "spectrum" of medical foundation models, ranging from general imaging models, modality-specific models, to organ/task-specific models, and highlight their challenges, opportunities and applications. We also discuss how foundation models can be leveraged in downstream medical tasks to enhance the accuracy and efficiency of medical image analysis, leading to more precise diagnosis and treatment decisions.


Subject(s)
Diagnostic Imaging , Humans , Diagnostic Imaging/methods , Forecasting
19.
Med Image Anal ; 91: 102999, 2024 Jan.
Article in English | MEDLINE | ID: mdl-37862866

ABSTRACT

Coronary CT angiography (CCTA) is an effective and non-invasive method for coronary artery disease diagnosis. Extracting an accurate coronary artery tree from CCTA image is essential for centerline extraction, plaque detection, and stenosis quantification. In practice, data quality varies. Sometimes, the arteries and veins have similar intensities and locate closely, which may confuse segmentation algorithms, even deep learning based ones, to obtain accurate arteries. However, it is not always feasible to re-scan the patient for better image quality. In this paper, we propose an artery and vein disentanglement network (AVDNet) for robust and accurate segmentation by incorporating the coronary vein into the segmentation task. This is the first work to segment coronary artery and vein at the same time. The AVDNet consists of an image based vessel recognition network (IVRN) and a topology based vessel refinement network (TVRN). IVRN learns to segment the arteries and veins, while TVRN learns to correct the segmentation errors based on topology consistency. We also design a novel inverse distance weighted dice (IDD) loss function to recover more thin vessel branches and preserve the vascular boundaries. Extensive experiments are conducted on a multi-center dataset of 700 patients. Quantitative and qualitative results demonstrate the effectiveness of the proposed method by comparing it with state-of-the-art methods and different variants. Prediction results of the AVDNet on the Automated Segmentation of Coronary Artery Challenge dataset are avaliabel at https://github.com/WennyJJ/Coronary-Artery-Vein-Segmentation for follow-up research.


Subject(s)
Algorithms , Coronary Vessels , Humans , Coronary Vessels/diagnostic imaging , Tomography, X-Ray Computed/methods , Coronary Angiography/methods , Computed Tomography Angiography/methods , Image Processing, Computer-Assisted/methods
20.
Article in English | MEDLINE | ID: mdl-38083011

ABSTRACT

Accurate liver tumor segmentation is a prerequisite for data-driven tumor analysis. Multiphase computed tomography (CT) with extensive liver tumor characteristics is typically used as the most crucial diagnostic basis. However, the large variations in contrast, texture, and tumor structure between CT phases limit the generalization capabilities of the associated segmentation algorithms. Inadequate feature integration across phases might also lead to a performance decrease. To address these issues, we present a domain-adversarial transformer (DA-Tran) network for segmenting liver tumors from multiphase CT images. A DA module is designed to generate domain-adapted feature maps from the non-contrast-enhanced (NC) phase, arterial (ART) phase, portal venous (PV) phase, and delay phase (DP) images. These domain-adapted feature maps are then combined with 3D transformer blocks to capture patch-structured similarity and global context attention. The experimental findings show that DA-Tran produces cutting-edge tumor segmentation outcomes, making it an ideal candidate for this co-segmentation challenge.


Subject(s)
Liver Neoplasms , Humans , Liver Neoplasms/diagnostic imaging , Algorithms , Arteries , Electric Power Supplies , Generalization, Psychological
SELECTION OF CITATIONS
SEARCH DETAIL