TransUNet: Rethinking the U-Net architecture design for medical image segmentation through the lens of transformers.

Medical image segmentation is crucial for healthcare, yet convolution-based methods like U-Net face limitations in modeling long-range dependencies. To address this, Transformers designed for sequence-to-sequence predictions have been integrated into medical image segmentation. However, a comprehensive understanding of Transformers' self-attention in U-Net components is lacking. TransUNet, first introduced in 2021, is widely recognized as one of the first models to integrate Transformer into medical image analysis. In this study, we present the versatile framework of TransUNet that encapsulates Transformers' self-attention into two key modules: (1) a Transformer encoder tokenizing image patches from a convolution neural network (CNN) feature map, facilitating global context extraction, and (2) a Transformer decoder refining candidate regions through cross-attention between proposals and U-Net features. These modules can be flexibly inserted into the U-Net backbone, resulting in three configurations: Encoder-only, Decoder-only, and Encoder+Decoder. TransUNet provides a library encompassing both 2D and 3D implementations, enabling users to easily tailor the chosen architecture. Our findings highlight the encoder's efficacy in modeling interactions among multiple abdominal organs and the decoder's strength in handling small targets like tumors. It excels in diverse medical applications, such as multi-organ segmentation, pancreatic tumor segmentation, and hepatic vessel segmentation. Notably, our TransUNet achieves a significant average Dice improvement of 1.06% and 4.30% for multi-organ segmentation and pancreatic tumor segmentation, respectively, when compared to the highly competitive nn-UNet, and surpasses the top-1 solution in the BrasTS2021 challenge. 2D/3D Code and models are available at https://github.com/Beckschen/TransUNet and https://github.com/Beckschen/TransUNet-3D, respectively.

RemixFormer++: A Multi-modal Transformer Model for Precision Skin Tumor Differential Diagnosis with Memory-efficient Attention.

Diagnosing malignant skin tumors accurately at an early stage can be challenging due to ambiguous and even confusing visual characteristics displayed by various categories of skin tumors. To improve diagnosis precision, all available clinical data from multiple sources, particularly clinical images, dermoscopy images, and medical history, could be considered. Aligning with clinical practice, we propose a novel Transformer model, named Remix-Former++ that consists of a clinical image branch, a dermoscopy image branch, and a metadata branch. Given the unique characteristics inherent in clinical and dermoscopy images, specialized attention strategies are adopted for each type. Clinical images are processed through a top-down architecture, capturing both localized lesion details and global contextual information. Conversely, dermoscopy images undergo a bottom-up processing with two-level hierarchical encoders, designed to pinpoint fine-grained structural and textural features. A dedicated metadata branch seamlessly integrates non-visual information by encoding relevant patient data. Fusing features from three branches substantially boosts disease classification accuracy. RemixFormer++ demonstrates exceptional performance on four single-modality datasets (PAD-UFES-20, ISIC 2017/2018/2019). Compared with the previous best method using a public multi-modal Derm7pt dataset, we achieved an absolute 5.3% increase in averaged F1 and 1.2% in accuracy for the classification of five skin tumors. Furthermore, using a large-scale in-house dataset of 10,351 patients with the twelve most common skin tumors, our method obtained an overall classification accuracy of 92.6%. These promising results, on par or better with the performance of 191 dermatologists through a comprehensive reader study, evidently imply the potential clinical usability of our method.

Multiple lymph nodes enlargement and fever as main manifestations of nocardiosis in immunocompetent individuals: Two case reports.

Nocardia farcinica is an aerobic gram-positive bacterium that is pathogenic to humans. It usually causes local and adjacent tissues' diseases at the entry of infection (most commonly occur in the lungs, skin, or central nervous system), which can also spread to other organs through the bloodstream such as joints, kidneys, and liver. However, these infections are often seen as opportunistic that occur in immunocompromised patients. Here, we report for the first time two immunocompetent patients lacking evidence of local infections, with multiple lymph node enlargements and fever as main clinical manifestations, finally diagnosed as nocardiosis by Metagenomic Next-Generation Sequencing testing (mNGS) from formalin-fixed and paraffin-embedded (FFPE) lymph node tissue, after all the other standard tests were negative. Both patients recovered after receiving anti-nocardia therapies. These two cases indicates that in healthy population, there may be more potential nocardia infections than we expected. Multiple lymph node enlargements and fever suggest a possibility of nocardiosis, especially in patients with fever of unknown origin (FUO). mNGS detection from FFPE lymph node tissue is an accurate, reliable and traceable method for diagnosis of nocardiosis.

Treatment outcomes and prognostic factors of the apical barrier technique with premixed calcium silicate-based putty in necrotic permanent teeth with open apices: a retrospective cohort study with up to six years follow-up.

OBJECTIVES: To evaluate treatment outcomes of the apical barrier technique with premixed calcium silicate-based putty for treating necrotic permanent teeth with open apices and to identify prognostic factors. MATERIALS AND METHODS: Permanent teeth with necrotic pulps and open apices treated by the apical barrier technique with premixed calcium silicate-based putty, with a minimum follow-up of 12 months, were included. Treatment outcomes were based on clinical signs, symptoms, and radiographic evaluation. The treatment outcome was dichotomized into success or failure according to strict and loose criteria. The chi-square test (or Fisher's exact test) and multiple logistic regression analysis were used to evaluate possible prognostic factors associated with treatment outcomes. RESULTS: Seventy-four teeth with a follow-up time of 12-72 months (mean, 25.74 ± 14.36 months) were included in the final evaluation. The success rate was 97.30% using the loose criteria and 66.22% using the strict criteria. Multiple logistic regression analysis indicated that the size of pre-operative periapical lesion (≥ 5 mm) (odds ratio [OR]: 18.96; P = 0.0153) and root canal underfilling (OR: 8.341; P = 0.0448) were significant predictors for treatment failure under the strict criteria. CONCLUSION: The apical barrier technique with premixed calcium silicate-based putty is a highly successful procedure for treating necrotic permanent teeth with open apices after an observation period of up to 6 years. Treatment success under the strict criteria is primarily affected by the size of the pre-operative periapical lesion and the apical extent of root-filling. CLINICAL RELEVANCE: Careful case selection and ensuring adequate root filling quality are essential to the successful outcome of the apical barrier technique with premixed calcium silicate-based putty.

Accurate Airway Tree Segmentation in CT Scans via Anatomy-aware Multi-class Segmentation and Topology-guided Iterative Learning.

Intrathoracic airway segmentation in computed tomography is a prerequisite for various respiratory disease analyses such as chronic obstructive pulmonary disease, asthma and lung cancer. Due to the low imaging contrast and noises execrated at peripheral branches, the topological-complexity and the intra-class imbalance of airway tree, it remains challenging for deep learning-based methods to segment the complete airway tree (on extracting deeper branches). Unlike other organs with simpler shapes or topology, the airway's complex tree structure imposes an unbearable burden to generate the "ground truth" label (up to 7 or 3 hours of manual or semi-automatic annotation per case). Most of the existing airway datasets are incompletely labeled/annotated, thus limiting the completeness of computer-segmented airway. In this paper, we propose a new anatomy-aware multi-class airway segmentation method enhanced by topology-guided iterative self-learning. Based on the natural airway anatomy, we formulate a simple yet highly effective anatomy-aware multi-class segmentation task to intuitively handle the severe intra-class imbalance of the airway. To solve the incomplete labeling issue, we propose a tailored iterative self-learning scheme to segment toward the complete airway tree. For generating pseudo-labels to achieve higher sensitivity (while retaining similar specificity), we introduce a novel breakage attention map and design a topology-guided pseudo-label refinement method by iteratively connecting breaking branches commonly existed from initial pseudo-labels. Extensive experiments have been conducted on four datasets including two public challenges. The proposed method achieves the top performance in both EXACT'09 challenge using average score and ATM'22 challenge on weighted average score. In a public BAS dataset and a private lung cancer dataset, our method significantly improves previous leading approaches by extracting at least (absolute) 6.1% more detected tree length and 5.2% more tree branches, while maintaining comparable precision.

Exosomal circHIF1A derived from hypoxic-induced carcinoma-associated fibroblasts promotes hepatocellular carcinoma cell malignant phenotypes and immune escape.

Hypoxia is a hallmark of solid tumors. Cancer-associated fibroblasts (CAFs) are an important component of the tumor microenvironment, and CAF-derived exosomes are involved in cancer genesis and progression. Here, this work investigated the role and mechanism of exosomal circHIF1A derived from hypoxia-induced CAFs in hepatocellular carcinoma (HCC) tumorigenesis. CAFs isolated from fresh HCC tissues were incubated in normoxia or hypoxia condition (N/CAFs or H/CAFs), and then the exosomes from N/CAFs or H/CAFs were isolated for functional analysis. Cell proliferation, migration and invasion were analyzed by cell counting kit-8, colony formation, and transwell assays. Immune evasion was evaluated by measuring the cytotoxicity and viability of CD8+T cells. qRT-PCR and western blotting analyses were used for the level measurement of genes and proteins. The binding between Hu antigen R (HuR) and circHIF1A or Programmed death ligand 1 (PD-L1) was analyzed by RNA immunoprecipitation assay. Functionally, we found that CAFs, especially CAFs under hypoxic stress (H/CAFs), promoted the proliferation, migration, invasion and EMT progression in HCC cells, as well as induced immune escape by suppressing CD8+T cell cytotoxicity and activity in an exosome-dependent manner. H/CAFs-derived exosomes showed highly expressed circHIF1A, and could secrete circHIF1A into HCC cells via exosomes. The oncogenic effects of H/CAFs-secreted exosomes were abolished by circHIF1A knockdown. Mechanistically, circHIF1A interacted with HuR to stabilize PD-L1 expression in HCC cells. Meanwhile, circHIF1A silencing suppressed HCC cell proliferation, mobility and immune escape by regulating PD-L1 expression. In all, exosomal circHIF1A derived from hypoxic-induced CAFs promoted the proliferation, migration, invasion, EMT progression and immune escape in HCC cells by up-regulating PD-L1 expression in a HuR-dependent manner.

Surface Topography Induces and Orients Nematic Swarms of Active Filaments: Considerations for Lab-On-A-Chip Devices.

Surface-bound molecular motors can drive the collective motion of cytoskeletal filaments in the form of nematic bands and polar flocks in reconstituted gliding assays. Although these "swarming transitions" are an emergent property of active filament collisions, they can be controlled and guided by tuning the surface chemistry or topography of the substrate. To date, the impact of surface topography on collective motion in active nematics is only partially understood, with most experimental studies focusing on the escape of a single filament from etched channels. Since the late 1990s, significant progress has been made to utilize the nonequilibrium properties of active filaments and create a range of functional nanodevices relevant to biosensing and parallel computation; however, the complexity of these swarming transitions presents a challenge when attempting to increase filament surface concentrations. In this work, we etch shallow, linear trenches into glass substrates to induce the formation of swarming nematic bands and investigate the mechanisms by which surface topography regulates the two-dimensional (2D) collective motion of driven filamentous actin (F-actin). We demonstrate that nematic swarms only appear at intermediate trench spacings and vanish if the trenches are made too narrow, wide, or tortuous. To rationalize these results, we simulate the F-actin as self-propelled, semiflexible chains subject to a soft, spatially modulated potential that encodes the energetic cost of bending a filament along the edge of a trench. In our model, we hypothesize that an individual filament experiences a penalty when its projected end-to-end distance is smaller than the trench spacing ("bending and turning"). However, chains that span the channel width glide above the trenches in a force- and torque-free manner ("crowd-surfing"). Our simulations demonstrate that collections of filaments form nematic bands only at intermediate trench spacings, consistent with our experimental findings.

A Colorectal Coordinate-Driven Method for Colorectum and Colorectal Cancer Segmentation in Conventional CT Scans.

Automated colorectal cancer (CRC) segmentation in medical imaging is the key to achieving automation of CRC detection, staging, and treatment response monitoring. Compared with magnetic resonance imaging (MRI) and computed tomography colonography (CTC), conventional computed tomography (CT) has enormous potential because of its broad implementation, superiority for the hollow viscera (colon), and convenience without needing bowel preparation. However, the segmentation of CRC in conventional CT is more challenging due to the difficulties presenting with the unprepared bowel, such as distinguishing the colorectum from other structures with similar appearance and distinguishing the CRC from the contents of the colorectum. To tackle these challenges, we introduce DeepCRC-SL, the first automated segmentation algorithm for CRC and colorectum in conventional contrast-enhanced CT scans. We propose a topology-aware deep learning-based approach, which builds a novel 1-D colorectal coordinate system and encodes each voxel of the colorectum with a relative position along the coordinate system. We then induce an auxiliary regression task to predict the colorectal coordinate value of each voxel, aiming to integrate global topology into the segmentation network and thus improve the colorectum's continuity. Self-attention layers are utilized to capture global contexts for the coordinate regression task and enhance the ability to differentiate CRC and colorectum tissues. Moreover, a coordinate-driven self-learning (SL) strategy is introduced to leverage a large amount of unlabeled data to improve segmentation performance. We validate the proposed approach on a dataset including 227 labeled and 585 unlabeled CRC cases by fivefold cross-validation. Experimental results demonstrate that our method outperforms some recent related segmentation methods and achieves the segmentation accuracy in DSC for CRC of 0.669 and colorectum of 0.892, reaching to the performance (at 0.639 and 0.890, respectively) of a medical resident with two years of specialized CRC imaging fellowship.

Establishment and verification of a prognostic model based on coagulation and fibrinolysis-related genes in hepatocellular carcinoma.

BACKGROUND: Studies have shown that coagulation and fibrinolysis (CFR) are correlated with Hepatocellular carcinoma (HCC) progression and prognosis. We aim to build a model based on CFR-correlated genes for risk assessment and prediction of HCC patient. METHODS: HCC samples were selected from The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) databases respectively. The Molecular Signatures Database (MSigDB) was used to select the CFR genes. RiskScore model were established by single sample gene set enrichment analysis (ssGSEA), weighted correlation network analysis (WGCNA), multivariate Cox regression analysis, LASSO regression analysis. RESULTS: PCDH17, PGF, PDE2A, FAM110D, FSCN1, FBLN5 were selected as the key genes and designed a RiskScore model. Those key genes were Differential expressions in HCC cell and patients. Overexpression PDE2A inhibited HCC cell migration and invasion. The higher the RiskScore, the lower the probability of survival. The model has high AUC values in the first, third and fifth year prediction curves, indicating that the model has strong prediction performance. The difference analysis of clinicopathological features found that a great proportion of high clinicopathological grade samples showed higher RiskScore. RiskScore were positively correlated with immune scores and TIDE scores. High levels of immune checkpoints and immunomodulators were observed in high RiskScore group. High RiskScore groups may benefit greatly from taking traditional chemotherapy drugs. CONCLUSIONS: We screened CFR related genes to design a RiskScore model, which could accurately evaluate the prognosis and survival status of HCC patients, providing certain value for optimizing the clinical treatment of cancer in the future.

CD200+ fibroblasts form a pro-resolving mesenchymal network in arthritis.

Fibroblasts are important regulators of inflammation, but whether fibroblasts change phenotype during resolution of inflammation is not clear. Here we use positron emission tomography to detect fibroblast activation protein (FAP) as a means to visualize fibroblast activation in vivo during inflammation in humans. While tracer accumulation is high in active arthritis, it decreases after tumor necrosis factor and interleukin-17A inhibition. Biopsy-based single-cell RNA-sequencing analyses in experimental arthritis show that FAP signal reduction reflects a phenotypic switch from pro-inflammatory MMP3+/IL6+ fibroblasts (high FAP internalization) to pro-resolving CD200+DKK3+ fibroblasts (low FAP internalization). Spatial transcriptomics of human joints indicates that pro-resolving niches of CD200+DKK3+ fibroblasts cluster with type 2 innate lymphoid cells, whereas MMP3+/IL6+ fibroblasts colocalize with inflammatory immune cells. CD200+DKK3+ fibroblasts stabilized the type 2 innate lymphoid cell phenotype and induced resolution of arthritis via CD200-CD200R1 signaling. Taken together, these data suggest a dynamic molecular regulation of the mesenchymal compartment during resolution of inflammation.

Exploring the Application of Large-Scale Pre-Trained Models on Adverse Weather Removal.

Image restoration under adverse weather conditions (e.g., rain, snow, and haze) is a fundamental computer vision problem that has important implications for various downstream applications. Distinct from early methods that are specially designed for specific types of weather, recent works tend to simultaneously remove various adverse weather effects based on either spatial feature representation learning or semantic information embedding. Inspired by various successful applications incorporating large-scale pre-trained models (e.g., CLIP), in this paper, we explore their potential benefits for leveraging large-scale pre-trained models in this task based on both spatial feature representation learning and semantic information embedding aspects: 1) spatial feature representation learning, we design a Spatially Adaptive Residual (SAR) encoder to adaptively extract degraded areas. To facilitate training of this model, we propose a Soft Residual Distillation (CLIP-SRD) strategy to transfer spatial knowledge from CLIP between clean and adverse weather images; 2) semantic information embedding, we propose a CLIP Weather Prior (CWP) embedding module to enable the network to adaptively respond to different weather conditions. This module integrates the sample-specific weather priors extracted by the CLIP image encoder with the distribution-specific information (as learned by a set of parameters) and embeds these elements using a cross-attention mechanism. Extensive experiments demonstrate that our proposed method can achieve state-of-the-art performance under various and severe adverse weather conditions. The code will be made available.

LViT: Language Meets Vision Transformer in Medical Image Segmentation.

Deep learning has been widely used in medical image segmentation and other aspects. However, the performance of existing medical image segmentation models has been limited by the challenge of obtaining sufficient high-quality labeled data due to the prohibitive data annotation cost. To alleviate this limitation, we propose a new text-augmented medical image segmentation model LViT (Language meets Vision Transformer). In our LViT model, medical text annotation is incorporated to compensate for the quality deficiency in image data. In addition, the text information can guide to generate pseudo labels of improved quality in the semi-supervised learning. We also propose an Exponential Pseudo label Iteration mechanism (EPI) to help the Pixel-Level Attention Module (PLAM) preserve local image features in semi-supervised LViT setting. In our model, LV (Language-Vision) loss is designed to supervise the training of unlabeled images using text information directly. For evaluation, we construct three multimodal medical segmentation datasets (image + text) containing X-rays and CT images. Experimental results show that our proposed LViT has superior segmentation performance in both fully-supervised and semi-supervised setting. The code and datasets are available at https://github.com/HUANGLIZI/LViT.

Large-scale pancreatic cancer detection via non-contrast CT and deep learning.

Pancreatic ductal adenocarcinoma (PDAC), the most deadly solid malignancy, is typically detected late and at an inoperable stage. Early or incidental detection is associated with prolonged survival, but screening asymptomatic individuals for PDAC using a single test remains unfeasible due to the low prevalence and potential harms of false positives. Non-contrast computed tomography (CT), routinely performed for clinical indications, offers the potential for large-scale screening, however, identification of PDAC using non-contrast CT has long been considered impossible. Here, we develop a deep learning approach, pancreatic cancer detection with artificial intelligence (PANDA), that can detect and classify pancreatic lesions with high accuracy via non-contrast CT. PANDA is trained on a dataset of 3,208 patients from a single center. PANDA achieves an area under the receiver operating characteristic curve (AUC) of 0.986-0.996 for lesion detection in a multicenter validation involving 6,239 patients across 10 centers, outperforms the mean radiologist performance by 34.1% in sensitivity and 6.3% in specificity for PDAC identification, and achieves a sensitivity of 92.9% and specificity of 99.9% for lesion detection in a real-world multi-scenario validation consisting of 20,530 consecutive patients. Notably, PANDA utilized with non-contrast CT shows non-inferiority to radiology reports (using contrast-enhanced CT) in the differentiation of common pancreatic lesion subtypes. PANDA could potentially serve as a new tool for large-scale pancreatic cancer screening.

Steatosis Quantification on Ultrasound Images by a Deep Learning Algorithm on Patients Undergoing Weight Changes.

INTRODUCTION: A deep learning algorithm to quantify steatosis from ultrasound images may change a subjective diagnosis to objective quantification. We evaluate this algorithm in patients with weight changes. MATERIALS AND METHODS: Patients (N = 101) who experienced weight changes ≥ 5% were selected for the study, using serial ultrasound studies retrospectively collected from 2013 to 2021. After applying our exclusion criteria, 74 patients from 239 studies were included. We classified images into four scanning views and applied the algorithm. Mean values from 3-5 images in each group were used for the results and correlated against weight changes. RESULTS: Images from the left lobe (G1) in 45 patients, right intercostal view (G2) in 67 patients, and subcostal view (G4) in 46 patients were collected. In a head-to-head comparison, G1 versus G2 or G2 versus G4 views showed identical steatosis scores (R2 > 0.86, p < 0.001). The body weight and steatosis scores were significantly correlated (R2 = 0.62, p < 0.001). Significant differences in steatosis scores between the highest and lowest body weight timepoints were found (p < 0.001). Men showed a higher liver steatosis/BMI ratio than women (p = 0.026). CONCLUSIONS: The best scanning conditions are 3-5 images from the right intercostal view. The algorithm objectively quantified liver steatosis, which correlated with body weight changes and gender.

Temperature Resistance Properties of Unidirectional Laminated Cf/SiC-Al Prepared by PIP and Vacuum Pressure Infiltration.

Material used for aero-engine fan blade requires excellent mechanical properties at high temperature (300 °C). Continuous carbon-fiber-reinforced silicon carbide ceramic matrix composites (Cf/SiC) are necessary candidates in this field, possessing low density, high strength, high modulus, and excellent high-temperature resistance. However, during the preparation process of Cf/SiC, there were inevitably residual pores and defects inside, resulting in insufficient compressive strength and reliability. The vacuum pressure melting infiltration process was used to infiltrate low melting point and high wettability aluminum alloys into the porous Cf/SiC composite material prepared by the precursor impregnation cracking process, repairing the residual pore defects inside the body. The porosity of porous Cf/SiC decreased from 49.65% to 5.1% after aluminum alloy repair and strengthening. The mechanical properties of Cf/SiC-Al composite materials strengthened by aluminum alloy repair after heat treatment were studied. The tensile strength of the as-prepared Cf/SiC-Al was 166 ± 10 MPa, which were degraded by 13~22% after heat treatment. The nonlinear sections of stress-displacement curve of as-treated samples were shorter than that of as-prepared sample. The hardness of aluminum alloy matrix after 300 °C 1 h heat treatment was 58 Hv, which was not obviously reduced compared with the sample without heat treatment. The vacuum infiltration of aluminum alloy is expected to have guiding significance for repairing and strengthening internal defects in ceramic matrix composites.

Evaluating the efficacy of endodontic microsurgery for teeth with an undeveloped root apex and periapical periodontitis after nonsurgical treatment failure.

AIM: To determine the efficacy of endodontic microsurgery for teeth with an undeveloped root apex and periapical periodontitis caused by an abnormal central cusp fracture after failed nonsurgical treatment. METHODOLOGY: Eighty teeth in 78 patients were subjected to endodontic microsurgery. All patients were clinically and radiologically examined 1 year postoperatively. The data were statistically analyzed using SPSS 27.0 software. RESULTS: Of the 80 teeth in 78 patients, periapical lesions had disappeared in 77 teeth at 1-year postoperative follow-up, with a success rate of approximately 96.3% (77/80). The efficacy of endodontic microsurgery was not affected by sex, age, extent of periapical lesions, and presence of the sinus tract. Between-group differences were not statistically significant (P > 0.05). CONCLUSIONS: Endodontic microsurgery can be an effective alternative treatment option for teeth with an undeveloped root apex and periapical periodontitis caused by an abnormal central cusp fracture after nonsurgical treatment failure.