Multi-branch attention Raman network and surface-enhanced Raman spectroscopy for the classification of neurological disorders.

Surface-enhanced Raman spectroscopy (SERS), a rapid, low-cost, non-invasive, ultrasensitive, and label-free technique, has been widely used in-situ and ex-situ biomedical diagnostics questions. However, analyzing and interpreting the untargeted spectral data remains challenging due to the difficulty of designing an optimal data pre-processing and modelling procedure. In this paper, we propose a Multi-branch Attention Raman Network (MBA-RamanNet) with a multi-branch attention module, including the convolutional block attention module (CBAM) branch, deep convolution module (DCM) branch, and branch weights, to extract more global and local information of characteristic Raman peaks which are more distinctive for classification tasks. CBAM, including channel and spatial aspects, is adopted to enhance the distinctive global information on Raman peaks. DCM is used to supplement local information of Raman peaks. Autonomously trained branch weights are applied to fuse the features of each branch, thereby optimizing the global and local information of the characteristic Raman peaks for identifying diseases. Extensive experiments are performed for two different neurological disorders classification tasks via untargeted serum SERS data. The results demonstrate that MBA-RamanNet outperforms commonly used CNN methods with an accuracy of 88.24% for the classification of healthy controls, mild cognitive impairment, Alzheimer's disease, and Non-Alzheimer's dementia; an accuracy of 90% for the classification of healthy controls, elderly depression, and elderly anxiety.

NB-TCM-CHM: Image dataset of the Chinese herbal medicine fruits and its application in classification through deep learning.

Chinese herbal medicine (CHM) is integral to a traditional Chinese medicine (TCM) system. Accurately identifying Chinese herbal medicine is crucial for quality control and prescription compounding verification. However, with many Chinese herbal medicines and some with similar appearances but different therapeutic effects, achieving precise identification is a challenging task. Traditional manual identification methods have certain limitations, including labor-intensive, inefficient. Deep learning techniques for Chinese herbal medicine identification can enhance accuracy, improve efficiency and lower coats. However, few high-quality Chinese herbal medicine datasets are currently available for deep learning applications. To alleviate this problem, this study constructed a dataset (Dataset 1) containing 3,384 images of 20 common Chinese herbal medicine fruits through web crawling. All images are annotated by TCM experts, making them suitable for training and testing Chinese herbal medicine identification methods. Furthermore, this study establishes another dataset (Dataset 2) of 400 images by taking pictures using smartphones to provide materials for the practical efficacy evaluation of Chinese herbal medicine identification methods. The two datasets form a Ningbo Traditional Chinese Medicine Chinese Herb Medicine (NB-TCM-CHM) Dataset. In Dataset 1 and Dataset 2, each type of Chinese medicine herb is stored in a separate folder, with the folder named after its name. The dataset can be used to develop Chinese herbal medicine identification algorithms based on deep learning and evaluate the performance of Chinese herbal medicine identification methods.

A review of traditional Chinese medicine diagnosis using machine learning: Inspection, auscultation-olfaction, inquiry, and palpation.

Traditional Chinese medicine (TCM) is an essential part of the Chinese medical system and is recognized by the World Health Organization as an important alternative medicine. As an important part of TCM, TCM diagnosis is a method to understand a patient's illness, analyze its state, and identify syndromes. In the long-term clinical diagnosis practice of TCM, four fundamental and effective diagnostic methods of inspection, auscultation-olfaction, inquiry, and palpation (IAOIP) have been formed. However, the diagnostic information in TCM is diverse, and the diagnostic process depends on doctors' experience, which is subject to a high-level subjectivity. At present, the research on the automated diagnosis of TCM based on machine learning is booming. Machine learning, which includes deep learning, is an essential part of artificial intelligence (AI), which provides new ideas for the objective and AI-related research of TCM. This paper aims to review and summarize the current research status of machine learning in TCM diagnosis. First, we review some key factors for the application of machine learning in TCM diagnosis, including data, data preprocessing, machine learning models, and evaluation metrics. Second, we review and summarize the research and applications of machine learning methods in TCM IAOIP and the synthesis of the four diagnostic methods. Finally, we discuss the challenges and research directions of using machine learning methods for TCM diagnosis.

Quasi-supervised learning for super-resolution PET.

Low resolution of positron emission tomography (PET) limits its diagnostic performance. Deep learning has been successfully applied to achieve super-resolution PET. However, commonly used supervised learning methods in this context require many pairs of low- and high-resolution (LR and HR) PET images. Although unsupervised learning utilizes unpaired images, the results are not as good as that obtained with supervised deep learning. In this paper, we propose a quasi-supervised learning method, which is a new type of weakly-supervised learning methods, to recover HR PET images from LR counterparts by leveraging similarity between unpaired LR and HR image patches. Specifically, LR image patches are taken from a patient as inputs, while the most similar HR patches from other patients are found as labels. The similarity between the matched HR and LR patches serves as a prior for network construction. Our proposed method can be implemented by designing a new network or modifying an existing network. As an example in this study, we have modified the cycle-consistent generative adversarial network (CycleGAN) for super-resolution PET. Our numerical and experimental results qualitatively and quantitatively show the merits of our method relative to the state-of-the-art methods. The code is publicly available at https://github.com/PigYang-ops/CycleGAN-QSDL.

Multi-modal tumor segmentation methods based on deep learning: a narrative review.

Background and Objective: Automatic tumor segmentation is a critical component in clinical diagnosis and treatment. Although single-modal imaging provides useful information, multi-modal imaging provides a more comprehensive understanding of the tumor. Multi-modal tumor segmentation has been an essential topic in medical image processing. With the remarkable performance of deep learning (DL) methods in medical image analysis, multi-modal tumor segmentation based on DL has attracted significant attention. This study aimed to provide an overview of recent DL-based multi-modal tumor segmentation methods. Methods: In in the PubMed and Google Scholar databases, the keywords "multi-modal", "deep learning", and "tumor segmentation" were used to systematically search English articles in the past 5 years. The date range was from 1 January 2018 to 1 June 2023. A total of 78 English articles were reviewed. Key Content and Findings: We introduce public datasets, evaluation methods, and multi-modal data processing. We also summarize common DL network structures, techniques, and multi-modal image fusion methods used in different tumor segmentation tasks. Finally, we conclude this study by presenting perspectives for future research. Conclusions: In multi-modal tumor segmentation tasks, DL technique is a powerful method. With the fusion methods of different modal data, the DL framework can effectively use the characteristics of different modal data to improve the accuracy of tumor segmentation.

Cation-crosslinkedκ-carrageenan sub-microgel medium for high-quality embedded bioprinting.

Three-dimensional (3D) bioprinting embedded within a microgel bath has emerged as a promising strategy for creating intricate biomimetic scaffolds. However, it remains a great challenge to construct tissue-scale structures with high resolution by using embedded 3D bioprinting due to the large particle size and polydispersity of the microgel medium, as well as its limited cytocompatibility. To address these issues, novel uniform sub-microgels of cell-friendly cationic-crosslinked kappa-carrageenan (κ-Car) are developed through an easy-to-operate mechanical grinding strategy. Theseκ-Car sub-microgels maintain a uniform submicron size of around 642 nm and display a rapid jamming-unjamming transition within 5 s, along with excellent shear-thinning and self-healing properties, which are critical for the high resolution and fidelity in the construction of tissue architecture via embedded 3D bioprinting. Utilizing this new sub-microgel medium, various intricate 3D tissue and organ structures, including the heart, lungs, trachea, branched vasculature, kidney, auricle, nose, and liver, are successfully fabricated with delicate fine structures and high shape fidelity. Moreover, the bone marrow mesenchymal stem cells encapsulated within the printed constructs exhibit remarkable viability exceeding 92.1% and robust growth. Thisκ-Car sub-microgel medium offers an innovative avenue for achieving high-quality embedded bioprinting, facilitating the fabrication of functional biological constructs with biomimetic structural organizations.

BPSegSys: A Brachial Plexus Nerve Trunk Segmentation System Using Deep Learning.

OBJECTIVE: Ultrasound-guided nerve block anesthesia (UGNB) is a high-tech visual nerve block anesthesia method that can be used to observe the target nerve and its surrounding structures, the puncture needle's advancement and local anesthetics spread in real time. The key in UGNB is nerve identification. With the help of deep learning methods, the automatic identification or segmentation of nerves can be realized, assisting doctors in completing nerve block anesthesia accurately and efficiently. METHODS: We established a public data set containing 320 ultrasound images of brachial plexus (BP). Three experienced doctors jointly produced the BP segmentation ground truth and labeled brachial plexus trunks. We designed a brachial plexus segmentation system (BPSegSys) based on deep learning. RESULTS: BPSegSys achieves experienced-doctor-level nerve identification performance in various experiments. We evaluated BPSegSys performance in terms of intersection-over-union (IoU). Considering three data set groups in our established public data set, the IoUs of BPSegSys were 0.5350, 0.4763 and 0.5043, respectively, which exceed the IoUs 0.5205, 0.4704 and 0.4979 of experienced doctors. In addition, we determined that BPSegSys can help doctors identify brachial plexus trunks more accurately, with IoU improvement up to 27%, which has significant clinical application value. CONCLUSION: We establish a data set for brachial plexus trunk identification and designed a BPSegSys to identify the brachial plexus trunks. BPSegSys achieves the doctor-level identification of the brachial plexus trunks and improves the accuracy and efficiency of doctors' identification of the brachial plexus trunks.

ST-GAN: A Swin Transformer-Based Generative Adversarial Network for Unsupervised Domain Adaptation of Cross-Modality Cardiac Segmentation.

Unsupervised domain adaptation (UDA) methods have shown great potential in cross-modality medical image segmentation tasks, where target domain labels are unavailable. However, the domain shift among different image modalities remains challenging, because the conventional UDA methods are based on convolutional neural networks (CNNs), which tend to focus on the texture of images and cannot establish the global semantic relevance of features due to the locality of CNNs. This paper proposes a novel end-to-end Swin Transformer-based generative adversarial network (ST-GAN) for cross-modality cardiac segmentation. In the generator of ST-GAN, we utilize the local receptive fields of CNNs to capture spatial information and introduce the Swin Transformer to extract global semantic information, which enables the generator to better extract the domain-invariant features in UDA tasks. In addition, we design a multi-scale feature fuser to sufficiently fuse the features acquired at different stages and improve the robustness of the UDA network. We extensively evaluated our method with two cross-modality cardiac segmentation tasks on the MS-CMR 2019 dataset and the M&Ms dataset. The results of two different tasks show the validity of ST-GAN compared with the state-of-the-art cross-modality cardiac image segmentation methods.

3D bioprinting of GelMA with enhanced extrusion printability through coupling sacrificial carrageenan.

The potential of 3D bioprinting in tissue engineering and regenerative medicine is enormous, but its implementation is hindered by the reliance on high-strength materials, which restricts the use of low-viscosity, biocompatible materials. Therefore, a major challenge for incorporating 3D bioprinting into tissue engineering is to develop a novel bioprinting platform that can reversibly provide high biological activity materials with a structural support. This study presents a room temperature printing system based on GelMA combined with carrageenan to address this challenge. By leveraging the wide temperature stability range and lubricating properties of carrageenan the room temperature stability of GelMA could be enhanced, as well as creating a solid ink to improve the performance of solid GelMA. Additionally, by utilizing the solubility of carrageenan at 37 °C, it becomes possible to prepare a porous GelMA structure while mimicking the unique extracellular matrix properties of osteocytes through residual carrageenan content and amplifying BMSCs' osteogenesis potential to some extent. Overall, this study provides an innovative technical platform for incorporating a low-viscosity ink into 3D bioprinting and resolves the long-standing contradiction between material printing performance and biocompatibility in bioprinting technology.

OII-DS: A benchmark Oral Implant Image Dataset for object detection and image classification evaluation.

In recent years, there is been a growing reliance on image analysis methods to bolster dentistry practices, such as image classification, segmentation and object detection. However, the availability of related benchmark datasets remains limited. Hence, we spent six years to prepare and test a bench Oral Implant Image Dataset (OII-DS) to support the work in this research domain. OII-DS is a benchmark oral image dataset consisting of 3834 oral CT imaging images and 15240 oral implant images. It serves the purpose of object detection and image classification. To demonstrate the validity of the OII-DS, for each function, the most representative algorithms and metrics are selected for testing and evaluation. For object detection, five object detection algorithms are adopted to test and four evaluation criteria are used to assess the detection of each of the five objects. Additionally, mean average precision serves as the evaluation metric for multi-objective detection. For image classification, 13 classifiers are used for testing and evaluating each of the five categories by meeting four evaluation criteria. Experimental results affirm the high quality of our data in OII-DS, rendering it suitable for evaluating object detection and image classification methods. Furthermore, OII-DS is openly available at the URL for non-commercial purpose: https://doi.org/10.6084/m9.figshare.22608790.

Deep learning at the edge enables real-time streaming ptychographic imaging.

Coherent imaging techniques provide an unparalleled multi-scale view of materials across scientific and technological fields, from structural materials to quantum devices, from integrated circuits to biological cells. Driven by the construction of brighter sources and high-rate detectors, coherent imaging methods like ptychography are poised to revolutionize nanoscale materials characterization. However, these advancements are accompanied by significant increase in data and compute needs, which precludes real-time imaging, feedback and decision-making capabilities with conventional approaches. Here, we demonstrate a workflow that leverages artificial intelligence at the edge and high-performance computing to enable real-time inversion on X-ray ptychography data streamed directly from a detector at up to 2 kHz. The proposed AI-enabled workflow eliminates the oversampling constraints, allowing low-dose imaging using orders of magnitude less data than required by traditional methods.

QS-ADN: quasi-supervised artifact disentanglement network for low-dose CT image denoising by local similarity among unpaired data.

Deep learning has been successfully applied to low-dose CT (LDCT) image denoising for reducing potential radiation risk. However, the widely reported supervised LDCT denoising networks require a training set of paired images, which is expensive to obtain and cannot be perfectly simulated. Unsupervised learning utilizes unpaired data and is highly desirable for LDCT denoising. As an example, an artifact disentanglement network (ADN) relies on unpaired images and obviates the need for supervision but the results of artifact reduction are not as good as those through supervised learning. An important observation is that there is often hidden similarity among unpaired data that can be utilized. This paper introduces a new learning mode, called quasi-supervised learning, to empower ADN for LDCT image denoising. For every LDCT image, the best matched image is first found from an unpaired normal-dose CT (NDCT) dataset. Then, the matched pairs and the corresponding matching degree as prior information are used to construct and train our ADN-type network for LDCT denoising. The proposed method is different from (but compatible with) supervised and semi-supervised learning modes and can be easily implemented by modifying existing networks. The experimental results show that the method is competitive with state-of-the-art methods in terms of noise suppression and contextual fidelity. The code and working dataset are publicly available athttps://github.com/ruanyuhui/ADN-QSDL.git.

Predicting Respiratory Rate from Electrocardiogram and Photoplethysmogram Using a Transformer-Based Model.

The respiratory rate (RR) serves as a critical physiological parameter in the context of both diagnostic and prognostic evaluations. Due to the challenges of direct measurement, RR is still predominantly measured through the traditional manual counting-breaths method in clinic practice. Numerous algorithms and machine learning models have been developed to predict RR using physiological signals, such as electrocardiogram (ECG) or/and photoplethysmogram (PPG) signals. Yet, the accuracy of these existing methods on available datasets remains limited, and their prediction on new data is also unsatisfactory for actual clinical applications. In this paper, we proposed an enhanced Transformer model with inception blocks for predicting RR based on both ECG and PPG signals. To evaluate the generalization capability on new data, our model was trained and tested using subject-level ten-fold cross-validation using data from both BIDMC and CapnoBase datasets. On the test set, our model achieved superior performance over five popular deep-learning-based methods with mean absolute error (1.2) decreased by 36.5% and correlation coefficient (0.85) increased by 84.8% compared to the best results of these models. In addition, we also proposed a new pipeline to preprocess ECG and PPG signals to improve model performance. We believe that the development of the TransRR model is expected to further expedite the clinical implementation of automatic RR estimation.

A Lightweight Network for Contextual and Morphological Awareness for Hepatic Vein Segmentation.

Accurate segmentation of the hepatic vein can improve the precision of liver disease diagnosis and treatment. Since the hepatic venous system is a small target and sparsely distributed, with various and diverse morphology, data labeling is difficult. Therefore, automatic hepatic vein segmentation is extremely challenging. We propose a lightweight contextual and morphological awareness network and design a novel morphology aware module based on attention mechanism and a 3D reconstruction module. The morphology aware module can obtain the slice similarity awareness mapping, which can enhance the continuous area of the hepatic veins in two adjacent slices through attention weighting. The 3D reconstruction module connects the 2D encoder and the 3D decoder to obtain the learning ability of 3D context with a very small amount of parameters. Compared with other SOTA methods, using the proposed method demonstrates an enhancement in the dice coefficient with few parameters on the two datasets. A small number of parameters can reduce hardware requirements and potentially have stronger generalization, which is an advantage in clinical deployment.

Hints of Growth Mechanism Left in Supercrystals.

Supercrystals of DNA-functionalized nanoparticles are visualized in three dimensions using X-ray ptychographic tomography, and their reciprocal spaces are mapped with small-angle X-ray scattering in order to better understand their internal defect structures. X-ray ptychographic tomography reveals various types of defects in an assembly that otherwise exhibits a single crystalline diffraction pattern. On average, supercrystals composed of smaller nanoparticles are smaller in size than supercrystals composed of larger particles. Additionally, supercrystals composed of small nanoparticles are typically aggregated into larger "necklace-like" structures. Within these larger structures, some but not all pairs of connected domains are coherent in their relative orientations. In contrast, supercrystals composed of larger nanoparticles with longer DNA ligands typically form faceted crystals. The combination of these two complementary X-ray techniques reveals that the crystalline assemblies grow by aggregation of smaller assemblies followed by rearrangement of nanoparticles.

3D PET/CT tumor segmentation based on nnU-Net with GCN refinement.

Objective. Whole-body positron emission tomography/computed tomography (PET/CT) scans are an important tool for diagnosing various malignancies (e.g. malignant melanoma, lymphoma, or lung cancer), and accurate segmentation of tumors is a key part of subsequent treatment. In recent years, convolutional neural network based segmentation methods have been extensively investigated. However, these methods often give inaccurate segmentation results, such as oversegmentation and undersegmentation. To address these issues, we propose a postprocessing method based on a graph convolutional network (GCN) to refine inaccurate segmentation results and improve the overall segmentation accuracy.Approach. First, nnU-Net is used as an initial segmentation framework, and the uncertainty in the segmentation results is analyzed. Certain and uncertain pixels are used to establish the nodes of a graph. Each node and its 6 neighbors form an edge, and 32 nodes are randomly selected as uncertain nodes to form edges. The highly uncertain nodes are used as the subsequent refinement targets. Second, the nnU-Net results of the certain nodes are used as labels to form a semisupervised graph network problem, and the uncertain part is optimized by training the GCN to improve the segmentation performance. This describes our proposed nnU-Net + GCN segmentation framework.Main results.We perform tumor segmentation experiments with the PET/CT dataset from the MICCIA2022 autoPET challenge. Among these data, 30 cases are randomly selected for testing, and the experimental results show that the false-positive rate is effectively reduced with nnU-Net + GCN refinement. In quantitative analysis, there is an improvement of 2.1% for the average Dice score, 6.4 for the 95% Hausdorff distance (HD95), and 1.7 for the average symmetric surface distance.Significance. The quantitative and qualitative evaluation results show that GCN postprocessing methods can effectively improve the tumor segmentation performance.

CLSSL-ResNet: Predicting malignancy of solitary pulmonary nodules from CT images by chimeric label with self-supervised learning.

BACKGROUND: Pulmonary granulomatous nodules (GN) with spiculation or lobulation have a similar morphological appearance to solid lung adenocarcinoma (SADC) under computed tomography (CT). However, these two kinds of solid pulmonary nodules (SPN) have different malignancies and are sometimes misdiagnosed. OBJECTIVE: This study aims to predict malignancies of SPNs by a deep learning model automatically. METHODS: A chimeric label with self-supervised learning (CLSSL) is proposed to pre-train a ResNet-based network (CLSSL-ResNet) for distinguishing isolated atypical GN from SADC in CT images. The malignancy, rotation, and morphology labels are integrated into a chimeric label and utilized to pre-train a ResNet50. The pre-trained ResNet50 is then transferred and fine-tuned to predict the malignancy of SPN. Two image datasets of 428 subjects (Dataset1, 307; Dataset2, 121) from different hospitals are collected. Dataset1 is divided into training, validation, and test data by a ratio of 7:1:2 to develop the model. Dataset2 is utilized as an external validation dataset. RESULTS: CLSSL-ResNet achieves an area under the ROC curve (AUC) of 0.944 and an accuracy (ACC) of 91.3%, which was much higher than that of the consensus of two experienced chest radiologists (77.3%). CLSSL-ResNet also outperforms other self-supervised learning models and many counterparts of other backbone networks. In Dataset2, AUC and ACC of CLSSL-ResNet are 0.923 and 89.3%, respectively. Additionally, the ablation experiment result indicates higher efficiency of the chimeric label. CONCLUSION: CLSSL with morphology labels can increase the ability of feature representation by deep networks. As a non-invasive method, CLSSL-ResNet can distinguish GN from SADC via CT images and may support clinical diagnoses after further validation.

VDVM: An automatic vertebrae detection and vertebral segment matching framework for C-arm X-ray image identification.

BACKGROUND: C-arm fluoroscopy, as an effective diagnosis and treatment method for spine surgery, can help doctors perform surgery procedures more precisely. In clinical surgery, the surgeon often determines the specific surgical location by comparing C-arm X-ray images with digital radiography (DR) images. However, this heavily relies on the doctor's experience. OBJECTIVE: In this study, we design a framework for automatic vertebrae detection as well as vertebral segment matching (VDVM) for the identification of vertebrae in C-arm X-ray images. METHODS: The proposed VDVM framework is mainly divided into two parts: vertebra detection and vertebra matching. In the first part, a data preprocessing method is used to improve the image quality of C-arm X-ray images and DR images. The YOLOv3 model is then used to detect the vertebrae, and the vertebral regions are extracted based on their position. In the second part, the Mobile-Unet model is first used to segment the vertebrae contour of the C-arm X-ray image and DR image based on vertebral regions respectively. The inclination angle of the contour is then calculated using the minimum bounding rectangle and corrected accordingly. Finally, a multi-vertebra strategy is applied to measure the visual information fidelity for the vertebral region, and the vertebrae are matched based on the measured results. RESULTS: We use 382 C-arm X-ray images and 203 full length X-ray images to train the vertebra detection model, and achieve a mAP of 0.87 in the test dataset of 31 C-arm X-ray images and 0.96 in the test dataset of 31 lumbar DR images. Finally, we achieve a vertebral segment matching accuracy of 0.733 on 31 C-arm X-ray images. CONCLUSIONS: A VDVM framework is proposed, which performs well for the detection of vertebrae and achieves good results in vertebral segment matching.

Raman spectromics method for fast and label-free genotype screening.

It is now understood that genes and their various mutations are associated with the onset and progression of diseases. However, routine genetic testing techniques are limited by their high cost, time consumption, susceptibility to contamination, complex operation, and data analysis difficulties, rendering them unsuitable for genotype screening in many cases. Therefore, there is an urgent need to develop a rapid, sensitive, user-friendly, and cost-effective method for genotype screening and analysis. In this study, we propose and investigate a Raman spectroscopic method for achieving fast and label-free genotype screening. The method was validated using spontaneous Raman measurements of wild-type Cryptococcus neoformans and its six mutants. An accurate identification of different genotypes was achieved by employing a one-dimensional convolutional neural network (1D-CNN), and significant correlations between metabolic changes and genotypic variations were revealed. Genotype-specific regions of interest were also localized and visualized using a gradient-weighted class activation mapping (Grad-CAM)-based spectral interpretable analysis method. Furthermore, the contribution of each metabolite to the final genotypic decision-making was quantified. The proposed Raman spectroscopic method demonstrated huge potential for fast and label-free genotype screening and analysis of conditioned pathogens.