An indirect estimation of x-ray spectrum via convolutional neural network and transmission measurement.

OBJECTIVE: In this work, we aim to propose an accurate and robust spectrum estimation method by synergistically combining X-ray imaging physics with a convolutional neural network (CNN). Approach: The approach relies on transmission measurements, and the estimated spectrum is formulated as a convolutional summation of a few model spectra generated using Monte Carlo simulation. The difference between the actual and estimated projections is utilized as the loss function to train the network. We contrasted this approach with the weighted sums of model spectra approach previously proposed. Comprehensive studies were performed to demonstrate the robustness and accuracy of the proposed approach in various scenarios. Main results: The results show the desirable accuracy of the CNN-based method for spectrum estimation. The ME and NRMSE were -0.021 keV and 3.04% for 80kVp, and 0.006 keV and 4.44% for 100kVp, superior to the previous approach. The robustness test and experimental study also demonstrated superior performances. The CNN-based approach yielded remarkably consistent results in phantoms with various material combinations, and the CNN-based approach was robust concerning spectrum generators and calibration phantoms. Significance: We proposed a method for estimating the real spectrum by integrating a deep learning model with real imaging physics. The results demonstrated that this method was accurate and robust in estimating the spectrum, and it is potentially helpful for broad X-ray imaging tasks.

Evaluation of low-dose computed tomography reconstruction using spatial-radon domain total generalized variation regularization.

The x-ray radiation dose in computed tomography (CT) examination has been a major concern for patients. Lowing the tube current and exposure time in data acquisition is a straightforward and cost-effective strategy to reduce the x-ray radiation dose. However, this will inevitably increase the noise fluctuations in measured projection data, and the corresponding CT image quality will be severely degraded if noise suppression is not performed during image reconstruction. To reconstruct high-quality low-dose CT image, we present a spatial-radon domain total generalized variation (SRDTGV) regularization for statistical iterative reconstruction (SIR) based on penalized weighted least-squares (PWLS) principle, which is called PWLS-SRDTGV for simplicity. The presented PWLS-SRDTGV model can simultaneously reconstruct high-quality CT image in space domain and its corresponding projection in radon domain. An efficient split Bregman algorithm was applied to minimize the cost function of the proposed reconstruction model. Qualitative and quantitative studies were performed to evaluate the effectiveness of the PWLS-SRDTGV image reconstruction algorithm using a digital 3D XCAT phantom and an anthropomorphic torso phantom. The experimental results demonstrate that PWLS-SRDTGV algorithm achieves notable gains in noise reduction, streak artifact suppression, and edge preservation compared with competing reconstruction approaches.

PNMC: Four-dimensional conebeam CT reconstruction combining prior network and motion compensation.

Four-dimensional conebeam computed tomography (4D CBCT) is an efficient technique to overcome motion artifacts caused by organ motion during breathing. 4D CBCT reconstruction in a single scan usually divides projections into different groups of sparsely sampled data based on the respiratory phases. The reconstructed images within each group present poor image quality due to the limited number of projections. To improve the image quality of 4D CBCT in a single scan, we propose a novel reconstruction scheme that combines prior knowledge with motion compensation. We apply the reconstructed images of the full projections within a single routine as prior knowledge, providing structural information for the network to enhance the restoration structure. The prior network (PN-Net) is proposed to extract features of prior knowledge and fuse them with the sparsely sampled data using an attention mechanism. The prior knowledge guides the reconstruction process to restore the approximate organ structure and alleviates severe streaking artifacts. The deformation vector field (DVF) extracted using deformable image registration among different phases is then applied in the motion-compensated ordered-subset simultaneous algebraic reconstruction algorithm to generate 4D CBCT images. Proposed method has been evaluated using simulated and clinical datasets and has shown promising results by comparative experiment. Compared with previous methods, our approach exhibits significant improvements across various evaluation metrics.

UBES: Unified scatter correction using ultrafast Boltzmann equation solver for conebeam CT.

A semi-analytical solution to the unified Boltzmann equation is constructed to exactly describe the scatter distribution on a flat-panel detector for high-quality conebeam CT (CBCT) imaging. The solver consists of three parts, including the phase space distribution estimator, the effective source constructor and the detector signal extractor. Instead of the tedious Monte Carlo solution, the derived Boltzmann equation solver achieves ultrafast computational capability for scatter signal estimation by combining direct analytical derivation and time-efficient one-dimensional numerical integration over the trajectory along each momentum of the photon phase space distribution. The execution of scatter estimation using the proposed ultrafast Boltzmann equation solver (UBES) for a single projection is finalized in around 0.4 seconds. We compare the performance of the proposed method with the state-of-the-art schemes, including a time-expensive Monte Carlo (MC) method and a conventional kernel-based algorithm using the same dataset, which is acquired from the CBCT scans of a head phantom and an abdominal patient. The evaluation results demonstrate that the proposed UBES method achieves comparable correction accuracy compared with the MC method, while exhibits significant improvements in image quality over learning and kernel-based methods. With the advantages of MC equivalent quality and superfast computational efficiency, the UBES method has the potential to become a standard solution to scatter correction in high-quality CBCT reconstruction.

Prediction of SBRT response in liver cancer by combining original and delta cone-beam CT radiomics: a pilot study.

This study aims to explore the feasibility of utilizing a combination of original and delta cone-beam CT (CBCT) radiomics for predicting treatment response in liver tumors undergoing stereotactic body radiation therapy (SBRT). A total of 49 patients are included in this study, with 36 receiving 5-fraction SBRT, 3 receiving 4-fraction SBRT, and 10 receiving 3-fraction SBRT. The CBCT and planning CT images from liver cancer patients who underwent SBRT are collected to extract overall 547 radiomics features. The CBCT features which are reproducible and interchangeable with pCT are selected for modeling analysis. The delta features between fractions are calculated to depict tumor change. The patients with 4-fraction SBRT are only used for screening robust features. In patients receiving 5-fraction SBRT, the predictive ability of both original and delta CBCT features for two-level treatment response (local efficacy vs. local non-efficacy; complete response (CR) vs. partial response (PR)) is assessed by utilizing multivariable logistic regression with leave-one-out cross-validation. Additionally, univariate analysis is conducted to validate the capability of CBCT features in identifying local efficacy in patients receiving 3-fraction SBRT. In patients receiving 5-fraction SBRT, the combined models incorporating original and delta CBCT radiomics features demonstrate higher area under the curve (AUC) values compared to models using either original or delta features alone for both classification tasks. The AUC values for predicting local efficacy vs. local non-efficacy are 0.58 for original features, 0.82 for delta features, and 0.90 for combined features. For distinguishing PR from CR, the respective AUC values for original, delta and combined features are 0.79, 0.80, and 0.89. In patients receiving 3-fraction SBRT, eight valuable CBCT radiomics features are identified for predicting local efficacy. The combination of original and delta radiomics derived from fractionated CBCT images in liver cancer patients undergoing SBRT shows promise in providing comprehensive information for predicting treatment response.

Phase-Tailoring Wx V1-x O2 Meta-Nanofiber Enables Temperature-Editing Energy Control.

Room-temperature phase change materials (RTPCMs) exhibit promise to address challenges in thermal energy storage and release, greatly aiding in numerous domains of human existence and productivity. The conventional RTPCMs undergo inevitable volume expansion, structural collapse, and diffusion of active ingredients while maintaining desirable phase change enthalpy and ideal phase change temperature. Here, a sol-gel 1D-induced growth approach is presented to fabricate meta nanofibers (Meta-NFs) comprised of vanadium dioxide with monoclinic crystal structure, and further achieve the editable phase change temperature from 68 to 37 °C through W-doping, which allowed for tailored length variation of the zigzag V-V bond. Subsequently, Meta-NFs are assembled into 3D aerogels with self-standing architecture, thereby enabling the independent use of the RTPCMs. The obtained metamaterials demonstrate not only the temperature-editing solid-solid phase transition, but also the stiffness of the ceramic matrix, exhibiting the thermal energy control capability at room temperature (37 °C), thermal insulation properties, temperature resistance, and flame retardancy. The effective creation of these fascinating metamaterials might offer new insights for next-generation and self-standing solid-solid RTPCMs.

4D-CT deformable image registration using unsupervised recursive cascaded full-resolution residual networks.

A novel recursive cascaded full-resolution residual network (RCFRR-Net) for abdominal four-dimensional computed tomography (4D-CT) image registration was proposed. The entire network was end-to-end and trained in the unsupervised approach, which meant that the deformation vector field, which presented the ground truth, was not needed during training. The network was designed by cascading three full-resolution residual subnetworks with different architectures. The training loss consisted of the image similarity loss and the deformation vector field regularization loss, which were calculated based on the final warped image and the fixed image, allowing all cascades to be trained jointly and perform the progressive registration cooperatively. Extensive network testing was conducted using diverse datasets, including an internal 4D-CT dataset, a public DIRLAB 4D-CT dataset, and a 4D cone-beam CT (4D-CBCT) dataset. Compared with the iteration-based demon method and two deep learning-based methods (VoxelMorph and recursive cascaded network), the RCFRR-Net achieved consistent and significant gains, which demonstrated that the proposed method had superior performance and generalization capability in medical image registration. The proposed RCFRR-Net was a promising tool for various clinical applications.

Virtual differential phase-contrast and dark-field imaging of x-ray absorption images via deep learning.

Weak absorption contrast in biological tissues has hindered x-ray computed tomography from accessing biological structures. Recently, grating-based imaging has emerged as a promising solution to biological low-contrast imaging, providing complementary and previously unavailable structural information of the specimen. Although it has been successfully applied to work with conventional x-ray sources, grating-based imaging is time-consuming and requires a sophisticated experimental setup. In this work, we demonstrate that a deep convolutional neural network trained with a generative adversarial network can directly convert x-ray absorption images into differential phase-contrast and dark-field images that are comparable to those obtained at both a synchrotron beamline and a laboratory facility. By smearing back all of the virtual projections, high-quality tomographic images of biological test specimens deliver the differential phase-contrast- and dark-field-like contrast and quantitative information, broadening the horizon of x-ray image contrast generation.

A generalizable new figure of merit for dose optimization in dual energy cone beam CT scanning protocols.

Objective. This study proposes and evaluates a new figure of merit (FOMn) for dose optimization of Dual-energy cone-beam CT (DE-CBCT) scanning protocols based on size-dependent modeling of radiation dose and multi-scale image quality.Approach. FOMn was defined using Z-score normalization and was proportional to the dose efficiency providing better multi-scale image quality, including comprehensive contrast-to-noise ratio (CCNR) and electron density (CED) for CatPhan604 inserts of various materials. Acrylic annuluses were combined with CatPhan604 to create four phantom sizes (diameters of the long axis are 200 mm, 270 mm, 350 mm, and 380 mm, respectively). DE-CBCT was decomposed using image-domain iterative methods based on Varian kV-CBCT images acquired using 25 protocols (100 kVp and 140 kVp combined with 5 tube currents).Main results. The accuracy of CED was approximately 1% for all protocols, but degraded monotonically with the increased phantom sizes. Combinations of lower voltage + higher current and higher voltage + lower current were optimal protocols balancing CCNR and dose. The most dose-efficient protocols for CED and CCNR were inconsistent, underlining the necessity of including multi-scale image quality in the evaluation and optimization of DE-CBCT. Pediatric and adult anthropomorphic phantom tests confirmed dose-efficiency of FOMn-recommended protocols.Significance. FOMn is a comprehensive metric that collectively evaluates radiation dose and multi-scale image quality for DE-CBCT. The models and data can also serve as lookup tables, suggesting personalized dose-efficient protocols for specific clinical imaging purposes.

The dose-related plateau effect of surviving fraction in normal tissue during the ultra-high-dose-rate radiotherapy.

Objective.Ultra-high-dose-rate radiotherapy, referred to as FLASH therapy, has been demonstrated to reduce the damage of normal tissue as well as inhibiting tumor growth compared with conventional dose-rate radiotherapy. The transient hypoxia may be a vital explanation for sparing the normal tissue. The heterogeneity of oxygen distribution for different doses and dose rates in the different radiotherapy schemes are analyzed. With these results, the influence of doses and dose rates on cell survival are evaluated in this work.Approach.The two-dimensional reaction-diffusion equations are used to describe the heterogeneity of the oxygen distribution in capillaries and tissue. A modified linear quadratic model is employed to characterize the surviving fraction at different doses and dose rates.Main results.The reduction of the damage to the normal tissue can be observed if the doses exceeds a minimum dose threshold under the ultra-high-dose-rate radiation. Also, the surviving fraction exhibits the 'plateau effect' under the ultra-high dose rates radiation, which signifies that within a specific range of doses, the surviving fraction either exhibits minimal variation or increases with the dose. For a given dose, the surviving fraction increases with the dose rate until tending to a stable value, which means that the protection in normal tissue reaches saturation.Significance.The emergence of the 'plateau effect' allows delivering the higher doses while minimizing damage to normal tissue. It is necessary to develop appropriate program of doses and dose rates for different irradiated tissue to achieve more efficient protection.

Application of multi-modality MRI-based radiomics in the pre-treatment prediction of RPS6K expression in hepatocellular carcinoma.

In this study, we aim to develop and validate a radiomics model for pretreatment prediction of RPS6K expression in hepatocellular carcinoma (HCC) patients, thus helping clinical decision-making of mTOR-inhibitor (mTORi) therapy. We retrospectively enrolled 147 HCC patients, who underwent curative hepatic resection at First Affiliated Hospital Zhejiang University School of Medicine. RPS6K expression was determined with immunohistochemistry staining. Patients were randomly split into training or validation cohorts on a 7:3 ratio. Radiomics features were extracted from T2-weighted and diffusion-weighted images. Machine learning algorithms including multiple logistic regression (MLR), supporting vector machine (SVM), random forest (RF), and artificial neural network (ANN) were applied to construct the predictive model. A nomogram was further built to visualize the possibility of RPS6K expression. The area under the receiver operating characteristic (AUC) was used to evaluate the performance of diagnostic models. 174 radiomics features were confirmed correlated with RPS6K expression. Amongst all built models, the ANN-based hybrid model exhibited best predictive ability with AUC of 0.887 and 0.826 in training and validation cohorts. ALB was identified as the key clinical index, and the nomogram displayed further improved ability with AUC of 0.917 and 0.845. In this study, we proved MRI-based radiomics model and nomogram can accurately predict RPS6K expression non-invasively, thus providing help for clinical decision making for mTORi therapy.

Preoperative radiomic signature based on CT images for noninvasive evaluation of localized nephroblastoma in pediatric patients.

Background: Nephron sparing nephrectomy may not reduce the prognosis of nephroblastoma in the absence of involvement of the renal capsule, sinus vessels, and lymph nodes, However, there is no accurate preoperative noninvasive evaluation method at present. Materials and methods: 105 nephroblastoma patients underwent contrast-enhanced CT scan between 2013 and 2020 in our hospital were retrospectively collected, including 59 cases with localized stage and 46 cases with non-localized stage, and then were divided into training cohort (n= 73) and validation cohort (n= 32) according to the order of CT scanning time. After lesion segmentation and data preprocessing, radiomic features were extracted from each volume of interest. The multi-step procedure including Pearson correlation analysis and sequential forward floating selection was performed to produce radiomic signature. Prediction model was constructed using the radiomic signature and Logistic Regression classifier for predicting the localized nephroblastoma in the training cohort. Finally, the model performance was validated in the validation cohort. Results: A total of 1652 radiomic features have been extracted, from which TOP 10 features were selected as the radiomic signature. The area under the receiver operating characteristic curve, accuracy, sensitivity and specificity of the prediction model were 0.796, 0.795, 0.732 and 0.875 for the training cohort respectively, and 0.710, 0.719, 0.611 and 0.857 for the validation cohort respectively. The result comparison with prediction models composed of different machine learning classifiers and different parameters also manifest the effectiveness of our radiomic model. Conclusion: A logistic regression model based on radiomic features extracted from preoperative CT images had good ability to noninvasively predict nephroblastoma without renal capsule, sinus vessel, and lymph node involvement.

Four-Dimensional Cone Beam CT Imaging Using a Single Routine Scan via Deep Learning.

A novel method is proposed to obtain four-dimensional (4D) cone-beam computed tomography (CBCT) images from a routine scan in patients with upper abdominal cancer. The projections are sorted according to the location of the lung diaphragm before being reconstructed to phase-sorted data. A multiscale-discriminator generative adversarial network (MSD-GAN) is proposed to alleviate the severe streaking artifacts in the original images. The MSD-GAN is trained using simulated CBCT datasets from patient planning CT images. The enhanced images are further used to estimate the deformable vector field (DVF) among breathing phases using a deformable image registration method. The estimated DVF is then applied in the motion-compensated ordered-subset simultaneous algebraic reconstruction approach to generate 4D CBCT images. The proposed MSD-GAN is compared with U-Net on the performance of image enhancement. Results show that the proposed method significantly outperforms the total variation regularization-based iterative reconstruction approach and the method using only MSD-GAN to enhance original phase-sorted images in simulation and patient studies on 4D reconstruction quality. The MSD-GAN also shows higher accuracy than the U-Net. The proposed method enables a practical way for 4D-CBCT imaging from a single routine scan in upper abdominal cancer treatment including liver and pancreatic tumors.

Image-based scatter correction for cone-beam CT using flip swin transformer U-shape network.

BACKGROUND: Cone beam computed tomography (CBCT) plays an increasingly important role in image-guided radiation therapy. However, the image quality of CBCT is severely degraded by excessive scatter contamination, especially in the abdominal region, hindering its further applications in radiation therapy. PURPOSE: To restore low-quality CBCT images contaminated by scatter signals, a scatter correction algorithm combining the advantages of convolutional neural networks (CNN) and Swin Transformer is proposed. METHODS: In this paper a scatter correction model for CBCT image, the Flip Swin Transformer U-shape network (FSTUNet) model, is proposed. In this model, the advantages of CNN in texture detail and Swin Transformer in global correlation are used to accurately extract shallow and deep features, respectively. Instead of using the original Swin Transformer tandem structure, we build the Flip Swin Transformer Block to achieve a more powerful inter-window association extraction. The validity and clinical relevance of the method is demonstrated through extensive experiments on a Monte Carlo (MC) simulation dataset and frequency split dataset generated by a validated method, respectively. RESULT: Experimental results on the MC simulated dataset show that the root mean square error of images corrected by the method is reduced from over 100 HU to about 7 HU. Both the structural similarity index measure (SSIM) and the universal quality index (UQI) are close to 1. Experimental results on the frequency split dataset demonstrate that the method not only corrects shading artifacts but also exhibits a high degree of structural consistency. In addition, comparison experiments show that FSTUNet outperforms UNet, Deep Residual Convolutional Neural Network (DRCNN), DSENet, Pix2pixGAN, and 3DUnet methods in both qualitative and quantitative metrics. CONCLUSIONS: Accurately capturing the features at different levels is greatly beneficial for reconstructing high-quality scatter-free images. The proposed FSTUNet method is an effective solution to CBCT scatter correction and has the potential to improve the accuracy of CBCT image-guided radiation therapy.

CT-based radiomics for the identification of colorectal cancer liver metastases sensitive to first-line irinotecan-based chemotherapy.

BACKGROUND: Chemosensitivity prediction in colorectal cancer patients with liver metastases has remained a research hotspot. Radiomics can extract features from patient imaging, and deep learning or machine learning can be used to build models to predict patient outcomes prior to chemotherapy. PURPOSE: In this study, the radiomics features and clinical data of colorectal cancer patients with liver metastases were used to predict their sensitivity to irinotecan-based chemotherapy. METHODS: A total of 116 patients with unresectable colorectal cancer liver metastases who received first-line irinotecan-based chemotherapy from January 2015 to January 2020 in our institution were retrospectively collected. Overall, 116 liver metastases were randomly divided into training (n = 81) and validation (n = 35) cohorts in a 7:3 ratio. The effect of chemotherapy was determined based on Response Evaluation Criteria in Solid Tumors. The lesions were divided into response and nonresponse groups. Regions of interest (ROIs) were manually segmented, and sample sizes of 1×1×1, 3×3×3, 5×5×5 mm3 were used to extract radiomics features. The relevant features were identified through Pearson correlation analysis and the MRMR algorithm, and the clinical data were merged into the artificial neural network. Finally, the p-model was obtained after repeated learning and testing. RESULTS: The p-model could distinguish responders in the training (area under the curve [AUC] 0.754, 95% CI 0.650-0.858) and validation cohorts (AUC 0.752 95% CI 0.581-0.904). AUC values of the pure image group model are 0.720 (95% CI 0.609-0.827) and 0.684 (95% CI 0.529-0.890) for the training and validation cohorts respectively. As for the clinical data model, AUC values of the training and validation cohorts are 0.638 (95% CI 0.500-0.757) and 0.545 (95% CI 0.360-0.785), respectively. The performances of the latter two are less than that of the former. CONCLUSION: The p-model has the potential to discriminate colorectal cancer patients sensitive to chemotherapy. This model holds promise as a noninvasive tool to predict the response of colorectal liver metastases to chemotherapy, allowing for personalized treatment planning.

An Unsupervised Learning-Based Regional Deformable Model for Automated Multi-Organ Contour Propagation.

The aim of this study is to evaluate a regional deformable model based on a deep unsupervised learning model for automatic contour propagation in breast cone-beam computed tomography-guided adaptive radiation therapy. A deep unsupervised learning model was introduced to map breast's tumor bed, clinical target volume, heart, left lung, right lung, and spinal cord from planning computed tomography to cone-beam CT. To improve the traditional image registration method's performance, we used a regional deformable framework based on the narrow-band mapping, which can mitigate the effect of the image artifacts on the cone-beam CT. We retrospectively selected 373 anonymized cone-beam CT volumes from 111 patients with breast cancer. The cone-beam CTs are divided into three sets. 311 / 20 / 42 cone-beam CT images were used for training, validating, and testing. The manual contour was used as reference for the testing set. We compared the results between the reference and the model prediction for evaluating the performance. The mean Dice between manual reference segmentations and the model predicted segmentations for breast tumor bed, clinical target volume, heart, left lung, right lung, and spinal cord were 0.78 ± 0.09, 0.90 ± 0.03, 0.88 ± 0.04, 0.94 ± 0.03, 0.95 ± 0.02, and 0.77 ± 0.07, respectively. The results demonstrated a good agreement between the reference and the proposed contours. The proposed deep learning-based regional deformable model technique can automatically propagate contours for breast cancer adaptive radiotherapy. Deep learning in contour propagation was promising, but further investigation was warranted.

A temporo-spatial description of moving tumor and organs by the probability of presence time proportion: concept and implementation for 4D dose calculation and optimization.

Objective. The binary definition of the internal target volume (ITV) artificially separates tumor from healthy organs at motion overlapping area for dose evaluation and optimization, bringing confusion about taking partial organs as tumor or adversely. In this work, the probability of presence time (PPT) proportion of a moving anatomic voxel at a geometric voxel is defined to construct a temporo-spatial description of moving objects. The geometric overlapping of tumor and organs in 3D space is distinguished by individual residence time proportion. The dose deposition at a geometric voxel is decomposed into individual dose delivered to tumor and organs for accumulative dose calculation and optimization.Approach.A novel PPT-based plan optimization strategy is proposed to generate an optimized non-uniform dose distribution based on the temporo-spatial relationship between tumor and organs.Main results.Results from a simulation study on phantoms show that the proposed method provides promising performance for surrounding organs at risk (OAR) avoidance with a reduction of mean and maximum dose at a range of 22.6%-23.1% and 23.6%-28.3% compared with ITV-based plans under different geometric conditions, while keeping the clinical target volume dose as prescription.Significance.The PPT definition constructs a unified framework to deal with the 4D temporo-spatial distribution, accumulative dose calculation and optimization of moving tumor and organs. The advantages of the PPT-based dose calculation and optimization approach are demonstrated by simulation study with significant reduction of OARs dose level compared with conventional ITV-based plan.

AI-Augmented Images for X-Ray Guiding Radiation Therapy Delivery.

Cancer accounts for nearly 10 million deaths in 2020 and is a leading cause of death worldwide. Radiation therapy is an effective modality to cure cancer. The ultimate goal of successful radiation therapy is to accurately deliver a safe and therapeutic dose of radiation to cancerous target cells while limiting radiation to the healthy tissue in the beam pathway and the surrounding normal tissue. With advancing radiation treatment technology, better imaging techniques are imperative for safe, high-dose delivery. Artificial intelligence developments on image-guided radiation therapy technologies, which range from Kilovoltage and Megavoltage modalities to two-dimensional and three-dimensional techniques, are discussed in depth.

ISA-Net: Improved spatial attention network for PET-CT tumor segmentation.

BACKGROUND AND OBJECTIVE: Achieving accurate and automated tumor segmentation plays an important role in both clinical practice and radiomics research. Segmentation in medicine is now often performed manually by experts, which is a laborious, expensive and error-prone task. Manual annotation relies heavily on the experience and knowledge of these experts. In addition, there is much intra- and interobserver variation. Therefore, it is of great significance to develop a method that can automatically segment tumor target regions. METHODS: In this paper, we propose a deep learning segmentation method based on multimodal positron emission tomography-computed tomography (PET-CT), which combines the high sensitivity of PET and the precise anatomical information of CT. We design an improved spatial attention network(ISA-Net) to increase the accuracy of PET or CT in detecting tumors, which uses multi-scale convolution operation to extract feature information and can highlight the tumor region location information and suppress the non-tumor region location information. In addition, our network uses dual-channel inputs in the coding stage and fuses them in the decoding stage, which can take advantage of the differences and complementarities between PET and CT. RESULTS: We validated the proposed ISA-Net method on two clinical datasets, a soft tissue sarcoma(STS) and a head and neck tumor(HECKTOR) dataset, and compared with other attention methods for tumor segmentation. The DSC score of 0.8378 on STS dataset and 0.8076 on HECKTOR dataset show that ISA-Net method achieves better segmentation performance and has better generalization. CONCLUSIONS: The method proposed in this paper is based on multi-modal medical image tumor segmentation, which can effectively utilize the difference and complementarity of different modes. The method can also be applied to other multi-modal data or single-modal data by proper adjustment.

Improved GAN: Using a transformer module generator approach for material decomposition.

Dual-energy computed tomography (CT) can be used for material decomposition, allowing for the precise quantitative mapping of body substances; this has a wide range of clinical applications, including disease diagnosis, treatment response evaluation and prognosis prediction. However, dual-energy CT has not yet become the mainstream technique in most clinical settings due to its limited accessibility. To fully take advantage of material quantification, researchers have attempted to use deep learning to generate material decomposition maps from conventional single-energy CT images, mainly by synthesizing another single-energy CT image from a conventional single-energy CT image to form a dual-energy CT image first and then generate material decomposition maps. This is not a straightforward process, and it potentially introduces many inaccuracies after multiple steps. In this work, we proposed a generative adversarial network (GAN) framework as the base and improved its generator; this approach combines convolutional neural networks (CNNs) and a transformer module to directly generate material decomposition maps from conventional single-energy CT images. Our model pays attention to both local and global information. Then, we compared our method with 6 competitive deep learning methods on water (calcium) and calcium (water) substrate density image datasets. The average PSNR, SSIM, MAE, and RMSE of the generated and ground truth of the water (calcium) substrate density images were 32.7207, 0.9685, 0.0323, and 0.0555, respectively. Furthermore, the average PSNR, SSIM, MAE, and RMSE of the generated and ground truth of the calcium (water) substrate density images were 30.2823, 0.9449, 0.0652, and 0.0715, respectively. Our model achieved better performance and stronger stability than competing approaches.