Geometry-Augmented Molecular Representation Learning for Property Prediction.

Accurate molecular representation plays a crucial role in expediting the process of drug discovery. Graph neural networks (GNNs) have demonstrated robust capabilities in molecular representation learning, adept at capturing structural and spatial information in molecular graphs. For molecular representation learning, most previous GNN methods are specialized in dealing with 2D or 3D molecular data formats. By further fusing the geometric attributes and structural features of molecules, we can elevate the performance of molecular representation. To realize this, we present a novel geometryaugmented molecular representation learning model, designed to effectively encode both the 2D structural and 3D spatial information inherent in molecular graphs. By incorporating structural and spatial information as attention biases in the graph Transformer framework, our model offers a comprehensive architecture that introduces molecular structural details at both atom and bond levels. We further propose a geometry information fusion module to encode the geometry information within 3D molecular graphs. The experimental results show the efficacy of our model, demonstrating its ability to achieve competitive performance when compared to state-ofthe-art (SOTA) models in various property prediction tasks.

Experimental validation of the accuracy of robotic-assisted radioactive seed implantation for tumor treatment.

An experimental validation of a robotic system for radioactive iodine-125 seed implantation (RISI) in tumor treatment was conducted using customized phantom models and animal models simulating liver and lung lesions. The robotic system, consisting of planning, navigation, and implantation modules, was employed to implant dummy radioactive seeds into the models. Fiducial markers were used for target localization. In phantom experiments across 40 cases, the mean errors between planned and actual seed positions were 0.98 ± 1.05 mm, 1.14 ± 0.62 mm, and 0.90 ± 1.05 mm in the x, y, and z directions, respectively. The x, y, and z directions correspond to the left-right, anterior-posterior, and superior-inferior anatomical planes. Silicone phantoms exhibiting significantly smaller x-axis errors compared to liver and lung phantoms (p < 0.05). Template assistance significantly reduced errors in all axes (p < 0.05). No significant dosimetric deviations were observed in parameters such as D90, V100, and V150 between plans and post-implant doses (p > 0.05). In animal experiments across 23 liver and lung cases, the mean implantation errors were 1.28 ± 0.77 mm, 1.66 ± 0.69 mm, and 1.86 ± 0.93 mm in the x, y, and z directions, slightly higher than in phantoms (p < 0.05), with no significant differences between liver and lung models. The dosimetric results closely matched planned values, confirming the accuracy of the robotic system for RISI, offering new possibilities in clinical tumor treatment.

Teacher-student collaborated multiple instance learning for pan-cancer PDL1 expression prediction from histopathology slides.

Programmed cell death ligand 1 (PDL1), as an important biomarker, is quantified by immunohistochemistry (IHC) with few established histopathological patterns. Deep learning aids in histopathological assessment, yet heterogeneity and lacking spatially resolved annotations challenge precise analysis. Here, we present a weakly supervised learning approach using bulk RNA sequencing for PDL1 expression prediction from hematoxylin and eosin (H&E) slides. Our method extends the multiple instance learning paradigm with the teacher-student framework, which assigns dynamic pseudo-labels for intra-slide heterogeneity and retrieves unlabeled instances using temporal ensemble model distillation. The approach, evaluated on 12,299 slides across 20 solid tumor types, achieves a weighted average area under the curve of 0.83 on fresh-frozen and 0.74 on formalin-fixed specimens for 9 tumors with PDL1 as an established biomarker. Our method predicts PDL1 expression patterns, validated by IHC on 20 slides, offering insights into histologies relevant to PDL1. This demonstrates the potential of deep learning in identifying diverse histological patterns for molecular changes from H&E images.

Exploring the influence of nasal vestibule structure on nasal obstruction using CFD and Machine Learning method.

Motivated by clinical findings about the nasal vestibule, this study analyzes the aerodynamic characteristics of the nasal vestibule and attempt to determine anatomical features which have a large influence on airflow through a combination of Computational Fluid Dynamics (CFD) and machine learning method. Firstly, the aerodynamic characteristics of the nasal vestibule are detailedly analyzed using the CFD method. Based on CFD simulation results, we divide the nasal vestibule into two types with distinctly different airflow patterns, which is consistent with clinical findings. Secondly, we explore the relationship between anatomical features and aerodynamic characteristics by developing a novel machine learning model which could predict airflow patterns based on several anatomical features. Feature mining is performed to determine the anatomical feature which has the greatest impact on respiratory function. The method is developed and validated on 41 unilateral nasal vestibules from 26 patients with nasal obstruction. The correctness of the CFD analysis and the developed model is verified by comparing them with clinical findings.

Contour-aware network with class-wise convolutions for 3D abdominal multi-organ segmentation.

Accurate delineation of multiple organs is a critical process for various medical procedures, which could be operator-dependent and time-consuming. Existing organ segmentation methods, which were mainly inspired by natural image analysis techniques, might not fully exploit the traits of the multi-organ segmentation task and could not accurately segment the organs with various shapes and sizes simultaneously. In this work, the characteristics of multi-organ segmentation are considered: the global count, position and scale of organs are generally predictable, while their local shape and appearance are volatile. Thus, we supplement the region segmentation backbone with a contour localization task to increase the certainty along delicate boundaries. Meantime, each organ has exclusive anatomical traits, which motivates us to deal with class variability with class-wise convolutions to highlight organ-specific features and suppress irrelevant responses at different field-of-views. To validate our method with adequate amounts of patients and organs, we constructed a multi-center dataset, which contains 110 3D CT scans with 24,528 axial slices, and provided voxel-level manual segmentations of 14 abdominal organs, which adds up to 1,532 3D structures in total. Extensive ablation and visualization studies on it validate the effectiveness of the proposed method. Quantitative analysis shows that we achieve state-of-the-art performance for most abdominal organs, and obtain 3.63 mm 95% Hausdorff Distance and 83.32% Dice Similarity Coefficient on an average.

Fuzzy Sparse Subspace Clustering for Infrared Image Segmentation.

Infrared image segmentation is a challenging task, due to interference of complex background and appearance inhomogeneity of foreground objects. A critical defect of fuzzy clustering for infrared image segmentation is that the method treats image pixels or fragments in isolation. In this paper, we propose to adopt self-representation from sparse subspace clustering in fuzzy clustering, aiming to introduce global correlation information into fuzzy clustering. Meanwhile, to apply sparse subspace clustering for non-linear samples from an infrared image, we leverage membership from fuzzy clustering to improve conventional sparse subspace clustering. The contributions of this paper are fourfold. First, by introducing self-representation coefficients modeled in sparse subspace clustering based on high-dimensional features, fuzzy clustering is capable of utilizing global information to resist complex background as well as intensity inhomogeneity of objects, so as to improve clustering accuracy. Second, fuzzy membership is tactfully exploited in the sparse subspace clustering framework. Thereby, the bottleneck of conventional sparse subspace clustering methods, that they could be barely applied to nonlinear samples, can be surmounted. Third, as we integrate fuzzy clustering and subspace clustering in a unified framework, features from two different aspects are employed, contributing to precise clustering results. Finally, we further incorporate neighbor information into clustering, thus effectively solving the uneven intensity problem in infrared image segmentation. Experiments examine the feasibility of proposed methods on various infrared images. Segmentation results demonstrate the effectiveness and efficiency of the proposed methods, which proves the superiority compared to other fuzzy clustering methods and sparse space clustering methods.

Electron transfer rules of minerals under pressure informed by machine learning.

Electron transfer is the most elementary process in nature, but the existing electron transfer rules are seldom applied to high-pressure situations, such as in the deep Earth. Here we show a deep learning model to obtain the electronegativity of 96 elements under arbitrary pressure, and a regressed unified formula to quantify its relationship with pressure and electronic configuration. The relative work function of minerals is further predicted by electronegativity, presenting a decreasing trend with pressure because of pressure-induced electron delocalization. Using the work function as the case study of electronegativity, it reveals that the driving force behind directional electron transfer results from the enlarged work function difference between compounds with pressure. This well explains the deep high-conductivity anomalies, and helps discover the redox reactivity between widespread Fe(II)-bearing minerals and water during ongoing subduction. Our results give an insight into the fundamental physicochemical properties of elements and their compounds under pressure.

Revelation of hidden 2D atmospheric turbulence strength fields from turbulence effects in infrared imaging.

Turbulence exists widely in the natural atmosphere and in industrial fluids. Strong randomness, anisotropy and mixing of multiple-scale eddies complicate the analysis and measurement of atmospheric turbulence. Although the spatially integrated strength of atmospheric turbulence can be roughly measured indirectly by Doppler radar or laser, direct measurement of two-dimensional (2D) strength fields of atmospheric turbulence is challenging. Here we attempt to solve this problem through infrared imaging. Specifically, we propose a physically boosted cooperative learning framework, termed the PBCL, to quantify 2D turbulence strength from infrared images. To demonstrate the capability of the PBCL, we constructed a dataset with 137,336 infrared images and corresponding 2D turbulence strength fields. The experimental results show that cooperative learning brings performance improvements, enabling the PBCL to simultaneously learn turbulence strength fields and inhibit adverse turbulence effects in images. Our work demonstrates the potential of imaging in measuring physical quantity fields.

Local extreme map guided multi-modal brain image fusion.

Multi-modal brain image fusion targets on integrating the salient and complementary features of different modalities of brain images into a comprehensive image. The well-fused brain image will make it convenient for doctors to precisely examine the brain diseases and can be input to intelligent systems to automatically detect the possible diseases. In order to achieve the above purpose, we have proposed a local extreme map guided multi-modal brain image fusion method. First, each source image is iteratively smoothed by the local extreme map guided image filter. Specifically, in each iteration, the guidance image is alternatively set to the local minimum map of the input image and local maximum map of previously filtered image. With the iteratively smoothed images, multiple scales of bright and dark feature maps of each source image can be gradually extracted from the difference image of every two continuously smoothed images. Then, the multiple scales of bright feature maps and base images (i.e., final-scale smoothed images) of the source images are fused by the elementwise-maximum fusion rule, respectively, and the multiple scales of dark feature maps of the source images are fused by the elementwise-minimum fusion rule. Finally, the fused bright feature map, dark feature map, and base image are integrated together to generate a single informative brain image. Extensive experiments verify that the proposed method outperforms eight state-of-the-art (SOTA) image fusion methods from both qualitative and quantitative aspects and demonstrates great application potential to clinical scenarios.

Does radiation therapy need more than two photon energies from Linac?

Purpose: Modern Linacs are equipped with multiple photon energies for radiation therapy, and proper energy is chosen for each case based on tumor characteristics and patient anatomy. The aim of this study is to investigate whether it is necessary to have more than two photons energies. Methods: The principle of photon energy synthesis is presented. It is shown that a photon beam of any intermediate energy (Esyn) can be synthesized from a linear combination of a low energy (Elow) and a high energy (Ehigh). The principle is validated on a wide range of scenarios: different intermediate photon energies on the same Linac; between Linacs from the same manufacturer or different manufacturers; open and wedge beams; and extensive photon energies available from published reference data. In addition, 3D dose distributions in water phantom are compared using Gamma analysis. The method is further demonstrated in clinical cases of various tumor sites and multiple treatment modalities. Experimental measurements are performed for IMRT plans and they are analyzed using the standard clinical protocol. Results: The synthesis coefficients vary with energy and field size. The root mean square error (RMSE) is within 1.1% for open and wedge fields. Excellent agreement was observed for British Journal of Radiology (BJR) data with an average RMSE of 0.11%. The 3D Gamma analysis shows a good match for all field sizes in the water phantom and all treatment modalities for the five clinical cases. The minimum gamma passing rate of 95.7% was achieved at 1%/1mm criteria for two measured dose distributions of IMRT plans. Conclusion: A Linac with two photon energies is capable of producing dosimetrically equivalent plans of any energy in-between through the photon energy synthesis, supporting the notion that there is no need to equip more than two photon energies on each Linac. This can significantly reduce the cost of equipment for radiation therapy.

Attention-Assisted Adversarial Model for Cerebrovascular Segmentation in 3D TOF-MRA Volumes.

Cerebrovascular segmentation in time-of-flight magnetic resonance angiography (TOF-MRA) volumes is essential for a variety of diagnostic and analytical applications. However, accurate cerebrovascular segmentation in 3D TOF-MRA is faced with multiple issues, including vast variations in cerebrovascular morphology and intensity, noisy background, and severe class imbalance between foreground cerebral vessels and background. In this work, a 3D adversarial network model called A-SegAN is proposed to segment cerebral vessels in TOF-MRA volumes. The proposed model is composed of a segmentation network A-SegS to predict segmentation maps, and a critic network A-SegC to discriminate predictions from ground truth. Based on this model, the aforementioned issues are addressed by the prevailing visual attention mechanism. First, A-SegS is incorporated with feature-attention blocks to filter out discriminative feature maps, though the cerebrovascular has varied appearances. Second, a hard-example-attention loss is exploited to boost the training of A-SegS on hard samples. Further, A-SegC is combined with an input-attention layer to attach importance to foreground cerebrovascular class. The proposed methods were evaluated on a self-constructed voxel-wise annotated cerebrovascular TOF-MRA segmentation dataset, and experimental results indicate that A-SegAN achieves competitive or better cerebrovascular segmentation results compared to other deep learning methods, effectively alleviating the above issues.

Voxel-Wise Adversarial FiboNet for 3D Cerebrovascular Segmentation on Magnetic Resonance Angiography Images.

Cerebrovascular segmentation is important in various clinical applications, such as surgical planning and computer-aided diagnosis. In order to achieve high segmentation performance, three challenging problems should be taken into consideration: (1) large variations in vascular anatomies and voxel intensities; (2) severe class imbalance between foreground and background voxels; (3) image noise with different magnitudes. Limited accuracy was achieved without considering these challenges in deep learning-based methods for cerebrovascular segmentation. To overcome the limitations, we propose an end-to-end adversarial model called FiboNet-VANGAN. Specifically, our contributions can be summarized as follows: (1) to relieve the first problem mentioned above, a discriminator is proposed to regularize for voxel-wise distribution consistency between the segmentation results and the ground truth; (2) to mitigate the problem of class imbalance, we propose to use the addition of cross-entropy and Dice coefficient as the loss function of the generator. Focal loss is utilized as the loss function of the discriminator; (3) a new feature connection is proposed, based on which a generator called FiboNet is built. By incorporating Dice coefficient in the training of FiboNet, noise robustness can be improved by a large margin. We evaluate our method on a healthy magnetic resonance angiography (MRA) dataset to validate its effectiveness. A brain atrophy MRA dataset is also collected to test the performance of each method on abnormal cases. Results show that the three problems in cerebrovascular segmentation mentioned above can be alleviated and high segmentation accuracy can be achieved on both datasets using our method.

Technical note: A fast and accurate analytical dose calculation algorithm for 125 I seed-loaded stent applications.

PURPOSE: The safety and clinical efficacy of 125 I seed-loaded stent for the treatment of portal vein tumor thrombosis (PVTT) have been shown. Accurate and fast dose calculation of the 125 I seeds with the presence of the stent is necessary for the plan optimization and evaluation. However, the dosimetric characteristics of the seed-loaded stents remain unclear and there is no fast dose calculation technique available. This paper aims to explore a fast and accurate analytical dose calculation method based on Monte Carlo (MC) dose calculation, which takes into account the effect of stent and tissue inhomogeneity. METHODS: A detailed model of the seed-loaded stent was developed using 3D modeling software and subsequently used in MC simulations to calculate the dose distribution around the stent. The dose perturbation caused by the presence of the stent was analyzed, and dose perturbation kernels (DPKs) were derived and stored for future use. Then, the dose calculation method from AAPM TG-43 was adapted by integrating the DPK and appropriate inhomogeneity correction factors (ICF) to calculate dose distributions analytically. To validate the proposed method, several comparisons were performed with other methods in water phantom and voxelized CT phantoms for three patients. RESULTS: The stent has a considerable dosimetric effect reducing the dose up to 47.2% for single-seed stent and 11.9%-16.1% for 16-seed stent. In a water phantom, dose distributions from MC simulations and TG-43-DP-ICF showed a good agreement with the relative error less than 3.3%. In voxelized CT phantoms, taking MC results as the reference, the relative errors of TG-43 method can be up to 33%, while those of TG-43-DP-ICF method were less than 5%. For a dose matrix with 256 × 256 × 46 grid (corresponding to a phantom of 17.2 × 17.2 × 11.5 cm3 ) for 16-seed-loaded stent, it only takes 17 s for TG-43-DP-ICF to compute, compared to 25 h for the full MC calculation. CONCLUSIONS: The combination of DPK and inhomogeneity corrections is an effective approach to handle both the presence of stent and tissue heterogeneity. Exhibiting good agreement with MC calculation and computational efficiency, the proposed TG-43-DP-ICF method is adequate for dose evaluation and optimization in seed-loaded stent implantation treatment planning.

A patient-independent CT intensity matching method using conditional generative adversarial networks (cGAN) for single x-ray projection-based tumor localization.

A convolutional neural network (CNN)-based tumor localization method with a single x-ray projection was previously developed by us. One finding is that the discrepancy in the discrepancy in the intensity between a digitally reconstructed radiograph (DRR) of a three-dimensional computed tomography (3D-CT) and the measured x-ray projection has an impact on the performance. To address this issue, a patient-dependent intensity matching process for 3D-CT was performed using 3D-cone-beam computed tomography (3D-CBCT) from the same patient, which was sometimes inefficient and could adversely affect the clinical implementation of the framework. To circumvent this, in this work, we propose and validate a patient-independent intensity matching method based on a conditional generative adversarial network (cGAN). A 3D cGAN was trained to approximate the mapping from 3D-CT to 3D-CBCT from previous patient data. By applying the trained network to a new patient, a synthetic 3D-CBCT could be generated without the need to perform an actual CBCT scan on that patient. The DRR of the synthetic 3D-CBCT was subsequently utilized in our CNN-based tumor localization scheme. The method was tested using data from 12 patients with the same imaging parameters. The resulting 3D-CBCT and DRR were compared with real ones to demonstrate the efficacy of the proposed method. The tumor localization errors were also analyzed. The difference between the synthetic and real 3D-CBCT had a median value of no more than 10 HU for all patients. The relative error between the DRR and the measured x-ray projection was less than 4.8% ± 2.0% for all patients. For the three patients with a visible tumor in the x-ray projections, the average tumor localization errors were below 1.7 and 0.9 mm in the superior-inferior and lateral directions, resepectively. A patient-independent CT intensity matching method was developed, based on which accurate tumor localization was achieved. It does not require an actual CBCT scan to be performed before treatment for each patient, therefore making it more efficient in the clinical workflow.

Arc Adjacency Matrix-Based Fast Ellipse Detection.

Fast and accurate ellipse detection is critical in certain computer vision tasks. In this paper, we propose an arc adjacency matrix-based ellipse detection (AAMED) method to fulfill this requirement. At first, after segmenting the edges into elliptic arcs, the digraph-based arc adjacency matrix (AAM) is constructed to describe their triple sequential adjacency states. Curvature and region constraints are employed to make the AAM sparse. Secondly, through bidirectionally searching the AAM, we can get all arc combinations which are probably true ellipse candidates. The cumulative-factor (CF) based cumulative matrices (CM) are worked out simultaneously. CF is irrelative to the image context and can be pre-calculated. CM is related to the arcs or arc combinations and can be calculated by the addition or subtraction of CF. Then the ellipses are efficiently fitted from these candidates through twice eigendecomposition of CM using Jacobi method. Finally, a comprehensive validation score is proposed to eliminate false ellipses effectively. The score is mainly influenced by the constraints about adaptive shape, tangent similarity, distribution compensation. Experiments show that our method outperforms the 12 state-of-the-art methods on 9 datasets as a whole, with reference to recall, precision, F-measure, and time-consumption.

Real-time tumor localization with single x-ray projection at arbitrary gantry angles using a convolutional neural network (CNN).

For tumor tracking therapy, precise knowledge of tumor position in real-time is very important. A technique using single x-ray projection based on a convolutional neural network (CNN) was recently developed which can achieve accurate tumor localization in real-time. However, this method was only validated at fixed gantry angles. In this study, an improved technique is developed to handle arbitrary gantry angles for rotational radiotherapy. To evaluate the highly complex relationship between x-ray projections at arbitrary angles and tumor motion, a special CNN was proposed. In this network, a binary region of interest (ROI) mask was applied on every extracted feature map. This avoids the overfitting problem due to gantry rotation by directing the network to neglect those irrelevant pixels whose intensity variation had nothing to do with breathing motion. In addition, an angle-dependent fully connection layer (ADFCL) was utilized to recover the mapping from extracted feature maps to tumor motion, which would vary with the gantry angles. The method was tested with images from 15 realistic patients and compared with a variant network of VGG, developed by Oxford University's Visual Geometry Group. The tumors were clearly visible on x-ray projections for five patients only. The average tumor localization error was under 1.8 mm and 1.0 mm in superior-inferior and lateral directions. For the other ten patients whose tumors were not clearly visible in the x-ray projection, a feature point localization error was computed to evaluate the proposed method, the mean value of which was no more than 1.5 mm and 1.0 mm in both directions for all patients. A tumor localization method for single x-ray projection at arbitrary angles based on a novel CNN was developed and validated in this study for real-time operation. This greatly expanded the applicability of the tumor localization framework to the rotation therapy.

Thermal Infrared Pedestrian Segmentation Based on Conditional GAN.

A novel thermal infrared pedestrian segmentation algorithm based on conditional generative adversarial network (IPS-cGAN) is proposed for intelligent vehicular applications. The convolution backbone architecture of the generator is based on the improved U-Net with residual blocks for well utilizing regional semantic information. Moreover, cross entropy loss for segmentation is introduced as the condition for the generator. SandwichNet, a novel convolutional network with symmetrical input, is proposed as the discriminator for real-fake segmented images. Based on the c-GAN framework, good segmentation performance could be achieved for thermal infrared pedestrians. Compared to some supervised and unsupervised segmentation algorithms, the proposed algorithm achieves higher accuracy with better robustness, especially for complex scenes.

Efficient Multiple Organ Localization in CT Image using 3D Region Proposal Network.

Organ localization is an essential preprocessing step for many medical image analysis tasks such as image registration, organ segmentation and lesion detection. In this work, we propose an efficient method for multiple organ localization in CT image using 3D region proposal network. Compared with other convolutional neural network based methods that successively detect the target organs in all slices to assemble the final 3D bounding box, our method is fully implemented in 3D manner, thus can take full advantages of the spatial context information in CT image to perform efficient organ localization with only one prediction. We also propose a novel backbone network architecture that generates high-resolution feature maps to further improve the localization performance on small organs. We evaluate our method on two clinical datasets, where 11 body organs and 12 head organs (or anatomical structures) are included. As our results shown, the proposed method achieves higher detection precision and localization accuracy than the current state-of-theart methods with approximate 4 to 18 times faster processing speed. Additionally, we have established a public dataset dedicated for organ localization on http://dx. doi.org/10.21227/df8g-pq27. The full implementation of the proposed method have also been made publicly available on https://github.com/superxuang/caffe_3d_faster_rcnn.

Cell Segmentation Based on FOPSO Combined With Shape Information Improved Intuitionistic FCM.

Fuzzy c-means (FCM) clustering algorithms have been proved to be effective image segmentation techniques. However, FCM clustering algorithms are sensitive to noises and initialization. They cannot effectively segment cell images with inhomogeneous gray value distributions and complex touching cells. Aiming to overcome these disadvantages, this paper proposes a cell image segmentation algorithm using fractional-order velocity based particle swarm optimization (FOPSO) combined with shape information improved intuitionistic FCM (SI-IFCM) clustering. Iterations are carried out between FOPSO and SI-IFCM to achieve final cell segmentation. Experimental results demonstrate that the proposed algorithm has advantages on cell image segmentation, with the highest recall (90.25%) and lowest false discovery rate (0.28%) compared with the state-of-the-art algorithms.

Similarity Measure-Based Possibilistic FCM With Label Information for Brain MRI Segmentation.

Magnetic resonance imaging (MRI) is extensively applied in clinical practice. Segmentation of the MRI brain image is significant to the detection of brain abnormalities. However, owing to the coexistence of intensity inhomogeneity and noise, dividing the MRI brain image into different clusters precisely has become an arduous task. In this paper, an improved possibilistic fuzzy c -means (FCM) method based on a similarity measure is proposed to improve the segmentation performance for MRI brain images. By introducing the new similarity measure, the proposed method is more effective for clustering the data with nonspherical distribution. Besides that, the new similarity measure could alleviate the "cluster-size sensitivity" problem that most FCM-based methods suffer from. Simultaneously, the proposed method could preserve image details as well as suppress image noises via the use of local label information. Experiments conducted on both synthetic and clinical images show that the proposed method is very effective, providing mitigation to the cluster-size sensitivity problem, resistance to noisy images, and applicability to data with more complex distribution.