Image Intrinsic-Based Unsupervised Monocular Depth Estimation in Endoscopy.

Unsupervised monocular depth estimation plays a vital role for endoscopy-based minimally invasive surgery (MIS). However, it remains challenging due to the distinctive imaging characteristics of endoscopy which disrupt the assumption of photometric consistency, a foundation relied upon by conventional methods. Distinct from recent approaches taking image pre-processing strategy, this paper introduces a pioneering solution through intrinsic image decomposition (IID) theory. Specifically, we propose a novel end-to-end intrinsic-based unsupervised monocular depth learning framework that is comprised of an image intrinsic decomposition module and a synthesis reconstruction module. This framework seamlessly integrates IID with unsupervised monocular depth estimation, and dedicated losses are meticulously designed to offer robust supervision for network training based on this novel integration. Noteworthy, we rely on the favorable property of the resulting albedo map of IID to circumvent the challenging images characteristics instead of pre-processing the input frames. The proposed method is extensively validated on SCARED and Hamlyn datasets, and better results are obtained than state-of-the-art techniques. Beside, its generalization ability and the effectiveness of the proposed components are also validated. This innovative method has the potential to elevate the quality of 3D reconstruction in monocular endoscopy, thereby enhancing the accuracy and robustness of augmented reality navigation technology in MIS. Our code will be available at: https://github.com/bobo909/IID-SfmLearner.

An end-to-end deep convolutional neural network-based dose engine for parotid gland cancer seed implant brachytherapy.

BACKGROUND: Seed implant brachytherapy (SIBT) is a promising treatment modality for parotid gland cancers (PGCs). However, the current clinical standard dose calculation method based on the American Association of Physicists in Medicine (AAPM) Task Group 43 (TG-43) Report oversimplifies patient anatomy as a homogeneous water phantom medium, leading to significant dose calculation errors due to heterogeneity surrounding the parotid gland. Monte Carlo Simulation (MCS) can yield accurate dose distributions but the long computation time hinders its wide application in clinical practice. PURPOSE: This paper aims to develop an end-to-end deep convolutional neural network-based dose engine (DCNN-DE) to achieve fast and accurate dose calculation for PGC SIBT. METHODS: A DCNN model was trained using the patient's CT images and TG-43-based dose maps as inputs, with the corresponding MCS-based dose maps as the ground truth. The DCNN model was enhanced based on our previously proposed model by incorporating attention gates (AGs) and large kernel convolutions. Training and evaluation of the model were performed using a dataset comprising 188 PGC I-125 SIBT patient cases, and its transferability was tested on an additional 16 non-PGC head and neck cancers (HNCs) I-125 SIBT patient cases. Comparison studies were conducted to validate the superiority of the enhanced model over the original one and compare their overall performance. RESULTS: On the PGC testing dataset, the DCNN-DE demonstrated the ability to generate accurate dose maps, with percentage absolute errors (PAEs) of 0.67% ± 0.47% for clinical target volume (CTV) D90 and 1.04% ± 1.33% for skin D0.1cc. The comparison studies revealed that incorporating AGs and large kernel convolutions resulted in 8.2% (p < 0.001) and 3.1% (p < 0.001) accuracy improvement, respectively, as measured by dose mean absolute error. On the non-PGC HNC dataset, the DCNN-DE exhibited good transferability, achieving a CTV D90 PAE of 1.88% ± 1.73%. The DCNN-DE can generate a dose map in less than 10 ms. CONCLUSIONS: We have developed and validated an end-to-end DCNN-DE for PGC SIBT. The proposed DCNN-DE enables fast and accurate dose calculation, making it suitable for application in the plan optimization and evaluation process of PGC SIBT.

RT-SRTS: Angle-agnostic real-time simultaneous 3D reconstruction and tumor segmentation from single X-ray projection.

Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a novel imaging method RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) and achieves real-time simultaneous 3D reconstruction and tumor segmentation from a single X-ray projection at any angle. Furthermore, the attention enhanced calibrator (AEC) and uncertain-region elaboration (URE) modules have been proposed to aid feature extraction and improve segmentation accuracy. The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. It not only delivers superior 3D reconstruction but also demonstrates commendable tumor segmentation results. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking. The efficacies of both AEC and URE have also been validated in ablation studies. The code of work is available at https://github.com/ZywooSimple/RT-SRTS.

A deep learning-based 3D Prompt-nnUnet model for automatic segmentation in brachytherapy of postoperative endometrial carcinoma.

PURPOSE: To create and evaluate a three-dimensional (3D) Prompt-nnUnet module that utilizes the prompts-based model combined with 3D nnUnet for producing the rapid and consistent autosegmentation of high-risk clinical target volume (HR CTV) and organ at risk (OAR) in high-dose-rate brachytherapy (HDR BT) for patients with postoperative endometrial carcinoma (EC). METHODS AND MATERIALS: On two experimental batches, a total of 321 computed tomography (CT) scans were obtained for HR CTV segmentation from 321 patients with EC, and 125 CT scans for OARs segmentation from 125 patients. The numbers of training/validation/test were 257/32/32 and 87/13/25 for HR CTV and OARs respectively. A novel comparison of the deep learning neural network 3D Prompt-nnUnet and 3D nnUnet was applied for HR CTV and OARs segmentation. Three-fold cross validation and several quantitative metrics were employed, including Dice similarity coefficient (DSC), Hausdorff distance (HD), 95th percentile of Hausdorff distance (HD95%), and intersection over union (IoU). RESULTS: The Prompt-nnUnet included two forms of parameters Predict-Prompt (PP) and Label-Prompt (LP), with the LP performing most similarly to the experienced radiation oncologist and outperforming the less experienced ones. During the testing phase, the mean DSC values for the LP were 0.96 ± 0.02, 0.91 ± 0.02, and 0.83 ± 0.07 for HR CTV, rectum and urethra, respectively. The mean HD values (mm) were 2.73 ± 0.95, 8.18 ± 4.84, and 2.11 ± 0.50, respectively. The mean HD95% values (mm) were 1.66 ± 1.11, 3.07 ± 0.94, and 1.35 ± 0.55, respectively. The mean IoUs were 0.92 ± 0.04, 0.84 ± 0.03, and 0.71 ± 0.09, respectively. A delineation time < 2.35 s per structure in the new model was observed, which was available to save clinician time. CONCLUSION: The Prompt-nnUnet architecture, particularly the LP, was highly consistent with ground truth (GT) in HR CTV or OAR autosegmentation, reducing interobserver variability and shortening treatment time.

Automatic planning for head and neck seed implant brachytherapy based on deep convolutional neural network dose engine.

BACKGROUND: Seed implant brachytherapy (SIBT) is an effective treatment modality for head and neck (H&N) cancers; however, current clinical planning requires manual setting of needle paths and utilizes inaccurate dose calculation algorithms. PURPOSE: This study aims to develop an accurate and efficient deep convolutional neural network dose engine (DCNN-DE) and an automatic SIBT planning method for H&N SIBT. METHODS: A cohort of 25 H&N patients who received SIBT was utilized to develop and validate the methods. The DCNN-DE was developed based on 3D-unet model. It takes single seed dose distribution from a modified TG-43 method, the CT image and a novel inter-seed shadow map (ISSM) as inputs, and predicts the dose map of accuracy close to the one from Monte Carlo simulations (MCS). The ISSM was proposed to better handle inter-seed attenuation. The accuracy and efficacy of the DCNN-DE were validated by comparing with other methods taking MCS dose as reference. For SIBT planning, a novel strategy inspired by clinical practice was proposed to automatically generate parallel or non-parallel potential needle paths that avoid puncturing bone and critical organs. A heuristic-based optimization method was developed to optimize the seed positions to meet clinical prescription requirements. The proposed planning method was validated by re-planning the 25 cases and comparing with clinical plans. RESULTS: The absolute percentage error in the TG-43 calculation for CTV V100 and D90 was reduced from 5.4% and 13.2% to 0.4% and 1.1% with DCNN-DE, an accuracy improvement of 93% and 92%, respectively. The proposed planning method could automatically obtain a plan in 2.5 ± 1.5 min. The generated plans were judged clinically acceptable with dose distribution comparable with those of the clinical plans. CONCLUSIONS: The proposed method can generate clinically acceptable plans quickly with high accuracy in dose evaluation, and thus has a high potential for clinical use in SIBT.

Does radiation therapy need more than two photon energies from Linac?

Purpose: Modern Linacs are equipped with multiple photon energies for radiation therapy, and proper energy is chosen for each case based on tumor characteristics and patient anatomy. The aim of this study is to investigate whether it is necessary to have more than two photons energies. Methods: The principle of photon energy synthesis is presented. It is shown that a photon beam of any intermediate energy (Esyn) can be synthesized from a linear combination of a low energy (Elow) and a high energy (Ehigh). The principle is validated on a wide range of scenarios: different intermediate photon energies on the same Linac; between Linacs from the same manufacturer or different manufacturers; open and wedge beams; and extensive photon energies available from published reference data. In addition, 3D dose distributions in water phantom are compared using Gamma analysis. The method is further demonstrated in clinical cases of various tumor sites and multiple treatment modalities. Experimental measurements are performed for IMRT plans and they are analyzed using the standard clinical protocol. Results: The synthesis coefficients vary with energy and field size. The root mean square error (RMSE) is within 1.1% for open and wedge fields. Excellent agreement was observed for British Journal of Radiology (BJR) data with an average RMSE of 0.11%. The 3D Gamma analysis shows a good match for all field sizes in the water phantom and all treatment modalities for the five clinical cases. The minimum gamma passing rate of 95.7% was achieved at 1%/1mm criteria for two measured dose distributions of IMRT plans. Conclusion: A Linac with two photon energies is capable of producing dosimetrically equivalent plans of any energy in-between through the photon energy synthesis, supporting the notion that there is no need to equip more than two photon energies on each Linac. This can significantly reduce the cost of equipment for radiation therapy.

Reorganizations of latency structures within the white matter from wakefulness to sleep.

Previous resting-state functional magnetic resonance imaging (fMRI) studies have revealed highly reproducible latency structures, reflecting the lead/lag relationship of BOLD fMRI signals in white matter (WM). With simultaneous electroencephalography and fMRI data from 35 healthy subjects who were instructed to sleep during imaging, we explored alterations of latency structures in the WM across wakefulness and nonrapid eye movement (NREM) sleep stages. Lagged cross-covariance was computed among voxelwise time series, followed by parabolic interpolation to determine the actual in-between latencies. WM regions, including the brainstem, internal capsule, optic radiation, genu of corpus callosum, and corona radiata, inconsistently changed temporal dynamics with respect to the rest of the WM across wakefulness and NREM sleep stages, as demonstrated when these regions were used as seeds for seed-based latency analysis. Latency analysis of resting-state networks, obtained by applying K-means clustering to a group-level functional connectivity matrix, identified a dominant direction of signaling, starting from the brainstem up to the internal capsule and then the corona radiata during wakefulness, which was reorganized according to stage transitions, e.g., the temporal organization of the internal capsule and corona radiata switched from unidirectional to bidirectional in the wakefulness to N3 transition. These findings suggest that WM BOLD signals are slow, dynamically modulated across wakefulness and NREM sleep stages and that they are involved in maintaining different levels of consciousness.

Applying pytorch toolkit to plan optimization for circular cone based robotic radiotherapy.

BACKGROUND: Robotic linac is ideally suited to deliver hypo-fractionated radiotherapy due to its compact head and flexible positioning. The non-coplanar treatment space improves the delivery versatility but the complexity also leads to prolonged optimization and treatment time. METHODS: In this study, we attempted to use the deep learning (pytorch) framework for the plan optimization of circular cone based robotic radiotherapy. The optimization problem was topologized into a simple feedforward neural network, thus the treatment plan optimization was transformed into network training. With this transformation, the pytorch toolkit with high-efficiency automatic differentiation (AD) for gradient calculation was used as the optimization solver. To improve the treatment efficiency, plans with fewer nodes and beams were sought. The least absolute shrinkage and selection operator (lasso) and the group lasso were employed to address the "sparsity" issue. RESULTS: The AD-S (AD sparse) approach was validated on 6 brain and 6 liver cancer cases and the results were compared with the commercial MultiPlan (MLP) system. It was found that the AD-S plans achieved rapid dose fall-off and satisfactory sparing of organs at risk (OARs). Treatment efficiency was improved by the reduction in the number of nodes (28%) and beams (18%), and monitor unit (MU, 24%), respectively. The computational time was shortened to 47.3 s on average. CONCLUSIONS: In summary, this first attempt of applying deep learning framework to the robotic radiotherapy plan optimization is promising and has the potential to be used clinically.

Latency structure of BOLD signals within white matter in resting-state fMRI.

PURPOSE: Previous studies have demonstrated that BOLD signals in gray matter in resting-state functional MRI (RSfMRI) have variable time lags, representing apparent propagations of fMRI BOLD signals in gray matter. We complemented existing findings and explored the corresponding variations of signal latencies in white matter. METHODS: We used data from the Brain Genomics Superstruct Project, consisting of 1412 subjects (both sexes included) and divided the dataset into ten equal groups to study both the patterns and reproducibility of latency estimates within white matter. We constructed latency matrices by computing cross-covariances between voxel pairs. We also applied a clustering analysis to identify functional networks within white matter, based on which latency analysis was also performed to investigate lead/lag relationship at network level. A dataset consisting of various sensory states (eyes closed, eyes open and eyes open with fixation) was also included to examine the relationship between latency structure and different states. RESULTS: Projections of voxel latencies from the latency matrices were highly correlated (average Pearson correlation coefficient = 0.89) across the subgroups, confirming the reproducibility and structure of signal lags in white matter. Analysis of latencies within and between networks revealed a similar pattern of inter- and intra-network communication to that reported for gray matter. Moreover, a dominant direction, from inferior to superior regions, of BOLD signal propagation was revealed by higher resolution clustering. The variations of lag structure within white matter are associated with different sensory states. CONCLUSIONS: These findings provide additional insight into the character and roles of white matter BOLD signals in brain functions.

Deep learning-based two-step organs at risk auto-segmentation model for brachytherapy planning in parotid gland carcinoma.

Purpose: Delineation of organs at risk (OARs) represents a crucial step for both tailored delivery of radiation doses and prevention of radiation-induced toxicity in brachytherapy. Due to lack of studies on auto-segmentation methods in head and neck cancers, our study proposed a deep learning-based two-step approach for auto-segmentation of organs at risk in parotid carcinoma brachytherapy. Material and methods: Computed tomography images of 200 patients with parotid gland carcinoma were used to train and evaluate our in-house developed two-step 3D nnU-Net-based model for OARs auto-segmentation. OARs during brachytherapy were defined as the auricula, condyle process, skin, mastoid process, external auditory canal, and mandibular ramus. Auto-segmentation results were compared to those of manual segmentation by expert oncologists. Accuracy was quantitatively evaluated in terms of dice similarity coefficient (DSC), Jaccard index, 95th-percentile Hausdorff distance (95HD), and precision and recall. Qualitative evaluation of auto-segmentation results was also performed. Results: The mean DSC values of each OAR were 0.88, 0.91, 0.75, 0.89, 0.74, and 0.93, respectively, indicating close resemblance of auto-segmentation results to those of manual contouring. In addition, auto-segmentation could be completed within a minute, as compared with manual segmentation, which required over 20 minutes. All generated results were deemed clinically acceptable. Conclusions: Our proposed deep learning-based two-step OARs auto-segmentation model demonstrated high efficiency and good agreement with gold standard manual contours. Thereby, this novel approach carries the potential in expediting the treatment planning process of brachytherapy for parotid gland cancers, while allowing for more accurate radiation delivery to minimize toxicity.

Correlated functional connectivity and glucose metabolism in brain white matter revealed by simultaneous MRI/positron emission tomography.

PURPOSE: There has been converging evidence of reliable detections of blood oxygenation level dependent (BOLD) signals evoked by neural stimulation and in a resting state in white matter (WM), within which few studies examined the relationship between BOLD functional signals and tissue metabolism. The purpose of the present study was to explore whether such relationship exists using combined functional MRI and positron emission tomography (PET) measurements of glucose uptake. METHODS: Functional and metabolic imaging data from 25 right-handed healthy human adults (aged 18-23 years, 18 females) were analyzed. Measures, including average resting state functional connectivity (FC) with respect to 82 Brodmann areas, fractional amplitude of low-frequency fluctuations (FALFF), and average fluorodeoxyglucose (FDG) uptake by PET, were computed for 48 predefined WM bundles. Pearson correlations across the bundles and 25 subjects studied were calculated among these measures. Linear mixed effects models were used to estimate the variance explainable by a predictor variable in the absence of inter-subject variations. RESULTS: Analysis of six separate imaging intervals found that average FC the bundles was significantly correlated with local FDG uptake (r = 0.25, p < 0.001), and the FC also covaried significantly with FALFF (r = 0.41, p < 0.001). When random effects from inter-subject variations were controlled, these correlations appeared to be medium to strong (r = 0.41 for FC vs. FDG uptake, and r = 0.65 for FALFF vs. FC). CONCLUSION: This study indicates that BOLD signals in WM are directly related to variations in metabolic demand and engagement with cortical processing and suggests they should be incorporated into more complete models of brain function.

Voxel-Wise Adversarial FiboNet for 3D Cerebrovascular Segmentation on Magnetic Resonance Angiography Images.

Cerebrovascular segmentation is important in various clinical applications, such as surgical planning and computer-aided diagnosis. In order to achieve high segmentation performance, three challenging problems should be taken into consideration: (1) large variations in vascular anatomies and voxel intensities; (2) severe class imbalance between foreground and background voxels; (3) image noise with different magnitudes. Limited accuracy was achieved without considering these challenges in deep learning-based methods for cerebrovascular segmentation. To overcome the limitations, we propose an end-to-end adversarial model called FiboNet-VANGAN. Specifically, our contributions can be summarized as follows: (1) to relieve the first problem mentioned above, a discriminator is proposed to regularize for voxel-wise distribution consistency between the segmentation results and the ground truth; (2) to mitigate the problem of class imbalance, we propose to use the addition of cross-entropy and Dice coefficient as the loss function of the generator. Focal loss is utilized as the loss function of the discriminator; (3) a new feature connection is proposed, based on which a generator called FiboNet is built. By incorporating Dice coefficient in the training of FiboNet, noise robustness can be improved by a large margin. We evaluate our method on a healthy magnetic resonance angiography (MRA) dataset to validate its effectiveness. A brain atrophy MRA dataset is also collected to test the performance of each method on abnormal cases. Results show that the three problems in cerebrovascular segmentation mentioned above can be alleviated and high segmentation accuracy can be achieved on both datasets using our method.

Technical note: A fast and accurate analytical dose calculation algorithm for 125 I seed-loaded stent applications.

PURPOSE: The safety and clinical efficacy of 125 I seed-loaded stent for the treatment of portal vein tumor thrombosis (PVTT) have been shown. Accurate and fast dose calculation of the 125 I seeds with the presence of the stent is necessary for the plan optimization and evaluation. However, the dosimetric characteristics of the seed-loaded stents remain unclear and there is no fast dose calculation technique available. This paper aims to explore a fast and accurate analytical dose calculation method based on Monte Carlo (MC) dose calculation, which takes into account the effect of stent and tissue inhomogeneity. METHODS: A detailed model of the seed-loaded stent was developed using 3D modeling software and subsequently used in MC simulations to calculate the dose distribution around the stent. The dose perturbation caused by the presence of the stent was analyzed, and dose perturbation kernels (DPKs) were derived and stored for future use. Then, the dose calculation method from AAPM TG-43 was adapted by integrating the DPK and appropriate inhomogeneity correction factors (ICF) to calculate dose distributions analytically. To validate the proposed method, several comparisons were performed with other methods in water phantom and voxelized CT phantoms for three patients. RESULTS: The stent has a considerable dosimetric effect reducing the dose up to 47.2% for single-seed stent and 11.9%-16.1% for 16-seed stent. In a water phantom, dose distributions from MC simulations and TG-43-DP-ICF showed a good agreement with the relative error less than 3.3%. In voxelized CT phantoms, taking MC results as the reference, the relative errors of TG-43 method can be up to 33%, while those of TG-43-DP-ICF method were less than 5%. For a dose matrix with 256 × 256 × 46 grid (corresponding to a phantom of 17.2 × 17.2 × 11.5 cm3 ) for 16-seed-loaded stent, it only takes 17 s for TG-43-DP-ICF to compute, compared to 25 h for the full MC calculation. CONCLUSIONS: The combination of DPK and inhomogeneity corrections is an effective approach to handle both the presence of stent and tissue heterogeneity. Exhibiting good agreement with MC calculation and computational efficiency, the proposed TG-43-DP-ICF method is adequate for dose evaluation and optimization in seed-loaded stent implantation treatment planning.

Deep Learning for Hemorrhagic Lesion Detection and Segmentation on Brain CT Images.

Stroke is an acute cerebral vascular disease that is likely to cause long-term disabilities and death. Immediate emergency care with accurate diagnosis of computed tomographic (CT) images is crucial for dealing with a hemorrhagic stroke. However, due to the high variability of a stroke's location, contrast, and shape, it is challenging and time-consuming even for experienced radiologists to locate them. In this paper, we propose a U-net based deep learning framework to automatically detect and segment hemorrhage strokes in CT brain images. The input of the network is built by concatenating the flipped image with the original CT slice which introduces symmetry constraints of the brain images into the proposed model. This enhances the contrast between hemorrhagic area and normal brain tissue. Various Deep Learning topologies are compared by varying the layers, batch normalization, dilation rates, and pre-train models. This could increase the respective filed and preserves more information on lesion characteristics. Besides, the adversarial training is also adopted in the proposed network to improve the accuracy of the segmentation. The proposed model is trained and evaluated on two different datasets, which achieve the competitive performance with human experts with the highest location accuracy 0.9859 for detection, 0.8033 Dice score, and 0.6919 IoU for segmentation. The results demonstrate the effectiveness, robustness, and advantages of the proposed deep learning model in automatically hemorrhage lesion diagnosis, which make it possible to be a clinical decision support tool in stroke diagnosis.

A patient-independent CT intensity matching method using conditional generative adversarial networks (cGAN) for single x-ray projection-based tumor localization.

A convolutional neural network (CNN)-based tumor localization method with a single x-ray projection was previously developed by us. One finding is that the discrepancy in the discrepancy in the intensity between a digitally reconstructed radiograph (DRR) of a three-dimensional computed tomography (3D-CT) and the measured x-ray projection has an impact on the performance. To address this issue, a patient-dependent intensity matching process for 3D-CT was performed using 3D-cone-beam computed tomography (3D-CBCT) from the same patient, which was sometimes inefficient and could adversely affect the clinical implementation of the framework. To circumvent this, in this work, we propose and validate a patient-independent intensity matching method based on a conditional generative adversarial network (cGAN). A 3D cGAN was trained to approximate the mapping from 3D-CT to 3D-CBCT from previous patient data. By applying the trained network to a new patient, a synthetic 3D-CBCT could be generated without the need to perform an actual CBCT scan on that patient. The DRR of the synthetic 3D-CBCT was subsequently utilized in our CNN-based tumor localization scheme. The method was tested using data from 12 patients with the same imaging parameters. The resulting 3D-CBCT and DRR were compared with real ones to demonstrate the efficacy of the proposed method. The tumor localization errors were also analyzed. The difference between the synthetic and real 3D-CBCT had a median value of no more than 10 HU for all patients. The relative error between the DRR and the measured x-ray projection was less than 4.8% ± 2.0% for all patients. For the three patients with a visible tumor in the x-ray projections, the average tumor localization errors were below 1.7 and 0.9 mm in the superior-inferior and lateral directions, resepectively. A patient-independent CT intensity matching method was developed, based on which accurate tumor localization was achieved. It does not require an actual CBCT scan to be performed before treatment for each patient, therefore making it more efficient in the clinical workflow.

Arc Adjacency Matrix-Based Fast Ellipse Detection.

Fast and accurate ellipse detection is critical in certain computer vision tasks. In this paper, we propose an arc adjacency matrix-based ellipse detection (AAMED) method to fulfill this requirement. At first, after segmenting the edges into elliptic arcs, the digraph-based arc adjacency matrix (AAM) is constructed to describe their triple sequential adjacency states. Curvature and region constraints are employed to make the AAM sparse. Secondly, through bidirectionally searching the AAM, we can get all arc combinations which are probably true ellipse candidates. The cumulative-factor (CF) based cumulative matrices (CM) are worked out simultaneously. CF is irrelative to the image context and can be pre-calculated. CM is related to the arcs or arc combinations and can be calculated by the addition or subtraction of CF. Then the ellipses are efficiently fitted from these candidates through twice eigendecomposition of CM using Jacobi method. Finally, a comprehensive validation score is proposed to eliminate false ellipses effectively. The score is mainly influenced by the constraints about adaptive shape, tangent similarity, distribution compensation. Experiments show that our method outperforms the 12 state-of-the-art methods on 9 datasets as a whole, with reference to recall, precision, F-measure, and time-consumption.

Real-time tumor localization with single x-ray projection at arbitrary gantry angles using a convolutional neural network (CNN).

For tumor tracking therapy, precise knowledge of tumor position in real-time is very important. A technique using single x-ray projection based on a convolutional neural network (CNN) was recently developed which can achieve accurate tumor localization in real-time. However, this method was only validated at fixed gantry angles. In this study, an improved technique is developed to handle arbitrary gantry angles for rotational radiotherapy. To evaluate the highly complex relationship between x-ray projections at arbitrary angles and tumor motion, a special CNN was proposed. In this network, a binary region of interest (ROI) mask was applied on every extracted feature map. This avoids the overfitting problem due to gantry rotation by directing the network to neglect those irrelevant pixels whose intensity variation had nothing to do with breathing motion. In addition, an angle-dependent fully connection layer (ADFCL) was utilized to recover the mapping from extracted feature maps to tumor motion, which would vary with the gantry angles. The method was tested with images from 15 realistic patients and compared with a variant network of VGG, developed by Oxford University's Visual Geometry Group. The tumors were clearly visible on x-ray projections for five patients only. The average tumor localization error was under 1.8 mm and 1.0 mm in superior-inferior and lateral directions. For the other ten patients whose tumors were not clearly visible in the x-ray projection, a feature point localization error was computed to evaluate the proposed method, the mean value of which was no more than 1.5 mm and 1.0 mm in both directions for all patients. A tumor localization method for single x-ray projection at arbitrary angles based on a novel CNN was developed and validated in this study for real-time operation. This greatly expanded the applicability of the tumor localization framework to the rotation therapy.

Efficient Multiple Organ Localization in CT Image using 3D Region Proposal Network.

Organ localization is an essential preprocessing step for many medical image analysis tasks such as image registration, organ segmentation and lesion detection. In this work, we propose an efficient method for multiple organ localization in CT image using 3D region proposal network. Compared with other convolutional neural network based methods that successively detect the target organs in all slices to assemble the final 3D bounding box, our method is fully implemented in 3D manner, thus can take full advantages of the spatial context information in CT image to perform efficient organ localization with only one prediction. We also propose a novel backbone network architecture that generates high-resolution feature maps to further improve the localization performance on small organs. We evaluate our method on two clinical datasets, where 11 body organs and 12 head organs (or anatomical structures) are included. As our results shown, the proposed method achieves higher detection precision and localization accuracy than the current state-of-theart methods with approximate 4 to 18 times faster processing speed. Additionally, we have established a public dataset dedicated for organ localization on http://dx. doi.org/10.21227/df8g-pq27. The full implementation of the proposed method have also been made publicly available on https://github.com/superxuang/caffe_3d_faster_rcnn.

Infrared simulation of large-scale urban scene through LOD.

The growing use of infrared (IR) imaging systems places increasing demands for simulating infrared images of real scenes. Utilizing images captured from unmanned aerial vehicles (UAV), we propose a semi-automatic pipeline to generate large-scale IR urban scenes in the form of levels of detail (LODs). It significantly reduces the cost of labor and time while providing detailed IR structures. Starting from the surface meshes generated by multi-view stereo (MVS) systems, we produce watertight LODs via semantic segmentation and structure-aware approximation. For each LOD, we divide the surfaces into triangle facets of specific scales. For each facet, one material attribute is attached, and the heat balance equations are solved to obtain the temperature. Three strategies are proposed to accelerate the thermal distribution calculation. Finally, by synthesizing the radiance distribution, the whole IR scenes are generated and rendered. Experiments on real urban scenes show that the proposed pipeline could effectively simulate IR scenes of large-scale urban scenes.

Automatic bladder segmentation from CT images using deep CNN and 3D fully connected CRF-RNN.

PURPOSE: Automatic approach for bladder segmentation from computed tomography (CT) images is highly desirable in clinical practice. It is a challenging task since the bladder usually suffers large variations of appearance and low soft-tissue contrast in CT images. In this study, we present a deep learning-based approach which involves a convolutional neural network (CNN) and a 3D fully connected conditional random fields recurrent neural network (CRF-RNN) to perform accurate bladder segmentation. We also propose a novel preprocessing method, called dual-channel preprocessing, to further advance the segmentation performance of our approach. METHODS: The presented approach works as following: first, we apply our proposed preprocessing method on the input CT image and obtain a dual-channel image which consists of the CT image and an enhanced bladder density map. Second, we exploit a CNN to predict a coarse voxel-wise bladder score map on this dual-channel image. Finally, a 3D fully connected CRF-RNN refines the coarse bladder score map and produce final fine-localized segmentation result. RESULTS: We compare our approach to the state-of-the-art V-net on a clinical dataset. Results show that our approach achieves superior segmentation accuracy, outperforming the V-net by a significant margin. The Dice Similarity Coefficient of our approach (92.24%) is 8.12% higher than that of the V-net. Moreover, the bladder probability maps performed by our approach present sharper boundaries and more accurate localizations compared with that of the V-net. CONCLUSION: Our approach achieves higher segmentation accuracy than the state-of-the-art method on clinical data. Both the dual-channel processing and the 3D fully connected CRF-RNN contribute to this improvement. The united deep network composed of the CNN and 3D CRF-RNN also outperforms a system where the CRF model acts as a post-processing method disconnected from the CNN.