A deep learning-based 3D Prompt-nnUnet model for automatic segmentation in brachytherapy of postoperative endometrial carcinoma.

PURPOSE: To create and evaluate a three-dimensional (3D) Prompt-nnUnet module that utilizes the prompts-based model combined with 3D nnUnet for producing the rapid and consistent autosegmentation of high-risk clinical target volume (HR CTV) and organ at risk (OAR) in high-dose-rate brachytherapy (HDR BT) for patients with postoperative endometrial carcinoma (EC). METHODS AND MATERIALS: On two experimental batches, a total of 321 computed tomography (CT) scans were obtained for HR CTV segmentation from 321 patients with EC, and 125 CT scans for OARs segmentation from 125 patients. The numbers of training/validation/test were 257/32/32 and 87/13/25 for HR CTV and OARs respectively. A novel comparison of the deep learning neural network 3D Prompt-nnUnet and 3D nnUnet was applied for HR CTV and OARs segmentation. Three-fold cross validation and several quantitative metrics were employed, including Dice similarity coefficient (DSC), Hausdorff distance (HD), 95th percentile of Hausdorff distance (HD95%), and intersection over union (IoU). RESULTS: The Prompt-nnUnet included two forms of parameters Predict-Prompt (PP) and Label-Prompt (LP), with the LP performing most similarly to the experienced radiation oncologist and outperforming the less experienced ones. During the testing phase, the mean DSC values for the LP were 0.96 ± 0.02, 0.91 ± 0.02, and 0.83 ± 0.07 for HR CTV, rectum and urethra, respectively. The mean HD values (mm) were 2.73 ± 0.95, 8.18 ± 4.84, and 2.11 ± 0.50, respectively. The mean HD95% values (mm) were 1.66 ± 1.11, 3.07 ± 0.94, and 1.35 ± 0.55, respectively. The mean IoUs were 0.92 ± 0.04, 0.84 ± 0.03, and 0.71 ± 0.09, respectively. A delineation time < 2.35 s per structure in the new model was observed, which was available to save clinician time. CONCLUSION: The Prompt-nnUnet architecture, particularly the LP, was highly consistent with ground truth (GT) in HR CTV or OAR autosegmentation, reducing interobserver variability and shortening treatment time.

Correlated functional connectivity and glucose metabolism in brain white matter revealed by simultaneous MRI/positron emission tomography.

PURPOSE: There has been converging evidence of reliable detections of blood oxygenation level dependent (BOLD) signals evoked by neural stimulation and in a resting state in white matter (WM), within which few studies examined the relationship between BOLD functional signals and tissue metabolism. The purpose of the present study was to explore whether such relationship exists using combined functional MRI and positron emission tomography (PET) measurements of glucose uptake. METHODS: Functional and metabolic imaging data from 25 right-handed healthy human adults (aged 18-23 years, 18 females) were analyzed. Measures, including average resting state functional connectivity (FC) with respect to 82 Brodmann areas, fractional amplitude of low-frequency fluctuations (FALFF), and average fluorodeoxyglucose (FDG) uptake by PET, were computed for 48 predefined WM bundles. Pearson correlations across the bundles and 25 subjects studied were calculated among these measures. Linear mixed effects models were used to estimate the variance explainable by a predictor variable in the absence of inter-subject variations. RESULTS: Analysis of six separate imaging intervals found that average FC the bundles was significantly correlated with local FDG uptake (r = 0.25, p < 0.001), and the FC also covaried significantly with FALFF (r = 0.41, p < 0.001). When random effects from inter-subject variations were controlled, these correlations appeared to be medium to strong (r = 0.41 for FC vs. FDG uptake, and r = 0.65 for FALFF vs. FC). CONCLUSION: This study indicates that BOLD signals in WM are directly related to variations in metabolic demand and engagement with cortical processing and suggests they should be incorporated into more complete models of brain function.

Infrared simulation of large-scale urban scene through LOD.

The growing use of infrared (IR) imaging systems places increasing demands for simulating infrared images of real scenes. Utilizing images captured from unmanned aerial vehicles (UAV), we propose a semi-automatic pipeline to generate large-scale IR urban scenes in the form of levels of detail (LODs). It significantly reduces the cost of labor and time while providing detailed IR structures. Starting from the surface meshes generated by multi-view stereo (MVS) systems, we produce watertight LODs via semantic segmentation and structure-aware approximation. For each LOD, we divide the surfaces into triangle facets of specific scales. For each facet, one material attribute is attached, and the heat balance equations are solved to obtain the temperature. Three strategies are proposed to accelerate the thermal distribution calculation. Finally, by synthesizing the radiance distribution, the whole IR scenes are generated and rendered. Experiments on real urban scenes show that the proposed pipeline could effectively simulate IR scenes of large-scale urban scenes.

Semi-automated infrared simulation on real urban scenes based on multi-view images.

The development of modern infrared applications require simulating thermal representations for targets of interest. However, generating geometric models for simulation has been a laborious, time-consuming work, which greatly limits the practical applications in real-world. In order to reduce the man-in-the-loop requirements, we devise a method that directly and semi-automatically simulates infrared signatures of real urban scenes. From raw meshes generated by multi-view stereo, we automatically produce a simplified watertight model through piecewise-planar 3D reconstruction. Model surface is subdivided into quality mesh elements to attach material attributes. For each element, heat balance equation is solved so as to render the whole scene by synthesizing the radiance distribution in infrared waveband. The credibility and effectiveness of our method are confirmed by comparing simulation results to the measured data in real-world. Our experiments on various types of buildings and large scale scene show that the proposed pipeline simulates meaningful infrared scenes while being robust and scalable.

Video stabilization in atmosphere turbulent conditions based on the Laplacian-Riesz pyramid.

Video stabilization in atmosphere turbulent conditions is aimed at removing spatiotemporally varying distortions from video recordings. Conventional shaky video stabilization approaches do not perform effectively under turbulent circumstances due to the erratic motion common to those conditions. Using complex-valued image pyramids, we propose a method to mitigate this erratic motion in videos. First, each frame of a video is decomposed into different spatial frequencies using the Laplacian pyramid. Second, a Riesz transform is adopted to extract the local amplitude and the local phase of each sub-band. Next, low-pass filters are designed to attenuate the local amplitude and phase variations to remove turbulence-induced distortions. Experimental results show that the proposed approach is efficient and provides stabilizing video in atmosphere turbulent conditions.

Modified anisotropic turbulence refractive-index fluctuations spectral model and its application in moderate-to-strong anisotropic turbulence.

In this study, the modified anisotropic turbulence refractive-index fluctuations spectral model is derived based on the extended Rytov approximation theory for the theoretical investigations of optical plane and spherical waves propagating through moderate-to-strong anisotropic non-Kolmogorov turbulence. The anisotropic factor which parameterizes the asymmetry of turbulence cells or eddies in the horizontal and vertical directions is introduced. The general spectral power law in the range of 3-4 is also considered compared with the conventional classic value of 11/3 for Kolmogorov turbulence. Based on the modified anisotropic turbulence refractive-index fluctuations spectrum, the analytic expressions of the irradiance scintillation index are also derived for optical plane and spherical waves propagating through moderate-to-strong anisotropic non-Kolmogorov turbulence. They are applicable in a wide range of turbulence strengths and can reduce correctly to the previously published results in the special cases of weak anisotropic turbulence and moderate-to-strong isotropic turbulence. Calculations are performed to analyze the derived models.

Generalized anisotropic turbulence spectra and applications in the optical waves' propagation through anisotropic turbulence.

Theoretical and experimental investigations have shown that the atmospheric turbulence exhibits both anisotropic and non-Kolmogorov properties. In this work, two theoretical atmosphere refractive-index fluctuations spectral models are derived for optical waves propagating through anisotropic non-Kolmogorov atmospheric turbulence. They consider simultaneously the finite turbulence inner and outer scales and the asymmetric property of turbulence eddies in the orthogonal xy-plane throughout the path. Two anisotropy factors which parameterize the asymmetry of turbulence eddies in both horizontal and vertical directions are introduced in the orthogonal xy-plane, so that the circular symmetry assumption of turbulence eddies in the xy-plane is no longer required. Deviations from the classic 11/3 power law behavior in the spectrum model are also allowed by assuming power law value variations between 3 and 4. Based on the derived anisotropic spectral model and the Rytov approximation theory, expressions for the variance of angle of arrival (AOA) fluctuations are derived for optical plane and spherical waves propagating through weak anisotropic non-Kolmogorov turbulence. Calculations are performed to analyze the derived spectral models and the variance of AOA fluctuations.

A novel greedy heuristic-based approach to intraoperative planning for permanent prostate brachytherapy.

This paper presents a greedy heuristic-based double iteration and rectification (DIR) approach to intraoperative planning for permanent prostate brachytherapy. The DIR approach adopts a greedy seed selection (GSS) procedure to obtain a preliminary plan. In this process, the potential seeds are evaluated according to their ability to irradiate target while spare organs at risk (OARs), and their impact on dosimetric homogeneity within target volume. A flexible termination condition is developed for the GSS procedure, which guarantees sufficient dose within target volume while avoids overdosing the OARs. The preliminary treatment plan generated by the GSS procedure is further refined by the double iteration (DI) and rectification procedure. The DI procedure removes the needles containing only one seed (single seed) and implements the GSS procedure again to get a temporary plan. The DI procedure terminates until the needles number of the temporary plan does not decrease. This process is guided by constantly removing undesired part rather than imposing extra constrains. Following the DI procedure, the rectification procedure attempts to replace the remaining single seeds with the acceptable ones within the existing needles. The change of dosimetric distribution (DD) after the replacement is evaluated to determine whether to accept or to withdraw the replacement. Experimental results demonstrate that the treatment plans obtained by the DIR approach caters to all clinical considerations. Compared with currently available methods, DIR approach is faster, more reliable, and more suitable for intraoperative treatment planning in the operation room.

Angle of arrival fluctuations considering turbulence outer scale for optical waves' propagation through moderate-to-strong non-Kolmogorov turbulence.

Based on the generalized von Kármán spectrum and the extended Rytov theory, new analytic expressions for the variance of angle of arrival (AOA) fluctuations are derived for optical plane and spherical waves propagating through moderate-to-strong non-Kolmogorov turbulence with horizontal path. They consider finite turbulence outer scale and general spectral power law value, and cover a wide range of non-Kolmogorov turbulence strength. When the turbulence outer scale is set to infinite, the new expressions can reduce correctly to previously published analytic expressions [J. Opt. Soc. Am. A, 302188 (2013]. The final results show that the increased turbulence outer scale value enlarges the variance of AOA fluctuations greatly under moderate-to-strong (or strong) non-Kolmogorov turbulence.

An end-to-end deep convolutional neural network-based dose engine for parotid gland cancer seed implant brachytherapy.

BACKGROUND: Seed implant brachytherapy (SIBT) is a promising treatment modality for parotid gland cancers (PGCs). However, the current clinical standard dose calculation method based on the American Association of Physicists in Medicine (AAPM) Task Group 43 (TG-43) Report oversimplifies patient anatomy as a homogeneous water phantom medium, leading to significant dose calculation errors due to heterogeneity surrounding the parotid gland. Monte Carlo Simulation (MCS) can yield accurate dose distributions but the long computation time hinders its wide application in clinical practice. PURPOSE: This paper aims to develop an end-to-end deep convolutional neural network-based dose engine (DCNN-DE) to achieve fast and accurate dose calculation for PGC SIBT. METHODS: A DCNN model was trained using the patient's CT images and TG-43-based dose maps as inputs, with the corresponding MCS-based dose maps as the ground truth. The DCNN model was enhanced based on our previously proposed model by incorporating attention gates (AGs) and large kernel convolutions. Training and evaluation of the model were performed using a dataset comprising 188 PGC I-125 SIBT patient cases, and its transferability was tested on an additional 16 non-PGC head and neck cancers (HNCs) I-125 SIBT patient cases. Comparison studies were conducted to validate the superiority of the enhanced model over the original one and compare their overall performance. RESULTS: On the PGC testing dataset, the DCNN-DE demonstrated the ability to generate accurate dose maps, with percentage absolute errors (PAEs) of 0.67% ± 0.47% for clinical target volume (CTV) D90 and 1.04% ± 1.33% for skin D0.1cc. The comparison studies revealed that incorporating AGs and large kernel convolutions resulted in 8.2% (p < 0.001) and 3.1% (p < 0.001) accuracy improvement, respectively, as measured by dose mean absolute error. On the non-PGC HNC dataset, the DCNN-DE exhibited good transferability, achieving a CTV D90 PAE of 1.88% ± 1.73%. The DCNN-DE can generate a dose map in less than 10 ms. CONCLUSIONS: We have developed and validated an end-to-end DCNN-DE for PGC SIBT. The proposed DCNN-DE enables fast and accurate dose calculation, making it suitable for application in the plan optimization and evaluation process of PGC SIBT.

Image Intrinsic-Based Unsupervised Monocular Depth Estimation in Endoscopy.

Unsupervised monocular depth estimation plays a vital role for endoscopy-based minimally invasive surgery (MIS). However, it remains challenging due to the distinctive imaging characteristics of endoscopy which disrupt the assumption of photometric consistency, a foundation relied upon by conventional methods. Distinct from recent approaches taking image pre-processing strategy, this paper introduces a pioneering solution through intrinsic image decomposition (IID) theory. Specifically, we propose a novel end-to-end intrinsic-based unsupervised monocular depth learning framework that is comprised of an image intrinsic decomposition module and a synthesis reconstruction module. This framework seamlessly integrates IID with unsupervised monocular depth estimation, and dedicated losses are meticulously designed to offer robust supervision for network training based on this novel integration. Noteworthy, we rely on the favorable property of the resulting albedo map of IID to circumvent the challenging images characteristics instead of pre-processing the input frames. The proposed method is extensively validated on SCARED and Hamlyn datasets, and better results are obtained than state-of-the-art techniques. Beside, its generalization ability and the effectiveness of the proposed components are also validated. This innovative method has the potential to elevate the quality of 3D reconstruction in monocular endoscopy, thereby enhancing the accuracy and robustness of augmented reality navigation technology in MIS. Our code will be available at: https://github.com/bobo909/IID-SfmLearner.

RT-SRTS: Angle-agnostic real-time simultaneous 3D reconstruction and tumor segmentation from single X-ray projection.

Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a novel imaging method RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) and achieves real-time simultaneous 3D reconstruction and tumor segmentation from a single X-ray projection at any angle. Furthermore, the attention enhanced calibrator (AEC) and uncertain-region elaboration (URE) modules have been proposed to aid feature extraction and improve segmentation accuracy. The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. It not only delivers superior 3D reconstruction but also demonstrates commendable tumor segmentation results. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking. The efficacies of both AEC and URE have also been validated in ablation studies. The code of work is available at https://github.com/ZywooSimple/RT-SRTS.

Automatic planning for head and neck seed implant brachytherapy based on deep convolutional neural network dose engine.

BACKGROUND: Seed implant brachytherapy (SIBT) is an effective treatment modality for head and neck (H&N) cancers; however, current clinical planning requires manual setting of needle paths and utilizes inaccurate dose calculation algorithms. PURPOSE: This study aims to develop an accurate and efficient deep convolutional neural network dose engine (DCNN-DE) and an automatic SIBT planning method for H&N SIBT. METHODS: A cohort of 25 H&N patients who received SIBT was utilized to develop and validate the methods. The DCNN-DE was developed based on 3D-unet model. It takes single seed dose distribution from a modified TG-43 method, the CT image and a novel inter-seed shadow map (ISSM) as inputs, and predicts the dose map of accuracy close to the one from Monte Carlo simulations (MCS). The ISSM was proposed to better handle inter-seed attenuation. The accuracy and efficacy of the DCNN-DE were validated by comparing with other methods taking MCS dose as reference. For SIBT planning, a novel strategy inspired by clinical practice was proposed to automatically generate parallel or non-parallel potential needle paths that avoid puncturing bone and critical organs. A heuristic-based optimization method was developed to optimize the seed positions to meet clinical prescription requirements. The proposed planning method was validated by re-planning the 25 cases and comparing with clinical plans. RESULTS: The absolute percentage error in the TG-43 calculation for CTV V100 and D90 was reduced from 5.4% and 13.2% to 0.4% and 1.1% with DCNN-DE, an accuracy improvement of 93% and 92%, respectively. The proposed planning method could automatically obtain a plan in 2.5 ± 1.5 min. The generated plans were judged clinically acceptable with dose distribution comparable with those of the clinical plans. CONCLUSIONS: The proposed method can generate clinically acceptable plans quickly with high accuracy in dose evaluation, and thus has a high potential for clinical use in SIBT.

Analytical expressions for the angle of arrival fluctuations for optical waves' propagation through moderate-to-strong non-Kolmogorov refractive turbulence.

The effects of moderate-to-strong non-Kolmogorov turbulence on the angle of arrival (AOA) fluctuations for plane and spherical waves are investigated in detail both analytically and numerically. New analytical expressions for the variance of AOA fluctuations are derived for moderate-to-strong non-Kolmogorov turbulence. The new expressions cover a wider range of non-Kolmogorov turbulence strength and reduce correctly to previously published analytic expressions for the cases of plane and spherical wave propagation through both weak non-Kolmogorov turbulence and moderate-to-strong Kolmogorov turbulence cases. The final results indicate that, as turbulence strength becomes greater, the expressions developed with the Rytov theory deviate from those given in this work. This deviation becomes greater with stronger turbulence, up to moderate-to-strong turbulence strengths. Furthermore, general spectral power law has significant influence on the variance of AOA fluctuations in non-Kolmogorov turbulence. These results are useful for understanding the potential impact of deviations from the standard Kolmogorv spectrum.

Transport of intensity phase imaging from multiple noisy intensities measured in unequally-spaced planes.

The noise problem is generally inevitable for phase retrieval by solving the transport of intensity equation (TIE). The noise effect can be alleviated by using multiple intensities to estimate the axial intensity derivative in the TIE. In this study, a method is proposed for estimating the intensity derivative by using multiple unevenly-spaced noisy measurements. The noise-minimized intensity derivative is approximated by a linear combination of the intensity data, in which the coefficients are obtained by solving a constrained optimization problem. The performance of the method is investigated by both the error analysis and the numerical simulations, and the results show that the method can reduce the noise effect on the retrieved phase. In addition, guidelines for the choice of the number of the intensity planes are given.

Atmospheric spectral model and theoretical expressions of irradiance scintillation index for optical wave propagating through moderate-to-strong non-Kolmogorov turbulence.

A new atmospheric spectral model and expressions of irradiance scintillation index are derived theoretically for optical wave propagating through moderate-to-strong non-Kolmogorov turbulence. They are developed under Andrews' assumption that small-scale irradiance fluctuations are modulated by large-scale irradiance fluctuations of the wave, and the geometrical optics approximation is adopted for mathematical development. A wide range of turbulence strength is considered instead of a limited range for weak turbulence. The atmospheric spectral model has a spectral power law value in the range of 3 to 4 instead of the standard power law value of 11/3. Numerical calculations are conducted to analyze the influences of spectral power law and turbulence strength.

Noise-suppressed image enhancement using multiscale top-hat selection transform through region extraction.

Enhancing an image through increasing the contrast of the image is one effective way of image enhancement. To well enhance an image and suppress the produced noises in the resulting image, a multiscale top-hat selection transform-based algorithm through extracting bright and dark image regions and increasing the contrast between them is proposed. First, the multiscale top-hat selection transform is discussed and then is used to extract the bright and dark image regions of each scale. Second, the final extracted bright and dark image regions are obtained through a maximum operation on all the extracted multiscale bright and dark image regions at all scales. Finally, by using a weight strategy, the image is enhanced through increasing the contrast of the image by adding the final bright regions on and subtracting the final dark regions from the original image. The weight parameters are used to adjust the effect of image enhancement. Because the multiscale top-hat selection transform is used to effectively extract the final image regions and discriminate the possible noise regions, the image is well enhanced and some noises are suppressed. Experimental results on different types of images show that our algorithm performs well for noise-suppressed image enhancement and is useful for different applications.

Multiple linear feature detection based on multiple-structuring-element center-surround top-hat transform.

Linear feature detection is an important technique in different applications of image processing. To detect linear features in different types of images, a simple but effective algorithm based on a multiple-structuring-element center-surround top-hat transform is proposed. The center-surround top-hat transform is discussed and analyzed. Based on the properties of this transform for image feature detection, multiple structuring elements are constructed corresponding to the possible linear features at different directions. The whole algorithm is divided into four parts. First, the algorithm uses the center-surround top-hat transform to detect all the possible linear features at different directions through constructing multiple structuring elements. Second, the detected linear feature regions at each direction are processed by a closing operation to remove the possible holes or unconnected regions. Third, the processed results of the detected linear feature regions at all directions are combined to form all the possible detected linear feature regions. Fourth, the combined result is refined by using some simple operations to form the final result. Experimental results on different types of images from different applications verified the effective performance of the proposed algorithm. Moreover, the experimental results indicate that the proposed algorithm could be used in different applications.

Latency structure of BOLD signals within white matter in resting-state fMRI.

PURPOSE: Previous studies have demonstrated that BOLD signals in gray matter in resting-state functional MRI (RSfMRI) have variable time lags, representing apparent propagations of fMRI BOLD signals in gray matter. We complemented existing findings and explored the corresponding variations of signal latencies in white matter. METHODS: We used data from the Brain Genomics Superstruct Project, consisting of 1412 subjects (both sexes included) and divided the dataset into ten equal groups to study both the patterns and reproducibility of latency estimates within white matter. We constructed latency matrices by computing cross-covariances between voxel pairs. We also applied a clustering analysis to identify functional networks within white matter, based on which latency analysis was also performed to investigate lead/lag relationship at network level. A dataset consisting of various sensory states (eyes closed, eyes open and eyes open with fixation) was also included to examine the relationship between latency structure and different states. RESULTS: Projections of voxel latencies from the latency matrices were highly correlated (average Pearson correlation coefficient = 0.89) across the subgroups, confirming the reproducibility and structure of signal lags in white matter. Analysis of latencies within and between networks revealed a similar pattern of inter- and intra-network communication to that reported for gray matter. Moreover, a dominant direction, from inferior to superior regions, of BOLD signal propagation was revealed by higher resolution clustering. The variations of lag structure within white matter are associated with different sensory states. CONCLUSIONS: These findings provide additional insight into the character and roles of white matter BOLD signals in brain functions.

Reorganizations of latency structures within the white matter from wakefulness to sleep.

Previous resting-state functional magnetic resonance imaging (fMRI) studies have revealed highly reproducible latency structures, reflecting the lead/lag relationship of BOLD fMRI signals in white matter (WM). With simultaneous electroencephalography and fMRI data from 35 healthy subjects who were instructed to sleep during imaging, we explored alterations of latency structures in the WM across wakefulness and nonrapid eye movement (NREM) sleep stages. Lagged cross-covariance was computed among voxelwise time series, followed by parabolic interpolation to determine the actual in-between latencies. WM regions, including the brainstem, internal capsule, optic radiation, genu of corpus callosum, and corona radiata, inconsistently changed temporal dynamics with respect to the rest of the WM across wakefulness and NREM sleep stages, as demonstrated when these regions were used as seeds for seed-based latency analysis. Latency analysis of resting-state networks, obtained by applying K-means clustering to a group-level functional connectivity matrix, identified a dominant direction of signaling, starting from the brainstem up to the internal capsule and then the corona radiata during wakefulness, which was reorganized according to stage transitions, e.g., the temporal organization of the internal capsule and corona radiata switched from unidirectional to bidirectional in the wakefulness to N3 transition. These findings suggest that WM BOLD signals are slow, dynamically modulated across wakefulness and NREM sleep stages and that they are involved in maintaining different levels of consciousness.