Weakly-supervised convolutional neural networks of renal tumor segmentation in abdominal CTA images.

BACKGROUND: Renal cancer is one of the 10 most common cancers in human beings. The laparoscopic partial nephrectomy (LPN) is an effective way to treat renal cancer. Localization and delineation of the renal tumor from pre-operative CT Angiography (CTA) is an important step for LPN surgery planning. Recently, with the development of the technique of deep learning, deep neural networks can be trained to provide accurate pixel-wise renal tumor segmentation in CTA images. However, constructing the training dataset with a large amount of pixel-wise annotations is a time-consuming task for the radiologists. Therefore, weakly-supervised approaches attract more interest in research. METHODS: In this paper, we proposed a novel weakly-supervised convolutional neural network (CNN) for renal tumor segmentation. A three-stage framework was introduced to train the CNN with the weak annotations of renal tumors, i.e. the bounding boxes of renal tumors. The framework includes pseudo masks generation, group and weighted training phases. Clinical abdominal CT angiographic images of 200 patients were applied to perform the evaluation. RESULTS: Extensive experimental results show that the proposed method achieves a higher dice coefficient (DSC) of 0.826 than the other two existing weakly-supervised deep neural networks. Furthermore, the segmentation performance is close to the fully supervised deep CNN. CONCLUSIONS: The proposed strategy improves not only the efficiency of network training but also the precision of the segmentation.

Measurement and Analysis of Lobar Lung Deformation After a Change of Patient Position During Video-Assisted Thoracoscopic Surgery.

Video-assisted thoracoscopic surgery (VATS) is a minimally invasive surgical technique for the diagnosis and treatment of early-stage lung cancer. During VATS, large lung deformation occurs as a result of a change of patient position and a pneumothorax (lung deflation), which hinders the intraoperative localization of pulmonary nodules. Modeling lung deformation during VATS for surgical navigation is desirable, but the mechanisms causing such deformation are yet not well-understood. In this study, we estimate, quantify and analyze the lung deformation occurring after a change of patient position during VATS. We used deformable image registration to estimate the lung deformation between a preoperative CT (in supine position) and an intraoperative CBCT (in lateral decubitus position) of six VATS clinical cases. We accounted for sliding motion between lobes and against the thoracic wall and obtained consistently low average target registration errors (under 1 mm). We observed large lung displacement (up to 40 mm); considerable sliding motion between lobes and against the thoracic wall (up to 30 mm); and localized volume changes indicating deformation. These findings demonstrate the complexity of the change of patient position phenomenon, which should necessarily be taken into account to model lung deformation for intraoperative guidance during VATS.

A multiple catheter tips tracking method in X-ray fluoroscopy images by a new lightweight segmentation network and Bayesian filtering.

During percutaneous coronary intervention, the guiding catheter plays an important role. Tracking the catheter tip placed at the coronary ostium in the X-ray fluoroscopy sequence can obtain image displacement information caused by the heart beating, which can help dynamic coronary roadmap overlap on X-ray fluoroscopy images. Due to a low exposure dose, the X-ray fluoroscopy is noisy and low contrast, which causes some difficulties in tracking. In this paper, we developed a new catheter tip tracking framework. First, a lightweight efficient catheter tip segmentation network is proposed and boosted by a self-distillation training mechanism. Then, the Bayesian filtering post-processing method is used to consider the sequence information to refine the single image segmentation results. By separating the segmentation results into several groups based on connectivity, our framework can track multiple catheter tips. The proposed tracking framework is validated on a clinical X-ray sequence dataset.

Development of a Numerical Model of High-Intensity Focused Ultrasound Treatment in Mobile and Elastic Organs: Application to a Beating Heart.

High-intensity focused ultrasound (HIFU) is a promising method used to treat cardiac arrhythmias, as it can induce lesions at a distance throughout myocardium thickness. Numerical modeling is commonly used for ultrasound probe development and optimization of HIFU treatment strategies. This study was aimed at describing a numerical method to simulate HIFU thermal ablation in elastic and mobile heart models. The ultrasound pressure field is computed on a 3-D orthonormal grid using the Rayleigh integral method, and the attenuation is calculated step by step between cells. The temperature distribution is obtained by resolution of the bioheat transfer equation on a 3-D non-orthogonally structured curvilinear grid using the finite-volume method. The simulation method is applied on two regions of the heart (atrioventricular node and ventricular apex) to compare the thermal effects of HIFU ablation depending on deformation, motion type and amplitude. The atrioventricular node requires longer sonication than the ventricular apex to reach the same lesion volume. Motion considerably influences treatment duration, lesion shape and distribution in cardiac HIFU treatment. These results emphasize the importance of considering local motion and deformation in numerical studies to define efficient and accurate treatment strategies.

Automatic segmentation for plan-of-the-day selection in CBCT-guided adaptive radiation therapy of cervical cancer.

Objective.Plan-of-the-day (PoD) adaptive radiation therapy (ART) is based on a library of treatment plans, among which, at each treatment fraction, the PoD is selected using daily images. However, this strategy is limited by PoD selection uncertainties. This work aimed to propose and evaluate a workflow to automatically and quantitatively identify the PoD for cervix cancer ART based on daily CBCT images.Approach.The quantification was based on the segmentation of the main structures of interest in the CBCT images (clinical target volume [CTV], rectum, bladder, and bowel bag) using a deep learning model. Then, the PoD was selected from the treatment plan library according to the geometrical coverage of the CTV. For the evaluation, the resulting PoD was compared to the one obtained considering reference CBCT delineations.Main results.In experiments on a database of 23 patients with 272 CBCT images, the proposed method obtained an agreement between the reference PoD and the automatically identified PoD for 91.5% of treatment fractions (99.6% when considering a 5% margin on CTV coverage).Significance.The proposed automatic workflow automatically selected PoD for ART using deep-learning methods. The results showed the ability of the proposed process to identify the optimal PoD in a treatment plan library.

Interest of the minimum edit distance to detect behaviour change of the elderly person.

In this article, a solution to detect the change of behaviour of the elderly person based on the person's activities of daily living is proposed. This work is based on the hypothesis that the person attaches importance to a rhythmic sequence of days and activities per day. The day of the elderly person is described by a succession of activities, and each activity is associated to a posture (lying down, sitting, standing, absent). Postures are estimated from image analysis measured by thermal or depth cameras in order to preserve the anonymity of the person. The change in posture succession is calculated using the minimum edit distance with respect to the routine day. The number of permutations/inversions reflects the change in the person's behaviour. The method was tested on two elderly persons recorded by thermal and depth cameras during 85 days in a retirement home. It is shown that for a person with a life change behaviour, the average number of permutations and interquartile range, before and after changes, are 41 [28], [48] and 57 [55-62] respectively compared to the learned routine day. The Wilcoxon test confirmed the significant difference between these two periods.Clinical Relevance- Monitoring the daily routine provides indicators for detecting changes in the behaviour of an elderly person.

Detection of changes in the behaviour of the elderly person.

In this paper, we propose a solution for detecting changes in the behaviour of the elderly person based on the monitoring of activities of daily living (ADL). The elderly person's daily routine is characterized by the following five indexes: 1) percentage of time lying down, 2) percentage of time sitting, 3) percentage of time standing, 4) percentage of time absent from home, and 5) number of falls during the day. In our framework, these indexes are computed using characteristics extracted from depth and thermal data. We hypothesize that elderly persons have a well-defined, regular life routine, organized around their environment, habits, and social relations. Then, given the indexes values, a day is defined as routine or non-routine day. Thus, looking for changes of day type allows to detect changes in a person's routine. The method has been tested on a database of depth and thermal images recorded in a nursing home over an 85 days period. These tests proved the reliability of the proposed method.

A hybrid, image-based and biomechanics-based registration approach to markerless intraoperative nodule localization during video-assisted thoracoscopic surgery.

The resection of small, low-dense or deep lung nodules during video-assisted thoracoscopic surgery (VATS) is surgically challenging. Nodule localization methods in clinical practice typically rely on the preoperative placement of markers, which may lead to clinical complications. We propose a markerless lung nodule localization framework for VATS based on a hybrid method combining intraoperative cone-beam CT (CBCT) imaging, free-form deformation image registration, and a poroelastic lung model with allowance for air evacuation. The difficult problem of estimating intraoperative lung deformations is decomposed into two more tractable sub-problems: (i) estimating the deformation due the change of patient pose from preoperative CT (supine) to intraoperative CBCT (lateral decubitus); and (ii) estimating the pneumothorax deformation, i.e. a collapse of the lung within the thoracic cage. We were able to demonstrate the feasibility of our localization framework with a retrospective validation study on 5 VATS clinical cases. Average initial errors in the range of 22 to 38 mm were reduced to the range of 4 to 14 mm, corresponding to an error correction in the range of 63 to 85%. To our knowledge, this is the first markerless lung deformation compensation method dedicated to VATS and validated on actual clinical data.

Meta grayscale adaptive network for 3D integrated renal structures segmentation.

Three-dimensional (3D) integrated renal structures (IRS) segmentation targets segmenting the kidneys, renal tumors, arteries, and veins in one inference. Clinicians will benefit from the 3D IRS visual model for accurate preoperative planning and intraoperative guidance of laparoscopic partial nephrectomy (LPN). However, no success has been reported in 3D IRS segmentation due to the inherent challenges in grayscale distribution: low contrast caused by the narrow task-dependent distribution range of regions of interest (ROIs), and the networks representation preferences caused by the distribution variation inter-images. In this paper, we propose the Meta Greyscale Adaptive Network (MGANet), the first deep learning framework to simultaneously segment the kidney, renal tumors, arteries and veins on CTA images in one inference. It makes innovations in two collaborate aspects: 1) The Grayscale Interest Search (GIS) adaptively focuses segmentation networks on task-dependent grayscale distributions via scaling the window width and center with two cross-correlated coefficients for the first time, thus learning the fine-grained representation for fine segmentation. 2) The Meta Grayscale Adaptive (MGA) learning makes an image-level meta-learning strategy. It represents diverse robust features from multiple distributions, perceives the distribution characteristic, and generates the model parameters to fuse features dynamically according to image's distribution, thus adapting the grayscale distribution variation. This study enrolls 123 patients and the average Dice coefficients of the renal structures are up to 87.9%. Fine selection of the task-dependent grayscale distribution ranges and personalized fusion of multiple representations on different distributions will lead to better 3D IRS segmentation quality. Extensive experiments with promising results on renal structures reveal powerful segmentation accuracy and great clinical significance in renal cancer treatment.

[From medical imaging to image-guided therapy]. / Représentation en sciences du vivant (3). De l'imagerie médicale à la thérapie guidée par l'image.

This survey on medical imaging provides a look into three major components. The first one deals with the full steps through which it must be apprehended: from the sensors to the reconstruction, from the image analysis up to its interpretation. The second aspect describes the physical principles used for imaging (magnetic resonance, acoustic, optics, etc.). The last section shows how imaging is involved in therapeutic procedures and in particular the new physical therapies. All along this paper, the research perspectives are sketched.

Deformable US/CT Image Registration with a Convolutional Neural Network for Cardiac Arrhythmia Therapy.

Image registration represents one of the fundamental techniques in medical imaging and image-guided interventions. In this paper, we present a Convolutional Neural Network (CNN) framework for deformable transesophageal US/CT image registration, for the cardiac arrhythmias, and guidance therapy purposes. The framework consists of a CNN, a spatial transformer, and a resampler. The CNN expects concatenated pairs of moving and fixed images as its input, and estimates as output the parameters for the spatial transformer, which generates the displacement vector field that allows the resampler to wrap the moving image into the fixed image. In our method, we train the model to maximize standard image matching objective functions that are based on the image intensities. The network can be applied to perform non-rigid registration of a pair of CT/US images directly in one pass, avoiding so the time consuming computation of the classical iterative method.

Fast and accurate segmentation method of active shape model with Rayleigh mixture model clustering for prostate ultrasound images.

BACKGROUND AND OBJECTIVE: The prostate cancer interventions, which need an accurate prostate segmentation, are performed under ultrasound imaging guidance. However, prostate ultrasound segmentation is facing two challenges. The first is the low signal-to-noise ratio and inhomogeneity of the ultrasound image. The second is the non-standardized shape and size of the prostate. METHODS: For prostate ultrasound image segmentation, this paper proposed an accurate and efficient method of Active shape model (ASM) with Rayleigh mixture model Clustering (ASM-RMMC). Firstly, Rayleigh mixture model (RMM) is adopted for clustering the image regions which present similar speckle distributions. These content-based clustered images are then used to initialize and guide the deformation of an ASM model. RESULTS: The performance of the proposed method is assessed on 30 prostate ultrasound images using four metrics as Mean Average Distance (MAD), Dice Similarity Coefficient (DSC), False Positive Error (FPE) and False Negative Error (FNE). The proposed ASM-RMMC reaches high segmentation accuracy with 95% ± 0.81% for DSC, 1.86 ±â€¯0.02 pixels for MAD, 2.10% ± 0.36% for FPE and 2.78% ± 0.71% for FNE, respectively. Moreover, the average segmentation time is less than 8 s when treating a single prostate ultrasound image through ASM-RMMC. CONCLUSIONS: This paper presents a method for prostate ultrasound image segmentation, which achieves high accuracy with less computational complexity and meets the clinical requirements.

HIFUNet: Multi-Class Segmentation of Uterine Regions From MR Images Using Global Convolutional Networks for HIFU Surgery Planning.

Accurate segmentation of uterus, uterine fibroids, and spine from MR images is crucial for high intensity focused ultrasound (HIFU) therapy but remains still difficult to achieve because of 1) the large shape and size variations among individuals, 2) the low contrast between adjacent organs and tissues, and 3) the unknown number of uterine fibroids. To tackle this problem, in this paper, we propose a large kernel Encoder-Decoder Network based on a 2D segmentation model. The use of this large kernel can capture multi-scale contexts by enlarging the valid receptive field. In addition, a deep multiple atrous convolution block is also employed to enlarge the receptive field and extract denser feature maps. Our approach is compared to both conventional and other deep learning methods and the experimental results conducted on a large dataset show its effectiveness.

Transrectal ultrasound image-based real-time augmented reality guidance in robot-assisted laparoscopic rectal surgery: a proof-of-concept study.

PURPOSE: Surgical treatments for low-rectal cancer require careful considerations due to the low location of cancer in rectums. Successful surgical outcomes highly depend on surgeons' ability to determine clear distal margins of rectal tumors. This is a challenge for surgeons in robot-assisted laparoscopic surgery, since tumors are often concealed in rectums and robotic surgical instruments do not provide tactile feedback for tissue diagnosis in real time. This paper presents the development and evaluation of an intraoperative ultrasound-based augmented reality framework for surgical guidance in robot-assisted rectal surgery. METHODS: Framework implementation consists in calibrating the transrectal ultrasound (TRUS) and the endoscopic camera (hand-eye calibration), generating a virtual model and registering it to the endoscopic image via optical tracking, and displaying the augmented view on a head-mounted display. An experimental validation setup is designed to evaluate the framework. RESULTS: The evaluation process yields a mean error of 0.9 mm for the TRUS calibration, a maximum error of 0.51 mm for the hand-eye calibration of endoscopic cameras, and a maximum RMS error of 0.8 mm for the whole framework. In the experiment with a rectum phantom, our framework guides the surgeon to accurately localize the simulated tumor and the distal resection margin. CONCLUSIONS: This framework is developed with our clinical partner, based on actual clinical conditions. The experimental protocol and the high level of accuracy show the feasibility of seamlessly integrating this framework within the surgical workflow.

Dense biased networks with deep priori anatomy and hard region adaptation: Semi-supervised learning for fine renal artery segmentation.

Fine renal artery segmentation on abdominal CT angiography (CTA) image is one of the most important tasks for kidney disease diagnosis and pre-operative planning. It will help clinicians locate each interlobar artery's blood-feeding region via providing the complete 3D renal artery tree masks. However, it is still a task of great challenges due to the large intra-scale changes, large inter-anatomy variation, thin structures, small volume ratio and small labeled dataset of the fine renal artery. In this paper, we propose the first semi-supervised 3D fine renal artery segmentation framework, DPA-DenseBiasNet, which combines deep prior anatomy (DPA), dense biased network (DenseBiasNet) and hard region adaptation loss (HRA): 1) Based on our proposed dense biased connection, the DenseBiasNet fuses multi-receptive field and multi-resolution feature maps for large intra-scale changes. This dense biased connection also obtains a dense information flow and dense gradient flow so that the training is accelerated and the accuracy is enhanced. 2) DPA features extracted from an autoencoder (AE) are embedded in DenseBiasNet to cope with the challenge of large inter-anatomy variation and thin structures. The AE is pre-trained (unsupervised) by numerous unlabeled data to achieve the representation ability of anatomy features and these features are embedded in DenseBiasNet. This process will not introduce incorrect labels as optimization targets and thus contributes to a stable semi-supervised training strategy that is suitable for sensitive thin structures. 3) The HRA selects the loss value calculation region dynamically according to the segmentation quality so the network will pay attention to the hard regions in the training process and keep the class balanced. Experiments demonstrated that DPA-DenseBiasNet had high predictive accuracy and generalization with the Dice coefficient of 0.884 which increased by 0.083 compared with 3D U-Net (Çiçek et al., 2016). This revealed our framework with great potential for the 3D fine renal artery segmentation in clinical practice.

Fast And Simple Automatic 3D Ultrasound Probe Calibration Based On 3D Printed Phantom And An Untracked Marker.

Tracking the pose of an ultrasound (US) probe is essential for an intraoperative US-based navigation system. The tracking requires mounting a marker on the US probe and calibrating the probe. The goal of the US probe calibration is to determine the rigid transformation between the coordinate system (CS) of the image and the CS of the marker mounted on the probe. We present a fast and automatic calibration method based on a 3D printed phantom and an untracked marker for three-dimensional (3D) US probe calibration. To simplify the conventional calibration procedures using and tracking at least two markers, we used only one marker and did not track it in the whole calibration process. Our automatic calibration method is fast, simple and does not require any experience from the user. The performance of our calibration method was evaluated by point reconstruction tests. The root mean square (RMS) of the point reconstruction errors was 1.39 mm.

Transesophageal 2D ultrasound to 3D computed tomography registration for the guidance of a cardiac arrhythmia therapy.

The work aims to develop a new image-processing method to improve the guidance of transesophageal high intensity focused ultrasound (HIFU) atrial fibrillation therapy. Our proposal is a novel registration approach that aligns intraoperative 2D ultrasound with preoperative 3D-CT information. This approach takes advantage of the anatomical constraints imposed at the transesophageal HIFU probe to simplify the registration process. Our proposed method has been evaluated on a physical phantom and on real clinical data.

Spherical harmonics based intrasubject 3-D kidney modeling/registration technique applied on partial information.

This paper presents a three-dimensional (3-D) shape reconstruction/intrapatient rigid registration technique used to establish a Nephron-Sparing Surgery preoperative planning. The usual preoperative imaging system is the Spiral CT Urography, which provides successive 3-D acquisitions of complementary information on kidney anatomy. Because the kidney is difficult to demarcate from the liver or from the spleen only limited information on its volume or surface is available. In this paper, we propose a methodology allowing a global kidney spatial representation on a spherical harmonics basis. The spherical harmonics are exploited to recover the kidney 3-D shape and also to perform intrapatient 3-D rigid registration. An evaluation performed on synthetic data showed that this technique presented lower performance then expected for the 3-D shape recovering but exhibited registration results slightly more accurate as the iterative closest point technique with faster computation time.

A new stationary gridline artifact suppression method based on the 2D discrete wavelet transform.

PURPOSE: In digital x-ray radiography, an antiscatter grid is inserted between the patient and the image receptor to reduce scattered radiation. If the antiscatter grid is used in a stationary way, gridline artifacts will appear in the final image. In most of the gridline removal image processing methods, the useful information with spatial frequencies close to that of the gridline is usually lost or degraded. In this study, a new stationary gridline suppression method is designed to preserve more of the useful information. METHODS: The method is as follows. The input image is first recursively decomposed into several smaller subimages using a multiscale 2D discrete wavelet transform. The decomposition process stops when the gridline signal is found to be greater than a threshold in one or several of these subimages using a gridline detection module. An automatic Gaussian band-stop filter is then applied to the detected subimages to remove the gridline signal. Finally, the restored image is achieved using the corresponding 2D inverse discrete wavelet transform. RESULTS: The processed images show that the proposed method can remove the gridline signal efficiently while maintaining the image details. The spectra of a 1D Fourier transform of the processed images demonstrate that, compared with some existing gridline removal methods, the proposed method has better information preservation after the removal of the gridline artifacts. Additionally, the performance speed is relatively high. CONCLUSIONS: The experimental results demonstrate the efficiency of the proposed method. Compared with some existing gridline removal methods, the proposed method can preserve more information within an acceptable execution time.

Benchmark for Algorithms Segmenting the Left Atrium From 3D CT and MRI Datasets.

Knowledge of left atrial (LA) anatomy is important for atrial fibrillation ablation guidance, fibrosis quantification and biophysical modelling. Segmentation of the LA from Magnetic Resonance Imaging (MRI) and Computed Tomography (CT) images is a complex problem. This manuscript presents a benchmark to evaluate algorithms that address LA segmentation. The datasets, ground truth and evaluation code have been made publicly available through the http://www.cardiacatlas.org website. This manuscript also reports the results of the Left Atrial Segmentation Challenge (LASC) carried out at the STACOM'13 workshop, in conjunction with MICCAI'13. Thirty CT and 30 MRI datasets were provided to participants for segmentation. Each participant segmented the LA including a short part of the LA appendage trunk and proximal sections of the pulmonary veins (PVs). We present results for nine algorithms for CT and eight algorithms for MRI. Results showed that methodologies combining statistical models with region growing approaches were the most appropriate to handle the proposed task. The ground truth and automatic segmentations were standardised to reduce the influence of inconsistently defined regions (e.g., mitral plane, PVs end points, LA appendage). This standardisation framework, which is a contribution of this work, can be used to label and further analyse anatomical regions of the LA. By performing the standardisation directly on the left atrial surface, we can process multiple input data, including meshes exported from different electroanatomical mapping systems.