Semi-automated three-dimensional segmentation for cardiac CT images using deep learning and randomly distributed points.

Given the prevalence of cardiovascular diseases (CVDs), the segmentation of the heart on cardiac computed tomography (CT) remains of great importance. Manual segmentation is time-consuming and intra-and inter-observer variabilities yield inconsistent and inaccurate results. Computer-assisted, and in particular, deep learning approaches to segmentation continue to potentially offer an accurate, efficient alternative to manual segmentation. However, fully automated methods for cardiac segmentation have yet to achieve accurate enough results to compete with expert segmentation. Thus, we focus on a semi-automated deep learning approach to cardiac segmentation that bridges the divide between a higher accuracy from manual segmentation and higher efficiency from fully automated methods. In this approach, we selected a fixed number of points along the surface of the cardiac region to mimic user interaction. Points-distance maps were then generated from these points selections, and a three-dimensional (3D) fully convolutional neural network (FCNN) was trained using points-distance maps to provide a segmentation prediction. Testing our method with different numbers of selected points, we achieved a Dice score from 0.742 to 0.917 across the four chambers. Specifically. Dice scores averaged 0.846 ± 0.059, 0.857 ± 0.052, 0.826 ± 0.062, and 0.824 ± 0.062 for the left atrium, left ventricle, right atrium, and right ventricle, respectively across all points selections. This point-guided, image-independent, deep learning segmentation approach illustrated a promising performance for chamber-by-chamber delineation of the heart in CT images.

A low-cost PVC-based dual-modality kidney phantom.

Phantoms are invaluable tools broadly used for research and training purposes designed to mimic tissues and structures in the body. In this paper, polyvinyl chloride (PVC)-plasticizer and silicone rubbers were explored as economical materials to reliably create long-lasting, realistic kidney phantoms with contrast under both ultrasound (US) and X-ray imaging. The radiodensity properties of varying formulations of soft PVC-based gels were characterized to allow adjustable image intensity and contrast. Using this data, a phantom creation workflow was established which can be easily adapted to match radiodensity values of other organs and soft tissues in the body. Internal kidney structures such as the medulla and ureter were created using a two-part molding process to allow greater phantom customization. The kidney phantoms were imaged under US and X-ray scanners to compare the contrast enhancement of a PVC-based medulla versus a silicone-based medulla. Silicone was found to have higher attenuation than plastic under X-ray imaging, but poor quality under US imaging. PVC was found to exhibit good contrast under X-ray imaging and excellent performance for US imaging. Finally, the durability and shelf life of our PVC-based phantoms were observed to be vastly superior to that of common agar-based phantoms. The work presented here allows extended periods of usage and storage for each kidney phantom while simultaneously preserving anatomical detail, contrast under dual-modality imaging, and low cost of materials.

An Augmented Reality-Assisted Visualization System for Potential Applications in Prostate Biopsy.

Ultrasound-guided biopsy is widely used for disease detection and diagnosis. We plan to register preoperative imaging, such as positron emission tomography / computed tomography (PET/CT) and/or magnetic resonance imaging (MRI), with real-time intraoperative ultrasound imaging for improved localization of suspicious lesions that may not be seen on ultrasound but visible on other imaging modalities. Once the image registration is completed, we will combine the images from two or more imaging modalities and use Microsoft HoloLens 2 augmented reality (AR) headset to display three-dimensional (3D) segmented lesions and organs from previously acquired images and real-time ultrasound images. In this work, we are developing a multi-modal, 3D augmented reality system for the potential use in ultrasound-guided prostate biopsy. Preliminary results demonstrate the feasibility of combining images from multiple modalities into an AR-guided system.

Identifying Unique Acoustic Signatures from Chemically-Crosslinked Microbubble Clusters Using Deep Learning.

Ultrasound contrast agents (UCA) are gas encapsulated microspheres that oscillate volumetrically when exposed to an ultrasound field producing a backscattered signal which can be used for improved ultrasound imaging and drug delivery. UCA's are being used widely for contrast-enhanced ultrasound imaging, but there is a need for improved UCAs to develop faster and more accurate contrast agent detection algorithms. Recently, we introduced a new class of lipid based UCAs called Chemically Cross-linked Microbubble Clusters (CCMCs). CCMCs are formed by the physical tethering of individual lipid microbubbles into a larger aggregate cluster. The advantages of these novel CCMCs are their ability to fuse together when exposed to low intensity pulsed ultrasound (US), potentially generating unique acoustic signatures that can enable better contrast agent detection. In this study, our main objective is to demonstrate that the acoustic response of CCMCs is unique and distinct when compared to individual UCAs using deep learning algorithms. Acoustic characterization of CCMCs and individual bubbles was performed using a broadband hydrophone or a clinical transducer attached to a Verasonics Vantage 256. A simple artificial neural network (ANN) was trained and used to classify raw 1D RF ultrasound data as either from CCMC or non-tethered individual bubble populations of UCAs. The ANN was able to classify CCMCs at an accuracy of 93.8% for data collected from broadband hydrophone and 90% for data collected using Verasonics with a clinical transducer. The results obtained suggest the acoustic response of CCMCs is unique and has the potential to be used in developing a novel contrast agent detection technique.

LED-based Hyperspectral Endoscopic Imaging.

Hyperspectral endoscopy can offer multiple advantages as compared to conventional endoscopy. Our goal is to design and develop a real-time hyperspectral endoscopic imaging system for the diagnosis of gastrointestinal (GI) tract cancers using a micro-LED array as an in-situ illumination source. The wavelengths of the system range from ultraviolet to visible and near infrared. To evaluate the use of the LED array for hyperspectral imaging, we designed a prototype system and conducted ex vivo experiments using normal and cancerous tissues of mice, chicken, and sheep. We compared the results of our LED-based approach with our reference hyperspectral camera system. The results confirm the similarity between the LED-based hyperspectral imaging system and the reference HSI camera. Our LED-based hyperspectral imaging system can be used not only as an endoscope but also as a laparoscopic or handheld devices for cancer detection and surgery.

Automatic Segmentation of Uterine Cavity and Placenta on MR Images Using Deep Learning.

Magnetic resonance imaging (MRI) is useful for the detection of abnormalities affecting maternal and fetal health. In this study, we used a fully convolutional neural network for simultaneous segmentation of the uterine cavity and placenta on MR images. We trained the network with MR images of 181 patients, with 157 for training and 24 for validation. The segmentation performance of the algorithm was evaluated using MR images of 60 additional patients that were not involved in training. The average Dice similarity coefficients achieved for the uterine cavity and placenta were 92% and 80%, respectively. The algorithm could estimate the volume of the uterine cavity and placenta with average errors of less than 1.1% compared to manual estimations. Automated segmentation, when incorporated into clinical use, has the potential to quantify, standardize, and improve placental assessment, resulting in improved outcomes for mothers and fetuses.

CascadeNet for hysterectomy prediction in pregnant women due to placenta accreta spectrum.

In severe cases, placenta accreta spectrum (PAS) requires emergency hysterectomy, endangering the life of both mother and fetus. Early prediction may reduce complications and aid in management decisions in these high-risk pregnancies. In this work, we developed a novel convolutional network architecture to combine MRI volumes, radiomic features, and custom feature maps to predict PAS severe enough to result in hysterectomy after fetal delivery in pregnant women. We trained, optimized, and evaluated the networks using data from 241 patients, in groups of 157, 24, and 60 for training, validation, and testing, respectively. We found the network using all three paths produced the best performance, with an AUC of 87.8, accuracy 83.3%, sensitivity of 85.0, and specificity of 82.5. This deep learning algorithm, deployed in clinical settings, may identify women at risk before birth, resulting in improved patient outcomes.

The Contribution of Thoracic Radiation Dose Volumes to Subsequent Development of Cardiovascular Disease in Cancer Survivors.

BACKGROUND: Although radiation therapy (RT) has been recognized for contributing to cardiovascular disease (CVD), it is unknown whether specific doses received by cardiovascular tissues influence development. OBJECTIVE: In this pilot study, we examined the contribution of RT dose distribution on the development of CVD events in patients with cancer within 5 years of RT. METHODS: A retrospective case-controlled design was used matching 28 cases receiving thoracic RT who subsequently developed an adverse CVD event with 28 controls based upon age, gender, and cancer type. Dose volume histograms of nongated computed tomography scans received during RT characterized the dose delivered to the heart. Heart chambers were segmented using an atlas approach, and radiomics features for the segmentation as well as planning dose in each chamber were tabulated for analysis. RESULT: No significant differences were observed in the RT dose statistics between groups, preexisting CVD, nor significant differences of RT doses delivered to distinct chambers of the heart. Cases were found to have greater CVD risk factors at the time of cancer diagnosis. Morphological significant differences for perimeter on border ( P = .043), equivalent spherical radius ( P = .050), and elongation ( P = .038) were observed, with preexisting CVD having the highest values (ie, larger hearts). CONCLUSION: Traditional CVD risk factors were more prevalent in the cases who developed CVD. No differences were observed in doses of RT. Of note, we observed significant differences in heart morphology and mass in known diseased hearts on the pretreatment scans. These new metrics may have implications for the measurement and quantification of CVD.

Technical note: The effect of image annotation with minimal manual interaction for semiautomatic prostate segmentation in CT images using fully convolutional neural networks.

PURPOSE: The goal is to study the performance improvement of a deep learning algorithm in three-dimensional (3D) image segmentation through incorporating minimal user interaction into a fully convolutional neural network (CNN). METHODS: A U-Net CNN was trained and tested for 3D prostate segmentation in computed tomography (CT) images. To improve the segmentation accuracy, the CNN's input images were annotated with a set of border landmarks to supervise the network for segmenting the prostate. The network was trained and tested again with annotated images after 5, 10, 15, 20, or 30 landmark points were used. RESULTS: Compared to fully automatic segmentation, the Dice similarity coefficient increased up to 9% when 5-30 sparse landmark points were involved, with the segmentation accuracy improving as more border landmarks were used. CONCLUSIONS: When a limited number of sparse border landmarks are used on the input image, the CNN performance approaches the interexpert observer difference observed in manual segmentation.

Placenta Accreta Spectrum and Hysterectomy Prediction Using MRI Radiomic Features.

In women with placenta accreta spectrum (PAS), patient management may involve cesarean hysterectomy at delivery. Magnetic resonance imaging (MRI) has been used for further evaluation of PAS and surgical planning. This work tackles two prediction problems: predicting presence of PAS and predicting hysterectomy using MR images of pregnant patients. First, we extracted approximately 2,500 radiomic features from MR images with two regions of interest: the placenta and the uterus. In addition to analyzing two regions of interest, we dilated the placenta and uterus masks by 5, 10, 15, and 20 mm to gain insights from the myometrium, where the uterus and placenta overlap in the case of PAS. This study cohort includes 241 pregnant women. Of these women, 89 underwent hysterectomy while 152 did not; 141 with suspected PAS, and 100 without suspected PAS. We obtained an accuracy of 0.88 for predicting hysterectomy and an accuracy of 0.92 for classifying suspected PAS. The radiomic analysis tool is further validated, it can be useful for aiding clinicians in decision making on the care of pregnant women.

Deep learning-based segmentation of the placenta and uterus on MR images.

Purpose: Magnetic resonance imaging has been recently used to examine the abnormalities of the placenta during pregnancy. Segmentation of the placenta and uterine cavity allows quantitative measures and further analyses of the organs. The objective of this study is to develop a segmentation method with minimal user interaction. Approach: We developed a fully convolutional neural network (CNN) for simultaneous segmentation of the uterine cavity and placenta in three dimensions (3D) while a minimal operator interaction was incorporated for training and testing of the network. The user interaction guided the network to localize the placenta more accurately. In the experiments, we trained two CNNs, one using 70 normal training cases and the other using 129 training cases including normal cases as well as cases with suspected placenta accreta spectrum (PAS). We evaluated the performance of the segmentation algorithms on two test sets: one with 20 normal cases and the other with 50 images from both normal women and women with suspected PAS. Results: For the normal test data, the average Dice similarity coefficient (DSC) was 92% and 82% for the uterine cavity and placenta, respectively. For the combination of normal and abnormal cases, the DSC was 88% and 83% for the uterine cavity and placenta, respectively. The 3D segmentation algorithm estimated the volume of the normal and abnormal uterine cavity and placenta with average volume estimation errors of 4% and 9%, respectively. Conclusions: The deep learning-based segmentation method provides a useful tool for volume estimation and analysis of the placenta and uterus cavity in human placental imaging.

Virtual Reality Assisted Cardiac Catheterization.

Cardiac catheterization is a delicate strategy often used during various heart procedures. However, the procedure carries a myriad of risks associated with it, including damage to the vessel or heart itself, blood clots, and arrhythmias. Many of these risks increase in probability as the length of the operation increases, creating a demand for a more accurate procedure while reducing the overall time required. To this end, we developed an adaptable virtual reality simulation and visualization method to provide essential information to the physician ahead of time with the goal of reducing potential risks, decreasing operation time, and improving the accuracy of cardiac catheterization procedures. We additionally conducted a phantom study to evaluate the impact of using our virtual reality system prior to a procedure.

Development of a real-time spectral imaging system using in-site micro-LED-based illumination and high-speed micro-camera for endoscopic applications.

We designed a compact, real-time LED-based endoscopic imaging system for the detection of various diseases including cancer. In gastrointestinal applications, conventional endoscopy cannot reliably differentiate tumor from normal tissue. Current hyperspectral imaging systems are too slow to be used for real-time endoscopic applications. We are investigating real-time spectral imaging for different tissue types. Our objective is to develop a catheter for real-time hyperspectral gastrointestinal endoscopy. The endoscope uses multiple wavelengths within UV, visible, and IR light spectra generated by a micro-LED array. We capture images with a monochrome micro camera, which is cost-effective and smaller than the current hyperspectral imagers. A wireless transceiver sends the captured images to a workstation for further processing, such as tumor detection. The spatial resolution of the system is defined by camera resolution and the distance to the object, while the number of LEDs in the multi-wavelength light source determines the spectral resolution. To investigate the properties and the limitations of our high-speed spectral imaging approach, we designed a prototype system. We conducted two experiments to measure the optimal forward voltages and lighting duration of the LEDs. These factors affect the maximum feasible imaging rate and resolution. The lighting duration of each LED can be shorter than 10 ms while producing an image with a high signal-to-noise ratio and no illumination interference. These results support the idea of using a high-speed camera and an LED-array for real-time hyperspectral endoscopic imaging.

Assessing reproducibility in Magnetic Resonance (MR) Radiomics features between Deep-Learning segmented and Expert Manual segmented data and evaluating their diagnostic performance in Pregnant Women with suspected Placenta Accreta Spectrum (PAS).

A Deep-Learning (DL) based segmentation tool was applied to a new magnetic resonance imaging dataset of pregnant women with suspected Placenta Accreta Spectrum (PAS). Radiomic features from DL segmentation were compared to those from expert manual segmentation via intraclass correlation coefficients (ICC) to assess reproducibility. An additional imaging marker quantifying the placental location within the uterus (PLU) was included. Features with an ICC > 0.7 were used to build logistic regression models to predict hysterectomy. Of 2059 features, 781 (37.9%) had ICC >0.7. AUC was 0.69 (95% CI 0.63-0.74) for manually segmented data and 0.78 (95% CI 0.73-0.83) for DL segmented data.

Development of a new polarized hyperspectral imaging microscope.

In this study, we proposed and designed a transmission mode polarized hyperspectral imaging microscope (PHSIM). The hyperspectral imaging (HSI) component is based on the snapscan with a hyperspectral camera. The HSI wavelength range is from 467-700 nm. Polarized light imaging is realized by the integration of two polarizers and two liquid crystal variable retarders (LCVR), which is capable of full Stokes polarimetric imaging. The new imaging device was tested for the detection of squamous cell carcinoma (SCC) in H&E stained oral tissue slides of 8 patients. One normal area and one cancerous area on each slide are selected to make the comparison. The preliminary results indicated that the spectral curves of the Stokes vector parameters (S0, S1, S2, S3) of the normal area on the H&E stained oral tissue slides are different from those of SCC in certain wavelength range. Further work is required to apply the new polarized hyperspectral imaging microscope to a large number of patient samples and to test the PHSIM system in different cancer types.

Abdominal muscle segmentation from CT using a convolutional neural network.

CT is widely used for diagnosis and treatment of a variety of diseases, including characterization of muscle loss. In many cases, changes in muscle mass, particularly abdominal muscle, indicate how well a patient is responding to treatment. Therefore, physicians use CT to monitor changes in muscle mass throughout the patient's course of treatment. In order to measure the muscle, radiologists must segment and review each CT slice manually, which is a time-consuming task. In this work, we present a fully convolutional neural network (CNN) for the segmentation of abdominal muscle on CT. We achieved a mean Dice similarity coefficient of 0.92, a mean precision of 0.93, and a mean recall of 0.91 in an independent test set. The CNN-based segmentation method can provide an automatic tool for the segmentation of abdominal muscle. As a result, the time required to obtain information about changes in abdominal muscle using the CNN takes a fraction of the time associated with manual segmentation methods and thus can provide a useful tool in the clinical application.

Development of a polarized hyperspectral microscope for cardiac fiber orientation imaging.

Myocardial fiber orientation is closely related to the functions of the heart. The development of imaging tools for depicting myocardial fiber orientation is important. We developed a polarized hyperspectral imaging microscope (PHSIM) for cardiac fiber orientation imaging, which is capable of polarimetric imaging and hyperspectral imaging. Polarimetric imaging is realized by the integration of two polarizers. Hyperspectral imaging is realized by snapscan Preliminary imaging experiments were implemented on an unstained paraffin embedded tissue slides of a chicken heart. We also set up a Monte Carlo simulation program based on the cylinder optical model to simulate the cardiac fiber structure of the sample and the optical setup of the PHSIM system, in which we can calculate the system output light intensity related to cardiac fiber orientation. According to the imaging and simulation results, there exists a variation of intensity of acquired images with the polar angles from the maximum to the minimum under different wavelengths, which should relate to the orientation of cardiac fibers. In addition, there is a shift of the polar angle where the maximum intensity appears when a rotation of the sample happened both in the simulation and imaging experiments. Further work is required for imaging more types of myocardial tissues at different parts and the design of a complete quantitative model to describe the relations among polar angles, wavelengths, and cardiac fiber orientations.

Siamese neural networks for the classification of high-dimensional radiomic features.

This study demonstrates that a variant of a Siamese neural network architecture is more effective at classifying high-dimensional radiomic features (extracted from T2 MRI images) than traditional models, such as a Support Vector Machine or Discriminant Analysis. Ninety-nine female patients, between the ages of 20 and 48, were imaged with T2 MRI. Using biopsy pathology, the patients were separated into two groups: those with breast cancer (N=55) and those with GLM (N=44). Lesions were segmented by a trained radiologist and the ROIs were used for radiomic feature extraction. The radiomic features include 536 published features from Aerts et al., along with 20 features recurrent quantification analysis features. A Student T-Test was used to select features found to be statistically significant between the two patient groups. These features were then used to train a Siamese neural network. The label given to test features was the label of whichever class the test features with the highest percentile similarity within the training group. Within the two highest-dimensional feature sets, the Siamese network produced an AUC of 0.853 and 0.894, respectively. This is compared to best non-Siamese model, Discriminant Analysis, which produced an AUC of 0.823 and 0.836 for the two respective feature sets. However, when it came to the lower-dimensional recurrent features and the top-20 most significant features from Aerts et al., the Siamese network performed on-par or worse than the competing models. The proposed Siamese neural network architecture can outperform competing other models in high-dimensional, low-sample size spaces with regards to tabular data.

Augmented Reality-Assisted Biopsy of Soft Tissue Lesions.

Guided biopsy of soft tissue lesions can be challenging in the presence of sensitive organs or when the lesion itself is small. Computed tomography (CT) is the most frequently used modality to target soft tissue lesions. In order to aid physicians, small field of view (FOV) low dose non-contrast CT volumes are acquired prior to intervention while the patient is on the procedure table to localize the lesion and plan the best approach. However, patient motion between the end of the scan and the start of the biopsy procedure can make it difficult for a physician to translate the lesion location from the CT onto the patient body, especially for a deep-seated lesion. In addition, the needle should be managed well in three-dimensional trajectories in order to reach the lesion and avoid vital structures. This is especially challenging for less experienced interventionists. These usually result in multiple additional image acquisitions during the course of procedure to ensure accurate needle placement, especially when multiple core biopsies are required. In this work, we present an augmented reality (AR)-guided biopsy system and procedure for soft tissue and lung lesions and quantify the results using a phantom study. We found an average error of 0.75 cm from the center of the lesion when AR guidance was used, compared to an error of 1.52 cm from the center of the lesion during unguided biopsy for soft tissue lesions while upon testing the system on lung lesions, an average error of 0.62 cm from the center of the tumor while using AR guidance versus a 1.12 cm error while relying on unguided biopsies. The AR-guided system is able to improve the accuracy and could be useful in the clinical application.

Image Guided Mitral Valve Replacement: Registration of 3D Ultrasound and 2D X-ray Images.

Mitral valve repair or replacement is important in the treatment of mitral regurgitation. For valve replacement, a transcatheter approach had the possibility of decrease the invasiveness of the procedure while retaining the benefit of replacement over repair. However, fluoroscopy images acquired during the procedure provide no anatomical information regarding the placement of the probe tip once the catheter has entered a cardiac chamber. By using 3D ultrasound and registering the 3D ultrasound images to the fluoroscopy images, a physician can gain a greater understanding of the mitral valve region during transcatheter mitral valve replacement surgery. In this work, we present a graphical user interface which allows the registration of two co-planar X-ray images with 3D ultrasound during mitral valve replacement surgery.