Hyperspectral Imaging for the Detection of Glioblastoma Tumor Cells in H&E Slides Using Convolutional Neural Networks.

Hyperspectral imaging (HSI) technology has demonstrated potential to provide useful information about the chemical composition of tissue and its morphological features in a single image modality. Deep learning (DL) techniques have demonstrated the ability of automatic feature extraction from data for a successful classification. In this study, we exploit HSI and DL for the automatic differentiation of glioblastoma (GB) and non-tumor tissue on hematoxylin and eosin (H&E) stained histological slides of human brain tissue. GB detection is a challenging application, showing high heterogeneity in the cellular morphology across different patients. We employed an HSI microscope, with a spectral range from 400 to 1000 nm, to collect 517 HS cubes from 13 GB patients using 20× magnification. Using a convolutional neural network (CNN), we were able to automatically detect GB within the pathological slides, achieving average sensitivity and specificity values of 88% and 77%, respectively, representing an improvement of 7% and 8% respectively, as compared to the results obtained using RGB (red, green, and blue) images. This study demonstrates that the combination of hyperspectral microscopic imaging and deep learning is a promising tool for future computational pathologies.

Deep Learning-Based Framework for In Vivo Identification of Glioblastoma Tumor using Hyperspectral Images of Human Brain.

The main goal of brain cancer surgery is to perform an accurate resection of the tumor, preserving as much normal brain tissue as possible for the patient. The development of a non-contact and label-free method to provide reliable support for tumor resection in real-time during neurosurgical procedures is a current clinical need. Hyperspectral imaging is a non-contact, non-ionizing, and label-free imaging modality that can assist surgeons during this challenging task without using any contrast agent. In this work, we present a deep learning-based framework for processing hyperspectral images of in vivo human brain tissue. The proposed framework was evaluated by our human image database, which includes 26 in vivo hyperspectral cubes from 16 different patients, among which 258,810 pixels were labeled. The proposed framework is able to generate a thematic map where the parenchymal area of the brain is delineated and the location of the tumor is identified, providing guidance to the operating surgeon for a successful and precise tumor resection. The deep learning pipeline achieves an overall accuracy of 80% for multiclass classification, improving the results obtained with traditional support vector machine (SVM)-based approaches. In addition, an aid visualization system is presented, where the final thematic map can be adjusted by the operating surgeon to find the optimal classification threshold for the current situation during the surgical procedure.

Primary Mucinous Carcinoma of the Thyroid: A Case Report, Literature Review, and Immunohistochemistry Summary.

Introduction: Primary mucinous carcinoma of the thyroid is an exceedingly rare malignancy that is histologically similar to mucinous carcinoma of other sites. Accurate diagnosis is a challenging yet crucial component of clinical management for both patients and our understanding of this rare disease. Case Presentation: We report the case of a 69-year-old male patient with primary mucinous carcinoma of the thyroid. Microscopic examination of a biopsy specimen showed fibrous tissue, which was extensively and irregularly infiltrated by a cytologically malignant epithelial neoplasm showing glandular differentiation with mucin production. Immunohistochemistry demonstrated that tumor cells were positive for TTF1, thyroglobulin, CK7, and PAX8. Co-expression of TTF1 and PAX8 is most commonly seen in thyroid tumors. These findings support our diagnosis of mucinous carcinoma of thyroid origin, which is rare and highly aggressive. Conclusion: In this report, we present the only documented case of primary mucinous carcinoma of the thyroid reported in the United States in the last decade. The diagnosis of primary mucinous carcinoma of the thyroid can be challenging. Therefore, we discuss and detail the clinicopathologic tumor profile and provide more current, detailed histological criteria to assist in the diagnosis of this rare disease.

Critical Left Subclavian Artery Stenosis With Retrograde Vertebral Flow: A Case Report and Literature Review.

Subclavian artery stenosis is a rare condition associated with significant morbidity and mortality, making prompt recognition and treatment essential. We present a case of left-sided subclavian artery occlusion with neurological symptoms, including vertigo, unsteady gait, and left upper extremity pain and paresthesia. The patient's symptoms had been progressing over several months. Her risk factors included age, hyperlipidemia, and poorly controlled blood pressure with resultant arteriosclerosis throughout her vasculature. An arteriogram demonstrated critical stenosis of the left subclavian with retrograde flow through the left vertebral artery. Aspirin and clopidogrel were initiated prior to successful balloon angioplasty and stenting. After stent placement, the patient had minimal residual subclavian stenosis and anterograde vertebral artery flow. In this case report, we discuss clinical presentation, typical examination and imaging findings, and treatment options for subclavian stenosis including medical management and revascularization procedures.

Impact of intravenous access site on attenuation for thoracic computed tomographic angiography: A time-matched, nested, case-control study.

BACKGROUND: The objective of this study was to evaluate whether the choice of intravenous access (IVA) site affects aortic attenuation during thoracic computed tomographic angiography (T-CTA) and any associated risks with intravenous device placement. METHODS: All T-CTA exams performed between 1/1/2013 and 8/14/2015 were retrospectively reviewed to identify those performed with contrast media injection via alternative (i.e. non-antecubital) IVA (n = 1769). Using time matching, antecubital IVA exams (n = 1769) were selected as controls. For each exam, attenuation was measured in the ascending aorta. Patient and technical data was subsequently collected from all 3538 patients included in this study. Multiple linear regression was used to determine if IVA site affected attenuation. Lastly, data related to extravasations for the entire T-CTA cohort were collected and compared. RESULTS: Hand/wrist, arm, and central venous access device IVA were all equivalent to antecubital IVA in terms of attenuation (P = 0.579, P = 0.599, and P = 0.522 respectively). Forearm and intraosseous IVA had significantly higher attenuation (P = 0.010 and P = 0.002, respectively) than antecubital IVA. Right-sided IVA was associated with a small attenuation increase of 11 Hounsfield Units (P < 0.001) compared to left-sided IVA. In terms of extravasation, antecubital IVA was equivalent to hand/wrist, forearm, and upper arm IVA (P = 0.778, P = 0.060, and P = 0.090 respectively). CONCLUSIONS: Satisfactory aortic attenuation achieved with non-antecubital IVA is equivalent to attenuation achieved with antecubital IVA for T-CTA imaging. The risk of contrast media extravasation in peripheral IVA devices was relatively low, however, appropriate IVA site selection should be considered an important factor for successful administration of contrast media for future imaging studies. This prevents undue harm to patients through preventable device failures when using a peripheral IV device in areas of high flexion/range of movements undergoing pressure injection for contrast media.

A Lost Bullet in the Coronary Sinus: A Cautionary Tale.

We present a case of venous bullet embolism to the right atrium following a gunshot wound (GSW) to the abdomen. A 53-year-old male presented after a GSW to the abdomen. His workup included a computed tomography (CT) scan demonstrating an aortic injury with aortocaval fistula. A radio-opaque object consistent with a bullet was visualized in the right atrium. First, this case details an important decision, choice of surgery versus an interventional approach. After repair of the aortocaval fistula, the patient underwent a planned attempt to extract the bullet through a right lateral thoracotomy approach utilizing cardiopulmonary bypass to facilitate a right atriotomy. Intraoperatively, the team was not able to localize the bullet in the right atrium despite fluoroscopic evaluation. A postoperative CT scan demonstrated that the bullet had migrated into the coronary sinus. Lastly, this case demonstrates successful positioning maneuvers to dislodge the bullet out of the heart and into the inferior vena cava, allowing for the endovascular extraction of the bullet.

Technical note: The effect of image annotation with minimal manual interaction for semiautomatic prostate segmentation in CT images using fully convolutional neural networks.

PURPOSE: The goal is to study the performance improvement of a deep learning algorithm in three-dimensional (3D) image segmentation through incorporating minimal user interaction into a fully convolutional neural network (CNN). METHODS: A U-Net CNN was trained and tested for 3D prostate segmentation in computed tomography (CT) images. To improve the segmentation accuracy, the CNN's input images were annotated with a set of border landmarks to supervise the network for segmenting the prostate. The network was trained and tested again with annotated images after 5, 10, 15, 20, or 30 landmark points were used. RESULTS: Compared to fully automatic segmentation, the Dice similarity coefficient increased up to 9% when 5-30 sparse landmark points were involved, with the segmentation accuracy improving as more border landmarks were used. CONCLUSIONS: When a limited number of sparse border landmarks are used on the input image, the CNN performance approaches the interexpert observer difference observed in manual segmentation.

Detecting Aggressive Papillary Thyroid Carcinoma Using Hyperspectral Imaging and Radiomic Features.

Hyperspectral imaging (HSI) and radiomics have the potential to improve the accuracy of tumor malignancy prediction and assessment. In this work, we extracted radiomic features of fresh surgical papillary thyroid carcinoma (PTC) specimen that were imaged with HSI. A total of 107 unique radiomic features were extracted. This study includes 72 ex-vivo tissue specimens from 44 patients with pathology-confirmed PTC. With the dilated hyperspectral images, the shape feature of least axis length was able to predict the tumor aggressiveness with a high accuracy. The HSI-based radiomic method may provide a useful tool to aid oncologists in determining tumors with intermediate to high risk and in clinical decision making.

Multiparametric Radiomics for Predicting the Aggressiveness of Papillary Thyroid Carcinoma Using Hyperspectral Images.

Papillary thyroid carcinoma (PTC) is primarily treated by surgical resection. During surgery, surgeons often need intraoperative frozen analysis and pathologic consultation in order to detect PTC. In some cases pathologists cannot determine if the tumor is aggressive until the operation has been completed. In this work, we have taken tumor classification a step further by determining the tumor aggressiveness of fresh surgical specimens. We employed hyperspectral imaging (HSI) in combination with multiparametric radiomic features to complete this task. The study cohort includes 72 ex-vivo tissue specimens from 44 patients with pathology-confirmed PTC. A total of 67 features were extracted from this data. Using machine learning classification methods, we were able to achieve an AUC of 0.85. Our study shows that hyperspectral imaging and multiparametric radiomic features could aid in the pathological detection of tumor aggressiveness using fresh surgical spemens obtained during surgery.

Rapid 3D reconstruction guided embolization for catastrophic bleeding following vacuum assisted breast biopsy; A case report and review of the literature.

The most clinically significant complication associated with stereotactic core needle biopsy of the breast is hematoma formation, which only occurs in less than 1% of biopsies and may require treatment. Cases of uncontrollable bleeding, refractory to repeated compression, resulting from biopsy are exceedingly rare. We present a case of catastrophic, uncontrollable bleeding and large hematoma formation resulting from stereotactic vacuum-assisted breast biopsy of a breast mass identified in screening mammography. Percutaneous embolization was planned and guided using 3D reconstructions from computed tomographic angiography, and bleeding was successfully controlled with micro-coil embolization.

Pixel-level Tumor Margin Assessment of Surgical Specimen with Hyperspectral Imaging and Deep Learning Classification.

Surgery is a major treatment method for squamous cell carcinoma (SCC). During surgery, insufficient tumor margin may lead to local recurrence of cancer. Hyperspectral imaging (HSI) is a promising optical imaging technique for in vivo cancer detection and tumor margin assessment. In this study, a fully convolutional network (FCN) was implemented for tumor classification and margin assessment on hyperspectral images of SCC. The FCN was trained and validated with hyperspectral images of 25 ex vivo SCC surgical specimens from 20 different patients. The network was evaluated per patient and achieved pixel-level tissue classification with an average area under the curve (AUC) of 0.88, as well as 0.83 accuracy, 0.84 sensitivity, and 0.70 specificity across all the 20 patients. The 95% Hausdorff distance of assessed tumor margin in 17 patients was less than 2 mm, and the classification time of each tissue specimen took less than 10 seconds. The proposed methods can potentially facilitate intraoperative tumor margin assessment and improve surgical outcomes.

Augmented reality visualization of hyperspectral imaging classifications for image-guided brain tumor phantom resection.

Wearable augmented reality (AR) is an emerging technology with enormous potential for use in the medical field, from training and procedure simulations to image-guided surgery. Medical AR seeks to enable surgeons to see tissue segmentations in real time. With the objective of achieving real-time guidance, the emphasis on speed produces the need for a fast method for imaging and classification. Hyperspectral imaging (HSI) is a non-contact, optical imaging modality that rapidly acquires hundreds of images of tissue at different wavelengths, which can be used to generate spectral data of the tissue. Combining HSI information and machine-learning algorithms allows for effective tissue classification. In this paper, we constructed a brain tissue phantom with porcine blood, yellow-dyed gelatin, and colorless gelatin to represent blood vessels, tumor, and normal brain tissue, respectively. Using a segmentation algorithm, hundreds of hyperspectral images were compiled to classify each of the pixels. Three segmentation labels were generated from the data, each with a different type of tissue. Our system virtually superimposes the HSI channels and segmentation labels of a brain tumor phantom onto the real scene using the HoloLens AR headset. The user can manipulate and interact with the segmentation labels and HSI channels by repositioning, rotating, changing visibility, and switching between them. All actions can be performed through either hand or voice controls. This creates a convenient and multifaceted visualization of brain tissue in real time with minimal user restrictions. We demonstrate the feasibility of a fast and practical HIS-AR technique for potential use of image-guided brain surgery.

In Vivo Cancer Detection in Animal Model Using Hyperspectral Image Classification with Wavelet Feature Extraction.

Hyperspectral imaging (HSI) is a promising optical imaging technique for cancer detection. However, quantitative methods need to be developed in order to utilize the rich spectral information and subtle spectral variation in such images. In this study, we explore the feasibility of using wavelet-based features from in vivo hyperspectral images for head and neck cancer detection. Hyperspectral reflectance data were collected from 12 mice bearing head and neck cancer. Catenation of 5-level wavelet decomposition outputs of hyperspectral images was used as a feature for tumor discrimination. A support vector machine (SVM) was utilized as the classifier. Seven types of mother wavelets were tested to select the one with the best performance. Classifications with raw reflectance spectra, 1-level wavelet decomposition output, and 2-level wavelet decomposition output, as well as the proposed feature were carried out for comparison. Our results show that the proposed wavelet-based feature yields better classification accuracy, and that using different type and order of mother wavelet achieves different classification results. The wavelet-based classification method provides a new approach for HSI detection of head and neck cancer in the animal model.

Development of a new polarized hyperspectral imaging microscope.

In this study, we proposed and designed a transmission mode polarized hyperspectral imaging microscope (PHSIM). The hyperspectral imaging (HSI) component is based on the snapscan with a hyperspectral camera. The HSI wavelength range is from 467-700 nm. Polarized light imaging is realized by the integration of two polarizers and two liquid crystal variable retarders (LCVR), which is capable of full Stokes polarimetric imaging. The new imaging device was tested for the detection of squamous cell carcinoma (SCC) in H&E stained oral tissue slides of 8 patients. One normal area and one cancerous area on each slide are selected to make the comparison. The preliminary results indicated that the spectral curves of the Stokes vector parameters (S0, S1, S2, S3) of the normal area on the H&E stained oral tissue slides are different from those of SCC in certain wavelength range. Further work is required to apply the new polarized hyperspectral imaging microscope to a large number of patient samples and to test the PHSIM system in different cancer types.

Incorporating minimal user input into deep learning based image segmentation.

Computer-assisted image segmentation techniques could help clinicians to perform the border delineation task faster with lower inter-observer variability. Recently, convolutional neural networks (CNNs) are widely used for automatic image segmentation. In this study, we used a technique to involve observer inputs for supervising CNNs to improve the accuracy of the segmentation performance. We added a set of sparse surface points as an additional input to supervise the CNNs for more accurate image segmentation. We tested our technique by applying minimal interactions to supervise the networks for segmentation of the prostate on magnetic resonance images. We used U-Net and a new network architecture that was based on U-Net (dual-input path [DIP] U-Net), and showed that our supervising technique could significantly increase the segmentation accuracy of both networks as compared to fully automatic segmentation using U-Net. We also showed DIP U-Net outperformed U-Net for supervised image segmentation. We compared our results to the measured inter-expert observer difference in manual segmentation. This comparison suggests that applying about 15 to 20 selected surface points can achieve a performance comparable to manual segmentation.

Hyperspectral Microscopic Imaging for Automatic Detection of Head and Neck Squamous Cell Carcinoma Using Histologic Image and Machine Learning.

The purpose of this study is to develop hyperspectral imaging (HSI) for automatic detection of head and neck cancer cells on histologic slides. A compact hyperspectral microscopic system is developed in this study. Histologic slides from 15 patients with squamous cell carcinoma (SCC) of the larynx and hypopharynx are imaged with the system. The proposed nuclei segmentation method based on principle component analysis (PCA) can extract most nuclei in the hyperspectral image without extracting other sub-cellular components. Both spectra-based support vector machine (SVM) and patch-based convolutional neural network (CNN) are used for nuclei classification. CNNs were trained with both hyperspectral images and pseudo RGB images of extracted nuclei, in order to evaluate the usefulness of extra information provided by hyperspectral imaging. The average accuracy of spectra-based SVM classification is 68%. The average AUC and average accuracy of the HSI patch-based CNN classification is 0.94 and 82.4%, respectively. The hyperspectral microscopic imaging and classification methods provide an automatic tool to aid pathologists in detecting SCC on histologic slides.

Tumor detection of the thyroid and salivary glands using hyperspectral imaging and deep learning.

The performance of hyperspectral imaging (HSI) for tumor detection is investigated in ex-vivo specimens from the thyroid (N = 200) and salivary glands (N = 16) from 82 patients. Tissues were imaged with HSI in broadband reflectance and autofluorescence modes. For comparison, the tissues were imaged with two fluorescent dyes. Additionally, HSI was used to synthesize three-band RGB multiplex images to represent the human-eye response and Gaussian RGBs, which are referred to as HSI-synthesized RGB images. Using histological ground truths, deep learning algorithms were developed for tumor detection. For the classification of thyroid tumors, HSI-synthesized RGB images achieved the best performance with an AUC score of 0.90. In salivary glands, HSI had the best performance with 0.92 AUC score. This study demonstrates that HSI could aid surgeons and pathologists in detecting tumors of the thyroid and salivary glands.

Hyperspectral and multispectral imaging in digital and computational pathology: a systematic review [Invited].

Hyperspectral imaging (HSI) and multispectral imaging (MSI) technologies have the potential to transform the fields of digital and computational pathology. Traditional digitized histopathological slides are imaged with RGB imaging. Utilizing HSI/MSI, spectral information across wavelengths within and beyond the visual range can complement spatial information for the creation of computer-aided diagnostic tools for both stained and unstained histological specimens. In this systematic review, we summarize the methods and uses of HSI/MSI for staining and color correction, immunohistochemistry, autofluorescence, and histopathological diagnostic research. Studies include hematology, breast cancer, head and neck cancer, skin cancer, and diseases of central nervous, gastrointestinal, and genitourinary systems. The use of HSI/MSI suggest an improvement in the detection of diseases and clinical practice compared with traditional RGB analysis, and brings new opportunities in histological analysis of samples, such as digital staining or alleviating the inter-laboratory variability of digitized samples. Nevertheless, the number of studies in this field is currently limited, and more research is needed to confirm the advantages of this technology compared to conventional imagery.

Fully automated segmentation of the right ventricle in patients with repaired Tetralogy of Fallot using U-Net.

Cardiac magnetic resonance (CMR) imaging is considered the standard imaging modality for volumetric analysis of the right ventricle (RV), an especially important practice in the evaluation of heart structure and function in patients with repaired Tetralogy of Fallot (rTOF). In clinical practice, however, this requires time-consuming manual delineation of the RV endocardium in multiple 2-dimensional (2D) slices at multiple phases of the cardiac cycle. In this work, we employed a U-Net based 2D convolutional neural network (CNN) classifier in the fully automatic segmentation of the RV blood pool. Our dataset was comprised of 5,729 short-axis cine CMR slices taken from 100 individuals with rTOF. Training of our CNN model was performed on images from 50 individuals while validation was performed on images from 10 individuals. Segmentation results were evaluated by Dice similarity coefficient (DSC) and Hausdorff distance (HD). Use of the CNN model on our testing group of 40 individuals yielded a median DSC of 90% and a median 95th percentile HD of 5.1 mm, demonstrating good performance in these metrics when compared to literature results. Our preliminary results suggest that our deep learning-based method can be effective in automating RV segmentation.

Conditional generative adversarial network for synthesizing hyperspectral images of breast cancer cells from digitized histology.

Hyperspectral imaging (HSI), which acquires up to hundreds of bands, has been proposed as a promising imaging modality for digitized histology beyond RGB imaging to provide more quantitative information to assist pathologists with disease detection in samples. While digitized RGB histology is quite standardized and easy to acquire, histological HSI often requires custom-made equipment and longer imaging times compared to RGB. In this work, we present a dataset of corresponding RGB digitized histology and histological HSI of breast cancer, and we develop a conditional generative adversarial network (GAN) to artificially synthesize HSI from standard RGB images of normal and cancer cells. The results of the GAN synthesized HSI are promising, showing structural similarity (SSIM) of approximately 80% and mean absolute error (MAE) of 6 to 11%. Further work is needed to establish the ability of generating HSI from RGB images on larger datasets.