Hyperspectral dark-field microscopy of human breast lumpectomy samples for tumor margin detection in breast-conserving surgery.

Significance: Hyperspectral dark-field microscopy (HSDFM) and data cube analysis algorithms demonstrate successful detection and classification of various tissue types, including carcinoma regions in human post-lumpectomy breast tissues excised during breast-conserving surgeries. Aim: We expand the application of HSDFM to the classification of tissue types and tumor subtypes in pre-histopathology human breast lumpectomy samples. Approach: Breast tissues excised during breast-conserving surgeries were imaged by the HSDFM and analyzed. The performance of the HSDFM is evaluated by comparing the backscattering intensity spectra of polystyrene microbead solutions with the Monte Carlo simulation of the experimental data. For classification algorithms, two analysis approaches, a supervised technique based on the spectral angle mapper (SAM) algorithm and an unsupervised technique based on the K-means algorithm are applied to classify various tissue types including carcinoma subtypes. In the supervised technique, the SAM algorithm with manually extracted endmembers guided by H&E annotations is used as reference spectra, allowing for segmentation maps with classified tissue types including carcinoma subtypes. Results: The manually extracted endmembers of known tissue types and their corresponding threshold spectral correlation angles for classification make a good reference library that validates endmembers computed by the unsupervised K-means algorithm. The unsupervised K-means algorithm, with no a priori information, produces abundance maps with dominant endmembers of various tissue types, including carcinoma subtypes of invasive ductal carcinoma and invasive mucinous carcinoma. The two carcinomas' unique endmembers produced by the two methods agree with each other within <2% residual error margin. Conclusions: Our report demonstrates a robust procedure for the validation of an unsupervised algorithm with the essential set of parameters based on the ground truth, histopathological information. We have demonstrated that a trained library of the histopathology-guided endmembers and associated threshold spectral correlation angles computed against well-defined reference data cubes serve such parameters. Two classification algorithms, supervised and unsupervised algorithms, are employed to identify regions with carcinoma subtypes of invasive ductal carcinoma and invasive mucinous carcinoma present in the tissues. The two carcinomas' unique endmembers used by the two methods agree to <2% residual error margin. This library of high quality and collected under an environment with no ambient background may be instrumental to develop or validate more advanced unsupervised data cube analysis algorithms, such as effective neural networks for efficient subtype classification.

Demonstration of a laparoscopic structured-illumination three-dimensional imaging system for guiding reconstructive bowel anastomosis.

This paper reports the development and system analysis of a laparoscopic system based on structured illumination technique capable of three-dimensional (3-D) reconstruction of porcine intestine during surgical anastomosis (connection of tubular structures). A calibration target is used to validate the system performance and results show a depth of field of 20 mm with an accuracy of 0.008 mm and precision of 0.25 mm. The imaging system is used to reconstruct a quantitative 3-D depth measurement of ex vivo porcine bowel tissues to mimic an end-to-end bowel anastomosis scenario. We demonstrate that the system can detect a suture in the tissue and map homogeneous surfaces of the intestine with different tissue pigments, affirming the feasibility for depth quantization for guiding and assisting medical diagnostic decisions in anastomosis surgery.

Three-dimensional facial digitization using advanced digital image correlation.

Presented in this paper is an effective technique to acquire the three-dimensional (3D) digital images of the human face without the use of active lighting and artificial patterns. The technique is based on binocular stereo imaging and digital image correlation, and it includes two key steps: camera calibration and image matching. The camera calibration involves a pinhole model and a bundle-adjustment approach, and the governing equations of the 3D digitization process are described. For reliable pixel-to-pixel image matching, the skin pores and freckles or lentigines on the human face serve as the required pattern features to facilitate the process. It employs feature-matching-based initial guess, multiple subsets, iterative optimization algorithm, and reliability-guided computation path to achieve fast and accurate image matching. Experiments have been conducted to demonstrate the validity of the proposed technique. The simplicity of the approach and the affordable cost of the implementation show its practicability in scientific and engineering applications.

Semi-Autonomous Laparoscopic Robotic Electro-surgery with a Novel 3D Endoscope.

This paper reports a robotic laparoscopic surgery system performing electro-surgery on porcine cadaver kidney, and evaluates its accuracy in an open loop control scheme to conduct targeting and cutting tasks guided by a novel 3D endoscope. We describe the design and integration of the novel laparoscopic imaging system that is capable of reconstructing the surgical field using structured light. A targeting task is first performed to determine the average positioning error of the system as guided by the laparoscopic camera. The imaging system is then used to reconstruct the surface of a porcine cadaver kidney, and generate a cutting trajectory with consistent depth. The paper concludes by using the robotic system in open loop control to cut this trajectory using a multi degree of freedom electro-surgical tool. It is demonstrated that for a cutting depth of 3 mm, the robotic surgical system follows the trajectory with an average depth of 2.44 mm and standard deviation of 0.34 mm. The average positional accuracy of the system was 2.74±0.99 mm.

Suture Maps Based on Structural Enhanced Imaging Endoscope for Laparoscopic Robotic Surgery.

A detection method for cutting scheme in 3D is proposed to assist robotic surgical manipulation, leading to an automatic suturing suggestion mapping.

A three-step reconstruction method for fluorescence molecular tomography based on compressive sensing.

Fluorescence molecular tomography (FMT) is a promising tool for real time in vivo quantification of neurotransmission (NT) as we pursue in our BRAIN initiative effort. However, the acquired image data are noisy and the reconstruction problem is ill-posed. Further, while spatial sparsity of the NT effects could be exploited, traditional compressive-sensing methods cannot be directly applied as the system matrix in FMT is highly coherent. To overcome these issues, we propose and assess a three-step reconstruction method. First, truncated singular value decomposition is applied on the data to reduce matrix coherence. The resultant image data are input to a homotopy-based reconstruction strategy that exploits sparsity via ℓ1 regularization. The reconstructed image is then input to a maximum-likelihood expectation maximization (MLEM) algorithm that retains the sparseness of the input estimate and improves upon the quantitation by accurate Poisson noise modeling. The proposed reconstruction method was evaluated in a three-dimensional simulated setup with fluorescent sources in a cuboidal scattering medium with optical properties simulating human brain cortex (reduced scattering coefficient: 9.2 cm-1, absorption coefficient: 0.1 cm-1) and tomographic measurements made using pixelated detectors. In different experiments, fluorescent sources of varying size and intensity were simulated. The proposed reconstruction method provided accurate estimates of the fluorescent source intensity, with a 20% lower root mean square error on average compared to the pure-homotopy method for all considered source intensities and sizes. Further, compared with conventional ℓ2 regularized algorithm, overall, the proposed method reconstructed substantially more accurate fluorescence distribution. The proposed method shows considerable promise and will be tested using more realistic simulations and experimental setups.

Listening to membrane potential: photoacoustic voltage-sensitive dye recording.

Voltage-sensitive dyes (VSDs) are designed to monitor membrane potential by detecting fluorescence changes in response to neuronal or muscle electrical activity. However, fluorescence imaging is limited by depth of penetration and high scattering losses, which leads to low sensitivity in vivo systems for external detection. By contrast, photoacoustic (PA) imaging, an emerging modality, is capable of deep tissue, noninvasive imaging by combining near-infrared light excitation and ultrasound detection. Here, we show that voltage-dependent quenching of dye fluorescence leads to a reciprocal enhancement of PA intensity. We synthesized a near-infrared photoacoustic VSD (PA-VSD), whose PA intensity change is sensitive to membrane potential. In the polarized state, this cyanine-based probe enhances PA intensity while decreasing fluorescence output in a lipid vesicle membrane model. A theoretical model accounts for how the experimental PA intensity change depends on fluorescence and absorbance properties of the dye. These results not only demonstrate PA voltage sensing but also emphasize the interplay of both fluorescence and absorbance properties in the design of optimized PA probes. Together, our results demonstrate PA sensing as a potential new modality for recording and external imaging of electrophysiological and neurochemical events in the brain.

A demonstration of structured-illumination-based technique using commercial surgical endoscope.

An integration of commercial surgical endoscope using structured illumination technique for three-dimensional reconstruction was performed on biological samples with a depth of field of 20 mm and a relative accuracy of 0.1%.

Experimental assessment of a 3-D plenoptic endoscopic imaging system.

An endoscopic imaging system using a plenoptic technique to reconstruct 3-D information is demonstrated and analyzed in this Letter. The proposed setup integrates a clinical surgical endoscope with a plenoptic camera to achieve a depth accuracy error of about 1 mm and a precision error of about 2 mm, within a 25 mm × 25 mm field of view, operating at 11 frames per second.

3-D endoscopic imaging using plenoptic camera.

Three-dimensional endoscopic imaging using plenoptic technique combined with F-matching algorithm has been pursued in this study. A custom relay optics was designed to integrate a commercial surgical straight endoscope with a plenoptic camera.

Multispectral tissue characterization for intestinal anastomosis optimization.

Intestinal anastomosis is a surgical procedure that restores bowel continuity after surgical resection to treat intestinal malignancy, inflammation, or obstruction. Despite the routine nature of intestinal anastomosis procedures, the rate of complications is high. Standard visual inspection cannot distinguish the tissue subsurface and small changes in spectral characteristics of the tissue, so existing tissue anastomosis techniques that rely on human vision to guide suturing could lead to problems such as bleeding and leakage from suturing sites. We present a proof-of-concept study using a portable multispectral imaging (MSI) platform for tissue characterization and preoperative surgical planning in intestinal anastomosis. The platform is composed of a fiber ring light-guided MSI system coupled with polarizers and image analysis software. The system is tested on ex vivo porcine intestine tissue, and we demonstrate the feasibility of identifying optimal regions for suture placement.

An average enumeration method of hyperspectral imaging data for quantitative evaluation of medical device surface contamination.

We propose a quantification method called Mapped Average Principal component analysis Score (MAPS) to enumerate the contamination coverage on common medical device surfaces. The method was adapted from conventional Principal Component Analysis (PCA) on non-overlapped regions of a full frame hyperspectral image to resolve the percentage of contamination from the substrate. The concept was proven by using a controlled contamination sample with artificial test soil and color simulating organic mixture, and was further validated using a bacterial system including biofilm on stainless steel surface. We also validate the results of MAPS with other statistical spectral analysis including Spectral Angle Mapper (SAM). The proposed method provides an alternative quantification method for hyperspectral imaging data, which can be easily implemented by basic PCA analysis.

Radiant exposure level comparison between Gaussian and top hat beams in various scanning patterns.

The radiant exposure of optical irradiation beams with different scanning parameters has been theoretically studied. We analyzed the difference in radiant exposure introduced by Gaussian and top hat beams. Various parameters such as scanning pattern, aperture position, beam size and scan spacing were also introduced in this study. We found that Gaussian beams introduce higher calculated radiant exposure to the aperture than top hat beams for certain beam size to aperture size ratios. However, as the scan spacing decreases, the radiant exposure difference calculated from Gaussian and top hat beams diminishes.

Three-dimensional phantoms for curvature correction in spatial frequency domain imaging.

The sensitivity to surface profile of non-contact optical imaging, such as spatial frequency domain imaging, may lead to incorrect measurements of optical properties and consequently erroneous extrapolation of physiological parameters of interest. Previous correction methods have focused on calibration-based, model-based, and computation-based approached. We propose an experimental method to correct the effect of surface profile on spectral images. Three-dimensional (3D) phantoms were built with acrylonitrile butadiene styrene (ABS) plastic using an accurate 3D imaging and an emergent 3D printing technique. In this study, our method was utilized for the correction of optical properties (absorption coefficient µ(a) and reduced scattering coefficient µ(s)') of objects obtained with a spatial frequency domain imaging system. The correction method was verified on three objects with simple to complex shapes. Incorrect optical properties due to surface with minimum 4 mm variation in height and 80 degree in slope were detected and improved, particularly for the absorption coefficients. The 3D phantom-based correction method is applicable for a wide range of purposes. The advantages and drawbacks of the 3D phantom-based correction methods are discussed in details.