Multicore fiber optic imaging reveals that astrocyte calcium activity in the mouse cerebral cortex is modulated by internal motivational state.

Astrocytes are a direct target of neuromodulators and can influence neuronal activity on broad spatial and temporal scales in response to a rise in cytosolic calcium. However, our knowledge about how astrocytes are recruited during different animal behaviors remains limited. To measure astrocyte activity calcium in vivo during normative behaviors, we utilize a high-resolution, long working distance multicore fiber optic imaging system that allows visualization of individual astrocyte calcium transients in the cerebral cortex of freely moving mice. We define the spatiotemporal dynamics of astrocyte calcium changes during diverse behaviors, ranging from sleep-wake cycles to the exploration of novel objects, showing that their activity is more variable and less synchronous than apparent in head-immobilized imaging conditions. In accordance with their molecular diversity, individual astrocytes often exhibit distinct thresholds and activity patterns during explorative behaviors, allowing temporal encoding across the astrocyte network. Astrocyte calcium events were induced by noradrenergic and cholinergic systems and modulated by internal state. The distinct activity patterns exhibited by astrocytes provides a means to vary their neuromodulatory influence in different behavioral contexts and internal states.

Automatic and real-time tissue sensing for autonomous intestinal anastomosis using hybrid MLP-DC-CNN classifier-based optical coherence tomography.

Anastomosis is a common and critical part of reconstructive procedures within gastrointestinal, urologic, and gynecologic surgery. The use of autonomous surgical robots such as the smart tissue autonomous robot (STAR) system demonstrates an improved efficiency and consistency of the laparoscopic small bowel anastomosis over the current da Vinci surgical system. However, the STAR workflow requires auxiliary manual monitoring during the suturing procedure to avoid missed or wrong stitches. To eliminate this monitoring task from the operators, we integrated an optical coherence tomography (OCT) fiber sensor with the suture tool and developed an automatic tissue classification algorithm for detecting missed or wrong stitches in real time. The classification results were updated and sent to the control loop of STAR robot in real time. The suture tool was guided to approach the object by a dual-camera system. If the tissue inside the tool jaw was inconsistent with the desired suture pattern, a warning message would be generated. The proposed hybrid multilayer perceptron dual-channel convolutional neural network (MLP-DC-CNN) classification platform can automatically classify eight different abdominal tissue types that require different suture strategies for anastomosis. In MLP, numerous handcrafted features (∼1955) were utilized including optical properties and morphological features of one-dimensional (1D) OCT A-line signals. In DC-CNN, intensity-based features and depth-resolved tissues' attenuation coefficients were fully exploited. A decision fusion technique was applied to leverage the information collected from both classifiers to further increase the accuracy. The algorithm was evaluated on 69,773 testing A-line data. The results showed that our model can classify the 1D OCT signals of small bowels in real time with an accuracy of 90.06%, a precision of 88.34%, and a sensitivity of 87.29%, respectively. The refresh rate of the displayed A-line signals was set as 300 Hz, the maximum sensing depth of the fiber was 3.6 mm, and the running time of the image processing algorithm was ∼1.56 s for 1,024 A-lines. The proposed fully automated tissue sensing model outperformed the single classifier of CNN, MLP, or SVM with optimized architectures, showing the complementarity of different feature sets and network architectures in classifying intestinal OCT A-line signals. It can potentially reduce the manual involvement of robotic laparoscopic surgery, which is a crucial step towards a fully autonomous STAR system.

Arc-to-line frame registration method for ultrasound and photoacoustic image-guided intraoperative robot-assisted laparoscopic prostatectomy.

PURPOSE: To achieve effective robot-assisted laparoscopic prostatectomy, the integration of transrectal ultrasound (TRUS) imaging system which is the most widely used imaging modality in prostate imaging is essential. However, manual manipulation of the ultrasound transducer during the procedure will significantly interfere with the surgery. Therefore, we propose an image co-registration algorithm based on a photoacoustic marker (PM) method, where the ultrasound/photoacoustic (US/PA) images can be registered to the endoscopic camera images to ultimately enable the TRUS transducer to automatically track the surgical instrument. METHODS: An optimization-based algorithm is proposed to co-register the images from the two different imaging modalities. The principle of light propagation and an uncertainty in PM detection were assumed in this algorithm to improve the stability and accuracy of the algorithm. The algorithm is validated using the previously developed US/PA image-guided system with a da Vinci surgical robot. RESULTS: The target-registration-error (TRE) is measured to evaluate the proposed algorithm. In both simulation and experimental demonstration, the proposed algorithm achieved a sub-centimeter accuracy which is acceptable in practical clinics (i.e., 1.15 ± 0.29 mm from the experimental evaluation). The result is also comparable with our previous approach (i.e., 1.05 ± 0.37 mm), and the proposed method can be implemented with a normal white light stereo camera and does not require highly accurate localization of the PM. CONCLUSION: The proposed frame registration algorithm enabled a simple yet efficient integration of commercial US/PA imaging system into laparoscopic surgical setting by leveraging the characteristic properties of acoustic wave propagation and laser excitation, contributing to automated US/PA image-guided surgical intervention applications.

Common-path optical coherence tomography guided vertical pneumodissection for DALK.

We reported a design and evaluation of an optical coherence tomography (OCT) sensor-integrated 27 gauge vertically inserted razor edge cannula (VIREC) for pneumatic dissection of Descemet's membrane (DM) from the stromal layer. The VIREC was inserted vertically at the apex of the cornea to the desired depth near DM. The study was performed using ex vivo bovine corneas (N = 5) and rabbit corneas (N = 5). A clean penumodissection of a stromal layer was successfully performed using VIREC without any stomal blanching on bovine eyes. The "big bubble" was generated in all five tests without perforation. Only micro bubbles were observed on rabbit eyes. The results proved that VIREC can be an effective surgical option for "big bubble" DALK.

Real-time intraoperative surgical guidance system in the da Vinci surgical robot based on transrectal ultrasound/photoacoustic imaging with photoacoustic markers: an ex vivo demonstration.

This paper introduces the first integrated real-time intraoperative surgical guidance system, in which an endoscope camera of da Vinci surgical robot and a transrectal ultrasound (TRUS) transducer are co-registered using photoacoustic markers that are detected in both fluorescence (FL) and photoacoustic (PA) imaging. The co-registered system enables the TRUS transducer to track the laser spot illuminated by a pulsed-laser-diode attached to the surgical instrument, providing both FL and PA images of the surgical region-of-interest (ROI). As a result, the generated photoacoustic marker is visualized and localized in the da Vinci endoscopic FL images, and the corresponding tracking can be conducted by rotating the TRUS transducer to display the PA image of the marker. A quantitative evaluation revealed that the average registration and tracking errors were 0.84 mm and 1.16°, respectively. This study shows that the co-registered photoacoustic marker tracking can be effectively deployed intraoperatively using TRUS+PA imaging providing functional guidance of the surgical ROI.

Depth-dependent attenuation and backscattering characterization of optical coherence tomography by stationary iterative method.

Significance: Extracting optical properties of tissue [e.g., the attenuation coefficient (µ) and the backscattering fraction] from the optical coherence tomography (OCT) images is a valuable tool for parametric imaging and related diagnostic applications. Previous attenuation estimation models depend on the assumption of the uniformity of the backscattering fraction (R) within layers or whole samples, which does not accurately represent real-world conditions. Aim: Our aim is to develop a robust and accurate model that calculates depth-wise values of attenuation and backscattering fractions simultaneously from OCT signals. Furthermore, we aim to develop an attenuation compensation model for OCT images that utilizes the optical properties we obtained to improve the visual representation of tissues. Approach: Using the stationary iteration method under suitable constraint conditions, we derived the approximated solutions of µ and R on a single scattering model. During the iteration, the estimated value of µ can be rectified by introducing the large variations of R, whereas the small ones were automatically ignored. Based on the calculation of the structure information, the OCT intensity with attenuation compensation was deduced and compared with the original OCT profiles. Results: The preliminary validation was performed in the OCT A-line simulation and Monte Carlo modeling, and the subsequent experiment was conducted on multi-layer silicone-dye-TiO2 phantoms and ex vivo cow eyes. Our method achieved robust and precise estimation of µ and R for both simulated and experimental data. Moreover, corresponding OCT images with attenuation compensation provided an improved resolution over the entire imaging range. Conclusions: Our proposed method was able to correct the estimation bias induced by the variations of R and provided accurate depth-resolved measurements of both µ and R simultaneously. The method does not require prior knowledge of the morphological information of tissue and represents more real-life tissues. Thus, it has the potential to help OCT imaging based disease diagnosis of complex and multi-layer biological tissue.

Multicore fiber optic imaging reveals that astrocyte calcium activity in the cerebral cortex is modulated by internal motivational state.

Astrocytes are a direct target of neuromodulators and can influence neuronal activity on broad spatial and temporal scales through their close proximity to synapses. However, our knowledge about how astrocytes are functionally recruited during different animal behaviors and their diverse effects on the CNS remains limited. To enable measurement of astrocyte activity patterns in vivo during normative behaviors, we developed a high-resolution, long working distance, multi-core fiber optic imaging platform that allows visualization of cortical astrocyte calcium transients through a cranial window in freely moving mice. Using this platform, we defined the spatiotemporal dynamics of astrocytes during diverse behaviors, ranging from circadian fluctuations to novelty exploration, showing that astrocyte activity patterns are more variable and less synchronous than apparent in head-immobilized imaging conditions. Although the activity of astrocytes in visual cortex was highly synchronized during quiescence to arousal transitions, individual astrocytes often exhibited distinct thresholds and activity patterns during explorative behaviors, in accordance with their molecular diversity, allowing temporal sequencing across the astrocyte network. Imaging astrocyte activity during self-initiated behaviors revealed that noradrenergic and cholinergic systems act synergistically to recruit astrocytes during state transitions associated with arousal and attention, which was profoundly modulated by internal state. The distinct activity patterns exhibited by astrocytes in the cerebral cortex may provide a means to vary their neuromodulatory influence in response to different behaviors and internal states.

Unsupervised and quantitative intestinal ischemia detection using conditional adversarial network in multimodal optical imaging.

Purpose: Intraoperative evaluation of bowel perfusion is currently dependent upon subjective assessment. Thus, quantitative and objective methods of bowel viability in intestinal anastomosis are scarce. To address this clinical need, a conditional adversarial network is used to analyze the data from laser speckle contrast imaging (LSCI) paired with a visible-light camera to identify abnormal tissue perfusion regions. Approach: Our vision platform was based on a dual-modality bench-top imaging system with red-green-blue (RGB) and dye-free LSCI channels. Swine model studies were conducted to collect data on bowel mesenteric vascular structures with normal/abnormal microvascular perfusion to construct the control or experimental group. Subsequently, a deep-learning model based on a conditional generative adversarial network (cGAN) was utilized to perform dual-modality image alignment and learn the distribution of normal datasets for training. Thereafter, abnormal datasets were fed into the predictive model for testing. Ischemic bowel regions could be detected by monitoring the erroneous reconstruction from the latent space. The main advantage is that it is unsupervised and does not require subjective manual annotations. Compared with the conventional qualitative LSCI technique, it provides well-defined segmentation results for different levels of ischemia. Results: We demonstrated that our model could accurately segment the ischemic intestine images, with a Dice coefficient and accuracy of 90.77% and 93.06%, respectively, in 2560 RGB/LSCI image pairs. The ground truth was labeled by multiple and independent estimations, combining the surgeons' annotations with fastest gradient descent in suspicious areas of vascular images. The total processing time was 0.05 s for an image size of 256 × 256 . Conclusions: The proposed cGAN can provide pixel-wise and dye-free quantitative analysis of intestinal perfusion, which is an ideal supplement to the traditional LSCI technique. It has potential to help surgeons increase the accuracy of intraoperative diagnosis and improve clinical outcomes of mesenteric ischemia and other gastrointestinal surgeries.

Convolutional neural network-based common-path optical coherence tomography A-scan boundary-tracking training and validation using a parallel Monte Carlo synthetic dataset.

We present a parallel Monte Carlo (MC) simulation platform for rapidly generating synthetic common-path optical coherence tomography (CP-OCT) A-scan image dataset for image-guided needle insertion. The computation time of the method has been evaluated on different configurations and 100000 A-scan images are generated based on 50 different eye models. The synthetic dataset is used to train an end-to-end convolutional neural network (Ascan-Net) to localize the Descemet's membrane (DM) during the needle insertion. The trained Ascan-Net has been tested on the A-scan images collected from the ex-vivo human and porcine cornea as well as simulated data and shows improved tracking accuracy compared to the result by using the Canny-edge detector.

Depth-resolved backscattering signal reconstruction based OCT attenuation compensation.

Optical coherence tomography (OCT) with a robust depth-resolved attenuation compensation method for a wide range of imaging applications is proposed and demonstrated. We derive a model for deducing the attenuation coefficients and the signal compensation value using the depth-dependent backscattering profiles, to mitigate under and over-estimation in tissue imaging. We validated the method using numerical simulation and phantoms, where we achieved stable and robust compensation results over the entire depth of samples. The comparison between other attenuation characterization models and our proposed model is also performed.

Downward viewing common-path optical coherence tomography guided hydro-dissection needle for DALK.

Deep anterior lamellar keratoplasty (DALK) is a partial-thickness cornea transplant procedure in which only the recipient's stroma is replaced, leaving the host's Descemet's membrane (DM) and endothelium intact. This highly challenging "Big Bubble" procedure requires micron accuracy to insert a hydro-dissection needle as close as possible to the DM. Here, we report the design and evaluation of a downward viewing common-path optical coherence tomography (OCT) guided hydro-dissection needle for DALK. This design offers the flexibility of using different insertion angles and needle sizes. With the fiber situated outside the needle and eye, the needle can use its' full lumen for a smoother air/fluid injection and image quality is improved. The common-path OCT probe uses a bare optical fiber with its tip cleaved at the right angle for both reference and sample arm which is encapsulated in a 25-gauge stainless still tube. The fiber was set up vertically with a half-ball epoxy lens at the end to provide an A-scan with an 11-degree downward field of view. The hydro dissection needle was set up at 70 degrees from vertical and the relative position between the fiber end and the needle tip remained constant during the insertion. The fiber and needle were aligned by a customized needle driver to allow the needle tip and tissue underneath to both be imaged within the same A-scan. Fresh porcine eyes (N = 5) were used for the studies. The needle tip position, the stroma, and DM were successfully identified from the A-scan during the whole insertion process. The results showed the downward viewing OCT distal sensor can accurately guide the needle insertion for DALK and improved the average insertion depth compared to freehand insertion.

Overcoming the Force Limitations of Magnetic Robotic Surgery: Magnetic Pulse Actuated Collisions for Tissue-Penetrating-Needle for Tetherless Interventions.

The field of magnetic robotics aims to obviate physical connections between the actuators and end-effectors. Such tetherless control may enable new ultra-minimally invasive surgical manipulations in clinical settings. While wireless actuation offers advantages in medical applications, the challenge of providing sufficient force to magnetic needles for tissue penetration remains a barrier to practical application. Applying sufficient force for tissue penetration is required for tasks such as biopsy, suturing, cutting, drug delivery, and accessing deep seated regions of complex structures in organs such as the eye. To expand the force landscape for such magnetic surgical tools, an impact-force based suture needle capable of penetrating in vitro and ex vivo samples with 3-DOF planar motion is proposed. Using custom-built 14G and 25G needles, we demonstrate generation of 410 mN penetration force, a 22.7-fold force increase with more than 20 times smaller volume compared to similar magnetically guided needles. With the MPACT-Needle, in vitro suturing of a gauze mesh onto an agar gel is demonstrated. In addition, we have reduced the tip size to 25G, which is a typical needle size for interventions in the eye, to demonstrate ex vivo penetration in a rabbit eye, mimicking procedures such as corneal injections and transscleral drug delivery.

Higher-order regression three-dimensional motion-compensation method for real-time optical coherence tomography volumetric imaging of the cornea.

SIGNIFICANCE: Optical coherence tomography (OCT) allows high-resolution volumetric three-dimensional (3D) imaging of biological tissues in vivo. However, 3D-image acquisition can be time-consuming and often suffers from motion artifacts due to involuntary and physiological movements of the tissue, limiting the reproducibility of quantitative measurements. AIM: To achieve real-time 3D motion compensation for corneal tissue with high accuracy. APPROACH: We propose an OCT system for volumetric imaging of the cornea, capable of compensating both axial and lateral motion with micron-scale accuracy and millisecond-scale time consumption based on higher-order regression. Specifically, the system first scans three reference B-mode images along the C-axis before acquiring a standard C-mode image. The difference between the reference and volumetric images is compared using a surface-detection algorithm and higher-order polynomials to deduce 3D motion and remove motion-related artifacts. RESULTS: System parameters are optimized, and performance is evaluated using both phantom and corneal (ex vivo) samples. An overall motion-artifact error of <4.61 microns and processing time of about 3.40 ms for each B-scan was achieved. CONCLUSIONS: Higher-order regression achieved effective and real-time compensation of 3D motion artifacts during corneal imaging. The approach can be expanded to 3D imaging of other ocular tissues. Implementing such motion-compensation strategies has the potential to improve the reliability of objective and quantitative information that can be extracted from volumetric OCT measurements.

Deep point cloud landmark localization for fringe projection profilometry.

Point clouds have been widely used due to their information being richer than images. Fringe projection profilometry (FPP) is one of the camera-based point cloud acquisition techniques that is being developed as a vision system for robotic surgery. For semi-autonomous robotic suturing, fluorescent fiducials were previously used on a target tissue as suture landmarks. This not only increases system complexity but also imposes safety concerns. To address these problems, we propose a numerical landmark localization algorithm based on a convolutional neural network (CNN) and a conditional random field (CRF). A CNN is applied to regress landmark heatmaps from the four-channel image data generated by the FPP. A CRF leveraging both local and global shape constraints is developed to better tune the landmark coordinates, reject extra landmarks, and recover missing landmarks. The robustness of the proposed method is demonstrated through ex vivo porcine intestine landmark localization experiments.

OCT-guided Robotic Subretinal Needle Injections: A Deep Learning-Based Registration Approach.

Subretinal injection (SI) is an ophthalmic surgical procedure that allows for the direct injection of therapeutic substances into the subretinal space to treat vitreoretinal disorders. Although this treatment has grown in popularity, various factors contribute to its difficulty. These include the retina's fragile, nonregenerative tissue, as well as hand tremor and poor visual depth perception. In this context, the usage of robotic devices may reduce hand tremors and facilitate gradual and controlled SI. For the robot to successfully move to the target area, it needs to understand the spatial relationship between the attached needle and the tissue. The development of optical coherence tomography (OCT) imaging has resulted in a substantial advancement in visualizing retinal structures at micron resolution. This paper introduces a novel foundation for an OCT-guided robotic steering framework that enables a surgeon to plan and select targets within the OCT volume. At the same time, the robot automatically executes the trajectories necessary to achieve the selected targets. Our contribution consists of a novel combination of existing methods, creating an intraoperative OCT-Robot registration pipeline. We combined straightforward affine transformation computations with robot kinematics and a deep neural network-determined tool-tip location in OCT. We evaluate our framework's capability in a cadaveric pig eye open-sky procedure and using an aluminum target board. Targeting the subretinal space of the pig eye produced encouraging results with a mean Euclidean error of 23.8µm.

Optimization of Near-Infrared Fluorescence Voltage-Sensitive Dye Imaging for Neuronal Activity Monitoring in the Rodent Brain.

Many currently employed clinical brain functional imaging technologies rely on indirect measures of activity such as hemodynamics resulting in low temporal and spatial resolutions. To improve upon this, optical systems were developed in conjunction with methods to deliver near-IR voltage-sensitive dye (VSD) to provide activity-dependent optical contrast to establish a clinical tool to facilitate direct monitoring of neuron depolarization through the intact skull. Following the previously developed VSD delivery protocol through the blood-brain barrier, IR-780 perchlorate VSD concentrations in the brain were varied and stimulus-evoked responses were observed. In this paper, a range of optimal VSD tissue concentrations was established that maximized fluorescence fractional change for detection of membrane potential responses to external stimuli through a series of phantom, in vitro, ex vivo, and in vivo experiments in mouse models.

Stabilizing the phase of swept-source optical coherence tomography by a wrapped Gaussian mixture model.

The phase of an optical coherence tomography (OCT) signal carries critical information about particle micro-displacements. However, swept-source OCT (SSOCT) suffers from phase instability problems due to trigger jitters from the swept source. In this Letter, a wrapped Gaussian mixture model (WGMM) is proposed to stabilize the phase of SSOCT systems. A closed-form iteration solution of the WGMM is derived using the expectation-maximization algorithm. Necessary approximations are made for real-time graphic processing unit implementation. The performance of the proposed method is demonstrated through ex vivo, in vivo, and flow phantom experiments. The results show the robustness of the method in different application scenarios.

Optimized OCT-based depth-resolved model for attenuation compensation using point-spread-function calibration.

Optical coherence tomography (OCT) with a robust depth-resolved attenuation compensation method for a wide range of imaging applications is proposed and demonstrated. The proposed novel OCT attenuation compensation algorithm introduces an optimized axial point spread function (PSF) to modify existing depth-resolved methods and mitigates under and overestimation in biological tissues, providing a uniform resolution over the entire imaging range. The preliminary study is implemented using A-mode numerical simulation, where this method achieved stable and robust compensation results over the entire depth of samples. The experiment results using phantoms and corneal imaging exhibit agreement with the simulation result evaluated using signal-to-noise (SNR) and contrast-to-noise (CNR) metrics.

Convolutional Neural Network-based Optical Coherence Tomography (OCT) A-scan Segmentation and Tracking Platform using Advanced Monte Carlo Simulation.

We reported a parallel Monte Carlo simulation platform for generating OCT cornea images and training the convolutional neural network. The trained network showed improved segmentation results when applied to the ex-vivo cornea A-scan images.

Molecular Radiative Energy Shifts under Strong Oscillating Fields.

Coherent manipulation of light-matter interactions is pivotal to the advancement of nanophotonics. Conventionally, the non-resonant optical Stark effect is harnessed for band engineering by intense laser pumping. However, this method is hindered by the transient Stark shifts and the high-energy laser pumping which, by itself, is precluded as a nanoscale optical source due to light diffraction. As an analog of photons in a laser, surface plasmons are uniquely positioned to coherently interact with matter through near-field coupling, thereby, providing a potential source of electric fields. Herein, the first demonstration of plasmonic Stark effect is reported and attributed to a newly uncovered energy-bending mechanism. As a complementary approach to the optical Stark effect, it is envisioned that the plasmonic Stark effect will advance fundamental understanding of coherent light-matter interactions and will also provide new opportunities for advanced optoelectronic tools, such as ultrafast all-optical switches and biological nanoprobes at lower light power levels.