Arc-to-line frame registration method for ultrasound and photoacoustic image-guided intraoperative robot-assisted laparoscopic prostatectomy.

PURPOSE: To achieve effective robot-assisted laparoscopic prostatectomy, the integration of transrectal ultrasound (TRUS) imaging system which is the most widely used imaging modality in prostate imaging is essential. However, manual manipulation of the ultrasound transducer during the procedure will significantly interfere with the surgery. Therefore, we propose an image co-registration algorithm based on a photoacoustic marker (PM) method, where the ultrasound/photoacoustic (US/PA) images can be registered to the endoscopic camera images to ultimately enable the TRUS transducer to automatically track the surgical instrument. METHODS: An optimization-based algorithm is proposed to co-register the images from the two different imaging modalities. The principle of light propagation and an uncertainty in PM detection were assumed in this algorithm to improve the stability and accuracy of the algorithm. The algorithm is validated using the previously developed US/PA image-guided system with a da Vinci surgical robot. RESULTS: The target-registration-error (TRE) is measured to evaluate the proposed algorithm. In both simulation and experimental demonstration, the proposed algorithm achieved a sub-centimeter accuracy which is acceptable in practical clinics (i.e., 1.15 ± 0.29 mm from the experimental evaluation). The result is also comparable with our previous approach (i.e., 1.05 ± 0.37 mm), and the proposed method can be implemented with a normal white light stereo camera and does not require highly accurate localization of the PM. CONCLUSION: The proposed frame registration algorithm enabled a simple yet efficient integration of commercial US/PA imaging system into laparoscopic surgical setting by leveraging the characteristic properties of acoustic wave propagation and laser excitation, contributing to automated US/PA image-guided surgical intervention applications.

Automatic search for photoacoustic marker using automated transrectal ultrasound.

Real-time transrectal ultrasound (TRUS) image guidance during robot-assisted laparoscopic radical prostatectomy has the potential to enhance surgery outcomes. Whether conventional or photoacoustic TRUS is used, the robotic system and the TRUS must be registered to each other. Accurate registration can be performed using photoacoustic (PA markers). However, this requires a manual search by an assistant [IEEE Robot. Autom. Lett8, 1287 (2023).10.1109/LRA.2022.3191788]. This paper introduces the first automatic search for PA markers using a transrectal ultrasound robot. This effectively reduces the challenges associated with the da Vinci-TRUS registration. This paper investigated the performance of three search algorithms in simulation and experiment: Weighted Average (WA), Golden Section Search (GSS), and Ternary Search (TS). For validation, a surgical prostate scenario was mimicked and various ex vivo tissues were tested. As a result, the WA algorithm can achieve 0.53°±0.30° average error after 9 data acquisitions, while the TS and GSS algorithm can achieve 0.29∘±0.31∘ and 0.48°±0.32° average errors after 28 data acquisitions.

Real-time intraoperative surgical guidance system in the da Vinci surgical robot based on transrectal ultrasound/photoacoustic imaging with photoacoustic markers: an ex vivo demonstration.

This paper introduces the first integrated real-time intraoperative surgical guidance system, in which an endoscope camera of da Vinci surgical robot and a transrectal ultrasound (TRUS) transducer are co-registered using photoacoustic markers that are detected in both fluorescence (FL) and photoacoustic (PA) imaging. The co-registered system enables the TRUS transducer to track the laser spot illuminated by a pulsed-laser-diode attached to the surgical instrument, providing both FL and PA images of the surgical region-of-interest (ROI). As a result, the generated photoacoustic marker is visualized and localized in the da Vinci endoscopic FL images, and the corresponding tracking can be conducted by rotating the TRUS transducer to display the PA image of the marker. A quantitative evaluation revealed that the average registration and tracking errors were 0.84 mm and 1.16°, respectively. This study shows that the co-registered photoacoustic marker tracking can be effectively deployed intraoperatively using TRUS+PA imaging providing functional guidance of the surgical ROI.

Laser diode photoacoustic point source detection: machine learning-based denoising and reconstruction.

A new development in photoacoustic (PA) imaging has been the use of compact, portable and low-cost laser diodes (LDs), but LD-based PA imaging suffers from low signal intensity recorded by the conventional transducers. A common method to improve signal strength is temporal averaging, which reduces frame rate and increases laser exposure to patients. To tackle this problem, we propose a deep learning method that will denoise point source PA radio-frequency (RF) data before beamforming with a very few frames, even one. We also present a deep learning method to automatically reconstruct point sources from noisy pre-beamformed data. Finally, we employ a strategy of combined denoising and reconstruction, which can supplement the reconstruction algorithm for very low signal-to-noise ratio inputs.

Synthetic radial aperture focusing to regulate manual volumetric scanning for economic transrectal ultrasound imaging.

In this paper, we present a volumetric transrectal ultrasound (TRUS) imaging under the presence of radial scanning angle disorientation (SAD) in a resource-limited diagnostic setting. Herein, we test our hypothesis that a synthetic radial aperture focusing (TRUS-rSAF) technique, in which a radial plane in target volume is reconstructed by coherent compounding of multiple transmittance/reception events, will reject a randomized SAD in a free-hand scanning setup based on external angular tracking. Based on an analytical model of the TRUS-rSAF technique, we first tested specific scenarios using a clinically available TRUS transducer under different SADs in a range of normal distributions (σ = 0.1°, 0.2°, 0.5°, 1°, 2°, and 5°). We found a benefit of the TRUS-rSAF technique for higher robustness when the SAD is contained within the radial synthetic aperture window, i.e., ±0.71° from a target scanning angle. However, no enhancement was found in spatial resolution because of the limited transmit beam field of the clinical TRUS transducer, limiting the synthetic aperture window. We further evaluated the TRUS-rSAF technique with a modified TRUS transducer for an extended synthetic aperture window to test whether higher spatial resolution and robustness to SAD can be obtained in the same evaluation setup. Widening of the synthetic aperture window (±3.54°, ± 5.91°, ± 8.27°, ± 10.63°, ± 12.99°, ± 15.35°) resulted in proportional enhancements of spatial resolution, but it also progressively built up sidelobe artifacts due to randomized synthesis with limited phase cancellations. The results suggest the need for careful calibration of the TRUS-rSAF technique to enable TRUS imaging with free-hand radial scanning and external angle tracking in resource-limited settings.

Wearable Mechatronic Ultrasound-integrated AR Navigation System for Lumbar Puncture Guidance.

As one of the most commonly performed spinal interventions in routine clinical practice, lumbar punctures are usually done with only hand palpation and trial-and-error. Failures can prolong procedure time and introduce complications such as cerebrospinal fluid leaks and headaches. Therefore, an effective needle insertion guidance method is desired. In this work, we present a complete lumbar puncture guidance system with the integration of (1) a wearable mechatronic ultrasound imaging device, (2) volume-reconstruction and bone surface estimation algorithms and (3) two alternative augmented reality user interfaces for needle guidance, including a HoloLens-based and a tablet-based solution. We conducted a quantitative evaluation of the end-to-end navigation accuracy, which shows that our system can achieve an overall needle navigation accuracy of 2.83 mm and 2.76 mm for the Tablet-based and the HoloLens-based solutions, respectively. In addition, we conducted a preliminary user study to qualitatively evaluate the effectiveness and ergonomics of our system on lumbar phantoms. The results show that users were able to successfully reach the target in an average of 1.12 and 1.14 needle insertion attempts for Tablet-based and HoloLens-based systems, respectively, exhibiting the potential to reduce the failure rates of lumbar puncture procedures with the proposed lumbar-puncture guidance.

A novel design framework of synthetic radial aperture focusing for volumetric transrectal ultrasound imaging.

In this paper, we present a novel design framework of synthetic radial aperture focusing for three-dimensional (3D) transrectal ultrasound imaging (TRUS-rSAF), in which multiple transmittance/reception events at different scanning angles are synthesized to reconstruct a radial plane in the target volume, securing high spatial resolution and texture uniformity. A theory-based design approach has not been available to push the envelope of the 3D rSAF technique. Herein, a closed-form analytical description of the TRUS-rSAF method is presented for the first time, effectively delineating spatial resolution and grating lobe positions in the radial dimension of a TRUS transducer. We demonstrate a solid optimization workflow based on the theoretical foundation to improve its spatiotemporal resolution, grating lobe artifacts, and signal-to-noise ratio. A specific design criterion was considered to outperform a clinical 3D TRUS imaging as a reference (TRUS-REF), where each radial plane is reconstructed with a single transmittance/reception event using a motorized actuator. The optimized TRUS-rSAF method significantly enhanced spatial resolution up to 50% over the TRUS-REF method while providing clinically effective temporal resolution (2-8 volume/sec) with negligible grating lobe artifacts. The results indicate that the proposed design approach would enable a novel TRUS imaging solution in clinics.

3D-printed Magnetic Resonance (MR)-based gynecological phantom for image-guided brachytherapy training.

PURPOSE/OBJECTIVES: There is a clinical need to develop anatomic phantoms for simulation-based learning in gynecological brachytherapy. Here, we provide a step-by-step approach to build a life-sized gynecological training phantom based on magnetic resonance imaging (MRI) of an individual patient. Our hypothesis is that this phantom can generate convincing ultrasound (US) images that are similar to patient scans. METHODS: Organs-at-risk were manually segmented using patient scans (MRI). The gynecological phantom was constructed using positive molds from 3D printing and polyvinyl chloride (PVC) plastisol. Tissue texture/acoustic properties were simulated using different plastic softener/hardener ratios and microbead densities. Nine readers (residents) were asked to evaluate 10 cases (1 ultrasound image per case) and categorize each as a "patient" or "phantom" image. To evaluate whether the phantom and patient images were equivalent, we used a multireader, multicase equivalence study design with two composite null hypotheses with proportion (pr) at H01: pr ≤ 0.35 and H02: pr ≥ 0.65. Readers were also asked to review US videos and identify the insertion of an interstitial needle into the pelvic phantom. Computed Tomography (CT) and magnetic resonance (MR) images of the phantom were acquired for a feasibility study. RESULTS: Readers correctly classified "patient" and "phantom" scans at pr = 53.3% ± 6.2% (p values 0.013 for H01 and 0.054 for H02, df = 5.96). Readers reviewed US videos and identified the interstitial needle 100% of the time in transabdominal view, and 78% in transrectal view. The phantom was CT and MR safe. CONCLUSIONS: We have outlined a manufacturing process to create a life-sized, gynecological phantom that is compatible with multi-modality imaging and can be used to simulate clinical scenarios in image-guided brachytherapy procedures.

System-level optimization in spectroscopic photoacoustic imaging of prostate cancer.

This study presents a system-level optimization of spectroscopic photoacoustic (PA) imaging for prostate cancer (PCa) detection in three folds. First, we present a spectral unmixing model to segregate spectral system error (SSE). We constructed two noise models (NMs) for the laser spectrotemporal fluctuation and the ultrasound system noise. We used these NMs in linear spectral unmixing to denoise and to achieve high temporal resolution. Second, we employed a simulation-aided wavelength optimization to select the most effective subset of wavelengths. NMs again were considered so that selected wavelengths were not only robust to the collinearity of optical absorbance, but also to noise. Third, we quantified the effect of frame averaging on improving spectral unmixing accuracy through theoretical analysis and numerical validation. To validate the whole framework, we performed comprehensive studies in simulation and an in vivo experiment which evaluated prostate-specific membrane antigen (PSMA) expression in PCa on a mice model. Both simulation analysis and in vivo studies confirmed that the proposed framework significantly enhances image signal-to-noise ratio (SNR) and spectral unmixing accuracy. It enabled more sensitive and faster PCa detection. Moreover, the proposed framework can be generalized to other spectroscopic PA imaging studies for noise reduction, wavelength optimization, and higher temporal resolution.

AutoInFocus, a new paradigm for ultrasound-guided spine intervention: a multi-platform validation study.

PURPOSE: Ultrasound-guided spine interventions often suffer from the insufficient visualization of key anatomical structures due to the complex shapes of the self-shadowing vertebrae. Therefore, we propose an ultrasound imaging paradigm, AutoInFocus (automatic insonification optimization with controlled ultrasound), to improve the key structure visibility. METHODS: A phased-array probe is used in conjunction with a motion platform to image a controlled workspace, and the resulting images from multiple insonification angles are combined to reveal the target anatomy. This idea is first evaluated in simulation and then realized as a robotic platform and a miniaturized patch device. A spine phantom (CIRS) and its CT scan were used in the evaluation experiments to quantitatively and qualitatively analyze the advantages of the proposed method over the traditional approach. RESULTS: We showed in simulation that the proposed system setup increased the visibility of interspinous space boundary, a key feature for lumbar puncture guidance, from 44.13 to 67.73% on average, and the 3D spine surface coverage from 14.31 to 35.87%, compared to traditional imaging setup. We also demonstrated the feasibility of both robotic and patch-based realizations in a spine phantom study. CONCLUSION: This work lays the foundation for a new imaging paradigm that leverages redundant and controlled insonification to allow for imaging optimization of the complex vertebrae anatomy, making it possible for high-quality visualization of key anatomies during ultrasound-guided spine interventions.

Bi-Directional Semi-Supervised Training of Convolutional Neural Networks for Ultrasound Elastography Displacement Estimation.

The performance of ultrasound elastography (USE) heavily depends on the accuracy of displacement estimation. Recently, convolutional neural networks (CNNs) have shown promising performance in optical flow estimation and have been adopted for USE displacement estimation. Networks trained on computer vision images are not optimized for USE displacement estimation since there is a large gap between the computer vision images and the high-frequency radio frequency (RF) ultrasound data. Many researchers tried to adopt the optical flow CNNs to USE by applying transfer learning to improve the performance of CNNs for USE. However, the ground-truth displacement in real ultrasound data is unknown, and simulated data exhibit a domain shift compared to the real data and are also computationally expensive to generate. To resolve this issue, semisupervised methods have been proposed in which the networks pretrained on computer vision images are fine-tuned using real ultrasound data. In this article, we employ a semisupervised method by exploiting the first- and second-order derivatives of the displacement field for regularization. We also modify the network structure to estimate both forward and backward displacements and propose to use consistency between the forward and backward strains as an additional regularizer to further enhance the performance. We validate our method using several experimental phantom and in vivo data. We also show that the network fine-tuned by our proposed method using experimental phantom data performs well on in vivo data similar to the network fine-tuned on in vivo data. Our results also show that the proposed method outperforms current deep learning methods and is comparable to computationally expensive optimization-based algorithms.

Photoacoustic assessment of the fetal brain and placenta as a method of non-invasive antepartum and intrapartum monitoring.

A noninvasive monitor for concurrent evaluation of placental and fetal sagittal sinus sO 2 for both antepartum surveillance at the late 2nd and 3rd trimesters and intrapartum monitoring would be a great advantage over current methods. A PA fetal brain and placental monitor has potential value to rapidly identify the fetus at risk for developing hypoxia and ischemia of a sufficient degree that brain injury or death may develop, which may be prevented by intervention with delivery and other follow-up treatments.

Transcranial photoacoustic characterization of neurovascular physiology during early-stage photothrombotic stroke in neonatal pigletsin vivo.

OBJECTIVE: Perinatal ischemic stroke is estimated to occur in 1/2300-1/5000 live births, but early differential diagnosis from global hypoxia-ischemia is often difficult. In this study, we tested the ability of a hand-held transcranial photoacoustic (PA) imaging probe to non-invasively detect a focal photothrombotic stroke (PTS) within 2 h of stroke onset in a gyrencephalic piglet brain. APPROACH: About 17 stroke lesions of approximately 1 cm2area were introduced randomly in anterior or posterior cortex via the light/dye PTS technique in anesthetized neonatal piglets (n= 11). The contralateral non-ischemic region served as control tissue for discrimination contrast for the PA hemoglobin metrics: oxygen saturation, total hemoglobin (tHb), and individual quantities of oxygenated and deoxygenated hemoglobin (HbO2and HbR). MAIN RESULTS: The PA-derived tissue oxygen saturation at 2 h yielded a significant separation between control and affected regions-of-interest (p< 0.0001), which were well matched with 24 h post-stroke cerebral infarction confirmed in the triphenyltetrazolium chloride-stained image. The quantity of HbO2also displayed a significant contrast (p= 0.021), whereas tHb and HbR did not. The analysis on receiver operating characteristic curves and multivariate data analysis also agreed with the results above. SIGNIFICANCE: This study shows that a hand-held transcranial PA neuroimaging device can detect a regional thrombotic stroke in the cerebral cortex of a neonatal piglet. In particular, we conclude that the oxygen saturation metric can be used alone to identify regional stroke lesions. The lack of change in tHb may be related to arbitrary hand-held imaging configuration and/or entrapment of red blood cells within the thrombotic stroke.

Optimization of Near-Infrared Fluorescence Voltage-Sensitive Dye Imaging for Neuronal Activity Monitoring in the Rodent Brain.

Many currently employed clinical brain functional imaging technologies rely on indirect measures of activity such as hemodynamics resulting in low temporal and spatial resolutions. To improve upon this, optical systems were developed in conjunction with methods to deliver near-IR voltage-sensitive dye (VSD) to provide activity-dependent optical contrast to establish a clinical tool to facilitate direct monitoring of neuron depolarization through the intact skull. Following the previously developed VSD delivery protocol through the blood-brain barrier, IR-780 perchlorate VSD concentrations in the brain were varied and stimulus-evoked responses were observed. In this paper, a range of optimal VSD tissue concentrations was established that maximized fluorescence fractional change for detection of membrane potential responses to external stimuli through a series of phantom, in vitro, ex vivo, and in vivo experiments in mouse models.

Dual contrast agents for fluorescence and photoacoustic imaging: evaluation in a murine model of prostate cancer.

Prostate-specific membrane antigen (PSMA) is a promising diagnostic and therapeutic target for prostate cancer (PC). Poly(amidoamine) [PAMAM] dendrimers serve as versatile scaffolds for imaging agents and drug delivery that can be tailored to different sizes and compositions depending upon the application. We have developed PSMA-targeted PAMAM dendrimers for real-time detection of PC using fluorescence (FL) and photoacoustic (PA) imaging. A generation-4, ethylenediamine core, amine-terminated dendrimer was consecutively conjugated with on average 10 lysine-glutamate-urea PSMA targeting moieties and a different number of sulfo-cyanine7.5 (Cy7.5) near-infrared dyes (2, 4, 6 and 8 denoted as conjugates II, III, IV and V, respectively). The remaining terminal primary amines were capped with butane-1,2-diol functionalities. We also prepared a conjugate composed of Cy7.5-lysine-suberic acid-lysine glutamate-urea (I) and control dendrimer conjugate (VI). Among all conjugates, IV showed superior in vivo target specificity in male NOD-SCID mice bearing isogenic PSMA+ PC3 PIP and PSMA- PC3 flu xenografts and suitable physicochemical properties for FL and PA imaging. Such agents may prove useful in PC cancer detection and subsequent surgical guidance during excision of PSMA-expressing lesions.

The Use of Three-dimensional Visualization Techniques for Prostate Procedures: A Systematic Review.

CONTEXT: As an emerging technique, three-dimensional (3D) visualization has become more popular and can facilitate education, training, surgical planning, and intraoperative guidance for prostate cancer surgery. OBJECTIVE: In this review, we aim to present the impact of 3D printing, virtual reality (VR), and augmented reality (AR) techniques for prostate cancer procedures, specifically prostate biopsy and radical prostatectomy (RP). EVIDENCE ACQUISITION: A systematic review was performed by two investigators according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) criteria. EVIDENCE SYNTHESIS: A total of 541 papers were identified in PubMed, Scopus, and Embase. Of these, 53 studies were identified for detailed review and 25 were qualified. Two more studies were identified from the references; thus, 27 studies were finally included in this systematic review. Nine papers reported on the use of 3D reconstructed models, mainly in education/training and intraoperative guidance; nine reported on VR, focusing on simulation training model and intraoperative guidance; and nine reported on AR technique with its best indication for surgical guidance in robotic RP. CONCLUSIONS: Three-dimensional visualization techniques have gradually been introduced and developed in prostate procedures, and demonstrate potential utility not only for education/training, but also for surgical planning and intraoperative guidance. Prospective studies are needed to demonstrate clinical utility and validation of these technologies. PATIENT SUMMARY: Despite low-quality evidence, promising signals were identified to demonstrate that three-dimensional visualization could help facilitate prostate procedures, in terms of education/training, surgical planning, and intraoperative guidance. It is still in a very early stage, and more studies need to be conducted to justify its widespread use.

Real-time, functional intra-operative localization of rat cavernous nerve network using near-infrared cyanine voltage-sensitive dye imaging.

Despite current progress achieved in the surgical technique of radical prostatectomy, post-operative complications such as erectile dysfunction and urinary incontinence persist at high incidence rates. In this paper, we present a methodology for functional intra-operative localization of the cavernous nerve (CN) network for nerve-sparing radical prostatectomy using near-infrared cyanine voltage-sensitive dye (VSD) imaging, which visualizes membrane potential variations in the CN and its branches (CNB) in real time. As a proof-of-concept experiment, we demonstrate a functioning complex nerve network in response to electrical stimulation of the CN, which was clearly differentiated from surrounding tissues in an in vivo rat prostate model. Stimulation of an erection was confirmed by correlative intracavernosal pressure (ICP) monitoring. Within 10 minutes, we performed trans-fascial staining of the CN by direct VSD administration. Our findings suggest the applicability of VSD imaging for real-time, functional imaging guidance during nerve-sparing radical prostatectomy.

An economic photoacoustic imaging platform using automatic laser synchronization and inverse beamforming.

We present a proof-of-concept of an automatic integration of photoacoustic (PA) imaging on clinical ultrasound (US) imaging platforms. Here we tackle two critical challenges: the laser synchronization and the inaccessibility to the beamformer core embedded in commercial US imaging platform. In particular, the line trigger frequency (LTF) estimation and the asynchronous synthetic aperture inverse beamforming (ASAIB) were developed and evaluated in both k-Wave simulation and phantom experiment. The proposed method is an economical solution to enable PA imaging on a greater number of US equipment to further thrive the PA imaging research community.

Transcranial photoacoustic imaging of NMDA-evoked focal circuit dynamics in the rat hippocampus.

OBJECTIVE: We report the transcranial functional photoacoustic (fPA) neuroimaging of N-methyl-D-aspartate (NMDA) evoked neural activity in the rat hippocampus. Concurrent quantitative electroencephalography (qEEG) and microdialysis were used to record real-time circuit dynamics and excitatory neurotransmitter concentrations, respectively. APPROACH: We hypothesized that location-specific fPA voltage-sensitive dye (VSD) contrast would identify neural activity changes in the hippocampus which correlate with NMDA-evoked excitatory neurotransmission. MAIN RESULTS: Transcranial fPA VSD imaging at the contralateral side of the microdialysis probe provided NMDA-evoked VSD responses with positive correlation to extracellular glutamate concentration changes. qEEG validated a wide range of glutamatergic excitation, which culminated in focal seizure activity after a high NMDA dose. We conclude that transcranial fPA VSD imaging can distinguish focal glutamate loads in the rat hippocampus, based on the VSD redistribution mechanism which is sensitive to the electrophysiologic membrane potential. SIGNIFICANCE: Our results suggest the future utility of this emerging technology in both laboratory and clinical sciences as an innovative functional neuroimaging modality.

Ultrasound Signal Detection with Multi-bounce Laser Microphone.

The multi-bounce laser microphone utilizes optical methods to detect the displacement of a gold-covered thin film diaphragm caused by ultrasound signal pressure waves. This sensitive all-optical sensing technique provides new opportunities for advanced ultrasound imaging as it is expected to achieve a higher detection signal-to-noise ratio (SNR) in a broader spectrum, as compared to conventional ultrasonic transducers. The technique does not involve signal time-averaging and the real-time enhancement in detection SNR stems from the amplification of signal strength due to multiple bouncing off the diaphragm. The system was previously developed for detecting acoustic signatures generated by explosives and were limited to lower than 10 kHz in frequency. To demonstrate its feasibility for biomedical imaging applications, preliminary experiments were conducted to show high fidelity detection of ultrasound waves with frequencies ranging from 100 kHz to in excess of 1 MHz. Experimental results are also presented in this work demonstrating the improved detection sensitivity of the multi-bounce laser microphone in detecting ultrasound signals when compared with a commercial Fabry-Perot type optical hydrophone. Furthermore, we also applied the multi-bounce laser microphone to detect photoacoustic signatures emitted by India ink when a LED bar is used as the excitation source without signal averaging.