A jointed feature fusion framework for photoacoustic image reconstruction.

The standard reconstruction of Photoacoustic (PA) computed tomography (PACT) image could cause the artifacts due to interferences or ill-posed setup. Recently, deep learning has been used to reconstruct the PA image with ill-posed conditions. In this paper, we propose a jointed feature fusion framework (JEFF-Net) based on deep learning to reconstruct the PA image using limited-view data. The cross-domain features from limited-view position-wise data and the reconstructed image are fused by a backtracked supervision. A quarter position-wise data (32 channels) is fed into model, which outputs another 3-quarters-view data (96 channels). Moreover, two novel losses are designed to restrain the artifacts by sufficiently manipulating superposed data. The experimental results have demonstrated the superior performance and quantitative evaluations show that our proposed method outperformed the ground-truth in some metrics by 135% (SSIM for simulation) and 40% (gCNR for in-vivo) improvement.

Deep learning adapted acceleration for limited-view photoacoustic image reconstruction.

The limited-view issue can cause a low-quality image in photoacoustic (PA) computed tomography due to the limitation of geometric condition. The model-based method is used to resolve this problem, which contains different regularization. To adapt fast and high-quality reconstruction of limited-view PA data, in this Letter, a model-based method that combines the mathematical variational model with deep learning is proposed to speed up and regularize the unrolled procedure of reconstruction. A deep neural network is designed to adapt the step of the gradient updated term of data consistency in the gradient descent procedure, which can obtain a high-quality PA image with only a few iterations. A comparison of different model-based methods shows that our proposed scheme has superior performances (over 0.05 for SSIM) with the same iteration (three times) steps. Finally, we find that our method obtains superior results (0.94 value of SSIM for in vivo) with a high robustness and accelerated reconstruction.

Programmable Acoustic Delay-Line Enabled Low-Cost Photoacoustic Tomography System.

Photoacoustic tomography (PAT) is an emerging technology for biomedical imaging that combines the superiorities of high optical contrast and acoustic penetration. In the PAT system, more PA signals are preferred to be detected from full field of view to reconstruct the PA images with higher fidelity. However, the requirement for more PA signals' detection leads to more time consumption for single-channel scanning-based PAT system or higher cost of data acquisition (DAQ) module for an array-based PAT system. To address this issue, we proposed a programmable acoustic delay-line (PADL) module to reduce DAQ cost and accelerate imaging speed for PAT system. The module is based on bidirectional conversion between acoustic signals and electrical signals, including ultrasound transmission in between to provide sufficient time delay. The acoustic delay-line module achieves tens or hundreds of microseconds' delay for each channel and is controlled by a programmable control unit. In this work, it achieves to merge four inputs of PA signals into one output signal, which can be recovered into original four PA signals in the digital domain after DAQ. The imaging experiments of pencil leads embedded in agar phantom are conducted by the PAT system equipped with the proposed PADL module, which demonstrated its feasibility to reduce the cost of the PAT system. An in vivo study of human finger PAT imaging with delay-line module verified its feasibility for biomedical imaging applications.

Size-adjustable ring-shape photoacoustic tomography imager in vivo.

Photoacoustic tomography (PAT) has become a novel biomedical imaging modality for scientific research and clinical diagnosis. It combines the advantages of spectroscopic optical absorption contrast and acoustic resolution with deep penetration. In this article, an imaging size-adjustable PAT system is proposed for potential clinical applications such as breast cancer detection and screening, which can adapt to imaging targets with various sizes. Comparing with the conventional PAT setup with a fixed radius ring shape ultrasound transducer (UT) array, the proposed system is more flexible for imaging diverse size targets based on sectorial ultrasound transducer arrays (SUTAs). Four SUTAs form a 128-channel UT array for photoacoustic detection, where each SUTA has 32 elements. Such four SUTAs are controlled by four stepper motors, respectively, and can change their distribution layout position to adapt for various imaging applications. In this proposed system, the radius of the imaging region of interest (ROI) can be adjusted from 50 to 100 mm, which is much more flexible than the conventional PAT system with a full ring UT array. The simulation experiments using the MATLAB k-wave toolbox demonstrate the feasibility of the proposed system. To further validate the proposed system, imaging of pencil leads made phantom, ex-vivo pork breast with indocyanine green (ICG) injected, and in-vivo human wrist, finger and ankle are conducted to prove its feasibility for potential clinical applications.

Learned Parameters and Increment for Iterative Photoacoustic Image Reconstruction via Deep Learning.

Photoacoustic (PA) tomography is a relatively new medical imaging technique that combines traditional ultrasound imaging and optical imaging, which has great application prospects in recent years. To reveal the light absorption coefficient of biological tissues, the images are reconstructed from PA signals by reconstruction algorithms. However, traditional model-based reconstruction method requires a huge number of iterations to obtain relatively good experimental results, which is quite time-consuming. In this paper, we propose to use deep learning method to replace brute parameter adjustment in model-based reconstruction, and speed up the rate of convergence by building convolutional neural networks (CNN). The parameters we defined in our model can be learned automatically. Meanwhile, our method can optimize the increment of gradient in each step of iteration. The numerical experiment validates our method, showing that only three iterations are needed to obtain the satisfactory image quality, which normally requires 10 iterations for tradition method. It demonstrated that efficiency of photoacoustic reconstruction can be greatly improved by our proposed method, compared with traditional model-based methods.

Deep learning enabled real-time photoacoustic tomography system via single data acquisition channel.

Photoacoustic computed tomography (PACT) combines the optical contrast of optical imaging and the penetrability of sonography. In this work, we develop a novel PACT system to provide real-time imaging, which is achieved by a 120-elements ultrasound array only using a single data acquisition (DAQ) channel. To reduce the channel number of DAQ, we superimpose 30 nearby channels' signals together in the analog domain, and shrinking to 4 channels of data (120/30 = 4). Furthermore, a four-to-one delay-line module is designed to combine these four channels' data into one channel before entering the single-channel DAQ, followed by decoupling the signals after data acquisition. To reconstruct the image from four superimposed 30-channels' PA signals, we train a dedicated deep learning model to reconstruct the final PA image. In this paper, we present the preliminary results of phantom and in-vivo experiments, which manifests its robust real-time imaging performance. The significance of this novel PACT system is that it dramatically reduces the cost of multi-channel DAQ module (from 120 channels to 1 channel), paving the way to a portable, low-cost and real-time PACT system.

Review of deep learning for photoacoustic imaging.

Machine learning has been developed dramatically and witnessed a lot of applications in various fields over the past few years. This boom originated in 2009, when a new model emerged, that is, the deep artificial neural network, which began to surpass other established mature models on some important benchmarks. Later, it was widely used in academia and industry. Ranging from image analysis to natural language processing, it fully exerted its magic and now become the state-of-the-art machine learning models. Deep neural networks have great potential in medical imaging technology, medical data analysis, medical diagnosis and other healthcare issues, and is promoted in both pre-clinical and even clinical stages. In this review, we performed an overview of some new developments and challenges in the application of machine learning to medical image analysis, with a special focus on deep learning in photoacoustic imaging. The aim of this review is threefold: (i) introducing deep learning with some important basics, (ii) reviewing recent works that apply deep learning in the entire ecological chain of photoacoustic imaging, from image reconstruction to disease diagnosis, (iii) providing some open source materials and other resources for researchers interested in applying deep learning to photoacoustic imaging.

Compressed sensing for photoacoustic computed tomography based on an untrained neural network with a shape prior.

Photoacoustic (PA) computed tomography (PACT) shows great potential in various preclinical and clinical applications. A great number of measurements are the premise that obtains a high-quality image, which implies a low imaging rate or a high system cost. The artifacts or sidelobes could pollute the image if we decrease the number of measured channels or limit the detected view. In this paper, a novel compressed sensing method for PACT using an untrained neural network is proposed, which decreases a half number of the measured channels and recovers enough details. This method uses a neural network to reconstruct without the requirement for any additional learning based on the deep image prior. The model can reconstruct the image only using a few detections with gradient descent. As an unlearned strategy, our method can cooperate with other existing regularization, and further improve the quality. In addition, we introduce a shape prior to easily converge the model to the image. We verify the feasibility of untrained network-based compressed sensing in PA image reconstruction and compare this method with a conventional method using total variation minimization. The experimental results show that our proposed method outperforms 32.72% (SSIM) with the traditional compressed sensing method in the same regularization. It could dramatically reduce the requirement for the number of transducers, by sparsely sampling the raw PA data, and improve the quality of PA image significantly.

Frequency-domain Dual-contrast Photoacoustic Imaging with Chirp Modulation.

Photoacoustic imaging has shown its great potential in biomedical imaging. A variety of imaging applications, like blood oxygenation for functional imaging, have been widely studied during the past few decades. Most of the previous works are based on the tissue's endogenous or nanoprobe's extraneous optical absorbance. In this paper, we proposed frequency-domain dual-contrast photoacoustic imaging aiming at exploring both optical absorption and mechanical property (e.g., viscoelasticity) of tissue. Instead of conventionally used pulsed excitation, a chirp-modulated laser signal is used to excite the sample to induce photoacoustic signals. On one hand, the optical absorption contrast is obtained by cross-correlating the PA signals with the chirp pattern. On the other hand, mechanical property is obtained by performing the Fourier transform to analyze the frequency spectrum. Experimental results revealed that samples with higher density-to-viscoelasticity ratio show larger quality factor in the received PA signals' spectrum. Both theoretical analysis and experimental demonstrations are performed to prove the feasibility of the proposed method.

Photoacoustic-guided Transcranial HIFU with Combined Time-reversal and Genetic Algorithm.

High intensity focused ultrasound (HIFU) is a noninvasive therapy used to induce tissue ablation for treating malignant tissues. Photoacoustic (PA) has recently been proposed as an alternative method to guide HIFU. In this paper, we present a method of HIFU guided by time-reversing the transcranial PA signals of an optically selective target in a nonselective background. To improve the focus performance on target area, we further propose to utilize the time-reversed PA signals as the initial population of Genetic Algorithm (GA) to optimize the focusing iteratively. In particular, we mimic both optical and acoustic parameters of the human brain and intracranial media in the simulation study. Experimental results show that the focusing accuracy of the proposed method has been significantly improved compared to just one-step PA time-reversal. At the same time, the combination of TR and GA makes the iteration time consumption of the optimization process less than other traditional algorithms without TR, showing its potential HIFU in clinical scenarios.

Low-Cost Multi-Wavelength Photoacoustic Imaging Based on Portable Continuous-Wave Laser Diode Module.

Photoacoustic imaging (PAI), an emerging imaging technique, exploits the merits of both optical and ultrasound imaging, equipped with optical contrast and deep penetration. Typical linear PAI relies on a nanosecond laser pulse to induce photoacoustic signals. To construct a multi-wavelength PAI system, a multi-wavelength nano-second laser source is required, which greatly increases the cost of the PAI system. However, according to the nonlinear photoacoustic effect, the amplitude of the photoacoustic signals will vary with different base temperatures of the tissue. Therefore, using continuous-wave lasers with different wavelengths to induce different temperature variations at the same point of the tissue, and then using a single-wavelength pulsed laser to induce photoacoustic signals has been an alternative method to achieve multi-wavelength PAI. In this paper, based on the nonlinear photoacoustic effect, we developed a continuous-wave multi-wavelength laser source to cut down the cost of the conventional multi-wavelength PAI system. The principle will be introduced firstly, followed by qualitative and quantitative experiments.

Y-Net: Hybrid deep learning image reconstruction for photoacoustic tomography in vivo.

Conventional reconstruction algorithms (e.g., delay-and-sum) used in photoacoustic imaging (PAI) provide a fast solution while many artifacts remain, especially for limited-view with ill-posed problem. In this paper, we propose a new convolutional neural network (CNN) framework Y-Net: a CNN architecture to reconstruct the initial PA pressure distribution by optimizing both raw data and beamformed images once. The network combines two encoders with one decoder path, which optimally utilizes more information from raw data and beamformed image. We compared our result with some ablation studies, and the results of the test set show better performance compared with conventional reconstruction algorithms and other deep learning method (U-Net). Both in-vitro and in-vivo experiments are used to validated our method, which still performs better than other existing methods. The proposed Y-Net architecture also has high potential in medical image reconstruction for other imaging modalities beyond PAI.

Enabling both time-domain and frequency-domain photoacoustic imaging by a fingertip laser diode system.

Photoacoustic imaging has attracted increasing research interest in recent years, due to its unique merit of combining light and sound. Enabling deep tissue imaging with high ultrasound spatial resolution and optical absorption contrast, photoacoustic imaging has been applied in various application scenarios including anatomical, functional and molecular imaging. However, the bulky and expensive laser source is one of the key bottlenecks that needs to be addressed for further compact system development. A photoacoustic imaging system based on a low-cost laser diode (LD) is one of the promising solutions. In this paper, we report a custom-made fingertip laser diode system enabling both pulsed and continuous modulation modes with shortest pulse-width of 40 ns, largest driving current of 13 A, and highest modulation frequency of 3 MHz, which is suitable for both time and frequency domain photoacoustic imaging. To the best of our knowledge, this may be the most compact laser source reported for photoacoustic imaging enabling both two modulation modes. Owing to its super-compact size, the proposed LD system could pave the pathway to a low-cost photoacoustic sensing and imaging device, even wearable photoacoustic biomedical sensors.

Hybrid Neural Network for Photoacoustic Imaging Reconstruction.

Photoacoustic imaging (PAI) is an emerging noninvasive imaging modality combining the advantages of ultrasound imaging and optical imaging. Image reconstruction is an essential topic in photoacoustic imaging, which is unfortunately an ill-posed problem due to the complex and unknown optical/acoustic parameters in tissue. Conventional algorithms used in photoacoustic imaging (e.g., delay-and-sum) provide a fast solution while many artifacts remain. Convolutional neural network (CNN) has shown state-of-the-art results in computer vision, and more and more work based on CNN has been studied in medical image processing recently. In this paper, we propose Y-Net: a CNN architecture to reconstruct the PA image by integrating both raw data and beamformed images as input. The network connected two encoders with one decoder path, which optimally utilizes more information from raw data and beamformed image. The results of the simulation showed a good performance compared with conventional deep-learning based algorithms and other model-based methods. The proposed Y-Net architecture has significant potential in medical image reconstruction beyond PAI.

Accelerated Photoacoustic Tomography Reconstruction via Recurrent Inference Machines.

Accelerated photoacoustic tomography (PAT) reconstruction is important for real-time photoacoustic imaging (PAI) applications. PAT requires a reconstruction algorithm to reconstruct the detected photoacoustic signal in order to obtain the detected image of the tissue, which is usually an inverse problem. Different from the typical method for solving the inverse problems that defines a model and chooses an inference procedure, we propose to use the Recurrent Inference Machines (RIM) as a framework for PAT reconstruction. Our model performs an accelerated iterative reconstruction, and directly learns to solve the inverse problem in PAT using the information from a forward model that is based on k-space methods. As shown in experiments, our method achieves faster high-resolution PAT reconstruction, and outperforms another method based on deep neural network in some respects.

Reconstruct the Photoacoustic Image Based On Deep Learning with Multi-frequency Ring-shape Transducer Array.

Photoacoustic tomography (PAT) combines the superiorities of both optical imaging and ultrasound imaging, which provides rich optical absorption contrast with 3D spatial information by applying reconstruction algorithms. Classical reconstruction algorithms, e.g. delay-and-sum, have been widely used in photoacoustic imaging. Recently, the deep neural networks have showed the potential to be used to reconstruct the PA images from raw photoacoustic data. In this paper, a framework of the neural network is proposed to approach the PA imaging reconstruction using multi-frequency ultrasound sensor data. Specifically, we trained an end-to-end network to compare the performance when the transducers surround the region of interest with three different center frequencies, which receive PA signals containing different frequency spectrum information from the target. In particular, we trained and tested the network using the factitious segmented vessels' PA images from fundus oculi CT imaging after converting to PA data. From the results of the numerical simulations, the proposed frameworks have shown much better performance compared with conventional reconstruction algorithms. Moreover, the time consumption of the proposed reconstruction method outperforms other conventional reconstruction algorithms, which enables its potential to apply in real-time imaging.

Optical spectroscopic ultrasound displacement imaging.

Photoacoustic imaging has been intensively studied in recent years, and many of the achievements have already been applied in important biomedical and clinical applications, e.g. spectroscopic photoacoustic imaging to extract functional and molecular information. However, spectroscopic photoacoustic imaging requires expensive and bulky tunable laser source, which severely hinder its further development towards portable device. In this paper, we propose a novel imaging method, named optical spectroscopic ultrasound displacement (OSUD) imaging, which enables optical spectroscopic imaging in deep scattering tissue using multiple low-cost continuous-wave laser sources and ultrasound imaging equipment. The principle of the OSUD imaging method will be introduced, and followed by preliminary experimental results. The OSUD imaging may provide another pathway to provide spectroscopic optical absorption contrast in deep scattering tissue beyond commonly used photoacoustic imaging.

Wavelet de-noising method with adaptive threshold selection for photoacoustic tomography.

Photoacoustic (PA) tomography enables imaging of optical absorption property in deep scattering tissue by listening to the PA wave. However, it is an open challenge that the conversion efficiency from light to sound based on PA effect is extremely low. The consequence is the poor signal-to-noise ratio (SNR) of PA signal especially in scenarios of low laser power and deep penetration. The conventional way to improve PA signal's SNR is data averaging, which however severely limits the imaging speed. In this paper, we propose a new adaptive wavelet threshold de-noising (aWTD) algorithm, and apply it in photoacoustic tomography to increase the PA signal's SNR without sacrificing the signal fidelity and imaging speed. PA image quality in terms of contrast is also significantly improved. The proposed method provides the potential to develop real-time low-cost PA tomography system with low-power laser source.

Hybrid multi-wavelength photoacoustic imaging.

Multi-wavelength photoacoustic (PA) imaging has been studied extensively to explore the spectroscopic absorption contrast of biological tissues. To generate strong PA signals, a high-power wavelength-tunable pulsed laser source has to be employed, which is bulky and quite expensive. In this paper, we propose a hybrid multi-wavelength PA imaging (hPAI) method based on combination of single-wavelength pulsed and multi-wavelength continuous-wave (CW) laser sources. By carefully controlling laser illumination sequence (pulse-CW-pulse), and extracting the PA signals' difference before and after heating of CW lasers, the optical absorption property of multi-wavelength CW lasers could be obtained. Compared with conventional PA imaging, the proposed hPAI shows much lower system cost due to the usage of single-wavelength pulsed laser and cheap CW lasers. Theoretical analysis and analytical model are presented in this paper, followed by proof-of-concept experimental results.

Hybrid multi-wavelength nonlinear photoacoustic sensing and imaging.

Multi-wavelength photoacoustic (PA) imaging has been studied extensively to explore the spectroscopic absorption contrast of biological tissues. To generate strong PA signals, a high-power wavelength tunable pulsed laser source has to be employed, which is bulky and quite expensive. In this Letter, we propose a hybrid multi-wavelength PA imaging (hPAI) method based on the combination of a single-wavelength pulsed laser source and multi-wavelength continuous-wave (CW) laser sources. By carefully controlling the laser illumination sequence (pulse-CW-pulse) and extracting the PA signal difference before and after the heating of CW lasers, the optical absorption property of multi-wavelength light illumination could be obtained. Compared with conventional PA imaging, the proposed hPAI shows a much lower system cost due to the usage of single-wavelength pulsed lasers and multiple inexpensive CW lasers. As the preliminary results show in this Letter, hPAI imaging has the potential to provide another pathway for high spectroscopic optical absorption contrast in PA imaging.