Research on a Cross-Domain Few-Shot Adaptive Classification Algorithm Based on Knowledge Distillation Technology.

With the development of deep learning and sensors and sensor collection methods, computer vision inspection technology has developed rapidly. The deep-learning-based classification algorithm requires the acquisition of a model with superior generalization capabilities through the utilization of a substantial quantity of training samples. However, due to issues such as privacy, annotation costs, and sensor-captured images, how to make full use of limited samples has become a major challenge for practical training and deployment. Furthermore, when simulating models and transferring them to actual image scenarios, discrepancies often arise between the common training sets and the target domain (domain offset). Currently, meta-learning offers a promising solution for few-shot learning problems. However, the quantity of supporting set data on the target domain remains limited, leading to limited cross-domain learning effectiveness. To address this challenge, we have developed a self-distillation and mixing (SDM) method utilizing a Teacher-Student framework. This method effectively transfers knowledge from the source domain to the target domain by applying self-distillation techniques and mixed data augmentation, learning better image representations from relatively abundant datasets, and achieving fine-tuning in the target domain. In comparison with nine classical models, the experimental results demonstrate that the SDM method excels in terms of training time and accuracy. Furthermore, SDM effectively transfers knowledge from the source domain to the target domain, even with a limited number of target domain samples.

Enhanced Photoacoustic Visualisation of Clinical Needles by Combining Interstitial and Extracorporeal Illumination of Elastomeric Nanocomposite Coatings.

Ultrasound (US) image guidance is widely used for minimally invasive procedures, but the invasive medical devices (such as metallic needles), especially their tips, can be poorly visualised in US images, leading to significant complications. Photoacoustic (PA) imaging is promising for visualising invasive devices and peripheral tissue targets. Light-emitting diodes (LEDs) acting as PA excitation sources facilitate the clinical translation of PA imaging, but the image quality is degraded due to the low pulse energy leading to insufficient contrast with needles at deep locations. In this paper, photoacoustic visualisation of clinical needles was enhanced by elastomeric nanocomposite coatings with superficial and interstitial illumination. Candle soot nanoparticle-polydimethylsiloxane (CSNP-PDMS) composites with high optical absorption and large thermal expansion coefficients were applied onto the needle exterior and the end-face of an optical fibre placed in the needle lumen. The excitation light was delivered at the surface by LED arrays and through the embedded optical fibre by a pulsed diode laser to improve the visibility of the needle tip. The performance was validated using an ex-vivo tissue model. An LED-based PA/US imaging system was used for imaging the needle out-of-plane and in-plane insertions over approach angles of 20 deg to 55 deg. The CSNP-PDMS composite conferred substantial visual enhancements on both the needle shaft and the tip, with an average of 1.7- and 1.6-fold improvements in signal-to-noise ratios (SNRs), respectively. With the extended light field involving extracorporeal and interstitial illumination and the highly absorbing coatings, enhanced visualisation of the needle shaft and needle tip was achieved with PA imaging, which could be helpful in current US-guided minimally invasive surgeries.

Intraoperative Needle Tip Tracking with an Integrated Fibre-Optic Ultrasound Sensor.

Ultrasound is an essential tool for guidance of many minimally-invasive surgical and interventional procedures, where accurate placement of the interventional device is critical to avoid adverse events. Needle insertion procedures for anaesthesia, fetal medicine and tumour biopsy are commonly ultrasound-guided, and misplacement of the needle may lead to complications such as nerve damage, organ injury or pregnancy loss. Clear visibility of the needle tip is therefore critical, but visibility is often precluded by tissue heterogeneities or specular reflections from the needle shaft. This paper presents the in vitro and ex vivo accuracy of a new, real-time, ultrasound needle tip tracking system for guidance of fetal interventions. A fibre-optic, Fabry-Pérot interferometer hydrophone is integrated into an intraoperative needle and used to localise the needle tip within a handheld ultrasound field. While previous, related work has been based on research ultrasound systems with bespoke transmission sequences, the new system-developed under the ISO 13485 Medical Devices quality standard-operates as an adjunct to a commercial ultrasound imaging system and therefore provides the image quality expected in the clinic, superimposing a cross-hair onto the ultrasound image at the needle tip position. Tracking accuracy was determined by translating the needle tip to 356 known positions in the ultrasound field of view in a tank of water, and by comparison to manual labelling of the the position of the needle in B-mode US images during an insertion into an ex vivo phantom. In water, the mean distance between tracked and true positions was 0.7 ± 0.4 mm with a mean repeatability of 0.3 ± 0.2 mm. In the tissue phantom, the mean distance between tracked and labelled positions was 1.1 ± 0.7 mm. Tracking performance was found to be independent of needle angle. The study demonstrates the performance and clinical compatibility of ultrasound needle tracking, an essential step towards a first-in-human study.

Focusing light through multimode fibres using a digital micromirror device: a comparison study of non-holographic approaches.

Focusing light through a multimode fibre (MMF) has attracted significant research interest, mainly driven by the need for miniature endoscopes in biomedicine. In recent years, digital micromirror devices (DMD) have become increasingly popular as a high-speed alternative to liquid-crystal spatial light modulators for light focusing via wavefront shaping based on binary amplitude modulations. To exploit the potentials and limitations of the state-of-the-art DMD-based wavefront shaping methods, in this study, for the first time, we compared four representative, non-holographic and DMD-based methods that are reported so far in literature with the same experimental and simulation conditions, including a real-valued intensity transmission matrix (RVITM)-based algorithm, a complex-valued transmission matrix (TM)-based algorithm, a conditional probability algorithm and a genetic algorithm. We investigated the maximum achievable peak-to-background ratio (PBR) in comparison to theoretical expectations, and further improved the performance of the RVITM-based method. With both numerical simulations and experiments, we found that the genetic algorithm offered the highest PBR but suffered from the lowest focusing speed, while the RVITM-based algorithm provided a comparable PBR to that of the genetic algorithm, and the highest focusing speed.

High-speed photoacoustic-guided wavefront shaping for focusing light in scattering media.

Wavefront shaping is becoming increasingly attractive as it promises to enable various biomedical applications by breaking through the optical diffusion limit that prevents light focusing at depths larger than ∼1mm in biological tissue. However, despite recent advancements in wavefront shaping technology, such as those exploiting non-invasive photoacoustic-guidance, in vivo demonstrations remain challenging mainly due to rapid tissue speckle decorrelation. In this work, we report a high-speed photoacoustic-guided wavefront shaping method with a relatively simple experimental setup, based on the characterization of a scattering medium with a real-valued intensity transmission matrix. We demonstrated light focusing through an optical diffuser by optimizing 4096 binary amplitude modulation modes of a digital micromirror device within ∼300ms, leading to a system runtime of 75 µs per input mode, which is 3 orders of magnitude smaller than the smallest runtime reported in literature so far using photoacoustic-guided wavefront shaping. Thus, our method is a solid step forward toward in vivo applications of wavefront shaping.

Biomedical Photoacoustic Imaging and Sensing Using Affordable Resources.

The photoacoustic (PA) effect, also called the optoacoustic effect, was discovered in the 1880s by Alexander Graham Bell and has been utilized for biomedical imaging and sensing applications since the early 1990s [...].

Seeing through multimode fibers with real-valued intensity transmission matrices.

Image transmission through multimode optical fibers has been an area of immense interests driven by the demand for miniature endoscopes in biomedicine and higher speed and capacity in telecommunications. Conventionally, a complex-valued transmission matrix is obtained experimentally to link the input and output light fields of a multimode fiber for image retrieval, which complicates the experimental setup and increases the computational complexity. Here, we report a simple and high-speed method for image retrieval based on our demonstration of a pseudo-linearity between the input and output light intensity distributions of multimode fibers. We studied the impact of several key parameters to image retrieval, including image pixel count, fiber core diameter and numerical aperture. We further demonstrated with experiments and numerical simulations that a wide variety of input binary and gray scale images could be faithfully retrieved from the corresponding output speckle patterns. Thus, it promises to be useful for highly miniaturized endoscopy in biomedicine and spatial-mode-division multiplexing in telecommunications.

Portable and Affordable Light Source-Based Photoacoustic Tomography.

Photoacoustic imaging is a hybrid imaging modality that offers the advantages of optical (spectroscopic contrast) and ultrasound imaging (scalable spatial resolution and imaging depth). This promising modality has shown excellent potential in a wide range of preclinical and clinical imaging and sensing applications. Even though photoacoustic imaging technology has matured in research settings, its clinical translation is not happening at the expected pace. One of the main reasons for this is the requirement of bulky and expensive pulsed lasers for excitation. To accelerate the clinical translation of photoacoustic imaging and explore its potential in resource-limited settings, it is of paramount importance to develop portable and affordable light sources that can be used as the excitation light source. In this review, we focus on the following aspects: (1) the basic theory of photoacoustic imaging; (2) inexpensive light sources and different implementations; and (3) important preclinical and clinical applications, demonstrated using affordable light source-based photoacoustics. The main focus will be on laser diodes and light-emitting diodes as they have demonstrated promise in photoacoustic tomography-the key technological developments in these areas will be thoroughly reviewed. We believe that this review will be a useful opus for both the beginners and experts in the field of biomedical photoacoustic imaging.

Handheld Real-Time LED-Based Photoacoustic and Ultrasound Imaging System for Accurate Visualization of Clinical Metal Needles and Superficial Vasculature to Guide Minimally Invasive Procedures.

Ultrasound imaging is widely used to guide minimally invasive procedures, but the visualization of the invasive medical device and the procedure’s target is often challenging. Photoacoustic imaging has shown great promise for guiding minimally invasive procedures, but clinical translation of this technology has often been limited by bulky and expensive excitation sources. In this work, we demonstrate the feasibility of guiding minimally invasive procedures using a dual-mode photoacoustic and ultrasound imaging system with excitation from compact arrays of light-emitting diodes (LEDs) at 850 nm. Three validation experiments were performed. First, clinical metal needles inserted into biological tissue were imaged. Second, the imaging depth of the system was characterized using a blood-vessel-mimicking phantom. Third, the superficial vasculature in human volunteers was imaged. It was found that photoacoustic imaging enabled needle visualization with signal-to-noise ratios that were 1.2 to 2.2 times higher than those obtained with ultrasound imaging, over insertion angles of 26 to 51 degrees. With the blood vessel mimicking phantom, the maximum imaging depth was 38 mm. The superficial vasculature of a human middle finger and a human wrist were clearly visualized in real-time. We conclude that the LED-based system is promising for guiding minimally invasive procedures with peripheral tissue targets.

Enhancement of instrumented ultrasonic tracking images using deep learning.

PURPOSE: Instrumented ultrasonic tracking provides needle localisation during ultrasound-guided minimally invasive percutaneous procedures. Here, a post-processing framework based on a convolutional neural network (CNN) is proposed to improve the spatial resolution of ultrasonic tracking images. METHODS: The custom ultrasonic tracking system comprised a needle with an integrated fibre-optic ultrasound (US) transmitter and a clinical US probe for receiving those transmissions and for acquiring B-mode US images. For post-processing of tracking images reconstructed from the received fibre-optic US transmissions, a recently-developed framework based on ResNet architecture, trained with a purely synthetic dataset, was employed. A preliminary evaluation of this framework was performed with data acquired from needle insertions in the heart of a fetal sheep in vivo. The axial and lateral spatial resolution of the tracking images were used as performance metrics of the trained network. RESULTS: Application of the CNN yielded improvements in the spatial resolution of the tracking images. In three needle insertions, in which the tip depth ranged from 23.9 to 38.4 mm, the lateral resolution improved from 2.11 to 1.58 mm, and the axial resolution improved from 1.29 to 0.46 mm. CONCLUSION: The results provide strong indications of the potential of CNNs to improve the spatial resolution of ultrasonic tracking images and thereby to increase the accuracy of needle tip localisation. These improvements could have broad applicability and impact across multiple clinical fields, which could lead to improvements in procedural efficiency and reductions in risk of complications.

Spatiotemporal singular value decomposition for denoising in photoacoustic imaging with a low-energy excitation light source.

Photoacoustic (PA) imaging is an emerging hybrid imaging modality that combines rich optical spectroscopic contrast and high ultrasonic resolution, and thus holds tremendous promise for a wide range of pre-clinical and clinical applications. Compact and affordable light sources such as light-emitting diodes (LEDs) and laser diodes (LDs) are promising alternatives to bulky and expensive solid-state laser systems that are commonly used as PA light sources. These could accelerate the clinical translation of PA technology. However, PA signals generated with these light sources are readily degraded by noise due to the low optical fluence, leading to decreased signal-to-noise ratio (SNR) in PA images. In this work, a spatiotemporal singular value decomposition (SVD) based PA denoising method was investigated for these light sources that usually have low fluence and high repetition rates. The proposed method leverages both spatial and temporal correlations between radiofrequency (RF) data frames. Validation was performed on simulations and in vivo PA data acquired from human fingers (2D) and forearm (3D) using a LED-based system. Spatiotemporal SVD greatly enhanced the PA signals of blood vessels corrupted by noise while preserving a high temporal resolution to slow motions, improving the SNR of in vivo PA images by 90.3%, 56.0%, and 187.4% compared to single frame-based wavelet denoising, averaging across 200 frames, and single frame without denoising, respectively. With a fast processing time of SVD (∼50 µs per frame), the proposed method is well suited to PA imaging systems with low-energy excitation light sources for real-time in vivo applications.

Video-rate dual-modal photoacoustic and fluorescence imaging through a multimode fibre towards forward-viewing endomicroscopy.

Multimode fibres (MMFs) are becoming increasingly attractive in optical endoscopy as they promise to enable unparallelled miniaturisation, spatial resolution and cost. However, high-speed imaging with wavefront shaping has been challenging. Here, we report the development of a video-rate dual-modal photoacoustic (PA) and fluorescence microscopy probe with a high-speed digital micromirror device (DMD) towards forward-viewing endomicroscopy. Optimal DMD patterns were obtained using a real-valued intensity transmission matrix algorithm to raster-scan a 1.5 µ m-diameter focused beam at the distal fibre tip for imaging. The PA imaging speed and spatial resolution were varied from ∼ 2 to 57 frames per second and from 1.7 to 3 µ m, respectively. Further, high-fidelity PA images of carbon fibres and mouse red blood cells were acquired at unprecedented speed. The capability of dual-modal imaging was demonstrated with phantoms. We anticipate that with further miniaturisation of the ultrasound detector, this probe could be integrated into medical needles to guide minimally invasive procedures.

Improving needle visibility in LED-based photoacoustic imaging using deep learning with semi-synthetic datasets.

Photoacoustic imaging has shown great potential for guiding minimally invasive procedures by accurate identification of critical tissue targets and invasive medical devices (such as metallic needles). The use of light emitting diodes (LEDs) as the excitation light sources accelerates its clinical translation owing to its high affordability and portability. However, needle visibility in LED-based photoacoustic imaging is compromised primarily due to its low optical fluence. In this work, we propose a deep learning framework based on U-Net to improve the visibility of clinical metallic needles with a LED-based photoacoustic and ultrasound imaging system. To address the complexity of capturing ground truth for real data and the poor realism of purely simulated data, this framework included the generation of semi-synthetic training datasets combining both simulated data to represent features from the needles and in vivo measurements for tissue background. Evaluation of the trained neural network was performed with needle insertions into blood-vessel-mimicking phantoms, pork joint tissue ex vivo and measurements on human volunteers. This deep learning-based framework substantially improved the needle visibility in photoacoustic imaging in vivo compared to conventional reconstruction by suppressing background noise and image artefacts, achieving 5.8 and 4.5 times improvements in terms of signal-to-noise ratio and the modified Hausdorff distance, respectively. Thus, the proposed framework could be helpful for reducing complications during percutaneous needle insertions by accurate identification of clinical needles in photoacoustic imaging.

Deep Learning for Instrumented Ultrasonic Tracking: From Synthetic Training Data to In Vivo Application.

Instrumented ultrasonic tracking is used to improve needle localization during ultrasound guidance of minimally invasive percutaneous procedures. Here, it is implemented with transmitted ultrasound pulses from a clinical ultrasound imaging probe, which is detected by a fiber-optic hydrophone integrated into a needle. The detected transmissions are then reconstructed to form the tracking image. Two challenges are considered with the current implementation of ultrasonic tracking. First, tracking transmissions are interleaved with the acquisition of B-mode images, and thus, the effective B-mode frame rate is reduced. Second, it is challenging to achieve an accurate localization of the needle tip when the signal-to-noise ratio is low. To address these challenges, we present a framework based on a convolutional neural network (CNN) to maintain spatial resolution with fewer tracking transmissions and enhance signal quality. A major component of the framework included the generation of realistic synthetic training data. The trained network was applied to unseen synthetic data and experimental in vivo tracking data. The performance of needle localization was investigated when reconstruction was performed with fewer (up to eightfold) tracking transmissions. CNN-based processing of conventional reconstructions showed that the axial and lateral spatial resolutions could be improved even with an eightfold reduction in tracking transmissions. The framework presented in this study will significantly improve the performance of ultrasonic tracking, leading to faster image acquisition rates and increased localization accuracy.

Ultrathin, high-speed, all-optical photoacoustic endomicroscopy probe for guiding minimally invasive surgery.

Photoacoustic (PA) endoscopy has shown significant potential for clinical diagnosis and surgical guidance. Multimode fibres (MMFs) are becoming increasingly attractive for the development of miniature endoscopy probes owing to their ultrathin size, low cost and diffraction-limited spatial resolution enabled by wavefront shaping. However, current MMF-based PA endomicroscopy probes are either limited by a bulky ultrasound detector or a low imaging speed that hindered their usability. In this work, we report the development of a highly miniaturised and high-speed PA endomicroscopy probe that is integrated within the cannula of a 20 gauge medical needle. This probe comprises a MMF for delivering the PA excitation light and a single-mode optical fibre with a plano-concave microresonator for ultrasound detection. Wavefront shaping with a digital micromirror device enabled rapid raster-scanning of a focused light spot at the distal end of the MMF for tissue interrogation. High-resolution PA imaging of mouse red blood cells covering an area 100 µm in diameter was achieved with the needle probe at ∼3 frames per second. Mosaicing imaging was performed after fibre characterisation by translating the needle probe to enlarge the field-of-view in real-time. The developed ultrathin PA endomicroscopy probe is promising for guiding minimally invasive surgery by providing functional, molecular and microstructural information of tissue in real-time.

Ultrasonic Needle Tracking with Dynamic Electronic Focusing.

Accurate identification of the needle tip is a key challenge with ultrasound-guided percutaneous interventions in regional anaesthesia, foetal surgery and cardiovascular medicine. In this study, we developed an ultrasonic needle tracking system in which the measured needle tip location was used to set the electronic focus of the external ultrasound imaging probe. In this system, needle tip tracking was enabled with a fibre-optic ultrasound sensor that was integrated into a needle stylet, and the A-lines recorded by the sensor were processed to generate tracking images of the needle tip. The needle tip position was estimated from the tracking images. The dependency of the tracking image on the electronic focal depth of the external ultrasound imaging probe was studied in a water bath and with needle insertions into a clinical training phantom. The variability in the estimated tracked position of the needle tip, with the needle tip at fixed depths in the imaging plane across a depth range from 0.5 to 7.5 cm, was studied. When the electronic focus was fixed, the variability of tracked position was found to increase with distance from that focus. The variability with the fixed focus was found to depend on the the relative distance between the needle tip and focal depth. It was found that with dynamic focusing, the maximum variability of tracked position was below 0.31 mm, as compared with 3.97 mm for a fixed focus.

Driving mechanism of drift-step-recovery diodes.

Parameters governing forward and reverse pumping of a device mechanism have an important influence on the cutoff and output characteristics of a drift-step-recovery diode (DSRD). Therefore, we analyzed the driving circuit for the DSRD that was used to adjust pumping parameter values. With multiple modules in parallel, a driving method exploiting narrow-pulse pumping is proposed in developing a high-repetition-frequency DSRD pulse generator. In experiments and simulations, one key factor affecting the working stability of this pulse generator operating at repetition frequencies in the megahertz range was found that helped in further optimizing the parameter settings of our 2 kV DSRD driving circuit. From this insight, a 2 kV-5 MHz all-solid-state high-repetition-frequency pulse generator based on the DSRD was developed.

Multispectral tissue mapping: developing a concept for the optical evaluation of liver disease.

Purpose: Alterations in the optical absorption behavior of liver tissue secondary to pathological processes can be evaluated by multispectral analysis, which is increasingly being explored as an imaging adjunct for use in liver surgery. Current methods are either invasive or have a limited wavelength spectrum, which restricts utility. This proof of concept study describes the development of a multispectral imaging (MSI) method called multispectral tissue mapping (MTM) that addresses these issues. Approach: The imaging system consists of a tunable excitation light source and a near-infrared camera. Following the development stage, proof of concept experiments are carried out where absorption spectra from colorectal cancer liver metastasis (CRLM), hepatocellular carcinoma (HCC), and liver steatosis specimen are acquired and compared to controls. Absorption spectra are compared to histopathology examination as the current gold standard for tissue assessment. Generalized linear mixed modeling is employed to compare absorption characteristics of individual pixels and to select wavelengths for false color image processing with the aim of visually enhancing cancer tissue. Results: Analysis of individual pixels revealed distinct absorption spectra therefore suggesting that MTM is possible. A prominent absorption peak at 1210 nm was found in lipid-rich animal tissues and steatotic liver specimen. Liver cancer tissue had a heterogeneous appearance on MSI. Subsequent statistical analysis suggests that measuring changes in absorption behavior may be a feasible method to estimate the pixel-based probability of cancer being present. In CRLM, this was observed throughout 1100 to 1700 nm, whereas in HCC it was concentrated around 1140 and 1430 nm. False color image processing visibly enhances contrast between cancer and normal liver tissues. Conclusions: The system's ability to enable no-touch MSI at 1100 to 1700 nm was demonstrated. Preliminary data suggest that MTM warrants further exploration as a potential imaging tool for the detection of liver cancer during surgery.

Photoacoustic imaging of the human placental vasculature.

Minimally invasive fetal interventions require accurate imaging from inside the uterine cavity. Twin-to-twin transfusion syndrome (TTTS), a condition considered in this study, occurs from abnormal vascular anastomoses in the placenta that allow blood to flow unevenly between the fetuses. Currently, TTTS is treated fetoscopically by identifying the anastomosing vessels, and then performing laser photocoagulation. However, white light fetoscopy provides limited visibility of placental vasculature, which can lead to missed anastomoses or incomplete photocoagulation. Photoacoustic (PA) imaging is an alternative imaging method that provides contrast for hemoglobin, and in this study, two PA systems were used to visualize chorionic (fetal) superficial and subsurface vasculature in human placentas. The first system comprised an optical parametric oscillator for PA excitation and a 2D Fabry-Pérot cavity ultrasound sensor; the second, light emitting diode arrays and a 1D clinical linear-array ultrasound imaging probe. Volumetric photoacoustic images were acquired from ex vivo normal term and TTTS-treated placentas. It was shown that superficial and subsurface branching blood vessels could be visualized to depths of approximately 7 mm, and that ablated tissue yielded negative image contrast. This study demonstrated the strong potential of PA imaging to guide minimally invasive fetal therapies.

Minimally invasive photoacoustic imaging: Current status and future perspectives.

Photoacoustic imaging (PAI) is an emerging biomedical imaging modality that is based on optical absorption contrast, capable of revealing distinct spectroscopic signatures of tissue at high spatial resolution and large imaging depths. However, clinical applications of conventional non-invasive PAI systems have been restricted to examinations of tissues at depths less than a few cm due to strong light attenuation. Minimally invasive photoacoustic imaging (miPAI) has greatly extended the landscape of PAI by delivering excitation light within tissue through miniature fibre-optic probes. In the past decade, various miPAI systems have been developed with demonstrated applicability in several clinical fields. In this article, we present an overview of the current status of miPAI and our thoughts on future perspectives.