RESUMO
With the development of deep learning and sensors and sensor collection methods, computer vision inspection technology has developed rapidly. The deep-learning-based classification algorithm requires the acquisition of a model with superior generalization capabilities through the utilization of a substantial quantity of training samples. However, due to issues such as privacy, annotation costs, and sensor-captured images, how to make full use of limited samples has become a major challenge for practical training and deployment. Furthermore, when simulating models and transferring them to actual image scenarios, discrepancies often arise between the common training sets and the target domain (domain offset). Currently, meta-learning offers a promising solution for few-shot learning problems. However, the quantity of supporting set data on the target domain remains limited, leading to limited cross-domain learning effectiveness. To address this challenge, we have developed a self-distillation and mixing (SDM) method utilizing a Teacher-Student framework. This method effectively transfers knowledge from the source domain to the target domain by applying self-distillation techniques and mixed data augmentation, learning better image representations from relatively abundant datasets, and achieving fine-tuning in the target domain. In comparison with nine classical models, the experimental results demonstrate that the SDM method excels in terms of training time and accuracy. Furthermore, SDM effectively transfers knowledge from the source domain to the target domain, even with a limited number of target domain samples.
RESUMO
Ultrasound (US) image guidance is widely used for minimally invasive procedures, but the invasive medical devices (such as metallic needles), especially their tips, can be poorly visualised in US images, leading to significant complications. Photoacoustic (PA) imaging is promising for visualising invasive devices and peripheral tissue targets. Light-emitting diodes (LEDs) acting as PA excitation sources facilitate the clinical translation of PA imaging, but the image quality is degraded due to the low pulse energy leading to insufficient contrast with needles at deep locations. In this paper, photoacoustic visualisation of clinical needles was enhanced by elastomeric nanocomposite coatings with superficial and interstitial illumination. Candle soot nanoparticle-polydimethylsiloxane (CSNP-PDMS) composites with high optical absorption and large thermal expansion coefficients were applied onto the needle exterior and the end-face of an optical fibre placed in the needle lumen. The excitation light was delivered at the surface by LED arrays and through the embedded optical fibre by a pulsed diode laser to improve the visibility of the needle tip. The performance was validated using an ex-vivo tissue model. An LED-based PA/US imaging system was used for imaging the needle out-of-plane and in-plane insertions over approach angles of 20 deg to 55 deg. The CSNP-PDMS composite conferred substantial visual enhancements on both the needle shaft and the tip, with an average of 1.7- and 1.6-fold improvements in signal-to-noise ratios (SNRs), respectively. With the extended light field involving extracorporeal and interstitial illumination and the highly absorbing coatings, enhanced visualisation of the needle shaft and needle tip was achieved with PA imaging, which could be helpful in current US-guided minimally invasive surgeries.
Assuntos
Nanocompostos , Agulhas , Iluminação , Análise Espectral , UltrassonografiaRESUMO
Ultrasound is an essential tool for guidance of many minimally-invasive surgical and interventional procedures, where accurate placement of the interventional device is critical to avoid adverse events. Needle insertion procedures for anaesthesia, fetal medicine and tumour biopsy are commonly ultrasound-guided, and misplacement of the needle may lead to complications such as nerve damage, organ injury or pregnancy loss. Clear visibility of the needle tip is therefore critical, but visibility is often precluded by tissue heterogeneities or specular reflections from the needle shaft. This paper presents the in vitro and ex vivo accuracy of a new, real-time, ultrasound needle tip tracking system for guidance of fetal interventions. A fibre-optic, Fabry-Pérot interferometer hydrophone is integrated into an intraoperative needle and used to localise the needle tip within a handheld ultrasound field. While previous, related work has been based on research ultrasound systems with bespoke transmission sequences, the new system-developed under the ISO 13485 Medical Devices quality standard-operates as an adjunct to a commercial ultrasound imaging system and therefore provides the image quality expected in the clinic, superimposing a cross-hair onto the ultrasound image at the needle tip position. Tracking accuracy was determined by translating the needle tip to 356 known positions in the ultrasound field of view in a tank of water, and by comparison to manual labelling of the the position of the needle in B-mode US images during an insertion into an ex vivo phantom. In water, the mean distance between tracked and true positions was 0.7 ± 0.4 mm with a mean repeatability of 0.3 ± 0.2 mm. In the tissue phantom, the mean distance between tracked and labelled positions was 1.1 ± 0.7 mm. Tracking performance was found to be independent of needle angle. The study demonstrates the performance and clinical compatibility of ultrasound needle tracking, an essential step towards a first-in-human study.
Assuntos
Tecnologia de Fibra Óptica , Agulhas , Gravidez , Feminino , Humanos , Ultrassonografia , Imagens de Fantasmas , Água , Ultrassonografia de Intervenção/métodosRESUMO
Focusing light through a multimode fibre (MMF) has attracted significant research interest, mainly driven by the need for miniature endoscopes in biomedicine. In recent years, digital micromirror devices (DMD) have become increasingly popular as a high-speed alternative to liquid-crystal spatial light modulators for light focusing via wavefront shaping based on binary amplitude modulations. To exploit the potentials and limitations of the state-of-the-art DMD-based wavefront shaping methods, in this study, for the first time, we compared four representative, non-holographic and DMD-based methods that are reported so far in literature with the same experimental and simulation conditions, including a real-valued intensity transmission matrix (RVITM)-based algorithm, a complex-valued transmission matrix (TM)-based algorithm, a conditional probability algorithm and a genetic algorithm. We investigated the maximum achievable peak-to-background ratio (PBR) in comparison to theoretical expectations, and further improved the performance of the RVITM-based method. With both numerical simulations and experiments, we found that the genetic algorithm offered the highest PBR but suffered from the lowest focusing speed, while the RVITM-based algorithm provided a comparable PBR to that of the genetic algorithm, and the highest focusing speed.
RESUMO
Wavefront shaping is becoming increasingly attractive as it promises to enable various biomedical applications by breaking through the optical diffusion limit that prevents light focusing at depths larger than â¼1mm in biological tissue. However, despite recent advancements in wavefront shaping technology, such as those exploiting non-invasive photoacoustic-guidance, in vivo demonstrations remain challenging mainly due to rapid tissue speckle decorrelation. In this work, we report a high-speed photoacoustic-guided wavefront shaping method with a relatively simple experimental setup, based on the characterization of a scattering medium with a real-valued intensity transmission matrix. We demonstrated light focusing through an optical diffuser by optimizing 4096 binary amplitude modulation modes of a digital micromirror device within â¼300ms, leading to a system runtime of 75 µs per input mode, which is 3 orders of magnitude smaller than the smallest runtime reported in literature so far using photoacoustic-guided wavefront shaping. Thus, our method is a solid step forward toward in vivo applications of wavefront shaping.
Assuntos
Luz , Técnicas Fotoacústicas/instrumentação , Espalhamento de RadiaçãoRESUMO
The photoacoustic (PA) effect, also called the optoacoustic effect, was discovered in the 1880s by Alexander Graham Bell and has been utilized for biomedical imaging and sensing applications since the early 1990s [...].
Assuntos
Técnicas Fotoacústicas , Diagnóstico por Imagem , Análise EspectralRESUMO
Image transmission through multimode optical fibers has been an area of immense interests driven by the demand for miniature endoscopes in biomedicine and higher speed and capacity in telecommunications. Conventionally, a complex-valued transmission matrix is obtained experimentally to link the input and output light fields of a multimode fiber for image retrieval, which complicates the experimental setup and increases the computational complexity. Here, we report a simple and high-speed method for image retrieval based on our demonstration of a pseudo-linearity between the input and output light intensity distributions of multimode fibers. We studied the impact of several key parameters to image retrieval, including image pixel count, fiber core diameter and numerical aperture. We further demonstrated with experiments and numerical simulations that a wide variety of input binary and gray scale images could be faithfully retrieved from the corresponding output speckle patterns. Thus, it promises to be useful for highly miniaturized endoscopy in biomedicine and spatial-mode-division multiplexing in telecommunications.
RESUMO
Photoacoustic imaging is a hybrid imaging modality that offers the advantages of optical (spectroscopic contrast) and ultrasound imaging (scalable spatial resolution and imaging depth). This promising modality has shown excellent potential in a wide range of preclinical and clinical imaging and sensing applications. Even though photoacoustic imaging technology has matured in research settings, its clinical translation is not happening at the expected pace. One of the main reasons for this is the requirement of bulky and expensive pulsed lasers for excitation. To accelerate the clinical translation of photoacoustic imaging and explore its potential in resource-limited settings, it is of paramount importance to develop portable and affordable light sources that can be used as the excitation light source. In this review, we focus on the following aspects: (1) the basic theory of photoacoustic imaging; (2) inexpensive light sources and different implementations; and (3) important preclinical and clinical applications, demonstrated using affordable light source-based photoacoustics. The main focus will be on laser diodes and light-emitting diodes as they have demonstrated promise in photoacoustic tomography-the key technological developments in these areas will be thoroughly reviewed. We believe that this review will be a useful opus for both the beginners and experts in the field of biomedical photoacoustic imaging.
RESUMO
Ultrasound imaging is widely used to guide minimally invasive procedures, but the visualization of the invasive medical device and the procedure’s target is often challenging. Photoacoustic imaging has shown great promise for guiding minimally invasive procedures, but clinical translation of this technology has often been limited by bulky and expensive excitation sources. In this work, we demonstrate the feasibility of guiding minimally invasive procedures using a dual-mode photoacoustic and ultrasound imaging system with excitation from compact arrays of light-emitting diodes (LEDs) at 850 nm. Three validation experiments were performed. First, clinical metal needles inserted into biological tissue were imaged. Second, the imaging depth of the system was characterized using a blood-vessel-mimicking phantom. Third, the superficial vasculature in human volunteers was imaged. It was found that photoacoustic imaging enabled needle visualization with signal-to-noise ratios that were 1.2 to 2.2 times higher than those obtained with ultrasound imaging, over insertion angles of 26 to 51 degrees. With the blood vessel mimicking phantom, the maximum imaging depth was 38 mm. The superficial vasculature of a human middle finger and a human wrist were clearly visualized in real-time. We conclude that the LED-based system is promising for guiding minimally invasive procedures with peripheral tissue targets.
Assuntos
Vasos Sanguíneos/diagnóstico por imagem , Metais , Agulhas , Técnicas Fotoacústicas/instrumentação , Ultrassonografia/instrumentação , Humanos , Imagens de Fantasmas , Análise EspectralRESUMO
Soft robots have morphological characteristics that make them preferred candidates, over their traditionally rigid counterparts, for executing physical interaction tasks with the environment. Therefore, equipping them with force sensing is essential for ensuring safety, enhancing their controllability, and adding autonomy. At the same time, it is necessary to preserve their inherent flexibility when integrating sensory units. Soft-fluidic actuators (SFAs) with hydraulic actuation address some of the challenges posed by the compressibility of pneumatic actuation while maintaining system compliance. This research further investigates the feasibility of utilizing the incompressible actuation fluid as the means of actuation and of multiaxial sensing. We have developed a hyperelastic model for the actuation pressure, acting as a baseline pressure. Any disparities from the baseline have been mapped to external forces, using the principle of pressure-based fluidic soft sensor. Computed tomography imaging has been used to examine inner deformation and validate the analytically derived actuation-pressure model. The induced stresses within the SFA are examined using COMSOL simulations, contributing to the development of a calibration algorithm, which accounts for geometric and cross-sectional nonlinearities and maps pressure variations with tip forces. Two force types (concentrated and distributed) acting on our SFA under different configurations are examined, using two experimental setups described as "Point Load" and "Distributed Force." The force sensing algorithm achieves high accuracy with a maximum absolute error of 0.32N for forces with a magnitude of up to 6N.
RESUMO
PURPOSE: Instrumented ultrasonic tracking provides needle localisation during ultrasound-guided minimally invasive percutaneous procedures. Here, a post-processing framework based on a convolutional neural network (CNN) is proposed to improve the spatial resolution of ultrasonic tracking images. METHODS: The custom ultrasonic tracking system comprised a needle with an integrated fibre-optic ultrasound (US) transmitter and a clinical US probe for receiving those transmissions and for acquiring B-mode US images. For post-processing of tracking images reconstructed from the received fibre-optic US transmissions, a recently-developed framework based on ResNet architecture, trained with a purely synthetic dataset, was employed. A preliminary evaluation of this framework was performed with data acquired from needle insertions in the heart of a fetal sheep in vivo. The axial and lateral spatial resolution of the tracking images were used as performance metrics of the trained network. RESULTS: Application of the CNN yielded improvements in the spatial resolution of the tracking images. In three needle insertions, in which the tip depth ranged from 23.9 to 38.4 mm, the lateral resolution improved from 2.11 to 1.58 mm, and the axial resolution improved from 1.29 to 0.46 mm. CONCLUSION: The results provide strong indications of the potential of CNNs to improve the spatial resolution of ultrasonic tracking images and thereby to increase the accuracy of needle tip localisation. These improvements could have broad applicability and impact across multiple clinical fields, which could lead to improvements in procedural efficiency and reductions in risk of complications.
Assuntos
Aprendizado Profundo , Ovinos , Animais , Ultrassom , Ultrassonografia/métodos , Agulhas , Redes Neurais de ComputaçãoRESUMO
Photoacoustic (PA) imaging is an emerging hybrid imaging modality that combines rich optical spectroscopic contrast and high ultrasonic resolution, and thus holds tremendous promise for a wide range of pre-clinical and clinical applications. Compact and affordable light sources such as light-emitting diodes (LEDs) and laser diodes (LDs) are promising alternatives to bulky and expensive solid-state laser systems that are commonly used as PA light sources. These could accelerate the clinical translation of PA technology. However, PA signals generated with these light sources are readily degraded by noise due to the low optical fluence, leading to decreased signal-to-noise ratio (SNR) in PA images. In this work, a spatiotemporal singular value decomposition (SVD) based PA denoising method was investigated for these light sources that usually have low fluence and high repetition rates. The proposed method leverages both spatial and temporal correlations between radiofrequency (RF) data frames. Validation was performed on simulations and in vivo PA data acquired from human fingers (2D) and forearm (3D) using a LED-based system. Spatiotemporal SVD greatly enhanced the PA signals of blood vessels corrupted by noise while preserving a high temporal resolution to slow motions, improving the SNR of in vivo PA images by 90.3%, 56.0%, and 187.4% compared to single frame-based wavelet denoising, averaging across 200 frames, and single frame without denoising, respectively. With a fast processing time of SVD (â¼50 µs per frame), the proposed method is well suited to PA imaging systems with low-energy excitation light sources for real-time in vivo applications.
RESUMO
Multimode fibres (MMFs) are becoming increasingly attractive in optical endoscopy as they promise to enable unparallelled miniaturisation, spatial resolution and cost. However, high-speed imaging with wavefront shaping has been challenging. Here, we report the development of a video-rate dual-modal photoacoustic (PA) and fluorescence microscopy probe with a high-speed digital micromirror device (DMD) towards forward-viewing endomicroscopy. Optimal DMD patterns were obtained using a real-valued intensity transmission matrix algorithm to raster-scan a 1.5 µ m-diameter focused beam at the distal fibre tip for imaging. The PA imaging speed and spatial resolution were varied from â¼ 2 to 57 frames per second and from 1.7 to 3 µ m, respectively. Further, high-fidelity PA images of carbon fibres and mouse red blood cells were acquired at unprecedented speed. The capability of dual-modal imaging was demonstrated with phantoms. We anticipate that with further miniaturisation of the ultrasound detector, this probe could be integrated into medical needles to guide minimally invasive procedures.
RESUMO
Photoacoustic imaging has shown great potential for guiding minimally invasive procedures by accurate identification of critical tissue targets and invasive medical devices (such as metallic needles). The use of light emitting diodes (LEDs) as the excitation light sources accelerates its clinical translation owing to its high affordability and portability. However, needle visibility in LED-based photoacoustic imaging is compromised primarily due to its low optical fluence. In this work, we propose a deep learning framework based on U-Net to improve the visibility of clinical metallic needles with a LED-based photoacoustic and ultrasound imaging system. To address the complexity of capturing ground truth for real data and the poor realism of purely simulated data, this framework included the generation of semi-synthetic training datasets combining both simulated data to represent features from the needles and in vivo measurements for tissue background. Evaluation of the trained neural network was performed with needle insertions into blood-vessel-mimicking phantoms, pork joint tissue ex vivo and measurements on human volunteers. This deep learning-based framework substantially improved the needle visibility in photoacoustic imaging in vivo compared to conventional reconstruction by suppressing background noise and image artefacts, achieving 5.8 and 4.5 times improvements in terms of signal-to-noise ratio and the modified Hausdorff distance, respectively. Thus, the proposed framework could be helpful for reducing complications during percutaneous needle insertions by accurate identification of clinical needles in photoacoustic imaging.
RESUMO
Instrumented ultrasonic tracking is used to improve needle localization during ultrasound guidance of minimally invasive percutaneous procedures. Here, it is implemented with transmitted ultrasound pulses from a clinical ultrasound imaging probe, which is detected by a fiber-optic hydrophone integrated into a needle. The detected transmissions are then reconstructed to form the tracking image. Two challenges are considered with the current implementation of ultrasonic tracking. First, tracking transmissions are interleaved with the acquisition of B-mode images, and thus, the effective B-mode frame rate is reduced. Second, it is challenging to achieve an accurate localization of the needle tip when the signal-to-noise ratio is low. To address these challenges, we present a framework based on a convolutional neural network (CNN) to maintain spatial resolution with fewer tracking transmissions and enhance signal quality. A major component of the framework included the generation of realistic synthetic training data. The trained network was applied to unseen synthetic data and experimental in vivo tracking data. The performance of needle localization was investigated when reconstruction was performed with fewer (up to eightfold) tracking transmissions. CNN-based processing of conventional reconstructions showed that the axial and lateral spatial resolutions could be improved even with an eightfold reduction in tracking transmissions. The framework presented in this study will significantly improve the performance of ultrasonic tracking, leading to faster image acquisition rates and increased localization accuracy.
Assuntos
Aprendizado Profundo , Processamento de Imagem Assistida por Computador/métodos , Agulhas , Redes Neurais de Computação , Ultrassom , Ultrassonografia/métodosRESUMO
Photoacoustic (PA) endoscopy has shown significant potential for clinical diagnosis and surgical guidance. Multimode fibres (MMFs) are becoming increasingly attractive for the development of miniature endoscopy probes owing to their ultrathin size, low cost and diffraction-limited spatial resolution enabled by wavefront shaping. However, current MMF-based PA endomicroscopy probes are either limited by a bulky ultrasound detector or a low imaging speed that hindered their usability. In this work, we report the development of a highly miniaturised and high-speed PA endomicroscopy probe that is integrated within the cannula of a 20 gauge medical needle. This probe comprises a MMF for delivering the PA excitation light and a single-mode optical fibre with a plano-concave microresonator for ultrasound detection. Wavefront shaping with a digital micromirror device enabled rapid raster-scanning of a focused light spot at the distal end of the MMF for tissue interrogation. High-resolution PA imaging of mouse red blood cells covering an area 100 µm in diameter was achieved with the needle probe at â¼3 frames per second. Mosaicing imaging was performed after fibre characterisation by translating the needle probe to enlarge the field-of-view in real-time. The developed ultrathin PA endomicroscopy probe is promising for guiding minimally invasive surgery by providing functional, molecular and microstructural information of tissue in real-time.
RESUMO
Accurate identification of the needle tip is a key challenge with ultrasound-guided percutaneous interventions in regional anaesthesia, foetal surgery and cardiovascular medicine. In this study, we developed an ultrasonic needle tracking system in which the measured needle tip location was used to set the electronic focus of the external ultrasound imaging probe. In this system, needle tip tracking was enabled with a fibre-optic ultrasound sensor that was integrated into a needle stylet, and the A-lines recorded by the sensor were processed to generate tracking images of the needle tip. The needle tip position was estimated from the tracking images. The dependency of the tracking image on the electronic focal depth of the external ultrasound imaging probe was studied in a water bath and with needle insertions into a clinical training phantom. The variability in the estimated tracked position of the needle tip, with the needle tip at fixed depths in the imaging plane across a depth range from 0.5 to 7.5 cm, was studied. When the electronic focus was fixed, the variability of tracked position was found to increase with distance from that focus. The variability with the fixed focus was found to depend on the the relative distance between the needle tip and focal depth. It was found that with dynamic focusing, the maximum variability of tracked position was below 0.31 mm, as compared with 3.97 mm for a fixed focus.
Assuntos
Agulhas , Ultrassom , Eletrônica , Imagens de Fantasmas , Ultrassonografia , Ultrassonografia de Intervenção/métodosRESUMO
Parameters governing forward and reverse pumping of a device mechanism have an important influence on the cutoff and output characteristics of a drift-step-recovery diode (DSRD). Therefore, we analyzed the driving circuit for the DSRD that was used to adjust pumping parameter values. With multiple modules in parallel, a driving method exploiting narrow-pulse pumping is proposed in developing a high-repetition-frequency DSRD pulse generator. In experiments and simulations, one key factor affecting the working stability of this pulse generator operating at repetition frequencies in the megahertz range was found that helped in further optimizing the parameter settings of our 2 kV DSRD driving circuit. From this insight, a 2 kV-5 MHz all-solid-state high-repetition-frequency pulse generator based on the DSRD was developed.
RESUMO
Purpose: Alterations in the optical absorption behavior of liver tissue secondary to pathological processes can be evaluated by multispectral analysis, which is increasingly being explored as an imaging adjunct for use in liver surgery. Current methods are either invasive or have a limited wavelength spectrum, which restricts utility. This proof of concept study describes the development of a multispectral imaging (MSI) method called multispectral tissue mapping (MTM) that addresses these issues. Approach: The imaging system consists of a tunable excitation light source and a near-infrared camera. Following the development stage, proof of concept experiments are carried out where absorption spectra from colorectal cancer liver metastasis (CRLM), hepatocellular carcinoma (HCC), and liver steatosis specimen are acquired and compared to controls. Absorption spectra are compared to histopathology examination as the current gold standard for tissue assessment. Generalized linear mixed modeling is employed to compare absorption characteristics of individual pixels and to select wavelengths for false color image processing with the aim of visually enhancing cancer tissue. Results: Analysis of individual pixels revealed distinct absorption spectra therefore suggesting that MTM is possible. A prominent absorption peak at 1210 nm was found in lipid-rich animal tissues and steatotic liver specimen. Liver cancer tissue had a heterogeneous appearance on MSI. Subsequent statistical analysis suggests that measuring changes in absorption behavior may be a feasible method to estimate the pixel-based probability of cancer being present. In CRLM, this was observed throughout 1100 to 1700 nm, whereas in HCC it was concentrated around 1140 and 1430 nm. False color image processing visibly enhances contrast between cancer and normal liver tissues. Conclusions: The system's ability to enable no-touch MSI at 1100 to 1700 nm was demonstrated. Preliminary data suggest that MTM warrants further exploration as a potential imaging tool for the detection of liver cancer during surgery.
RESUMO
Minimally invasive fetal interventions require accurate imaging from inside the uterine cavity. Twin-to-twin transfusion syndrome (TTTS), a condition considered in this study, occurs from abnormal vascular anastomoses in the placenta that allow blood to flow unevenly between the fetuses. Currently, TTTS is treated fetoscopically by identifying the anastomosing vessels, and then performing laser photocoagulation. However, white light fetoscopy provides limited visibility of placental vasculature, which can lead to missed anastomoses or incomplete photocoagulation. Photoacoustic (PA) imaging is an alternative imaging method that provides contrast for hemoglobin, and in this study, two PA systems were used to visualize chorionic (fetal) superficial and subsurface vasculature in human placentas. The first system comprised an optical parametric oscillator for PA excitation and a 2D Fabry-Pérot cavity ultrasound sensor; the second, light emitting diode arrays and a 1D clinical linear-array ultrasound imaging probe. Volumetric photoacoustic images were acquired from ex vivo normal term and TTTS-treated placentas. It was shown that superficial and subsurface branching blood vessels could be visualized to depths of approximately 7 mm, and that ablated tissue yielded negative image contrast. This study demonstrated the strong potential of PA imaging to guide minimally invasive fetal therapies.