Rib region detection for scanning path planning for fully automated robotic abdominal ultrasonography.

PURPOSE: Scanning path planning is an essential technology for fully automated ultrasound (US) robotics. During biliary scanning, the subcostal boundary is critical body surface landmarks for scanning path planning but are often invisible, depending on the individual. This study developed a method of estimating the rib region for scanning path planning toward fully automated robotic US systems. METHODS: We proposed a method for determining the rib region using RGB-D images and respiratory variation. We hypothesized that detecting the rib region would be possible based on changes in body surface position due to breathing. We generated a depth difference image by finding the difference between the depth image taken at the resting inspiratory position and the depth image taken at the maximum inspiratory position, which clearly shows the rib region. The boundary position of the subcostal was then determined by applying training using the YOLOv5 object detection model to this depth difference image. RESULTS: In the experiments with healthy subjects, the proposed method of rib detection using the depth difference image marked an intersection over union (IoU) of 0.951 and average confidence of 0.77. The average error between the ground truth and predicted positions was 16.5 mm in 3D space. The results were superior to rib detection using only the RGB image. CONCLUSION: The proposed depth difference imaging method, which measures respiratory variation, was able to accurately estimate the rib region without contact and physician intervention. It will be useful for planning the scan path during the biliary imaging.

Functional ultrasound reveals effects of MRI acoustic noise on brain function.

Loud acoustic noise from the scanner during functional magnetic resonance imaging (fMRI) can affect functional connectivity (FC) observed in the resting state, but the exact effect of the MRI acoustic noise on resting state FC is not well understood. Functional ultrasound (fUS) is a neuroimaging method that visualizes brain activity based on relative cerebral blood volume (rCBV), a similar neurovascular coupling response to that measured by fMRI, but without the audible acoustic noise. In this study, we investigated the effects of different acoustic noise levels (silent, 80 dB, and 110 dB) on FC by measuring resting state fUS (rsfUS) in awake mice in an environment similar to fMRI measurement. Then, we compared the results to those of resting state fMRI (rsfMRI) conducted using an 11.7 Tesla scanner. RsfUS experiments revealed a significant reduction in FC between the retrosplenial dysgranular and auditory cortexes (0.56 ± 0.07 at silence vs 0.05 ± 0.05 at 110 dB, p=.01) and a significant increase in FC anticorrelation between the infralimbic and motor cortexes (-0.21 ± 0.08 at silence vs -0.47 ± 0.04 at 110 dB, p=.017) as acoustic noise increased from silence to 80 dB and 110 dB, with increased consistency of FC patterns between rsfUS and rsfMRI being found with the louder noise conditions. Event-related auditory stimulation experiments using fUS showed strong positive rCBV changes (16.5% ± 2.9% at 110 dB) in the auditory cortex, and negative rCBV changes (-6.7% ± 0.8% at 110 dB) in the motor cortex, both being constituents of the brain network that was altered by the presence of acoustic noise in the resting state experiments. Anticorrelation between constituent brain regions of the default mode network (such as the infralimbic cortex) and those of task-positive sensorimotor networks (such as the motor cortex) is known to be an important feature of brain network antagonism, and has been studied as a biological marker of brain disfunction and disease. This study suggests that attention should be paid to the acoustic noise level when using rsfMRI to evaluate the anticorrelation between the default mode network and task-positive sensorimotor network.

Brain-wide mapping of resting-state networks in mice using high-frame rate functional ultrasound.

Functional ultrasound (fUS) imaging is a method for visualizing deep brain activity based on cerebral blood volume changes coupled with neural activity, while functional MRI (fMRI) relies on the blood-oxygenation-level-dependent signal coupled with neural activity. Low-frequency fluctuations (LFF) of fMRI signals during resting-state can be measured by resting-state fMRI (rsfMRI), which allows functional imaging of the whole brain, and the distributions of resting-state network (RSN) can then be estimated from these fluctuations using independent component analysis (ICA). This procedure provides an important method for studying cognitive and psychophysiological diseases affecting specific brain networks. The distributions of RSNs in the brain-wide area has been reported primarily by rsfMRI. RSNs using rsfMRI are generally computed from the time-course of fMRI signals for more than 5 min. However, a recent dynamic functional connectivity study revealed that RSNs are still not perfectly stable even after 10 min. Importantly, fUS has a higher temporal resolution and stronger correlation with neural activity compared with fMRI. Therefore, we hypothesized that fUS applied during the resting-state for a shorter than 5 min would provide similar RSNs compared to fMRI. High temporal resolution rsfUS data were acquired at 10 Hz in awake mice. The quality of the default mode network (DMN), a well-known RSN, was evaluated using signal-noise separation (SNS) applied to different measurement durations of rsfUS. The results showed that the SNS did not change when the measurement duration was increased to more than 210 s. Next, we measured short-duration rsfUS multi-slice measurements in the brain-wide area. The results showed that rsfUS with the short duration succeeded in detecting RSNs distributed in the brain-wide area consistent with RSNs detected by 11.7-T MRI under awake conditions (medial prefrontal cortex and cingulate cortex in the anterior DMN, retrosplenial cortex and visual cortex in the posterior DMN, somatosensory and motor cortexes in the lateral cortical network, thalamus, dorsal hippocampus, and medial cerebellum), confirming the reliability of the RSNs detected by rsfUS. However, bilateral RSNs located in the secondary somatosensory cortex, ventral hippocampus, auditory cortex, and lateral cerebellum extracted from rsfUS were different from the unilateral RSNs extracted from rsfMRI. These findings indicate the potential of rsfUS as a method for analyzing functional brain networks and should encourage future research to elucidate functional brain networks and their relationships with disease model mice.

Study on method of organ section retention and tracking through deep learning in automated diagnostic and therapeutic robotics.

PURPOSE: In high-intensity focused ultrasound (HIFU) treatment of the kidney and liver, tracking the organs is essential because respiratory motions make continuous cauterization of the affected area difficult and may cause damage to other parts of the body. In this study, we propose a tracking system for rotational scanning, and propose and evaluate a method for estimating the angles of organs in ultrasound images. METHODS: We proposed AEMA, AEMAD, and AEMAD++ as methods for estimating the angles of organs in ultrasound images, using RUDS and a phantom to acquire 90-degree images of a kidney from the long-axis image to the short-axis image as a data set. Six datasets were used, with five for preliminary preparation and one for testing, while the initial position was shifted by 2 mm in the contralateral axis direction. The test data set was evaluated by estimating the angle using each method. RESULTS: The accuracy and processing speed of angle estimation for AEMA, AEMAD, and AEMAD++ were 23.8% and 0.33 FPS for AEMAD, 32.0% and 0.56 FPS for AEMAD, and 29.5% and 3.20 FPS for AEMAD++, with tolerance of ± 2.5 degrees. AEMAD++ offered the best speed and accuracy. CONCLUSION: In the phantom experiment, AEMAD++ showed the effectiveness of tracking the long-axis image of the kidney in rotational scanning. In the future, we will add either the area of surrounding organs or the internal structure of the kidney as a new feature to validate the results.

Suppression of Clothing-Induced Acoustic Attenuation in Robotic Auscultation.

For patients who are often embarrassed and uncomfortable when exposing their breasts and having them touched by physicians of different genders during auscultation, we are developing a robotic system that performs auscultation over clothing. As the technical issue, the sound obtained through the clothing is often attenuated. This study aims to investigate clothing-induced acoustic attenuation and develop a suppression method for it. Because the attenuation is due to the loss of energy as sound propagates through a medium with viscosity, we hypothesized that the attenuation is improved by compressing clothing and shortening the sound propagation distance. Then, the amplitude spectrum of the heart sound was obtained over clothes of different thicknesses and materials in a phantom study and human trial at varying contact forces with a developed passive-actuated end-effector. Our results demonstrate the feasibility of the attenuation suppression method by applying an optimum contact force, which varied according to the clothing condition. In the phantom experiments, the attenuation rate was improved maximumly by 48% when applying the optimal contact force (1 N). In human trials, the attenuation rate was under the acceptable attenuation (40%) when applying the optimal contact force in all combinations in each subject. The proposed method promises the potential of robotic auscultation toward eliminating gender bias.

Body surface registration considering individual differences with non-rigid iterative closest point.

PURPOSE: In telemedicine such as remote auscultation, patients themselves or non-medical people such as patient's parents need to place the stethoscope on their body surface in appropriate positions instead of the physicians. Meanwhile, as the position depends on the individual difference of body shape, there is a demand for the efficient navigation to place the medical equipment. METHODS: In this paper, we have proposed a non-rigid iterative closest point (ICP)-based registration method for localizing the auscultation area considering the individual difference of body surface. The proposed system provides the listening position by applying the body surface registration between the patient and reference model with the specified auscultation area. Our novelty is that selecting the utilized reference model similar to the patient body among several types of the prepared reference model increases the registration accuracy. RESULTS: Simulation results showed that the registration error increases due to deviations of the body shape between the targeted models and reference model. Experimental results demonstrated that the proposed non-rigid ICP registration is capable of estimating the auscultation area with average error 5-19 mm when selecting the most similar reference model. The statistical analysis showed high correlation between the registration accuracy and similarity of the utilized models. CONCLUSION: The proposed non-rigid ICP registration is a promising new method that provides accurate auscultation area takes into account the individual difference of body shape. Our hypothesis that the registration accuracy depends on the similarity of both body surfaces is validated through simulation study and human trial.

A VS ultrasound diagnostic system with kidney image evaluation functions.

PURPOSE: An inevitable feature of ultrasound-based diagnoses is that the quality of the ultrasound images produced depends directly on the skill of the physician operating the probe. This is because physicians have to constantly adjust the probe position to obtain a cross section of the target organ, which is constantly shifting due to patient respiratory motions. Therefore, we developed an ultrasound diagnostic robot that works in cooperation with a visual servo system based on deep learning that will help alleviate the burdens imposed on physicians. METHODS: Our newly developed robotic ultrasound diagnostic system consists of three robots: an organ tracking robot (OTR), a robotic bed, and a robotic supporting arm. Additionally, we used different image processing methods (YOLOv5s and BiSeNet V2) to detect the target kidney location, as well as to evaluate the appropriateness of the obtained ultrasound images (ResNet 50). Ultimately, the image processing results are transmitted to the OTR for use as motion commands. RESULTS: In our experiments, the highest effective tracking rate (0.749) was obtained by YOLOv5s with Kalman filtering, while the effective tracking rate was improved by about 37% in comparison with cases without such filtering. Additionally, the appropriateness probability of the ultrasound images obtained during the tracking process was also the highest and most stable. The second highest tracking efficiency value (0.694) was obtained by BiSeNet V2 with Kalman filtering and was a 75% improvement over the case without such filtering. CONCLUSION: While the most efficient tracking achieved is based on the combination of YOLOv5s and Kalman filtering, the combination of BiSeNet V2 and Kalman filtering was capable of detecting the kidney center of gravity closer to the kidney's actual motion state. Furthermore, this model could also measure the cross-sectional area, maximum diameter, and other detailed information of the target kidney, which meant it is more practical for use in actual diagnoses.

Towards fully automated robotic platform for remote auscultation.

BACKGROUND: Since most developed countries are facing an increase in the number of patients per healthcare worker due to a declining birth rate and an ageing population, relatively simple and safe diagnosis tasks may need to be performed using robotics and automation technologies, without specialists and hospitals. This study presents an automated robotic platform for remote auscultation, which is a highly cost-effective screening tool for detecting abnormal clinical signs. METHOD: The developed robotic platform is composed of a 6-degree-of-freedom cooperative robotic arm, LiDAR camera, and a spring-based mechanism holding an electric stethoscope. The platform enables autonomous stethoscope positioning based on external body information acquired using the LiDAR camera-based multi-way registration; the platform also ensures safe and flexible contact, maintaining the contact force within a certain range through the passive-actuated mechanism. RESULTS: Our preliminary results confirm that the robotic platform enables estimation of the landing positions required for cardiac examinations based on the depth and landmark information of the body surface. It also handles the stethoscope while maintaining the contact force without relying on the push-in displacement by the robotic arm. CONCLUSION: The developed robotic platform enables the estimation of the landing positions and handling the stethoscope while maintaining the contact force, which promises the potential of automatic remote auscultation.

A visualization method for a wide range of rising temperature induced by high-intensity focused ultrasound using a tissue-mimicking phantom.

PURPOSE: High-intensity focused ultrasound (HIFU) treatment requires prior evaluation of the HIFU transducer output. A method using micro-capsulated thermochromic liquid crystal (MTLC) to evaluate the temperature distribution in the media during HIFU exposure has been previously developed. However, the color-coded temperature range of commercial MTLC is approximately 10 °C, which is insufficient for temperature measurement for HIFU exposure. We created two layers of tissue-mimicking phantoms with different color-coded temperature ranges, and a new visualization method was developed by utilizing the axisymmetric pressure distribution of a HIFU focus. METHODS: A two-layer phantom with two sensitivity ranges was created. The HIFU transducer was set to align the focal point to the boundary between the two layers. Images of the upper and lower layers were flipped along the boundary between the two layers such that they overlapped with each other, assuming the pressure distribution of HIFU to be axisymmetric. RESULTS: The experimental and simulation results were compared to evaluate the accuracy of the phantom temperature measurement. The experimental time profile of the temperature and spatial distribution around the HIFU focus matched well with that of the simulation. However, there is room for improvement in the accuracy in the axial direction of HIFU focus. CONCLUSION: Users can apply our proposed method in clinical practice to promptly assess the output of the HIFU transducer before treatment.

Evaluation of Contact Force between Aneurysm Model and Coil for Embolization of Intracranial Aneurysms.

Objective: To ensure safe coil embolization for intracranial aneurysms, it is important to investigate the contact force between the coil and the aneurysm wall. However, it is unclear how the catheter tip position and the diameter of the secondary loop of the coil influence the contact force. In this study, we measured the contact force between a coil and an aneurysm biomodel under different conditions. Methods: A commercially available coil was inserted through a microcatheter into a silicone rubber aneurysm model at a constant speed (1 mm/s) using an automatic stage, and the contact force between the coil and the aneurysm wall was measured by a force sensor attached on the aneurysm model. The inner diameter of the spherical aneurysm was 5 mm. The effects of varying the position of the catheter tip (near dome, center, near neck) and the diameter of the secondary coil (4.5 mm) were evaluated. Results: When the catheter tip was inserted more deeply into the aneurysm (especially near the dome), the contact force increased. The contact force also increased as the secondary coil diameter was increased with the catheter tip near and in the center of the dome. Conclusion: These results suggest that the catheter tip position and the secondary coil diameter affect the contact force. In particular, the contact force should be considered large with the catheter tip near the dome to ensure safe coil deployment.

Numerical analysis and experimental observation of guidewire motion in a blood vessel model.

We have developed a computer-based system to simulate a guidewire in blood vessels for surgical planning, intra-operative assistance, and to facilitate the design of new guidewires. In this study, we compared simulation results with experimental results for validation of the simulation system. First, we inserted a commercial guidewire into a poly (vinyl alcohol) hydrogel blood vessel model using a two-axis automatic stage and measured the position of the guidewire tip and the contact force between the guidewire and the vessel. The experimental apparatus can be used not only for the validation of numerical analyses, but also as a simulation system. Second, similarly to the experiment, the motion of the guidewire in the blood vessel model was calculated when the proximal part of the guidewire model was pushed and twisted. The model of the guidewire is constructed with viscoelastic springs and segments, and the proximal part of the guidewire model is constrained by the fixed catheter model. Collisions between the guidewire and the vessel are calculated, and the contact forces are determined according to the stiffness of the vessel wall. The same tendency was seen in the trajectories and the contact force of both the experimental and simulated guidewire tips.

Temperature distributions measurement of high intensity focused ultrasound using a thin-film thermocouple array and estimation of thermal error caused by viscous heating.

To improve the throughput of high intensity focused ultrasound (HIFU) treatment, we have considered a focus switching method at two points. For this method, it is necessary to evaluate the thermal distribution under exposure to ultrasound. The thermal distribution was measured using a prototype thin-film thermocouple array, which has the advantage of minimizing the influence of the thermocouple on the acoustic and temperature fields. Focus switching was employed to enlarge the area of temperature increase and evaluate the proposed evaluation parameters with respect to safety and uniformity. The results indicate that focus switching can effectively expand the thermal lesion while maintaining a steep thermal boundary. In addition, the influence caused by the thin-film thermocouple array was estimated experimentally. This thermocouple was demonstrated to be an effective tool for the measurement of temperature distributions induced by HIFU.

Localized elasticity measurement for detection of coagulation during HIFU therapy.

High intensity focused ultrasound (HIFU) treatment is one minimally invasive treatment method for cancer. Visualizing the internal treatment area of a body during HIFU treatment is required in order to achieve appropriate beam positioning and HIFU dosage. The objective of this work is to develop an ultrasound monitoring system for thermally induced coagulation. Localized motion imaging (LMI) is a monitoring method used to detect a localized mechanical response that is dependent on changes in tissue stiffness caused by thermal coagulation. In LMI, amplitude modulated HIFU causes oscillation of tissues in the HIFU focal area. The elastic modulus at a coagulated area increases and can be detected as an area with decreased oscillation amplitude., Localized control of the oscillation by changing the modulation frequency was conducted to increase the detection sensitivity for small coagulated areas in porcine liver. 2 and 7.5 MHz transducers were employed for HIFU and imaging, respectively. The amplitude modulation frequency was changed in the range from 50 to 200 Hz. The acoustic intensity of HIFU was 2.0 kW/cm2 at the focus and the exposure time was 45 s. The decrease in the amplitude of tissue oscillation at the focal point was detected within 5-10 s of HIFU exposure at the highest modulation frequency. The detected amplitude was decreased to 0.2, which indicates that for LMI, a high modulation frequency is suitable for the detection of small coagulation areas or areas of initial coagulation.

A novel high intensity focused ultrasound robotic system for breast cancer treatment.

High intensity focused ultrasound (HIFU) is a promising technique for cancer treatment owing to its minimal invasiveness and safety. However, skin burn, long treatment time and incomplete ablation are main shortcomings of this method. This paper presents a novel HIFU robotic system for breast cancer treatment. The robot has 4 rotational degrees of freedom with the workspace located in a water tank for HIFU beam imaging and ablation treatment. The HIFU transducer combined with a diagnostic 2D linear ultrasound probe is mounted on the robot end-effector, which is rotated around the HIFU focus when ablating the tumor. HIFU beams are visualized by the 2D probe using beam imaging. Skin burn can be prevented or alleviated by avoiding long time insonification towards the same skin area. The time cost could be significantly reduced, as there is no need to interrupt the ablation procedure for cooling the skin. In addition, our proposed robot control strategies can avoid incomplete ablation. Experiments were carried out and the results showed the effectiveness of our proposed system.

Ultrasound-mediated gene transfection in vitro: effect of ultrasonic parameters on efficiency and cell viability.

Ultrasound (US)-mediated gene transfection in the presence of microbubbles is a recently developed and promising non-viral gene delivery method. Optimising the parameters used in ultrasonic transfection is urgently required in order to realise higher transfection efficiencies in clinical settings. This study examined the effect of ultrasound exposure parameters on plasmid DNA transfection in mouse embryonic fibroblast cell lines using perfluorobutane bubbles. Variations in US intensity (0-11 W/cm2), pulse repetition frequency (PRF, 50-50,000 Hz), duty ratio (10 to 50%), exposure time (0-120 s) and microbubble volume concentration (0 to 10%) were tested, and the microbubble volume concentration was also monitored during exposure. Through the experiments, the mechanism of how variations in parameters influence US-mediated gene transfection was discussed, which can provide a basis for future applications of ultrasound mediated transfection.

Visual servoing for a US-guided therapeutic HIFU system by coagulated lesion tracking: a phantom study.

BACKGROUND: Applying ultrasound (US)-guided high-intensity focused ultrasound (HIFU) therapy for kidney tumours is currently very difficult, due to the unclearly observed tumour area and renal motion induced by human respiration. In this research, we propose new methods by which to track the indistinct tumour area and to compensate the respiratory tumour motion for US-guided HIFU treatment. METHODS: For tracking indistinct tumour areas, we detect the US speckle change created by HIFU irradiation. In other words, HIFU thermal ablation can coagulate tissue in the tumour area and an intraoperatively created coagulated lesion (CL) is used as a spatial landmark for US visual tracking. Specifically, the condensation algorithm was applied to robust and real-time CL speckle pattern tracking in the sequence of US images. Moreover, biplanar US imaging was used to locate the three-dimensional position of the CL, and a three-actuator system drives the end-effector to compensate for the motion. Finally, we tested the proposed method by using a newly devised phantom model that enables both visual tracking and a thermal response by HIFU irradiation. RESULTS: In the experiment, after generation of the CL in the phantom kidney, the end-effector successfully synchronized with the phantom motion, which was modelled by the captured motion data for the human kidney. The accuracy of the motion compensation was evaluated by the error between the end-effector and the respiratory motion, the RMS error of which was approximately 2 mm. CONCLUSION: This research shows that a HIFU-induced CL provides a very good landmark for target motion tracking. By using the CL tracking method, target motion compensation can be realized in the US-guided robotic HIFU system.

Three-dimensional computer-controlled acoustic pressure scanning and quantification of focused ultrasound.

We propose an automated needle hydrophone-based scanning system to measure high-resolution 3-D acoustic pressure distributions generated by high-intensity focused ultrasound (HIFU). The hardware consists of a host computer, subsystems for HIFU generation, and an oscilloscope to sample the pressure response from a sensor in a water tank. Software was developed to control the hardware subsystems, to search for the initial scan position, and to design the scanning path and volume. A preoperative scanning plan with three perpendicular planes is used to manipulate the position of the HIFU transducer and to automate the acquisition of the spatial acoustic pressure distribution. The post-processing process displays the scanning results, compensates time delays caused by continuous linear scans, and quantifies the focal region. A method to minimize the displacement error induced by the time delay improves the scanning speed of a conventional needle hydrophone-based scanning system. Moreover, a noise-robust, automatic-focus searching algorithm using Gaussian function fitting reduces the total number of iterations and prevents the initial scanning position search from diverging. Finally, the minimum-volume enclosing ellipsoid approximation is used to quantify the size and orientation of the 3-D focal region thresholded by the minimum pressure of interest for various input conditions and to test the reproducibility of the scanning system.