Real-time nonstandard-shaped gold fiducial marker tracking on x-ray fluoroscopic images for prostate radiotherapy.

Objective.The prostate moves in accordance with the movement of surrounding organs. Tumor position can change by ≥3 mm during radiotherapy. Given the difficulties of visualizing the prostate fluoroscopically, fiducial markers are generally implanted into the prostate to monitor its motion during treatment. Recently, internally motion guidance methods of the prostate using a 99.5% gold/0.5% iron flexible notched wire fiducial marker (Gold Anchor® , Naslund Medical AB, Huddinge, Sweden), which requires a 22 gauge needle, has been used. However, because the notched wire can retain its linear shape, acquire a spiral shape, or roll into an irregular ball, detecting it on fluoroscopic images in real-time incurs higher computation costs.Approach.We developed a fiducial tracking algorithm to achieve real-time computation. The marker is detected on the first image frame using a shape filter that employs inter-class variance for the marker likelihood calculated by the filter, focusing on the large difference in densities between the marker and its surroundings. After the second frame, the marker is tracked by adding to the shape filter the similarity to the template cropped from the area around the marker position detected in the first frame. We retrospectively evaluated the algorithm's marker tracking accuracy for ten prostate cases, analyzing two fractions in each case.Main results.Tracking positional accuracy averaged over all patients was 0.13 ± 0.04 mm (mean ± standard deviation, Euclidean distance) and 0.25 ± 0.09 mm (95th percentile). Computation time was 2.82 ± 0.20 ms/frame averaged over all frames.Significance.Our algorithm successfully and stably tracked irregularly-shaped markers in real time.

Underwater imaging dataset in a very shallow water environment of Pramuka Island, Seribu Island District, Indonesia.

In this article, we present a dataset of underwater videos captured through manual dives in a complex and unstructured seabed area dominated by harbor structures and coral reefs. The area is shallow (0.5 - 7.0 m depth) with an enclosed embayment for the harbor area, offering protection from ocean currents and waves. The coral reef area is located in a more open ocean sloping gently toward the deeper seafloor, leading to a more pronounced rolling shutter effect and camera motion. The dataset was collected using a GoPro Hero 10 camera, employing a standard wide lens with a horizontal field of view (FoV) of 109° and 768 × 432 image resolution. The camera is also equipped with an Inertial Measurement Unit (IMU) sensor, comprising a 200 Hz frequency accelerometer and gyroscope. During underwater deployment, the camera is protected with a 5 mm thick flat glass panel. This camera setting hence creates three medium layers of water-glass-air leading to additional refraction distortion. To address the refraction distortion, the dataset has been subject to pre-calibration utilizing flat refractive geometry found in the Pinax camera model. The Pinax camera model for the underwater imagery is calculated by combining the aspects of pinhole calibration parameters with axial camera projection. The main aim of the dataset collection is to facilitate the testing and evaluation of underwater imaging algorithms that are used in underwater robotics, such as computer vision, photogrammetry, and Simultaneous Localization and Mapping (SLAM).

Deep learning enables MV-based real-time image guided radiation therapy for prostate cancer patients.

Objective. Using MV images for real-time image guided radiation therapy (IGRT) is ideal as it does not require additional imaging equipment, adds no additional imaging dose and provides motion data in the treatment beam frame of reference. However, accurate tracking using MV images is challenging due to low contrast and modulated fields. Here, a novel real-time marker tracking system based on a convolutional neural network (CNN) classifier was developed and evaluated on retrospectively acquired patient data for MV-based IGRT for prostate cancer patients.Approach. MV images, acquired from 29 volumetric modulated arc therapy (VMAT) prostate cancer patients treated in a multi-institutional clinical trial, were used to train and evaluate a CNN-based marker tracking system. The CNN was trained using labelled MV images from 9 prostate cancer patients (35 fractions) with implanted markers. CNN performance was evaluated on an independent cohort of unseen MV images from 20 patients (78 fractions), using a Precision-Recall curve (PRC), area under the PRC plot (AUC) and sensitivity and specificity. The accuracy of the tracking system was evaluated on the same unseen dataset and quantified by calculating mean absolute (±1 SD) and [1st, 99th] percentiles of the geometric tracking error in treatment beam co-ordinates using manual identification as the ground truth.Main results. The CNN had an AUC of 0.99, sensitivity of 98.31% and specificity of 99.87%. The mean absolute geometric tracking error was 0.30 ± 0.27 and 0.35 ± 0.31 mm in the lateral and superior-inferior directions of the MV images, respectively. The [1st, 99th] percentiles of the error were [-1.03, 0.90] and [-1.12, 1.12] mm in the lateral and SI directions, respectively.Significance. The high classification performance on unseen MV images demonstrates the CNN can successfully identify implanted prostate markers. Furthermore, the sub-millimetre accuracy and precision of the marker tracking system demonstrates potential for adaptation to real-time applications.

Performance Evaluation of Deformable Image Registration Algorithms Using Computed Tomography of Multiple Lung Metastases.

Purpose: Various deformable image registration (DIR) methods have been used to evaluate organ deformations in 4-dimensional computed tomography (4D CT) images scanned during the respiratory motions of a patient. This study assesses the performance of 10 DIR algorithms using 4D CT images of 5 patients with fiducial markers (FMs) implanted during the postoperative radiosurgery of multiple lung metastases. Methods: To evaluate DIR algorithms, 4D CT images of 5 patients were used, and ground-truths of FMs and tumors were generated by physicians based on their medical expertise. The positions of FMs and tumors in each 4D CT phase image were determined using 10 DIR algorithms, and the deformed results were compared with ground-truth data. Results: The target registration errors (TREs) between the FM positions estimated by optical flow algorithms and the ground-truth ranged from 1.82 ± 1.05 to 1.98 ± 1.17 mm, which is within the uncertainty of the ground-truth position. Two algorithm groups, namely, optical flow and demons, were used to estimate tumor positions with TREs ranging from 1.29 ± 1.21 to 1.78 ± 1.75 mm. With respect to the deformed position for tumors, for the 2 DIR algorithm groups, the maximum differences of the deformed positions for gross tumor volume tracking were approximately 4.55 to 7.55 times higher than the mean differences. Errors caused by the aforementioned difference in the Hounsfield unit values were also observed. Conclusions: We quantitatively evaluated 10 DIR algorithms using 4D CT images of 5 patients and compared the results with ground-truth data. The optical flow algorithms showed reasonable FM-tracking results in patient 4D CT images. The iterative optical flow method delivered the best performance in this study. With respect to the tumor volume, the optical flow and demons algorithms delivered the best performance.

Improved Position Accuracy of Foot-Mounted Inertial Sensor by Discrete Corrections from Vision-Based Fiducial Marker Tracking.

In this paper, we present a novel pedestrian indoor positioning system that uses sensor fusion between a foot-mounted inertial measurement unit (IMU) and a vision-based fiducial marker tracking system. The goal is to provide an after-action review for first responders during training exercises. The main contribution of this work comes from the observation that different walking types (e.g., forward walking, sideways walking, backward walking) lead to different levels of position and heading error. Our approach takes this into account when accumulating the error, thereby leading to more-accurate estimations. Through experimentation, we show the variation in error accumulation and the improvement in accuracy alter when and how often to activate the camera tracking system, leading to better balance between accuracy and power consumption overall. The IMU and vision-based systems are loosely coupled using an extended Kalman filter (EKF) to ensure accurate and unobstructed positioning computation. The motion model of the EKF is derived from the foot-mounted IMU data and the measurement model from the vision system. Existing indoor positioning systems for training exercises require extensive active infrastructure installation, which is not viable for exercises taking place in a remote area. With the use of passive infrastructure (i.e., fiducial markers), the positioning system can accurately track user position over a longer duration of time and can be easily integrated into the environment. We evaluated our system on an indoor trajectory of 250 m. Results show that even with discrete corrections, near a meter level of accuracy can be achieved. Our proposed system attains the positioning accuracy of 0.55 m for a forward walk, 1.05 m for a backward walk, and 1.68 m for a sideways walk with a 90% confidence level.

Integrating XMALab and DeepLabCut for high-throughput XROMM.

Marker tracking is a major bottleneck in studies involving X-ray reconstruction of moving morphology (XROMM). Here, we tested whether DeepLabCut, a new deep learning package built for markerless tracking, could be applied to videoradiographic data to improve data processing throughput. Our novel workflow integrates XMALab, the existing XROMM marker tracking software, and DeepLabCut while retaining each program's utility. XMALab is used for generating training datasets, error correction and 3D reconstruction, whereas the majority of marker tracking is transferred to DeepLabCut for automatic batch processing. In the two case studies that involved an in vivo behavior, our workflow achieved a 6 to 13-fold increase in data throughput. In the third case study, which involved an acyclic, post-mortem manipulation, DeepLabCut struggled to generalize to the range of novel poses and did not surpass the throughput of XMALab alone. Deployed in the proper context, this new workflow facilitates large scale XROMM studies that were previously precluded by software constraints.

Utilizing the TrueBeam Advanced Imaging Package to monitor intrafraction motion with periodic kV imaging and automatic marker detection during VMAT prostate treatments.

BACKGROUND: Fiducial markers are frequently used before treatment for image-guided patient setup in radiation therapy (RT), but can also be used during treatment for image-guided intrafraction motion detection. This report describes our implementation of automatic marker detection with periodic kV imaging (TrueBeam v2.5) to monitor and correct intrafraction motion during prostate RT. METHODS: We evaluated the reproducibility and accuracy of software fiducial detection using a phantom with 3 implanted fiducial markers. Clinical implementation for patients with intraprostatic fiducials receiving volumetric modulated arc therapy (VMAT) utilized periodic on-board kV imaging with 10 s intervals during treatment delivery. For each image, the software automatically identified fiducial locations and determined whether their distance relative to planned locations were within a 3 mm tolerance. Motion was corrected if either ≥2 fiducials in a single image or ≥1 fiducial in sequential images were out of tolerance. RESULTS: Phantom studies demonstrated poorer performance of linear fiducials compared to collapsible fiducials, and wide variability to accurately detect fiducials across eight software settings. For any given setting, results were relatively reproducible and precise to ~0.5 mm. Across 17 patients treated with a median of 20 fractions, the software recommended a shift in 44% of fractions, and a shift was actually implemented after visual confirmation of movement greater than the 3 mm threshold in 20% of fractions. Adjustment of our approach led to improved accuracy for the latter (n = 7) patient subset. On average, table repositioning added 3.0 ± 0.3 min to patient time on table. Periodic kV imaging increased skin dose by an estimated 1 cGy per treatment arc. CONCLUSIONS: Periodic kV imaging with automatic detection of motion during VMAT prostate treatments is commercially available, and can be successfully implemented to mitigate effects of intrafraction motion with careful attention to software settings.

Retrospective assessment of a single fiducial marker tracking regimen with robotic stereotactic body radiation therapy for liver tumours.

AIM: To investigate tumour motion tracking uncertainties in the CyberKnife Synchrony system with single fiducial marker in liver tumours. BACKGROUND: In the fiducial-based CyberKnife real-time tumour motion tracking system, multiple fiducial markers are generally used to enable translation and rotation corrections during tracking. However, sometimes a single fiducial marker is employed when rotation corrections are not estimated during treatment. MATERIALS AND METHODS: Data were analysed for 32 patients with liver tumours where one fiducial marker was implanted. Four-dimensional computed tomography (CT) scans were performed to determine the internal target volume (ITV). Before the first treatment fraction, the CT scans were repeated and the marker migration was determined. Log files generated by the Synchrony system were obtained after each treatment and the correlation model errors were calculated. Intra-fractional spine rotations were examined on the spine alignment images before and after each treatment. RESULTS: The mean (standard deviation) ITV margin was 4.1 (2.3) mm, which correlated weakly with the distance between the fiducial marker and the tumour. The mean migration distance of the marker was 1.5 (0.7) mm. The overall mean correlation model error was 1.03 (0.37) mm in the radial direction. The overall mean spine rotations were 0.27° (0.31), 0.25° (0.22), and 0.23° (0.26) for roll, pitch, and yaw, respectively. The treatment time was moderately associated with the correlation model errors and weakly related to spine rotation in the roll and yaw planes. CONCLUSIONS: More caution and an additional safety margins are required when tracking a single fiducial marker.

Development of a robust and cost-effective 3D respiratory motion monitoring system using the kinect device: Accuracy comparison with the conventional stereovision navigation system.

BACKGROUND AND OBJECTIVE: To develop and validate a robust and cost-effective 3D respiratory monitoring system based on a Kinect device with a custom-made simple marker. METHODS: A 3D respiratory monitoring system comprising the simple marker and the Microsoft Kinect v2 device was developed. The marker was designed for simple and robust detection, and the tracking algorithm was developed using the depth, RGB, and infra-red images acquired from the Kinect sensor. A Kalman filter was used to suppress movement noises. The major movements of the marker attached to the four different locations of body surface were determined from the initially collected tracking points of the marker while breathing. The signal level of respiratory motion with the tracking point was estimated along the major direction vector. The accuracy of the results was evaluated through a comparison with those of the conventional stereovision navigation system (NDI Polaris Spectra). RESULTS: Sixteen normal volunteers were enrolled to evaluate the accuracy of this system. The correlation coefficients between the respiratory motion signal from the Kinect device and conventional navigation system ranged from 0.970 to 0.999 and from 0.837 to 0.995 at the abdominal and thoracic surfaces, respectively. The respiratory motion signal from this system was obtained at 27-30 frames/s. CONCLUSIONS: This system with the Kinect v2 device and simple marker could be used for cost-effective, robust and accurate 3D respiratory motion monitoring. In addition, this system is as reliable for respiratory motion signal generation and as practically useful as the conventional stereovision navigation system and is less sensitive to patient posture.

Image Analysis Technique for Material Behavior Evaluation in Civil Structures.

The article presents a hybrid monitoring technique for the measurement of the deformation field. The goal is to obtain information about crack propagation in existing structures, for the purpose of monitoring their state of health. The measurement technique is based on the capture and analysis of a digital image set. Special markers were used on the surface of the structures that can be removed without damaging existing structures as the historical masonry. The digital image analysis was done using software specifically designed in Matlab to follow the tracking of the markers and determine the evolution of the deformation state. The method can be used in any type of structure but is particularly suitable when it is necessary not to damage the surface of structures. A series of experiments carried out on masonry walls of the Oliverian Museum (Pesaro, Italy) and Palazzo Silvi (Perugia, Italy) have allowed the validation of the procedure elaborated by comparing the results with those derived from traditional measuring techniques.

Development of a real-time internal and external marker tracking system for particle therapy: a phantom study using patient tumor trajectory data.

Target motion-induced uncertainty in particle therapy is more complicated than that in X-ray therapy, requiring more accurate motion management. Therefore, a hybrid motion-tracking system that can track internal tumor motion and as well as an external surrogate of tumor motion was developed. Recently, many correlation tests between internal and external markers in X-ray therapy have been developed; however, the accuracy of such internal/external marker tracking systems, especially in particle therapy, has not yet been sufficiently tested. In this article, the process of installing an in-house hybrid internal/external motion-tracking system is described and the accuracy level of tracking system was acquired. Our results demonstrated that the developed in-house external/internal combined tracking system has submillimeter accuracy, and can be clinically used as a particle therapy system as well as a simulation system for moving tumor treatment.

Validation of XMALab software for marker-based XROMM.

Marker-based XROMM requires software tools for: (1) correcting fluoroscope distortion; (2) calibrating X-ray cameras; (3) tracking radio-opaque markers; and (4) calculating rigid body motion. In this paper we describe and validate XMALab, a new open-source software package for marker-based XROMM (C++ source and compiled versions on Bitbucket). Most marker-based XROMM studies to date have used XrayProject in MATLAB. XrayProject can produce results with excellent accuracy and precision, but it is somewhat cumbersome to use and requires a MATLAB license. We have designed XMALab to accelerate the XROMM process and to make it more accessible to new users. Features include the four XROMM steps (listed above) in one cohesive user interface, real-time plot windows for detecting errors, and integration with an online data management system, XMAPortal. Accuracy and precision of XMALab when tracking markers in a machined object are ±0.010 and ±0.043 mm, respectively. Mean precision for nine users tracking markers in a tutorial dataset of minipig feeding was ±0.062 mm in XMALab and ±0.14 mm in XrayProject. Reproducibility of 3D point locations across nine users was 10-fold greater in XMALab than in XrayProject, and six degree-of-freedom bone motions calculated with a joint coordinate system were 3- to 6-fold more reproducible in XMALab. XMALab is also suitable for tracking white or black markers in standard light videos with optional checkerboard calibration. We expect XMALab to increase both the quality and quantity of animal motion data available for comparative biomechanics research.

Fully automated tracking of cardiac structures using radiopaque markers and high-frequency videofluoroscopy in an in vivo ovine model: from three-dimensional marker coordinates to quantitative analyses.

PURPOSE: Recently, algorithms were developed to track radiopaque markers in the heart fully automated. However, the methodology did not allow to assign the exact anatomical location to each marker. In this case study we describe the steps from the generation of three-dimensional marker coordinates to quantitative data analyses in an in vivo ovine model. METHODS: In one adult sheep, twenty silver balls were sutured to the right side of the heart: 10 to the tricuspid annulus, one to the anterior tricuspid leaflet and nine to the epicardial surface of the right ventricle. In addition, 13 cylindrical tantalum markers were implanted into the left ventricle. Data were acquired with a biplanar X-ray acquisition system (Neurostar R, Siemens AG, 500 Hz). Radiopaque marker coordinates were determined fully automated using novel tracking algorithms. RESULTS: The anatomical marker locations were identified using a 3-dimensional model of a single frame containing all tracked markers. First, cylindrical markers were manually separated from spherical markers, thus allowing to distinguish right from left heart markers. The fast moving leaflet marker was identified by using video loops constructed of all recorded frames. Rotation of the 3-dimensional model allowed the identification of the precise anatomical position for each marker. Data sets were then analyzed quantitatively using customized software. CONCLUSIONS: The method presented in this case study allowed quantitative data analyses of radiopaque cardiac markers that were tracked fully automated with high temporal resolution. However, marker identification still requires substantial manual work. Future improvements including the implication of marker identification algorithms and data analysis software could allow almost real-time quantitative analyses of distinct cardiac structures with high temporal and spatial resolution.

Leaf Length Tracker: a novel approach to analyse leaf elongation close to the thermal limit of growth in the field.

Leaf growth in monocot crops such as wheat and barley largely follows the daily temperature course, particularly under cold but humid springtime field conditions. Knowledge of the temperature response of leaf extension, particularly variations close to the thermal limit of growth, helps define physiological growth constraints and breeding-related genotypic differences among cultivars. Here, we present a novel method, called 'Leaf Length Tracker' (LLT), suitable for measuring leaf elongation rates (LERs) of cereals and other grasses with high precision and high temporal resolution under field conditions. The method is based on image sequence analysis, using a marker tracking approach to calculate LERs. We applied the LLT to several varieties of winter wheat (Triticum aestivum), summer barley (Hordeum vulgare), and ryegrass (Lolium perenne), grown in the field and in growth cabinets under controlled conditions. LLT is easy to use and we demonstrate its reliability and precision under changing weather conditions that include temperature, wind, and rain. We found that leaf growth stopped at a base temperature of 0°C for all studied species and we detected significant genotype-specific differences in LER with rising temperature. The data obtained were statistically robust and were reproducible in the tested environments. Using LLT, we were able to detect subtle differences (sub-millimeter) in leaf growth patterns. This method will allow the collection of leaf growth data in a wide range of future field experiments on different graminoid species or varieties under varying environmental or treatment conditions.

Diel growth patterns of young soybean (Glycine max) leaflets are synchronous throughout different positions on a plant.

Leaf growth is controlled by various internal and external factors. Leaves of dicotyledonous plants show pronounced diel (24 h) growth patterns that are controlled by the circadian clock. To date, it is still uncertain whether diel leaf growth patterns remain constant throughout the development of a plant. In this study, we followed growth from the primary leaves to leaflets of the seventh trifoliate leaf of soybean (Glycine max) on the same plants with a recently developed imaging-based method under controlled conditions and at a high temporal resolution. We found that all leaflets displayed a consistent diel growth pattern with maximum growth towards the end of the night. In some leaves, growth maxima occurred somewhat later - at dawn - as long as the leaves were still in a very early developmental stage. Yet, overall, diel growth patterns of leaves from different positions within the canopy were highly synchronous. Therefore, the diel growth pattern of any leaf at a given point in time is representative for the overall diel growth pattern of the plant leaf canopy and a deviation from the normal diel growth pattern can indicate that the plant is currently facing stress.