Quantification of gallium cryo-FIB milling damage in biological lamellae.

Cryogenic electron microscopy (cryo-EM) can reveal the molecular details of biological processes in their native, cellular environment at atomic resolution. However, few cells are sufficiently thin to permit imaging with cryo-EM. Thinning of frozen cells to <500 nm lamellae by focused-ion-beam (FIB) milling has enabled visualization of cellular structures with cryo-EM. FIB milling represents a significant advance over prior approaches because of its ease of use, scalability, and lack of large-scale sample distortions. However, the amount of damage it causes to a thinned cell section has not yet been determined. We recently described an approach for detecting and identifying single molecules in cryo-EM images of cells using 2D template matching (2DTM). 2DTM is sensitive to small differences between a molecular model (template) and the detected structure (target). Here, we use 2DTM to demonstrate that under the standard conditions used for machining lamellae of biological samples, FIB milling introduces a layer of variable damage that extends to a depth of 60 nm from each lamella surface. This layer of damage limits the recovery of information for in situ structural biology. We find that the mechanism of FIB milling damage is distinct from radiation damage during cryo-EM imaging. By accounting for both electron scattering and FIB milling damage, we estimate that FIB milling damage with current protocols will negate the potential improvements from lamella thinning beyond 90 nm.

MarkerDetector: A method for robust fiducial marker detection in electron micrographs using wavelet-based template.

Fiducial marker detection in electron micrographs becomes an important and challenging task with the development of large-field electron microscopy. The fiducial marker detection plays an important role in several steps during the process of electron micrographs, such as the alignment and parameter calibrations. However, limited by the conditions of low signal-to-noise ratio (SNR) in the electron micrographs, the performance of fiducial marker detection is severely affected. In this work, we propose the MarkerDetector, a novel algorithm for detecting fiducial markers in electron micrographs. The proposed MarkerDetector is built upon the following contributions: Firstly, a wavelet-based template generation algorithm is devised in MarkerDetector. By adopting a shape-based criterion, a high-quality template can be obtained. Secondly, a robust marker determination strategy is devised by utilizing statistic-based filtering, which can guarantee the correctness of the detected fiducial markers. The average running time of our algorithm is 1.67 seconds with promising accuracy, indicating its practical feasibility for applications in electron micrographs.

Hooded crows (Corvus cornix) manufacture objects relative to a mental template.

It was recently found that not only tool-specialized New Caledonian crows, but also Goffin cockatoos can manufacture physical objects in accordance with a mental template. That is, they can emulate features of existing objects when they manufacture new items. Both species spontaneously ripped pieces of card into large strips if they had previously learned that a large template was rewarded, and small strips when they previously learned that a small template was rewarded. Among New Caledonian crows, this cognitive ability was suggested as a potential mechanism underlying the transmission of natural tool designs. Here, we tested for the same ability in another non-specialised tool user-Hooded crows (Corvus cornix). Crows were exposed to pre-made template objects, varying first in colour and then in size, and were rewarded only if they chose pre-made objects that matched the template. In subsequent tests, birds were given the opportunity to manufacture versions of these objects. All three crows ripped paper pieces from the same colour material as the rewarded template, and, crucially, also manufactured objects that were more similar in size to previously rewarded, than unrewarded, templates, despite the birds being rewarded at random in both tests. Therefore, we found the ability to manufacture physical objects relative to a mental template in yet another bird species not specialized in using or making foraging tools in the wild, but with a high level of brain and cognitive development.

Accurate Localization of First and Second Heart Sounds via Template Matching in Forcecardiography Signals.

Cardiac auscultation is an essential part of physical examination and plays a key role in the early diagnosis of many cardiovascular diseases. The analysis of phonocardiography (PCG) recordings is generally based on the recognition of the main heart sounds, i.e., S1 and S2, which is not a trivial task. This study proposes a method for an accurate recognition and localization of heart sounds in Forcecardiography (FCG) recordings. FCG is a novel technique able to measure subsonic vibrations and sounds via small force sensors placed onto a subject's thorax, allowing continuous cardio-respiratory monitoring. In this study, a template-matching technique based on normalized cross-correlation was used to automatically recognize heart sounds in FCG signals recorded from six healthy subjects at rest. Distinct templates were manually selected from each FCG recording and used to separately localize S1 and S2 sounds, as well as S1-S2 pairs. A simultaneously recorded electrocardiography (ECG) trace was used for performance evaluation. The results show that the template matching approach proved capable of separately classifying S1 and S2 sounds in more than 96% of all heartbeats. Linear regression, correlation, and Bland-Altman analyses showed that inter-beat intervals were estimated with high accuracy. Indeed, the estimation error was confined within 10 ms, with negligible impact on heart rate estimation. Heart rate variability (HRV) indices were also computed and turned out to be almost comparable with those obtained from ECG. The preliminary yet encouraging results of this study suggest that the template matching approach based on normalized cross-correlation allows very accurate heart sounds localization and inter-beat intervals estimation.

Phase-Resolved Partial Discharge (PRPD) Pattern Recognition Using Image Processing Template Matching.

This paper proposes a new method for recognizing, extracting, and processing Phase-Resolved Partial Discharge (PRPD) patterns from two-dimensional plots to identify specific defect types affecting electrical equipment without human intervention while retaining the principals that make PRPD analysis an effective diagnostic technique. The proposed method does not rely on training complex deep learning algorithms which demand substantial computational resources and extensive datasets that can pose significant hurdles for the application of on-line partial discharge monitoring. Instead, the developed Cosine Cluster Net (CCNet) model, which is an image processing pipeline, can extract and process patterns from any two-dimensional PRPD plot before employing the cosine similarity function to measure the likeness of the patterns to predefined templates of known defect types. The PRPD pattern recognition capabilities of the model were tested using several manually classified PRPD images available in the existing literature. The model consistently produced similarity scores that identified the same defect type as the one from the manual classification. The successful defect type reporting from the initial trials of the CCNet model together with the speed of the identification, which typically does not exceed four seconds, indicates potential for real-time applications.

Robust Multi-Modal Image Registration for Image Fusion Enhancement in Infrastructure Inspection.

Efficient multi-modal image fusion plays an important role in the non-destructive evaluation (NDE) of infrastructures, where an essential challenge is the precise visualizing of defects. While automatic defect detection represents a significant advancement, the determination of the precise location of both surface and subsurface defects simultaneously is crucial. Hence, visible and infrared data fusion strategies are essential for acquiring comprehensive and complementary information to detect defects across vast structures. This paper proposes an infrared and visible image registration method based on Euclidean evaluation together with a trade-off between key-point threshold and non-maximum suppression. Moreover, we employ a multi-modal fusion strategy to investigate the robustness of our image registration results.

Flexible template matching for observational study design.

Matching is a popular design for inferring causal effect with observational data. Unlike model-based approaches, it is a nonparametric method to group treated and control subjects with similar characteristics together, hence to re-create a randomization-like scenario. The application of matched design for real world data may be limited by: (1) the causal estimand of interest; (2) the sample size of different treatment arms. We propose a flexible design of matching, based on the idea of template matching, to overcome these challenges. It first identifies the template group which is representative of the target population, then match subjects from the original data to this template group and make inference. We provide theoretical justification on how it unbiasedly estimates the average treatment effect using matched pairs and the average treatment effect on the treated when the treatment group has a bigger sample size. We also propose using the triplet matching algorithm to improve matching quality and devise a practical strategy to select the template size. One major advantage of matched design is that it allows both randomization-based or model-based inference, with the former being more robust. For the commonly used binary outcome in medical research, we adopt a randomization inference framework of attributable effects in matched data, which allows heterogeneous effects and can incorporate sensitivity analysis for unmeasured confounding. We apply our design and analytical strategy to a trauma care evaluation study.

ECG-Free Heartbeat Detection in Seismocardiography Signals via Template Matching.

Cardiac monitoring can be performed by means of an accelerometer attached to a subject's chest, which produces the Seismocardiography (SCG) signal. Detection of SCG heartbeats is commonly carried out by taking advantage of a simultaneous electrocardiogram (ECG). SCG-based long-term monitoring would certainly be less obtrusive and easier to implement without an ECG. Few studies have addressed this issue using a variety of complex approaches. This study proposes a novel approach to ECG-free heartbeat detection in SCG signals via template matching, based on normalized cross-correlation as heartbeats similarity measure. The algorithm was tested on the SCG signals acquired from 77 patients with valvular heart diseases, available from a public database. The performance of the proposed approach was assessed in terms of sensitivity and positive predictive value (PPV) of the heartbeat detection and accuracy of inter-beat intervals measurement. Sensitivity and PPV of 96% and 97%, respectively, were obtained by considering templates that included both systolic and diastolic complexes. Regression, correlation, and Bland-Altman analyses carried out on inter-beat intervals reported slope and intercept of 0.997 and 2.8 ms (R2 > 0.999), as well as non-significant bias and limits of agreement of ±7.8 ms. The results are comparable or superior to those achieved by far more complex algorithms, also based on artificial intelligence. The low computational burden of the proposed approach makes it suitable for direct implementation in wearable devices.

Co-Design Dedicated System for Efficient Object Tracking Using Swarm Intelligence-Oriented Search Strategies.

The template matching technique is one of the most applied methods to find patterns in images, in which a reduced-size image, called a target, is searched within another image that represents the overall environment. In this work, template matching is used via a co-design system. A hardware coprocessor is designed for the computationally demanding step of template matching, which is the calculation of the normalized cross-correlation coefficient. This computation allows invariance in the global brightness changes in the images, but it is computationally more expensive when using images of larger dimensions, or even sets of images. Furthermore, we investigate the performance of six different swarm intelligence techniques aiming to accelerate the target search process. To evaluate the proposed design, the processing time, the number of iterations, and the success rate were compared. The results show that it is possible to obtain approaches capable of processing video images at 30 frames per second with an acceptable average success rate for detecting the tracked target. The search strategies based on PSO, ABC, FFA, and CS are able to meet the processing time of 30 frame/s, yielding average accuracy rates above 80% for the pipelined co-design implementation. However, FWA, EHO, and BFOA could not achieve the required timing restriction, and they achieved an acceptance rate around 60%. Among all the investigated search strategies, the PSO provides the best performance, yielding an average processing time of 16.22 ms coupled with a 95% success rate.

Heartbeat Detection in Gyrocardiography Signals without Concurrent ECG Tracings.

A heartbeat generates tiny mechanical vibrations, mainly due to the opening and closing of heart valves. These vibrations can be recorded by accelerometers and gyroscopes applied on a subject's chest. In particular, the local 3D linear accelerations and 3D angular velocities of the chest wall are referred to as seismocardiograms (SCG) and gyrocardiograms (GCG), respectively. These signals usually exhibit a low signal-to-noise ratio, as well as non-negligible amplitude and morphological changes due to changes in posture and the sensors' location, respiratory activity, as well as other sources of intra-subject and inter-subject variability. These factors make heartbeat detection a complex task; therefore, a reference electrocardiogram (ECG) lead is usually acquired in SCG and GCG studies to ensure correct localization of heartbeats. Recently, a template matching technique based on cross correlation has proven to be particularly effective in recognizing individual heartbeats in SCG signals. This study aims to verify the performance of this technique when applied on GCG signals. Tests were conducted on a public database consisting of SCG, GCG, and ECG signals recorded synchronously on 100 patients with valvular heart diseases. The results show that the template matching technique identified heartbeats in GCG signals with a sensitivity and positive predictive value (PPV) of 87% and 92%, respectively. Regression, correlation, and Bland-Altman analyses carried out on inter-beat intervals obtained from GCG and ECG (assumed as reference) reported a slope of 0.995, an intercept of 4.06 ms (R2 > 0.99), a Pearson's correlation coefficient of 0.9993, and limits of agreement of about ±13 ms with a negligible bias. A comparison with the results of a previous study obtained on SCG signals from the same database revealed that GCG enabled effective cardiac monitoring in significantly more patients than SCG (95 vs. 77). This result suggests that GCG could ensure more robust and reliable cardiac monitoring in patients with heart diseases with respect to SCG.

Vision-Based Jigsaw Puzzle Solving with a Robotic Arm.

This study proposed two algorithms for reconstructing jigsaw puzzles by using a color compatibility feature. Two realistic application cases were examined: one involved using the original image, while the other did not. We also calculated the transformation matrix to obtain the real positions of each puzzle piece and transmitted the positional information to the robotic arm, which then put each puzzle piece in its correct position. The algorithms were tested on 35-piece and 70-piece puzzles, achieving an average success rate of 87.1%. Compared with the human visual system, the proposed methods demonstrated enhanced accuracy when handling more complex textural images.

Large Displacement Detection Using Improved Lucas-Kanade Optical Flow.

Displacement is critical when it comes to the evaluation of civil structures. Large displacement can be dangerous. There are many methods that can be used to monitor structural displacements, but every method has its benefits and limitations. Lucas-Kanade (LK) optical flow is recognized as a superior computer vision displacement tracking method, but it only applies to small displacement monitoring. An upgraded LK optical flow method is developed in this study and used to detect large displacement motions. One motion controlled by a multiple purpose testing system (MTS) and a free-falling experiment were designed to verify the developed method. The results provided by the upgraded LK optical flow method showed 97 percent accuracy when compared with the movement of the MTS piston. In order to capture the free-falling large displacement, the pyramid and warp optical flow methods are included in the upgraded LK optical flow method and compared with the results of template matching. The warping algorithm with the second derivative Sobel operator provides accurate displacements with 96% average accuracy.

Mushroom Detection and Three Dimensional Pose Estimation from Multi-View Point Clouds.

Agricultural robotics is an up and coming field which deals with the development of robotic systems able to tackle a multitude of agricultural tasks efficiently. The case of interest, in this work, is mushroom collection in industrial mushroom farms. Developing such a robot, able to select and out-root a mushroom, requires delicate actions that can only be conducted if a well-performing perception module exists. Specifically, one should accurately detect the 3D pose of a mushroom in order to facilitate the smooth operation of the robotic system. In this work, we develop a vision module for 3D pose estimation of mushrooms from multi-view point clouds using multiple RealSense active-stereo cameras. The main challenge is the lack of annotation data, since 3D annotation is practically infeasible on a large scale. To address this, we developed a novel pipeline for mushroom instance segmentation and template matching, where a 3D model of a mushroom is the only data available. We evaluated, quantitatively, our approach over a synthetic dataset of mushroom scenes, and we, further, validated, qualitatively, the effectiveness of our method over a set of real data, collected by different vision settings.

Printing Defect Detection Based on Scale-Adaptive Template Matching and Image Alignment.

Printing defects are extremely common in the manufacturing industry. Although some studies have been conducted to detect printing defects, the stability and practicality of the printing defect detection has received relatively little attention. Currently, printing defect detection is susceptible to external environmental interference such as illuminance and noise, which leads to poor detection rates and poor practicality. This research develops a printing defect detection method based on scale-adaptive template matching and image alignment. Firstly, the research introduces a convolutional neural network (CNN) to adaptively extract deep feature vectors from templates and target images at a low-resolution version. Then, a feature map cross-correlation (FMCC) matching metric is proposed to measure the similarity of the feature map between the templates and target images, and the matching position is achieved by a proposed location refinement method. Finally, the matching image and the template are both sent to the image alignment module, so as to detect printing defects. The experimental results show that the accuracy of the proposed method reaches 93.62%, which can quickly and accurately find the location of the defect. Simultaneously, it is also proven that our method achieves state-of-the-art defect detection performance with strong real-time detection and anti-interference capabilities.

ECG-Free Heartbeat Detection in Seismocardiography and Gyrocardiography Signals Provides Acceptable Heart Rate Variability Indices in Healthy and Pathological Subjects.

Cardio-mechanical monitoring techniques, such as Seismocardiography (SCG) and Gyrocardiography (GCG), have received an ever-growing interest in recent years as potential alternatives to Electrocardiography (ECG) for heart rate monitoring. Wearable SCG and GCG devices based on lightweight accelerometers and gyroscopes are particularly appealing for continuous, long-term monitoring of heart rate and its variability (HRV). Heartbeat detection in cardio-mechanical signals is usually performed with the support of a concurrent ECG lead, which, however, limits their applicability in standalone cardio-mechanical monitoring applications. The complex and variable morphology of SCG and GCG signals makes the ECG-free heartbeat detection task quite challenging; therefore, only a few methods have been proposed. Very recently, a template matching method based on normalized cross-correlation (NCC) has been demonstrated to provide very accurate detection of heartbeats and estimation of inter-beat intervals in SCG and GCG signals of pathological subjects. In this study, the accuracy of HRV indices obtained with this template matching method is evaluated by comparison with ECG. Tests were performed on two public datasets of SCG and GCG signals from healthy and pathological subjects. Linear regression, correlation, and Bland-Altman analyses were carried out to evaluate the agreement of 24 HRV indices obtained from SCG and GCG signals with those obtained from ECG signals, simultaneously acquired from the same subjects. The results of this study show that the NCC-based template matching method allowed estimating HRV indices from SCG and GCG signals of healthy subjects with acceptable accuracy. On healthy subjects, the relative errors on time-domain indices ranged from 0.25% to 15%, on frequency-domain indices ranged from 10% to 20%, and on non-linear indices were within 8%. The estimates obtained on signals from pathological subjects were affected by larger errors. Overall, GCG provided slightly better performances as compared to SCG, both on healthy and pathological subjects. These findings provide, for the first time, clear evidence that monitoring HRV via SCG and GCG sensors without concurrent ECG is feasible with the NCC-based template matching method for heartbeat detection.

An Active Multi-Object Ultrafast Tracking System with CNN-Based Hybrid Object Detection.

This study proposes a visual tracking system that can detect and track multiple fast-moving appearance-varying targets simultaneously with 500 fps image processing. The system comprises a high-speed camera and a pan-tilt galvanometer system, which can rapidly generate large-scale high-definition images of the wide monitored area. We developed a CNN-based hybrid tracking algorithm that can robustly track multiple high-speed moving objects simultaneously. Experimental results demonstrate that our system can track up to three moving objects with velocities lower than 30 m per second simultaneously within an 8-m range. The effectiveness of our system was demonstrated through several experiments conducted on simultaneous zoom shooting of multiple moving objects (persons and bottles) in a natural outdoor scene. Moreover, our system demonstrates high robustness to target loss and crossing situations.

Transformer-Based Multiple-Object Tracking via Anchor-Based-Query and Template Matching.

Multiple object tracking (MOT) plays an important role in intelligent video-processing tasks, which aims to detect and track all moving objects in a scene. Joint-detection-and-tracking (JDT) methods are thriving in MOT tasks, because they accomplish the detection and data association in a single stage. However, the slow training convergence and insufficient data association limit the performance of JDT methods. In this paper, the anchor-based query (ABQ) is proposed to improve the design of the JDT methods for faster training convergence. By augmenting the coordinates of the anchor boxes into the learnable queries of the decoder, the ABQ introduces explicit prior spatial knowledge into the queries to focus the query-to-feature learning of the JDT methods on the local region, which leads to faster training speed and better performance. Moreover, a new template matching (TM) module is designed for the JDT methods, which enables the JDT methods to associate the detection results and trajectories with historical features. Finally, a new transformer-based MOT method, ABQ-Track, is proposed. Extensive experiments verify the effectiveness of the two modules, and the ABQ-Track surpasses the performance of the baseline JDT methods, TransTrack. Specifically, the ABQ-Track only needs to train for 50 epochs to achieve convergence, while that for TransTrack is 150 epochs.

Robust H-K Curvature Map Matching for Patient-to-CT Registration in Neurosurgical Navigation Systems.

Image-to-patient registration is a coordinate system matching process between real patients and medical images to actively utilize medical images such as computed tomography (CT) during surgery. This paper mainly deals with a markerless method utilizing scan data of patients and 3D data from CT images. The 3D surface data of the patient are registered to CT data using computer-based optimization methods such as iterative closest point (ICP) algorithms. However, if a proper initial location is not set up, the conventional ICP algorithm has the disadvantages that it takes a long converging time and also suffers from the local minimum problem during the process. We propose an automatic and robust 3D data registration method that can accurately find a proper initial location for the ICP algorithm using curvature matching. The proposed method finds and extracts the matching area for 3D registration by converting 3D CT data and 3D scan data to 2D curvature images and by performing curvature matching between them. Curvature features have characteristics that are robust to translation, rotation, and even some deformation. The proposed image-to-patient registration is implemented with the precise 3D registration of the extracted partial 3D CT data and the patient's scan data using the ICP algorithm.

Cluster Analysis of Seismicity in the Eastern Gulf of Corinth Based on a Waveform Template Matching Catalog.

The Corinth Rift, in Central Greece, is one of the most seismically active areas in Europe. In the eastern part of the Gulf of Corinth, which has been the site of numerous large and destructive earthquakes in both historic and modern times, a pronounced earthquake swarm occurred in 2020-2021 at the Perachora peninsula. Herein, we present an in-depth analysis of this sequence, employing a high-resolution relocated earthquake catalog, further enhanced by the application of a multi-channel template matching technique, producing additional detections of over 7600 events between January 2020 and June 2021. Single-station template matching enriches the original catalog thirty-fold, providing origin times and magnitudes for over 24,000 events. We explore the variable levels of spatial and temporal resolution in the catalogs of different completeness magnitudes and also of variable location uncertainties. We characterize the frequency-magnitude distributions using the Gutenberg-Richter scaling relation and discuss possible b-value temporal variations that appear during the swarm and their implications for the stress levels in the area. The evolution of the swarm is further analyzed through spatiotemporal clustering methods, while the temporal properties of multiplet families indicate that short-lived seismic bursts, associated with the swarm, dominate the catalogs. Multiplet families present clustering effects at all time scales, suggesting triggering by aseismic factors, such as fluid diffusion, rather than constant stress loading, in accordance with the spatiotemporal migration patterns of seismicity.

Flatfeet Severity-Level Detection Based on Alignment Measuring.

Flat foot is a postural deformity in which the plantar part of the foot is either completely or partially contacted with the ground. In recent clinical practices, X-ray radiographs have been introduced to detect flat feet because they are more affordable to many clinics than using specialized devices. This research aims to develop an automated model that detects flat foot cases and their severity levels from lateral foot X-ray images by measuring three different foot angles: the Arch Angle, Meary's Angle, and the Calcaneal Inclination Angle. Since these angles are formed by connecting a set of points on the image, Template Matching is used to allocate a set of potential points for each angle, and then a classifier is used to select the points with the highest predicted likelihood to be the correct point. Inspired by literature, this research constructed and compared two models: a Convolutional Neural Network-based model and a Random Forest-based model. These models were trained on 8000 images and tested on 240 unseen cases. As a result, the highest overall accuracy rate was 93.13% achieved by the Random Forest model, with mean values for all foot types (normal foot, mild flat foot, and moderate flat foot) being: 93.38 precision, 92.56 recall, 96.46 specificity, 95.42 accuracy, and 92.90 F-Score. The main conclusions that were deduced from this research are: (1) Using transfer learning (VGG-16) as a feature-extractor-only, in addition to image augmentation, has greatly increased the overall accuracy rate. (2) Relying on three different foot angles shows more accurate estimations than measuring a single foot angle.