Ultra-Low-Power Sensor Nodes for Real-Time Synchronous and High-Accuracy Timing Wireless Data Acquisition.

This paper presents an energy-efficient and high-accuracy sampling synchronization approach for real-time synchronous data acquisition in wireless sensor networks (saWSNs). A proprietary protocol based on time-division multiple access (TDMA) and deep energy-efficient coding in sensor firmware is proposed. A real saWSN model based on 2.4 GHz nRF52832 system-on-chip (SoC) sensors was designed and experimentally tested. The obtained results confirmed significant improvements in data synchronization accuracy (even by several times) and power consumption (even by a hundred times) compared to other recently reported studies. The results demonstrated a sampling synchronization accuracy of 0.8 µs and ultra-low power consumption of 15 µW per 1 kb/s throughput for data. The protocol was well designed, stable, and importantly, lightweight. The complexity and computational performance of the proposed scheme were small. The CPU load for the proposed solution was <2% for a sampling event handler below 200 Hz. Furthermore, the transmission reliability was high with a packet error rate (PER) not exceeding 0.18% for TXPWR ≥ -4 dBm and 0.03% for TXPWR ≥ 3 dBm. The efficiency of the proposed protocol was compared with other solutions presented in the manuscript. While the number of new proposals is large, the technical advantage of our solution is significant.

A Virtual Testing Framework for Real-Time Validation of Automotive Software Systems Based on Hardware in the Loop and Fault Injection.

To validate safety-related automotive software systems, experimental tests are conducted at different stages of the V-model, which are referred as "X-in-the-loop (XIL) methods". However, these methods have significant drawbacks in terms of cost, time, effort and effectiveness. In this study, based on hardware-in-the-loop (HIL) simulation and real-time fault injection (FI), a novel testing framework has been developed to validate system performance under critical abnormal situations during the development process. The developed framework provides an approach for the real-time analysis of system behavior under single and simultaneous sensor/actuator-related faults during virtual test drives without modeling effort for fault mode simulations. Unlike traditional methods, the faults are injected programmatically and the system architecture is ensured without modification to meet the real-time constraints. Moreover, a virtual environment is modeled with various environmental conditions, such as weather, traffic and roads. The validation results demonstrate the effectiveness of the proposed framework in a variety of driving scenarios. The evaluation results demonstrate that the system behavior via HIL simulation has a high accuracy compared to the non-real-time simulation method with an average relative error of 2.52. The comparative study with the state-of-the-art methods indicates that the proposed approach exhibits superior accuracy and capability. This, in turn, provides a safe, reliable and realistic environment for the real-time validation of complex automotive systems at a low cost, with minimal time and effort.

Contraction assessment of abdominal muscles using automated segmentation designed for wearable ultrasound applications.

PURPOSE: Wearable ultrasound devices can be used to continuously monitor muscle activity. One possible application is to provide real-time feedback during physiotherapy, to show a patient whether an exercise is performed correctly. Algorithms which automatically analyze the data can be of importance to overcome the need for manual assessment and annotations and speed up evaluations especially when considering real-time video sequences. They even could be used to present feedback in an understandable manner to patients in a home-use scenario. The following work investigates three deep learning based segmentation approaches for abdominal muscles in ultrasound videos during a segmental stabilizing exercise. The segmentations are used to automatically classify the contraction state of the muscles. METHODS: The first approach employs a simple 2D network, while the remaining two integrate the time information from the videos either via additional tracking or directly into the network architecture. The contraction state is determined by comparing measures such as muscle thickness and center of mass between rest and exercise. A retrospective analysis is conducted but also a real-time scenario is simulated, where classification is performed during exercise. RESULTS: Using the proposed segmentation algorithms, 71% of the muscle states are classified correctly in the retrospective analysis in comparison to 90% accuracy with manual reference segmentation. For the real-time approach the majority of given feedback during exercise is correct when the retrospective analysis had come to the correct result, too. CONCLUSION: Both retrospective and real-time analysis prove to be feasible. While no substantial differences between the algorithms were observed regarding classification, the networks incorporating the time information showed temporally more consistent segmentations. Limitations of the approaches as well as reasons for failing cases in segmentation, classification and real-time assessment are discussed and requirements regarding image quality and hardware design are derived.

Implementation of Lightweight Convolutional Neural Networks with an Early Exit Mechanism Utilizing 40 nm CMOS Process for Fire Detection in Unmanned Aerial Vehicles.

The advancement of unmanned aerial vehicles (UAVs) enables early detection of numerous disasters. Efforts have been made to automate the monitoring of data from UAVs, with machine learning methods recently attracting significant interest. These solutions often face challenges with high computational costs and energy usage. Conventionally, data from UAVs are processed using cloud computing, where they are sent to the cloud for analysis. However, this method might not meet the real-time needs of disaster relief scenarios. In contrast, edge computing provides real-time processing at the site but still struggles with computational and energy efficiency issues. To overcome these obstacles and enhance resource utilization, this paper presents a convolutional neural network (CNN) model with an early exit mechanism designed for fire detection in UAVs. This model is implemented using TSMC 40 nm CMOS technology, which aids in hardware acceleration. Notably, the neural network has a modest parameter count of 11.2 k. In the hardware computation part, the CNN circuit completes fire detection in approximately 230,000 cycles. Power-gating techniques are also used to turn off inactive memory, contributing to reduced power consumption. The experimental results show that this neural network reaches a maximum accuracy of 81.49% in the hardware implementation stage. After automatic layout and routing, the CNN hardware accelerator can operate at 300 MHz, consuming 117 mW of power.

First-in-human real-time AI-assisted instrument deocclusion during augmented reality robotic surgery.

The integration of Augmented Reality (AR) into daily surgical practice is withheld by the correct registration of pre-operative data. This includes intelligent 3D model superposition whilst simultaneously handling real and virtual occlusions caused by the AR overlay. Occlusions can negatively impact surgical safety and as such deteriorate rather than improve surgical care. Robotic surgery is particularly suited to tackle these integration challenges in a stepwise approach as the robotic console allows for different inputs to be displayed in parallel to the surgeon. Nevertheless, real-time de-occlusion requires extensive computational resources which further complicates clinical integration. This work tackles the problem of instrument occlusion and presents, to the authors' best knowledge, the first-in-human on edge deployment of a real-time binary segmentation pipeline during three robot-assisted surgeries: partial nephrectomy, migrated endovascular stent removal, and liver metastasectomy. To this end, a state-of-the-art real-time segmentation and 3D model pipeline was implemented and presented to the surgeon during live surgery. The pipeline allows real-time binary segmentation of 37 non-organic surgical items, which are never occluded during AR. The application features real-time manual 3D model manipulation for correct soft tissue alignment. The proposed pipeline can contribute towards surgical safety, ergonomics, and acceptance of AR in minimally invasive surgery.

A Novel Bio-Inspired Bat Node Scheduling Algorithm for Dependable Safety-Critical Wireless Sensor Network Systems.

The multi-objective optimization (MOO) problem in wireless sensor networks (WSNs) is concerned with optimizing the operation of the WSN across three dimensions: coverage, connectivity, and lifetime. Most works in the literature address only one or two dimensions of this problem at a time, except for the randomized coverage-based scheduling (RCS) algorithm and the clique-based scheduling algorithm. More recently, a Hidden Markov Model (HMM)-based algorithm was proposed that improves on the latter two; however, the question remains open if further improvement is possible as previous algorithms explore solutions in terms of local minima and local maxima, not in terms of the full search space globally. Therefore, the main contribution of this paper is to propose a new scheduling algorithm based on bio-inspired computation (the bat algorithm) to address this limitation. First, the algorithm defines a fitness and objective function over a search space, which returns all possible sleep and wake-up schedules for each node in the WSN. This yields a (scheduling) solution space that is then organized by the Pareto sorting algorithm, whose output coordinates are the distance of each node to the base station and the residual energy of the node. We evaluated our results by comparing the bat and HMM node scheduling algorithms implemented in MATLAB. Our results show that network lifetime has improved by 30%, coverage by 40%, and connectivity by 26.7%. In principle, the obtained solution will be the best scheduling that guarantees the best network lifetime performance as well as the best coverage and connectedness for ensuring the dependability of safety-critical WSNs.

Inter- and intra-session variability of compression strain gauge for the adductor groin squeeze test on soccer athletes.

The importance of hip adductor strength for injury prevention and performance benefits is well documented. The purpose of this study was to establish the intra- and inter-day variability of peak force (PF) of a groin squeeze protocol using a custom-designed compression strain gauge device. Sixteen semi-professional soccer players completed three trials over three separate testing occasions with at least 24-h rest between each session. The main findings were that the compression strain gauge was a reliable device for measuring PF within and between days. All intraclass correlations were higher than 0.80 and coefficients of variations were below 10% across the different sessions and trials. Due to the information gained through the compression strain gauge, the higher sampling frequency utilized, portability, and the relatively affordable price, this device offers an effective alternative for measuring maximal strength for hip adduction.

Implementation and analysis of a parallel kalman filter algorithm for lidar localization based on CUDA technology.

Introduction: Navigation satellite systems can fail to work or work incorrectly in a number of conditions: signal shadowing, electromagnetic interference, atmospheric conditions, and technical problems. All of these factors can significantly affect the localization accuracy of autonomous driving systems. This emphasizes the need for other localization technologies, such as Lidar. Methods: The use of the Kalman filter in combination with Lidar can be very effective in various applications due to the synergy of their capabilities. The Kalman filter can improve the accuracy of lidar measurements by taking into account the noise and inaccuracies present in the measurements. Results: In this paper, we propose a parallel Kalman algorithm in three-dimensional space to speed up the computational speed of Lidar localization. At the same time, the initial localization accuracy of the latter is preserved. A distinctive feature of the proposed approach is that the Kalman localization algorithm itself is parallelized, rather than the process of building a map for navigation. The proposed algorithm allows us to obtain the result 3.8 times faster without compromising the localization accuracy, which was 3% for both cases, making it effective for real-time decision-making. Discussion: The reliability of this result is confirmed by a preliminary theoretical estimate of the acceleration rate based on Ambdahl's law. Accelerating the Kalman filter with CUDA for Lidar localization can be of significant practical value, especially in real-time and in conditions where large amounts of data from Lidar sensors need to be processed.

A Multiplier-Free Convolution Neural Network Hardware Accelerator for Real-Time Bearing Condition Detection of CNC Machinery.

In various industrial domains, machinery plays a pivotal role, with bearing failure standing out as the most prevalent cause of malfunction, contributing to approximately 41% to 44% of all operational breakdowns. To address this issue, this research employs a lightweight neural network, boasting a mere 8.69 K parameters, tailored for implementation on an FPGA (field-programmable gate array). By integrating an incremental network quantization approach and fixed-point operation techniques, substantial memory savings amounting to 63.49% are realized compared to conventional 32-bit floating-point operations. Moreover, when executed on an FPGA, this work facilitates real-time bearing condition detection at an impressive rate of 48,000 samples per second while operating on a minimal power budget of just 342 mW. Remarkably, this system achieves an accuracy level of 95.12%, showcasing its effectiveness in predictive maintenance and the prevention of costly machinery failures.

A Real-Time Vessel Detection and Tracking System Based on LiDAR.

Vessel detection and tracking is of utmost importance to river traffic. Efficient detection and tracking technology offer an effective solution to address challenges related to river traffic safety and congestion. Traditional image-based object detection and tracking algorithms encounter issues such as target ID switching, difficulties in feature extraction, reduced robustness due to occlusion, target overlap, and changes in brightness and contrast. To detect and track vessels more accurately, a vessel detection and tracking algorithm based on the LiDAR point cloud was proposed. For vessel detection, statistical filtering algorithms were integrated into the Euclidean clustering algorithm to mitigate the effect of ripples on vessel detection. Our detection accuracy of vessels improved by 3.3% to 8.3% compared to three conventional algorithms. For vessel tracking, L-shape fitting of detected vessels can improve the efficiency of tracking, and a simple and efficient tracking algorithm is presented. By comparing three traditional tracking algorithms, an improvement in multiple object tracking accuracy (MOTA) and a reduction in ID switch times and number of missed detections were achieved. The results demonstrate that LiDAR point cloud-based vessel detection can significantly enhance the accuracy of vessel detection and tracking.

Weakly Hard Real-Time Model for Control Systems: A Survey.

The concept of weakly hard real-time systems can be used to model real-time systems that may tolerate occasional deadline misses in a bounded and predictable manner. This model applies to many practical applications and is particularly interesting in the context of real-time control systems. In practice, applying hard real-time constraints may be too rigid since a certain amount of deadline misses is acceptable in some applications. In order to maintain system stability, limitations on the amount and distribution of violated deadlines need to be imposed. These limitations can be formally expressed as weakly hard real-time constraints. Current research in the field of weakly hard real-time task scheduling is focused on designing scheduling algorithms that guarantee the fulfillment of constraints, while aiming to maximize the total number of timely completed task instances. This paper provides an extensive literature review of the work related to the weakly hard real-time system model and its link to the field of control systems design. The weakly hard real-time system model and the corresponding scheduling problem are described. Furthermore, an overview of system models derived from the generalized weakly hard real-time system model is provided, with an emphasis on models that apply to real-time control systems. The state-of-the-art algorithms for scheduling tasks with weakly hard real-time constraints are described and compared. Finally, an overview of controller design methods that rely on the weakly hard real-time model is given.

CNN Hardware Accelerator for Real-Time Bearing Fault Diagnosis.

This paper introduces a one-dimensional convolutional neural network (CNN) hardware accelerator. It is crafted to conduct real-time assessments of bearing conditions using economical hardware components, implemented on a field-programmable gate array evaluation platform, negating the necessity to transfer data to a cloud-based server. The adoption of the down-sampling technique augments the visible time span of the signal in an image, thereby enhancing the accuracy of the bearing condition diagnosis. Furthermore, the proposed method of quaternary quantization enhances precision and shrinks the memory demand for the neural network model by an impressive 89%. Provided that the current signal data sampling rate stands at 64 K samples/s, the proposed design can accomplish real-time fault diagnosis at a clock frequency of 100 MHz. Impressively, the response duration of the proposed CNN hardware system is a mere 0.28 s, with the fault diagnosis precision reaching a remarkable 96.37%.

A low-cost edge AI-chip-based system for real-time algae species classification and HAB prediction.

Harmful Algal Blooms (HAB) are damaging to ecosystem functions and pose challenges to environmental and fisheries management. The key to HAB management and understanding the complex algal growth dynamics is the development of robust systems for real-time monitoring of algae populations and species. Previous algae classification studies mainly rely on the combination of an in-situ imaging flow cytometer and an off-site lab-based algae classification model such as Random Forest (RF) for the analysis of high-throughput images. An on-site AI algae monitoring system on top of an edge AI chip embedded with the proposed Algal Morphology Deep Neural Network (AMDNN) model is developed to achieve real-time algae species classification and HAB prediction. Based on a detailed examination of real-world algae images, dataset augmentation is first performed: consisting of orientation, flipping, blurring, and Resizing with Aspect ratio Preserved (RAP). The dataset augmentation is shown to significantly improve classification performance which is superior to that of the competitive RF model. And the attention heatmaps show that for relatively regular-shaped algal species (e.g., Vicicitus), the model weights the color and texture information heavily; while the shape-related features are more important for complex-shaped algae (e.g., Chaetoceros). The AMDNN is tested on a dataset of 11,250 algae images containing the 25 most common HAB classes in Hong Kong subtropical waters with 99.87% test accuracy. Based on the fast and accurate algae classification, the AI-chip-based on-site system is applied to a one-month dataset in February 2020; the predicted trends of total cell counts and targeted HAB species counts are in good agreement with observations. The proposed edge AI algae monitoring system provides a platform for the development of practical HAB early warning systems that can effectively support environmental risk and fisheries management.

Multilevel hysteresis modulation (MHM) for sliding mode controlled dual inverter PV systems.

In this article, proposes an adaptive multiband hysteresis modulation and its characterization based on a robust nonlinear sliding-mode control (RNSMC) strategy of a dual two-level inverter topologies for photovoltaic (PV) system subject to robust and fast-tracking characteristics and external disturbance rejection. Using the RNSMC based novel, vector control schemes are designed and implemented to improve its robustness and against the rejection of external disturbances. The control scheme ensures the active power is stable across the whole operation of the PV systems (standalone and grid-connected). The reachability and stability analysis are obtaining the robustness of the controller. Tsypkin's method is used to realize the switching frequency characterization. The development and implementation approach of the proposed control strategy is thoroughly detailed, and its effectiveness is validated on the simulation and real-time laboratory tests. Moreover, the proposed scheme is superior in keeping the enhanced efficiency, multilevel voltage waveforms, and total harmonic distortion (THD) reduction in current within the necessary standard limitations even under different operating conditions.

A Real-Time Endoscope Motion Tracker.

OBJECTIVE: In colonoscopy, it is desirable to accurately localize the position of the endoscope's distal tip. Current tip localization techniques are not sufficient for recording the position and movement of the tip, nor is its rotation measured. We hypothesize that integration of multiple tracking modalities can effectively record the endoscope's motion in real time and continuously corrects cumulative errors. METHODS: A dual modality tracking method is developed to measure the motion of the endoscope's insertion tube in real time, including insertion length, rotation angle, and their velocities. Optical trackballs were used to measure the endoscope insertion tube's motion and cameras were used to correct cumulative errors. RESULTS: The accuracy of insertion length and rotational angle were measured. For speeds ≤ 10 mm/s, the median and 90th percentile insertion position errors were 0.88 mm and 2.2 mm, respectively. The insertion position error increases with the speed, reaching a maximum of 10 mm for speeds < 40 mm/s. 11° and 21° were the median and 90th percentile rotation angle errors for angular speeds < 40°/s. Cumulative errors are sufficiently reduced by the imaging modality. CONCLUSION: The prototype device can precisely measure an unmodified endoscope's position, rotation, and motion in real time without significant accumulative error. The prototype device is small and compatible with existing commercial endoscopes as an add-on accessory, which could be used for reporting, localizing the lesions in follow up procedures, operational guidance, quality assurance, and training. Clinical and Translational Impact Statement-This preclinical research develops an endoscope tracker that can be integrated into colonoscopy training, automatically record endoscope motion, and be further developed to improve polyp and tumor localization during colonoscopy.

Using Python Modules in Real-Time Plasma Systems for Fusion.

One of the most important applications of sensors is feedback control, in which an algorithm is applied to data that are collected from sensors in order to drive system actuators and achieve the desired outputs of the target plant. One of the most challenging applications of this control is represented by magnetic confinement fusion, in which real-time systems are responsible for the confinement of plasma at a temperature of several million degrees within a toroidal container by means of strong electromagnetic fields. Due to the fast dynamics of the underlying physical phenomena, data that are collected from electromagnetic sensors must be processed in real time. In most applications, real-time systems are implemented in C++; however, Python applications are now becoming more and more widespread, which has raised potential interest in their applicability in real-time systems. In this study, a framework was set up to assess the applicability of Python in real-time systems. For this purpose, a reference operating system configuration was chosen, which was optimized for real time, together with a reference framework for real-time data management. Within this framework, the performance of modules that computed PID control and FFT transforms was compared for C++ and Python implementations, respectively. Despite the initial concerns about Python applicability in real-time systems, it was found that the worst-case execution time (WCET) could also be safely defined for modules that were implemented in Python, thereby confirming that they could be considered for real-time applications.

Optimal priority assignment for real-time systems: a coevolution-based approach.

In real-time systems, priorities assigned to real-time tasks determine the order of task executions, by relying on an underlying task scheduling policy. Assigning optimal priority values to tasks is critical to allow the tasks to complete their executions while maximizing safety margins from their specified deadlines. This enables real-time systems to tolerate unexpected overheads in task executions and still meet their deadlines. In practice, priority assignments result from an interactive process between the development and testing teams. In this article, we propose an automated method that aims to identify the best possible priority assignments in real-time systems, accounting for multiple objectives regarding safety margins and engineering constraints. Our approach is based on a multi-objective, competitive coevolutionary algorithm mimicking the interactive priority assignment process between the development and testing teams. We evaluate our approach by applying it to six industrial systems from different domains and several synthetic systems. The results indicate that our approach significantly outperforms both our baselines, i.e., random search and sequential search, and solutions defined by practitioners. Our approach scales to complex industrial systems as an offline analysis method that attempts to find near-optimal solutions within acceptable time, i.e., less than 16 hours.

GPU-Accelerated PD-IPM for Real-Time Model Predictive Control in Integrated Missile Guidance and Control Systems.

This paper addresses the problem of real-time model predictive control (MPC) in the integrated guidance and control (IGC) of missile systems. When the primal-dual interior point method (PD-IPM), which is a convex optimization method, is used as an optimization solution for the MPC, the real-time performance of PD-IPM degenerates due to the elevated computation time in checking the Karush-Kuhn-Tucker (KKT) conditions in PD-IPM. This paper proposes a graphics processing unit (GPU)-based method to parallelize and accelerate PD-IPM for real-time MPC. The real-time performance of the proposed method was tested and analyzed on a widely-used embedded system. The comparison results with the conventional PD-IPM and other methods showed that the proposed method improved the real-time performance by reducing the computation time significantly.

Formal Verification of Real-Time Autonomous Robots: An Interdisciplinary Approach.

Due to the severe consequences of their possible failure, robotic systems must be rigorously verified as to guarantee that their behavior is correct and safe. Such verification, carried out on a model, needs to cover various behavioral properties (e.g., safety and liveness), but also, given the timing constraints of robotic missions, real-time properties (e.g., schedulability and bounded response). In addition, in order to obtain valid and useful verification results, the model must faithfully represent the underlying robotic system and should therefore take into account all possible behaviors of the robotic software under the actual hardware and OS constraints (e.g., the scheduling policy and the number of cores). These requirements put the rigorous verification of robotic systems at the intersection of at least three communities: the robotic community, the formal methods community, and the real-time systems community. Verifying robotic systems is thus a complex, interdisciplinary task that involves a number of disciplines/techniques (e.g., model checking, schedulability analysis, component-based design) and faces a number of challenges (e.g., formalization, automation, scalability). For instance, the use of formal verification (formal methods community) is hindered by the state-space explosion problem, whereas schedulability analysis (real-time systems) is not suitable for behavioral properties. Moreover, current real-time implementations of robotic software are limited in terms of predictability and efficiency, leading to, e.g., unnecessary latencies. This is flagrant, in particular, at the level of locking protocols in robotic software. Such situation may benefit from major theoretical and practical findings of the real-time systems community. In this paper, we propose an interdisciplinary approach that, by joining forces of the different communities, provides a scalable and unified means to efficiently implement and rigorously verify real-time robots. First, we propose a scalable two-step verification solution that combines formal methods and schedulability analysis to verify both behavioral and real-time properties. Second, we devise a new multi-resource locking mechanism that is efficient, predictable, and suitable for real-time robots and show how it improves the latter's real-time behavior. In both cases, we show, using a real drone example, how our approach compares favorably to that in the literature. This paper is a major extension of the RTCSA 2020 publication "A Two-Step Hybrid Approach for Verifying Real-Time Robotic Systems."

Real-Time System for Daily Modal Split Estimation and OD Matrices Generation Using IoT Data: A Case Study of Tartu City.

In recent years, we have witnessed the emergence of the implementation and integration of significant working solutions in transportation, especially within the smart city concept. A lot of cities in Europe and around the world support this initiative of making their cities smarter for enhanced mobility and a sustainable environment. In this paper, we present a case study of Tartu city, where we developed and designed a daily real-time system for extracting and performing a modal split analysis. Our web-based platform relied on an optimization approach for calibrating our simulation in order to perform the analysis with the use of real data streams from IoT devices installed around the city. The results obtained from our system demonstrated acceptable performance versus the quality of the available data source. In addition, our platform provides downloadable OD matrices for each mode of mobility for the community.