Constructing transferable and interpretable machine learning models for black carbon concentrations.

Black carbon (BC) has received increasing attention from researchers due to its adverse health effects. However, in-situ BC measurements are often not included as a regulated variable in air quality monitoring networks. Machine learning (ML) models have been studied extensively to serve as virtual sensors to complement the reference instruments. This study evaluates and compares three white-box (WB) and four black-box (BB) ML models to estimate BC concentrations, with the focus to show their transferability and interpretability. We train the models with the long-term air pollutant and weather measurements in Barcelona urban background site, and test them in other European urban and traffic sites. Despite the difference in geographical locations and measurement sites, BC correlates the strongest with particle number concentration of accumulation mode (PNacc, r = 0.73-0.85) and nitrogen dioxide (NO2, r = 0.68-0.85) and the weakest with meteorological parameters. Due to its similarity of correlation behaviour, the ML models trained in Barcelona performs prominently at the traffic site in Helsinki (R2 = 0.80-0.86; mean absolute error MAE = 3.90-4.73 %) and at the urban background site in Dresden (R2 = 0.79-0.84; MAE = 4.23-4.82 %). WB models appear to explain less variability of BC than BB models, long short-term memory (LSTM) model of which outperforms the rest of the models. In terms of interpretability, we adopt several methods for individual model to quantify and normalize the relative importance of each input feature. The overall static relative importance commonly used for WB models demonstrate varying results from the dynamic values utilized to show local contribution used for BB models. PNacc and NO2 on average have the strongest absolute static contribution; however, they simultaneously impact the estimation positively and negatively at different sites. This comprehensive analysis demonstrates that the possibility of these interpretable air pollutant ML models to be transfered across space and time.

Enabling Artificial Intelligent Virtual Sensors in an IoT Environment.

The demands for a large number of sensors increase as the proliferation of Internet of Things (IoT) and smart cities applications are continuing at a rapid pace. This also increases the cost of the infrastructure and the installation and maintenance overhead and creates significant performance degradation in the end-to-end communication, monitoring, and orchestration of the various connected devices. In order to solve the problem of increasing sensor demands, this paper suggests replacing physical sensors with machine learning (ML) models. These software-based artificial intelligence models are called virtual sensors. Extensive research and simulation comparisons between fourteen ML models provide a solid ground decision when it comes to the selection of the most accurate model to replace physical sensors, such as temperature and humidity sensors. In this problem at hand, the virtual and physical sensors are designed to be scattered in a smart home, while being connected and run on the same IoT platform. Thus, this paper also introduces a custom lightweight IoT platform that runs on a Raspberry Pi equipped with physical temperature and humidity sensors, which may also execute the virtual sensors. The evaluation results of the devised virtual sensors in a smart home scenario are promising and corroborate the applicability of the proposed methodology.

Virtual MEG sensors based on beamformer and independent component analysis can reconstruct epileptic activity as measured on simultaneous intracerebral recordings.

The prevailing gold standard for presurgical determination of epileptogenic brain networks is intracerebral EEG, a potent yet invasive approach. Magnetoencephalography (MEG) is a state-of-the art non-invasive method for investigating epileptiform discharges. However, it is not clear at what level the precision offered by MEG can reach that of SEEG. Here, we present a strategy for non-invasively retrieving the constituents of the interictal network, with high spatial and temporal precision. Our method is based on MEG and a combination of spatial filtering and independent component analysis (ICA). We validated this approach in twelve patients with drug-resistant focal epilepsy, thanks to the unprecedented ground truth provided by simultaneous recordings of MEG and SEEG. A minimum variance adaptive beamformer estimated the source time series and ICA was used to further decompose these time series into network constituents (MEG-ICs), each having a time series (virtual electrode) and a topography (spatial distribution of amplitudes in the brain). We show that MEG has a considerable sensitivity of 0.80 and 0.84 and a specificity of 0.93 and 0.91 for reconstructing deep and superficial sources, respectively, when compared to the ground truth (SEEG). For each epileptic MEG-IC (n = 131), we found at least one significantly correlating SEEG contact close to zero lag after correcting for multiple comparisons. All the patients except one had at least one epileptic component that was highly correlated (Spearman rho>0.3) with that of SEEG traces. MEG-ICs correlated well with SEEG traces. The strength of correlation coefficients did not depend on the depth of the SEEG contacts or the clinical outcome of the patient. A significant proportion of the MEG-ICs (n = 83/131) were localized in proximity with their maximally correlating SEEG, within a mean distance of 20±12.18mm. Our research is the first to validate the MEG-retrieved beamformer IC sources against SEEG-derived ground truth in a simultaneous MEG-SEEG framework. Observations from the present study suggest that non-invasive MEG source components may potentially provide additional information, comparable to SEEG in a number of instances.

Virtual implantation using conventional scalp EEG delineates seizure onset and predicts surgical outcome in children with epilepsy.

OBJECTIVE: Delineation of the seizure onset zone (SOZ) is required in children with drug resistant epilepsy (DRE) undergoing neurosurgery. Intracranial EEG (icEEG) serves as gold standard but has limitations. Here, we examine the utility of virtual implantation with electrical source imaging (ESI) on ictal scalp EEG for mapping the SOZ and predict surgical outcome. METHODS: We retrospectively analyzed EEG data from 35 children with DRE who underwent surgery and dichotomized into seizure-free (SF) and non-seizure-free (NSF). We estimated virtual sensors (VSs) at brain locations that matched icEEG implantation and compared ictal patterns at VSs vs icEEG. We calculated the agreement between VSs SOZ and clinically defined SOZ and built receiver operating characteristic (ROC) curves to test whether it predicted outcome. RESULTS: Twenty-one patients were SF after surgery. Moderate agreement between virtual and icEEG patterns was observed (kappa = 0.45, p < 0.001). Virtual SOZ agreement with clinically defined SOZ was higher in SF vs NSF patients (66.6% vs 41.6%, p = 0.01). Anatomical concordance of virtual SOZ with clinically defined SOZ predicted outcome (AUC = 0.73; 95% CI: 0.57-0.89; sensitivity = 66.7%; specificity = 78.6%; accuracy = 71.4%). CONCLUSIONS: Virtual implantation on ictal scalp EEG can approximate the SOZ and predict outcome. SIGNIFICANCE: SOZ mapping with VSs may contribute to tailoring icEEG implantation and predict outcome.

A Discrete-Time Extended Kalman Filter Approach Tailored for Multibody Models: State-Input Estimation.

Model-based force estimation is an emerging methodology in the mechatronic community given the possibility to exploit physically inspired high-fidelity models in tandem with ready-to-use cheap sensors. In this work, an inverse input load identification methodology is presented combining high-fidelity multibody models with a Kalman filter-based estimator and providing the means for an accurate and computationally efficient state-input estimation strategy. A particular challenge addressed in this work is the handling of the redundant state-description encountered in common multibody model descriptions. A novel linearization framework is proposed on the time-discretized equations in order to extract the required system model matrices for the Kalman filter. The presented framework is experimentally validated on a slider-crank mechanism. The nonlinear kinematics and dynamics are well represented through a rigid multibody model with lumped flexibilities to account for localized interaction phenomena among bodies. The proposed methodology is validated estimating the input torque delivered by a driver electro-motor together with the system states and comparing the experimental data with the estimated quantities. The results show the stability and accuracy of the estimation framework by only employing the angular motor velocity, measured by the motor encoder sensor and available in most of the commercial electro-motors.

BLAINDER-A Blender AI Add-On for Generation of Semantically Labeled Depth-Sensing Data.

Common Machine-Learning (ML) approaches for scene classification require a large amount of training data. However, for classification of depth sensor data, in contrast to image data, relatively few databases are publicly available and manual generation of semantically labeled 3D point clouds is an even more time-consuming task. To simplify the training data generation process for a wide range of domains, we have developed the BLAINDER add-on package for the open-source 3D modeling software Blender, which enables a largely automated generation of semantically annotated point-cloud data in virtual 3D environments. In this paper, we focus on classical depth-sensing techniques Light Detection and Ranging (LiDAR) and Sound Navigation and Ranging (Sonar). Within the BLAINDER add-on, different depth sensors can be loaded from presets, customized sensors can be implemented and different environmental conditions (e.g., influence of rain, dust) can be simulated. The semantically labeled data can be exported to various 2D and 3D formats and are thus optimized for different ML applications and visualizations. In addition, semantically labeled images can be exported using the rendering functionalities of Blender.

Velocity estimation of flexible dynamics systems using virtual sensing.

In this paper, multiple virtual sensors derived from the equations of motion of a flexible dynamics system, are utilized to form a novel virtual sensing system able to estimate the angular and translation velocities. Measuring only the elastic displacement of the flexible body by a strain gauge, angular velocities are estimated by the first virtual sensor composed of a kinetics equation related to the flexibility, and the nonlinear Kalman and particle filters. Moreover, for translational velocities to be also estimated, specific forces are measured by accelerometers. The second virtual sensor including the equation of translational motion, is added to the virtual sensing system, leading not only the velocities to be estimated with high performance but the accumulated errors of the sensors to be corrected.

A Fuzzy Virtual Actuator for Automated Guided Vehicles.

In the last decades, virtual sensors have found increasing attention in the research community. Virtual sensors employ mathematical models and different sources of information such as actuator states or sensors, which are already existing in a system, in order to generate virtual measurements. Additionally, in recent years, the concept of virtual actuators has been proposed by leading researchers. Virtual actuators are parts of a fault-tolerant control strategy and aim to accommodate faults and to achieve a safe operation of a faulty plant. This paper describes a novel concept for a fuzzy virtual actuator applied to an automated guided vehicle (AGV). The application of fuzzy logic rules allows integrating expert knowledge or experimental data into the decision making of the virtual actuator. The AGV under consideration disposes of an innovative steering concept, which leads to considerable advantages in terms of maneuverability, but requires an elaborate control system. The application of the virtual actuator allows the accommodation of several possible faults, such as a slippery surface under one of the drive modules of the AGV.

Fault diagnosis of an automated guided vehicle with torque and motion forces estimation: A case study.

The paper is devoted to developing a new fault detection scheme for an Automated Guided Vehicle (AGV) on the basis of so-called virtual sensors (VSs) which provide the information regarding the current status of a vehicle. This set contains the estimates of lateral and longitudinal forces as well as the torque. The paper proposes a novel robust VSs design scheme which yields such estimates taking into account inevitable disturbances/noise and modelling uncertainty without any knowledge about tire models used in the AGV. The obtained estimates are used to generate the residuals and to diagnose the current status of the vehicle. Finally, the paper shows the experimental results concerning the application of the developed methods to fault detection of the self-designed and constructed AGV.

A Novel Chiller Sensors Fault Diagnosis Method Based on Virtual Sensors.

Sensor fault detection and diagnosis (FDD) has great significance for ensuring the energy saving and normal operation of the air conditioning system. Chiller systems serving as an important part of central air conditioning systems are the major energy consumer in commercial and industrial buildings. In order to ensure the normal operation of the chiller system, virtual sensors have been proposed to detect and diagnose sensor faults. However, the performance of virtual sensors could be easily impacted by abnormal data. To solve this problem, virtual sensors combined with the maximal information coefficient (MIC) and a long short-term memory (LSTM) network is proposed for chiller sensor fault diagnosis. Firstly, MIC, which has the ability to quantify the degree of relevance in a data set, is applied to examine all potentially interesting relationships between sensors. Subsequently, sensors with high correlation are divided into several groups by the grouping thresholds. Two virtual sensors, which are constructed in each group by LSTM with different input sensors and corresponding to the same physical sensor, could have the ability to predict the value of physical sensors. High correlation sensors in each group improve the fitting effect of virtual sensors. Finally, sensor faults can be diagnosed by the absolute deviation which is generated by comparing the virtual sensors' output with the actual value measured from the air-cooled chiller. The performance of the proposed method is evaluated by using a real data set. Experimental results indicate that virtual sensors can be well constructed and the proposed method achieves a significant performance along with a low false alarm rate.

Virtual Sensors for Optimal Integration of Human Activity Data.

Sensors are becoming more and more ubiquitous as their price and availability continue to improve, and as they are the source of information for many important tasks. However, the use of sensors has to deal with noise and failures. The lack of reliability in the sensors has led to many forms of redundancy, but simple solutions are not always the best, and the precise way in which several sensors are combined has a big impact on the overall result. In this paper, we discuss how to deal with the combination of information coming from different sensors, acting thus as "virtual sensors", in the context of human activity recognition, in a systematic way, aiming for optimality. To achieve this goal, we construct meta-datasets containing the "signatures" of individual datasets, and apply machine-learning methods in order to distinguish when each possible combination method could be actually the best. We present specific results based on experimentation, supporting our claims of optimality.

Collaborative Sensing with Interactive Learning using Dynamic Intelligent Virtual Sensors.

Although the availability of sensor data is becoming prevalent across many domains, it still remains a challenge to make sense of the sensor data in an efficient and effective manner in order to provide users with relevant services. The concept of virtual sensors provides a step towards this goal, however they are often used to denote homogeneous types of data, generally retrieved from a predetermined group of sensors. The DIVS (Dynamic Intelligent Virtual Sensors) concept was introduced in previous work to extend and generalize the notion of a virtual sensor to a dynamic setting with heterogenous sensors. This paper introduces a refined version of the DIVS concept by integrating an interactive machine learning mechanism, which enables the system to take input from both the user and the physical world. The paper empirically validates some of the properties of the DIVS concept. In particular, we are concerned with the distribution of different budget allocations for labelled data, as well as proactive labelling user strategies. We report on results suggesting that a relatively good accuracy can be achieved despite a limited budget in an environment with dynamic sensor availability, while proactive labeling ensures further improvements in performance.

A Method for Fault Detection and Diagnostics in Ventilation Units Using Virtual Sensors.

Buildings represent a significant portion of global energy consumption. Ventilation units are complex components, often customized for the specific building, responsible for a large part of energy consumption. Their faults impact buildings' energy efficiency and occupancy comfort. In order to ensure their correct operation, proper fault detection and diagnostics methods must be applied. Hardware redundancy, an effective approach to detect faults, leads to increased costs and space requirements. We propose exploiting physical relations inside ventilation units to create virtual sensors from other sensors' readings, introducing redundancy in the system. We use two different measures to detect when a virtual sensor deviates from the physical one: coefficient of determination for linear models, and acceptable range. We tested our method on a real building at the University of Southern Denmark, developing three virtual sensors: temperature, airflow, and fan speed. We employed linear regression models, statistical models, and non-linear regression models. All models detected an anomalous strong oscillation in the temperature sensors. Readings fell outside the acceptable range and the coefficient of determination dropped. Our method showed promising results by introducing redundancy in the system, which can benefit several applications, such as fault detection and diagnostics and fault-tolerant control. Future work will be necessary to discover thresholds and set up automatic fault detection and diagnostics.

VLSI Design of Trusted Virtual Sensors.

This work presents a Very Large Scale Integration (VLSI) design of trusted virtual sensors providing a minimum unitary cost and very good figures of size, speed and power consumption. The sensed variable is estimated by a virtual sensor based on a configurable and programmable PieceWise-Affine hyper-Rectangular (PWAR) model. An algorithm is presented to find the best values of the programmable parameters given a set of (empirical or simulated) input-output data. The VLSI design of the trusted virtual sensor uses the fast authenticated encryption algorithm, AEGIS, to ensure the integrity of the provided virtual measurement and to encrypt it, and a Physical Unclonable Function (PUF) based on a Static Random Access Memory (SRAM) to ensure the integrity of the sensor itself. Implementation results of a prototype designed in a 90-nm Complementary Metal Oxide Semiconductor (CMOS) technology show that the active silicon area of the trusted virtual sensor is 0.86 mm 2 and its power consumption when trusted sensing at 50 MHz is 7.12 mW. The maximum operation frequency is 85 MHz, which allows response times lower than 0.25 µ s. As application example, the designed prototype was programmed to estimate the yaw rate in a vehicle, obtaining root mean square errors lower than 1.1%. Experimental results of the employed PUF show the robustness of the trusted sensing against aging and variations of the operation conditions, namely, temperature and power supply voltage (final value as well as ramp-up time).

Automatic detection and visualisation of MEG ripple oscillations in epilepsy.

High frequency oscillations (HFOs, 80-500 Hz) in invasive EEG are a biomarker for the epileptic focus. Ripples (80-250 Hz) have also been identified in non-invasive MEG, yet detection is impeded by noise, their low occurrence rates, and the workload of visual analysis. We propose a method that identifies ripples in MEG through noise reduction, beamforming and automatic detection with minimal user effort. We analysed 15 min of presurgical resting-state interictal MEG data of 25 patients with epilepsy. The MEG signal-to-noise was improved by using a cross-validation signal space separation method, and by calculating ~ 2400 beamformer-based virtual sensors in the grey matter. Ripples in these sensors were automatically detected by an algorithm optimized for MEG. A small subset of the identified ripples was visually checked. Ripple locations were compared with MEG spike dipole locations and the resection area if available. Running the automatic detection algorithm resulted in on average 905 ripples per patient, of which on average 148 ripples were visually reviewed. Reviewing took approximately 5 min per patient, and identified ripples in 16 out of 25 patients. In 14 patients the ripple locations showed good or moderate concordance with the MEG spikes. For six out of eight patients who had surgery, the ripple locations showed concordance with the resection area: 4/5 with good outcome and 2/3 with poor outcome. Automatic ripple detection in beamformer-based virtual sensors is a feasible non-invasive tool for the identification of ripples in MEG. Our method requires minimal user effort and is easily applicable in a clinical setting.

Development of a Virtual Collision Sensor for Industrial Robots.

Collision detection is a fundamental issue for the safety of a robotic cell. While several common methods require specific sensors or the knowledge of the robot dynamic model, the proposed solution is constituted by a virtual collision sensor for industrial manipulators, which requires as inputs only the motor currents measured by the standard sensors that equip a manipulator and the estimated currents provided by an internal dynamic model of the robot (i.e., the one used inside its controller), whose structure, parameters and accuracy are not known. The collision detection is achieved by comparing the absolute value of the current residue with a time-varying, positive-valued threshold function, including an estimate of the model error and a bias term, corresponding to the minimum collision torque to be detected. The value of such a term, defining the sensor sensitivity, can be simply imposed as constant, or automatically customized for a specific robotic application through a learning phase and a subsequent adaptation process, to achieve a more robust and faster collision detection, as well as the avoidance of any false collision warnings, even in case of slow variations of the robot behavior. Experimental results are provided to confirm the validity of the proposed solution, which is already adopted in some industrial scenarios.

Evaluation of Sensor Configurations for Robotic Surgical Instruments.

Designing surgical instruments for robotic-assisted minimally-invasive surgery (RAMIS) is challenging due to constraints on the number and type of sensors imposed by considerations such as space or the need for sterilization. A new method for evaluating the usability of virtual teleoperated surgical instruments based on virtual sensors is presented. This method uses virtual prototyping of the surgical instrument with a dual physical interaction, which allows testing of different sensor configurations in a real environment. Moreover, the proposed approach has been applied to the evaluation of prototypes of a two-finger grasper for lump detection by remote pinching. In this example, the usability of a set of five different sensor configurations, with a different number of force sensors, is evaluated in terms of quantitative and qualitative measures in clinical experiments with 23 volunteers. As a result, the smallest number of force sensors needed in the surgical instrument that ensures the usability of the device can be determined. The details of the experimental setup are also included.

Virtual sensor for failure detection, identification and recovery in the transition phase of a morphing aircraft.

The Helicopter Adaptive Aircraft (HADA) is a morphing aircraft which is able to take-off as a helicopter and, when in forward flight, unfold the wings that are hidden under the fuselage, and transfer the power from the main rotor to a propeller, thus morphing from a helicopter to an airplane. In this process, the reliable folding and unfolding of the wings is critical, since a failure may determine the ability to perform a mission, and may even be catastrophic. This paper proposes a virtual sensor based Fault Detection, Identification and Recovery (FDIR) system to increase the reliability of the HADA aircraft. The virtual sensor is able to capture the nonlinear interaction between the folding/unfolding wings aerodynamics and the HADA airframe using the navigation sensor measurements. The proposed FDIR system has been validated using a simulation model of the HADA aircraft, which includes real phenomena as sensor noise and sampling characteristics and turbulence and wind perturbations.