Data-driven approach to quantify trust in medical devices using Bayesian networks.

Bayesian networks are increasingly used to quantify the uncertainty of subjective and stochastic concepts such as trust. In this article, we propose a data-driven approach to estimate Bayesian parameters in the domain of wearable medical devices. Our approach extracts the probability of a trust factor being in a specific state directly from the devices (e.g. sensor quality). The strength of the relationship between related factors is defined by expert knowledge and incorporated into the model. We use propagation rules from requirements engineering to estimate how much each trust factor contributes to the related intermediate nodes in the network and ultimately compute the trust score. The trust score is a relative measure of trustworthiness when different devices are evaluated in the same test conditions and using the same Bayesian structure. To evaluate our approach, we developed Bayesian networks for the trust quantification of similar wearable devices from two manufacturers under identical test conditions and noise levels. The results demonstrated the learnability and generalizability of our approach.

Evaluation methodology for deep learning imputation models.

There is growing interest in imputing missing data in tabular datasets using deep learning. Existing deep learning-based imputation models have been commonly evaluated using root mean square error (RMSE) as the predictive accuracy metric. In this article, we investigate the limitations of assessing deep learning-based imputation models by conducting a comparative analysis between RMSE and alternative metrics in the statistical literature including qualitative, predictive accuracy, statistical distance, and descriptive statistics. We design a new aggregated metric, called reconstruction loss (RL), to evaluate deep learning-based imputation models. We also develop and evaluate a novel imputation evaluation methodology based on RL. To minimize model and dataset biases, we use a regression imputation model and two different deep learning imputation models: denoising autoencoders and generative adversarial nets. We also use two tabular datasets from different industry sectors: health care and financial. Our results show that the proposed methodology is effective in evaluating multiple properties of the deep learning-based imputation model's reconstruction performance.

From Group-Level Statistics to Single-Subject Prediction: Machine Learning Detection of Concussion in Retired Athletes.

There has been increased effort to understand the neurophysiological effects of concussion aimed to move diagnosis and identification beyond current subjective behavioral assessments that suffer from poor sensitivity. Recent evidence suggests that event-related potentials (ERPs) measured with electroencephalography (EEG) are persistent neurophysiological markers of past concussions. However, as such evidence is limited to group-level analyzes, the extent to which they enable concussion detection at the individual-level is unclear. One promising avenue of research is the use of machine learning to create quantitative predictive models that can detect prior concussions in individuals. In this paper, we translate the recent group-level findings from ERP studies of concussed individuals into a machine learning framework for performing single-subject prediction of past concussion. We found that a combination of statistics of single-subject ERPs and wavelet features yielded a classification accuracy of 81% with a sensitivity of 82% and a specificity of 80%, improving on current practice. Notably, the model was able to detect concussion effects in individuals who sustained their last injury as much as 30 years earlier. However, failure to detect past concussions in a subset of individuals suggests that the clear effects found in group-level analyses may not provide us with a full picture of the neurophysiological effects of concussion.

Machine Learning and Mobile Health Monitoring Platforms: A Case Study on Research and Implementation Challenges.

Machine learning-based patient monitoring systems are generally deployed on remote servers for analyzing heterogeneous data. While recent advances in mobile technology provide new opportunities to deploy such systems directly on mobile devices, the development and deployment challenges are not being extensively studied by the research community. In this paper, we systematically investigate challenges associated with each stage of the development and deployment of a machine learning-based patient monitoring system on a mobile device. For each class of challenges, we provide a number of recommendations that can be used by the researchers, system designers, and developers working on mobile-based predictive and monitoring systems. The results of our investigation show that when developers are dealing with mobile platforms, they must evaluate the predictive systems based on its classification and computational performance. Accordingly, we propose a new machine learning training and deployment methodology specifically tailored for mobile platforms that incorporates metrics beyond traditional classifier performance.