EEG as a potential ground truth for the assessment of cognitive state in software development activities: A multimodal imaging study.

Cognitive human error and recent cognitive taxonomy on human error causes of software defects support the intuitive idea that, for instance, mental overload, attention slips, and working memory overload are important human causes for software bugs. In this paper, we approach the EEG as a reliable surrogate to MRI-based reference of the programmer's cognitive state to be used in situations where heavy imaging techniques are infeasible. The idea is to use EEG biomarkers to validate other less intrusive physiological measures, that can be easily recorded by wearable devices and useful in the assessment of the developer's cognitive state during software development tasks. Herein, our EEG study, with the support of fMRI, presents an extensive and systematic analysis by inspecting metrics and extracting relevant information about the most robust features, best EEG channels and the best hemodynamic time delay in the context of software development tasks. From the EEG-fMRI similarity analysis performed, we found significant correlations between a subset of EEG features and the Insula region of the brain, which has been reported as a region highly related to high cognitive tasks, such as software development tasks. We concluded that despite a clear inter-subject variability of the best EEG features and hemodynamic time delay used, the most robust and predominant EEG features, across all the subjects, are related to the Hjorth parameter Activity and Total Power features, from the EEG channels F4, FC4 and C4, and considering in most of the cases a hemodynamic time delay of 4 seconds used on the hemodynamic response function. These findings should be taken into account in future EEG-fMRI studies in the context of software debugging.

How Reliable Are Ultra-Short-Term HRV Measurements during Cognitively Demanding Tasks?

Ultra-short-term HRV features assess minor autonomous nervous system variations such as variations resulting from cognitive stress peaks during demanding tasks. Several studies compare ultra-short-term and short-term HRV measurements to investigate their reliability. However, existing experiments are conducted in low cognitively demanding environments. In this paper, we propose to evaluate these measurements' reliability under cognitively demanding tasks using a near real-life setting. For this purpose, we selected 31 HRV features, extracted from data collected from 21 programmers performing code comprehension, and compared them across 18 different time frames, ranging from 3 min to 10 s. Statistical significance and correlation tests were performed between the features extracted using the larger window (3 min) and the same features extracted with the other 17 time frames. We paired these analyses with Bland-Altman plots to inspect how the extraction window size affects the HRV features. The main results show 13 features that presented at least 50% correlation when using 60-second windows. The HF and mNN features achieved around 50% correlation using a 30-second window. The 30-second window was the smallest time frame considered to have reliable measurements. Furthermore, the mNN feature proved to be quite robust to the shortening of the time resolution.

Software Bug Detection Causes a Shift From Bottom-Up to Top-Down Effective Connectivity Involving the Insula Within the Error-Monitoring Network.

The neural correlates of software programming skills have been the target of an increasing number of studies in the past few years. Those studies focused on error-monitoring during software code inspection. Others have studied task-related cognitive load as measured by distinct neurophysiological measures. Most studies addressed only syntax errors (shallow level of code monitoring). However, a recent functional MRI (fMRI) study suggested a pivotal role of the insula during error-monitoring when challenging deep-level analysis of code inspection was required. This raised the hypothesis that the insula is causally involved in deep error-monitoring. To confirm this hypothesis, we carried out a new fMRI study where participants performed a deep source-code comprehension task that included error-monitoring to detect bugs in the code. The generality of our paradigm was enhanced by comparison with a variety of tasks related to text reading and bugless source-code understanding. Healthy adult programmers (N = 21) participated in this 3T fMRI experiment. The activation maps evoked by error-related events confirmed significant activations in the insula [p(Bonferroni) < 0.05]. Importantly, a posterior-to-anterior causality shift was observed concerning the role of the insula: in the absence of error, causal directions were mainly bottom-up, whereas, in their presence, the strong causal top-down effects from frontal regions, in particular, the anterior cingulate cortex was observed.

On the accuracy of code complexity metrics: A neuroscience-based guideline for improvement.

Complexity is the key element of software quality. This article investigates the problem of measuring code complexity and discusses the results of a controlled experiment to compare different views and methods to measure code complexity. Participants (27 programmers) were asked to read and (try to) understand a set of programs, while the complexity of such programs is assessed through different methods and perspectives: (a) classic code complexity metrics such as McCabe and Halstead metrics, (b) cognitive complexity metrics based on scored code constructs, (c) cognitive complexity metrics from state-of-the-art tools such as SonarQube, (d) human-centered metrics relying on the direct assessment of programmers' behavioral features (e.g., reading time, and revisits) using eye tracking, and (e) cognitive load/mental effort assessed using electroencephalography (EEG). The human-centered perspective was complemented by the subjective evaluation of participants on the mental effort required to understand the programs using the NASA Task Load Index (TLX). Additionally, the evaluation of the code complexity is measured at both the program level and, whenever possible, at the very low level of code constructs/code regions, to identify the actual code elements and the code context that may trigger a complexity surge in the programmers' perception of code comprehension difficulty. The programmers' cognitive load measured using EEG was used as a reference to evaluate how the different metrics can express the (human) difficulty in comprehending the code. Extensive experimental results show that popular metrics such as V(g) and the complexity metric from SonarSource tools deviate considerably from the programmers' perception of code complexity and often do not show the expected monotonic behavior. The article summarizes the findings in a set of guidelines to improve existing code complexity metrics, particularly state-of-the-art metrics such as cognitive complexity from SonarSource tools.

Can EEG Be Adopted as a Neuroscience Reference for Assessing Software Programmers' Cognitive Load?

An emergent research area in software engineering and software reliability is the use of wearable biosensors to monitor the cognitive state of software developers during software development tasks. The goal is to gather physiologic manifestations that can be linked to error-prone scenarios related to programmers' cognitive states. In this paper we investigate whether electroencephalography (EEG) can be applied to accurately identify programmers' cognitive load associated with the comprehension of code with different complexity levels. Therefore, a controlled experiment involving 26 programmers was carried. We found that features related to Theta, Alpha, and Beta brain waves have the highest discriminative power, allowing the identification of code lines and demanding higher mental effort. The EEG results reveal evidence of mental effort saturation as code complexity increases. Conversely, the classic software complexity metrics do not accurately represent the mental effort involved in code comprehension. Finally, EEG is proposed as a reference, in particular, the combination of EEG with eye tracking information allows for an accurate identification of code lines that correspond to peaks of cognitive load, providing a reference to help in the future evaluation of the space and time accuracy of programmers' cognitive state monitored using wearable devices compatible with software development activities.

Correlated alpha activity with the facial expression processing network in a simultaneous EEG-fMRI experiment.

The relationship between EEG and fMRI data is poorly covered in the literature. Extensive work has been conducted in resting-state and epileptic activity, highlighting a negative correlation between the alpha power band of the EEG and the BOLD activity in the default-mode-network. The identification of an appropriate task-specific relationship between fMRI and EEG data for predefined regions-of-interest, would allow the transfer of interventional paradigms (such as BOLD-based neurofeedback sessions) from fMRI to EEG, enhancing its application range by lowering its costs and improving its flexibility. In this study, we present an analysis of the correlation between task-specific alpha band fluctuations and BOLD activity in the facial expressions processing network. We characterized the network ROIs through a stringent localizer and identified two clusters on the scalp (one frontal, one parietal-occipital) with marked alpha fluctuations, related to the task. We then check whether such power variations throughout the time correlate with the BOLD activity in the network. Our results show statistically significant negative correlations between the alpha power in both clusters and for all the ROIs of the network. The correlation levels have still not met the requirements for transferring the protocol to an EEG setup, but they pave the way towards a better understand on how frontal and parietal-occipital alpha relates to the activity of the facial expressions processing network.

Assessment of cardiovascular function from multi-Gaussian fitting of a finger photoplethysmogram.

Monitoring of cardiovascular function on a beat-to-beat basis is fundamental for protecting patients in different settings including emergency medicine and interventional cardiology, but still faces technical challenges and several limitations. In the present study, we propose a new method for the extraction of cardiovascular performance surrogates from analysis of the photoplethysmographic (PPG) signal alone.We propose using a multi-Gaussian (MG) model consisting of five Gaussian functions to decompose the PPG pulses into its main physiological components. From the analysis of these components, we aim to extract estimators of the left ventricular ejection time, blood pressure and vascular tone changes. Using a multi-derivative analysis of the components related with the systolic ejection, we investigate which are the characteristic points that best define the left ventricular ejection time (LVET). Six LVET estimates were compared with the echocardiographic LVET in a database comprising 68 healthy and cardiovascular diseased volunteers. The best LVET estimate achieved a low absolute error (15.41 ± 13.66 ms), and a high correlation (ρ = 0.78) with the echocardiographic reference.To assess the potential use of the temporal and morphological characteristics of the proposed MG model components as surrogates for blood pressure and vascular tone, six parameters have been investigated: the stiffness index (SI), the T1_d and T1_2 (defined as the time span between the MG model forward and reflected waves), the reflection index (RI), the R1_d and the R1_2 (defined as their amplitude ratio). Their association to reference values of blood pressure and total peripheral resistance was investigated in 43 volunteers exhibiting hemodynamic instability. A good correlation was found between the majority of the extracted and reference parameters, with an exception to R1_2 (amplitude ratio between the main forward wave and the first reflection wave), which correlated low with all the reference parameters. The highest correlation ([Formula: see text] = 0.45) was found between T1_2 and the total peripheral resistance index (TPRI); while in the patients that experienced syncope, the highest agreement ([Formula: see text] = 0.57) was found between SI and systolic blood pressure (SBP) and mean blood pressure (MBP).In conclusion, the presented method for the extraction of surrogates of cardiovascular performance might improve patient monitoring and warrants further investigation.

Wearable sensors in syncope management.

Syncope is a common disorder with a lifetime prevalence of about 40%. Implantable cardiac electronic devices, including implantable loop recorders (ILR) and implantable cardioverter-defibrillators (ICD), are well established in syncope management. However, despite the successful use of ILR and ICD, diagnosis and therapy still remain challenging in many patients due to the complex hemodynamic interplay of cardiac and vascular adaptations during impending syncopes. Wearable sensors might overcome some limitations, including misdiagnosis and inappropriate defibrillator shocks, because a variety of physiological measures can now be easily acquired by a single non-invasive device at high signal quality. In neurally-mediated syncope (NMS), which is the most common cause of syncope, advanced signal processing methodologies paved the way to develop devices for early syncope detection. In contrast to the relatively benign NMS, in arrhythmia-related syncopes immediate therapeutical intervention, predominantly by electrical defibrillation, is often mandatory. However, in patients with a transient risk of arrhythmia-related syncope, limitations of ICD therapy might outweigh their potential therapeutic benefits. In this context the wearable cardioverter-defibrillator offers alternative therapeutical options for some high-risk patients. Herein, we review recent evidence demonstrating that wearable sensors might be useful to overcome some limitations of implantable devices in syncope management.

A low-complex multi-channel methodology for noise detection in phonocardiogram signals.

The phonocardiography (PCG) is an important technique for the diagnosis of several heart conditions. However, the PCG signal is highly prone to noise, which can be an obstacle for the detection and interpretation of physiological heart sounds. Thus, the detection and elimination of noise present in PCG signals is crucial for the accurate analysis of heart sounds, especially in p-health environments. Noise can be introduced by various internal factors (e.g., respiration and laughing) and by external conditions (e.g., phone ringing or door closing). To mention also that the noise frequency components are typically overlapped with the PCG spectrum, increasing the complexity of the analysis. The purpose of the present work consists in the detection of noisy periods willfully introduced during the performance of three different sets of tasks. The developed method returns the classification of the signal content, in a window-by-window analysis and can be divided in two distinct phases. The first step consists in the search for a noise free window using a feature obtained from the PCG time-domain. In the second step, the noise free window is compared with the remaining signal. The classification between clean and contaminated PCG is performed using two features from the frequency domain. The algorithm was able to discriminate clean from contamined PCG sections with an average sensitivity and specificity of 95.59% and 92.68%, respectively.

Detection of hemodynamic adaptations during impending syncope: implementation of a robust algorithm based on pulse arrival time measurements only.

Syncopes are a major public health concern since they can cause severe injuries e.g. by associated falls. We previously demonstrated the feasibility of syncope prediction based on the pulse arrival time (PAT) analysis. Importantly, algorithms for early detection of impending syncope need to be robust against measurement noise, in particular photoplethysmography (PPG) artifacts, causing false detection. We introduce in this work an algorithm concept to deal with artifacts as well as to detect the onset of syncope based on tracking of relative PAT changes only. Our method has been shown useful to improve detection performance for measurements during impending syncope in patients undergoing head-up tilt table testing which might improve syncope prediction.

Characterization of surrogate parameters for blood pressure regulation in neurally-mediated syncope.

Neurally Mediated Syncope (NMS) is often cited as the most common cause of syncope. It can lead to severe consequences such as injuries, high rates of hospitalization and reduced quality of life, especially in elderly populations. Therefore, information about the syncope triggers and reflex mechanisms would be of a great value in the development of a cost-effective p-health system for the prediction of syncope episodes, by enhancing patients' quality of life and reducing the incidence of syncope related disorders/conditions. In the present paper we study the characterization of syncope reflex mechanisms and blood pressure changes from the analysis of several non-invasive modalities (ECG, ICG and PPG). Several parameters were extracted in order to characterize the chronotropic, inotropic and vascular tone changes. Thus, we evaluate the ability of parameters such as Heart Rate (HR), Pre-Ejection Period (PEP) and Left Ventricular Ejection Time (LVET) to characterize the physiological mechanisms behind the development of reflex syncope and their potential syncope prediction capability. The significant parameter changes (e.g. HR from 12.9% to -12.4%, PEP from 14.9% to -3.8% and LVET from -14.4% to 12.3%) found in the present work suggest the feasibility of these surrogates to characterize the blood pressure regulation mechanisms during impending syncope.

Pulse Arrival Time as surrogate for systolic blood pressure changes during impending neurally mediated syncope.

Blood pressure regulation failures cause neurally mediated syncope often resulting in a fall. A warning device might help to make patients aware of an impending critical event or even trigger the patient to perform countermeasures such as lying down or isometric exercises. We previously demonstrated that the Pulse Arrival Time (PAT) methodology is a potential approach to enable early detection of impending faints. The aim of the present study was to evaluate whether PAT can be used as an easy to measure beat-to-beat surrogate for systolic blood pressure (SBP) changes during a passive standing exercise (head-up tilt table testing (HUTT)). A significant PAT increase of more than 10 % was accompanied with a critical SBP decrease in syncope patients. Although PAT is in general not considered as a good measure of absolute blood pressure we found strong correlations (R>0.89, P<0.01) of SBP and PAT after PAT began to increase. Therefore, our data suggest that the pulse arrival time is useful to monitor blood pressure changes in patients with neurally mediated syncope. This might open up new avenues to prevent falls in these patients.