Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data.

Machine learning (ML) is increasingly used in cognitive, computational and clinical neuroscience. The reliable and efficient application of ML requires a sound understanding of its subtleties and limitations. Training ML models on datasets with imbalanced classes is a particularly common problem, and it can have severe consequences if not adequately addressed. With the neuroscience ML user in mind, this paper provides a didactic assessment of the class imbalance problem and illustrates its impact through systematic manipulation of data imbalance ratios in (i) simulated data and (ii) brain data recorded with electroencephalography (EEG), magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI). Our results illustrate how the widely-used Accuracy (Acc) metric, which measures the overall proportion of successful predictions, yields misleadingly high performances, as class imbalance increases. Because Acc weights the per-class ratios of correct predictions proportionally to class size, it largely disregards the performance on the minority class. A binary classification model that learns to systematically vote for the majority class will yield an artificially high decoding accuracy that directly reflects the imbalance between the two classes, rather than any genuine generalizable ability to discriminate between them. We show that other evaluation metrics such as the Area Under the Curve (AUC) of the Receiver Operating Characteristic (ROC), and the less common Balanced Accuracy (BAcc) metric - defined as the arithmetic mean between sensitivity and specificity, provide more reliable performance evaluations for imbalanced data. Our findings also highlight the robustness of Random Forest (RF), and the benefits of using stratified cross-validation and hyperprameter optimization to tackle data imbalance. Critically, for neuroscience ML applications that seek to minimize overall classification error, we recommend the routine use of BAcc, which in the specific case of balanced data is equivalent to using standard Acc, and readily extends to multi-class settings. Importantly, we present a list of recommendations for dealing with imbalanced data, as well as open-source code to allow the neuroscience community to replicate and extend our observations and explore alternative approaches to coping with imbalanced data.

Revealing robust neural correlates of conscious and unconscious visual processing: Activation likelihood estimation meta-analyses.

Our ability to consciously perceive information from the visual scene relies on a myriad of intrinsic neural mechanisms. Functional neuroimaging studies have sought to identify the neural correlates of conscious visual processing and to further dissociate from those pertaining to preconscious and unconscious visual processing. However, delineating what core brain regions are involved in eliciting a conscious percept remains a challenge, particularly regarding the role of prefrontal-parietal regions. We performed a systematic search of the literature that yielded a total of 54 functional neuroimaging studies. We conducted two quantitative meta-analyses using activation likelihood estimation to identify reliable patterns of activation engaged by i. conscious (n = 45 studies, comprising 704 participants) and ii. unconscious (n = 16 studies, comprising 262 participants) visual processing during various task performances. Results of the meta-analysis specific to conscious percepts quantitatively revealed reliable activations across a constellation of regions comprising the bilateral inferior frontal junction, intraparietal sulcus, dorsal anterior cingulate, angular gyrus, temporo-occipital cortex and anterior insula. Neurosynth reverse inference revealed conscious visual processing to be intertwined with cognitive terms related to attention, cognitive control and working memory. Results of the meta-analysis on unconscious percepts revealed consistent activations in the lateral occipital complex, intraparietal sulcus and precuneus. These findings highlight the notion that conscious visual processing readily engages higher-level regions including the inferior frontal junction and unconscious processing reliably recruits posterior regions, mainly the lateral occipital complex.

NeuroPycon: An open-source python toolbox for fast multi-modal and reproducible brain connectivity pipelines.

Recent years have witnessed a massive push towards reproducible research in neuroscience. Unfortunately, this endeavor is often challenged by the large diversity of tools used, project-specific custom code and the difficulty to track all user-defined parameters. NeuroPycon is an open-source multi-modal brain data analysis toolkit which provides Python-based template pipelines for advanced multi-processing of MEG, EEG, functional and anatomical MRI data, with a focus on connectivity and graph theoretical analyses. Importantly, it provides shareable parameter files to facilitate replication of all analysis steps. NeuroPycon is based on the NiPype framework which facilitates data analyses by wrapping many commonly-used neuroimaging software tools into a common Python environment. In other words, rather than being a brain imaging software with is own implementation of standard algorithms for brain signal processing, NeuroPycon seamlessly integrates existing packages (coded in python, Matlab or other languages) into a unified python framework. Importantly, thanks to the multi-threaded processing and computational efficiency afforded by NiPype, NeuroPycon provides an easy option for fast parallel processing, which critical when handling large sets of multi-dimensional brain data. Moreover, its flexible design allows users to easily configure analysis pipelines by connecting distinct nodes to each other. Each node can be a Python-wrapped module, a user-defined function or a well-established tool (e.g. MNE-Python for MEG analysis, Radatools for graph theoretical metrics, etc.). Last but not least, the ability to use NeuroPycon parameter files to fully describe any pipeline is an important feature for reproducibility, as they can be shared and used for easy replication by others. The current implementation of NeuroPycon contains two complementary packages: The first, called ephypype, includes pipelines for electrophysiology analysis and a command-line interface for on the fly pipeline creation. Current implementations allow for MEG/EEG data import, pre-processing and cleaning by automatic removal of ocular and cardiac artefacts, in addition to sensor or source-level connectivity analyses. The second package, called graphpype, is designed to investigate functional connectivity via a wide range of graph-theoretical metrics, including modular partitions. The present article describes the philosophy, architecture, and functionalities of the toolkit and provides illustrative examples through interactive notebooks. NeuroPycon is available for download via github (https://github.com/neuropycon) and the two principal packages are documented online (https://neuropycon.github.io/ephypype/index.html, and https://neuropycon.github.io/graphpype/index.html). Future developments include fusion of multi-modal data (eg. MEG and fMRI or intracranial EEG and fMRI). We hope that the release of NeuroPycon will attract many users and new contributors, and facilitate the efforts of our community towards open source tool sharing and development, as well as scientific reproducibility.

Neuronal mechanisms of motion detection underlying blindsight assessed by functional magnetic resonance imaging (fMRI).

Brain imaging offers a valuable tool to observe functional brain plasticity by showing how sensory inputs reshape cortical activations after a visual impairment. Following a unilateral post-chiasmatic lesion affecting the visual cortex, patients may suffer a contralateral visual loss referred to homonymous hemianopia. Nevertheless, these patients preserve the ability to unconsciously detect, localize and discriminate visual stimuli presented in their impaired visual field. To investigate this paradox, known as blindsight, we conducted a study using functional magnetic resonance imaging (fMRI) to evaluate the structural and functional impact of such lesion in a 33-year old patient (ML), who suffers a complete right hemianopia without macular sparing and showing strong evidences of blindsight. We thus performed whole brain and sliced thalamic fMRI scan sequences during an event-related motion detection task. We provided evidence of the neuronal fingerprint of blindsight by acquiring and associating neural correlates, specific structures and functional networks of the midbrain during blindsight performances which may help to better understand this condition. Accurate performance demonstrated the presence of residual vision and the ability to unconsciously perceive motion presented in the blind hemifield, although her reaction time was significantly higher in her blind-field. When the normal hemifield was stimulated, we observed significant contralateral activations in primary and secondary visual areas as well as motion specific areas, such as the supramarginal gyrus and middle temporal area. We also demonstrated sub-thalamic activations within the superior colliculi (SC) and the pulvinar. These results suggest a role of secondary subcortical structures in normal spontaneous motion detection. In a similar way, when the lesioned hemifield was stimulated, we observed contralateral activity in extrastriate areas with no activation of the primary lesioned visual cortex. Moreover, we observed activations within the SC when the blind hemifield was stimulated. However, we observed unexpected ipsilateral activations within the same motion specific areas, as well as bilateral frontal activations. These results highlight the importance of abnormal secondary pathways bypassing the primary visual area (V1) in residual vision. This reorganization in the structure and function of the visual pathways correlates with behavioral changes, thus offering a plausible explanation for the blindsight phenomenon. Our results may potentially impact the development of rehabilitation strategies to target subcortical pathways.

From Cortical Blindness to Conscious Visual Perception: Theories on Neuronal Networks and Visual Training Strategies.

Homonymous hemianopia (HH) is the most common cortical visual impairment leading to blindness in the contralateral hemifield. It is associated with many inconveniences and daily restrictions such as exploration and visual orientation difficulties. However, patients with HH can preserve the remarkable ability to unconsciously perceive visual stimuli presented in their blindfield, a phenomenon known as blindsight. Unfortunately, the nature of this captivating residual ability is still misunderstood and the rehabilitation strategies in terms of visual training have been insufficiently exploited. This article discusses type I and type II blindsight in a neuronal framework of altered global workspace, resulting from inefficient perception, attention and conscious networks. To enhance synchronization and create global availability for residual abilities to reach visual consciousness, rehabilitation tools need to stimulate subcortical extrastriate pathways through V5/MT. Multisensory bottom-up compensation combined with top-down restitution training could target pre-existing and new neuronal mechanisms to recreate a framework for potential functionality.