A method for synchronized use of EEG and eye tracking in fully immersive VR.

This study explores the synchronization of multimodal physiological data streams, in particular, the integration of electroencephalography (EEG) with a virtual reality (VR) headset featuring eye-tracking capabilities. A potential use case for the synchronized data streams is demonstrated by implementing a hybrid steady-state visually evoked potential (SSVEP) based brain-computer interface (BCI) speller within a fully immersive VR environment. The hardware latency analysis reveals an average offset of 36 ms between EEG and eye-tracking data streams and a mean jitter of 5.76 ms. The study further presents a proof of concept brain-computer interface (BCI) speller in VR, showcasing its potential for real-world applications. The findings highlight the feasibility of combining commercial EEG and VR technologies for neuroscientific research and open new avenues for studying brain activity in ecologically valid VR environments. Future research could focus on refining the synchronization methods and exploring applications in various contexts, such as learning and social interactions.

CLET: Computation of Latencies in Event-related potential Triggers using photodiode on virtual reality apparatuses.

To investigate event-related activity in human brain dynamics as measured with EEG, triggers must be incorporated to indicate the onset of events in the experimental protocol. Such triggers allow for the extraction of ERP, i.e., systematic electrophysiological responses to internal or external stimuli that must be extracted from the ongoing oscillatory activity by averaging several trials containing similar events. Due to the technical setup with separate hardware sending and recording triggers, the recorded data commonly involves latency differences between the transmitted and received triggers. The computation of these latencies is critical for shifting the epochs with respect to the triggers sent. Otherwise, timing differences can lead to a misinterpretation of the resulting ERPs. This study presents a methodical approach for the CLET using a photodiode on a non-immersive VR (i.e., LED screen) and an immersive VR (i.e., HMD). Two sets of algorithms are proposed to analyze the photodiode data. The experiment designed for this study involved the synchronization of EEG, EMG, PPG, photodiode sensors, and ten 3D MoCap cameras with a VR presentation platform (Unity). The average latency computed for LED screen data for a set of white and black stimuli was 121.98 ± 8.71 ms and 121.66 ± 8.80 ms, respectively. In contrast, the average latency computed for HMD data for the white and black stimuli sets was 82.80 ± 7.63 ms and 69.82 ± 5.52 ms. The codes for CLET and analysis, along with datasets, tables, and a tutorial video for using the codes, have been made publicly available.