Tracking Your Mind's Eye during Recollection: Decoding the Long-Term Recall of Short Audiovisual Clips.

Unlike familiarity, recollection involves the ability to reconstruct mentally previous events that results in a strong sense of reliving. According to the reinstatement hypothesis, this specific feature emerges from the reactivation of cortical patterns involved during information exposure. Over time, the retrieval of specific details becomes more difficult, and memories become increasingly supported by familiarity judgments. The multiple trace theory (MTT) explains the gradual loss of episodic details by a transformation in the memory representation, a view that is not shared by the standard consolidation model. In this study, we tested the MTT in light of the reinstatement hypothesis. The temporal dynamics of mental imagery from long-term memory were investigated and tracked over the passage of time. Participant EEG activity was recorded during the recall of short audiovisual clips that had been watched 3 weeks, 1 day, or a few hours beforehand. The recall of the audiovisual clips was assessed using a Remember/Know/New procedure, and snapshots of clips were used as recall cues. The decoding matrices obtained from the multivariate pattern analyses revealed sustained patterns that occurred at long latencies (>500 msec poststimulus onset) that faded away over the retention intervals and that emerged from the same neural processes. Overall, our data provide further evidence toward the MTT and give new insights into the exploration of our "mind's eye."

Memory for Repeated Images in Rapid-Serial-Visual-Presentation Streams of Thousands of Images.

Human observers readily detect targets in stimuli presented briefly and in rapid succession. Here, we show that even without predefined targets, humans can spot repetitions in streams of thousands of images. We presented sequences of natural images reoccurring a number of times interleaved with either one or two distractors, and we asked participants to detect the repetitions and to identify the repeated images after a delay that could last for minutes. Performance improved with the number of repeated-image presentations up to a ceiling around seven repetitions and was above chance even after only two to three presentations. The task was easiest for slow streams; performance dropped with increasing image-presentation rate but stabilized above 15 Hz and remained well above chance even at 120 Hz. To summarize, we reveal that the human brain has an impressive capacity to detect repetitions in rapid-serial-visual-presentation streams and to remember repeated images over a time course of minutes.

Object categorization in visual periphery is modulated by delayed foveal noise.

Behavioral studies in humans indicate that peripheral vision can do object recognition to some extent. Moreover, recent studies have shown that some information from brain regions retinotopic to visual periphery is somehow fed back to regions retinotopic to the fovea and disrupting this feedback impairs object recognition in human. However, it is unclear to what extent the information in visual periphery contributes to human object categorization. Here, we designed two series of rapid object categorization tasks to first investigate the performance of human peripheral vision in categorizing natural object images at different eccentricities and abstraction levels (superordinate, basic, and subordinate). Then, using a delayed foveal noise mask, we studied how modulating the foveal representation impacts peripheral object categorization at any of the abstraction levels. We found that peripheral vision can quickly and accurately accomplish superordinate categorization, while its performance in finer categorization levels dramatically drops as the object presents further in the periphery. Also, we found that a 300-ms delayed foveal noise mask can significantly disturb categorization performance in basic and subordinate levels, while it has no effect on the superordinate level. Our results suggest that human peripheral vision can easily process objects at high abstraction levels, and the information is fed back to foveal vision to prime foveal cortex for finer categorizations when a saccade is made toward the target object.

Finding faces, animals, and vehicles in far peripheral vision.

Neuroimaging studies have shown that faces exhibit a central visual field bias, as compared to buildings and scenes. With a saccadic choice task, Crouzet, Kirchner, and Thorpe (2010) demonstrated a speed advantage for the detection of faces with stimuli located 8° from fixation. We used the same paradigm to examine whether the face advantage, relative to other categories (animals and vehicles), extends across the whole visual field (from 10° to 80° eccentricity) or whether it is limited to the central visual field. Pairs of photographs of natural scenes (a target and a distractor) were displayed simultaneously left and right of central fixation for 1s on a panoramic screen. Participants were asked to saccade to a target stimulus (faces, animals, or vehicles). The distractors were images corresponding to the two other categories. Eye movements were recorded with a head-mounted eye tracker. Only the first saccade was measured. Experiment 1 showed that (a) in terms of speed of categorization, faces maintain their advantage over animals and vehicles across the whole visual field, up to 80° and (b) even in crowded conditions (an object embedded in a scene), performance was above chance for the three categories of stimuli at 80° eccentricity. Experiment 2 showed that, when compared to another category with a high degree of within category structural similarity (cars), faces keep their advantage at all eccentricities. These results suggest that the bias for faces is not limited to the central visual field, at least in a categorization task.

At 120 msec you can spot the animal but you don't yet know it's a dog.

Earlier studies suggested that the visual system processes information at the basic level (e.g., dog) faster than at the subordinate (e.g., Dalmatian) or superordinate (e.g., animals) levels. However, the advantage of the basic category over the superordinate category in object recognition has been challenged recently, and the hierarchical nature of visual categorization is now a matter of debate. To address this issue, we used a forced-choice saccadic task in which a target and a distractor image were displayed simultaneously on each trial and participants had to saccade as fast as possible toward the image containing animal targets based on different categorization levels. This protocol enables us to investigate the first 100-120 msec, a previously unexplored temporal window, of visual object categorization. The first result is a surprising stability of the saccade latency (median RT ∼ 155 msec) regardless of the animal target category and the dissimilarity of target and distractor image sets. Accuracy was high (around 80% correct) for categorization tasks that can be solved at the superordinate level but dropped to almost chance levels for basic level categorization. At the basic level, the highest accuracy (62%) was obtained when distractors were restricted to another dissimilar basic category. Computational simulations based on the saliency map model showed that the results could not be predicted by pure bottom-up saliency differences between images. Our results support a model of visual recognition in which the visual system can rapidly access relatively coarse visual representations that provide information at the superordinate level of an object, but where additional visual analysis is required to allow more detailed categorization at the basic level.

Auditory gist: recognition of very short sounds from timbre cues.

Sounds such as the voice or musical instruments can be recognized on the basis of timbre alone. Here, sound recognition was investigated with severely reduced timbre cues. Short snippets of naturally recorded sounds were extracted from a large corpus. Listeners were asked to report a target category (e.g., sung voices) among other sounds (e.g., musical instruments). All sound categories covered the same pitch range, so the task had to be solved on timbre cues alone. The minimum duration for which performance was above chance was found to be short, on the order of a few milliseconds, with the best performance for voice targets. Performance was independent of pitch and was maintained when stimuli contained less than a full waveform cycle. Recognition was not generally better when the sound snippets were time-aligned with the sound onset compared to when they were extracted with a random starting time. Finally, performance did not depend on feedback or training, suggesting that the cues used by listeners in the artificial gating task were similar to those relevant for longer, more familiar sounds. The results show that timbre cues for sound recognition are available at a variety of time scales, including very short ones.

Processing of short auditory stimuli: the rapid audio sequential presentation paradigm (RASP).

Human listeners seem to be remarkably able to recognise acoustic sound sources based on timbre cues. Here we describe a psychophysical paradigm to estimate the time it takes to recognise a set of complex sounds differing only in timbre cues: both in terms of the minimum duration of the sounds and the inferred neural processing time. Listeners had to respond to the human voice while ignoring a set of distractors. All sounds were recorded from natural sources over the same pitch range and equalised to the same duration and power. In a first experiment, stimuli were gated in time with a raised-cosine window of variable duration and random onset time. A voice/non-voice (yes/no) task was used. Performance, as measured by d', remained above chance for the shortest sounds tested (2 ms); d's above 1 were observed for durations longer than or equal to 8 ms. Then, we constructed sequences of short sounds presented in rapid succession. Listeners were asked to report the presence of a single voice token that could occur at a random position within the sequence. This method is analogous to the "rapid sequential visual presentation" paradigm (RSVP), which has been used to evaluate neural processing time for images. For 500-ms sequences made of 32-ms and 16-ms sounds, d' remained above chance for presentation rates of up to 30 sounds per second. There was no effect of the pitch relation between successive sounds: identical for all sounds in the sequence or random for each sound. This implies that the task was not determined by streaming or forward masking, as both phenomena would predict better performance for the random pitch condition. Overall, the recognition of familiar sound categories such as the voice seems to be surprisingly fast, both in terms of the acoustic duration required and of the underlying neural time constants.

Fast recognition of musical sounds based on timbre.

Human listeners seem to have an impressive ability to recognize a wide variety of natural sounds. However, there is surprisingly little quantitative evidence to characterize this fundamental ability. Here the speed and accuracy of musical-sound recognition were measured psychophysically with a rich but acoustically balanced stimulus set. The set comprised recordings of notes from musical instruments and sung vowels. In a first experiment, reaction times were collected for three target categories: voice, percussion, and strings. In a go/no-go task, listeners reacted as quickly as possible to members of a target category while withholding responses to distractors (a diverse set of musical instruments). Results showed near-perfect accuracy and fast reaction times, particularly for voices. In a second experiment, voices were recognized among strings and vice-versa. Again, reaction times to voices were faster. In a third experiment, auditory chimeras were created to retain only spectral or temporal features of the voice. Chimeras were recognized accurately, but not as quickly as natural voices. Altogether, the data suggest rapid and accurate neural mechanisms for musical-sound recognition based on selectivity to complex spectro-temporal signatures of sound sources.

First Profile of Phenolic Compounds from Maltese Extra Virgin Olive Oils Using Liquid-Liquid Extraction and Liquid Chromatography-Mass Spectrometry.

This study presents the profile of phenolic extracts from different Extra Virgin Olive Oils (EVOOs) from Malta and is the first study that characterizes the phenolic profile of the Maltese EVOOs Bidni (B) and Malti (M) using liquid-liquid extraction (LLE) and Liquid Chromatography-Mass Spectrometry (LC-MS). The total phenolic content (TPC), ortho diphenolic content (TdPC) and flavonoid content (TFC) were determined using the Folin-Ciocalteau assay, the Arnow's assay and the Aluminium Chloride method respectively. Results show that the B variety had the highest TPC, TdPC and TFC. Using LC-MS analysis, over 30 phenolic compounds were identified belonging to different classes of phenolic compounds.

Oscillations, phase-of-firing coding, and spike timing-dependent plasticity: an efficient learning scheme.

Recent experiments have established that information can be encoded in the spike times of neurons relative to the phase of a background oscillation in the local field potential-a phenomenon referred to as "phase-of-firing coding" (PoFC). These firing phase preferences could result from combining an oscillation in the input current with a stimulus-dependent static component that would produce the variations in preferred phase, but it remains unclear whether these phases are an epiphenomenon or really affect neuronal interactions-only then could they have a functional role. Here we show that PoFC has a major impact on downstream learning and decoding with the now well established spike timing-dependent plasticity (STDP). To be precise, we demonstrate with simulations how a single neuron equipped with STDP robustly detects a pattern of input currents automatically encoded in the phases of a subset of its afferents, and repeating at random intervals. Remarkably, learning is possible even when only a small fraction of the afferents ( approximately 10%) exhibits PoFC. The ability of STDP to detect repeating patterns had been noted before in continuous activity, but it turns out that oscillations greatly facilitate learning. A benchmark with more conventional rate-based codes demonstrates the superiority of oscillations and PoFC for both STDP-based learning and the speed of decoding: the oscillation partially formats the input spike times, so that they mainly depend on the current input currents, and can be efficiently learned by STDP and then recognized in just one oscillation cycle. This suggests a major functional role for oscillatory brain activity that has been widely reported experimentally.

Ultra-rapid sensory responses in the human frontal eye field region.

Most of what we know about the human frontal eye field (FEF) is extrapolated from studies in animals. There is ample evidence that this region is crucial for eye movements. However, evidence is accumulating that this region also plays a role in sensory processing and that it belongs to a "fast brain" system. We set out to investigate these issues in humans, using intracerebral recordings in patients with drug-refractory epilepsy. Event-related potential recordings were obtained from 11 epileptic patients from within the FEF region while they passed a series of visual and auditory perceptual tests. No eye movement was required. Ultra-rapid responses were observed, with mean onset latencies at 24 ms after stimulus to auditory stimuli and 45 ms to visual stimuli. Such early responses were compatible with cortical routes as assessed with simultaneous recordings in primary auditory and visual cortices. Components were modulated very early by the sensory characteristics of the stimuli, in the 30-60 ms period for auditory stimuli and in the 45-60 ms period for visual stimuli. Although the frontal lobes in humans are generally viewed as being involved in high-level cognitive processes, these results indicate that the human FEF is a remarkably quickly activated multimodal region that belongs to a network of low-level neocortical sensory areas.

Fast saccades toward faces: face detection in just 100 ms.

Previous work has demonstrated that the human visual system can detect animals in complex natural scenes very efficiently and rapidly. In particular, using a saccadic choice task, H. Kirchner and S. J. Thorpe (2006) found that when two images are simultaneously flashed in the left and right visual fields, saccades toward the side with an animal can be initiated in as little as 120-130 ms. Here we show that saccades toward human faces are even faster, with the earliest reliable saccades occurring in just 100-110 ms, and mean reaction times of roughly 140 ms. Intriguingly, it appears that these very fast saccades are not completely under instructional control, because when faces were paired with photographs of vehicles, fast saccades were still biased toward faces even when the subject was targeting vehicles. Finally, we tested whether these very fast saccades might only occur in the simple case where the images are presented left and right of fixation by showing they also occur when the images are presented above and below fixation. Such results impose very serious constraints on the sorts of processing model that can be invoked and demonstrate that face-selective behavioral responses can be generated extremely rapidly.

Stimulus Onset Hub: an Open-Source, Low Latency, and Opto-Isolated Trigger Box for Neuroscientific Research Replicability and Beyond.

Accurate stimulus onset timing is critical to almost all behavioral research. Auditory, visual, or manual response time stimulus onsets are typically sent through wires to various machines that record data such as: eye gaze positions, electroencephalography, stereo electroencephalography, and electrocorticography. These stimulus onsets are collated and analyzed according to experimental condition. If there is variability in the temporal accuracy of the delivery of these onsets to external systems, the quality of the resulting data and scientific analyses will degrade. Here, we describe an approximately 200 dollar Arduino based system and associated open-source codebase that achieved a maximum of 4 microseconds of delay from the inputs to the outputs while electrically opto-isolating the connected external systems. Using an oscilloscope, the device is configurable for the different environmental conditions particular to each laboratory (e.g., light sensor type, screen type, speaker type, stimulus type, temperature, etc). This low-cost open-source project delivered electrically isolated digital stimulus onset Transistor-Transistor Logic triggers with an input/output delay of 4 µs, and was successfully tested with seven different external systems that record eye and neurological data.

Microsaccades during high speed continuous visual search.

Here, we provide an analysis of the microsaccades that occurred during continuous visual search and targeting of small faces that we pasted either into cluttered background photos or into a simple gray background. Subjects continuously used their eyes to target singular 3-degree upright or inverted faces in changing scenes. As soon as the participant's gaze reached the target face, a new face was displayed in a different and random location. Regardless of the experimental context (e.g. background scene, no background scene), or target eccentricity (from 4 to 20 degrees of visual angle), we found that the microsaccade rate dropped to near zero levels within only 12 milliseconds after stimulus onset. There were almost never any microsaccades after stimulus onset and before the first saccade to the face. One subject completed 118 consecutive trials without a single microsaccade. However, in about 20% of the trials, there was a single microsaccade that occurred almost immediately after the preceding saccade's offset. These microsaccades were task oriented because their facial landmark targeting distributions matched those of saccades within both the upright and inverted face conditions. Our findings show that a single feedforward pass through the visual hierarchy for each stimulus is likely all that is needed to effectuate prolonged continuous visual search. In addition, we provide evidence that microsaccades can serve perceptual functions like correcting saccades or effectuating task-oriented goals during continuous visual search.

Recording local field potential and neuronal activity with tetrodes in epileptic patients.

BACKGROUND: Recordings with tetrodes have proven to be more effective in isolating single neuron spiking activity than with single microwires. However, tetrodes have never been used in humans. We report on the characteristics, safety, compatibility with clinical intracranial recordings in epileptic patients, and performance, of a new type of hybrid electrode equipped with tetrodes. NEW METHOD: 240 standard clinical macroelectrodes and 102 hybrid electrodes were implanted in 28 patients. Hybrids (diameter 800 µm) are made of 6 or 9 macro-contacts and 2 or 3 tetrodes (diameter 70-80 µm). RESULTS: No clinical complication or adverse event was associated with the hybrids. Impedance and noise of recordings were stable over time. The design enabled multiscale spatial analyses that revealed physiopathological events which were sometimes specific to one tetrode, but could not be recorded on the macro-contacts. After spike sorting, the single-unit yield was similar to other hybrid electrodes and was sometimes as high as >10 neurons per tetrode. COMPARISON WITH EXISTING METHOD(S): This new hybrid electrode has a smaller diameter than other available hybrid electrodes. It provides novel spatial information due to the configuration of the tetrodes. The single-unit yield appears promising. CONCLUSIONS: This new hybrid electrode is safe, easy to use, and works satisfactorily for conducting multi-scale seizure and physiological analyses.

Regularity is not a key factor for encoding repetition in rapid image streams.

Human observers readily detect targets and repetitions in streams of rapidly presented visual stimuli. It seems intuitive that regularly spaced repeating items should be easier to detect than irregularly spaced ones, since regularity adds predictability and in addition has ecological relevance. Here, we show that this is not necessarily the case, and we point out the intrinsic difficulty in addressing this question. We presented long RSVP streams of never-before-seen natural images containing repetition sequences; an image appearing six times interleaved by one or more non-repeating distractors, and asked participants to detect the repetitions and to afterwards identify the repeated images. We found that the ability to detect and memorize repeated images was preserved even with irregular sequences, and conclude that temporal regularity is not a key factor for detection and memory for repeating images in RSVP streams. These findings have implications for models of repetition processing.

Neuronal spiking activity highlights a gradient of epileptogenicity in human tuberous sclerosis lesions.

OBJECTIVE: The mechanisms underlying epileptogenicity in tuberous sclerosis complex (TSC) are poorly understood. METHODS: We analysed neuronal spiking activity (84 neurons), fast ripples (FRs), local field potentials and intracranial electroencephalogram during interictal epileptiform discharges (IEDs) in the tuber and perituber of a patient using novel hybrid electrodes equipped with tetrodes. RESULTS: IEDs were recorded in the tuber and perituber. FRs were recorded only in the tuber and only with the microelectrodes. A larger proportion of neurons in the tuber (57%) than in the perituber (17%) had firing-rates modulated around IEDs. CONCLUSIONS: A multi-scale analysis of neuronal activity, FRs and IEDs indicates a gradient of epileptogenicity running from the tuber to the perituber. SIGNIFICANCE: We demonstrate, for the first time in vivo, a gradient of epileptogenicity from the tuber to the perituber, which paves the way for future models of epilepsy in TSC. Our results also question the extent of the neurosurgical resection, including or not the perituber, that needs to be made in these patients.

Unsupervised learning of visual features through spike timing dependent plasticity.

Spike timing dependent plasticity (STDP) is a learning rule that modifies synaptic strength as a function of the relative timing of pre- and postsynaptic spikes. When a neuron is repeatedly presented with similar inputs, STDP is known to have the effect of concentrating high synaptic weights on afferents that systematically fire early, while postsynaptic spike latencies decrease. Here we use this learning rule in an asynchronous feedforward spiking neural network that mimics the ventral visual pathway and shows that when the network is presented with natural images, selectivity to intermediate-complexity visual features emerges. Those features, which correspond to prototypical patterns that are both salient and consistently present in the images, are highly informative and enable robust object recognition, as demonstrated on various classification tasks. Taken together, these results show that temporal codes may be a key to understanding the phenomenal processing speed achieved by the visual system and that STDP can lead to fast and selective responses.

Parallel processing in high-level categorization of natural images.

Models of visual processing often include an initial parallel stage that is restricted to relatively low-level features, whereas activation of higher-level object descriptions is generally assumed to require attention. Here we report that even high-level object representations can be accessed in parallel: in a rapid animal versus non-animal categorization task, both behavioral and electrophysiological data show that human subjects were as fast at responding to two simultaneously presented natural images as they were to a single one. The implication is that even complex natural images can be processed in parallel without the need for sequential focal attention.

Unsupervised Feature Learning With Winner-Takes-All Based STDP.

We present a novel strategy for unsupervised feature learning in image applications inspired by the Spike-Timing-Dependent-Plasticity (STDP) biological learning rule. We show equivalence between rank order coding Leaky-Integrate-and-Fire neurons and ReLU artificial neurons when applied to non-temporal data. We apply this to images using rank-order coding, which allows us to perform a full network simulation with a single feed-forward pass using GPU hardware. Next we introduce a binary STDP learning rule compatible with training on batches of images. Two mechanisms to stabilize the training are also presented : a Winner-Takes-All (WTA) framework which selects the most relevant patches to learn from along the spatial dimensions, and a simple feature-wise normalization as homeostatic process. This learning process allows us to train multi-layer architectures of convolutional sparse features. We apply our method to extract features from the MNIST, ETH80, CIFAR-10, and STL-10 datasets and show that these features are relevant for classification. We finally compare these results with several other state of the art unsupervised learning methods.