Online spike-based recognition of digits with ultrafast microlaser neurons.

Classification and recognition tasks performed on photonic hardware-based neural networks often require at least one offline computational step, such as in the increasingly popular reservoir computing paradigm. Removing this offline step can significantly improve the response time and energy efficiency of such systems. We present numerical simulations of different algorithms that utilize ultrafast photonic spiking neurons as receptive fields to allow for image recognition without an offline computing step. In particular, we discuss the merits of event, spike-time and rank-order based algorithms adapted to this system. These techniques have the potential to significantly improve the efficiency and effectiveness of optical classification systems, minimizing the number of spiking nodes required for a given task and leveraging the parallelism offered by photonic hardware.

The fate of visual long term memories for images across weeks in adults and children.

What is the content and the format of visual memories in Long Term Memory (LTM)? Is it similar in adults and children? To address these issues, we investigated, in both adults and 9-year-old children, how visual LTM is affected over time and whether visual vs semantic features are affected differentially. In a learning phase, participants were exposed to hundreds of meaningless and meaningful images presented once or twice for either 120 ms or 1920 ms. Memory was assessed using a recognition task either immediately after learning or after a delay of three or six weeks. The results suggest that multiple and extended exposures are crucial for retaining an image for several weeks. Although a benefit was observed in the meaningful condition when memory was assessed immediately after learning, this benefit tended to disappear over weeks, especially when the images were presented twice for 1920 ms. This pattern was observed for both adults and children. Together, the results call into question the dominant models of LTM for images: although semantic information enhances the encoding & maintaining of images in LTM when assessed immediately, this seems not critical for LTM over weeks.

Analyzing time-to-first-spike coding schemes: A theoretical approach.

Spiking neural networks (SNNs) using time-to-first-spike (TTFS) codes, in which neurons fire at most once, are appealing for rapid and low power processing. In this theoretical paper, we focus on information coding and decoding in those networks, and introduce a new unifying mathematical framework that allows the comparison of various coding schemes. In an early proposal, called rank-order coding (ROC), neurons are maximally activated when inputs arrive in the order of their synaptic weights, thanks to a shunting inhibition mechanism that progressively desensitizes the neurons as spikes arrive. In another proposal, called NoM coding, only the first N spikes of M input neurons are propagated, and these "first spike patterns" can be readout by downstream neurons with homogeneous weights and no desensitization: as a result, the exact order between the first spikes does not matter. This paper also introduces a third option-"Ranked-NoM" (R-NoM), which combines features from both ROC and NoM coding schemes: only the first N input spikes are propagated, but their order is readout by downstream neurons thanks to inhomogeneous weights and linear desensitization. The unifying mathematical framework allows the three codes to be compared in terms of discriminability, which measures to what extent a neuron responds more strongly to its preferred input spike pattern than to random patterns. This discriminability turns out to be much higher for R-NoM than for the other codes, especially in the early phase of the responses. We also argue that R-NoM is much more hardware-friendly than the original ROC proposal, although NoM remains the easiest to implement in hardware because it only requires binary synapses.

First Profile of Phenolic Compounds from Maltese Extra Virgin Olive Oils Using Liquid-Liquid Extraction and Liquid Chromatography-Mass Spectrometry.

This study presents the profile of phenolic extracts from different Extra Virgin Olive Oils (EVOOs) from Malta and is the first study that characterizes the phenolic profile of the Maltese EVOOs Bidni (B) and Malti (M) using liquid-liquid extraction (LLE) and Liquid Chromatography-Mass Spectrometry (LC-MS). The total phenolic content (TPC), ortho diphenolic content (TdPC) and flavonoid content (TFC) were determined using the Folin-Ciocalteau assay, the Arnow's assay and the Aluminium Chloride method respectively. Results show that the B variety had the highest TPC, TdPC and TFC. Using LC-MS analysis, over 30 phenolic compounds were identified belonging to different classes of phenolic compounds.

24-Month-olds and over remember novel object names after a single learning event.

In the context of word learning, it is commonly assumed that repetition is required for young children to form and maintain in memory an association between a novel word and its corresponding object. For instance, at 2 years of age, children are able to disambiguate word-related situations in one shot but are not able to further retain this newly acquired knowledge. It has been proposed that multiple fast-mapping experiences would be required to promote word retention or that the inferential reasoning needs to be accompanied by explicit labeling of the target. We hypothesized that when 2-year-olds simply encounter an unambiguous learning context, word learning may be fast and maintained in time. We also assumed that, under this condition, even a single exposure to an object would be sufficient to form a memory trace of its name that would survive a delay. To test these hypotheses, 2- and 4-year-olds were ostensively taught three arbitrary word-object pairs using a 15-s video sequence during which each object was manually displayed and labeled three times in a row. Retention was measured after a 30-min distractive period using a forced-choice procedure. Our results provide evidence that declarative memory does not need repetition to be formed and maintained, for at least a 30-min period, by children as young as 2 years. This finding suggests that the mechanisms required for extremely rapid and robust word acquisition not only are present in preschoolers with developed language and cognitive skills but also are already operative at a younger age.

Recording local field potential and neuronal activity with tetrodes in epileptic patients.

BACKGROUND: Recordings with tetrodes have proven to be more effective in isolating single neuron spiking activity than with single microwires. However, tetrodes have never been used in humans. We report on the characteristics, safety, compatibility with clinical intracranial recordings in epileptic patients, and performance, of a new type of hybrid electrode equipped with tetrodes. NEW METHOD: 240 standard clinical macroelectrodes and 102 hybrid electrodes were implanted in 28 patients. Hybrids (diameter 800 µm) are made of 6 or 9 macro-contacts and 2 or 3 tetrodes (diameter 70-80 µm). RESULTS: No clinical complication or adverse event was associated with the hybrids. Impedance and noise of recordings were stable over time. The design enabled multiscale spatial analyses that revealed physiopathological events which were sometimes specific to one tetrode, but could not be recorded on the macro-contacts. After spike sorting, the single-unit yield was similar to other hybrid electrodes and was sometimes as high as >10 neurons per tetrode. COMPARISON WITH EXISTING METHOD(S): This new hybrid electrode has a smaller diameter than other available hybrid electrodes. It provides novel spatial information due to the configuration of the tetrodes. The single-unit yield appears promising. CONCLUSIONS: This new hybrid electrode is safe, easy to use, and works satisfactorily for conducting multi-scale seizure and physiological analyses.

Stimulus Onset Hub: an Open-Source, Low Latency, and Opto-Isolated Trigger Box for Neuroscientific Research Replicability and Beyond.

Accurate stimulus onset timing is critical to almost all behavioral research. Auditory, visual, or manual response time stimulus onsets are typically sent through wires to various machines that record data such as: eye gaze positions, electroencephalography, stereo electroencephalography, and electrocorticography. These stimulus onsets are collated and analyzed according to experimental condition. If there is variability in the temporal accuracy of the delivery of these onsets to external systems, the quality of the resulting data and scientific analyses will degrade. Here, we describe an approximately 200 dollar Arduino based system and associated open-source codebase that achieved a maximum of 4 microseconds of delay from the inputs to the outputs while electrically opto-isolating the connected external systems. Using an oscilloscope, the device is configurable for the different environmental conditions particular to each laboratory (e.g., light sensor type, screen type, speaker type, stimulus type, temperature, etc). This low-cost open-source project delivered electrically isolated digital stimulus onset Transistor-Transistor Logic triggers with an input/output delay of 4 µs, and was successfully tested with seven different external systems that record eye and neurological data.

Microsaccades during high speed continuous visual search.

Here, we provide an analysis of the microsaccades that occurred during continuous visual search and targeting of small faces that we pasted either into cluttered background photos or into a simple gray background. Subjects continuously used their eyes to target singular 3-degree upright or inverted faces in changing scenes. As soon as the participant's gaze reached the target face, a new face was displayed in a different and random location. Regardless of the experimental context (e.g. background scene, no background scene), or target eccentricity (from 4 to 20 degrees of visual angle), we found that the microsaccade rate dropped to near zero levels within only 12 milliseconds after stimulus onset. There were almost never any microsaccades after stimulus onset and before the first saccade to the face. One subject completed 118 consecutive trials without a single microsaccade. However, in about 20% of the trials, there was a single microsaccade that occurred almost immediately after the preceding saccade's offset. These microsaccades were task oriented because their facial landmark targeting distributions matched those of saccades within both the upright and inverted face conditions. Our findings show that a single feedforward pass through the visual hierarchy for each stimulus is likely all that is needed to effectuate prolonged continuous visual search. In addition, we provide evidence that microsaccades can serve perceptual functions like correcting saccades or effectuating task-oriented goals during continuous visual search.

Tracking Your Mind's Eye during Recollection: Decoding the Long-Term Recall of Short Audiovisual Clips.

Unlike familiarity, recollection involves the ability to reconstruct mentally previous events that results in a strong sense of reliving. According to the reinstatement hypothesis, this specific feature emerges from the reactivation of cortical patterns involved during information exposure. Over time, the retrieval of specific details becomes more difficult, and memories become increasingly supported by familiarity judgments. The multiple trace theory (MTT) explains the gradual loss of episodic details by a transformation in the memory representation, a view that is not shared by the standard consolidation model. In this study, we tested the MTT in light of the reinstatement hypothesis. The temporal dynamics of mental imagery from long-term memory were investigated and tracked over the passage of time. Participant EEG activity was recorded during the recall of short audiovisual clips that had been watched 3 weeks, 1 day, or a few hours beforehand. The recall of the audiovisual clips was assessed using a Remember/Know/New procedure, and snapshots of clips were used as recall cues. The decoding matrices obtained from the multivariate pattern analyses revealed sustained patterns that occurred at long latencies (>500 msec poststimulus onset) that faded away over the retention intervals and that emerged from the same neural processes. Overall, our data provide further evidence toward the MTT and give new insights into the exploration of our "mind's eye."

Object categorization in visual periphery is modulated by delayed foveal noise.

Behavioral studies in humans indicate that peripheral vision can do object recognition to some extent. Moreover, recent studies have shown that some information from brain regions retinotopic to visual periphery is somehow fed back to regions retinotopic to the fovea and disrupting this feedback impairs object recognition in human. However, it is unclear to what extent the information in visual periphery contributes to human object categorization. Here, we designed two series of rapid object categorization tasks to first investigate the performance of human peripheral vision in categorizing natural object images at different eccentricities and abstraction levels (superordinate, basic, and subordinate). Then, using a delayed foveal noise mask, we studied how modulating the foveal representation impacts peripheral object categorization at any of the abstraction levels. We found that peripheral vision can quickly and accurately accomplish superordinate categorization, while its performance in finer categorization levels dramatically drops as the object presents further in the periphery. Also, we found that a 300-ms delayed foveal noise mask can significantly disturb categorization performance in basic and subordinate levels, while it has no effect on the superordinate level. Our results suggest that human peripheral vision can easily process objects at high abstraction levels, and the information is fed back to foveal vision to prime foveal cortex for finer categorizations when a saccade is made toward the target object.

Regularity is not a key factor for encoding repetition in rapid image streams.

Human observers readily detect targets and repetitions in streams of rapidly presented visual stimuli. It seems intuitive that regularly spaced repeating items should be easier to detect than irregularly spaced ones, since regularity adds predictability and in addition has ecological relevance. Here, we show that this is not necessarily the case, and we point out the intrinsic difficulty in addressing this question. We presented long RSVP streams of never-before-seen natural images containing repetition sequences; an image appearing six times interleaved by one or more non-repeating distractors, and asked participants to detect the repetitions and to afterwards identify the repeated images. We found that the ability to detect and memorize repeated images was preserved even with irregular sequences, and conclude that temporal regularity is not a key factor for detection and memory for repeating images in RSVP streams. These findings have implications for models of repetition processing.

Memory for Repeated Images in Rapid-Serial-Visual-Presentation Streams of Thousands of Images.

Human observers readily detect targets in stimuli presented briefly and in rapid succession. Here, we show that even without predefined targets, humans can spot repetitions in streams of thousands of images. We presented sequences of natural images reoccurring a number of times interleaved with either one or two distractors, and we asked participants to detect the repetitions and to identify the repeated images after a delay that could last for minutes. Performance improved with the number of repeated-image presentations up to a ceiling around seven repetitions and was above chance even after only two to three presentations. The task was easiest for slow streams; performance dropped with increasing image-presentation rate but stabilized above 15 Hz and remained well above chance even at 120 Hz. To summarize, we reveal that the human brain has an impressive capacity to detect repetitions in rapid-serial-visual-presentation streams and to remember repeated images over a time course of minutes.

Neuronal spiking activity highlights a gradient of epileptogenicity in human tuberous sclerosis lesions.

OBJECTIVE: The mechanisms underlying epileptogenicity in tuberous sclerosis complex (TSC) are poorly understood. METHODS: We analysed neuronal spiking activity (84 neurons), fast ripples (FRs), local field potentials and intracranial electroencephalogram during interictal epileptiform discharges (IEDs) in the tuber and perituber of a patient using novel hybrid electrodes equipped with tetrodes. RESULTS: IEDs were recorded in the tuber and perituber. FRs were recorded only in the tuber and only with the microelectrodes. A larger proportion of neurons in the tuber (57%) than in the perituber (17%) had firing-rates modulated around IEDs. CONCLUSIONS: A multi-scale analysis of neuronal activity, FRs and IEDs indicates a gradient of epileptogenicity running from the tuber to the perituber. SIGNIFICANCE: We demonstrate, for the first time in vivo, a gradient of epileptogenicity from the tuber to the perituber, which paves the way for future models of epilepsy in TSC. Our results also question the extent of the neurosurgical resection, including or not the perituber, that needs to be made in these patients.

High Resolution Human Eye Tracking During Continuous Visual Search.

While several studies have shown human subjects' impressive ability to detect faces in individual images in paced settings (Crouzet et al., 2010), we here report the details of an eye movement dataset in which subjects rapidly and continuously targeted single faces embedded in different scenes at rates approaching six face targets each second (including blinks and eye movement times). In this paper, we describe details of a large publicly available eye movement dataset of this new psychophysical paradigm (Martin et al., 2018). The paradigm produced high-resolution eye-tracking data from an experiment on continuous upright and inverted 3° sized face detection in both background and no-background conditions. The new "Zapping" paradigm allowed large amounts of trials to be completed in a short amount of time. For example, our three studies encompassed a total of 288,000 trials done in 72 separate experiments, and yet only took approximately 40 hours of recording for the three experimental cohorts. Each subject did 4000 trials split into eight blocks of 500 consecutive trials in one of the four different experimental conditions: {upright, inverted} × {scene, no scene}. For each condition, there are several covariates of interest, including: temporal eye positions sampled at 1250 hz, saccades, saccade reaction times, microsaccades, pupil dynamics, target luminances, and global contrasts.

Zapping 500 faces in less than 100 seconds: Evidence for extremely fast and sustained continuous visual search.

A number of studies have shown human subjects' impressive ability to detect faces in individual images, with saccade reaction times starting as fast as 100 ms after stimulus onset. Here, we report evidence that humans can rapidly and continuously saccade towards single faces embedded in different scenes at rates approaching 6 faces/scenes each second (including blinks and eye movement times). These observations are impressive, given that humans usually make no more than 2 to 5 saccades per second when searching a single scene with eye movements. Surprisingly, attempts to hide the faces by blending them into a large background scene had little effect on targeting rates, saccade reaction times, or targeting accuracy. Upright faces were found more quickly and more accurately than inverted faces; both with and without a cluttered background scene, and over a large range of eccentricities (4°-16°). The fastest subject in our study made continuous saccades to 500 small 3° upright faces at 4° eccentricities in only 96 seconds. The maximum face targeting rate ever achieved by any subject during any sequence of 7 faces during Experiment 3 for the no scene and upright face condition was 6.5 faces targeted/second. Our data provide evidence that the human visual system includes an ultra-rapid and continuous object localization system for upright faces. Furthermore, these observations indicate that continuous paradigms such as the one we have used can push humans to make remarkably fast reaction times that impose strong constraints and challenges on models of how, where, and when visual processing occurs in the human brain.

Unsupervised Feature Learning With Winner-Takes-All Based STDP.

We present a novel strategy for unsupervised feature learning in image applications inspired by the Spike-Timing-Dependent-Plasticity (STDP) biological learning rule. We show equivalence between rank order coding Leaky-Integrate-and-Fire neurons and ReLU artificial neurons when applied to non-temporal data. We apply this to images using rank-order coding, which allows us to perform a full network simulation with a single feed-forward pass using GPU hardware. Next we introduce a binary STDP learning rule compatible with training on batches of images. Two mechanisms to stabilize the training are also presented : a Winner-Takes-All (WTA) framework which selects the most relevant patches to learn from along the spatial dimensions, and a simple feature-wise normalization as homeostatic process. This learning process allows us to train multi-layer architectures of convolutional sparse features. We apply our method to extract features from the MNIST, ETH80, CIFAR-10, and STL-10 datasets and show that these features are relevant for classification. We finally compare these results with several other state of the art unsupervised learning methods.

STDP-based spiking deep convolutional neural networks for object recognition.

Previous studies have shown that spike-timing-dependent plasticity (STDP) can be used in spiking neural networks (SNN) to extract visual features of low or intermediate complexity in an unsupervised manner. These studies, however, used relatively shallow architectures, and only one layer was trainable. Another line of research has demonstrated - using rate-based neural networks trained with back-propagation - that having many layers increases the recognition robustness, an approach known as deep learning. We thus designed a deep SNN, comprising several convolutional (trainable with STDP) and pooling layers. We used a temporal coding scheme where the most strongly activated neurons fire first, and less activated neurons fire later or not at all. The network was exposed to natural images. Thanks to STDP, neurons progressively learned features corresponding to prototypical patterns that were both salient and frequent. Only a few tens of examples per category were required and no label was needed. After learning, the complexity of the extracted features increased along the hierarchy, from edge detectors in the first layer to object prototypes in the last layer. Coding was very sparse, with only a few thousands spikes per image, and in some cases the object category could be reasonably well inferred from the activity of a single higher-order neuron. More generally, the activity of a few hundreds of such neurons contained robust category information, as demonstrated using a classifier on Caltech 101, ETH-80, and MNIST databases. We also demonstrate the superiority of STDP over other unsupervised techniques such as random crops (HMAX) or auto-encoders. Taken together, our results suggest that the combination of STDP with latency coding may be a key to understanding the way that the primate visual system learns, its remarkable processing speed and its low energy consumption. These mechanisms are also interesting for artificial vision systems, particularly for hardware solutions.

Extremely long-term memory and familiarity after 12 years.

In 2006 Mitchell demonstrated that implicit memory was robust to decay. He showed that the ability to identify fragments of pictures seen 17 years before was significantly higher than for new stimuli. Is this true only for implicit memory? In this study, we tested whether explicit memory was still possible for drawings (n = 144) that had been presented once or three times, two seconds each time on average, approximately 12 years earlier. Surprisingly, our data reveal that our participants were able to recognize pictures above chance level. Preserved memory was mainly observed in the youngest subjects, for stimuli seen three times. Despite the fact that confidence judgments were low, reports suggest that recognition could be based on a strong sense of familiarity. These data extend Mitchell's findings and show that familiarity can also be robust to decay.

Waking Up Buried Memories of Old TV Programs.

Although it has been demonstrated that visual and auditory stimuli can be recalled decades after the initial exposure, previous studies have generally not ruled out the possibility that the material may have been seen or heard during the intervening period. Evidence shows that reactivations of a long-term memory trace play a role in its update and maintenance. In the case of remote or very long-term memories, it is most likely that these reactivations are triggered by the actual re-exposure to the stimulus. In this study we decided to explore whether it is possible to recall stimuli that could not have been re-experienced in the intervening period. We tested the ability of French participants (N = 34, 31 female) to recall 50 TV programs broadcast on average for the last time 44 years ago (from the 60's and early 70's). Potential recall was elicited by the presentation of short audiovisual excerpts of these TV programs. The absence of potential re-exposure to the material was strictly controlled by selecting TV programs that have never been rebroadcast and were not available in the public domain. Our results show that six TV programs were particularly well identified on average across the 34 participants with a median percentage of 71.7% (SD = 13.6, range: 48.5-87.9%). We also obtained 50 single case reports with associated information about the viewing of 23 TV programs including the 6 previous ones. More strikingly, for two cases, retrieval of the title was made spontaneously without the need of a four-proposition choice. These results suggest that re-exposures to the stimuli are not necessary to maintain a memory for a lifetime. These new findings raise fundamental questions about the underlying mechanisms used by the brain to store these very old sensory memories.

Long Term Memory for Noise: Evidence of Robust Encoding of Very Short Temporal Acoustic Patterns.

Recent research has demonstrated that humans are able to implicitly encode and retain repeating patterns in meaningless auditory noise. Our study aimed at testing the robustness of long-term implicit recognition memory for these learned patterns. Participants performed a cyclic/non-cyclic discrimination task, during which they were presented with either 1-s cyclic noises (CNs) (the two halves of the noise were identical) or 1-s plain random noises (Ns). Among CNs and Ns presented once, target CNs were implicitly presented multiple times within a block, and implicit recognition of these target CNs was tested 4 weeks later using a similar cyclic/non-cyclic discrimination task. Furthermore, robustness of implicit recognition memory was tested by presenting participants with looped (shifting the origin) and scrambled (chopping sounds into 10- and 20-ms bits before shuffling) versions of the target CNs. We found that participants had robust implicit recognition memory for learned noise patterns after 4 weeks, right from the first presentation. Additionally, this memory was remarkably resistant to acoustic transformations, such as looping and scrambling of the sounds. Finally, implicit recognition of sounds was dependent on participant's discrimination performance during learning. Our findings suggest that meaningless temporal features as short as 10 ms can be implicitly stored in long-term auditory memory. Moreover, successful encoding and storage of such fine features may vary between participants, possibly depending on individual attention and auditory discrimination abilities. Significance Statement Meaningless auditory patterns could be implicitly encoded and stored in long-term memory.Acoustic transformations of learned meaningless patterns could be implicitly recognized after 4 weeks.Implicit long-term memories can be formed for meaningless auditory features as short as 10 ms.Successful encoding and long-term implicit recognition of meaningless patterns may strongly depend on individual attention and auditory discrimination abilities.