Task-related hemodynamic responses are modulated by reward and task engagement.

Hemodynamic recordings from visual cortex contain powerful endogenous task-related responses that may reflect task-related arousal, or "task engagement" distinct from attention. We tested this hypothesis with hemodynamic measurements (intrinsic-signal optical imaging) from monkey primary visual cortex (V1) while the animals' engagement in a periodic fixation task over several hours was varied through reward size and as animals took breaks. With higher rewards, animals appeared more task-engaged; task-related responses were more temporally precise at the task period (approximately 10-20 seconds) and modestly stronger. The 2-5 minute blocks of high-reward trials led to ramp-like decreases in mean local blood volume; these reversed with ramp-like increases during low reward. The blood volume increased even more sharply when the animal shut his eyes and disengaged completely from the task (5-10 minutes). We propose a mechanism that controls vascular tone, likely along with local neural responses in a manner that reflects task engagement over the full range of timescales tested.

Stimulus-related neuroimaging in task-engaged subjects is best predicted by concurrent spiking.

The implicit goal of functional magnetic resonance imaging is to infer local neural activity. There is considerable debate, however, as to whether imaging correlates most linearly with local spiking or some local field potential (LFP) measurement. Through simultaneous neuroimaging (intrinsic-signal optical imaging) and electrode recordings from alert, task-engaged macaque monkeys, we showed previously that local electrophysiology correlates with only a specific stimulus-related imaging component. Here we show that this stimulus-related component--obtained by subtracting a substantial task-related component--is particularly linear with local spiking over a comprehensive range of response strengths. Matches to concurrent LFP measurements are, to varying degrees, poorer. As a control, we also tried matching the full imaging signal to local electrophysiology without subtracting task-related components. These control matches were consistently worse; they were, however, slightly better for gamma LFP than spiking, potentially resolving discrepancies between our findings and earlier reports favoring LFP.

Anticipatory haemodynamic signals in sensory cortex not predicted by local neuronal activity.

Haemodynamic signals underlying functional brain imaging (for example, functional magnetic resonance imaging (fMRI)) are assumed to reflect metabolic demand generated by local neuronal activity, with equal increases in haemodynamic signal implying equal increases in the underlying neuronal activity. Few studies have compared neuronal and haemodynamic signals in alert animals to test for this assumed correspondence. Here we present evidence that brings this assumption into question. Using a dual-wavelength optical imaging technique that independently measures cerebral blood volume and oxygenation, continuously, in alert behaving monkeys, we find two distinct components to the haemodynamic signal in the alert animals' primary visual cortex (V1). One component is reliably predictable from neuronal responses generated by visual input. The other component-of almost comparable strength-is a hitherto unknown signal that entrains to task structure independently of visual input or of standard neural predictors of haemodynamics. This latter component shows predictive timing, with increases of cerebral blood volume in anticipation of trial onsets even in darkness. This trial-locked haemodynamic signal could be due to an accompanying V1 arterial pumping mechanism, closely matched in time, with peaks of arterial dilation entrained to predicted trial onsets. These findings (tested in two animals) challenge the current understanding of the link between brain haemodynamics and local neuronal activity. They also suggest the existence of a novel preparatory mechanism in the brain that brings additional arterial blood to cortex in anticipation of expected tasks.

Spatial homogeneity and task-synchrony of the trial-related hemodynamic signal.

There is growing evidence that functional brain images in alert task-engaged subjects contain task-related but stimulus-independent signals in addition to stimulus-evoked responses. It is important to separate these different components when analyzing the neuroimaging signal. Using intrinsic-signal optical imaging combined with electrophysiology we had earlier reported a particular 'trial-related signal' in the primary visual cortex (V1) of alert monkeys performing periodic fixation tasks. This signal periodically modulated V1 tissue blood volume, in time with anticipated trial onsets. Unlike visually evoked blood volume changes, however, this signal was present even in total darkness. Further, it could not be predicted by concurrently recorded spiking or local field potentials. Here we use our earlier recording techniques to analyze the spatial distribution of this trial-related signal over our imaged area (10mm square, subdivided into a 16×16 grid, i.e. at 625 µm resolution). We show that the signal is spatially coherent and essentially homogeneous over the imaged region and fails to be predicted by concurrent electrode recordings even at the resolution of a single grid square at the electrode tip. As a corollary we show that the signal is critically linked to the animals' engagement in a task. Not only does the trial-related signal entrain accurately and precisely to any task timing at which the animal was willing to perform; the signal also loses the entrained trial-locked pattern dramatically, within a single trial, when the animal stops performing correctly. Thus the signal is very unlikely to be an ongoing task-independent vascular oscillation. These findings will help categorize the likely distinct varieties of non-stimulus-related signals evoked during behavioral tasks, and lead to a further understanding of the elements comprising the net neuroimaging response.

Spatiotemporal precision and hemodynamic mechanism of optical point spreads in alert primates.

In functional brain imaging there is controversy over which hemodynamic signal best represents neural activity. Intrinsic signal optical imaging (ISOI) suggests that the best signal is the early darkening observed at wavelengths absorbed preferentially by deoxyhemoglobin (HbR). It is assumed that this darkening or "initial dip" reports local conversion of oxyhemoglobin (HbO) to HbR, i.e., oxygen consumption caused by local neural activity, thus giving the most specific measure of such activity. The blood volume signal, by contrast, is believed to be more delayed and less specific. Here, we used multiwavelength ISOI to simultaneously map oxygenation and blood volume [i.e., total hemoglobin (HbT)] in primary visual cortex (V1) of the alert macaque. We found that the hemodynamic "point spread," i.e., impulse response to a minimal visual stimulus, was as rapid and retinotopically specific when imaged by using blood volume as when using the initial dip. Quantitative separation of the imaged signal into HbR, HbO, and HbT showed, moreover, that the initial dip was dominated by a fast local increase in HbT, with no increase in HbR. We found only a delayed HbR decrease that was broader in retinotopic spread than HbO or HbT. Further, we show that the multiphasic time course of typical ISOI signals and the strength of the initial dip may reflect the temporal interplay of monophasic HbO, HbR, and HbT signals. Characterizing the hemodynamic response is important for understanding neurovascular coupling and elucidating the physiological basis of imaging techniques such as fMRI.

COVID-19 masks increase the influence of face recognition algorithm decisions on human decisions in unfamiliar face matching.

Face masks, recently adopted to reduce the spread of COVID-19, have had the unintended consequence of increasing the difficulty of face recognition. In security applications, face recognition algorithms are used to identify individuals and present results for human review. This combination of human and algorithm capabilities, known as human-algorithm teaming, is intended to improve total system performance. However, prior work has shown that human judgments of face pair similarity-confidence can be biased by an algorithm's decision even in the case of an error by that algorithm. This can reduce team effectiveness, particularly for difficult face pairs. We conducted two studies to examine whether face masks, now routinely present in security applications, impact the degree to which this cognitive bias is experienced by humans. We first compared the influence of algorithm's decisions on human similarity-confidence ratings in the presence and absence of face masks and found that face masks more than doubled the influence of algorithm decisions on human similarity-confidence ratings. We then investigated if this increase in cognitive bias was dependent on perceived algorithm accuracy by also presenting algorithm accuracy rates in the presence of face masks. We found that making humans aware of the potential for algorithm errors mitigated the increase in cognitive bias due to face masks. Our findings suggest that humans reviewing face recognition algorithm decisions should be made aware of the potential for algorithm errors to improve human-algorithm team performance.

What could underlie the trial-related signal? A response to the commentaries by Drs. Kleinschmidt and Muller, and Drs. Handwerker and Bandettini.

Here we address two recent commentaries on our finding of an anticipatory trial-related signal that could not be predicted by concurrent electrode recordings. In addition, we offer a broad discussion regarding what our findings do and do not say about local neural activity underlying imaging signals.

Human-algorithm teaming in face recognition: How algorithm outcomes cognitively bias human decision-making.

In face recognition applications, humans often team with algorithms, reviewing algorithm results to make an identity decision. However, few studies have explicitly measured how algorithms influence human face matching performance. One study that did examine this interaction found a concerning deterioration of human accuracy in the presence of algorithm errors. We conducted an experiment to examine how prior face identity decisions influence subsequent human judgements about face similarity. 376 volunteers were asked to rate the similarity of face pairs along a scale. Volunteers performing the task were told that they were reviewing identity decisions made by different sources, either a computer or human, or were told to make their own judgement without prior information. Replicating past results, we found that prior identity decisions, presented as labels, influenced volunteers' own identity judgements. We extend these results as follows. First, we show that the influence of identity decision labels was independent of indicated decision source (human or computer) despite volunteers' greater distrust of human identification ability. Second, applying a signal detection theory framework, we show that prior identity decision labels did not reduce volunteers' attention to the face pair. Discrimination performance was the same with and without the labels. Instead, prior identity decision labels altered volunteers' internal criterion used to judge a face pair as "matching" or "non-matching". This shifted volunteers' face pair similarity judgements by a full step along the response scale. Our work shows how human face matching is affected by prior identity decision labels and we discuss how this may limit the total accuracy of human-algorithm teams performing face matching tasks.

Sniff Invariant Odor Coding.

Sampling regulates stimulus intensity and temporal dynamics at the sense organ. Despite variations in sampling behavior, animals must make veridical perceptual judgments about external stimuli. In olfaction, odor sampling varies with respiration, which influences neural responses at the olfactory periphery. Nevertheless, rats were able to perform fine odor intensity judgments despite variations in sniff kinetics. To identify the features of neural activity supporting stable intensity perception, in awake mice we measured responses of mitral/tufted (MT) cells to different odors and concentrations across a range of sniff frequencies. Amplitude and latency of the MT cells' responses vary with sniff duration. A fluid dynamics (FD) model based on odor concentration kinetics in the intranasal cavity can account for this variability. Eliminating sniff waveform dependence of MT cell responses using the FD model allows for significantly better decoding of concentration. This suggests potential schemes for sniff waveform invariant odor concentration coding.

Simultaneously estimating the task-related and stimulus-evoked components of hemodynamic imaging measurements.

Task-related hemodynamic responses contribute prominently to functional magnetic resonance imaging (fMRI) recordings. They reflect behaviorally important brain states, such as arousal and attention, and can dominate stimulus-evoked responses, yet they remain poorly understood. To help characterize these responses, we present a method for parametrically estimating both stimulus-evoked and task-related components of hemodynamic responses from subjects engaged in temporally predictable tasks. The stimulus-evoked component is modeled by convolving a hemodynamic response function (HRF) kernel with spiking. The task-related component is modeled by convolving a Fourier-series task-related function (TRF) kernel with task timing. We fit this model with simultaneous electrode recordings and intrinsic-signal optical imaging from the primary visual cortex of alert, task-engaged monkeys. With high [Formula: see text], the model returns HRFs that are consistent across experiments and recording sites for a given animal and TRFs that entrain to task timing independent of stimulation or local spiking. When the task schedule conflicts with that of stimulation, the TRF remains locked to the task emphasizing its behavioral origins. The current approach is strikingly more robust to fluctuations than earlier ones and gives consistently, if modestly, better fits. This approach could help parse the distinct components of fMRI recordings made in the context of a task.

Temporal associations and prior-list intrusions in free recall.

When asked to recall the words from a just-presented target list, subjects occasionally recall words that were not on the list. These intrusions either appeared on earlier lists (prior-list intrusions, or PLIs) or had not appeared over the course of the experiment (extra-list intrusions). The authors examined the factors that elicit PLIs in free recall. A reanalysis of earlier studies revealed that PLIs tend to come from semantic associates as well as from recently studied lists, with the rate of PLIs decreasing sharply with list recency. The authors report 3 new experiments in which some items in a given list also appeared on earlier lists. Although repetition enhanced recall of list items, subjects were significantly more likely to make PLIs following the recall of repeated items, suggesting that temporal associations formed in earlier lists can induce recall errors. The authors interpret this finding as evidence for the interacting roles of associative and contextual retrieval processes in recall. Although contextual information helps to focus recall on words in the target list, it does not form an impermeable boundary between current- and prior-list experiences.

Going beyond a single list: modeling the effects of prior experience on episodic free recall.

We present an extension of the search of associative memory (SAM) model that simulates the effects of both prior semantic knowledge and prior episodic experience on episodic free recall. The model incorporates a memory store for preexisting semantic associations, a contextual drift mechanism, a memory search mechanism that uses both episodic and semantic associations, and a large lexicon including both words from prior lists and unpresented words. These features enabled the model to successfully account for the effects of prior semantic knowledge and prior episodic learning on the pattern of correct recalls and intrusions observed in free recall experiments.

Neural Coding of Perceived Odor Intensity.

Stimulus intensity is a fundamental perceptual feature in all sensory systems. In olfaction, perceived odor intensity depends on at least two variables: odor concentration; and duration of the odor exposure or adaptation. To examine how neural activity at early stages of the olfactory system represents features relevant to intensity perception, we studied the responses of mitral/tufted cells (MTCs) while manipulating odor concentration and exposure duration. Temporal profiles of MTC responses to odors changed both as a function of concentration and with adaptation. However, despite the complexity of these responses, adaptation and concentration dependencies behaved similarly. These similarities were visualized by principal component analysis of average population responses and were quantified by discriminant analysis in a trial-by-trial manner. The qualitative functional dependencies of neuronal responses paralleled psychophysics results in humans. We suggest that temporal patterns of MTC responses in the olfactory bulb contribute to an internal perceptual variable: odor intensity.

Single scale for odor intensity in rat olfaction.

Humans and laboratory animals are thought to discriminate sensory objects using elemental perceptual features computed by neural circuits in the brain. However, it is often difficult to identify the perceptual features that animals use to make specific comparisons. In olfaction, changes in the concentration of a given odor lead to discriminable changes in both its perceived quality and intensity. Humans use perceived intensity to compare quantities of different odors. Here we establish that laboratory rats also use perceived intensity to compare concentrations of different odors and reveal the perceptual organization of this elemental feature. We first trained rats to classify concentrations of single odors as high or low. When subsequently classifying concentrations of two odors presented on different trials of the same session, rats made errors consistent with using a single intensity criterion for both odors. This allowed us to investigate the relative perceived intensity of different odor pairs. Odor intensity was not only a function of concentration, but varied also with molecular weight and exposure time. These findings demonstrate the role of perceived intensity as an elemental perceptual feature of odors in rat olfaction.

Rodent ultrasonic vocalizations are bound to active sniffing behavior.

During rodent active behavior, multiple orofacial sensorimotor behaviors, including sniffing and whisking, display rhythmicity in the theta range (~5-10 Hz). During specific behaviors, these rhythmic patterns interlock, such that execution of individual motor programs becomes dependent on the state of the others. Here we performed simultaneous recordings of the respiratory cycle and ultrasonic vocalization emission by adult rats and mice in social settings. We used automated analysis to examine the relationship between breathing patterns and vocalization over long time periods. Rat ultrasonic vocalizations (USVs, "50 kHz") were emitted within stretches of active sniffing (5-10 Hz) and were largely absent during periods of passive breathing (1-4 Hz). Because ultrasound was tightly linked to the exhalation phase, the sniffing cycle segmented vocal production into discrete calls and imposed its theta rhythmicity on their timing. In turn, calls briefly prolonged exhalations, causing an immediate drop in sniffing rate. Similar results were obtained in mice. Our results show that ultrasonic vocalizations are an integral part of the rhythmic orofacial behavioral ensemble. This complex behavioral program is thus involved not only in active sensing but also in the temporal structuring of social communication signals. Many other social signals of mammals, including monkey calls and human speech, show structure in the theta range. Our work points to a mechanism for such structuring in rodent ultrasonic vocalizations.

The neuroimaging signal is a linear sum of neurally distinct stimulus- and task-related components.

Neuroimaging (for example, functional magnetic resonance imaging) signals are taken as a uniform proxy for local neural activity. By simultaneously recording electrode and neuroimaging (intrinsic optical imaging) signals in alert, task-engaged macaque visual cortex, we recently observed a large anticipatory trial-related neuroimaging signal that was poorly related to local spiking or field potentials. We used these same techniques to study the interactions of this trial-related signal with stimulus-evoked responses over the full range of stimulus intensities, including total darkness. We found that the two signals could be separated, and added linearly over this full range. The stimulus-evoked component was related linearly to local spiking and, consequently, could be used to obtain precise and reliable estimates of local neural activity. The trial-related signal likely has a distinct neural mechanism, however, and failure to account for it properly could lead to substantial errors when estimating local neural spiking from the neuroimaging signal.

Zooming in on mouse vision.

An examination of the micro-organization of visual cortex using two-photon calcium imaging provides a new level of insight into retinotopic maps, finding that retinotopy is scrambled on fine scales in mouse primary visual cortex.

Spatial Relationship between Flavoprotein Fluorescence and the Hemodynamic Response in the Primary Visual Cortex of Alert Macaque Monkeys.

Flavoprotein fluorescence imaging (FFI) is a novel intrinsic optical signal that is steadily gaining ground as a valuable imaging tool in neuroscience research due to its closer relationship with local metabolism relative to the more commonly used hemodynamic signals. We have developed a technique for FFI imaging in the primary visual cortex (V1) of alert monkeys. Due to the nature of neurovascular coupling, hemodynamic signals are known to spread beyond the locus of metabolic activity. To determine whether FFI signals could provide a more focal measure of cortical activity in alert animals, we compared FFI and hemodynamic point spreads (i.e. responses to a minimal visual stimulus) and functional mapping signals over V1 in macaques performing simple fixation tasks. FFI responses were biphasic, with an early and focal fluorescence increase followed by a delayed and spatially broader fluorescence decrease. As expected, the early fluorescence increase, indicating increased local oxidative metabolism, was somewhat narrower than the simultaneously observed hemodynamic response. However, the later FFI decrease was broader than the hemodynamic response and started prior to the cessation of visual stimulation suggesting different mechanisms underlying the two phases of the fluorescence signal. FFI mapping signals were free of vascular artifacts and comparable in amplitude to hemodynamic mapping signals. These results indicate that the FFI response may be a more local and direct indicator of cortical metabolism than the hemodynamic response in alert animals.

Reaction times of manual responses to a visual stimulus at the goal of a planned memory-guided saccade in the monkey.

Monkeys demonstrate improved contrast sensitivity at the goal of a planned memory-guided saccade (Science 299:81-86, 2003). Such perceptual improvements have been ascribed to an endogenous attentional advantage induced by the saccade plan. Speeded reaction times have also been used as evidence for attention. We therefore asked whether the attentional advantage at the goal of a planned memory-guided saccade led to speeded manual reaction times following probes presented at the saccade goal in a simple detection task. We found that monkeys showed slower manual reaction times when the probe appeared at the memorized goal of the planned saccade when compared to manual reaction times following a probe that appeared opposite the saccade goal. Flashing a distractor at the saccade goal after target presentation appeared to slow reaction times further. Our data, combined with prior results, suggest that a spatially localized inhibition operates on the neural representation of the saccade goal. This inhibition may be closely related or identical to the processes underlying inhibition-of-return. We also found that if the same detection task was interleaved with a difficult perceptual discrimination task, manual reaction times became faster when the probe was at the saccade goal. We interpret these results as being an effect of task difficulty; the more difficult interleaved task may have engaged endogenous attentional resources more effectively, allowing it to override the inhibition at the saccade goal. We construct and discuss a simple working hypothesis for the relationship between the effects of prior attention on neural activity in salience maps and on performance in detection and discrimination tasks.