Creating a Home for Timing Researchers: Then, Now, and the Future.

Our ability to perceive event duration and order is critical in every aspect of our lives, from everyday tasks like coordinating our limbs to walk safely, to uniquely human activities like planning our children's future. Many theoretical accounts of timing have been proposed to explain the mechanisms underlying our ability to estimate time and unify events in time. Continuous progress is being met in further refining and extending current theories, with the aim not only to advance our understanding of timing and time perception, but also to make timing more accessible and applicable to daily life. For this to be possible, cross-disciplinary thinking is required, which is something one cannot easily attain in a scientific conference, rather it requires a community. Having a community with an interest and/or expertise in timing can allow for cross-fertilization of ideas. This chapter introduced the story of the Timing Research Forum or else TRF.

The timing database: An open-access, live repository for interval timing studies.

Interval timing refers to the ability to perceive and remember intervals in the seconds to minutes range. Our contemporary understanding of interval timing is derived from relatively small-scale, isolated studies that investigate a limited range of intervals with a small sample size, usually based on a single task. Consequently, the conclusions drawn from individual studies are not readily generalizable to other tasks, conditions, and task parameters. The current paper presents a live database that presents raw data from interval timing studies (currently composed of 68 datasets from eight different tasks incorporating various interval and temporal order judgments) with an online graphical user interface to easily select, compile, and download the data organized in a standard format. The Timing Database aims to promote and cultivate key and novel analyses of our timing ability by making published and future datasets accessible as open-source resources for the entire research community. In the current paper, we showcase the use of the database by testing various core ideas based on data compiled across studies (i.e., temporal accuracy, scalar property, location of the point of subjective equality, malleability of timing precision). The Timing Database will serve as the repository for interval timing studies through the submission of new datasets.

Do the colors of your letters depend on your language? Language-dependent and universal influences on grapheme-color synesthesia in seven languages.

Grapheme-color synesthetes experience graphemes as having a consistent color (e.g., "N is turquoise"). Synesthetes' specific associations (which letter is which color) are often influenced by linguistic properties such as phonetic similarity, color terms ("Y is yellow"), and semantic associations ("D is for dog and dogs are brown"). However, most studies of synesthesia use only English-speaking synesthetes. Here, we measure the effect of color terms, semantic associations, and non-linguistic shape-color associations on synesthetic associations in Dutch, English, Greek, Japanese, Korean, Russian, and Spanish. The effect size of linguistic influences (color terms, semantic associations) differed significantly between languages. In contrast, the effect size of non-linguistic influences (shape-color associations), which we predicted to be universal, indeed did not differ between languages. We conclude that language matters (outcomes are influenced by the synesthete's language) and that synesthesia offers an exceptional opportunity to study influences on letter representations in different languages.

Combined effects of motor response, sensory modality, and stimulus intensity on temporal reproduction.

The ability to estimate a filled interval of time is affected by numerous non-temporal factors, such as the sensory modality, duration, and the intensity of the stimulus. Here we explore the role of modality (auditory or visual), stimulus intensity (low vs. high), and motor response speed on the ability to reproduce the duration of short (<1 s) filled intervals. In accordance with the literature, the reproduced duration was affected by both the modality and the intensity of the stimulus; longer reproduction times were generally observed for visual as compared to auditory stimuli, and for low as compared to high-intensity stimuli. We used general estimating equations in order to determine whether these factors independently affected participants' ability to reproduce a given duration, after eliminating the variability associated with reaction time, since it covaried with the reproduced durations. This analysis revealed that stimulus duration, modality, and intensity were all significant independent predictors of the reproduced durations. Additionally, duration interacted with intensity when reproducing auditory intervals. That is, after taking into account the general speeding-up effect that high-intensity stimuli have on responses, they seem to have an additional effect on the rate of the internal clock. These results support previous evidence suggesting that auditory and visual clocks run at different speeds.

The perception and passage of time during public speaking.

Several studies have shown that anxious individuals experience a slower passage of time under threat conditioning. Anxiety-evoking situations have also been proposed to elevate arousal levels, which, in turn, alter one's time percept. However, the effect of social stressors on time perception remains significantly neglected. The current research aimed to investigate the impact of anxiety levels on time estimation and passage of time judgments during public speaking in healthy adults. Participants were recruited from a pool of students that had to give a presentation as part of a university course or their teaching duties. Following the presentation, they were asked to make retrospective time estimations on the duration of the latter, as well as to provide passage of time judgments. Self-reported questionnaires related to affective states, public speaking anxiety, and performance were also administered. Analysis showed that higher levels of public speaking anxiety predicted temporal overestimation and slower "feel" duration and passage of time. Moreover, the relationship between public speaking anxiety and passage of time was mediated by participants' mood states, which remained significant after -indirectly- controlling for fear of evaluation. Overall, our observations suggest that anxiety levels during public presentation significantly predict altered perception and experience of time. The latter can be explained by the speaker's mood status. Identifying the mechanisms that modulate timing under psychological stressors could complement our understanding regarding their impact on educational and social settings, as well as set the ground towards the development of early intervention and prevention strategies for those who suffer from stress-related disorders.

Nameability effects and short-term memory limitations on order perception and enumeration of brief sounds.

Research has shown that perceiving the order of successive auditory stimuli could be affected by their nameability. The present research re-examined this hypothesis, using tasks requiring participants to report the order of successively presented (with no interstimulus gaps) environmental (i.e., easily named stimuli) and abstract (i.e., hard-to-name stimuli) sounds of short duration (i.e., 200 ms). Using the same sequences, we also examined the accuracy of the sounds perceived by administering enumeration tasks. Data analyses showed that accuracy in the ordering tasks was equally low for both environmental and abstract sounds, whereas accuracy in the enumeration tasks was higher for the former as compared to the latter sounds. Importantly, overall accuracy in the enumeration tasks did not reach ceiling levels, suggesting some limitations in the perception of successively presented stimuli. Overall, naming fluency seemed to affect sound enumeration, but no effects were obtained for order perception. Furthermore, an effect of each sound's location in a sequence on ordering accuracy was noted. Our results question earlier notions suggesting that order perception is mediated by stimuli's nameability and leave open the possibility that memory capacity limits may play a role.

Time perception in film viewing: A modulation of scene's duration estimates as a function of film editing.

Filmmakers and editors have empirically developed techniques to ensure the spatiotemporal continuity of a film's narration. In terms of time, editing techniques (e.g., elliptical, overlapping, or cut minimization) allow for the manipulation of the perceived duration of events as they unfold on screen. More specifically, a scene can be edited to be time compressed, expanded, or real-time in terms of its perceived duration. Despite the consistent application of these techniques in filmmaking, their perceptual outcomes have not been experimentally validated. Given that viewing a film is experienced as a precise simulation of the physical world, the use of cinematic material to examine aspects of time perception allows for experimentation with high ecological validity, while filmmakers gain more insight on how empirically developed techniques influence viewers' time percept. Here, we investigated how such time manipulation techniques of an action affect a scene's perceived duration. Specifically, we presented videos depicting different actions (e.g., a woman talking on the phone), edited according to the techniques applied for temporal manipulation and asked participants to make verbal estimations of the presented scenes' perceived durations. Analysis of data revealed that the duration of expanded scenes was significantly overestimated as compared to that of compressed and real-time scenes, as was the duration of real-time scenes as compared to that of compressed scenes. Therefore, our results validate the empirical techniques applied for the modulation of a scene's perceived duration. We also found interactions on time estimates of scene type and editing technique as a function of the characteristics and the action of the scene presented. Thus, these findings add to the discussion that the content and characteristics of a scene, along with the editing technique applied, can also modulate perceived duration. Our findings are discussed by considering current timing frameworks, as well as attentional saliency algorithms measuring the visual saliency of the presented stimuli.

Multisensory integration deficits in Schizophrenia and Autism evidenced in behaviour but not event related potentials.

The process of integrating information from different sensory channels, known as multisensory integration (MSI) was assessed in two disorders, Autism Spectrum Disorder (ASD) and Schizophrenia (SCZ). 32 healthy controls (HC), 35 SCZ patients, and 23 ASD patients performed an audiovisual (AV) synchronous target detection task while reaction time (RT) and scalp recorded electrophysiological (EEG) activity were measured. MSI in the AV condition resulted in faster and less variable RTs compared to the unimodal conditions. Using our novel bootstrap method, MSI gain was observed in 78 % of HC, 26 % of ASD, and 48 % of SCZ patients. At the neural level, MSI in the AV condition resulted in larger amplitude of sensory evoked responses and cognitive P3 response compared to the corresponding unimodal conditions. These neural effects of MSI were not related to the behavioural MSI gain identified at the individual level and could not explain the deficits in behavioural MSI of patient groups. In conclusion, a robust MSI gain deficit in RT was observed in both patient groups that was not reflected in early perceptual and cognitive electro-cortical responses, suggesting that behavioural MSI deficits in ASD and SCZ may arise at late processing stages such as response selection.

Attention and schema violations of real world scenes differentially modulate time perception.

In the real world, object arrangement follows a number of rules. Some of the rules pertain to the spatial relations between objects and scenes (i.e., syntactic rules) and others about the contextual relations (i.e., semantic rules). Research has shown that violation of semantic rules influences interval timing with the duration of scenes containing such violations to be overestimated as compared to scenes with no violations. However, no study has yet investigated whether both semantic and syntactic violations can affect timing in the same way. Furthermore, it is unclear whether the effect of scene violations on timing is due to attentional or other cognitive accounts. Using an oddball paradigm and real-world scenes with or without semantic and syntactic violations, we conducted two experiments on whether time dilation will be obtained in the presence of any type of scene violation and the role of attention in any such effect. Our results from Experiment 1 showed that time dilation indeed occurred in the presence of syntactic violations, while time compression was observed for semantic violations. In Experiment 2, we further investigated whether these estimations were driven by attentional accounts, by utilizing a contrast manipulation of the target objects. The results showed that an increased contrast led to duration overestimation for both semantic and syntactic oddballs. Together, our results indicate that scene violations differentially affect timing due to violation processing differences and, moreover, their effect on timing seems to be sensitive to attentional manipulations such as target contrast.

Exposure to multisensory and visual static or moving stimuli enhances processing of nonoptimal visual rhythms.

Research has shown that visual moving and multisensory stimuli can efficiently mediate rhythmic information. It is possible, therefore, that the previously reported auditory dominance in rhythm perception is due to the use of nonoptimal visual stimuli. Yet it remains unknown whether exposure to multisensory or visual-moving rhythms would benefit the processing of rhythms consisting of nonoptimal static visual stimuli. Using a perceptual learning paradigm, we tested whether the visual component of the multisensory training pair can affect processing of metric simple two integer-ratio nonoptimal visual rhythms. Participants were trained with static (AVstat), moving-inanimate (AVinan), or moving-animate (AVan) visual stimuli along with auditory tones and a regular beat. In the pre- and posttraining tasks, participants responded whether two static-visual rhythms differed or not. Results showed improved posttraining performance for all training groups irrespective of the type of visual stimulation. To assess whether this benefit was auditory driven, we introduced visual-only training with a moving or static stimulus and a regular beat (Vinan). Comparisons between Vinan and Vstat showed that, even in the absence of auditory information, training with visual-only moving or static stimuli resulted in an enhanced posttraining performance. Overall, our findings suggest that audiovisual and visual static or moving training can benefit processing of nonoptimal visual rhythms.

The Blursday database as a resource to study subjective temporalities during COVID-19.

The COVID-19 pandemic and associated lockdowns triggered worldwide changes in the daily routines of human experience. The Blursday database provides repeated measures of subjective time and related processes from participants in nine countries tested on 14 questionnaires and 15 behavioural tasks during the COVID-19 pandemic. A total of 2,840 participants completed at least one task, and 439 participants completed all tasks in the first session. The database and all data collection tools are accessible to researchers for studying the effects of social isolation on temporal information processing, time perspective, decision-making, sleep, metacognition, attention, memory, self-perception and mindfulness. Blursday includes quantitative statistics such as sleep patterns, personality traits, psychological well-being and lockdown indices. The database provides quantitative insights on the effects of lockdown (stringency and mobility) and subjective confinement on time perception (duration, passage of time and temporal distances). Perceived isolation affects time perception, and we report an inter-individual central tendency effect in retrospective duration estimation.

Opportunities and Limitations of Mobile Neuroimaging Technologies in Educational Neuroscience.

As the field of educational neuroscience continues to grow, questions have emerged regarding the ecological validity and applicability of this research to educational practice. Recent advances in mobile neuroimaging technologies have made it possible to conduct neuroscientific studies directly in naturalistic learning environments. We propose that embedding mobile neuroimaging research in a cycle (Matusz, Dikker, Huth, & Perrodin, 2019), involving lab-based, seminaturalistic, and fully naturalistic experiments, is well suited for addressing educational questions. With this review, we take a cautious approach, by discussing the valuable insights that can be gained from mobile neuroimaging technology, including electroencephalography and functional near-infrared spectroscopy, as well as the challenges posed by bringing neuroscientific methods into the classroom. Research paradigms used alongside mobile neuroimaging technology vary considerably. To illustrate this point, studies are discussed with increasingly naturalistic designs. We conclude with several ethical considerations that should be taken into account in this unique area of research.

Audiovisual temporal adaptation of speech: temporal order versus simultaneity judgments.

The temporal perception of simple auditory and visual stimuli can be modulated by exposure to asynchronous audiovisual speech. For instance, research using the temporal order judgment (TOJ) task has shown that exposure to temporally misaligned audiovisual speech signals can induce temporal adaptation that will influence the TOJs of other (simpler) audiovisual events (Navarra et al. (2005) Cognit Brain Res 25:499-507). Given that TOJ and simultaneity judgment (SJ) tasks appear to reflect different underlying mechanisms, we investigated whether adaptation to asynchronous speech inputs would also influence SJ task performance. Participants judged whether a light flash and a noise burst, presented at varying stimulus onset asynchronies, were simultaneous or not, or else they discriminated which of the two sensory events appeared to have occurred first. While performing these tasks, participants monitored a continuous speech stream for target words that were either presented in synchrony, or with the audio channel lagging 300 ms behind the video channel. We found that the sensitivity of participant's TOJ and SJ responses was reduced when the background speech stream was desynchronized. A significant modulation of the point of subjective simultaneity (PSS) was also observed in the SJ task but, interestingly, not in the TOJ task, thus supporting previous claims that TOJ and SJ tasks may tap somewhat different aspects of temporal perception.

Evaluating the influence of the 'unity assumption' on the temporal perception of realistic audiovisual stimuli.

Vatakis, A. and Spence, C. (in press) [Crossmodal binding: Evaluating the 'unity assumption' using audiovisual speech stimuli. Perception &Psychophysics] recently demonstrated that when two briefly presented speech signals (one auditory and the other visual) refer to the same audiovisual speech event, people find it harder to judge their temporal order than when they refer to different speech events. Vatakis and Spence argued that the 'unity assumption' facilitated crossmodal binding on the former (matching) trials by means of a process of temporal ventriloquism. In the present study, we investigated whether the 'unity assumption' would also affect the binding of non-speech stimuli (video clips of object action or musical notes). The auditory and visual stimuli were presented at a range of stimulus onset asynchronies (SOAs) using the method of constant stimuli. Participants made unspeeded temporal order judgments (TOJs) regarding which modality stream had been presented first. The auditory and visual musical and object action stimuli were either matched (e.g., the sight of a note being played on a piano together with the corresponding sound) or else mismatched (e.g., the sight of a note being played on a piano together with the sound of a guitar string being plucked). However, in contrast to the results of Vatakis and Spence's recent speech study, no significant difference in the accuracy of temporal discrimination performance for the matched versus mismatched video clips was observed. Reasons for this discrepancy are discussed.

Facilitation of multisensory integration by the "unity effect" reveals that speech is special.

Whenever two or more sensory inputs are highly consistent in one or more dimension(s), observers will be more likely to perceive them as a single multisensory event rather than as separate unimodal events. For audiovisual speech, but not for other noncommunicative events, participants exhibit a "unity effect," whereby they are less sensitive to temporal asynchrony (i.e., that are more likely to bind the multisensory signals together) for matched (than for mismatched) speech events. This finding suggests that the modulation of multisensory integration by the unity effect in humans may be specific to speech. To test this hypothesis directly, we investigated whether the unity effect would also influence the multisensory integration of vocalizations from another primate species, the rhesus monkey. Human participants made temporal order judgments for both matched and mismatched audiovisual stimuli presented at a range of stimulus-onset asynchronies. The unity effect was examined with (1) a single call-type across two different monkeys, (2) two different call-types from the same monkey, (3) human versus monkey "cooing," and (4) speech sounds produced by a male and a female human. The results show that the unity effect only influenced participants' performance for the speech stimuli; no effect was observed for monkey vocalizations or for the human imitations of monkey calls. These findings suggest that the facilitation of multisensory integration by the unity effect is specific to human speech signals.

"Walking" through the sensory, cognitive, and temporal degradations of healthy aging.

As we age, there is a wide range of changes in motor, sensory, cognitive, and temporal processing due to alterations in the functioning of the central nervous and musculoskeletal systems. Specifically, aging is associated with degradations in gait; altered processing of the individual sensory systems; modifications in executive control, memory, and attention; and changes in temporal processing. These age-related alterations are often inter-related and have been suggested to result from shared neural substrates. Additionally, the overlap between these brain areas and those controlling walking raises the possibility of facilitating performance in several tasks by introducing protocols that can efficiently target all four domains. Attempts to counteract these negative effects of normal aging have been focusing on research to prevent falls and/or enhance cognitive processes, while ignoring the potential multisensory benefits accompanying old age. Research shows that the aging brain tends to increasingly rely on multisensory integration to compensate for degradations in individual sensory systems and for altered neural functioning. This review covers the age-related changes in the above-mentioned domains and the potential to exploit the benefits associated with multisensory integration in aging so as to improve one's mobility and enhance sensory, cognitive, and temporal processing.

Robust intentional binding for causally-linked sequences of naturalistic events but not for abstract event sequences.

Past studies have shown that when a voluntary action produces a sensory effect, the action and the effect will be perceived as being closer in time. This subjective temporal 'attraction' is known as intentional binding (IB). Induction of IB is dependent on the intentionality of one's actions, the predictability of the effect, and the causality between the action and the effect. Previous investigations of IB have utilized abstract stimuli (e.g., flashes and beeps) with adaptation so as to associate the abstract action-effect link. Yet, events from our everyday experiences already have an inherent action-effect link. We, thus, investigated, for the first time, IB under naturalistic, multisensory stimulation by manipulating the intentionality, predictability, and causal event link. A total of five experiments without adaptation were conducted examining IB with: abstract stimuli (Experiment 1), naturalistic effects (Exp. 2), naturalistic action cue and effect matching (Exp. 3), naturalistic action cue and effect mismatching (Exp. 4), and naturalistic action cue and effect matching but mismatched response mapping (Exp. 5). Analyses of the data showed the absence of IB for abstract stimuli without action-effect adaptation (Exp. 1) and for effects that were not inherently causal or predictable of one's action (Exp. 2, 4, and 5). IB, however, was induced when the naturalistic sequence of action cue-effect was casually linked and predictable in terms of timing (Exp. 3). Overall, our results showed that induction of IB is dependent on the inherent causal and predictable association of an event from the cue to act to the consequence of that action, an association that is already present in everyday multisensory events.

How 'special' is the human face? Evidence from an audiovisual temporal order judgment task.

Upright and inverted audiovisual video clips of a monkey producing a 'coo' and a human imitating this vocalization were presented at a range of stimulus onset asynchronies. Participants made temporal order judgments regarding which modality stream appeared to have been presented first. The results showed that inverting the dynamic human visual display led to a significant differences in the point of subjective simultaneity, with the inverted human faces requiring more time to be processed compared with the upright displays. No such inversion effect was found for the monkey visual displays. These results demonstrate that the effect of inversion on the temporal perception of audiovisual speech stimuli are driven by the viewing of a human face rather than by the integration of audiovisual speech.

Temporal recalibration during asynchronous audiovisual speech perception.

We investigated the consequences of monitoring an asynchronous audiovisual speech stream on the temporal perception of simultaneously presented vowel-consonant-vowel (VCV) audiovisual speech video clips. Participants made temporal order judgments (TOJs) regarding whether the speech-sound or the visual-speech gesture occurred first, for video clips presented at various different stimulus onset asynchronies. Throughout the experiment, half of the participants also monitored a continuous stream of words presented audiovisually, superimposed over the VCV video clips. The continuous (adapting) speech stream could either be presented in synchrony, or else with the auditory stream lagging by 300 ms. A significant shift (13 ms in the direction of the adapting stimulus in the point of subjective simultaneity) was observed in the TOJ task when participants monitored the asynchronous speech stream. This result suggests that the consequences of adapting to asynchronous speech extends beyond the case of simple audiovisual stimuli (as has recently been demonstrated by Navarra et al. in Cogn Brain Res 25:499-507, 2005) and can even affect the perception of more complex speech stimuli.