Modal and amodal cognition: an overarching principle in various domains of psychology.

Accounting for how the human mind represents the internal and external world is a crucial feature of many theories of human cognition. Central to this question is the distinction between modal as opposed to amodal representational formats. It has often been assumed that one but not both of these two types of representations underlie processing in specific domains of cognition (e.g., perception, mental imagery, and language). However, in this paper, we suggest that both formats play a major role in most cognitive domains. We believe that a comprehensive theory of cognition requires a solid understanding of these representational formats and their functional roles within and across different domains of cognition, the developmental trajectory of these representational formats, and their role in dysfunctional behavior. Here we sketch such an overarching perspective that brings together research from diverse subdisciplines of psychology on modal and amodal representational formats so as to unravel their functional principles and their interactions.

Is there hierarchical generalization in response-effect learning?

Ideomotor theory is an influential approach to understand goal-directed behavior. In this framework, response-effect (R-E) learning is assumed as a prerequisite for voluntary action: Once associations between motor actions and their effects in the environment have been formed, the anticipation of these effects will automatically activate the associated motor pattern. R-E learning is typically investigated with (induction) experiments that comprise an acquisition phase, where R-E associations are presumably learned, and a subsequent test phase, where the previous effects serve as stimuli for a response. While most studies used stimuli in the test phase that were identical to the effects in the acquisition phase, one study reported generalization from exemplars to their superordinate category (Hommel et al., Vis Cogn 10:965-986, 2003, Exp. 1). However, studies on so-called R-E compatibility did not report such generalization. We aimed to conceptually replicate Experiment 1 of Hommel et al. (Vis Cogn 10:965-986, 2003) with a free-choice test phase. While we did observe effects consistent with R-E learning when the effects in the acquisition phase were identical to the stimuli in the test phase, we did not observe evidence for generalization. We discuss this with regard to recent studies suggesting that individual response biases might rather reflect rapidly inferred propositional knowledge instead of learned R-E associations.

Neural mechanisms of response priming do not support veridical unconscious processing.

Studies using the priming paradigm often infer that unconscious processes have more veridical access to the world than conscious processes. These interpretations are based on a standard reasoning that erroneously infers good sensitivity of indirect measures from a clear priming effect. To correct for this fallacy, researchers should explicitly compute the sensitivities from indirect measures and compare them against the sensitivities of direct measures. Recent results suggest that indirect behavioral measures are not more sensitive than direct measures and challenge interpretations about veridical unconscious processing. We add to these behavioral results by focusing on neurophysiological indirect measures. In two EEG experiments, we investigate whether event related potentials (ERPs) are more sensitive to different visual stimuli than direct measures. The results show the opposite effect: higher sensitivities for direct than indirect measures. Therefore-contrasting commonly held belief-we find no evidence for more veridical unconscious than conscious processes in ERP measures.

Grasping follows Weber's law: How to use response variability as a proxy for JND.

Weber's law is a fundamental psychophysical principle. It states that the just noticeable difference (JND) between stimuli increases with stimulus magnitude; consequently, larger stimuli should be estimated with larger variability. However, visually guided grasping seems to violate this expectation: When repeatedly grasping large objects, the variability is similar to that when grasping small objects. Based on this result, it was often concluded that grasping violated Weber's law. This astonishing finding generated a flurry of research, with contradictory results and potentially far-reaching implications for theorizing about the functional architecture of the brain. We show that previous studies ignored nonlinearities in the scaling of the grasping response. These nonlinearities result from, for example, the finger span being limited such that the opening of the fingers reaches a ceiling for large objects. We describe how to mathematically take these nonlinearities into account and apply this approach to our own data, as well as to the data of three influential studies on this topic. In all four datasets, we found that-when appropriately estimated-JNDs increase with object size, as expected by Weber's law. We conclude that grasping obeys Weber's law, as do essentially all sensory dimensions.

Garner-Interference in left-handed awkward grasping.

The Perception-Action Model (PAM) claims to provide a coherent interpretation of data from all areas of the visual neurosciences, most notably data from neuropsychological patients and from behavioral experiments in healthy people. Here, we tested two claims that are part of the core version of the PAM: (a) certain actions (natural, highly practiced, and right-handed) are controlled by the dorsal vision for action pathway, while other actions (awkward, unpracticed, or left-handed) are controlled by the ventral vision for perception pathway. (b) Only the dorsal pathway operates in an analytical fashion, being able to selectively focus on the task-relevant dimension of an object (Ganel and Goodale, Nature 426(6967):664-667, 2003). We show that one of these claims must be wrong: using the same test for analytical processing as Ganel and Goodale (2003), we found that even an action that should clearly be ventral (left-handed awkward grasping) shows analytical processing just as a dorsal task does (right-handed natural precision grasping). These results are at odds with the PAM and point to an inconsistency of the model.

Learning and transfer of response-effect relations.

Acting means changing the environment according to one's own goals, and this often requires bodily movements as responses. How these responses are selected is a central question in contemporary cognitive psychology. The ideomotor principle offers a simple answer based on two assumptions: An agent first learns an association between a response and its effects. Later, this association can be used in a reverse way: When the agent wants to achieve a desired effect and activates its representation, the associated response representation becomes activated as well. This reversed use of the learned association is considered the means to select the required response. In three experiments, we addressed two questions related to the first assumption: First, we tested whether effect representations generalise to more abstract conceptual knowledge. This is important, because outside the laboratory and in novel situations, effects are variable and not always exactly identical, such that generalisation is necessary for successful actions. Second, the nature of the response-effect relation has been debated recently, and more data are necessary to put theorising on firm empirical ground. Results of our experiments suggest that (a) abstraction to conceptual knowledge seems to occur only under very restricted situations, and (b) it seems that no (implicit) associations between responses and effects are learned, but rather (explicit) propositional knowledge in the form of rules.

Human grasp point selection.

When we grasp an object, our visuomotor system has to solve an intricate problem: how to find the best out of an infinity of possible contact points of the fingers with the object? The contact point selection model (CoPS) we present here solves this problem and predicts human grasp point selection in precision grip grasping by combining a few basic rules that have been identified in human and robotic grasping. Usually, not all of the rules can be perfectly satisfied. Therefore, we assessed their relative importance by creating simple stimuli that put them into conflict with each other in pairs. Based on these conflict experiments we made model-based grasp point predictions for another experiment with a novel set of complexly shaped objects. The results show that our model predicts the human choice of grasp points very well, and that observers' preferences for their natural grasp angles is as important as physical stability constraints. Incorporating a human grasp point selection model like the one presented here could markedly improve current approaches to cortically guided arm and hand prostheses by making movements more natural while also allowing for a more efficient use of the available information.

No evidence for contextual cueing beyond explicit recognition.

Many studies claim that visual regularities can be learned unconsciously and without explicit awareness. For example in the contextual cueing paradigm, studies often make claims using a standard reasoning based on two results: (1) a reliable response time (RT) difference between repeated vs. new stimulus displays and (2) a close-to-chance sensitivity when participants are asked to explicitly recognize repeated stimulus displays. From this pattern of results, studies routinely conclude that the sensitivity of RT responses is higher than that of explicit responses-an empirical situation we call Indirect Task Advantage (ITA). Many studies further infer from an ITA that RT effects were driven by a form of recognition that exceeds explicit memory: implicit recognition. However, this reasoning is flawed because the sensitivity underlying RT effects is never computed. To properly establish a difference, a sensitivity comparison is required. We apply this sensitivity comparison in a reanalysis of 20 contextual cueing studies showing that not a single study provides consistent evidence for ITAs. Responding to recent correlation-based arguments, we also demonstrate the absence of evidence for ITAs at the level of individual participants. This lack of ITAs has serious consequences for the field: If RT effects can be fully explained by weak but above-chance explicit recognition sensitivity, what is the empirical content of the label "implicit"? Thus, theoretical discussions in this paradigm-and likely in other paradigms using this standard reasoning-require serious reassessment because the current data from contextual cueing studies is insufficient to consider recognition as implicit.

Object processing in visual perception and action in children and adults.

We investigated whether 6- and 7-year-olds and 9- and 10-year-olds, as well as adults, process object dimensions independent of or in interaction with one another in a perception and action task by adapting Ganel and Goodale's method for testing adults (Nature, 2003, Vol. 426, pp. 664-667). In addition, we aimed to confirm Ganel and Goodale's results in adults to reliably compare their processing strategies with those of children. Specifically, we tested the abilities of children and adults to perceptually classify (perception task) or grasp (action task) the width of a rectangular object while ignoring its length. We found that adults process object dimensions in interaction with one another in visual perception but independent of each other in action, thereby replicating Ganel and Goodale's results. Children processed object dimensions interactively in visual perception, and there was also some evidence for interactive processing in action. Possible reasons for these differences in object processing between children and adults are discussed.

Advancing research on unconscious priming: When can scientists claim an indirect task advantage?

Current literature holds that many cognitive functions can be performed outside consciousness. Evidence for this view comes from unconscious priming. In a typical experiment, visual stimuli are masked such that participants are close to chance performance when directly asked to which of two categories the stimuli belong. This close-to-zero sensitivity is seen as evidence that participants cannot consciously report the category of the masked stimuli. Nevertheless, the category of the masked stimuli can indirectly affect responses to other stimuli (e.g., reaction times or brain activity)-an effect called priming. The priming effect is seen as evidence for a higher sensitivity to the masked stimuli in the indirect responses as compared with the direct responses. Such an apparent difference in sensitivities is taken as evidence that processing occurred unconsciously. But we show that this "standard reasoning of unconscious priming" is flawed: Sensitivities are not properly compared, creating the wrong impression of a difference in sensitivities even if there is none. We describe the appropriate way to determine sensitivities, replicate the behavioral part of a landmark study, develop methods to estimate sensitivities from reported summary statistics of published studies, and use these methods to reanalyze 15 highly influential studies. Results show that the interpretations of many studies need to be changed and that a community effort is required to reassess the vast literature on unconscious priming. This process will allow scientists to learn more about the true boundary conditions of unconscious priming, thereby advancing the scientific understanding of consciousness. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Working memory maintenance of grasp-target information in the human posterior parietal cortex.

Event-related functional magnetic resonance imaging was applied to identify cortical areas involved in maintaining target information in working memory used for an upcoming grasping action. Participants had to grasp with their thumb and index finger of the dominant right hand three-dimensional objects of different size and orientation. Reaching-to-grasp movements were performed without visual feedback either immediately after object presentation or after a variable delay of 2-12 s. The right inferior parietal cortex demonstrated sustained neural activity throughout the delay, which overlapped with activity observed during encoding of the grasp target. Immediate and delayed grasping activated similar motor-related brain areas and showed no differential activity. The results suggest that the right inferior parietal cortex plays an important functional role in working memory maintenance of grasp-related information. Moreover, our findings confirm the assumption that brain areas engaged in maintaining information are also involved in encoding the same information, and thus extend previous findings on working memory function of the posterior parietal cortex in saccadic behavior to reach-to-grasp movements.

Card posting does not rely on visual orientation: A challenge to past neuropsychological dissociations.

A common set of tasks frequently employed in the neuropsychological assessment of patients with visuomotor or perceptual deficits are the card-posting and the perceptual orientation matching tasks. In the posting task, patients have to post a card (or their hand) through a slot of varying orientations while the matching task requires them to indicate the slot's orientation as accurately as possible. Observations that damage to different areas of the brain (dorsal vs. ventral stream) is associated with selective impairment in one of the tasks - but not the other - has led to the suggestion that different cortical pathways process visual orientation information for perception versus action. In three experiments, we show that this conclusion may be premature as posting does not seem to rely on the processing of visual orientation information but is instead performed using obstacle avoidance strategies that require an accurate judgement of egocentric distances between the card's and the slot's edges. Specifically, we found that while matching is susceptible to the oblique effect (i.e., common perceptual orientation bias with higher accuracy for cardinal than oblique orientations), this was not the case for posting, neither in immediate nor in memory-guided conditions. In contrast to matching, posting errors primarily depended on biomechanical demands and reflected a preference for performing efficient and comfortable movements. Thus, we suggest that previous dissociations between perceptual and visuomotor performance in letter posting tasks are better explained by impairments in egocentric and allocentric spatial processing than by independent visual processing systems.

Group decisions based on confidence weighted majority voting.

BACKGROUND: It has repeatedly been reported that, when making decisions under uncertainty, groups outperform individuals. Real groups are often replaced by simulated groups: Instead of performing an actual group discussion, individual responses are aggregated by a numerical computation. While studies have typically used unweighted majority voting (MV) for this aggregation, the theoretically optimal method is confidence weighted majority voting (CWMV)-if independent and accurate confidence ratings from the individual group members are available. To determine which simulations (MV vs. CWMV) reflect real group processes better, we applied formal cognitive modeling and compared simulated group responses to real group responses. RESULTS: Simulated group decisions based on CWMV matched the accuracy of real group decisions, while simulated group decisions based on MV showed lower accuracy. CWMV predicted the confidence that groups put into their group decisions well. However, real groups treated individual votes to some extent more equally weighted than suggested by CWMV. Additionally, real groups tend to put lower confidence into their decisions compared to CWMV simulations. CONCLUSION: Our results highlight the importance of taking individual confidences into account when simulating group decisions: We found that real groups can aggregate individual confidences in a way that matches statistical aggregations given by CWMV to some extent. This implies that research using simulated group decisions should use CWMV instead of MV as a benchmark to compare real groups to.

Number processing outside awareness? Systematically testing sensitivities of direct and indirect measures of consciousness.

In priming research, it is often argued that humans can discriminate stimuli outside consciousness. For example, the semantic meaning of numbers can be processed even when the numbers are so strongly masked that participants are not aware of them. These claims are typically based on a certain pattern of results: Direct measures indicate no conscious awareness of the masked stimuli, while indirect measures show clear priming effects of the same stimuli on reaction times or neurophysiological measures. From this pattern, preserved (unconscious) processing in the indirect task is concluded. However, this widely used standard reasoning is problematic and leads to spurious claims of unconscious processing. Such problems can be avoided by comparing sensitivities of direct and indirect measures. Many studies are affected by these problems, such that a reassessment of the literature is needed. Here, we investigated whether numbers can be processed unconsciously. In three experiments, we replicated and extended well-established effects of number priming over a wide range of stimulus visibilities. We then compared the standard reasoning to a sensitivity analysis, where direct and indirect effects are compared using the same metric. Results show that the sensitivities of indirect measures did not exceed those of direct measures, thereby indicating no evidence for preserved unconscious processing when awareness of the stimuli is low. Instead, it seems that at low visibility there is residual processing that affects direct and indirect measures to a similar degree. This suggests that similar processing modes cause those effects in direct and indirect measures.

Grasping isoluminant stimuli.

We used a virtual reality setup to let participants grasp discs, which differed in luminance, chromaticity and size. Current theories on perception and action propose a division of labor in the brain into a color proficient perception pathway and a less color-capable action pathway. In this study, we addressed the question whether isoluminant stimuli, which provide only a chromatic but no luminance contrast for action planning, are harder to grasp than stimuli providing luminance contrast or both kinds of contrast. Although we found that grasps of isoluminant stimuli had a slightly steeper slope relating the maximum grip aperture to disc size, all other measures of grip quality were unaffected. Overall, our results do not support the view that isoluminance of stimulus and background impedes the planning of a grasping movement.

Corrective processes in grasping after perturbations of object size.

Researchers proposed that humans may achieve grip adaptation to a new object size by reprogramming and substituting the initially planned motor program. The authors investigated corrective processes in grasping by using a size perturbation paradigm. In 3 experiments, they investigated how grip adjustments are influenced by different perturbation times (early or late), the visibility of the moving hand, and different perturbation sizes (small or large). Results indicated that individuals execute corrections faster after late perturbations. The availability of visual information about the hand had minimal effect on the corrections, suggesting that feedforward mechanisms are involved. Moreover, participants achieved adjustments mainly by smooth changes of the aperture over time, contradicting the researchers' assumption that a new movement is programmed and superimposed.

Differences in fixations between grasping and viewing objects.

Where exactly do people look when they grasp an object? An object is usually contacted at two locations, whereas the gaze can only be at one location at the time. We investigated participants' fixation locations when they grasp objects with the contact positions of both index finger and thumb being visible and compared these to fixation locations when they only viewed the objects. Participants grasped with the index finger at the top and the thumb at the bottom of a flat shape. The main difference between grasping and viewing was that after a saccade roughly directed to the object's center of gravity, participants saccaded more upward and more into the direction of a region that was difficult to contact during grasping. A control experiment indicated that it was not the upper part of the shape that attracted fixation, while the results were consistent with an attraction by the index finger. Participants did not try to fixate both contact locations. Fixations were closer to the object's center of gravity in the viewing than in the grasping task. In conclusion, participants adapt their eye movements to the need of the task, such as acquiring information about regions with high required contact precision in grasping, even with small (graspable) objects. We suggest that in grasping, the main function of fixations is to acquire visual feedback of the approaching digits.

Grasping visual illusions: Consistent data and no dissociation.

The finding that the Ebbinghaus/Titchener illusion deceives perception but not grasping is usually seen as strong evidence for Goodale and Milner's (1992) notion of two parallel visual systems, one being conscious and deceived by the illusion (vision-for-perception) and the other being unconscious and not deceived (vision-for-action). However, this finding is controversial and led to studies with seemingly contradictory results. We argue that these results are not as contradictory as it might seem. Instead, studies consistently show similar effects of the illusion on grasping. The perceptual effects are strongly dependent on the specific perceptual measure employed. If, however, some methodological precautions are used, then these diverse perceptual results can be reconciled and point to a single internal size estimate that is used for perception and for grasping. This suggests that the Ebbinghaus illusion deceives a common representation of object size that is used by perception and action.

Effects of the Ebbinghaus illusion on children's perception and grasping.

We investigated the development of the Ebbinghaus illusion in children's perception and grasping. A previous study (Hanisch et al. 2001) had reported negative illusion effects on 5- to 12-year-olds' grasping as compared to their perception. We attempted to replicate this finding and to test different hypotheses based on a direct influence of the context elements on the trajectories of the fingers which could explain this reversal of the illusion effects. For 5- to 7- and 9- to 11-year-olds we observed the classical illusion effects in perception. Illusion effects were perfectly similar for perception and grasping in 9- to 11-year-olds, while there was a non-significant trend toward smaller illusion effects in grasping for the 5- to 7-year-olds. This could be due to a slightly different effect of the illusion on younger children's grasping. However, it seems clear that there are no qualitative changes, as a reversal of the illusion effects in grasping of younger children. Finally, we show that our grasping data conform well to the motor literature for children's grasping, thereby strengthening our conclusions.