Dense Sample Deep Learning.

Deep learning (DL), a variant of the neural network algorithms originally proposed in the 1980s (Rumelhart et al., 1986), has made surprising progress in artificial intelligence (AI), ranging from language translation, protein folding (Jumper et al., 2021), autonomous cars, and, more recently, human-like language models (chatbots). All that seemed intractable until very recently. Despite the growing use of DL networks, little is understood about the learning mechanisms and representations that make these networks effective across such a diverse range of applications. Part of the answer must be the huge scale of the architecture and, of course, the large scale of the data, since not much has changed since 1986. But the nature of deep learned representations remains largely unknown. Unfortunately, training sets with millions or billions of tokens have unknown combinatorics, and networks with millions or billions of hidden units can't easily be visualized and their mechanisms can't be easily revealed. In this letter, we explore these challenges with a large (1.24 million weights VGG) DL in a novel high-density sample task (five unique tokens with more than 500 exemplars per token), which allows us to more carefully follow the emergence of category structure and feature construction. We use various visualization methods for following the emergence of the classification and the development of the coupling of feature detectors and structures that provide a type of graphical bootstrapping. From these results, we harvest some basic observations of the learning dynamics of DL and propose a new theory of complex feature construction based on our results.

Causally informed activity flow models provide mechanistic insight into network-generated cognitive activations.

Brain activity flow models estimate the movement of task-evoked activity over brain connections to help explain network-generated task functionality. Activity flow models have been shown to accurately generate task-evoked brain activations across a wide variety of brain regions and task conditions. However, these models have had limited explanatory power, given known issues with causal interpretations of the standard functional connectivity measures used to parameterize activity flow models. We show here that functional/effective connectivity (FC) measures grounded in causal principles facilitate mechanistic interpretation of activity flow models. We progress from simple to complex FC measures, with each adding algorithmic details reflecting causal principles. This reflects many neuroscientists' preference for reduced FC measure complexity (to minimize assumptions, minimize compute time, and fully comprehend and easily communicate methodological details), which potentially trades off with causal validity. We start with Pearson correlation (the current field standard) to remain maximally relevant to the field, estimating causal validity across a range of FC measures using simulations and empirical fMRI data. Finally, we apply causal-FC-based activity flow modeling to a dorsolateral prefrontal cortex region (DLPFC), demonstrating distributed causal network mechanisms contributing to its strong activation during a working memory task. Notably, this fully distributed model is able to account for DLPFC working memory effects traditionally thought to rely primarily on within-region (i.e., not distributed) recurrent processes. Together, these results reveal the promise of parameterizing activity flow models using causal FC methods to identify network mechanisms underlying cognitive computations in the human brain.

Latent functional connectivity underlying multiple brain states.

Functional connectivity (FC) studies have predominantly focused on resting state, where ongoing dynamics are thought to reflect the brain's intrinsic network architecture, which is thought to be broadly relevant because it persists across brain states (i.e., is state-general). However, it is unknown whether resting state is the optimal state for measuring intrinsic FC. We propose that latent FC, reflecting shared connectivity patterns across many brain states, better captures state-general intrinsic FC relative to measures derived from resting state alone. We estimated latent FC independently for each connection using leave-one-task-out factor analysis in seven highly distinct task states (24 conditions) and resting state using fMRI data from the Human Connectome Project. Compared with resting-state connectivity, latent FC improves generalization to held-out brain states, better explaining patterns of connectivity and task-evoked activation. We also found that latent connectivity improved prediction of behavior outside the scanner, indexed by the general intelligence factor (g). Our results suggest that FC patterns shared across many brain states, rather than just resting state, better reflect state-general connectivity. This affirms the notion of "intrinsic" brain network architecture as a set of connectivity properties persistent across brain states, providing an updated conceptual and mathematical framework of intrinsic connectivity as a latent factor.

Experimental evidence for a child-to-adolescent switch in human amygdala-prefrontal cortex communication: A cross-sectional pilot study.

Interactions between the amygdala and prefrontal cortex are fundamental to human emotion. Despite the central role of frontoamygdala communication in adult emotional learning and regulation, little is known about how top-down control emerges during human development. In the present cross-sectional pilot study, we experimentally manipulated prefrontal engagement to test its effects on the amygdala during development. Inducing dorsal anterior cingulate cortex (dACC) activation resulted in developmentally-opposite effects on amygdala reactivity during childhood versus adolescence, such that dACC activation was followed by increased amygdala reactivity in childhood but reduced amygdala reactivity in adolescence. Bayesian network analyses revealed an age-related switch between childhood and adolescence in the nature of amygdala connectivity with the dACC and ventromedial PFC (vmPFC). Whereas adolescence was marked by information flow from dACC and vmPFC to amygdala (consistent with that observed in adults), the reverse information flow, from the amygdala to dACC and vmPFC, was dominant in childhood. The age-related switch in information flow suggests a potential shift from bottom-up co-excitatory to top-down regulatory frontoamygdala connectivity and may indicate a profound change in the circuitry supporting maturation of emotional behavior. These findings provide novel insight into the developmental construction of amygdala-cortical connections and implications for the ways in which childhood experiences may influence subsequent prefrontal function.

Computations and Connectivity Underlying Aversive Counterfactuals.

Background: Mentally simulating counterfactuals (scenarios that have not actually occurred) is a sophisticated human cognitive function underlying creativity, planning, and daydreaming. One example is the "would you rather" game, in which forced choices are made between outlandish negative counterfactuals. Materials and Methods: We measured behavioral and neural correlates while participants made "would you rather" choices framed as approaching or avoiding aversive counterfactual scenarios (e.g., illnesses, car accidents). Results: We found in two independent cohorts that participants were highly susceptible to framing effects when making these decisions, taking significantly longer to respond to approach frames compared with avoidance. Brain imaging showed that choices to approach and avoid resulted in a pattern of activation consistent with a network associated with responding to aversive stimuli, identified via a coordinate-based meta-analysis of 238 studies. Bayesian graph connectivity analysis showed that network connectivity differed by choice frame, with significantly stronger connectivity for approach choices compared with avoidance choices among primarily limbic nodes (putamen, insula, caudate, and amygdala). Computational modeling of behavior revealed that approach frames led to significantly longer nondecision times, increased evidence required to make decisions, and faster evidence accumulation than avoidance frames. Stronger network connectivity between corticostriatal and limbic regions was associated with rate of evidence accumulation and length of nondecision time during approach choices. For avoidance choices, prefrontal connectivity was related to nondecision time. Conclusions: These results suggest that "would you rather" decisions about aversive counterfactuals differentially recruit limbic circuit connectivity based on choice frame. Impact statement We measured brain connectivity and latent cognitive variables underlying aversive counterfactual choices. We found a replicable reaction time effect whereby approach decisions were slower than avoidance decisions. Computational modeling identified that the latent cognitive variable of evidence accumulation was related to strength of connectivity between corticostriatal and limbic nodes during approach decisions. Multidimensional scaling (MDS) and clustering revealed a three-dimensional choice structure that differed between individuals, and between approach and avoidance choices within individuals. Our results suggest that cognitive evaluations of aversive counterfactuals involve flexible representations that can be altered by choice framing. These findings have broad implications for prospective decision making.

The Stochastic Delta Rule: Faster and More Accurate Deep Learning Through Adaptive Weight Noise.

Multilayer neural networks have led to remarkable performance on many kinds of benchmark tasks in text, speech, and image processing. Nonlinear parameter estimation in hierarchical models is known to be subject to overfitting and misspecification. One approach to these estimation and related problems (e.g., saddle points, colinearity, feature discovery) is called Dropout. The Dropout algorithm removes hidden units according to a binomial random variable with probability p prior to each update, creating random "shocks" to the network that are averaged over updates (thus creating weight sharing). In this letter, we reestablish an older parameter search method and show that Dropout is a special case of this more general model, stochastic delta rule (SDR), published originally in 1990. Unlike Dropout, SDR redefines each weight in the network as a random variable with mean µwij and standard deviation σwij. Each weight random variable is sampled on each forward activation, consequently creating an exponential number of potential networks with shared weights (accumulated in the mean values). Both parameters are updated according to prediction error, thus resulting in weight noise injections that reflect a local history of prediction error and local model averaging. SDR therefore implements a more sensitive local gradient-dependent simulated annealing per weight converging in the limit to a Bayes optimal network. We run tests on standard benchmarks (CIFAR and ImageNet) using a modified version of DenseNet and show that SDR outperforms standard Dropout in top-5 validation error by approximately 13% with DenseNet-BC 121 on ImageNet and find various validation error improvements in smaller networks. We also show that SDR reaches the same accuracy that Dropout attains in 100 epochs in as few as 40 epochs, as well as improvements in training error by as much as 80%.

Advancing functional connectivity research from association to causation.

Cognition and behavior emerge from brain network interactions, such that investigating causal interactions should be central to the study of brain function. Approaches that characterize statistical associations among neural time series-functional connectivity (FC) methods-are likely a good starting point for estimating brain network interactions. Yet only a subset of FC methods ('effective connectivity') is explicitly designed to infer causal interactions from statistical associations. Here we incorporate best practices from diverse areas of FC research to illustrate how FC methods can be refined to improve inferences about neural mechanisms, with properties of causal neural interactions as a common ontology to facilitate cumulative progress across FC approaches. We further demonstrate how the most common FC measures (correlation and coherence) reduce the set of likely causal models, facilitating causal inferences despite major limitations. Alternative FC measures are suggested to immediately start improving causal inferences beyond these common FC measures.

Attentional Bias in Human Category Learning: The Case of Deep Learning.

Category learning performance is influenced by both the nature of the category's structure and the way category features are processed during learning. Shepard (1964, 1987) showed that stimuli can have structures with features that are statistically uncorrelated (separable) or statistically correlated (integral) within categories. Humans find it much easier to learn categories having separable features, especially when attention to only a subset of relevant features is required, and harder to learn categories having integral features, which require consideration of all of the available features and integration of all the relevant category features satisfying the category rule (Garner, 1974). In contrast to humans, a single hidden layer backpropagation (BP) neural network has been shown to learn both separable and integral categories equally easily, independent of the category rule (Kruschke, 1993). This "failure" to replicate human category performance appeared to be strong evidence that connectionist networks were incapable of modeling human attentional bias. We tested the presumed limitations of attentional bias in networks in two ways: (1) by having networks learn categories with exemplars that have high feature complexity in contrast to the low dimensional stimuli previously used, and (2) by investigating whether a Deep Learning (DL) network, which has demonstrated humanlike performance in many different kinds of tasks (language translation, autonomous driving, etc.), would display human-like attentional bias during category learning. We were able to show a number of interesting results. First, we replicated the failure of BP to differentially process integral and separable category structures when low dimensional stimuli are used (Garner, 1974; Kruschke, 1993). Second, we show that using the same low dimensional stimuli, Deep Learning (DL), unlike BP but similar to humans, learns separable category structures more quickly than integral category structures. Third, we show that even BP can exhibit human like learning differences between integral and separable category structures when high dimensional stimuli (face exemplars) are used. We conclude, after visualizing the hidden unit representations, that DL appears to extend initial learning due to feature development thereby reducing destructive feature competition by incrementally refining feature detectors throughout later layers until a tipping point (in terms of error) is reached resulting in rapid asymptotic learning.

Differences in atypical resting-state effective connectivity distinguish autism from schizophrenia.

Autism and schizophrenia share overlapping genetic etiology, common changes in brain structure and common cognitive deficits. A number of studies using resting state fMRI have shown that machine learning algorithms can distinguish between healthy controls and individuals diagnosed with either autism spectrum disorder or schizophrenia. However, it has not yet been determined whether machine learning algorithms can be used to distinguish between the two disorders. Using a linear support vector machine, we identify features that are most diagnostic for each disorder and successfully use them to classify an independent cohort of subjects. We find both common and divergent connectivity differences largely in the default mode network as well as in salience, and motor networks. Using divergent connectivity differences, we are able to distinguish autistic subjects from those with schizophrenia. Understanding the common and divergent connectivity changes associated with these disorders may provide a framework for understanding their shared cognitive deficits.

Survival and Transstadial Persistence of Trypanosoma cruzi in the bed bug (Hemiptera: Cimicidae).

Bed bug populations are increasing around the world at an alarming rate and have become a major public health concern. The appearance of bed bug populations in areas where Chagas disease is endemic raises questions about the role of these insects in the transmission of Trypanosoma cruzi, the etiological agent of the disease. In a series of laboratory evaluations, bed bug adults and nymphs were experimentally fed with T. cruzi-infected blood to assess the ability of T. cruzi to survive inside the bed bug and throughout the insect's molting process. Live T. cruzi were observed in gut contents of experimentally infected bed bug adults via light microscopy and the identity of the parasite was confirmed via polymerase chain reaction analysis. T. cruzi persisted at least 97-d postinfection in adult bed bugs. Nymphal stage bed bugs that were infected with T. cruzi maintained the parasite after molting, indicating that transstadial passage of T. cruzi in bed bugs took place. This report provides further evidence of acquisition, maintenance, and for the first time, transstadial persistence of T. cruzi in bed bugs.

Modeling Causal Relationships among Brain Areas in the Mesocorticolimbic System during Resting-State in Cocaine Users Utilizing a Graph Theoretic Approach.

OBJECTIVE: While effective connectivity (EC, causal interaction) between brain areas has been investigated in chronic users of cocaine as they view cocaine pictures cues, no study has examined EC while they take part in a resting-state scan. This resting-state fMRI study aims to investigate the causal interaction among brain areas in the mesocorticolimbic system (MCLS), which is involved in reward and motivation, in cocaine users (vs. controls). METHOD: Twenty cocaine users and 17 healthy controls finished a structural and a resting-state scan. Mean voxel-based time series data were obtained from brain regions of interest (ROIs) from the MCLS, and were input into a Bayesian search algorithm called IMaGES. RESULTS: The causal interaction pattern was different between the two groups. The feed-forward pattern found in cocaine smokers, between 7 ROIs of the MCLS during resting-state [ventral tegmental area (VTA)→hippocampus (HIPP)→ventral striatum (VenStri)→orbital frontal cortex (OFC), medial frontal cortex (MFC), anterior cingulate cortex (ACC), dorsolateral prefrontal cortex (DLPFC)], was absent in controls. That is, the subcortical VenStri area had a causal influence on four cortical brain areas only in cocaine users. CONCLUSIONS: During the resting-state scan, the VTA of cocaine smokers abstinent for at least 72 hours, but not controls, begins causal connections to limbic, midbrain, and frontal regions in the MCLS in a feed-forward manner. Following replication, further studies may assess if changes over time in EC during resting-state predict cocaine treatment efficacy and outcome.

Back to the future: The return of cognitive functionalism.

The claims that learning systems must build causal models and provide explanations of their inferences are not new, and advocate a cognitive functionalism for artificial intelligence. This view conflates the relationships between implicit and explicit knowledge representation. We present recent evidence that neural networks do engage in model building, which is implicit, and cannot be dissociated from the learning process.

Pediatric Euthanasia and Palliative Care Can Work Together.

Since the Netherlands produced the Groningen protocol describing the methods to be used for pediatric euthanasia and Belgium passed laws authorizing euthanasia for children who consent to it, the issue of pediatric euthanasia has become a relevant topic to discuss. Most rejections of pediatric euthanasia fall into 1 or more of 3 categories, each of which has problems. This article shows how several recent arguments against pediatric euthanasia fail to prove that pediatric euthanasia is unacceptable. It does not follow from this that the practice is permissible but rather that if one is to reject such a practice, stronger arguments will need to be made, especially in countries where adult euthanasia or assisted suicide is already permitted.