Using cognitive psychology to understand GPT-3.

We study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3's decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3's behavior is impressive: It solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multiarmed bandit task, and shows signatures of model-based reinforcement learning. Yet, we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. Taken together, these results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.

Unifying Principles of Generalization: Past, Present, and Future.

Generalization, defined as applying limited experiences to novel situations, represents a cornerstone of human intelligence. Our review traces the evolution and continuity of psychological theories of generalization, from its origins in concept learning (categorizing stimuli) and function learning (learning continuous input-output relationships) to domains such as reinforcement learning and latent structure learning. Historically, there have been fierce debates between approaches based on rule-based mechanisms, which rely on explicit hypotheses about environmental structure, and approaches based on similarity-based mechanisms, which leverage comparisons to prior instances. Each approach has unique advantages: Rules support rapid knowledge transfer, while similarity is computationally simple and flexible. Today, these debates have culminated in the development of hybrid models grounded in Bayesian principles, effectively marrying the precision of rules with the flexibility of similarity. The ongoing success of hybrid models not only bridges past dichotomies but also underscores the importance of integrating both rules and similarity for a comprehensive understanding of human generalization.

Development of oncolytic and gene therapy vectors based on adenovirus serotype 4 as an alternative to adenovirus serotype 5.

BACKGROUND: Adenoviral vectors are among the most frequently used vectors for gene therapy and cancer treatment. Most vectors are derived from human adenovirus (Ad) serotype 5 despite limited applicability caused by pre-existing immunity and unfavorable liver tropism, whereas the other more than 100 known human serotypes remain largely unused. Here, we screened a library of human Ad types and identified Ad4 as a promising candidate vector. METHODS: Reporter-gene-expressing viruses representative of the natural human Ad diversity were used to transduce an array of muscle cell lines and two- or three-dimensional tumor cultures. The time-course of transgene expression was monitored by fluorescence or luminescence measurements. To generate replication-deficient Ad4 vector genomes, successive homologous recombination was applied. RESULTS: Ad4, 17 and 50 transduced human cardiomyocytes more efficiently than Ad5, whereas Ad37 was found to be superior in rhabdomyocytes. Despite its moderate transduction efficiency, Ad4 showed efficient and long-lasting gene expression in papillomavirus (HPV) positive tumor organoids. Therefore, we aimed to harness the potential of Ad4 for improved muscle transduction or oncolytic virotherapy of HPV-positive tumors. We deleted the E1 and E3 transcription units to produce first generation Ad vectors for gene therapy. The E1- and E1/E3-deleted vectors were replication-competent in HEK293 cells stably expressing E1 but not in the other cell lines tested. Furthermore, we show that the Ad5 E1 transcription unit can complement the replication of E1-deleted Ad4 vectors. CONCLUSIONS: Our Ad4-based gene therapy vector platform contributes to the development of improved Ad vectors based on non-canonical serotypes for a broad range of applications.

Task imprinting: Another mechanism of representational change?

Research from several areas suggests that mental representations adapt to the specific tasks we carry out in our environment. In this study, we propose a mechanism of adaptive representational change, task imprinting. Thereby, we introduce a computational model, which portrays task imprinting as an adaptation to specific task goals via selective storage of helpful representations in long-term memory. We test the main qualitative prediction of the model in four behavioral experiments using healthy young adults as participants. In each experiment, we assess participants' baseline representations in the beginning of the experiment, then expose participants to one of two tasks intended to shape representations differently according to our model, and finally assess any potential change in representations. Crucially, the tasks used to measure representations differ in the amount that strategic, judgmental processes play a role. The results of Experiments 1 and 2 allow us to exclude the option that representations used in more perceptual tasks become biased categorically. The results of Experiment 4 make it likely that people strategically decide given the specific task context whether they use categorical information or not. One signature of representational change was however observed: category learning practice increased the perceptual sensitivity over and above mere exposure to the same stimuli.

Preschoolers search longer when there is more information to be gained.

What drives children to explore and learn when external rewards are uncertain or absent? Across three studies, we tested whether information gain itself acts as an internal reward and suffices to motivate children's actions. We measured 24-56-month-olds' persistence in a game where they had to search for an object (animal or toy), which they never find, hidden behind a series of doors, manipulating the degree of uncertainty about which specific object was hidden. We found that children were more persistent in their search when there was higher uncertainty, and therefore, more information to be gained with each action, highlighting the importance of research on artificial intelligence to invest in curiosity-driven algorithms. RESEARCH HIGHLIGHTS: Across three studies, we tested whether information gain itself acts as an internal reward and suffices to motivate preschoolers' actions. We measured preschoolers' persistence when searching for an object behind a series of doors, manipulating the uncertainty about which specific object was hidden. We found that preschoolers were more persistent when there was higher uncertainty, and therefore, more information to be gained with each action. Our results highlight the importance of research on artificial intelligence to invest in curiosity-driven algorithms.

Meta-learning: Data, architecture, and both.

We are encouraged by the many positive commentaries on our target article. In this response, we recapitulate some of the points raised and identify synergies between them. We have arranged our response based on the tension between data and architecture that arises in the meta-learning framework. We additionally provide a short discussion that touches upon connections to foundation models.

A clearer view of Southern Ocean air-sea interaction using surface heat flux asymmetry.

Progress in understanding Southern Ocean heat exchange and wind forcing is discussed and new results presented. These include a metric of the zonal asymmetry between surface ocean heat gain in the Atlantic/Indian sector and heat loss in the Pacific sector. The asymmetry arises from an intersector variation in the humidity gradient between the sea surface and near-surface atmosphere. This gradient increases by 60% in the Pacific sector enabling a 20 Wm-2 stronger latent heat loss compared with the Atlantic/Indian sector. The new metric is used for intercomparison of atmospheric reanalyses and CMIP6 climate simulations. CMIP6 has weaker Atlantic/Indian sector heat gain compared with the reanalyses primarily due to Indian Ocean sector differences. The potential for surface flux buoys to provide an observation-based counterpart to the asymmetry metric is explored. Over the past decade, flux buoys have been deployed at two sites (south of Tasmania and upstream of Drake Passage). The data record provided by these moorings is assessed and an argument developed for a third buoy to sample the Atlantic/Indian sector of the asymmetry metric. To close, we assess evidence that the main westerly wind belt has strengthened and moved southward in recent decades using the ERA5 reanalysis. This article is part of a discussion meeting issue 'Heat and carbon uptake in the Southern Ocean: the state of the art and future priorities'.

Meta-Learned Models of Cognition.

Psychologists and neuroscientists extensively rely on computational models for studying and analyzing the human mind. Traditionally, such computational models have been hand-designed by expert researchers. Two prominent examples are cognitive architectures and Bayesian models of cognition. While the former requires the specification of a fixed set of computational structures and a definition of how these structures interact with each other, the latter necessitates the commitment to a particular prior and a likelihood function which - in combination with Bayes' rule - determine the model's behavior. In recent years, a new framework has established itself as a promising tool for building models of human cognition: the framework of meta-learning. In contrast to the previously mentioned model classes, meta-learned models acquire their inductive biases from experience, i.e., by repeatedly interacting with an environment. However, a coherent research program around meta-learned models of cognition is still missing to this day. The purpose of this article is to synthesize previous work in this field and establish such a research program. We accomplish this by pointing out that meta-learning can be used to construct Bayes-optimal learning algorithms, allowing us to draw strong connections to the rational analysis of cognition. We then discuss several advantages of the meta-learning framework over traditional methods and reexamine prior work in the context of these new insights.

Structured, uncertainty-driven exploration in real-world consumer choice.

Making good decisions requires people to appropriately explore their available options and generalize what they have learned. While computational models can explain exploratory behavior in constrained laboratory tasks, it is unclear to what extent these models generalize to real-world choice problems. We investigate the factors guiding exploratory behavior in a dataset consisting of 195,333 customers placing 1,613,967 orders from a large online food delivery service. We find important hallmarks of adaptive exploration and generalization, which we analyze using computational models. In particular, customers seem to engage in uncertainty-directed exploration and use feature-based generalization to guide their exploration. Our results provide evidence that people use sophisticated strategies to explore complex, real-world environments.

Correction: Similarities and differences in spatial and non-spatial cognitive maps.

[This corrects the article DOI: 10.1371/journal.pcbi.1008149.].

Similarities and differences in spatial and non-spatial cognitive maps.

Learning and generalization in spatial domains is often thought to rely on a "cognitive map", representing relationships between spatial locations. Recent research suggests that this same neural machinery is also recruited for reasoning about more abstract, conceptual forms of knowledge. Yet, to what extent do spatial and conceptual reasoning share common computational principles, and what are the implications for behavior? Using a within-subject design we studied how participants used spatial or conceptual distances to generalize and search for correlated rewards in successive multi-armed bandit tasks. Participant behavior indicated sensitivity to both spatial and conceptual distance, and was best captured using a Bayesian model of generalization that formalized distance-dependent generalization and uncertainty-guided exploration as a Gaussian Process regression with a radial basis function kernel. The same Gaussian Process model best captured human search decisions and judgments in both domains, and could simulate realistic learning curves, where we found equivalent levels of generalization in spatial and conceptual tasks. At the same time, we also find characteristic differences between domains. Relative to the spatial domain, participants showed reduced levels of uncertainty-directed exploration and increased levels of random exploration in the conceptual domain. Participants also displayed a one-directional transfer effect, where experience in the spatial task boosted performance in the conceptual task, but not vice versa. While confidence judgments indicated that participants were sensitive to the uncertainty of their knowledge in both tasks, they did not or could not leverage their estimates of uncertainty to guide exploration in the conceptual task. These results support the notion that value-guided learning and generalization recruit cognitive-map dependent computational mechanisms in spatial and conceptual domains. Yet both behavioral and model-based analyses suggest domain specific differences in how these representations map onto actions.

Development of directed and random exploration in children.

Are young children just random explorers who learn serendipitously? Or are even young children guided by uncertainty-directed sampling, seeking to explore in a systematic fashion? We study how children between the ages of 4 and 9 search in an explore-exploit task with spatially correlated rewards, where exhaustive exploration is infeasible and not all options can be experienced. By combining behavioral data with a computational model that decomposes search into similarity-based generalization, uncertainty-directed exploration, and random exploration, we map out developmental trajectories of generalization and exploration. The behavioral data show strong developmental differences in children's capability to exploit environmental structure, with performance and adaptiveness of sampling decisions increasing with age. Through model-based analyses, we disentangle different forms of exploration, finding signature of both uncertainty-directed and random exploration. The amount of random exploration strongly decreases as children get older, supporting the notion of a developmental "cooling off" process that modulates the randomness in sampling. However, even at the youngest age range, children do not solely rely on random exploration. Even as random exploration begins to taper off, children are actively seeking out options with high uncertainty in a goal-directed fashion, and using inductive inferences to generalize their experience to novel options. Our findings provide critical insights into the behavioral and computational principles underlying the developmental trajectory of learning and exploration.

Genomic and phylogenetic characterization of ChPV2, a novel goat PV closely related to the Xi-PV1 species infecting bovines.

BACKGROUND: Papillomaviruses (PVs) infecting artiodactyls are very diverse, and only second in number to PVs infecting primates. PVs associated to lesions in economically important ruminant species have been isolated from cattle and sheep. METHODS: Potential PV DNA from teat lesions of a Damascus goat was isolated, cloned and sequenced. The PV genome was analyzed using bioinformatics approaches to detect open reading frames and to predict potential features of encoded proteins as well as putative regulatory elements. Sequence comparison and phylogenetic analyses using the concatenated E1E2L2L1 nucleotide and amino acid alignments was used to reveal the relationship of the new PV to the known PV diversity and its closest relevants. RESULTS: We isolated and characterized the full-genome of novel Capra hircus papillomavirus. We identified the E6, E7, E1, E2, L2, L1 open reading frames with protein coding potential and putative active elements in the ChPV2 proteins and putative regulatory genome elements. Sequence similarities of L1 and phylogenetic analyses using concatenated E1E2L2L1 nucleotide and amino acid alignments suggest the classification as a new PV type designated ChPV2 with a phylogenetic position within the XiPV genus, basal to the XiPV1 species. ChPV2 is not closely related to ChPV1, the other known goat PV isolated from healthy skin, although both of them belong confidently into a clade composed of PVs infecting cervids and bovids. Interestingly, ChPV2 contains an E6 open reading frame whereas all closely related PVs do not CONCLUSION: ChPV2 is a novel goat PV closely related to the Xi-PV1 species infecting bovines. Phylogenetic relationships and genome architecture of ChPV2 and closely related PV types suggest at least two independent E6 losses within the XiPV clade.

Finding structure in multi-armed bandits.

How do humans search for rewards? This question is commonly studied using multi-armed bandit tasks, which require participants to trade off exploration and exploitation. Standard multi-armed bandits assume that each option has an independent reward distribution. However, learning about options independently is unrealistic, since in the real world options often share an underlying structure. We study a class of structured bandit tasks, which we use to probe how generalization guides exploration. In a structured multi-armed bandit, options have a correlation structure dictated by a latent function. We focus on bandits in which rewards are linear functions of an option's spatial position. Across 5 experiments, we find evidence that participants utilize functional structure to guide their exploration, and also exhibit a learning-to-learn effect across rounds, becoming progressively faster at identifying the latent function. Our experiments rule out several heuristic explanations and show that the same findings obtain with non-linear functions. Comparing several models of learning and decision making, we find that the best model of human behavior in our tasks combines three computational mechanisms: (1) function learning, (2) clustering of reward distributions across rounds, and (3) uncertainty-guided exploration. Our results suggest that human reinforcement learning can utilize latent structure in sophisticated ways to improve efficiency.

Searching for Rewards Like a Child Means Less Generalization and More Directed Exploration.

How do children and adults differ in their search for rewards? We considered three different hypotheses that attribute developmental differences to (a) children's increased random sampling, (b) more directed exploration toward uncertain options, or (c) narrower generalization. Using a search task in which noisy rewards were spatially correlated on a grid, we compared the ability of 55 younger children (ages 7 and 8 years), 55 older children (ages 9-11 years), and 50 adults (ages 19-55 years) to successfully generalize about unobserved outcomes and balance the exploration-exploitation dilemma. Our results show that children explore more eagerly than adults but obtain lower rewards. We built a predictive model of search to disentangle the unique contributions of the three hypotheses of developmental differences and found robust and recoverable parameter estimates indicating that children generalize less and rely on directed exploration more than adults. We did not, however, find reliable differences in terms of random sampling.

Designer nuclease-mediated gene correction via homology-directed repair in an in vitro model of canine hemophilia B.

BACKGROUND: Gene correction at specific target loci provides a powerful strategy for overcoming genetic diseases. In the present study, we aimed to use an in vitro model for canine hemophilia B containing a single point mutation in the catalytic domain of the canine coagulation factor IX (cFIX) gene. To correct the defective gene via homology-directed repair (HDR), we designed transcription-activator like effector nucleases and clustered regularly interspaced short palindromic repeats including Cas9 (CRISPR/Cas9) for introduction of double-strand breaks at the mutation site. METHODS: To generate a stable cell line containing the mutated cFIX locus, a 2-kb genomic DNA fragment derived from a hemophilia B dog was amplified and integrated utilizing the phiC31 integrase system. Designer nucleases were assembled and cloned into vectors for constitutive and inducible expression. To detect mutations, insertions and deletions, and HDR events after nuclease treatment T7E1 assays, an amplification-refractory mutation system-quantitative polymerase chain reaction and pyrosequencing were performed. RESULTS: To perform HDR correction experiments, we established a cell line carrying the mutated cFIX locus. In HDR approaches we either explored a wild-type or an optimized cFIX sequence and we found that our modified HDR cassette showed higher gene correction efficiencies of up to 6.4%. Furthermore, we compared inducible and constitutive designer nuclease expression systems and found that the inducible system resulted in comparable HDR efficiencies. CONCLUSIONS: In conclusion, the present study demonstrates the potential of this strategy for gene therapeutic approaches in vitro and in a canine model for hemophilia B.

Long-term quality of life changes after primary septorhinoplasty.

PURPOSE: Health-related quality of life measurements are gaining in importance in clinical medicine. Little is known about the long-term quality of life changes after septorhinoplasty. This study was designed to analyse the long-term quality of life impacts of septorhinoplasty, using disease-specific instruments-rhinoplasty outcome evaluation (ROE) and Functional Rhinoplasty Outcome Inventory-17 (FROI-17); as well as a generic instrument-Short-Form 36 Health Survey (SF-36). METHODS: Patients completed the FROI-17, the ROE and the SF-36 preoperatively and at 12 and 60 months postoperatively. General demographic and clinical information (age, gender, allergies, medication, medical and surgical history) were collected from all patients. RESULTS: We report a significant increase in disease-specific QOL after primary septorhinoplasty (as measured with the ROE & FROI-17) and in two scales of the SF-36 generic instrument (role-functioning physical and mental health) 1 year after surgery. Our patients showed further significant increase in disease-specific QOL (FROI-17) after their primary septorhinoplasty (1 year vs. 5 years postoperatively). SF-36 results showed significant improvements 5 years postoperatively (compared to preoperative scores) in six out of eight scales (physical functioning, role-functioning physical, bodily pain, vitality, social functioning and mental health). CONCLUSION: Septorhinoplasty can improve disease-specific and non-disease-specific QOL in the short- and long-term postoperative period. These improvements remain measurable 5 years after surgery.

Where do hypotheses come from?

Why are human inferences sometimes remarkably close to the Bayesian ideal and other times systematically biased? In particular, why do humans make near-rational inferences in some natural domains where the candidate hypotheses are explicitly available, whereas tasks in similar domains requiring the self-generation of hypotheses produce systematic deviations from rational inference. We propose that these deviations arise from algorithmic processes approximating Bayes' rule. Specifically in our account, hypotheses are generated stochastically from a sampling process, such that the sampled hypotheses form a Monte Carlo approximation of the posterior. While this approximation will converge to the true posterior in the limit of infinite samples, we take a small number of samples as we expect that the number of samples humans take is limited. We show that this model recreates several well-documented experimental findings such as anchoring and adjustment, subadditivity, superadditivity, the crowd within as well as the self-generation effect, the weak evidence, and the dud alternative effects. We confirm the model's prediction that superadditivity and subadditivity can be induced within the same paradigm by manipulating the unpacking and typicality of hypotheses. We also partially confirm our model's prediction about the effect of time pressure and cognitive load on these effects.

Compositional inductive biases in function learning.

How do people recognize and learn about complex functional structure? Taking inspiration from other areas of cognitive science, we propose that this is achieved by harnessing compositionality: complex structure is decomposed into simpler building blocks. We formalize this idea within the framework of Bayesian regression using a grammar over Gaussian process kernels, and compare this approach with other structure learning approaches. Participants consistently chose compositional (over non-compositional) extrapolations and interpolations of functions. Experiments designed to elicit priors over functional patterns revealed an inductive bias for compositional structure. Compositional functions were perceived as subjectively more predictable than non-compositional functions, and exhibited other signatures of predictability, such as enhanced memorability and reduced numerosity. Taken together, these results support the view that the human intuitive theory of functions is inherently compositional.

Advances of Recombinant Adenoviral Vectors in Preclinical and Clinical Applications.

Adenoviruses (Ad) have the potential to induce severe infections in vulnerable patient groups. Therefore, understanding Ad biology and antiviral processes is important to comprehend the signaling cascades during an infection and to initiate appropriate diagnostic and therapeutic interventions. In addition, Ad vector-based vaccines have revealed significant potential in generating robust immune protection and recombinant Ad vectors facilitate efficient gene transfer to treat genetic diseases and are used as oncolytic viruses to treat cancer. Continuous improvements in gene delivery capacity, coupled with advancements in production methods, have enabled widespread application in cancer therapy, vaccine development, and gene therapy on a large scale. This review provides a comprehensive overview of the virus biology, and several aspects of recombinant Ad vectors, as well as the development of Ad vector, are discussed. Moreover, we focus on those Ads that were used in preclinical and clinical applications including regenerative medicine, vaccine development, genome engineering, treatment of genetic diseases, and virotherapy in tumor treatment.