Not every credible interval is credible: Evaluating robustness in the presence of contamination in Bayesian data analysis.

As Bayesian methods become more popular among behavioral scientists, they will inevitably be applied in situations that violate the assumptions underpinning typical models used to guide statistical inference. With this in mind, it is important to know something about how robust Bayesian methods are to the violation of those assumptions. In this paper, we focus on the problem of contaminated data (such as data with outliers or conflicts present), with specific application to the problem of estimating a credible interval for the population mean. We evaluate five Bayesian methods for constructing a credible interval, using toy examples to illustrate the qualitative behavior of different approaches in the presence of contaminants, and an extensive simulation study to quantify the robustness of each method. We find that the "default" normal model used in most Bayesian data analyses is not robust, and that approaches based on the Bayesian bootstrap are only robust in limited circumstances. A simple parametric model based on Tukey's "contaminated normal model" and a model based on the t-distribution were markedly more robust. However, the contaminated normal model had the added benefit of estimating which data points were discounted as outliers and which were not.

On the likelihood of "encapsulating all uncertainty".

The assignment of personal probabilities to form a forensic practitioner's likelihood ratio is a mental operation subject to all the frailties of human memory, perception and judgment. While we agree that beliefs expressed as coherent probabilities are neither 'right' nor 'wrong' we argue that debate over this fact obscures both the requirement for and consideration of the 'helpfulness' of practitioner's opinions. We also question the extent to which a likelihood ratio based on personal probabilities can realistically be expected to 'encapsulate all uncertainty'. Courts cannot rigorously assess a forensic practitioner's bare assertions of belief regarding evidential strength. At a minimum, information regarding the uncertainty both within and between the opinions of practitioners is required.

Learning and choosing in an uncertain world: An investigation of the explore-exploit dilemma in static and dynamic environments.

How do people solve the explore-exploit trade-off in a changing environment? In this paper we present experimental evidence from an "observe or bet" task, in which people have to determine when to engage in information-seeking behavior and when to switch to reward-taking actions. In particular we focus on the comparison between people's behavior in a changing environment and their behavior in an unchanging one. Our experimental work is motivated by rational analysis of the problem that makes strong predictions about information search and reward seeking in static and changeable environments. Our results show a striking agreement between human behavior and the optimal policy, but also highlight a number of systematic differences. In particular, we find that while people often employ suboptimal strategies the first time they encounter the learning problem, most people are able to approximate the correct strategy after minimal experience. In order to describe both the manner in which people's choices are similar to but slightly different from an optimal standard, we introduce four process models for the observe or bet task and evaluate them as potential theories of human behavior.

How do people learn from negative evidence? Non-monotonic generalizations and sampling assumptions in inductive reasoning.

A robust finding in category-based induction tasks is for positive observations to raise the willingness to generalize to other categories while negative observations lower the willingness to generalize. This pattern is referred to as monotonic generalization. Across three experiments we find systematic non-monotonicity effects, in which negative observations raise the willingness to generalize. Experiments 1 and 2 show that this effect emerges in hierarchically structured domains when a negative observation from a different category is added to a positive observation. They also demonstrate that this is related to a specific kind of shift in the reasoner's hypothesis space. Experiment 3 shows that the effect depends on the assumptions that the reasoner makes about how inductive arguments are constructed. Non-monotonic reasoning occurs when people believe the facts were put together by a helpful communicator, but monotonicity is restored when they believe the observations were sampled randomly from the environment.

Erroneous gambling-related beliefs as illusions of primary and secondary control: a confirmatory factor analysis.

Different classification systems for erroneous beliefs about gambling have been proposed, consistently alluding to 'illusion of control' and 'gambler's fallacy' categories. None of these classification systems have, however, considered the how the illusion of control and the gambler's fallacy might be interrelated. In this paper, we report the findings of a confirmatory factor analysis that examines the proposal that most erroneous gambling-related beliefs can be defined in terms of Rothbaum et al.'s (J Pers Soc Psychol, doi: 10.1037/0022-3514.42.1.5 , 1982) distinction between 'primary' and 'secondary' illusory control, with the former being driven to a large extent by the well-known gambler's fallacy and the latter being driven by a complex of beliefs about supernatural forces such as God and luck. A survey consisting of 100 items derived from existing instruments was administered to 329 participants. The analysis confirmed the existence of two latent structures (beliefs in primary and secondary control), while also offering support to the idea that gambler's fallacy-style reasoning may underlie both perceived primary control and beliefs about the cyclical nature of luck, a form of perceived secondary control. The results suggest the need for a greater focus on the role of underlying processes or belief structures as factors that foster susceptibility to specific beliefs in gambling situations. Addressing and recognising the importance of these underlying factors may also have implications for cognitive therapy treatments for problem gambling.

Learning time-varying categories.

Many kinds of objects and events in our world have a strongly time-dependent quality. However, most theories about concepts and categories either are insensitive to variation over time or treat it as a nuisance factor that produces irrational order effects during learning. In this article, we present two category learning experiments in which we explored peoples' ability to learn categories whose structure is strongly time-dependent. We suggest that order effects in categorization may in part reflect a sensitivity to changing environments, and that understanding dynamically changing concepts is an important part of developing a full account of human categorization.

Better explanations of lexical and semantic cognition using networks derived from continued rather than single-word associations.

In this article, we describe the most extensive set of word associations collected to date. The database contains over 12,000 cue words for which more than 70,000 participants generated three responses in a multiple-response free association task. The goal of this study was (1) to create a semantic network that covers a large part of the human lexicon, (2) to investigate the implications of a multiple-response procedure by deriving a weighted directed network, and (3) to show how measures of centrality and relatedness derived from this network predict both lexical access in a lexical decision task and semantic relatedness in similarity judgment tasks. First, our results show that the multiple-response procedure results in a more heterogeneous set of responses, which lead to better predictions of lexical access and semantic relatedness than do single-response procedures. Second, the directed nature of the network leads to a decomposition of centrality that primarily depends on the number of incoming links or in-degree of each node, rather than its set size or number of outgoing links. Both studies indicate that adequate representation formats and sufficiently rich data derived from word associations represent a valuable type of information in both lexical and semantic processing.

Epistemic trust: modeling children's reasoning about others' knowledge and intent.

A core assumption of many theories of development is that children can learn indirectly from other people. However, indirect experience (or testimony) is not constrained to provide veridical information. As a result, if children are to capitalize on this source of knowledge, they must be able to infer who is trustworthy and who is not. How might a learner make such inferences while at the same time learning about the world? What biases, if any, might children bring to this problem? We address these questions with a computational model of epistemic trust in which learners reason about the helpfulness and knowledgeability of an informant. We show that the model captures the competencies shown by young children in four areas: (1) using informants' accuracy to infer how much to trust them; (2) using informants' recent accuracy to overcome effects of familiarity; (3) inferring trust based on consensus among informants; and (4) using information about mal-intent to decide not to trust. The model also explains developmental changes in performance between 3 and 4 years of age as a result of changing default assumptions about the helpfulness of other people.

The Impact of Shift Work on Diabetes Self-Management Activities.

BACKGROUND: Once a person is diagnosed with diabetes, aggressive management is imperative to minimize poor glycemic control devastating outcomes. However, for some patients reaching optimum blood glucose levels is challenging due to the complexity of diabetes care. To achieve good blood glucose control, patients affected by diabetes must engage in self-care activities that include routine blood glucose check, dietary control, physical activity, medication regimen, and routine medical provider visits. Diabetes-associated self-care activities aimed to reach good blood glucose control can be hindered by multiple factors including shift work. OBJECTIVE: To evaluate self-management activities of individuals affected by diabetes who are employed as shift workers. This study also informs primary care nurse practitioners of the challenges shift workers face in managing their disease. METHODS: This was a cross-sectional descriptive study. Participants were individuals affected by type II diabetes from a single primary care practice (N = 86); 45 were individuals working on the dayshift and 41 worked on the evening/night shift. Each participant completed the diabetes self-management questionnaire and author-developed demographic/supplemental questionnaire. CONCLUSIONS: There were no differences in self-reported diabetes management activities (i.e., physical activity, glucose management, and healthcare use) between the two groups. Thirty-nine percent of participants working shifts reported worse sleep patterns compared to their dayshift counterparts (X 2[1, N = 85] = 8.73, p = .003). Evening/night shift workers also reported more symptoms such as leg pain, fungal infection, numbness of the feet and legs, dizziness, and vision changes (X 2[1, N = 79] = 43.037, p < .001). IMPLICATIONS FOR PRACTICE: A better understanding of the impact that shift work has on diabetes care may help healthcare providers formulate meaningful treatment plans to meet the needs of evening/night shift diabetic workers. The use of a patient-centered medical home is one strategy.

Similarity, distance, and categorization: a discussion of Smith's (2006) warning about "colliding parameters".

The idea that categorization decisions rely on subjective impressions of similarities between stimuli has been prevalent in much of the literature over the past 30 years and has led to the development of a large number of models that apply some kind of decision rule to similarity measures. A recent article by Smith (2006) has argued that these similarity-choice models of categorization have a substantial design flaw, in which the similarity and the choice components effectively cancel one another out. As a consequence of this cancellation, it is claimed, the relationship between distance and category membership probabilities is linear in these models. In this article, I discuss these claims and show mathematically that in those cases in which it is sensible to discuss the relationship between category distance and category membership at all, the function relating the two is approximately logistic. Empirical data are used to show that a logistic function can be observed in appropriate contexts.

Does response scaling cause the generalized context model to mimic a prototype model?

Smith and Minda (1998, 2002) argued that the response scaling parameter y in the exemplar-based generalized context model (GCM) makes the model unnecessarily complex and allows it to mimic the behavior of a prototype model. We evaluated this criticism in two ways. First, we estimated the complexity of the GCM with and without the yparameter and also compared its complexity to that of a prototype model. Next, we assessed the extent to which the models mimic each other, using two experimental designs (Nosofsky & Zaki, 2002, Experiment 3; Smith & Minda, 1998, Experiment 2), chosen because these designs are thought to differ in the degree to which they can discriminate the models. The results show that y can increase the complexity of the GCM, but this complexity does not necessarily allow mimicry. Furthermore, if statistical model selection methods such as minimum description length are adopted as the measure of model performance, the models will be highly discriminable, irrespective of design.

None of the above: A Bayesian account of the detection of novel categories.

Every time we encounter a new object, action, or event, there is some chance that we will need to assign it to a novel category. We describe and evaluate a class of probabilistic models that detect when an object belongs to a category that has not previously been encountered. The models incorporate a prior distribution that is influenced by the distribution of previous objects among categories, and we present 2 experiments that demonstrate that people are also sensitive to this distributional information. Two additional experiments confirm that distributional information is combined with similarity when both sources of information are available. We compare our approach to previous models of unsupervised categorization and to several heuristic-based models, and find that a hierarchical Bayesian approach provides the best account of our data. (PsycINFO Database Record

Bayesian models of cognition revisited: Setting optimality aside and letting data drive psychological theory.

Recent debates in the psychological literature have raised questions about the assumptions that underpin Bayesian models of cognition and what inferences they license about human cognition. In this paper we revisit this topic, arguing that there are 2 qualitatively different ways in which a Bayesian model could be constructed. The most common approach uses a Bayesian model as a normative standard upon which to license a claim about optimality. In the alternative approach, a descriptive Bayesian model need not correspond to any claim that the underlying cognition is optimal or rational, and is used solely as a tool for instantiating a substantive psychological theory. We present 3 case studies in which these 2 perspectives lead to different computational models and license different conclusions about human cognition. We demonstrate how the descriptive Bayesian approach can be used to answer different sorts of questions than the optimal approach, especially when combined with principled tools for model evaluation and model selection. More generally we argue for the importance of making a clear distinction between the 2 perspectives. Considerable confusion results when descriptive models and optimal models are conflated, and if Bayesians are to avoid contributing to this confusion it is important to avoid making normative claims when none are intended. (PsycINFO Database Record

Global model analysis by parameter space partitioning.

To model behavior, scientists need to know how models behave. This means learning what other behaviors a model can produce besides the one generated by participants in an experiment. This is a difficult problem because of the complexity of psychological models (e.g., their many parameters) and because the behavioral precision of models (e.g., interval-scale performance) often mismatches their testable precision in experiments, where qualitative, ordinal predictions are the norm. Parameter space partitioning is a solution that evaluates model performance at a qualitative level. There exists a partition on the model's parameter space that divides it into regions that correspond to each data pattern. Three application examples demonstrate its potential and versatility for studying the global behavior of psychological models.

Leaping to Conclusions: Why Premise Relevance Affects Argument Strength.

Everyday reasoning requires more evidence than raw data alone can provide. We explore the idea that people can go beyond this data by reasoning about how the data was sampled. This idea is investigated through an examination of premise non-monotonicity, in which adding premises to a category-based argument weakens rather than strengthens it. Relevance theories explain this phenomenon in terms of people's sensitivity to the relationships among premise items. We show that a Bayesian model of category-based induction taking premise sampling assumptions and category similarity into account complements such theories and yields two important predictions: First, that sensitivity to premise relationships can be violated by inducing a weak sampling assumption; and second, that premise monotonicity should be restored as a result. We test these predictions with an experiment that manipulates people's assumptions in this regard, showing that people draw qualitatively different conclusions in each case.

The helpfulness of category labels in semi-supervised learning depends on category structure.

The study of semi-supervised category learning has generally focused on how additional unlabeled information with given labeled information might benefit category learning. The literature is also somewhat contradictory, sometimes appearing to show a benefit to unlabeled information and sometimes not. In this paper, we frame the problem differently, focusing on when labels might be helpful to a learner who has access to lots of unlabeled information. Using an unconstrained free-sorting categorization experiment, we show that labels are useful to participants only when the category structure is ambiguous and that people's responses are driven by the specific set of labels they see. We present an extension of Anderson's Rational Model of Categorization that captures this effect.

Structure at every scale: A semantic network account of the similarities between unrelated concepts.

Similarity plays an important role in organizing the semantic system. However, given that similarity cannot be defined on purely logical grounds, it is important to understand how people perceive similarities between different entities. Despite this, the vast majority of studies focus on measuring similarity between very closely related items. When considering concepts that are very weakly related, little is known. In this article, we present 4 experiments showing that there are reliable and systematic patterns in how people evaluate the similarities between very dissimilar entities. We present a semantic network account of these similarities showing that a spreading activation mechanism defined over a word association network naturally makes correct predictions about weak similarities, whereas, though simpler, models based on direct neighbors between word pairs derived using the same network cannot. (PsycINFO Database Record

The structure of sequential effects.

There is a long history of research into sequential effects, extending more than one hundred years. The pattern of sequential effects varies widely with both experimental conditions as well as for different individuals performing the same experiment. Yet this great diversity of results is poorly understood, particularly with respect to individual variation, which save for some passing mentions has largely gone unreported in the literature. Here we seek to understand the way in which sequential effects vary by identifying the causes underlying the differences observed in sequential effects. In order to achieve this goal we perform principal component analysis on a dataset of 158 individual results from participants performing different experiments with the aim of identifying hidden variables responsible for sequential effects. We find a latent structure consisting of 3 components related to sequential effects-2 main and 1 minor. A relationship between the 2 main components and the separate processing of stimuli and of responses is proposed on the basis of previous empirical evidence. It is further speculated that the minor component of sequential effects arises as the consequence of processing delays. Independently of the explanation for the latent variables encountered, this work provides a unified descriptive model for a wide range of different types of sequential effects previously identified in the literature. In addition to explaining individual differences themselves, it is demonstrated how the latent structure uncovered here is useful in understanding the classical problem of the dependence of sequential effects on the interval between successive stimuli.

Common and distinctive features in stimulus similarity: a modified version of the contrast model.

Featural representations of similarity data assume that people represent stimuli in terms of a set of discrete properties. In this article, we consider the differences in featural representations that arise from making four different assumptions about how similarity is measured. Three of these similarity models--the common features model, the distinctive features model, and Tversky's seminal contrast model-have been considered previously. The other model is new and modifies the contrast model by assuming that each individual feature only ever acts as a common or distinctive feature. Each of the four models is tested on previously examined similarity data, relating to kinship terms, and on a new data set, relating to faces. In fitting the models, we have used the geometric complexity criterion to balance the competing demands of data-fit and model complexity. The results show that both common and distinctive features are important for stimulus representation, and we argue that the modified contrast model combines these two components in a more effective and interpretable way than Tversky's original formulation.

Extending the ALCOVE model of category learning to featural stimulus domains.

The ALCOVE model of category learning, despite its considerable success in accounting for human performance across a wide range of empirical tasks, is limited by its reliance on spatial stimulus representations. Some stimulus domains are better suited to featural representation, characterizing stimuli in terms of the presence or absence of discrete features, rather than as points in a multidimensional space. We report on empirical data measuring human categorization performance across a featural stimulus domain and show that ALCOVE is unable to capture fundamental qualitative aspects of this performance. In response, a featural version of the ALCOVE model is developed, replacing the spatial stimulus representations that are usually generated by multidimensional scaling with featural representations generated by additive clustering. We demonstrate that this featural version of ALCOVE is able to capture human performance where the spatial model failed, explaining the difference in terms of the contrasting representational assumptions made by the two approaches. Finally, we discuss ways in which the ALCOVE categorization model might be extended further to use "hybrid" representational structures combining spatial and featural components.