Automating hybrid collective intelligence in open-ended medical diagnostics.

Collective intelligence has emerged as a powerful mechanism to boost decision accuracy across many domains, such as geopolitical forecasting, investment, and medical diagnostics. However, collective intelligence has been mostly applied to relatively simple decision tasks (e.g., binary classifications). Applications in more open-ended tasks with a much larger problem space, such as emergency management or general medical diagnostics, are largely lacking, due to the challenge of integrating unstandardized inputs from different crowd members. Here, we present a fully automated approach for harnessing collective intelligence in the domain of general medical diagnostics. Our approach leverages semantic knowledge graphs, natural language processing, and the SNOMED CT medical ontology to overcome a major hurdle to collective intelligence in open-ended medical diagnostics, namely to identify the intended diagnosis from unstructured text. We tested our method on 1,333 medical cases diagnosed on a medical crowdsourcing platform: The Human Diagnosis Project. Each case was independently rated by ten diagnosticians. Comparing the diagnostic accuracy of single diagnosticians with the collective diagnosis of differently sized groups, we find that our method substantially increases diagnostic accuracy: While single diagnosticians achieved 46% accuracy, pooling the decisions of ten diagnosticians increased this to 76%. Improvements occurred across medical specialties, chief complaints, and diagnosticians' tenure levels. Our results show the life-saving potential of tapping into the collective intelligence of the global medical community to reduce diagnostic errors and increase patient safety.

Visual social information use in collective foraging.

Collective dynamics emerge from individual-level decisions, yet we still poorly understand the link between individual-level decision-making processes and collective outcomes in realistic physical systems. Using collective foraging to study the key trade-off between personal and social information use, we present a mechanistic, spatially-explicit agent-based model that combines individual-level evidence accumulation of personal and (visual) social cues with particle-based movement. Under idealized conditions without physical constraints, our mechanistic framework reproduces findings from established probabilistic models, but explains how individual-level decision processes generate collective outcomes in a bottom-up way. In clustered environments, groups performed best if agents reacted strongly to social information, while in uniform environments, individualistic search was most beneficial. Incorporating different real-world physical and perceptual constraints profoundly shaped collective performance, and could even buffer maladaptive herding by facilitating self-organized exploration. Our study uncovers the mechanisms linking individual cognition to collective outcomes in human and animal foraging and paves the way for decentralized robotic applications.

The rostral micro-tooth morphology of blue marlin, Makaira nigricans.

Billfish rostra potentially have several functions; however, their role in feeding is unequivocal in some species. Recent work linked morphological variation in rostral micro-teeth to differences in feeding behavior in two billfish species, the striped marlin (Kajikia audax) and the sailfish (Istiophorus platypterus). Here, we present the rostral micro-tooth morphology for a third billfish species, the blue marlin (Makaira nigricans), for which the use of the rostrum in feeding behavior is still undocumented from systematic observations in the wild. We measured the micro-teeth on rostrum tips of blue marlin, striped marlin, and sailfish using a micro-computed tomography approach and compared the tooth morphology among the three species. This was done after an analysis of video-recorded hunting behavior of striped marlin and sailfish revealed that both species strike prey predominantly with the first third of the rostrum, which provided the justification to focus our analysis on the rostrum tips. In blue marlin, intact micro-teeth were longer compared to striped marlin but not to sailfish. Blue marlin had a higher fraction of broken teeth than both striped marlin and sailfish, and broken teeth were distributed more evenly on the rostrum. Micro-tooth regrowth was equally low in both marlin species but higher in sailfish. Based on the differences and similarities in the micro-tooth morphology between the billfish species, we discuss potential feeding-related rostrum use in blue marlin. We put forward the hypothesis that blue marlin might use their rostra in high-speed dashes as observed in striped marlin, rather than in the high-precision rostral strikes described for sailfish, possibly focusing on larger prey organisms.

Avoiding costly mistakes in groups: The evolution of error management in collective decision making.

Individuals continuously have to balance the error costs of alternative decisions. A wealth of research has studied how single individuals navigate this, showing that individuals develop response biases to avoid the more costly error. We, however, know little about the dynamics in groups facing asymmetrical error costs and when social influence amplifies either safe or risky behavior. Here, we investigate this by modeling the decision process and information flow with a drift-diffusion model extended to the social domain. In the model individuals first gather independent personal information; they then enter a social phase in which they can either decide early based on personal information, or wait for additional social information. We combined the model with an evolutionary algorithm to derive adaptive behavior. We find that under asymmetric costs, individuals in large cooperative groups do not develop response biases because such biases amplify at the collective level, triggering false information cascades. Selfish individuals, however, undermine the group's performance for their own benefit by developing higher response biases and waiting for more information. Our results have implications for our understanding of the social dynamics in groups facing asymmetrical errors costs, such as animal groups evading predation or police officers holding a suspect at gunpoint.

Crowd control: Reducing individual estimation bias by sharing biased social information.

Cognitive biases are widespread in humans and animals alike, and can sometimes be reinforced by social interactions. One prime bias in judgment and decision-making is the human tendency to underestimate large quantities. Previous research on social influence in estimation tasks has generally focused on the impact of single estimates on individual and collective accuracy, showing that randomly sharing estimates does not reduce the underestimation bias. Here, we test a method of social information sharing that exploits the known relationship between the true value and the level of underestimation, and study if it can counteract the underestimation bias. We performed estimation experiments in which participants had to estimate a series of quantities twice, before and after receiving estimates from one or several group members. Our purpose was threefold: to study (i) whether restructuring the sharing of social information can reduce the underestimation bias, (ii) how the number of estimates received affects the sensitivity to social influence and estimation accuracy, and (iii) the mechanisms underlying the integration of multiple estimates. Our restructuring of social interactions successfully countered the underestimation bias. Moreover, we find that sharing more than one estimate also reduces the underestimation bias. Underlying our results are a human tendency to herd, to trust larger estimates than one's own more than smaller estimates, and to follow disparate social information less. Using a computational modeling approach, we demonstrate that these effects are indeed key to explain the experimental results. Overall, our results show that existing knowledge on biases can be used to dampen their negative effects and boost judgment accuracy, paving the way for combating other cognitive biases threatening collective systems.

Parasite infection disrupts escape behaviours in fish shoals.

Many prey species have evolved collective responses to avoid predation. They rapidly transfer information about potential predators to trigger and coordinate escape waves. Predation avoidance behaviour is often manipulated by trophically transmitted parasites, to facilitate their transmission to the next host. We hypothesized that the presence of infected, behaviourally altered individuals might disturb the spread of escape waves. We used the tapeworm Schistocephalus solidus, which increases risk-taking behaviour and decreases social responsiveness of its host, the three-spined stickleback, to test this hypothesis. Three subgroups of sticklebacks were placed next to one another in separate compartments with shelter. The middle subgroup contained either uninfected or infected sticklebacks. We confronted an outer subgroup with an artificial bird strike and studied how the escape response spread through the subgroups. With uninfected sticklebacks in the middle, escape waves spread rapidly through the entire shoal and fish remained in shelter thereafter. With infected sticklebacks in the middle, the escape wave was disrupted and uninfected fish rarely used the shelter. Infected individuals can disrupt the transmission of flight responses, thereby not only increasing their own predation risk but also that of their uninfected shoal members. Our study uncovers a potentially far-reaching fitness consequence of grouping with infected individuals.

Strategies for integrating disparate social information.

Social information use is widespread in the animal kingdom, helping individuals rapidly acquire useful knowledge and adjust to novel circumstances. In humans, the highly interconnected world provides ample opportunities to benefit from social information but also requires navigating complex social environments with people holding disparate or conflicting views. It is, however, still largely unclear how people integrate information from multiple social sources that (dis)agree with them, and among each other. We address this issue in three steps. First, we present a judgement task in which participants could adjust their judgements after observing the judgements of three peers. We experimentally varied the distribution of this social information, systematically manipulating its variance (extent of agreement among peers) and its skewness (peer judgements clustering either near or far from the participant's judgement). As expected, higher variance among peers reduced their impact on behaviour. Importantly, observing a single peer confirming a participant's own judgement markedly decreased the influence of other-more distant-peers. Second, we develop a framework for modelling the cognitive processes underlying the integration of disparate social information, combining Bayesian updating with simple heuristics. Our model accurately accounts for observed adjustment strategies and reveals that people particularly heed social information that confirms personal judgements. Moreover, the model exposes strong inter-individual differences in strategy use. Third, using simulations, we explore the possible implications of the observed strategies for belief updating. These simulations show how confirmation-based weighting can hamper the influence of disparate social information, exacerbate filter bubble effects and deepen group polarization. Overall, our results clarify what aspects of the social environment are, and are not, conducive to changing people's minds.

Females facilitate male food patch discovery in a wild fish population.

Responding to the information provided by others is an important foraging strategy in many species. Through social foraging, individuals can more efficiently find unpredictable resources and thereby increase their foraging success. When individuals are more socially responsive to particular phenotypes than others, however, the advantage they obtain from foraging socially is likely to depend on the phenotype composition of the social environment. We tested this hypothesis by performing experimental manipulations of guppy, Poecilia reticulata, sex compositions in the wild. Males found fewer novel food patches in the absence of females than in mixed-sex compositions, while female patch discovery did not differ regardless of the presence or absence of males. We argue that these results were driven by sex-dependent mechanisms of social association: Markov chain-based fission-fusion modelling revealed that less social individuals found fewer patches and that males reduced sociality when females were absent. In contrast, females were similarly social with or without males. Our findings highlight the relevance of considering how individual- and population-level traits interact in shaping the advantages of social foraging in the wild.

Boosting medical diagnostics by pooling independent judgments.

Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decision making in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involving more than 140 doctors making more than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.

Parasite-infected sticklebacks increase the risk-taking behaviour of uninfected group members.

Trophically transmitted parasites frequently increase their hosts' risk-taking behaviour, to facilitate transmission to the next host. Whether such elevated risk-taking can spill over to uninfected group members is, however, unknown. To investigate this, we confronted groups of 6 three-spined sticklebacks, Gasterosteus aculeatus, containing 0, 2, 4 or 6 experimentally infected individuals with a simulated bird attack and studied their risk-taking behaviour. As a parasite, we used the tapeworm Schistocephalus solidus, which increases the risk-taking of infected sticklebacks, to facilitate transmission to its final host, most often piscivorous birds. Before the attack, infected and uninfected individuals did not differ in their risk-taking. However, after the attack, individuals in groups with only infected members showed lower escape responses and higher risk-taking than individuals from groups with only uninfected members. Importantly, uninfected individuals adjusted their risk-taking behaviour to the number of infected group members, taking more risk with an increasing number of infected group members. Infected individuals, however, did not adjust their risk-taking to the number of uninfected group members. Our results show that behavioural manipulation by parasites does not only affect the infected host, but also uninfected group members, shedding new light on the social dynamics involved in host-parasite interactions.

Proto-cooperation: group hunting sailfish improve hunting success by alternating attacks on grouping prey.

We present evidence of a novel form of group hunting. Individual sailfish (Istiophorus platypterus) alternate attacks with other group members on their schooling prey (Sardinella aurita). While only 24% of attacks result in prey capture, multiple prey are injured in 95% of attacks, resulting in an increase of injured fish in the school with the number of attacks. How quickly prey are captured is positively correlated with the level of injury of the school, suggesting that hunters can benefit from other conspecifics' attacks on the prey. To explore this, we built a mathematical model capturing the dynamics of the hunt. We show that group hunting provides major efficiency gains (prey caught per unit time) for individuals in groups of up to 70 members. We also demonstrate that a free riding strategy, where some individuals wait until the prey are sufficiently injured before attacking, is only beneficial if the cost of attacking is high, and only then when waiting times are short. Our findings provide evidence that cooperative benefits can be realized through the facilitative effects of individuals' hunting actions without spatial coordination of attacks. Such 'proto-cooperation' may be the pre-cursor to more complex group-hunting strategies.

Earlier social information has a stronger influence on judgments.

People's decisions are often informed by the choices of others. Evidence accumulation models provide a mechanistic account of how such social information enters the choice process. Previous research taking this approach has suggested two fundamentally different cognitive mechanisms by which people incorporate social information. On the one hand, individuals may update their evidence level instantaneously when observing social information. On the other hand, they may gradually integrate social information over time. These accounts make different predictions on how the timing of social information impacts its influence. The former predicts that timing has no impact on social information uptake. The latter predicts that social information which arrives earlier has a stronger impact because its impact increases over time. We tested both predictions in two studies in which participants first observed a perceptual stimulus. They then entered a deliberation phase in which social information arrived either early or late before reporting their judgment. In Experiment 1, early social information remained visible until the end and was thus displayed for longer than late social information. In Experiment 2, which was preregistered, early and late social information were displayed for an equal duration. In both studies, early social information had a larger impact on individuals' judgments. Further, an evidence accumulation analysis found that social information integration was best explained by both an immediate update of evidence and continuous integration over time. Because in social systems, timing plays a key role (e.g., propagation of information in social networks), our findings inform theories explaining the temporal evolution of social impact and the emergent social dynamics.

Collective Intelligence Increases Diagnostic Accuracy in a General Practice Setting.

BACKGROUND: General practitioners (GPs) work in an ill-defined environment where diagnostic errors are prevalent. Previous research indicates that aggregating independent diagnoses can improve diagnostic accuracy in a range of settings. We examined whether aggregating independent diagnoses can also improve diagnostic accuracy for GP decision making. In addition, we investigated the potential benefit of such an approach in combination with a decision support system (DSS). METHODS: We simulated virtual groups using data sets from 2 previously published studies. In study 1, 260 GPs independently diagnosed 9 patient cases in a vignette-based study. In study 2, 30 GPs independently diagnosed 12 patient actors in a patient-facing study. In both data sets, GPs provided diagnoses in a control condition and/or DSS condition(s). Each GP's diagnosis, confidence rating, and years of experience were entered into a computer simulation. Virtual groups of varying sizes (range: 3-9) were created, and different collective intelligence rules (plurality, confidence, and seniority) were applied to determine each group's final diagnosis. Diagnostic accuracy was used as the performance measure. RESULTS: Aggregating independent diagnoses by weighing them equally (i.e., the plurality rule) substantially outperformed average individual accuracy, and this effect increased with increasing group size. Selecting diagnoses based on confidence only led to marginal improvements, while selecting based on seniority reduced accuracy. Combining the plurality rule with a DSS further boosted performance. DISCUSSION: Combining independent diagnoses may substantially improve a GP's diagnostic accuracy and subsequent patient outcomes. This approach did, however, not improve accuracy in all patient cases. Therefore, future work should focus on uncovering the conditions under which collective intelligence is most beneficial in general practice. HIGHLIGHTS: We examined whether aggregating independent diagnoses of GPs can improve diagnostic accuracy.Using data sets of 2 previously published studies, we composed virtual groups of GPs and combined their independent diagnoses using 3 collective intelligence rules (plurality, confidence, and seniority).Aggregating independent diagnoses by weighing them equally substantially outperformed average individual GP accuracy, and this effect increased with increasing group size.Combining independent diagnoses may substantially improve GP's diagnostic accuracy and subsequent patient outcomes.

Collective incentives reduce over-exploitation of social information in unconstrained human groups.

Collective dynamics emerge from countless individual decisions. Yet, we poorly understand the processes governing dynamically-interacting individuals in human collectives under realistic conditions. We present a naturalistic immersive-reality experiment where groups of participants searched for rewards in different environments, studying how individuals weigh personal and social information and how this shapes individual and collective outcomes. Capturing high-resolution visual-spatial data, behavioral analyses revealed individual-level gains-but group-level losses-of high social information use and spatial proximity in environments with concentrated (vs. distributed) resources. Incentivizing participants at the group (vs. individual) level facilitated adaptation to concentrated environments, buffering apparently excessive scrounging. To infer discrete choices from unconstrained interactions and uncover the underlying decision mechanisms, we developed an unsupervised Social Hidden Markov Decision model. Computational results showed that participants were more sensitive to social information in concentrated environments frequently switching to a social relocation state where they approach successful group members. Group-level incentives reduced participants' overall responsiveness to social information and promoted higher selectivity over time. Finally, mapping group-level spatio-temporal dynamics through time-lagged regressions revealed a collective exploration-exploitation trade-off across different timescales. Our study unravels the processes linking individual-level strategies to emerging collective dynamics, and provides tools to investigate decision-making in freely-interacting collectives.

Accurate decisions in an uncertain world: collective cognition increases true positives while decreasing false positives.

In a wide range of contexts, including predator avoidance, medical decision-making and security screening, decision accuracy is fundamentally constrained by the trade-off between true and false positives. Increased true positives are possible only at the cost of increased false positives; conversely, decreased false positives are associated with decreased true positives. We use an integrated theoretical and experimental approach to show that a group of decision-makers can overcome this basic limitation. Using a mathematical model, we show that a simple quorum decision rule enables individuals in groups to simultaneously increase true positives and decrease false positives. The results from a predator-detection experiment that we performed with humans are in line with these predictions: (i) after observing the choices of the other group members, individuals both increase true positives and decrease false positives, (ii) this effect gets stronger as group size increases, (iii) individuals use a quorum threshold set between the average true- and false-positive rates of the other group members, and (iv) individuals adjust their quorum adaptively to the performance of the group. Our results have broad implications for our understanding of the ecology and evolution of group-living animals and lend themselves for applications in the human domain such as the design of improved screening methods in medical, forensic, security and business applications.

Situational social influence leading to non-compliance with conservation rules.

It is well established that the decisions that we make can be strongly influenced by the behaviour of others. However, testing how social influence can lead to non-compliance with conservation rules during an individual's decision-making process has received little research attention. We synthesise advances in understanding of conformity and rule-breaking in individuals and in groups, and take a situational approach to studying the social dynamics and ensuing social identity changes that can lead to non-compliant decision-making. We focus on situational social influence contagion that are copresent (i.e., same space and same time) or trace-based (i.e., behavioural traces in the same space). We then suggest approaches for testing how situational social influence can lead to certain behaviours in non-compliance with conservation rules.

The balance of the sexes.

A large-scale experiment demonstrates sex differences in cooperation and competition that can explain group size variation in ostriches.

How experts' own inconsistency relates to their confidence and between-expert disagreement.

People routinely rely on experts' advice to guide their decisions. However, experts are known to make inconsistent judgments when judging the same case twice. Previous research on expert inconsistency has largely focused on individual or situational factors; here we focus directly on the cases themselves. First, using a theoretical model, we study how within-expert inconsistency and confidence are related to how strongly experts agree on a case. Second, we empirically test the model's predictions in two real-world datasets with a diagnostic ground truth from follow-up research: diagnosticians rating the same mammograms or images of the lower spine twice. Our modeling and empirical analyses converge on the same novel results: The more experts disagree in their initial decisions about a case (i.e., as consensus decreases), the less confident individual experts are in their initial decision-despite not knowing the level of consensus-and the more likely they are to judge that same case differently when facing it again months later, regardless of whether the expert consensus is correct. Our results suggest the following advice when faced with two conflicting decisions from a single expert: In the absence of more predictive cues, choose the more confident decision.

Time pressure reduces misinformation discrimination ability but does not alter response bias.

Many parts of our social lives are speeding up, a process known as social acceleration. How social acceleration impacts people's ability to judge the veracity of online news, and ultimately the spread of misinformation, is largely unknown. We examined the effects of accelerated online dynamics, operationalised as time pressure, on online misinformation evaluation. Participants judged the veracity of true and false news headlines with or without time pressure. We used signal detection theory to disentangle the effects of time pressure on discrimination ability and response bias, as well as on four key determinants of misinformation susceptibility: analytical thinking, ideological congruency, motivated reflection, and familiarity. Time pressure reduced participants' ability to accurately distinguish true from false news (discrimination ability) but did not alter their tendency to classify an item as true or false (response bias). Key drivers of misinformation susceptibility, such as ideological congruency and familiarity, remained influential under time pressure. Our results highlight the dangers of social acceleration online: People are less able to accurately judge the veracity of news online, while prominent drivers of misinformation susceptibility remain present. Interventions aimed at increasing deliberation may thus be fruitful avenues to combat online misinformation.