CHAI: consensus clustering through similarity matrix integration for cell-type identification.

Several methods have been developed to computationally predict cell-types for single cell RNA sequencing (scRNAseq) data. As methods are developed, a common problem for investigators has been identifying the best method they should apply to their specific use-case. To address this challenge, we present CHAI (consensus Clustering tHrough similArIty matrix integratIon for single cell-type identification), a wisdom of crowds approach for scRNAseq clustering. CHAI presents two competing methods which aggregate the clustering results from seven state-of-the-art clustering methods: CHAI-AvgSim and CHAI-SNF. CHAI-AvgSim and CHAI-SNF demonstrate superior performance across several benchmarking datasets. Furthermore, both CHAI methods outperform the most recent consensus clustering method, SAME-clustering. We demonstrate CHAI's practical use case by identifying a leader tumor cell cluster enriched with CDH3. CHAI provides a platform for multiomic integration, and we demonstrate CHAI-SNF to have improved performance when including spatial transcriptomics data. CHAI overcomes previous limitations by incorporating the most recent and top performing scRNAseq clustering algorithms into the aggregation framework. It is also an intuitive and easily customizable R package where users may add their own clustering methods to the pipeline, or down-select just the ones they want to use for the clustering aggregation. This ensures that as more advanced clustering algorithms are developed, CHAI will remain useful to the community as a generalized framework. CHAI is available as an open source R package on GitHub: https://github.com/lodimk2/chai.

COFFEE: consensus single cell-type specific inference for gene regulatory networks.

The inference of gene regulatory networks (GRNs) is crucial to understanding the regulatory mechanisms that govern biological processes. GRNs may be represented as edges in a graph, and hence, it have been inferred computationally for scRNA-seq data. A wisdom of crowds approach to integrate edges from several GRNs to create one composite GRN has demonstrated improved performance when compared with individual algorithm implementations on bulk RNA-seq and microarray data. In an effort to extend this approach to scRNA-seq data, we present COFFEE (COnsensus single cell-type speciFic inFerence for gEnE regulatory networks), a Borda voting-based consensus algorithm that integrates information from 10 established GRN inference methods. We conclude that COFFEE has improved performance across synthetic, curated, and experimental datasets when compared with baseline methods. Additionally, we show that a modified version of COFFEE can be leveraged to improve performance on newer cell-type specific GRN inference methods. Overall, our results demonstrate that consensus-based methods with pertinent modifications continue to be valuable for GRN inference at the single cell level. While COFFEE is benchmarked on 10 algorithms, it is a flexible strategy that can incorporate any set of GRN inference algorithms according to user preference. A Python implementation of COFFEE may be found on GitHub: https://github.com/lodimk2/coffee.

Deepfake detection by human crowds, machines, and machine-informed crowds.

The recent emergence of machine-manipulated media raises an important societal question: How can we know whether a video that we watch is real or fake? In two online studies with 15,016 participants, we present authentic videos and deepfakes and ask participants to identify which is which. We compare the performance of ordinary human observers with the leading computer vision deepfake detection model and find them similarly accurate, while making different kinds of mistakes. Together, participants with access to the model's prediction are more accurate than either alone, but inaccurate model predictions often decrease participants' accuracy. To probe the relative strengths and weaknesses of humans and machines as detectors of deepfakes, we examine human and machine performance across video-level features, and we evaluate the impact of preregistered randomized interventions on deepfake detection. We find that manipulations designed to disrupt visual processing of faces hinder human participants' performance while mostly not affecting the model's performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance.

Promoting Erroneous Divergent Opinions Increases the Wisdom of Crowds.

The aggregation of many lay judgments generates surprisingly accurate estimates. This phenomenon, called the "wisdom of crowds," has been demonstrated in domains such as medical decision-making and financial forecasting. Previous research identified two factors driving this effect: the accuracy of individual assessments and the diversity of opinions. Most available strategies to enhance the wisdom of crowds have focused on improving individual accuracy while neglecting the potential of increasing opinion diversity. Here, we study a complementary approach to reduce collective error by promoting erroneous divergent opinions. This strategy proposes to anchor half of the crowd to a small value and the other half to a large value before eliciting and averaging all estimates. Consistent with our mathematical modeling, four experiments (N = 1,362 adults) demonstrated that this method is effective for estimation and forecasting tasks. Beyond the practical implications, these findings offer new theoretical insights into the epistemic value of collective decision-making.

Adaptive social networks promote the wisdom of crowds.

Social networks continuously change as new ties are created and existing ones fade. It is widely acknowledged that our social embedding has a substantial impact on what information we receive and how we form beliefs and make decisions. However, most empirical studies on the role of social networks in collective intelligence have overlooked the dynamic nature of social networks and its role in fostering adaptive collective intelligence. Therefore, little is known about how groups of individuals dynamically modify their local connections and, accordingly, the topology of the network of interactions to respond to changing environmental conditions. In this paper, we address this question through a series of behavioral experiments and supporting simulations. Our results reveal that, in the presence of plasticity and feedback, social networks can adapt to biased and changing information environments and produce collective estimates that are more accurate than their best-performing member. To explain these results, we explore two mechanisms: 1) a global-adaptation mechanism where the structural connectivity of the network itself changes such that it amplifies the estimates of high-performing members within the group (i.e., the network "edges" encode the computation); and 2) a local-adaptation mechanism where accurate individuals are more resistant to social influence (i.e., adjustments to the attributes of the "node" in the network); therefore, their initial belief is disproportionately weighted in the collective estimate. Our findings substantiate the role of social-network plasticity and feedback as key adaptive mechanisms for refining individual and collective judgments.

Harnessing the power of collective intelligence in dentistry: a pilot study in Victoria, Australia.

BACKGROUND: In many dental settings, diagnosis and treatment planning is the responsibility of a single clinician, and this process is inevitably influenced by the clinician's own heuristics and biases. Our aim was to test whether collective intelligence increases the accuracy of individual diagnoses and treatment plans, and whether such systems have potential to improve patient outcomes in a dental setting. METHODS: This pilot project was carried out to assess the feasibility of the protocol and appropriateness of the study design. We used a questionnaire survey and pre-post study design in which dental practitioners were involved in the diagnosis and treatment planning of two simulated cases. Participants were provided the opportunity to amend their original diagnosis/treatment decisions after viewing a consensus report made to simulate a collaborative setting. RESULTS: Around half (55%, n = 17) of the respondents worked in group private practices, however most practitioners (74%, n = 23) did not collaborate when planning treatment. Overall, the average practitioners' self-confidence score in managing different dental disciplines was 7.22 (s.d. 2.20) on a 1-10 scale. Practitioners tended to change their mind after viewing the consensus response, particularly for the complex case compared to the simple case (61.5% vs 38.5%, respectively). Practitioners' confidence ratings were also significantly higher (p < 0.05) after viewing the consensus for complex case. CONCLUSION: Our pilot study shows that collective intelligence in the form of peers' opinion can lead to modifications in diagnosis and treatment planning by dentists. Our results lay the foundations for larger scale investigations on whether peer collaboration can improve diagnostic accuracy, treatment planning and, ultimately, oral health outcomes.

Crowd-Sourced Identification of Characteristics of Collective Human Motion.

Crowd simulations are used extensively to study the dynamics of human collectives. Such studies are underpinned by specific movement models, which encode rules and assumptions about how people navigate a space and handle interactions with others. These models often give rise to macroscopic simulated crowd behaviours that are statistically valid, but which lack the noisy microscopic behaviours that are the signature of believable real crowds. In this article, we use an existing Turing test for crowds to identify realistic features of real crowds that are generally omitted from simulation models. Our previous study using this test established that untrained individuals have difficulty in classifying movies of crowds as real or simulated, and that such people often have an idealised view of how crowds move. In this follow-up study (with new participants) we perform a second trial, which now includes a training phase (showing participants movies of real crowds). We find that classification performance significantly improves after training, confirming the existence of features that allow participants to identify real crowds. High-performing individuals are able to identify the features of real crowds that should be incorporated into future simulations if they are to be considered realistic.

Historical roots of implicit bias in slavery.

Implicit racial bias remains widespread, even among individuals who explicitly reject prejudice. One reason for the persistence of implicit bias may be that it is maintained through structural and historical inequalities that change slowly. We investigated the historical persistence of implicit bias by comparing modern implicit bias with the proportion of the population enslaved in those counties in 1860. Counties and states more dependent on slavery before the Civil War displayed higher levels of pro-White implicit bias today among White residents and less pro-White bias among Black residents. These associations remained significant after controlling for explicit bias. The association between slave populations and implicit bias was partially explained by measures of structural inequalities. Our results support an interpretation of implicit bias as the cognitive residue of past and present structural inequalities.

The wisdom of partisan crowds.

Theories in favor of deliberative democracy are based on the premise that social information processing can improve group beliefs. While research on the "wisdom of crowds" has found that information exchange can increase belief accuracy on noncontroversial factual matters, theories of political polarization imply that groups will become more extreme-and less accurate-when beliefs are motivated by partisan political bias. A primary concern is that partisan biases are associated not only with more extreme beliefs, but also with a diminished response to social information. While bipartisan networks containing both Democrats and Republicans are expected to promote accurate belief formation, politically homogeneous networks are expected to amplify partisan bias and reduce belief accuracy. To test whether the wisdom of crowds is robust to partisan bias, we conducted two web-based experiments in which individuals answered factual questions known to elicit partisan bias before and after observing the estimates of peers in a politically homogeneous social network. In contrast to polarization theories, we found that social information exchange in homogeneous networks not only increased accuracy but also reduced polarization. Our results help generalize collective intelligence research to political domains.

Monitoring social-distance in wide areas during pandemics: a density map and segmentation approach.

With the relaxation of the containment measurements around the globe, monitoring the social distancing in crowded public spaces is of great importance to prevent a new massive wave of COVID-19 infections. Recent works in that matter have limited themselves by assessing social distancing in corridors up to small crowds by detecting each person individually, considering the full body in the image. In this work, we propose a new framework for monitoring the social-distance using end-to-end Deep Learning, to detect crowds violating social-distancing in wide areas, where important occlusions may be present. Our framework consists in the creation of new ground truth social distance labels, based on the ground truth density maps, and the proposal of two different solutions, a density-map-based and a segmentation-based, to detect crowds violating social-distancing constraints. We assess the results of both approaches by using the generated ground truth from the PET2009 and CityStreet datasets. We show that our framework performs well at providing the zones where people are not following the social-distance, even when heavily occluded or far away from the camera, compared to current detection and tracking approaches.

Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms.

Achieving the upper limits of face identification accuracy in forensic applications can minimize errors that have profound social and personal consequences. Although forensic examiners identify faces in these applications, systematic tests of their accuracy are rare. How can we achieve the most accurate face identification: using people and/or machines working alone or in collaboration? In a comprehensive comparison of face identification by humans and computers, we found that forensic facial examiners, facial reviewers, and superrecognizers were more accurate than fingerprint examiners and students on a challenging face identification test. Individual performance on the test varied widely. On the same test, four deep convolutional neural networks (DCNNs), developed between 2015 and 2017, identified faces within the range of human accuracy. Accuracy of the algorithms increased steadily over time, with the most recent DCNN scoring above the median of the forensic facial examiners. Using crowd-sourcing methods, we fused the judgments of multiple forensic facial examiners by averaging their rating-based identity judgments. Accuracy was substantially better for fused judgments than for individuals working alone. Fusion also served to stabilize performance, boosting the scores of lower-performing individuals and decreasing variability. Single forensic facial examiners fused with the best algorithm were more accurate than the combination of two examiners. Therefore, collaboration among humans and between humans and machines offers tangible benefits to face identification accuracy in important applications. These results offer an evidence-based roadmap for achieving the most accurate face identification possible.

Cellular Network Radio Monitoring and Management through Virtual UE Probes: A Study Case Based on Crowded Events.

Although log processing of network equipment is a common technique in cellular network management, several factors make said task challenging, especially during mass attendance events. The present paper assesses classic methods for cellular network measurement and acquisition, showing how the use of on-the-field user probes can provide relevant capabilities to the analysis of cellular network performance. Therefore, a framework for the deployment of this kind of system is proposed here based on the development of a new hardware virtualization platform with radio frequency capabilities. Accordingly, an analysis of the characteristics and requirements for the use of virtual probes was performed. Moreover, the impact that social events (e.g., sports matches) may have on the service provision was evaluated through a real cellular scenario. For this purpose, a long-term measurement study during crowded events (i.e., football matches) in a stadium has been conducted, and the performances of different services and operators under realistic settings has been evaluated. As a result, several considerations are presented that can be used for better management of future networks.

"It's Not Too Aggressive": Key Features of Social Branding Anti-Tobacco Interventions for High-Risk Young Adults.

Purpose. Peer crowd-targeted campaigns are a novel approach to engage high-risk young adults in tobacco use prevention and cessation. We elicited the perspectives of young adult key informants to understand how and why two social branding interventions were effective: (1) "COMMUNE," designed for "Hipsters" as a movement of artists and musicians against Big Tobacco, and (2) "HAVOC," designed for "Partiers" as an exclusive, smoke-free clubbing experience. Design. Qualitative study (27 semistructured qualitative phone interviews). Setting. Intervention events held in bars in multiple U.S. cities. Participants: Twenty-seven key informants involved in COMMUNE or HAVOC as organizers (e.g., musicians, event coordinators) or event attendees. Measures. We conducted semistructured, in-depth interviews. Participants described intervention events and features that worked or did not work well. Analysis. We used an inductive-deductive approach to thematically code interview transcripts, integrating concepts from intervention design literature and emergent themes. Results: Participants emphasized the importance of fun, interactive, social environments that encouraged a sense of belonging. Anti-tobacco messaging was subtle and nonjudgmental and resonated with their interests, values, and aesthetics. Young adults who represented the intervention were admired and influential among peers, and intervention promotional materials encouraged brand recognition and social status. Conclusion. Anti-tobacco interventions for high-risk young adults should encourage fun experiences; resonate with their interests, values, and aesthetics; and use subtle, nonjudgmental messaging.

Peer Crowd Identification of Young and Early Middle Adulthood Customers at Vape Shops.

Vape shops specialize in the sales of e-cigarettes and other vaping products. In recent studies, young adults who use e-cigarettes have tended to identify with at-risk peer crowds. This is the first study to examine vape shop customers' clientele. Composed primarily of young adults and persons in early middle adulthood, we speculated that a relatively high prevalence of those who appeared to bystanders as radical/extreme (at-risk) customers would be identified as such at these shops. We recruited vape shops throughout Southern California (N = 44 shops), and trained teams of data collectors visited each of the consented vape shops, making note of 451 customers' appearance, including features such as manner of dress, presence of tattoos, and hairstyles. Customers were then coded as either belonging to a conventional, progressive, or radical/extreme crowd based on outward appearance. Of the customers observed, 223 (49%) were rated as appearing to be in the conventional crowd; 169 (38%) were rated as appearing to be in the progressive crowd, and only 59 (13%) were rated as appearing to be in the radical/extreme crowd. The conventional crowd tended to appear older. Clientele may reflect that more conventional young and early middle age adults are tempted to visit vape shops due to perceptions of greater acceptability or safety of e-cigarettes. E-cigarette mass media campaigns aimed at protecting potential vape shop customers from harm may need to depict more conservative-looking characters.

The wisdom of stalemates: consensus and clustering as filtering mechanisms for improving collective accuracy.

Groups of organisms, from bacteria to fish schools to human societies, depend on their ability to make accurate decisions in an uncertain world. Most models of collective decision-making assume that groups reach a consensus during a decision-making bout, often through simple majority rule. In many natural and sociological systems, however, groups may fail to reach consensus, resulting in stalemates. Here, we build on opinion dynamics and collective wisdom models to examine how stalemates may affect the wisdom of crowds. For simple environments, where individuals have access to independent sources of information, we find that stalemates improve collective accuracy by selectively filtering out incorrect decisions (an effect we call stalemate filtering). In complex environments, where individuals have access to both shared and independent information, this effect is even more pronounced, restoring the wisdom of crowds in regions of parameter space where large groups perform poorly when making decisions using majority rule. We identify network properties that tune the system between consensus and accuracy, providing mechanisms by which animals, or evolution, could dynamically adjust the collective decision-making process in response to the reward structure of the possible outcomes. Overall, these results highlight the adaptive potential of stalemate filtering for improving the decision-making abilities of group-living animals.

The wisdom of crowds for visual search.

Decision-making accuracy typically increases through collective integration of people's judgments into group decisions, a phenomenon known as the wisdom of crowds. For simple perceptual laboratory tasks, classic signal detection theory specifies the upper limit for collective integration benefits obtained by weighted averaging of people's confidences, and simple majority voting can often approximate that limit. Life-critical perceptual decisions often involve searching large image data (e.g., medical, security, and aerial imagery), but the expected benefits and merits of using different pooling algorithms are unknown for such tasks. Here, we show that expected pooling benefits are significantly greater for visual search than for single-location perceptual tasks and the prediction given by classic signal detection theory. In addition, we show that simple majority voting obtains inferior accuracy benefits for visual search relative to averaging and weighted averaging of observers' confidences. Analysis of gaze behavior across observers suggests that the greater collective integration benefits for visual search arise from an interaction between the foveated properties of the human visual system (high foveal acuity and low peripheral acuity) and observers' nonexhaustive search patterns, and can be predicted by an extended signal detection theory framework with trial to trial sampling from a varying mixture of high and low target detectabilities across observers (SDT-MIX). These findings advance our theoretical understanding of how to predict and enhance the wisdom of crowds for real world search tasks and could apply more generally to any decision-making task for which the minority of group members with high expertise varies from decision to decision.

Network dynamics of social influence in the wisdom of crowds.

A longstanding problem in the social, biological, and computational sciences is to determine how groups of distributed individuals can form intelligent collective judgments. Since Galton's discovery of the "wisdom of crowds" [Galton F (1907) Nature 75:450-451], theories of collective intelligence have suggested that the accuracy of group judgments requires individuals to be either independent, with uncorrelated beliefs, or diverse, with negatively correlated beliefs [Page S (2008) The Difference: How the Power of Diversity Creates Better Groups, Firms, Schools, and Societies]. Previous experimental studies have supported this view by arguing that social influence undermines the wisdom of crowds. These results showed that individuals' estimates became more similar when subjects observed each other's beliefs, thereby reducing diversity without a corresponding increase in group accuracy [Lorenz J, Rauhut H, Schweitzer F, Helbing D (2011) Proc Natl Acad Sci USA 108:9020-9025]. By contrast, we show general network conditions under which social influence improves the accuracy of group estimates, even as individual beliefs become more similar. We present theoretical predictions and experimental results showing that, in decentralized communication networks, group estimates become reliably more accurate as a result of information exchange. We further show that the dynamics of group accuracy change with network structure. In centralized networks, where the influence of central individuals dominates the collective estimation process, group estimates become more likely to increase in error.

How social information can improve estimation accuracy in human groups.

In our digital and connected societies, the development of social networks, online shopping, and reputation systems raises the questions of how individuals use social information and how it affects their decisions. We report experiments performed in France and Japan, in which subjects could update their estimates after having received information from other subjects. We measure and model the impact of this social information at individual and collective scales. We observe and justify that, when individuals have little prior knowledge about a quantity, the distribution of the logarithm of their estimates is close to a Cauchy distribution. We find that social influence helps the group improve its properly defined collective accuracy. We quantify the improvement of the group estimation when additional controlled and reliable information is provided, unbeknownst to the subjects. We show that subjects' sensitivity to social influence permits us to define five robust behavioral traits and increases with the difference between personal and group estimates. We then use our data to build and calibrate a model of collective estimation to analyze the impact on the group performance of the quantity and quality of information received by individuals. The model quantitatively reproduces the distributions of estimates and the improvement of collective performance and accuracy observed in our experiments. Finally, our model predicts that providing a moderate amount of incorrect information to individuals can counterbalance the human cognitive bias to systematically underestimate quantities and thereby improve collective performance.

Multiple Object Tracking for Dense Pedestrians by Markov Random Field Model with Improvement on Potentials.

Pedestrian tracking in dense crowds is a challenging task, even when using a multi-camera system. In this paper, a new Markov random field (MRF) model is proposed for the association of tracklet couplings. Equipped with a new potential function improvement method, this model can associate the small tracklet coupling segments caused by dense pedestrian crowds. The tracklet couplings in this paper are obtained through a data fusion method based on image mutual information. This method calculates the spatial relationships of tracklet pairs by integrating position and motion information, and adopts the human key point detection method for correction of the position data of incomplete and deviated detections in dense crowds. The MRF potential function improvement method for dense pedestrian scenes includes assimilation and extension processing, as well as a message selective belief propagation algorithm. The former enhances the information of the fragmented tracklets by means of a soft link with longer tracklets and expands through sharing to improve the potentials of the adjacent nodes, whereas the latter uses a message selection rule to prevent unreliable messages of fragmented tracklet couplings from being spread throughout the MRF network. With the help of the iterative belief propagation algorithm, the potentials of the model are improved to achieve valid association of the tracklet coupling fragments, such that dense pedestrians can be tracked more robustly. Modular experiments and system-level experiments are conducted using the PETS2009 experimental data set, where the experimental results reveal that the proposed method has superior tracking performance.

Crowd Sourcing: Do Peer Crowd Prototypes Match Reality?

During the transition into high school, adolescents sort large sets of unfamiliar peers into prototypical peer crowds thought to share similar values, behaviors, and interests (e.g., Jocks). Often, such sorting is based solely on appearance. This study investigates the accuracy of this sorting process in relation to actual characteristics using video and survey data from a longitudinal sample of U.S. youths who attended high school in the mid- to late-2000s. To simulate this sorting process, we asked same-birth-cohort strangers to view short videos of youths at age 15 and to classify those strangers into likely crowd membership. We then compared the classifications they made to how adolescents characterized themselves at that same time point. Results show that peer crowd classification predicts aspects of unknown peers' mental health, academic achievement, extracurricular involvement, social status, and risk-taking behaviors.