Self-certification: A novel method for increasing sharing discernment on social media.

The proliferation of misinformation on social media platforms has given rise to growing demands for effective intervention strategies that increase sharing discernment (i.e. increase the difference in the probability of sharing true posts relative to the probability of sharing false posts). One suggested method is to encourage users to deliberate on the veracity of the information prior to sharing. However, this strategy is undermined by individuals' propensity to share posts they acknowledge as false. In our study, across three experiments, in a simulated social media environment, participants were shown social media posts and asked whether they wished to share them and, sometimes, whether they believed the posts to be truthful. We observe that requiring users to verify their belief in a news post's truthfulness before sharing it markedly curtails the dissemination of false information. Thus, requiring self-certification increased sharing discernment. Importantly, requiring self-certification didn't hinder users from sharing content they genuinely believed to be true because participants were allowed to share any posts that they indicated were true. We propose self-certification as a method that substantially curbs the spread of misleading content on social media without infringing upon the principle of free speech.

ChatGPT's advice is perceived as better than that of professional advice columnists.

ChatGPT is a high-performance large language model that has the potential to significantly improve human-computer interactions. It can provide advice on a range of topics, but it is unclear how good this advice is relative to that provided by competent humans, especially in situations where empathy is required. Here, we report the first investigation of whether ChatGPT's responses are perceived as better than those of humans in a task where humans were attempting to be empathetic. Fifty social dilemma questions were randomly selected from 10 well-known advice columns. In a pre-registered survey, participants (N = 404) were each shown one question, along with the corresponding response by an advice columnist and by ChatGPT. ChatGPT's advice was perceived as more balanced, complete, empathetic, helpful, and better than the advice provided by professional advice columnists (all values of p < 0.001). Although participants could not determine which response was written by ChatGPT (54%, p = 0.29), most participants preferred that their own social dilemma questions be answered by a human than by a computer (77%, p < 0.001). ChatGPT's responses were longer than those produced by the advice columnists (mean 280.9 words vs. 142.2 words, p < 0.001). In a second pre-registered survey, each ChatGPT answer was constrained to be approximately the same length as that of the advice columnist (mean 143.2 vs. 142.2 words, p = 0.95). This survey (N = 401) replicated the above findings, showing that the benefit of ChatGPT was not solely due to it writing longer answers.

Validating a forced-choice method for eliciting quality-of-reasoning judgments.

In this paper we investigate the criterion validity of forced-choice comparisons of the quality of written arguments with normative solutions. Across two studies, novices and experts assessing quality of reasoning through a forced-choice design were both able to choose arguments supporting more accurate solutions-62.2% (SE = 1%) of the time for novices and 74.4% (SE = 1%) for experts-and arguments produced by larger teams-up to 82% of the time for novices and 85% for experts-with high inter-rater reliability, namely 70.58% (95% CI = 1.18) agreement for novices and 80.98% (95% CI = 2.26) for experts. We also explored two methods for increasing efficiency. We found that the number of comparative judgments needed could be substantially reduced with little accuracy loss by leveraging transitivity and producing quality-of-reasoning assessments using an AVL tree method. Moreover, a regression model trained to predict scores based on automatically derived linguistic features of participants' judgments achieved a high correlation with the objective accuracy scores of the arguments in our dataset. Despite the inherent subjectivity involved in evaluating differing quality of reasoning, the forced-choice paradigm allows even novice raters to perform beyond chance and can provide a valid, reliable, and efficient method for producing quality-of-reasoning assessments at scale.

The role of narrative in collaborative reasoning and intelligence analysis: A case study.

This paper explores the significance of narrative in collaborative reasoning using a qualitative case study of two teams of intelligence analysts who took part in an exercise using an online collaborative platform. Digital ethnographic methods were used to analyze the chat transcripts of analysts as they reasoned with evidence provided in a difficult, fictional intelligence-type problem and produced a final intelligence report. These chat transcripts provided a powerful "microscope" into the reasoning processes and interactions involved in complex, collaborative reasoning. We found that Individuals and teams used narrative to solve the kinds of complex problems organizations and intelligence agencies face daily. We observed that team members generated what we term "micro-narratives", which provided a means for testing, assessing and weighing alternative hypotheses through mental simulation in the context of collaborative reasoning. The creation of micro-narratives assisted in the teams' reasoning with evidence, an integral part of collaborative reasoning and intelligence analysis. Micro-narratives were combined into, and compared with, an ideal or 'virtual' narrative which informed the judgements the team came to in their final intelligence report. The case study developed in this paper provides evidence that narrative thought processes play an important role in complex collaborative problem-solving, reasoning with evidence and problem-solving. This is contrary to a widespread perception that narrative thinking is fundamentally distinct from formal, logical reasoning.