ChatGPT outperforms crowd workers for text-annotation tasks.

Many NLP applications require manual text annotations for a variety of tasks, notably to train classifiers or evaluate the performance of unsupervised models. Depending on the size and degree of complexity, the tasks may be conducted by crowd workers on platforms such as MTurk as well as trained annotators, such as research assistants. Using four samples of tweets and news articles (n = 6,183), we show that ChatGPT outperforms crowd workers for several annotation tasks, including relevance, stance, topics, and frame detection. Across the four datasets, the zero-shot accuracy of ChatGPT exceeds that of crowd workers by about 25 percentage points on average, while ChatGPT's intercoder agreement exceeds that of both crowd workers and trained annotators for all tasks. Moreover, the per-annotation cost of ChatGPT is less than $0.003-about thirty times cheaper than MTurk. These results demonstrate the potential of large language models to drastically increase the efficiency of text classification.

Empathy-based counterspeech can reduce racist hate speech in a social media field experiment.

Despite heightened awareness of the detrimental impact of hate speech on social media platforms on affected communities and public discourse, there is little consensus on approaches to mitigate it. While content moderation-either by governments or social media companies-can curb online hostility, such policies may suppress valuable as well as illicit speech and might disperse rather than reduce hate speech. As an alternative strategy, an increasing number of international and nongovernmental organizations (I/NGOs) are employing counterspeech to confront and reduce online hate speech. Despite their growing popularity, there is scant experimental evidence on the effectiveness and design of counterspeech strategies (in the public domain). Modeling our interventions on current I/NGO practice, we randomly assign English-speaking Twitter users who have sent messages containing xenophobic (or racist) hate speech to one of three counterspeech strategies-empathy, warning of consequences, and humor-or a control group. Our intention-to-treat analysis of 1,350 Twitter users shows that empathy-based counterspeech messages can increase the retrospective deletion of xenophobic hate speech by 0.2 SD and reduce the prospective creation of xenophobic hate speech over a 4-wk follow-up period by 0.1 SD. We find, however, no consistent effects for strategies using humor or warning of consequences. Together, these results advance our understanding of the central role of empathy in reducing exclusionary behavior and inform the design of future counterspeech interventions.

Tokenization of social media engagements increases the sharing of false (and other) news but penalization moderates it.

Some major social media companies are announcing plans to tokenize user engagements, derived from blockchain-based decentralized social media. This would bring financial and reputational incentives for engagement, which might lead users to post more objectionable content. Previous research showed that financial or reputational incentives for accuracy decrease the willingness to share misinformation. However, it is unclear to what extent such outcome would change if engagements instead of accuracy were incentivized, which is a more realistic scenario. To address this question, we conducted a survey experiment to examine the effects of hypothetical token incentives. We find that a simple nudge about the possibility of earning token-based points for the achieved user engagements increases the willingness to share different kinds of news, including misinformation. The presence of penalties for objectionable posts diminishes the positive effect of tokenization rewards on misinformation sharing, but it does not eliminate it. These results have policy implications for content moderation practices if platforms embrace decentralization and engagement tokenization.

Social Media and Policy Responses to the COVID-19 Pandemic in Switzerland.

We study the role of social media in debates regarding two policy responses to COVID-19 in Switzerland: face-mask rules and contact-tracing apps. We use a dictionary classifier to categorize 612'177 tweets by parties, politicians, and the public as well as 441'458 articles published in 76 newspapers between February and August 2020. We distinguish between "problem" (COVID-19) and "solutions" (face masks and contact-tracing apps) and, using a vector autoregression approach, we analyze the relationship between their salience on social and traditional media, as well as among different groups on social media. We find that overall attention to COVID-19 was not driven by endogenous dynamics between the different actors. By contrast, the debate on face masks was led by the attentive public and by politicians, whereas parties and newspapers followed. The results illustrate how social media challenge the capacity of party and media elites to craft a consensus regarding the appropriateness of different measures as responses to a major crisis.

Nous étudions le rôle des réseaux sociaux dans les débats concernant deux réponses politiques au COVID­19 en Suisse: les règles relatives aux masques de protection et les applications de traçage des contacts. Nous catégorisons 612'177 tweets ainsi que 441'458 articles publiés dans 76 journaux entre février et août 2020 en distinguant le "problème" (COVID­19) des "solutions" (masques de protection et applications de traçage des contacts). Ensuite, nous analysons la relation entre leur saillance sur les réseaux sociaux et dans les médias traditionnels, ainsi qu'entre différents groupes sur les réseaux sociaux. L'attention portée à COVID­19 n'a pas été caractérisée par une dynamique endogène entre les différents acteurs. En revanche, le débat sur les masques de protection a été mené par le public attentif et par les politiciens, tandis que les partis et les journaux ont suivi. Les résultats illustrent la façon dont les réseaux sociaux remettent en question la capacité des élites à élaborer un consensus sur les différentes mesures.