People who share encounters with racism are silenced online by humans and machines, but a guideline-reframing intervention holds promise.

Are members of marginalized communities silenced on social media when they share personal experiences of racism? Here, we investigate the role of algorithms, humans, and platform guidelines in suppressing disclosures of racial discrimination. In a field study of actual posts from a neighborhood-based social media platform, we find that when users talk about their experiences as targets of racism, their posts are disproportionately flagged for removal as toxic by five widely used moderation algorithms from major online platforms, including the most recent large language models. We show that human users disproportionately flag these disclosures for removal as well. Next, in a follow-up experiment, we demonstrate that merely witnessing such suppression negatively influences how Black Americans view the community and their place in it. Finally, to address these challenges to equity and inclusion in online spaces, we introduce a mitigation strategy: a guideline-reframing intervention that is effective at reducing silencing behavior across the political spectrum.

Leveraging body-worn camera footage to assess the effects of training on officer communication during traffic stops.

Can training police officers on how to best interact with the public actually improve their interactions with community members? This has been a challenging question to answer. Interpersonal aspects of policing are consequential but largely invisible in administrative records commonly used for evaluation. In this study, we offer a solution: body-worn camera footage captures police-community interactions and how they might change as a function of training. Using this footage-as-data approach, we consider changes in officers' communication following procedural justice training in Oakland, CA, USA, one module of which sought to increase officer-communicated respect during traffic stops. We applied natural language processing tools and expert annotations of traffic stop recordings to detect whether officers enacted the five behaviors recommended in this module. Compared with recordings of stops that occurred prior to the training, we find that officers employed more of these techniques in posttraining stops; officers were more likely to express concern for drivers' safety, offer reassurance, and provide explicit reasons for the stop. These methods demonstrate the promise of a footage-as-data approach to capture and affect change in police-community interactions.

AI generates covertly racist decisions about people based on their dialect.

Hundreds of millions of people now interact with language models, with uses ranging from help with writing1,2 to informing hiring decisions3. However, these language models are known to perpetuate systematic racial prejudices, making their judgements biased in problematic ways about groups such as African Americans4-7. Although previous research has focused on overt racism in language models, social scientists have argued that racism with a more subtle character has developed over time, particularly in the United States after the civil rights movement8,9. It is unknown whether this covert racism manifests in language models. Here, we demonstrate that language models embody covert racism in the form of dialect prejudice, exhibiting raciolinguistic stereotypes about speakers of African American English (AAE) that are more negative than any human stereotypes about African Americans ever experimentally recorded. By contrast, the language models' overt stereotypes about African Americans are more positive. Dialect prejudice has the potential for harmful consequences: language models are more likely to suggest that speakers of AAE be assigned less-prestigious jobs, be convicted of crimes and be sentenced to death. Finally, we show that current practices of alleviating racial bias in language models, such as human preference alignment, exacerbate the discrepancy between covert and overt stereotypes, by superficially obscuring the racism that language models maintain on a deeper level. Our findings have far-reaching implications for the fair and safe use of language technology.

Learning the Meanings of Function Words From Grounded Language Using a Visual Question Answering Model.

Interpreting a seemingly simple function word like "or," "behind," or "more" can require logical, numerical, and relational reasoning. How are such words learned by children? Prior acquisition theories have often relied on positing a foundation of innate knowledge. Yet recent neural-network-based visual question answering models apparently can learn to use function words as part of answering questions about complex visual scenes. In this paper, we study what these models learn about function words, in the hope of better understanding how the meanings of these words can be learned by both models and children. We show that recurrent models trained on visually grounded language learn gradient semantics for function words requiring spatial and numerical reasoning. Furthermore, we find that these models can learn the meanings of logical connectives and and or without any prior knowledge of logical reasoning as well as early evidence that they are sensitive to alternative expressions when interpreting language. Finally, we show that word learning difficulty is dependent on the frequency of models' input. Our findings offer proof-of-concept evidence that it is possible to learn the nuanced interpretations of function words in a visually grounded context by using non-symbolic general statistical learning algorithms, without any prior knowledge of linguistic meaning.

Assessing the potential of GPT-4 to perpetuate racial and gender biases in health care: a model evaluation study.

BACKGROUND: Large language models (LLMs) such as GPT-4 hold great promise as transformative tools in health care, ranging from automating administrative tasks to augmenting clinical decision making. However, these models also pose a danger of perpetuating biases and delivering incorrect medical diagnoses, which can have a direct, harmful impact on medical care. We aimed to assess whether GPT-4 encodes racial and gender biases that impact its use in health care. METHODS: Using the Azure OpenAI application interface, this model evaluation study tested whether GPT-4 encodes racial and gender biases and examined the impact of such biases on four potential applications of LLMs in the clinical domain-namely, medical education, diagnostic reasoning, clinical plan generation, and subjective patient assessment. We conducted experiments with prompts designed to resemble typical use of GPT-4 within clinical and medical education applications. We used clinical vignettes from NEJM Healer and from published research on implicit bias in health care. GPT-4 estimates of the demographic distribution of medical conditions were compared with true US prevalence estimates. Differential diagnosis and treatment planning were evaluated across demographic groups using standard statistical tests for significance between groups. FINDINGS: We found that GPT-4 did not appropriately model the demographic diversity of medical conditions, consistently producing clinical vignettes that stereotype demographic presentations. The differential diagnoses created by GPT-4 for standardised clinical vignettes were more likely to include diagnoses that stereotype certain races, ethnicities, and genders. Assessment and plans created by the model showed significant association between demographic attributes and recommendations for more expensive procedures as well as differences in patient perception. INTERPRETATION: Our findings highlight the urgent need for comprehensive and transparent bias assessments of LLM tools such as GPT-4 for intended use cases before they are integrated into clinical care. We discuss the potential sources of these biases and potential mitigation strategies before clinical implementation. FUNDING: Priscilla Chan and Mark Zuckerberg.

Escalated police stops of Black men are linguistically and psychologically distinct in their earliest moments.

Across the United States, police chiefs, city officials, and community leaders alike have highlighted the need to de-escalate police encounters with the public. This concern about escalation extends from encounters involving use of force to routine car stops, where Black drivers are disproportionately pulled over. Yet, despite the calls for action, we know little about the trajectory of police stops or how escalation unfolds. In study 1, we use methods from computational linguistics to analyze police body-worn camera footage from 577 stops of Black drivers. We find that stops with escalated outcomes (those ending in arrest, handcuffing, or a search) diverge from stops without these outcomes in their earliest moments-even in the first 45 words spoken by the officer. In stops that result in escalation, officers are more likely to issue commands as their opening words to the driver and less likely to tell drivers the reason why they are being stopped. In study 2, we expose Black males to audio clips of the same stops and find differences in how escalated stops are perceived: Participants report more negative emotion, appraise officers more negatively, worry about force being used, and predict worse outcomes after hearing only the officer's initial words in escalated versus non-escalated stops. Our findings show that car stops that end in escalated outcomes sometimes begin in an escalated fashion, with adverse effects for Black male drivers and, in turn, police-community relations.

Computational analysis of 140 years of US political speeches reveals more positive but increasingly polarized framing of immigration.

We classify and analyze 200,000 US congressional speeches and 5,000 presidential communications related to immigration from 1880 to the present. Despite the salience of antiimmigration rhetoric today, we find that political speech about immigration is now much more positive on average than in the past, with the shift largely taking place between World War II and the passage of the Immigration and Nationality Act in 1965. However, since the late 1970s, political parties have become increasingly polarized in their expressed attitudes toward immigration, such that Republican speeches today are as negative as the average congressional speech was in the 1920s, an era of strict immigration quotas. Using an approach based on contextual embeddings of text, we find that modern Republicans are significantly more likely to use language that is suggestive of metaphors long associated with immigration, such as "animals" and "cargo," and make greater use of frames like "crime" and "legality." The tone of speeches also differs strongly based on which nationalities are mentioned, with a striking similarity between how Mexican immigrants are framed today and how Chinese immigrants were framed during the era of Chinese exclusion in the late 19th century. Overall, despite more favorable attitudes toward immigrants and the formal elimination of race-based restrictions, nationality is still a major factor in how immigrants are spoken of in Congress.

Diversifying history: A large-scale analysis of changes in researcher demographics and scholarly agendas.

BACKGROUND: In recent years, interest has grown in whether and to what extent demographic diversity sparks discovery and innovation in research. At the same time, topic modeling has been employed to discover differences in what women and men write about. This study engages these two strands of scholarship to explore associations between changing researcher demographics and research questions asked in the discipline of history. Specifically, we analyze developments in history as women entered the field. METHODS: We focus on author gender in diachronic analysis of history dissertations from 1980 (when online data is first available) to 2015 and a select set of general history journals from 1950 to 2015. We use correlated topic modeling and network visualizations to map developments in research agendas over time and to examine how women and men have contributed to these developments. RESULTS: Our summary snapshot of aggregate interests of women and men for the period 1950 to 2015 identifies new topics associated with women authors: gender and women's history, body history, family and households, consumption and consumerism, and sexuality. Diachronic analysis demonstrates that while women pioneered topics such as gender and women's history or the history of sexuality, these topics broaden over time to become methodological frameworks that historians widely embraced and that changed in interesting ways as men engaged with them. Our analysis of history dissertations surface correlations between advisor/advisee gender pairings and choice of dissertation topic. CONCLUSIONS: Overall, this quantitative longitudinal study suggests that the growth in women historians has coincided with the broadening of research agendas and an increased sensitivity to new topics and methodologies in the field.

Language in popular American culture constructs the meaning of healthy and unhealthy eating: Narratives of craveability, excitement, and social connection in movies, television, social media, recipes, and food reviews.

Many people want to eat healthier but struggle to do so, in part due to a dominant perception that healthy foods are at odds with hedonic goals. Is the perception that healthy foods are less appealing than unhealthy foods represented in language across popular entertainment media and social media? Six studies analyzed dialogue about food in six cultural products - creations of a culture that reflect its perspectives - including movies, television, social media posts, food recipes, and food reviews. In Study 1 (N = 617 movies) and Study 2 (N = 27 television shows), healthy foods were described with fewer appealing descriptions (e.g., "couldn't stop eating"; d = 0.59 and d = 0.37, respectively) and more unappealing descriptions (e.g., "I hate peas"; d = -.57 and d = -.63, respectively) than unhealthy foods in characters' speech from the film and television industries. Using sources with richer descriptive language, Studies 3-6 analyzed popular American restaurants' Facebook posts (Study 3, N = 2275), recipe descriptions from Allrecipes.com (Study 4, N = 1000), Yelp reviews from six U.S. cities (Study 5, N = 4403), and Twitter tweets (Study 6, N = 10,000) for seven specific themes. Meta-analytic results across Studies 3-6 showed that healthy foods were specifically described as less craveworthy (d = 0.51, 95% CI: 0.44-0.59), less exciting (d = 0.40, 95% CI: 0.31-0.49), and less social (d = 0.36, 95% CI: 0.04-0.68) than unhealthy foods. Machine learning methods further generalized patterns across 1.6 million tweets spanning 42 different foods representing a range of nutritional quality. These data suggest that strategies to encourage healthy choices must counteract pervasive narratives that dissociate healthy foods from craveability, excitement, and social connection in individuals' everyday lives.

A Suite of Mobile Conversational Agents for Daily Stress Management (Popbots): Mixed Methods Exploratory Study.

BACKGROUND: Approximately 60%-80% of the primary care visits have a psychological stress component, but only 3% of patients receive stress management advice during these visits. Given recent advances in natural language processing, there is renewed interest in mental health chatbots. Conversational agents that can understand a user's problems and deliver advice that mitigates the effects of daily stress could be an effective public health tool. However, such systems are complex to build and costly to develop. OBJECTIVE: To address these challenges, our aim is to develop and evaluate a fully automated mobile suite of shallow chatbots-we call them Popbots-that may serve as a new species of chatbots and further complement human assistance in an ecosystem of stress management support. METHODS: After conducting an exploratory Wizard of Oz study (N=14) to evaluate the feasibility of a suite of multiple chatbots, we conducted a web-based study (N=47) to evaluate the implementation of our prototype. Each participant was randomly assigned to a different chatbot designed on the basis of a proven cognitive or behavioral intervention method. To measure the effectiveness of the chatbots, the participants' stress levels were determined using self-reported psychometric evaluations (eg, web-based daily surveys and Patient Health Questionnaire-4). The participants in these studies were recruited through email and enrolled on the web, and some of them participated in follow-up interviews that were conducted in person or on the web (as necessary). RESULTS: Of the 47 participants, 31 (66%) completed the main study. The findings suggest that the users viewed the conversations with our chatbots as helpful or at least neutral and came away with increasingly positive sentiment toward the use of chatbots for proactive stress management. Moreover, those users who used the system more often (ie, they had more than or equal to the median number of conversations) noted a decrease in depression symptoms compared with those who used the system less often based on a Wilcoxon signed-rank test (W=91.50; Z=-2.54; P=.01; r=0.47). The follow-up interviews with a subset of the participants indicated that half of the common daily stressors could be discussed with chatbots, potentially reducing the burden on human coping resources. CONCLUSIONS: Our work suggests that suites of shallow chatbots may offer benefits for both users and designers. As a result, this study's contributions include the design and evaluation of a novel suite of shallow chatbots for daily stress management, a summary of benefits and challenges associated with random delivery of multiple conversational interventions, and design guidelines and directions for future research into similar systems, including authoring chatbot systems and artificial intelligence-enabled recommendation algorithms.

The thin blue waveform: Racial disparities in officer prosody undermine institutional trust in the police.

How do routine police encounters build or undermine community trust, and how might they contribute to racial gaps in citizen perceptions of the police? Procedural justice theory posits that officers' interpersonal communication toward the public plays a formative role, but experimental tests of this hypothesis have been constrained by the difficulties of measuring and manipulating this dimension of officer-citizen interactions. Officer-worn body camera recordings provide a novel means to overcome both of these challenges. Across five studies with laboratory and community samples, we use footage from traffic stops to examine how officers communicate to drivers and whether racial disparities in officers' communication erode institutional trust in the police. Specifically, we consider the cumulative effects of one subtle interpersonal cue: an officer's tone of voice. In Studies 1A, 1B, and 1C, participants rated thin slices of officer speech. Participants were blind to the content of the officer's words and the race of their interlocutor, yet they evaluated officers' tone toward White (vs. Black) men more positively. By manipulating participants' exposure to repeated interactions, we demonstrate that even these paraverbal aspects of police interactions shape how citizens construe the police generally (Study 2), and that racial disparities in prosodic cues undermine trust in institutions such as police departments (Study 3). Participants' trust in the police, and personal experiences of fairness, in turn, correlated with their perceptions of officer prosody across studies. Taken together, these data illustrate a cycle through which interpersonal aspects of police encounters erode institutional trust across race. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Gender Differences in Patient Perceptions of Physicians' Communal Traits and the Impact on Physician Evaluations.

Background: Communal traits, such as empathy, warmth, and consensus-building, are not highly valued in the medical hierarchy. Devaluing communal traits is potentially harmful for two reasons. First, data suggest that patients may prefer when physicians show communal traits. Second, if female physicians are more likely to be perceived as communal, devaluing communal traits may increase the gender inequity already prevalent in medicine. We test for both these effects. Materials and Methods: This study analyzed 22,431 Press Ganey outpatient surveys assessing 480 physicians collected from 2016 to 2017 at a large tertiary hospital. The surveys asked patients to provide qualitative comments and quantitative Likert-scale ratings assessing physician effectiveness. We coded whether patients described physicians with "communal" language using a validated word scale derived from previous work. We used multivariate logistic regressions to assess whether (1) patients were more likely to describe female physicians using communal language and (2) patients gave higher quantitative ratings to physicians they described with communal language, when controlling for physician, patient, and comment characteristics. Results: Female physicians had higher odds of being described with communal language than male physicians (odds ratio 1.29, 95% confidence interval 1.18-1.40, p < 0.001). In addition, patients gave higher quantitative ratings to physicians they described with communal language. These results were robust to inclusion of controls. Conclusions: Female physicians are more likely to be perceived as communal. Being perceived as communal is associated with higher quantitative ratings, including likelihood to recommend. Our study indicates a need to reevaluate what types of behaviors academic hospitals reward in their physicians.

Five-star prices, appealing healthy item descriptions? Expensive restaurants' descriptive menu language.

Objective: Prior research shows that America's top-selling inexpensive casual dining restaurants use less appealing language to describe healthy menu items than standard items. This may suggest to diners that healthy options are less tasty and enjoyable. The present research asked whether expensive restaurants also use less appealing language to describe healthy items, or whether healthy items are described with equally appealing language as standard items in high status dining contexts. Method: Using Yelp, the name and description of every food item were recorded from the menus of 160 top-rated expensive restaurants across 8 U.S. cities (Nitems = 3,295; Nwords = 32,516). Healthy menu items were defined as salads and side vegetables, and standard items as all other dishes (excluding desserts), with high interrater reliability (K = .89). Descriptive words were categorized into 22 predefined themes, and log likelihood analyses compared normalized theme frequencies from standard item and healthy item descriptions. Results: Healthy items were described with 4.8-times fewer American region words, 2.7-times fewer exciting words, 1.4-times fewer tasty words, and significantly fewer portion size, spicy, artisanal, and foreign region words. Unlike inexpensive restaurants, however, expensive restaurants did not use any health-focused themes to promote healthy items and used several appealing themes more frequently in healthy item descriptions. Conclusions: Like inexpensive restaurants, expensive American restaurants described healthy items as less appealing and less authentically American than standard foods, but to a lesser extent. Implications for ordering behavior and solutions for improving the appeal of healthy menu items are discussed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Assessing the accuracy of automatic speech recognition for psychotherapy.

Accurate transcription of audio recordings in psychotherapy would improve therapy effectiveness, clinician training, and safety monitoring. Although automatic speech recognition software is commercially available, its accuracy in mental health settings has not been well described. It is unclear which metrics and thresholds are appropriate for different clinical use cases, which may range from population descriptions to individual safety monitoring. Here we show that automatic speech recognition is feasible in psychotherapy, but further improvements in accuracy are needed before widespread use. Our HIPAA-compliant automatic speech recognition system demonstrated a transcription word error rate of 25%. For depression-related utterances, sensitivity was 80% and positive predictive value was 83%. For clinician-identified harm-related sentences, the word error rate was 34%. These results suggest that automatic speech recognition may support understanding of language patterns and subgroup variation in existing treatments but may not be ready for individual-level safety surveillance.

The Diversity-Innovation Paradox in Science.

Prior work finds a diversity paradox: Diversity breeds innovation, yet underrepresented groups that diversify organizations have less successful careers within them. Does the diversity paradox hold for scientists as well? We study this by utilizing a near-complete population of ∼1.2 million US doctoral recipients from 1977 to 2015 and following their careers into publishing and faculty positions. We use text analysis and machine learning to answer a series of questions: How do we detect scientific innovations? Are underrepresented groups more likely to generate scientific innovations? And are the innovations of underrepresented groups adopted and rewarded? Our analyses show that underrepresented groups produce higher rates of scientific novelty. However, their novel contributions are devalued and discounted: For example, novel contributions by gender and racial minorities are taken up by other scholars at lower rates than novel contributions by gender and racial majorities, and equally impactful contributions of gender and racial minorities are less likely to result in successful scientific careers than for majority groups. These results suggest there may be unwarranted reproduction of stratification in academic careers that discounts diversity's role in innovation and partly explains the underrepresentation of some groups in academia.

Racial disparities in automated speech recognition.

Automated speech recognition (ASR) systems, which use sophisticated machine-learning algorithms to convert spoken language to text, have become increasingly widespread, powering popular virtual assistants, facilitating automated closed captioning, and enabling digital dictation platforms for health care. Over the last several years, the quality of these systems has dramatically improved, due both to advances in deep learning and to the collection of large-scale datasets used to train the systems. There is concern, however, that these tools do not work equally well for all subgroups of the population. Here, we examine the ability of five state-of-the-art ASR systems-developed by Amazon, Apple, Google, IBM, and Microsoft-to transcribe structured interviews conducted with 42 white speakers and 73 black speakers. In total, this corpus spans five US cities and consists of 19.8 h of audio matched on the age and gender of the speaker. We found that all five ASR systems exhibited substantial racial disparities, with an average word error rate (WER) of 0.35 for black speakers compared with 0.19 for white speakers. We trace these disparities to the underlying acoustic models used by the ASR systems as the race gap was equally large on a subset of identical phrases spoken by black and white individuals in our corpus. We conclude by proposing strategies-such as using more diverse training datasets that include African American Vernacular English-to reduce these performance differences and ensure speech recognition technology is inclusive.

Universals of word order reflect optimization of grammars for efficient communication.

The universal properties of human languages have been the subject of intense study across the language sciences. We report computational and corpus evidence for the hypothesis that a prominent subset of these universal properties-those related to word order-result from a process of optimization for efficient communication among humans, trading off the need to reduce complexity with the need to reduce ambiguity. We formalize these two pressures with information-theoretic and neural-network models of complexity and ambiguity and simulate grammars with optimized word-order parameters on large-scale data from 51 languages. Evolution of grammars toward efficiency results in word-order patterns that predict a large subset of the major word-order correlations across languages.

A Framework for the Computational Linguistic Analysis of Dehumanization.

Dehumanization is a pernicious psychological process that often leads to extreme intergroup bias, hate speech, and violence aimed at targeted social groups. Despite these serious consequences and the wealth of available data, dehumanization has not yet been computationally studied on a large scale. Drawing upon social psychology research, we create a computational linguistic framework for analyzing dehumanizing language by identifying linguistic correlates of salient components of dehumanization. We then apply this framework to analyze discussions of LGBTQ people in the New York Times from 1986 to 2015. Overall, we find increasingly humanizing descriptions of LGBTQ people over time. However, we find that the label homosexual has emerged to be much more strongly associated with dehumanizing attitudes than other labels, such as gay. Our proposed techniques highlight processes of linguistic variation and change in discourses surrounding marginalized groups. Furthermore, the ability to analyze dehumanizing language at a large scale has implications for automatically detecting and understanding media bias as well as abusive language online.

Seekers, Providers, Welcomers, and Storytellers: Modeling Social Roles in Online Health Communities.

Participants in online communities often enact different roles when participating in their communities. For example, some in cancer support communities specialize in providing disease-related information or socializing new members. This work clusters the behavioral patterns of users of a cancer support community into specific functional roles. Based on a series of quantitative and qualitative evaluations, this research identified eleven roles that members occupy, such as welcomer and story sharer. We investigated role dynamics, including how roles change over members' lifecycles, and how roles predict long-term participation in the community. We found that members frequently change roles over their history, from ones that seek resources to ones offering help, while the distribution of roles is stable over the community's history. Adopting certain roles early on predicts members' continued participation in the community. Our methodology will be useful for facilitating better use of members' skills and interests in support of community-building efforts.

Word embeddings quantify 100 years of gender and ethnic stereotypes.

Word embeddings are a powerful machine-learning framework that represents each English word by a vector. The geometric relationship between these vectors captures meaningful semantic relationships between the corresponding words. In this paper, we develop a framework to demonstrate how the temporal dynamics of the embedding helps to quantify changes in stereotypes and attitudes toward women and ethnic minorities in the 20th and 21st centuries in the United States. We integrate word embeddings trained on 100 y of text data with the US Census to show that changes in the embedding track closely with demographic and occupation shifts over time. The embedding captures societal shifts-e.g., the women's movement in the 1960s and Asian immigration into the United States-and also illuminates how specific adjectives and occupations became more closely associated with certain populations over time. Our framework for temporal analysis of word embedding opens up a fruitful intersection between machine learning and quantitative social science.