Multimodal analysis of disinformation and misinformation.

The use of disinformation and misinformation campaigns in the media has attracted much attention from academics and policy-makers. Multimodal analysis or the analysis of two or more semiotic systems-language, gestures, images, sounds, among others-in their interrelation and interaction is essential to understanding dis-/misinformation efforts because most human communication goes beyond just words. There is a confluence of many disciplines (e.g. computer science, linguistics, political science, communication studies) that are developing methods and analytical models of multimodal communication. This literature review brings research strands from these disciplines together, providing a map of the multi- and interdisciplinary landscape for multimodal analysis of dis-/misinformation. It records the substantial growth starting from the second quarter of 2020-the start of the COVID-19 epidemic in Western Europe-in the number of studies on multimodal dis-/misinformation coming from the field of computer science. The review examines that category of studies in more detail. Finally, the review identifies gaps in multimodal research on dis-/misinformation and suggests ways to bridge these gaps including future cross-disciplinary research directions. Our review provides scholars from different disciplines working on dis-/misinformation with a much needed bird's-eye view of the rapidly emerging research of multimodal dis-/misinformation.

Measuring the Volatility of the Political agenda in Public Opinion and News Media.

Recent election surprises, regime changes, and political shocks indicate that political agendas have become more fast-moving and volatile. The ability to measure the complex dynamics of agenda change and capture the nature and extent of volatility in political systems is therefore more crucial than ever before. This study proposes a definition and operationalization of volatility that combines insights from political science, communications, information theory, and computational techniques. The proposed measures of fractionalization and agenda change encompass the shifting salience of issues in the agenda as a whole and allow the study of agendas across different domains. We evaluate these metrics and compare them to other measures such as issue-level survival rates and the Pedersen Index, which uses public-opinion poll data to measure public agendas, as well as traditional media content to measure media agendas in the UK and Germany. We show how these measures complement existing approaches and could be employed in future agenda-setting research.

Measuring relative opinion from location-based social media: A case study of the 2016 U.S. presidential election.

Social media has become an emerging alternative to opinion polls for public opinion collection, while it is still posing many challenges as a passive data source, such as structurelessness, quantifiability, and representativeness. Social media data with geotags provide new opportunities to unveil the geographic locations of users expressing their opinions. This paper aims to answer two questions: 1) whether quantifiable measurement of public opinion can be obtained from social media and 2) whether it can produce better or complementary measures compared to opinion polls. This research proposes a novel approach to measure the relative opinion of Twitter users towards public issues in order to accommodate more complex opinion structures and take advantage of the geography pertaining to the public issues. To ensure that this new measure is technically feasible, a modeling framework is developed including building a training dataset by adopting a state-of-the-art approach and devising a new deep learning method called Opinion-Oriented Word Embedding. With a case study of tweets on the 2016 U.S. presidential election, we demonstrate the predictive superiority of our relative opinion approach and we show how it can aid visual analytics and support opinion predictions. Although the relative opinion measure is proved to be more robust than polling, our study also suggests that the former can advantageously complement the latter in opinion prediction.

Estimating Traffic Disruption Patterns with Volunteered Geographic Information.

Accurate understanding and forecasting of traffic is a key contemporary problem for policymakers. Road networks are increasingly congested, yet traffic data is often expensive to obtain, making informed policy-making harder. This paper explores the extent to which traffic disruption can be estimated using features from the volunteered geographic information site OpenStreetMap (OSM). We use OSM features as predictors for linear regressions of counts of traffic disruptions and traffic volume at 6,500 points in the road network within 112 regions of Oxfordshire, UK. We show that more than half the variation in traffic volume and disruptions can be explained with OSM features alone, and use cross-validation and recursive feature elimination to evaluate the predictive power and importance of different land use categories. Finally, we show that using OSM's granular point of interest data allows for better predictions than the broader categories typically used in studies of transportation and land use.

Using immersive virtual reality to improve the beliefs and intentions of influenza vaccine avoidant 18-to-49-year-olds: Considerations, effects, and lessons learned.

OBJECTIVE: Only one-third of adults 18-49 years old in the United States receive a recommended annual influenza vaccination. This study examined whether supplementing vaccine information statements (VIS) with an immersive virtual reality (VR), short video or electronic pamphlet story designed to convey the community immunity benefits of influenza vaccination would improve influenza vaccine avoidant participants' influenza-related perceptions as well as their influenza vaccination-related beliefs, confidence and intentions. METHOD: A one-way between-subjects experimental design compared the effects of adding a supplemental education experience prior to VIS exposure with flu vaccine avoidant 18-to-49-year-olds. The 171 participants recruited from the community were randomly assigned to one of three modality treatment conditions [VR, video, or e-pamphlet (i.e., story board presented via electronic tablet)] or a VIS-only control condition. RESULTS: Compared to the modalities, the VR intervention created a stronger perception of presence (i.e., feeling of "being there" in the story), which, in turn, increased participants' concern about transmitting influenza to others and raised vaccination intention. Increased concern about transmitting influenza to others was associated with positive effects on influenza vaccination-related beliefs, including confidence that one's flu vaccination would protect others. Neither the e-pamphlet nor the video intervention were able to elicit a sense of presence nor were they able to improve the impact of the VIS on the outcome measures. CONCLUSIONS: Immersive VR has much potential to increase understanding of key immunization concepts, such as community immunity, through creative executions that increase a sense of presence. Given the need to increase influenza vaccination uptake among 18-to-49-year-olds, and the projected growth in VR accessibility and use, additional applications and assessments related to vaccination communication and education are needed and warranted. By increasing the ability to convey key vaccine and immunization concepts, immersive VR could help address vaccination hesitancy and acceptance challenges.

Diagnosing the performance of human mobility models at small spatial scales using volunteered geographical information.

Accurate modelling of local population movement patterns is a core, contemporary concern for urban policymakers, affecting both the short-term deployment of public transport resources and the longer-term planning of transport infrastructure. Yet, while macro-level population movement models (such as the gravity and radiation models) are well developed, micro-level alternatives are in much shorter supply, with most macro-models known to perform poorly at smaller geographical scales. In this paper, we take a first step to remedy this deficit, by leveraging two novel datasets to analyse where and why macro-level models of human mobility break down. We show how freely available data from OpenStreetMap concerning land use composition of different areas around the county of Oxfordshire in the UK can be used to diagnose mobility models and understand the types of trips they over- and underestimate when compared with empirical volumes derived from aggregated, anonymous smartphone location data. We argue for new modelling strategies that move beyond rough heuristics such as distance and population towards a detailed, granular understanding of the opportunities presented in different regions.

How digital design shapes political participation: A natural experiment with social information.

Political behaviour increasingly takes place on digital platforms, where people are presented with a range of social information-real-time feedback about the behaviour of peers and reference groups-which can stimulate (or depress) participation. This social information is hypothesized to impact the distribution of political activity, stimulating participation in mobilizations that are increasing in popularity, and depressing participation in those that appear to be less popular, leading to a non-normal distribution. Changes to these platforms can generate natural experiments allowing for an estimate of the impact of different kinds of social information on participation. This paper tests the hypothesis that social information shapes the distribution of political mobilizations by examining the introduction of trending information to the homepage of the UK government petition platform. The introduction of the trending feature did not increase the overall number of signatures per day, but the distribution of signatures across petitions changed significantly-the most popular petitions gained more signatures at the expense of those with fewer signatories. We further find significant differences between petitions trending at different ranks on the homepage. This evidence suggests that the ubiquity of trending information on digital platforms is introducing instability into political markets, as has been shown for cultural markets. As well as highlighting the importance of digital design in shaping political behaviour, the findings suggest that a non-negligible group of individuals visit the homepage of the site looking for petitions to sign, without having decided the issues they wish to support in advance. These 'aimless petitioners' are particularly susceptible to changes in social information.

The October 2014 United States Treasury bond flash crash and the contributory effect of mini flash crashes.

We investigate the causal uncertainty surrounding the flash crash in the U.S. Treasury bond market on October 15, 2014, and the unresolved concern that no clear link has been identified between the start of the flash crash at 9:33 and the opening of the U.S. equity market at 9:30. We consider the contributory effect of mini flash crashes in equity markets, and find that the number of equity mini flash crashes in the three-minute window between market open and the Treasury Flash Crash was 2.6 times larger than the number experienced in any other three-minute window in the prior ten weekdays. We argue that (a) this statistically significant finding suggests that mini flash crashes in equity markets both predicted and contributed to the October 2014 U.S. Treasury Bond Flash Crash, and (b) mini-flash crashes are important phenomena with negative externalities that deserve much greater scholarly attention.

Understanding Editing Behaviors in Multilingual Wikipedia.

Multilingualism is common offline, but we have a more limited understanding of the ways multilingualism is displayed online and the roles that multilinguals play in the spread of content between speakers of different languages. We take a computational approach to studying multilingualism using one of the largest user-generated content platforms, Wikipedia. We study multilingualism by collecting and analyzing a large dataset of the content written by multilingual editors of the English, German, and Spanish editions of Wikipedia. This dataset contains over two million paragraphs edited by over 15,000 multilingual users from July 8 to August 9, 2013. We analyze these multilingual editors in terms of their engagement, interests, and language proficiency in their primary and non-primary (secondary) languages and find that the English edition of Wikipedia displays different dynamics from the Spanish and German editions. Users primarily editing the Spanish and German editions make more complex edits than users who edit these editions as a second language. In contrast, users editing the English edition as a second language make edits that are just as complex as the edits by users who primarily edit the English edition. In this way, English serves a special role bringing together content written by multilinguals from many language editions. Nonetheless, language remains a formidable hurdle to the spread of content: we find evidence for a complexity barrier whereby editors are less likely to edit complex content in a second language. In addition, we find that multilinguals are less engaged and show lower levels of language proficiency in their second languages. We also examine the topical interests of multilingual editors and find that there is no significant difference between primary and non-primary editors in each language.

Detection and quantification of species authenticity and adulteration in crabmeat using visible and near-infrared spectroscopy.

Seafood processing often removes morphological properties of seafood species that enable the consumer to distinguish one type of organism from another. For this reason, species substitution is the most common form of economic adulteration in the seafood industry. Visible and near-infrared spectroscopy (Vis/NIR) has been used to detect and quantify species authenticity and adulteration in crabmeat samples. Atlantic blue crabmeat was adulterated with blue swimmer crabmeat in 10% increments. Water absorption bands dominated the main features in the crabmeat spectra, with a decrease in sample absorbance with increasing adulteration percentage. Several data pretreatments, i.e., moving average, combing, first and second derivatives, and multiplicative scatter correction, in addition to the raw data, were investigated for prediction and quantitative data analysis using partial least-squares. In addition, quantitative analysis was done using the full spectrum and a sequential approach in which 50 wavelengths were added sequentially to determine a new model and find an optimal solution. The results suggest that Vis/NIR spectroscopy is a suitable technology that can be applied to detect and quantify species authenticity and adulteration in crabmeat.

Quantitative analysis and detection of adulteration in crab meat using visible and near-infrared spectroscopy.

Visible and near-infrared spectroscopy (VIS/NIR) has been used to detect economic adulteration of crab meat samples. Atlantic blue and blue swimmer crab meat samples were adulterated with surimi-based imitation crab meat in 10% increments. Waveform evaluation revealed that the main features seen in the spectral data arise from water absorptions with a decrease in sample absorbance with increasing adulteration level. Prediction and quantitative analysis was done using raw data, a 15-point smoothing average, a first derivative, a second derivative, and 150 wavelength spectral data gathered from a correlogram. Regression analysis included partial least squares (PLS) and principal component analysis (PCR). Both models were able to perform similarly in predicting crab meat adulteration. The best model for both PLS and PCR used the first derivative spectral data gathered from the correlogram, with a standard error of prediction (SEP) of 0.252 and 0.244, respectively. The results suggest that VIS/NIR technology can be successfully used to detect adulteration in crab meat samples adulterated with surimi-based imitation crab meat.