Auditing YouTube's recommendation system for ideologically congenial, extreme, and problematic recommendations.

Algorithms of social media platforms are often criticized for recommending ideologically congenial and radical content to their users. Despite these concerns, evidence on such filter bubbles and rabbit holes of radicalization is inconclusive. We conduct an audit of the platform using 100,000 sock puppets that allow us to systematically and at scale isolate the influence of the algorithm in recommendations. We test 1) whether recommended videos are congenial with regard to users' ideology, especially deeper in the watch trail and whether 2) recommendations deeper in the trail become progressively more extreme and come from problematic channels. We find that YouTube's algorithm recommends congenial content to its partisan users, although some moderate and cross-cutting exposure is possible and that congenial recommendations increase deeper in the trail for right-leaning users. We do not find meaningful increases in ideological extremity of recommendations deeper in the trail, yet we show that a growing proportion of recommendations comes from channels categorized as problematic (e.g., "IDW," "Alt-right," "Conspiracy," and "QAnon"), with this increase being most pronounced among the very-right users. Although the proportion of these problematic recommendations is low (max of 2.5%), they are still encountered by over 36.1% of users and up to 40% in the case of very-right users.

Multiple imputation in data that grow over time: a comparison of three strategies.

Multiple imputation is a recommended technique to deal with missing data. We study the problem where the investigator has already created imputations before the arrival of the next wave of data. The newly arriving data contain missing values that need to be imputed. The standard method (RE-IMPUTE) is to combine the new and old data before imputation, and re-impute all missing values in the combined data. We study the properties of two methods that impute the missing data in the new part only, thus preserving the historic imputations. Method NEST multiply imputes the new data conditional on each filled-in old data m2>1 times. Method APPEND is the special case of NEST with m2=1, thus appending each filled-in data by single imputation. We found that NEST and APPEND have the same validity as RE-IMPUTE for monotone missing data-patterns. NEST and APPEND also work well when relations within waves are stronger than between waves and for moderate percentages of missing data. We do not recommend the use of NEST or APPEND when relations within time points are weak and when associations between time points are strong.

Evaluation of approaches for accommodating interactions and non-linear terms in multiple imputation of incomplete three-level data.

Three-level data structures arising from repeated measures on individuals clustered within larger units are common in health research studies. Missing data are prominent in such studies and are often handled via multiple imputation (MI). Although several MI approaches can be used to account for the three-level structure, including adaptations to single- and two-level approaches, when the substantive analysis model includes interactions or quadratic effects, these too need to be accommodated in the imputation model. In such analyses, substantive model compatible (SMC) MI has shown great promise in the context of single-level data. Although there have been recent developments in multilevel SMC MI, to date only one approach that explicitly handles incomplete three-level data is available. Alternatively, researchers can use pragmatic adaptations to single- and two-level MI approaches, or two-level SMC-MI approaches. We describe the available approaches and evaluate them via simulations in the context of three three-level random effects analysis models involving an interaction between the incomplete time-varying exposure and time, an interaction between the time-varying exposure and an incomplete time-fixed confounder, or a quadratic effect of the exposure. Results showed that all approaches considered performed well in terms of bias and precision when the target analysis involved an interaction with time, but the three-level SMC MI approach performed best when the target analysis involved an interaction between the time-varying exposure and an incomplete time-fixed confounder, or a quadratic effect of the exposure. We illustrate the methods using data from the Childhood to Adolescence Transition Study.

Graphical and numerical diagnostic tools to assess suitability of multiple imputations and imputation models.

Multiple imputation has become a popular approach for analyzing incomplete data. Many software packages are available to multiply impute the missing values and to analyze the resulting completed data sets. However, diagnostic tools to check the validity of the imputations are limited, and the majority of the currently available methods need considerable knowledge of the imputation model. In many practical settings, however, the imputer and the analyst may be different individuals or from different organizations, and the analyst model may or may not be congenial to the model used by the imputer. This article develops and evaluates a set of graphical and numerical diagnostic tools for two practical purposes: (i) for an analyst to determine whether the imputations are reasonable under his/her model assumptions without actually knowing the imputation model assumptions; and (ii) for an imputer to fine tune the imputation model by checking the key characteristics of the observed and imputed values. The tools are based on the numerical and graphical comparisons of the distributions of the observed and imputed values conditional on the propensity of response. The methodology is illustrated using simulated data sets created under a variety of scenarios. The examples focus on continuous and binary variables, but the principles can be used to extend methods for other types of variables. Copyright © 2016 John Wiley & Sons, Ltd.

Bootstrap inference for multiple imputation under uncongeniality and misspecification.

Multiple imputation has become one of the most popular approaches for handling missing data in statistical analyses. Part of this success is due to Rubin's simple combination rules. These give frequentist valid inferences when the imputation and analysis procedures are so-called congenial and the embedding model is correctly specified, but otherwise may not. Roughly speaking, congeniality corresponds to whether the imputation and analysis models make different assumptions about the data. In practice, imputation models and analysis procedures are often not congenial, such that tests may not have the correct size, and confidence interval coverage deviates from the advertised level. We examine a number of recent proposals which combine bootstrapping with multiple imputation and determine which are valid under uncongeniality and model misspecification. Imputation followed by bootstrapping generally does not result in valid variance estimates under uncongeniality or misspecification, whereas certain bootstrap followed by imputation methods do. We recommend a particular computationally efficient variant of bootstrapping followed by imputation.

Relative efficiency of joint-model and full-conditional-specification multiple imputation when conditional models are compatible: The general location model.

Estimating the parameters of a regression model of interest is complicated by missing data on the variables in that model. Multiple imputation is commonly used to handle these missing data. Joint model multiple imputation and full-conditional specification multiple imputation are known to yield imputed data with the same asymptotic distribution when the conditional models of full-conditional specification are compatible with that joint model. We show that this asymptotic equivalence of imputation distributions does not imply that joint model multiple imputation and full-conditional specification multiple imputation will also yield asymptotically equally efficient inference about the parameters of the model of interest, nor that they will be equally robust to misspecification of the joint model. When the conditional models used by full-conditional specification multiple imputation are linear, logistic and multinomial regressions, these are compatible with a restricted general location joint model. We show that multiple imputation using the restricted general location joint model can be substantially more asymptotically efficient than full-conditional specification multiple imputation, but this typically requires very strong associations between variables. When associations are weaker, the efficiency gain is small. Moreover, full-conditional specification multiple imputation is shown to be potentially much more robust than joint model multiple imputation using the restricted general location model to mispecification of that model when there is substantial missingness in the outcome variable.

Putting congeniality effects into context: Investigating the role of context in attitude memory using multiple paradigms.

In three experiments, we evaluated remembering and intentional forgetting of attitude statements that were either congruent or incongruent with participants' own political attitudes. In Experiment 1, significant directed forgetting was obtained for incongruent statements, but not for congruent statements. In addition, in the remember group, recall was better for incongruent statements than congruent statements. To explain these findings, we propose a contextual competition at retrieval hypothesis, according to which incongruent statements become more strongly associated with their episodic context during encoding than do congruent statements. At the time of retrieval, incongruent statements compete with congruent statements due to the greater amount of contextual information stored in their memory trace. We tested this hypothesis in Experiment 2 by studying free recall of congruent and incongruent statements in a mixed-pure list design. In Experiment 3, memory for incongruent and congruent statements was tested under recognition test conditions that varied in terms of how much direct retrieval of contextual details they required. Overall, the results supported the contextual competition hypothesis, and they indicate the importance of context strength in both the remembering and intentional forgetting of attitude information.