Modeling Evasive Response Bias in Randomized Response: Cheater Detection Versus Self-protective No-Saying.

Randomized response is an interview technique for sensitive questions designed to eliminate evasive response bias. Since this elimination is only partially successful, two models have been proposed for modeling evasive response bias: the cheater detection model for a design with two sub-samples with different randomization probabilities and the self-protective no sayers model for a design with multiple sensitive questions. This paper shows the correspondence between these models, and introduces models for the new, hybrid "ever/last year" design that account for self-protective no saying and cheating. The model for one set of ever/last year questions has a degree of freedom that can be used for the inclusion of a response bias parameter. Models with multiple degrees of freedom are introduced for extensions of the design with a third randomized response question and a second set of ever/last year questions. The models are illustrated with two surveys on doping use. We conclude with a discussion of the pros and cons of the ever/last year design and its potential for future research.

The analysis of randomized response "ever" and "last year" questions: A non-saturated Multinomial model.

Randomized response (RR) is a well-known interview technique designed to eliminate evasive response bias that arises from asking sensitive questions. The most frequently asked questions in RR are either whether respondents were "ever" carriers of the sensitive characteristic, or whether they were carriers in a recent period, for instance, "last year". The present paper proposes a design in which both questions are asked, and derives a multinomial model for the joint analysis of these two questions. Compared to the separate analyses with the binomial model, the model makes a useful distinction between last year and former carriers of the sensitive characteristic, it is more efficient in estimating the prevalence of last year carriers, and it has a degree of freedom that allows for a goodness-of-fit test. Furthermore, it is easily extended to a multinomial logistic regression model to investigate the effects of covariates on the prevalence estimates. These benefits are illustrated in two studies on the use of anabolic androgenic steroids in the Netherlands, one using Kuk and one using both the Kuk and forced response. A salient result of our analyses is that the multinomial model provided ample evidence of response biases in the forced response condition.

Refinement of the extended crosswise model with a number sequence randomizer: Evidence from three different studies in the UK.

The Extended Crosswise Model (ECWM) is a randomized response model with neutral response categories, relatively simple instructions, and the availability of a goodness-of-fit test. This paper refines this model with a number sequence randomizer that virtually precludes the possibility to give evasive responses. The motivation for developing this model stems from a strategic priority of WADA (World Anti-Doping Agency) to monitor the prevalence of doping use by elite athletes. For this model we derived a maximum likelihood estimator that allows for binary logistic regression analysis. Three studies were conducted on online platforms with a total of over 6, 000 respondents; two on controlled substance use and one on compliance with COVID-19 regulations in the UK during the first lockdown. The results of these studies are promising. The goodness-of-fit tests showed little to no evidence for response biases, and the ECWM yielded higher prevalence estimates than direct questions for sensitive questions, and similar ones for non-sensitive questions. Furthermore, the randomizer with the shortest number sequences yielded the smallest response error rates on a control question with known prevalence.

Functionality of the Crosswise Model for Assessing Sensitive or Transgressive Behavior: A Systematic Review and Meta-Analysis.

Tools for reliable assessment of socially sensitive or transgressive behavior warrant constant development. Among them, the Crosswise Model (CM) has gained considerable attention. We systematically reviewed and meta-analyzed empirical applications of CM and addressed a gap for quality assessment of indirect estimation models. Guided by the PRISMA protocol, we identified 45 empirical studies from electronic database and reference searches. Thirty of these were comparative validation studies (CVS) comparing CM and direct question (DQ) estimates. Six prevalence studies exclusively used CM. One was a qualitative study. Behavior investigated were substance use and misuse (k = 13), academic misconduct (k = 8), and corruption, tax evasion, and theft (k = 7) among others. Majority of studies (k = 39) applied the "more is better" hypothesis. Thirty-five studies relied on birthday distribution and 22 of these used P = 0.25 for the non-sensitive item. Overall, 11 studies were assessed as high-, 31 as moderate-, and two as low quality (excluding the qualitative study). The effect of non-compliance was assessed in eight studies. From mixed CVS results, the meta-analysis indicates that CM outperforms DQ on the "more is better" validation criterion, and increasingly so with higher behavior sensitivity. However, little difference was observed between DQ and CM estimates for items with DQ prevalence estimate around 50%. Based on empirical evidence available to date, our study provides support for the superiority of CM to DQ in assessing sensitive/transgressive behavior. Despite some limitations, CM is a valuable and promising tool for population level investigation.

A multilevel structural equation model for assessing a drug effect on a patient-reported outcome measure in on-demand medication data.

We analyze data from a clinical trial investigating the effect of an on-demand drug for women with low sexual desire. These data consist of a varying number of measurements/events across patients of when the drug was taken, including data on a patient-reported outcome consisting of five items measuring an unobserved construct (latent variable). Traditionally, these data are aggregated prior to analysis by composing one sum score per event and averaging this sum score over all observed events. In this paper, we explain the drawbacks of this aggregating approach. One drawback is that these averages have different standard errors because the variance of the underlying events differs between patients and because the number of events per patient differs. Another drawback is the implicit assumption that all items have equal weight in relation to the latent variable being measured. We propose a multilevel structural equation model, treating the events (level 1) as nested observations within patients (level 2), as alternative analysis method to overcome these drawbacks. The model we apply includes a factor model measuring a latent variable at the level of the event and at the level of the patient. Then, in the same model, the latent variables are regressed on covariates to assess the drug effect. We discuss the inferences obtained about the efficacy of the on-demand drug using our proposed model. We further illustrate how to test for measurement invariance across grouping covariates and levels using the same model.

Automatic multilabel detection of ICD10 codes in Dutch cardiology discharge letters using neural networks.

Standard reference terminology of diagnoses and risk factors is crucial for billing, epidemiological studies, and inter/intranational comparisons of diseases. The International Classification of Disease (ICD) is a standardized and widely used method, but the manual classification is an enormously time-consuming endeavor. Natural language processing together with machine learning allows automated structuring of diagnoses using ICD-10 codes, but the limited performance of machine learning models, the necessity of gigantic datasets, and poor reliability of terminal parts of these codes restricted clinical usability. We aimed to create a high performing pipeline for automated classification of reliable ICD-10 codes in the free medical text in cardiology. We focussed on frequently used and well-defined three- and four-digit ICD-10 codes that still have enough granularity to be clinically relevant such as atrial fibrillation (I48), acute myocardial infarction (I21), or dilated cardiomyopathy (I42.0). Our pipeline uses a deep neural network known as a Bidirectional Gated Recurrent Unit Neural Network and was trained and tested with 5548 discharge letters and validated in 5089 discharge and procedural letters. As in clinical practice discharge letters may be labeled with more than one code, we assessed the single- and multilabel performance of main diagnoses and cardiovascular risk factors. We investigated using both the entire body of text and only the summary paragraph, supplemented by age and sex. Given the privacy-sensitive information included in discharge letters, we added a de-identification step. The performance was high, with F1 scores of 0.76-0.99 for three-character and 0.87-0.98 for four-character ICD-10 codes, and was best when using complete discharge letters. Adding variables age/sex did not affect results. For model interpretability, word coefficients were provided and qualitative assessment of classification was manually performed. Because of its high performance, this pipeline can be useful to decrease the administrative burden of classifying discharge diagnoses and may serve as a scaffold for reimbursement and research applications.

Mothers' emotions after pediatric burn injury: Longitudinal associations with posttraumatic stress and depressive symptoms 18 months postburn.

BACKGROUND: Various emotions are implicated in posttraumatic stress disorder (PTSD). Longitudinal studies examining temporal associations between emotions and posttraumatic stress may reveal who is at risk of chronic psychological problems. This study examined the longitudinal relationships of mothers' trauma-related emotions with posttraumatic stress and depressive symptoms after pediatric burn injury. METHODS: Data from two cohort studies were used (n = 296). Mothers reported the intensity of burn-related emotions within the first month (T1) and 12 months postburn (T2). The Impact of Event Scale (IES) and the Hospital and Anxiety Depression Scale (HADS-D; depression subscale) were administered at T1 and 18 months postburn (T3). RESULTS: Based on two exploratory factor analyses, emotion variables were combined into acute and long-term basic emotions (fear, sadness, horror, anger) and self-conscious emotions (guilt, shame). The path model showed a positive relationship between acute and long-term basic emotions. Higher long-term basic emotions were related to persistence of posttraumatic stress and depressive symptoms. Acute self-conscious emotions showed associations with posttraumatic stress and depressive symptoms at T1 and were longitudinally related to depressive, but not posttraumatic stress, symptoms. LIMITATIONS: The posttraumatic stress measure was not based on DSM-5 PTSD criteria and results require replication using these criteria. CONCLUSIONS: This study suggests that mothers' acute self-conscious and long-term basic emotions in relation to their child's burn injury are involved in the development of posttraumatic stress and depressive symptoms. Clinically, assessing and monitoring parents' early posttraumatic stress, depressive symptoms and burn-related emotions may be useful to identify parents at risk.

Multilevel analyses of on-demand medication data, with an application to the treatment of Female Sexual Interest/Arousal Disorder.

Data from clinical trials investigating on-demand medication often consist of an intentionally varying number of measurements per patient. These measurements are often observations of discrete events of when the medication was taken, including for example data on symptom severity. In addition to the varying number of observations between patients, the data have another important feature: they are characterized by a hierarchical structure in which the events are nested within patients. Traditionally, the observed events of patients are aggregated into means and subsequently analyzed using, for example, a repeated measures ANOVA. This procedure has drawbacks. One drawback is that these patient means have different standard errors, first, because the variance of the underlying events differs between patients and second, because the number of events per patient differs. In this paper, we argue that such data should be analyzed by applying a multilevel analysis using the individual observed events as separate nested observations. Such a multilevel approach handles this drawback and it also enables the examination of varying drug effects across patients by estimating random effects. We show how multilevel analyses can be applied to on-demand medication data from a clinical trial investigating the efficacy of a drug for women with low sexual desire. We also explore linear and quadratic time effects that can only be performed when the individual events are considered as separate observations and we discuss several important statistical topics relevant for multilevel modeling. Taken together, the use of a multilevel approach considering events as nested observations in these types of data is advocated as it is more valid and provides more information than other (traditional) methods.

Optimizing predictive performance of criminal recidivism models using registration data with binary and survival outcomes.

In a recidivism prediction context, there is no consensus on which modeling strategy should be followed for obtaining an optimal prediction model. In previous papers, a range of statistical and machine learning techniques were benchmarked on recidivism data with a binary outcome. However, two important tree ensemble methods, namely gradient boosting and random forests were not extensively evaluated. In this paper, we further explore the modeling potential of these techniques in the binary outcome criminal prediction context. Additionally, we explore the predictive potential of classical statistical and machine learning methods for censored time-to-event data. A range of statistical manually specified statistical and (semi-)automatic machine learning models is fitted on Dutch recidivism data, both for the binary outcome case and censored outcome case. To enhance generalizability of results, the same models are applied to two historical American data sets, the North Carolina prison data. For all datasets, (semi-) automatic modeling in the binary case seems to provide no improvement over an appropriately manually specified traditional statistical model. There is however evidence of slightly improved performance of gradient boosting in survival data. Results on the reconviction data from two sources suggest that both statistical and machine learning should be tried out for obtaining an optimal model. Even if a flexible black-box model does not improve upon the predictions of a manually specified model, it can serve as a test whether important interactions are missing or other misspecification of the model are present and can thus provide more security in the modeling process.

Doing philosophy effectively II: A replication and elaboration of student learning in classroom teaching.

An important aim of teaching philosophy in Dutch secondary schools is to learn about philosophy (that is, the great philosophers) by doing philosophy. In an earlier study published in PLoS ONE, we focused on the relationship between student learning activities and teacher behavior by analyzing eight lessons. Correspondence analysis revealed that doing philosophy was more effective in some lessons than in others. We replicated this finding in the current study, using 10 new lessons, and elaborated on the relationship between the likely causes for doing philosophy effectively. The data suggest that conducting a dialogue in the form of a philosophical discussion is sufficient for achieving an effective lesson, whereas the teachers' guidance being shared with the students is a necessary but not sufficient condition for achieving an effective lesson.

A stochastic simulation model to study respondent-driven recruitment.

Respondent-driven detection is a chain recruitment method used to sample contact persons of infected persons in order to enhance case finding. It starts with initial individuals, so-called seeds, who are invited for participation. Afterwards, seeds receive a fixed number of coupons to invite individuals with whom they had contact during a specific time period. Recruitees are then asked to do the same, resulting in successive waves of contact persons who are connected in one recruitment tree. However, often the majority of participants fail to invite others, or invitees do not accept an invitation, and recruitment stops after several waves. A mathematical model can help to analyse how various factors influence peer recruitment and to understand under which circumstances sustainable recruitment is possible. We implemented a stochastic simulation model, where parameters were suggested by empirical data from an online survey, to determine the thresholds for obtaining large recruitment trees and the number of waves needed to reach a steady state in the sample composition for individual characteristics. We also examined the relationship between mean and variance of the number of invitations sent out by participants and the probability of obtaining a large recruitment tree. Our main finding is that a situation where participants send out any number of coupons between one and the maximum number is more effective in reaching large recruitment trees, compared to a situation where the majority of participants does not send out any invitations and a smaller group sends out the maximum number of invitations. The presented model is a helpful tool that can assist public health professionals in preparing research and contact tracing using online respondent-driven detection. In particular, it can provide information on the required minimum number of successfully sent invitations to reach large recruitment trees, a certain sample composition or certain number of waves.

Psychometric Properties of the Sexual Event Diary in a Sample of Dutch Women With Female Sexual Interest/Arousal Disorder.

BACKGROUND: Efficacy of on-demand drugs for women with hypoactive sexual desire disorder or female sexual interest/arousal disorder (FSIAD) should be assessed using a validated instrument that assesses the discrete sexual events during which the on-demand drug is taken, because this type of assessment is more proximate to an on-demand drug's efficacy compared to instruments that assess sexual function over longer periods of time. AIM: The aim of this study was to assess the psychometric properties of the Dutch translation of the previously validated 11-item Sexual Event Diary (SED) for measuring sexual satisfaction and sexual functioning during discrete sexual events. METHODS: Psychometric assessment was performed on data of 1,840 SEDs from 139 women with hypoactive sexual desire disorder/FSIAD, collected during a randomized clinical cross-over trial conducted in the Netherlands. OUTCOMES: Item scores of the SED at the event level, and at subject level, summarized item scores during the placebo run-in period (PRI) and active treatment period, and score changes from PRI to active treatment period. RESULTS: Reliability and convergent validity were confirmed. All item scores showed the ability to discriminate between known groups. Larger mean score changes from PRI were observed in groups with known benefit from the medication, as compared to those with no benefit. Guyatt effect sizes ranged from 0.51-1.02, thereby demonstrating ability to detect change. CLINICAL TRANSLATION: The Dutch version of the SED is an excellent instrument for assessing female sexual functioning and sexual satisfaction during discrete sexual events and for assessing these concepts over longer periods of time. CONCLUSIONS: Data were collected in a randomized, well-controlled trial. The large number of data points gave high statistical power, and the results confirmed previous findings. However, care is needed when generalizing the SED's validity to other areas of research, eg, recreational drug use and sexual risky behaviors, since the current validation study has not used such data. Consistent with the US-English version, the Dutch version of the SED is a reliable, valid, and responsive instrument, and suitable for use in evaluating effects of on-demand drugs in women with FSIAD. van Nes Y, Bloemers J, Kessels R, et al. Psychometric Properties of the Sexual Event Diary in a Sample of Dutch Women With Female Sexual Interest/Arousal Disorder. J Sex Med 2018;15:722-731.

The Sexual Event Diary (SED): Development and Validation of a Standardized Questionnaire for Assessing Female Sexual Functioning During Discrete Sexual Events.

BACKGROUND: The efficacy of on-demand drugs for hypoactive sexual desire disorder (HSDD) or female sexual interest/arousal disorder (FSIAD) should be assessed using a validated instrument that assesses the discrete sexual events during which the on-demand drug is taken. AIM: To develop and validate an event log for measuring sexual satisfaction and sexual functioning of discrete sexual events. METHODS: Psychometric assessment was carried out on data of 10,959 Sexual Event Diaries (SEDs) collected during three clinical trials in a total of 421 women with HSDD. Cognitive debriefing interviews were held with 16 women with HSDD. OUTCOMES: Item scores of the SED at the event level and at the subject level, summarized item scores of women during the baseline establishment and active treatment periods, and score changes in women from baseline establishment to active treatment. RESULTS: Several items of the initial 16-item SED items showed weak validity. The 16-item SED was refined to the 11-item SED. The reliability, content, and convergent validity of the 11-item SED were confirmed. For most 11-item SED item scores, the ability to discriminate between known groups was confirmed. Larger mean score changes from the baseline establishment period were found in those with than in those without known benefit from the medication, and Guyatt effect sizes ranged from 0.73 to 1.58, thereby demonstrating the ability to detect change. CLINICAL TRANSLATION: The SED is a good tool for assessing sexual function during a discrete sexual event and for assessing the sexual function of women over longer periods. STRENGTHS AND LIMITATIONS: The validation of the SED was performed on data from nearly 11,000 sexual events, gathered as part of a drug development program for HSDD and FSIAD. This amount of data provides very robust results when related to drug use for HSDD and FSIAD, but caution is advised when generalizing the validity of the SED directly to other areas of research (eg, recreational drug use and sexual risky behaviors), because such data were not used in this validation. CONCLUSIONS: The 11-item SED is a reliable, valid, and responsive instrument and suitable for use in evaluating the effects of on-demand drugs in women with HSDD or FSIAD. van Nes Y, Bloemers J, van der Heijden PGM, et al. The Sexual Event Diary (SED): Development and Validation of a Standardized Questionnaire for Assessing Female Sexual Functioning During Discrete Sexual Events. J Sex Med 2017;14:1438-1450.

The multidimensional randomized response design: Estimating different aspects of the same sensitive behavior.

The conventional randomized response design is unidimensional in the sense that it measures a single dimension of a sensitive attribute, like its prevalence, frequency, magnitude, or duration. This paper introduces a multidimensional design characterized by categorical questions that each measure a different aspect of the same sensitive attribute. The benefits of the multidimensional design are (i) a substantial gain in power and efficiency, and the potential to (ii) evaluate the goodness-of-fit of the model, and (iii) test hypotheses about evasive response biases in case of a misfit. The method is illustrated for a two-dimensional design measuring both the prevalence and the magnitude of social security fraud.

Tracking social contact networks with online respondent-driven detection: who recruits whom?

BACKGROUND: Transmission of respiratory pathogens in a population depends on the contact network patterns of individuals. To accurately understand and explain epidemic behaviour information on contact networks is required, but only limited empirical data is available. Online respondent-driven detection can provide relevant epidemiological data on numbers of contact persons and dynamics of contacts between pairs of individuals. We aimed to analyse contact networks with respect to sociodemographic and geographical characteristics, vaccine-induced immunity and self-reported symptoms. METHODS: In 2014, volunteers from two large participatory surveillance panels in the Netherlands and Belgium were invited for a survey. Participants were asked to record numbers of contacts at different locations and self-reported influenza-like-illness symptoms, and to invite 4 individuals they had met face to face in the preceding 2 weeks. We calculated correlations between linked individuals to investigate mixing patterns. RESULTS: In total 1560 individuals completed the survey who reported in total 30591 contact persons; 488 recruiter-recruit pairs were analysed. Recruitment was assortative by age, education, household size, influenza vaccination status and sentiments, indicating that participants tended to recruit contact persons similar to themselves. We also found assortative recruitment by symptoms, reaffirming our objective of sampling contact persons whom a participant may infect or by whom a participant may get infected in case of an outbreak. Recruitment was random by sex and numbers of contact persons. Relationships between pairs were influenced by the spatial distribution of peer recruitment. CONCLUSIONS: Although complex mechanisms influence online peer recruitment, the observed statistical relationships reflected the observed contact network patterns in the general population relevant for the transmission of respiratory pathogens. This provides useful and innovative input for predictive epidemic models relying on network information.

Doing Philosophy Effectively: Student Learning in Classroom Teaching.

An important aim of teaching philosophy in Dutch secondary schools is to learn about philosophy (i.e., the great philosophers) by doing philosophy. We examined doing philosophy and focused specifically on the relationship between student learning activities and teacher behavior; in doing so, a qualitative cross-case analysis of eight philosophy lessons was performed. The effectiveness of doing philosophy was operationalized into five learning activities comprising rationalizing, analyzing, testing, producing criticism, and reflecting, and scored by means of qualitative graphical time registration. Using CA we find a quantitative one-dimensional scale for the lessons that contrasts lessons that are more and less effective in terms of learning and teaching. A relationship was found between teaching by teachers and doing philosophy by students. In particular we found students to produce a higher level of doing philosophy with teachers who chose to organize a philosophical discussion with shared guidance by the teacher together with the students.

Distance comparisons in virtual reality: effects of path, context, and age.

In this large scale, individual differences study (N = 521), the effects of cardinal axes of an environment and the path taken between locations on distance comparisons were assessed. The main goal was to identify if and to what extent previous findings in simple 2D tasks can be generalized to a more dynamic, three-dimensional virtual reality environment. Moreover, effects of age and gender were assessed. After memorizing the locations of six objects in a circular environment, participants were asked to judge the distance between objects they encountered. Results indicate that categorization (based on the cardinal axes) was present, as distances within one quadrant were judged as being closer together, even when no visual indication of the cardinal axes was given. Moreover, strong effects of the path taken between object locations were found; objects that were near on the path taken were perceived as being closer together than objects that were further apart on this path, regardless of the metric distance between the objects. Males outperformed females in distance comparison, but did not differ in the extent of the categorization and path effects. Age also affected performance; the categorization and path effects were highly similar across the age range tested, but the general ability to estimate distances does show a clear pattern increase during development and decrease with aging.

Correspondence: Some general points regarding Ledberg and Wennberg, BMC Medical Research Methodology 2014 April 27;14:58.

The purpose of this note is to contribute some general points on a recent paper by Ledberg and Wennberg (BMC Med Res Meth 14:58, 2014) which need to be rectified. They advocate the capture-removal estimator. First, we will discuss drawbacks of this estimator in comparison to the Lincoln-Petersen estimator. Second, we show that their evaluation of the Chao estimator is flawed. We conclude that some statements in Ledberg and Wennberg with respect to Chao's estimator and removal estimation need to be taken with great caution.

Enhancing Syndromic Surveillance With Online Respondent-Driven Detection.

OBJECTIVES: We investigated the feasibility of combining an online chain recruitment method (respondent-driven detection) and participatory surveillance panels to collect previously undetected information on infectious diseases via social networks of participants. METHODS: In 2014, volunteers from 2 large panels in the Netherlands were invited to complete a survey focusing on symptoms of upper respiratory tract infections and to invite 4 individuals they had met in the preceding 2 weeks to take part in the study. We compared sociodemographic characteristics among panel participants, individuals who volunteered for our survey, and individuals recruited via respondent-driven detection. RESULTS: Starting from 1015 panel members, the survey spread through all provinces of the Netherlands and all age groups in 83 days. A total of 433 individuals completed the survey via peer recruitment. Participants who reported symptoms were 6.1% (95% confidence interval = 5.4, 6.9) more likely to invite contact persons than were participants who did not report symptoms. Participants with symptoms invited more symptomatic recruits to take part than did participants without symptoms. CONCLUSIONS: Our findings suggest that online respondent-driven detection can enhance identification of symptomatic patients by making use of individuals' local social networks.