Bot or Not? Detecting and Managing Participant Deception When Conducting Digital Research Remotely: Case Study of a Randomized Controlled Trial.

BACKGROUND: Evaluating digital interventions using remote methods enables the recruitment of large numbers of participants relatively conveniently and cheaply compared with in-person methods. However, conducting research remotely based on participant self-report with little verification is open to automated "bots" and participant deception. OBJECTIVE: This paper uses a case study of a remotely conducted trial of an alcohol reduction app to highlight and discuss (1) the issues with participant deception affecting remote research trials with financial compensation; and (2) the importance of rigorous data management to detect and address these issues. METHODS: We recruited participants on the internet from July 2020 to March 2022 for a randomized controlled trial (n=5602) evaluating the effectiveness of an alcohol reduction app, Drink Less. Follow-up occurred at 3 time points, with financial compensation offered (up to £36 [US $39.23]). Address authentication and telephone verification were used to detect 2 kinds of deception: "bots," that is, automated responses generated in clusters; and manual participant deception, that is, participants providing false information. RESULTS: Of the 1142 participants who enrolled in the first 2 months of recruitment, 75.6% (n=863) of them were identified as bots during data screening. As a result, a CAPTCHA (Completely Automated Public Turing Test to Tell Computers and Humans Apart) was added, and after this, no more bots were identified. Manual participant deception occurred throughout the study. Of the 5956 participants (excluding bots) who enrolled in the study, 298 (5%) were identified as false participants. The extent of this decreased from 110 in November 2020, to a negligible level by February 2022 including a number of months with 0. The decline occurred after we added further screening questions such as attention checks, removed the prominence of financial compensation from social media advertising, and added an additional requirement to provide a mobile phone number for identity verification. CONCLUSIONS: Data management protocols are necessary to detect automated bots and manual participant deception in remotely conducted trials. Bots and manual deception can be minimized by adding a CAPTCHA, attention checks, a requirement to provide a phone number for identity verification, and not prominently advertising financial compensation on social media. TRIAL REGISTRATION: ISRCTN Number ISRCTN64052601; https://doi.org/10.1186/ISRCTN64052601.

The Effect of the First UK COVID-19 Lockdown on Users of the Drink Less App: Interrupted Time Series Analysis of Sociodemographic Characteristics, Engagement, and Alcohol Reduction.

BACKGROUND: The first UK COVID-19 lockdown had a polarizing impact on drinking behavior and may have impacted engagement with digital interventions to reduce alcohol consumption. OBJECTIVE: We examined the effect of lockdown on engagement, alcohol reduction, and the sociodemographic characteristics of users of the popular and widely available alcohol reduction app Drink Less. METHODS: This was a natural experiment. The study period spanned 468 days between March 24, 2019, and July 3, 2020, with the introduction of UK lockdown measures beginning on March 24, 2020. Users were 18 years or older, based in the United Kingdom, and interested in drinking less. Interrupted time series analyses using generalized additive mixed models (GAMMs) were conducted for each outcome variable (ie, sociodemographic characteristics, app downloads and engagement levels, alcohol consumption, and extent of alcohol reduction) for existing (downloaded the app prelockdown) and new (downloaded the app during the lockdown) users of the app. RESULTS: Among existing users of the Drink Less app, there were increases in the time spent on the app per day (B=0.01, P=.01), mean units of alcohol recorded per day (B>0.00 P=.02), and mean heavy drinking (>6 units) days (B>0.00, P=.02) during the lockdown. Previous declines in new app downloads plateaued during the lockdown (incidence rate ratio [IRR]=1.00, P=.18). Among new app users, there was an increase in the proportion of female users (B>0.00, P=.04) and those at risk of alcohol dependence (B>0.00, P=.01) and a decrease in the proportion of nonmanual workers (B>-0.00, P=.04). Among new app users, there were step increases in the mean number of alcohol units per day (B=20.12, P=.03), heavy-drinking days (B=1.38, P=.01), and the number of days the app was used (B=2.05, P=.02), alongside a step decrease in the percentage of available screens viewed (B=-0.03, P=.04), indicating users were using less of the intervention components within the app. CONCLUSIONS: Following the first UK lockdown, there was evidence of increases in engagement and alcohol consumption among new and existing users of the Drink Less app.

Impact of personality status on the outcomes and cost of cognitive-behavioural therapy for health anxiety.

BACKGROUND: Health anxiety, hypochondriasis and personality disturbance commonly coexist. The impact of personality status was assessed in a secondary analysis of a randomised controlled trial (RCT). AIMS: To test the impact of personality status using ICD-11 criteria on the clinical and cost outcomes of treatment with cognitive-behavioural therapy for health anxiety (CBT-HA) and standard care over 2 years. METHOD: Personality dysfunction was assessed at baseline in 444 patients before randomisation and independent assessment of costs and outcomes made on four occasions over 2 years. RESULTS: In total, 381 patients (86%) had some personality dysfunction with 184 (41%) satisfying the ICD criteria for personality disorder. Those with no personality dysfunction showed no treatment differences (P = 0.90) and worse social function with CBT-HA compared with standard care (P<0.03) whereas all other personality groups showed greater improvement with CBT-HA maintained over 2 years (P<0.001). Less benefit was shown in those with more severe personality disorder (P<0.05). Costs were less with CBT-HA except for non-significant greater differences in those with moderate or severe personality disorder. CONCLUSIONS: The results contradict the hypothesis that personality disorder impairs response to CBT in health anxiety in both the short and medium term.

Methodological Insights on Recruitment and Retention From a Remote Randomized Controlled Trial Examining the Effectiveness of an Alcohol Reduction App: Descriptive Analysis Study.

BACKGROUND: Randomized controlled trials (RCTs) with no in-person contact (ie, remote) between researchers and participants offer savings in terms of cost and time but present unique challenges. OBJECTIVE: The goal of this study is to examine the differences between different forms of remote recruitment (eg, National Health Service [NHS] website, social media, and radio advertising) in the proportion of participants recruited, demographic diversity, follow-up rates, and cost. We also examine the cost per participant of sequential methods of follow-up (emails, phone calls, postal surveys, and postcards). Finally, our experience with broader issues around study advertising and participant deception is discussed. METHODS: We conducted a descriptive analysis of 5602 increasing-and-higher-risk drinkers (Alcohol Use Disorders Identification Test score ≥8), taking part in a 2-arm, parallel group, remote RCT with a 1:1 allocation, comparing the intervention (Drink Less app) with usual digital care (NHS alcohol advice web page). Participants were recruited between July 2020 and March 2022 and compensated with gift vouchers of up to £36 (a currency exchange rate of £1=US $1.26988 is applicable) for completing follow-up surveys, with 4 stages of follow-up: email reminders, phone calls, postal survey, and postcard. RESULTS: The three main recruitment methods were advertisements on (1) social media (2483/5602, 44.32%), (2) the NHS website (1961/5602, 35.01%), and (3) radio and newspapers (745/5602, 13.3%), with the remaining methods of recruitment accounting 7.37% (413/5602) of the sample. The overall recruitment cost per participant varied from £0 to £11.01. Costs were greater when recruiting participants who were men (£0-£28.85), from an ethnic minority group (£0-£303.81), and more disadvantaged (£0-£49.12). Targeted approaches were useful for recruiting more men but less useful in achieving diversity in ethnicity and socioeconomic status. Follow-up at 6 months was 79.58% (4458/5602). Of those who responded, 92.4% (4119/4458) responded by email. Each additional stage of follow-up resulted in an additional 2-3 percentage points of the overall sample being followed up, although phone calls, postal surveys, and postcards were more resource intensive than email reminders. CONCLUSIONS: For remote RCTs, researchers could benefit from using a range of recruitment methods and cost-targeted approaches to achieve demographic diversity. Automated emails with substantial financial incentives for prompt completion can achieve good follow-up rates, and sequential, offline follow-up options, such as phone calls and postal surveys, can further increase follow-up rates but are comparatively expensive. We also make broader recommendations focused on striking the right balance when designing remote RCTs. Careful planning, ongoing maintenance, and dynamic decision-making are required throughout a trial to balance the competing demands of participation among those eligible, deceptive participation among those who are not eligible, and ensuring no postrandomization bias is introduced by data-checking protocols.

Effectiveness of a smartphone app (Drink Less) versus usual digital care for reducing alcohol consumption among increasing-and-higher-risk adult drinkers in the UK: a two-arm, parallel-group, double-blind, randomised controlled trial.

Background: Digital interventions, including apps and websites, can be effective for reducing alcohol consumption. However, many are not evidence- or theory-informed and have not been evaluated. We tested the effectiveness of the Drink Less app for reducing alcohol consumption compared with usual digital care in the UK. Methods: In this two-arm, parallel group, double-blind, randomised controlled trial, we enrolled increasing-and-higher-risk drinkers (AUDIT ≥ 8) in the UK, who were motivated to reduce their alcohol consumption and willing to use a digital intervention to do so, via online methods. Participants were randomly assigned (1:1), using an online algorithm, to receive a web link to download the Drink Less app (intervention) or to the NHS alcohol advice webpage (usual digital care). Researchers were masked to group allocation. Participants were followed up at one, three and six months. The primary outcome was self-reported weekly alcohol consumption at six months, adjusting for baseline consumption. The full analytic sample was used in most analyses, though missing data was treated in different ways. The primary, pre-registered intention-to-treat analysis assumed baseline-carried-forwards. Secondary pre-registered analyses also focused on the full analytic sample and used alternatives including multiple imputation and last observation carried forwards. This trial is registered with the ISRCTN registry, ISRCTN64052601. Findings: Between 07/13/2020 and 03/29/2022, 5602 people were randomly assigned to the Drink Less app (n = 2788) or comparator (n = 2814) groups. Six-month follow-up rates were 79% and 80%, respectively. The primary pre-registered conservative intention-to-treat approach assuming non-responders were drinking at baseline levels of consumption, found a non-significant greater reduction of 0.98 units in weekly alcohol consumption in the intervention group at 6-month follow-up (95% CI -2.67 to 0.70). The data were insensitive to detect the hypothesised effect (Bayes factor = 1.17). Data were not missing completely at random, with 6-month follow-up rates differing in terms of education, occupation, and income. We therefore conducted the pre-registered sensitivity analysis using multiple imputation, showing that the Drink Less app resulted in a 2.00-unit greater weekly reduction at 6-month follow-up compared with the NHS alcohol advice webpage (95% CI -3.76 to -0.24). Fewer than 0.1% of participants in both arms who responded to one, three or six-month follow-up reported adverse events linked to participation in the trial. Interpretation: The Drink Less app may be effective in reducing the alcohol consumption in increasing-and-higher-risk drinkers motivated to reduce their consumption. Funding: NIHR Public Health Research Programme.

Do engagement and behavioural mechanisms underpin the effectiveness of the Drink Less app?

This is a process evaluation of a large UK-based randomised controlled trial (RCT) (n = 5602) evaluating the effectiveness of recommending an alcohol reduction app, Drink Less, compared with usual digital care in reducing alcohol consumption in increasing and higher risk drinkers. The aim was to understand whether participants' engagement ('self-reported adherence') and behavioural characteristics were mechanisms of action underpinning the effectiveness of Drink Less. Self-reported adherence with both digital tools was over 70% (Drink Less: 78.0%, 95% CI = 77.6-78.4; usual digital care: 71.5%, 95% CI = 71.0-71.9). Self-reported adherence to the intervention (average causal mediation effect [ACME] = -0.250, 95% CI = -0.42, -0.11) and self-monitoring behaviour (ACME = -0.235, 95% CI = -0.44, -0.03) both partially mediated the effect of the intervention (versus comparator) on alcohol reduction. Following the recommendation (self-reported adherence) and the tracking (self-monitoring behaviour) feature of the Drink Less app appear to be important mechanisms of action for alcohol reduction among increasing and higher risk drinkers.

Influence of apparently negative personality characteristics on the long-term outcome of health anxiety: Secondary analysis of a randomized controlled trial.

BACKGROUND: It is known that personality has an influence on the outcome of mental state disorders, but detailed studies on its long-term impact are few. We examined the influence of personality status on the 8-year outcome of health anxiety and its relationship to the effects of cognitive behaviour therapy in a randomized controlled trial. AIMS: This study aims to examine both the usefulness of the diagnosis of personality disorder and an additional measure of pathological dependence, in predicting the outcome of medical patients with health anxiety treated with cognitive behaviour therapy. Because the influence of personality is often shown in the long term, these assessments covered the period of 8 years after randomization. An additional aim is to examine the costs of different levels of personality dysfunction in each treatment group. METHOD: Personality dysfunction, using both ICD-10 and ICD-11 classifications of severity, was assessed at baseline by interview in a randomized controlled trial. Patients were also assessed for pathological dependence using the Dependent Personality Questionnaire, also scored along a severity dimension. Four hundred forty-four patients from medical clinics with pathological health anxiety were treated with a modified form of cognitive behaviour therapy for health anxiety (CBT-HA) or standard care. Total costs over follow-up were calculated from hospital data and compared by personality group. RESULTS: At baseline, 381 (86%) had some personality dysfunction, mainly at the lower level of personality difficulty (not formally a disorder). One hundred eighty four (41%) had a personality disorder. A similar proportion was found with regard to dependent personality. Using the ICD-10 classification, 153 patients (34.6%) had a personality disorder, with 83 (54.2%) having anxious or dependent personality disorder, 20 (13.1%) having an anankastic disorder, but also with 66 (43.1%) having mixed disorder. During initial treatment, those with personality disorder adhered more closely to CBT-HA, and after 8 years, they had a significantly better outcome than those with personality difficulty and no personality disorder (p < 0.002). Similar results were found in those scoring high on the Dependent Personality Questionnaire. All these differences increased over the follow-up period. Costs were similar in all groups but were somewhat higher in the CBT-HA one; this finding is hypothesised to be due to fuller hospital treatment once health anxiety is discounted. CONCLUSION: Personality disorder in people with health anxiety, particularly in those who have anxious and dependent traits, reinforces the benefits of cognitive behaviour therapy, particularly in the longer term. © 2020 The Authors Personality and Mental Health Published by John Wiley & Sons Ltd.

Evaluating the effectiveness of the smartphone app, Drink Less, compared with the NHS alcohol advice webpage, for the reduction of alcohol consumption among hazardous and harmful adult drinkers in the UK at 6-month follow-up: protocol for a randomised controlled trial.

BACKGROUND AND AIMS: Digital interventions are effective for reducing alcohol consumption but evidence is limited regarding smartphone apps. Drink Less is a theory- and evidence-informed app to help people reduce their alcohol consumption that has been refined in terms of its content and design for usability across the sociodemographic spectrum. We aim to evaluate the effectiveness and cost-effectiveness of recommending Drink Less at reducing alcohol consumption compared with usual digital care. DESIGN: Two-arm individually randomised controlled trial. SETTING: Online trial in the United Kingdom (UK). PARTICIPANTS: Hazardous or harmful drinkers (Alcohol Use Disorders Identification Test [AUDIT] score ≥8) aged 18+ who want to drink less alcohol (n = 5562). Participants will be recruited from July 2020 to May 2022 using multiple strategies with a focus on remote digital methods. INTERVENTION AND COMPARATOR: Participants will be randomised to receive either an email recommending that they use Drink Less (intervention) or view the National Health Service (NHS) webpage on alcohol advice (comparator). MEASUREMENTS: The primary outcome is change in self-reported weekly alcohol consumption, assessed using the extended AUDIT-Consumption, between baseline and 6-month follow-up. Secondary outcomes include change in self-reported weekly alcohol consumption assessed at 1- and 3-month follow-ups, and the proportion of hazardous drinkers; alcohol-related problems and injury; health-related quality of life; and use of health services assessed at 6-month follow-up. Effectiveness will be examined with adjusted regression models, adjusting for baseline alcohol consumption and using an intention-to-treat approach. A mixed-methods process evaluation will assess engagement, acceptability and mechanism of action. Economic evaluations will be conducted using both a short- and longer-term time horizon. COMMENTS: This study will establish the effectiveness and cost-effectiveness of the Drink Less app at reducing alcohol consumption among hazardous and harmful adult drinkers and will be the first randomised controlled trial of an alcohol reduction app for the general population in the United Kingdom. This study will inform the decision on whether it is worth investing resources in large-scale implementation.

Cognitive-behaviour therapy for health anxiety in medical patients (CHAMP): a randomised controlled trial with outcomes to 5 years.

BACKGROUND: Health anxiety is an under-recognised but frequent cause of distress that is potentially treatable, but there are few studies in secondary care. OBJECTIVE: To determine the clinical effectiveness and cost-effectiveness of a modified form of cognitive-behaviour therapy (CBT) for health anxiety (CBT-HA) compared with standard care in medical outpatients. DESIGN: Randomised controlled trial. SETTING: Five general hospitals in London, Middlesex and Nottinghamshire. PARTICIPANTS: A total of 444 patients aged 16-75 years seen in cardiology, endocrinology, gastroenterology, neurology and respiratory medicine clinics who scored ≥ 20 points on the Health Anxiety Inventory (HAI) and satisfied diagnostic requirements for hypochondriasis. Those with current psychiatric disorders were excluded, but those with concurrent medical illnesses were not. INTERVENTIONS: Cognitive-behaviour therapy for health anxiety - between 4 and 10 1-hour sessions of CBT-HA from a health professional or psychologist trained in the treatment. Standard care was normal practice in primary and secondary care. MAIN OUTCOME MEASURES: Primary - researchers masked to allocation assessed patients at baseline, 3, 6, 12, 24 months and 5 years. The primary outcome was change in the HAI score between baseline and 12 months. Main secondary outcomes - costs of care in the two groups after 24 and 60 months, change in health anxiety (HAI), generalised anxiety and depression [Hospital Anxiety and Depression Scale (HADS)] scores, social functioning using the Social Functioning Questionnaire and quality of life using the EuroQol-5 Dimensions (EQ-5D), at 6, 12, 24 and 60 months, and deaths over 5 years. RESULTS: Of the 28,991 patients screened over 21 months, 5769 had HAI scores of ≥ 20 points. Improvement in HAI scores at 3 months was significantly greater in the CBT-HA group (mean number of sessions = 6) than in the standard care, and this was maintained over the 5-year period (overall p < 0.0001), with no loss of efficacy between 2 and 5 years. Differences in the generalised anxiety (p = 0.0018) and depression scores (p = 0.0065) on the HADS were similar in both groups over the 5-year period. Gastroenterology and cardiology patients showed the greatest CBT gains. The outcomes for nurses were superior to those of other therapists. Deaths (n = 24) were similar in both groups; those in standard care died earlier than those in CBT-HA. Patients with mild personality disturbance and higher dependence levels had the best outcome with CBT-HA. Total costs were similar in both groups over the 5-year period (£12,590.58 for CBT-HA; £13,334.94 for standard care). CBT-HA was not cost-effective in terms of quality-adjusted life-years, as measured using the EQ-5D, but was cost-effective in terms of HAI outcomes, and offset the cost of treatment. LIMITATIONS: Many eligible patients were not randomised and the population treated may not be representative. CONCLUSIONS: CBT-HA is a highly effective treatment for pathological health anxiety with lasting benefit over 5 years. It also improves generalised anxiety and depressive symptoms more than standard care. The presence of personality abnormality is not a bar to successful outcome. CBT-HA may also be cost-effective, but the high costs of concurrent medical illnesses obscure potential savings. This treatment deserves further research in medical settings. TRIAL REGISTRATION: Current Controlled Trials ISRCTN14565822. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 21, No. 50. See the NIHR Journals Library website for further project information.

Preliminary studies of the ICD-11 classification of personality disorder in practice.

OBJECTIVE: This study aims to compare ICD-10 and putative ICD-11 classifications of personality disorder in different clinical populations. DESIGN: Prospective recording of ICD-10 and ICD-11 personality disorder classifications was carried out in (1) an anxious medical population, (2) an acute psychiatric in-patient population and (3) a retrospective recording of a mixed anxiety depression cohort in which all baseline data were scored from baseline information using the ICD-11 classification and compared with the original ICD-10 assessments. METHOD: Comparison of ICD-10 and ICD-11 prevalence of personality disorder in each population was carried out. RESULTS: Data from 722 patients were recorded. Using the ICD-10 criteria, the prevalence of generic personality disorder was 33.8% compared with 40.4% using the ICD-11 ones (χ2 = 6.7; P < 0.01), with 103 (14.3%) discordant assessments. Using the severity definitions in ICD-11, 34.3% of patients had personality difficulty. Severity level varied greatly by population; severe personality disorder was five times more common in the inpatient group. The four domain traits originally denoted as qualifying severity in ICD-11, negative affective, dissocial, anankastic and detached, were linked to anxious, borderline, dissocial, anankastic and schizoid personality disorders in ICD-10. Many patients had pathology in two or more domains. CONCLUSIONS: The ICD-11 classification of personality disorder yields somewhat higher levels of personality dysfunction than ICD-10, possibly because the age range for the onset of diagnosis is now flexible. The range of severity levels make the classification more useful than ICD-10 in clinical practice as it identifies the greater pathology necessary for intervention.

Prevalence of health anxiety problems in medical clinics.

OBJECTIVES: To determine the prevalence of significant health anxiety (hypochondriasis) in patients aged 16-75 in cardiology, respiratory medicine, neurological, endocrine and gastrointestinal clinics in general hospitals in London, Middlesex and North Nottinghamshire. METHOD: The Health Anxiety Inventory (HAI) (short form) was administered to patients attending the five clinics over a 21 month period and all those who scored 20 or more invited to take part in a further assessment for a randomised controlled trial. RESULTS: Of 43,205 patients attending the clinics 28,991 (67.1%) were assessed and of these, after exclusion of ineligible patients 5747 (19.8%) had significant health anxiety. 444 subsequently agreed to take part in a randomised controlled trial of treatment. The prevalence levels varied by clinic with neurology (24.7%) having the highest prevalence followed by respiratory medicine (20.9%), gastroenterology (19.5%), cardiology (19.1%), and endocrinology (17.5%). CONCLUSION: Abnormal health anxiety is common and a significant problem in those attending medical clinics and deserves greater awareness.