Describing diversity of real world data sources in pharmacoepidemiologic studies: The DIVERSE scoping review.

PURPOSE: Real-world evidence (RWE) is increasingly used for medical regulatory decisions, yet concerns persist regarding its reproducibility and hence validity. This study addresses reproducibility challenges associated with diversity across real-world data sources (RWDS) repurposed for secondary use in pharmacoepidemiologic studies. Our aims were to identify, describe and characterize practices, recommendations and tools for collecting and reporting diversity across RWDSs, and explore how leveraging diversity could improve the quality of evidence. METHODS: In a preliminary phase, keywords for a literature search and selection tool were designed using a set of documents considered to be key by the coauthors. Next, a systematic search was conducted up to December 2021. The resulting documents were screened based on titles and abstracts, then based on full texts using the selection tool. Selected documents were reviewed to extract information on topics related to collecting and reporting RWDS diversity. A content analysis of the topics identified explicit and latent themes. RESULTS: Across the 91 selected documents, 12 topics were identified: 9 dimensions used to describe RWDS (organization accessing the data source, data originator, prompt, inclusion of population, content, data dictionary, time span, healthcare system and culture, and data quality), tools to summarize such dimensions, challenges, and opportunities arising from diversity. Thirty-six themes were identified within the dimensions. Opportunities arising from data diversity included multiple imputation and standardization. CONCLUSIONS: The dimensions identified across a large number of publications lay the foundation for formal guidance on reporting diversity of data sources to facilitate interpretation and enhance replicability and validity of RWE.

Best Practices and Recommendations for Research Using Virtual Real-Time Data Collection: Protocol for Virtual Data Collection Studies.

BACKGROUND: The COVID-19 pandemic and the subsequent need for social distancing required the immediate pivoting of research modalities. Research that had previously been conducted in person had to pivot to remote data collection. Researchers had to develop data collection protocols that could be conducted remotely with limited or no evidence to guide the process. Therefore, the use of web-based platforms to conduct real-time research visits surged despite the lack of evidence backing these novel approaches. OBJECTIVE: This paper aims to review the remote or virtual research protocols that have been used in the past 10 years, gather existing best practices, and propose recommendations for continuing to use virtual real-time methods when appropriate. METHODS: Articles (n=22) published from 2013 to June 2023 were reviewed and analyzed to understand how researchers conducted virtual research that implemented real-time protocols. "Real-time" was defined as data collection with a participant through a live medium where a participant and research staff could talk to each other back and forth in the moment. We excluded studies for the following reasons: (1) studies that collected participant or patient measures for the sole purpose of engaging in a clinical encounter; (2) studies that solely conducted qualitative interview data collection; (3) studies that conducted virtual data collection such as surveys or self-report measures that had no interaction with research staff; (4) studies that described research interventions but did not involve the collection of data through a web-based platform; (5) studies that were reviews or not original research; (6) studies that described research protocols and did not include actual data collection; and (7) studies that did not collect data in real time, focused on telehealth or telemedicine, and were exclusively intended for medical and not research purposes. RESULTS: Findings from studies conducted both before and during the COVID-19 pandemic suggest that many types of data can be collected virtually in real time. Results and best practice recommendations from the current protocol review will be used in the design and implementation of a substudy to provide more evidence for virtual real-time data collection over the next year. CONCLUSIONS: Our findings suggest that virtual real-time visits are doable across a range of participant populations and can answer a range of research questions. Recommended best practices for virtual real-time data collection include (1) providing adequate equipment for real-time data collection, (2) creating protocols and materials for research staff to facilitate or guide participants through data collection, (3) piloting data collection, (4) iteratively accepting feedback, and (5) providing instructions in multiple forms. The implementation of these best practices and recommendations for future research are further discussed in the paper. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/53790.

Plan and Operations of the National Health and Nutrition Examination Survey, August 2021-August 2023.

The continuous National Health and Nutrition Examination Survey began data collection in 1999 and proceeded without interruption until operations were suspended in March 2020 in response to the COVID-19 pandemic. Once the Division of Health and Nutrition Examination Surveys was able to determine and resume safe field operations, the next survey cycle was conducted between August 2021 and August 2023. This report describes the survey content, procedures, and methodologies implemented in the August 2021-August 2023 National Health and Nutrition Examination Survey cycle.

Navigating the Complexities of Artificial Intelligence-Enabled Real-World Data Collection for Oncology Pharmacovigilance.

This new editorial discusses the promise and challenges of successful integration of natural language processing methods into electronic health records for timely, robust, and fair oncology pharmacovigilance.

Lifetime residential history collection and processing for environmental data linkages in the ABCD study.

By using geospatial information such as participants' residential history along with external datasets of environmental exposures, ongoing studies can enrich their cohorts to investigate the role of the environment on brain-behavior health outcomes. However, challenges may arise if clear guidance and key quality control steps are not taken at the outset of data collection of residential information. Here, we detail the protocol development aimed at improving the collection of lifetime residential address information from the Adolescent Brain Cognitive Development (ABCD) Study. This protocol generates a workflow for minimizing gaps in residential information, improving data collection processes, and reducing misclassification error in exposure estimates.

A practical guide to cross-cultural and multi-sited data collection in the biological and behavioural sciences.

Researchers in the biological and behavioural sciences are increasingly conducting collaborative, multi-sited projects to address how phenomena vary across ecologies. These types of projects, however, pose additional workflow challenges beyond those typically encountered in single-sited projects. Through specific attention to cross-cultural research projects, we highlight four key aspects of multi-sited projects that must be considered during the design phase to ensure success: (1) project and team management; (2) protocol and instrument development; (3) data management and documentation; and (4) equitable and collaborative practices. Our recommendations are supported by examples from our experiences collaborating on the Evolutionary Demography of Religion project, a mixed-methods project collecting data across five countries in collaboration with research partners in each host country. To existing discourse, we contribute new recommendations around team and project management, introduce practical recommendations for exploring the validity of instruments through qualitative techniques during piloting, highlight the importance of good documentation at all steps of the project, and demonstrate how data management workflows can be strengthened through open science practices. While this project was rooted in cross-cultural human behavioural ecology and evolutionary anthropology, lessons learned from this project are applicable to multi-sited research across the biological and behavioural sciences.

Deriving life-course residential histories in brain bank cohorts: A feasibility study.

INTRODUCTION: The exposome is theorized to interact with biological mechanisms to influence risk for Alzheimer's disease but is not well-integrated into existing Alzheimer's Disease Research Center (ADRC) brain bank data collection. METHODS: We apply public data tracing, an iterative, dual abstraction and validation process rooted in rigorous historic archival methods, to develop life-course residential histories for 1254 ADRC decedents. RESULTS: The median percentage of the life course with an address is 78.1% (IQR 24.9); 56.5% of the sample has an address for at least 75% of their life course. Archivists had 89.7% agreement at the address level. This method matched current residential survey methodology 97.4% on average. DISCUSSION: This novel method demonstrates feasibility, reproducibility, and rigor for historic data collection. To our knowledge, this is the first study to show that public data tracing methods for brain bank decedent residential history development can be used to better integrate the social exposome with biobank specimens. HIGHLIGHTS: Public data tracing compares favorably to survey-based residential history. Public data tracing is feasible and reproducible between archivists. Archivists achieved 89.7% agreement at the address level. This method identifies residences for nearly 80% of life-years, on average. This novel method enables brain banks to add social characterizations.

Getting our ducks in a row: The need for data utility comparisons of healthcare systems data for clinical trials.

BACKGROUND: Better use of healthcare systems data, collected as part of interactions between patients and the healthcare system, could transform planning and conduct of randomised controlled trials. Multiple challenges to widespread use include whether healthcare systems data captures sufficiently well the data traditionally captured on case report forms. "Data Utility Comparison Studies" (DUCkS) assess the utility of healthcare systems data for RCTs by comparison to data collected by the trial. Despite their importance, there are few published UK examples of DUCkS. METHODS-AND-RESULTS: Building from ongoing and selected recent examples of UK-led DUCkS in the literature, we set out experience-based considerations for the conduct of future DUCkS. Developed through informal iterative discussions in many forums, considerations are offered for planning, protocol development, data, analysis and reporting, with comparisons at "patient-level" or "trial-level", depending on the item of interest and trial status. DISCUSSION: DUCkS could be a valuable tool in assessing where healthcare systems data can be used for trials and in which trial teams can play a leading role. There is a pressing need for trials to be more efficient in their delivery and research waste must be reduced. Trials have been making inconsistent use of healthcare systems data, not least because of an absence of evidence of utility. DUCkS can also help to identify challenges in using healthcare systems data, such as linkage (access and timing) and data quality. We encourage trial teams to incorporate and report DUCkS in trials and funders and data providers to support them.

Capture, Coding, and Reporting of Health Care and Medicine Information in Australian Workers' Compensation Systems.

OBJECTIVE: This study aims to characterize the approaches to collecting, coding, and reporting health care and medicines data within Australian workers' compensation schemes. METHODS: We conducted a cross-sectional survey of data and information professionals in major Australian workers' compensation jurisdictions. Questionnaires were developed with input from key informants and a review of existing documentation. RESULTS: Twenty-five participants representing regulators (40%) and insurers (60%) with representation from all Australian jurisdictions were included. Health care and medicines data sources, depth, coding standards, and reporting practices exhibited significant variability across the Australian workers' compensation schemes. CONCLUSIONS: Substantial variability exists in the capture, coding, and reporting of health care and medicine data in Australian workers' compensation jurisdictions. There are opportunities to advance understanding of medicines and health service delivery in these schemes through greater harmonization of data collection, data coding, and reporting.

Lessons Learned From Telephone-Based Data Collection for Health and Demographic Surveillance Systems During the COVID-19 Pandemic in Indonesia.

The Sleman Health and Demographic Surveillance System (HDSS) is a longitudinal survey held routinely since 2014 to collect demographic, social, and health changes in Sleman Regency, Special Region of Yogyakarta, Indonesia. During the COVID-19 pandemic in Indonesia, we needed to adjust our method of conducting data collection from in-person to telephone interviews. We describe the Sleman HDSS data collection strategy used and the opportunities it presented. First, the Sleman HDSS team completed a feasibility study and adjusted the standard operational procedures to conduct telephone interviews. Then, the Sleman HDSS team collected data via a telephone interview in September-October 2020. Ten interviewers were equipped with an e-HDSS data collection application installed on an Android-based tablet to collect data. The sample targeted was 5,064 households. The telephone-based data collection successfully interviewed 1,674 households (33% response rate) in 17 subdistricts. We changed the data collection strategy so that the Sleman HDSS could still be conducted and we could get the latest data from the population. Compared to in-person interviewing, data collection via telephone was sufficiently practical. The telephone interview was a safe and viable data collection method. To increase the response rate, telephone number activation could be checked, ways of building rapport could be improved, and engagement could be improved by using social capital.

Leveraging Ethnic Backgrounds to Improve Collection of Race, Ethnicity, and Language Data.

INTRODUCTION: Healthcare disparities may be exacerbated by upstream incapacity to collect high-quality and accurate race, ethnicity, and language (REaL) data. There are opportunities to remedy these data barriers. We present the Denver Health (DH) REaL initiative, which was implemented in 2021. METHODS: Denver Health is a large safety net health system. After assessing the state of REaL data at DH, we developed a standard script, implemented training, and adapted our electronic health record to collect this information starting with an individual's ethnic background followed by questions on race, ethnicity, and preferred language. We analyzed the data for completeness after REaL implementation. RESULTS: A total of 207,490 patients who had at least one in-person registration encounter before and after the DH REaL implementation were included in our analysis. There was a significant decline in missing values for race (7.9%-0.5%, p < .001) and for ethnicity (7.6%-0.3%, p < .001) after implementation. Completely of language data also improved (3%-1.6%, p < .001). A year after our implementation, we knew over 99% of our cohort's self-identified race and ethnicity. CONCLUSIONS: Our initiative significantly reduced missing data by successfully leveraging ethnic background as the starting point of our REaL data collection.

A comparative study of clinical trial and real-world data in patients with diabetic kidney disease.

A growing body of research is focusing on real-world data (RWD) to supplement or replace randomized controlled trials (RCTs). However, due to the disparities in data generation mechanisms, differences are likely and necessitate scrutiny to validate the merging of these datasets. We compared the characteristics of RCT data from 5734 diabetic kidney disease patients with corresponding RWD from electronic health records (EHRs) of 23,523 patients. Demographics, diagnoses, medications, laboratory measurements, and vital signs were analyzed using visualization, statistical comparison, and cluster analysis. RCT and RWD sets exhibited significant differences in prevalence, longitudinality, completeness, and sampling density. The cluster analysis revealed distinct patient subgroups within both RCT and RWD sets, as well as clusters containing patients from both sets. We stress the importance of validation to verify the feasibility of combining RCT and RWD, for instance, in building an external control arm. Our results highlight general differences between RCT and RWD sets, which should be considered during the planning stages of an RCT-RWD study. If they are, RWD has the potential to enrich RCT data by providing first-hand baseline data, filling in missing data or by subgrouping or matching individuals, which calls for advanced methods to mitigate the differences between datasets.

Online interviews for qualitative health research in Africa: a scoping review.

Online interviews can be powerful tools in global health research. In this article, we review the literature on the use of and challenges associated with online interviews in health research in Africa and make recommendations for future online qualitative studies. The scoping review methodology was used. We searched on Medline and Embase in March 2022 for qualitative articles that used internet-based interviews as a data collection method. Following full-text reviews, we included nine articles. We found that online interviews were typically conducted via Microsoft Teams, Zoom, Skype, WhatsApp, Facebook Messaging and E-mail chats. Online interviews were used in Africa because of the restrictions imposed by the coronavirus disease 2019 pandemic and the need to sample participants across multiple countries or communities. Recruitment for online interviews occurred online, interviews were characterised by inaudible sounds, the inability to use video options and the challenges of including people with low income and education. We recommend that researchers critically evaluate the feasibility of online interviews within a particular African locality before fully implementing this data collection approach. Researchers may also collaborate with community-based organisations to help recruit a more socioeconomically diverse sample because of the potential of excluding participants with limited internet access.

The impact of electronic versus paper-based data capture on data collection logistics and on missing scores in thyroid cancer patients.

PURPOSE: The purpose of this study was to investigate the impact of the type of data capture on the time and help needed for collecting patient-reported outcomes as well as on the proportion of missing scores. METHODS: In a multinational prospective study, thyroid cancer patients from 17 countries completed a validated questionnaire measuring quality of life. Electronic data capture was compared to the paper-based approach using multivariate logistic regression. RESULTS: A total of 437 patients were included, of whom 13% used electronic data capture. The relation between data capture and time needed was modified by the emotional functioning of the patients. Those with clinical impairments in that respect needed more time to complete the questionnaire when they used electronic data capture compared to paper and pencil (ORadj 24.0; p = 0.006). This was not the case when patients had sub-threshold emotional problems (ORadj 1.9; p = 0.48). The odds of having the researcher reading the questions out (instead of the patient doing this themselves) (ORadj 0.1; p = 0.01) and of needing any help (ORadj 0.1; p = 0.01) were lower when electronic data capture was used. The proportion of missing scores was equivalent in both groups (ORadj 0.4, p = 0.42). CONCLUSIONS: The advantages of electronic data capture, such as real-time assessment and fewer data entry errors, may come at the price of more time required for data collection when the patients have mental health problems. As this is not uncommon in thyroid cancer, researchers need to choose the type of data capture wisely for their particular research question.

Comparative performance of national civil registration and vital statistics systems: a global assessment.

Objective: To assess the current state of the world's civil registration and vital statistics systems based on publicly available data and to propose strategic development pathways, including priority interventions, for countries at different levels of civil registration and vital statistics performance. Methods: We applied a performance assessment framework to publicly available data, using a composite indicator highly correlated with civil registration and vital statistics performance which we then adjusted for data incomparability and missing values. Findings: Globally, civil registration and vital statistics systems score on average 0.70 (0-1 scale), with substantial variations across countries and regions. Scores ranged from less than 0.50 in emerging systems to nearly 1.00 in the most developed systems. Approximately one fifth of the world's population live in the 43 countries with low system performance (< 0.477). Irrespective of system development, health sector indicators consistently scored lower than other determinants of civil registration and vital statistics performance. Conclusion: From our assessment, we provide three main recommendations for how the health sector can contribute to improving civil registration and vital statistics systems: (i) enhanced health sector engagement in birth and death notification; (ii) a more systematic approach to training cause of death diagnostics; and (iii) leadership in the implementation of verbal autopsy methods. Four different civil registration and vital statistics improvement pathways for countries at different levels of system development are proposed, that can constitute a blueprint for regional civil registration and vital statistics strengthening activities that countries can adapt and refine to suit their capabilities, resources, and particular challenges.

A qualitative exploration of forensic pathology service staff perceptions of the implementation barriers and facilitators of manual- and electronic injury mortality surveillance system methods in South Africa.

BACKGROUND: Injury mortality surveillance systems are critical to monitor changes in a population's injury outcomes so that relevant injury prevention responses may be adopted. This is particularly the case in South Africa, where the injury burden is nearly twice the global rate. Regular evaluations of surveillance systems are pivotal to strengthening surveillance capacity, performance, and cost effectiveness. The National Injury Mortality Surveillance System (NIMSS) is an injury mortality surveillance system that is currently focused in Mpumalanga and utilises manual and electronic web-based systems for data collection. This study explored Forensic Pathology Service (FPS) staff perceptions of the implementation barriers and facilitators of manual- and electronic injury mortality surveillance system methods. METHODS: A qualitative study was employed using purposive sampling. Forty-seven participants, aged 29 to 59 years comprising 31 males and 16 females were recruited across 21 FPS facilities that serve the province. The formative evaluation occurred over the November 2019 to November 2022 period. Twelve focus group discussions were thematically analysed to determine emerging themes and patterns related to the use of the system using the WHO surveillance system guidelines as a framework. RESULTS: The key themes concerning the barriers and facilitators were located along WHO attributes of simplicity, acceptability, timeliness, flexibility, data quality and stability. Distinctions between the manual and e-surveillance systems were drawn upon across the attributes highlighting their experience with the system, user preference, and its contextual relevance. With Mpumalanga predominantly rural, internet connectivity was a common issue, with most participants consequently showing a preference for the manual system, even though the electronic system's automated internal validation process was of benefit. The data quality however remained similar for both methods. With program stability and flexibility, the manual system proved more beneficial as the dataset was reported to be easily transferrable across computer devices. CONCLUSION: Obtaining FPS perceptions of their experiences with the system methodologies are pertinent for the enhancement of injury surveillance systems so to improve prospective engagements with the systems. This will facilitate timely and accurate injury mortality information which is vital to inform public policy, and injury control and prevention responses.