Best Practices and Recommendations for Research Using Virtual Real-Time Data Collection: Protocol for Virtual Data Collection Studies.

BACKGROUND: The COVID-19 pandemic and the subsequent need for social distancing required the immediate pivoting of research modalities. Research that had previously been conducted in person had to pivot to remote data collection. Researchers had to develop data collection protocols that could be conducted remotely with limited or no evidence to guide the process. Therefore, the use of web-based platforms to conduct real-time research visits surged despite the lack of evidence backing these novel approaches. OBJECTIVE: This paper aims to review the remote or virtual research protocols that have been used in the past 10 years, gather existing best practices, and propose recommendations for continuing to use virtual real-time methods when appropriate. METHODS: Articles (n=22) published from 2013 to June 2023 were reviewed and analyzed to understand how researchers conducted virtual research that implemented real-time protocols. "Real-time" was defined as data collection with a participant through a live medium where a participant and research staff could talk to each other back and forth in the moment. We excluded studies for the following reasons: (1) studies that collected participant or patient measures for the sole purpose of engaging in a clinical encounter; (2) studies that solely conducted qualitative interview data collection; (3) studies that conducted virtual data collection such as surveys or self-report measures that had no interaction with research staff; (4) studies that described research interventions but did not involve the collection of data through a web-based platform; (5) studies that were reviews or not original research; (6) studies that described research protocols and did not include actual data collection; and (7) studies that did not collect data in real time, focused on telehealth or telemedicine, and were exclusively intended for medical and not research purposes. RESULTS: Findings from studies conducted both before and during the COVID-19 pandemic suggest that many types of data can be collected virtually in real time. Results and best practice recommendations from the current protocol review will be used in the design and implementation of a substudy to provide more evidence for virtual real-time data collection over the next year. CONCLUSIONS: Our findings suggest that virtual real-time visits are doable across a range of participant populations and can answer a range of research questions. Recommended best practices for virtual real-time data collection include (1) providing adequate equipment for real-time data collection, (2) creating protocols and materials for research staff to facilitate or guide participants through data collection, (3) piloting data collection, (4) iteratively accepting feedback, and (5) providing instructions in multiple forms. The implementation of these best practices and recommendations for future research are further discussed in the paper. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/53790.

Plan and Operations of the National Health and Nutrition Examination Survey, August 2021-August 2023.

The continuous National Health and Nutrition Examination Survey began data collection in 1999 and proceeded without interruption until operations were suspended in March 2020 in response to the COVID-19 pandemic. Once the Division of Health and Nutrition Examination Surveys was able to determine and resume safe field operations, the next survey cycle was conducted between August 2021 and August 2023. This report describes the survey content, procedures, and methodologies implemented in the August 2021-August 2023 National Health and Nutrition Examination Survey cycle.

Navigating the Complexities of Artificial Intelligence-Enabled Real-World Data Collection for Oncology Pharmacovigilance.

This new editorial discusses the promise and challenges of successful integration of natural language processing methods into electronic health records for timely, robust, and fair oncology pharmacovigilance.

Describing diversity of real world data sources in pharmacoepidemiologic studies: The DIVERSE scoping review.

PURPOSE: Real-world evidence (RWE) is increasingly used for medical regulatory decisions, yet concerns persist regarding its reproducibility and hence validity. This study addresses reproducibility challenges associated with diversity across real-world data sources (RWDS) repurposed for secondary use in pharmacoepidemiologic studies. Our aims were to identify, describe and characterize practices, recommendations and tools for collecting and reporting diversity across RWDSs, and explore how leveraging diversity could improve the quality of evidence. METHODS: In a preliminary phase, keywords for a literature search and selection tool were designed using a set of documents considered to be key by the coauthors. Next, a systematic search was conducted up to December 2021. The resulting documents were screened based on titles and abstracts, then based on full texts using the selection tool. Selected documents were reviewed to extract information on topics related to collecting and reporting RWDS diversity. A content analysis of the topics identified explicit and latent themes. RESULTS: Across the 91 selected documents, 12 topics were identified: 9 dimensions used to describe RWDS (organization accessing the data source, data originator, prompt, inclusion of population, content, data dictionary, time span, healthcare system and culture, and data quality), tools to summarize such dimensions, challenges, and opportunities arising from diversity. Thirty-six themes were identified within the dimensions. Opportunities arising from data diversity included multiple imputation and standardization. CONCLUSIONS: The dimensions identified across a large number of publications lay the foundation for formal guidance on reporting diversity of data sources to facilitate interpretation and enhance replicability and validity of RWE.

Pilot Study for a Model for the Collection of Waiting Lists Data.

The analysis of data on waiting lists in Italy is regulated by the PNGLA (National Plan for the Governance of Waiting Lists). However, the Plan does not specify the characteristics of the data to be returned by the Regions for the purposes of monitoring, with the result that it is frequently either in aggregate form, unreadable, or incomplete, and therefore cannot be analysed in any meaningful way. Fondazione the Bridge and AGENAS, with the University of Genoa and the University of Pavia, conducted a pilot study on a methodological model for the collection of waiting lists data. The model proved to be effective and replicable, also providing a more valuable opportunity to analyse waiting lists data.

EMinsight: a tool to capture cryoEM microscope configuration and experimental outcomes for analysis and deposition.

The widespread adoption of cryoEM technologies for structural biology has pushed the discipline to new frontiers. A significant worldwide effort has refined the single-particle analysis (SPA) workflow into a reasonably standardized procedure. Significant investments of development time have been made, particularly in sample preparation, microscope data-collection efficiency, pipeline analyses and data archiving. The widespread adoption of specific commercial microscopes, software for controlling them and best practices developed at facilities worldwide has also begun to establish a degree of standardization to data structures coming from the SPA workflow. There is opportunity to capitalize on this moment in the maturation of the field, to capture metadata from SPA experiments and correlate the metadata with experimental outcomes, which is presented here in a set of programs called EMinsight. This tool aims to prototype the framework and types of analyses that could lead to new insights into optimal microscope configurations as well as to define methods for metadata capture to assist with the archiving of cryoEM SPA data. It is also envisaged that this tool will be useful to microscope operators and facilities looking to rapidly generate reports on SPA data-collection and screening sessions.

Conducting a Systematic Review of the Literature.

This overview of the systematic review provides guidance regarding how and when to use this approach to a research question. High quality systematic reviews are essential to assist health care practitioners keep current with the large and rapidly growing body of scientific evidence. The systematic review is a transparent and reproducible synthesis of all the available evidence on a clearly defined research question or topic. Key stages in conducting a systematic review include clarification of aims and methods in a protocol, finding all of the relevant research, data collection, quality assessments, synthesizing evidence, and interpreting the findings. This short report provides examples for the various stages and steps of the systematic review research approach.

Developing Sampling Weights for Statistical Analysis of Parent-Child Pair Data From the National Health Interview Survey.

The National Health Interview Survey (NHIS), conducted by the National Center for Health Statistics since 1957, is the principal source of information on the health of the U.S. civilian noninstitutionalized population. NHIS selects one adult (Sample Adult) and, when applicable, one child (Sample Child) randomly within a family (through 2018) or a household (2019 and forward). Sampling weights for the separate analysis of data from Sample Adults and Sample Children are provided annually by the National Center for Health Statistics. A growing interest in analysis of parent-child pair data using NHIS has been observed, which necessitated the development of appropriate analytic weights. Objective This report explains how dyad weights were created such that data users can analyze NHIS data from both Sample Children and their mothers or fathers, respectively. Methods Using data from the 2019 NHIS, adult-child pair-level sampling weights were developed by combining each pair's conditional selection probability with their household-level sampling weight. The calculated pair weights were then adjusted for pair-level nonresponse, and large sampling weights were trimmed at the 99th percentile of the derived sampling weights. Examples of analyzing parent-child pair data by means of domain estimation methods (that is, statistical analysis for subpopulations or subgroups) are included in this report. Conclusions The National Center for Health Statistics has created dyad or pair weights that can be used for studies using parent-child pairs in NHIS. This method could potentially be adapted to other surveys with similar sampling design and statistical needs.

From physiological complexity to data interactions-A case study of recordings from exercise monitoring.

The popularity of nonlinear analysis has been growing simultaneously with the technology of effort monitoring. Therefore, considering the simple methods of physiological data collection and the approaches from the information domain, we proposed integrating univariate and bivariate analysis for the rest and effort comparison. Two sessions separated by an intensive training program were studied. Nine subjects participated in the first session (S1) and seven in the second session (S2). The protocol included baseline (BAS), exercise, and recovery phase. During all phases, electrocardiogram (ECG) was recorded. For the analysis, we selected corresponding data lengths of BAS and exercise usually lasting less than 5 min. We found the utility of the differences between original data and their surrogates for sample entropy Sdiff and Kullback-Leibler divergence KLDdiff. Sdiff of heart rate variability was negative in BAS and exercise but its sensitivity for phases discrimination was not satisfactory. We studied the bivariate analysis of RR intervals and corresponding QT peaks by Interlayer Mutual Information (IMI) and average edge overlap (AVO) markers. While the IMI parameter decreases in exercise conditions, AVO increased in effort compared to BAS. These findings conclude that researchers should consider a bivariate analysis of extracted RR intervals and corresponding QT datasets, when only ECG is recorded during tests.

Remote Monitoring and Data Collection for Decentralized Clinical Trials.

Importance: Less than 5% of patients with cancer enroll in a clinical trial, partly due to financial and logistic burdens, especially among underserved populations. The COVID-19 pandemic marked a substantial shift in the adoption of decentralized trial operations by pharmaceutical companies. Objective: To assess the current global state of adoption of decentralized trial technologies, understand factors that may be driving or preventing adoption, and highlight aspirations and direction for industry to enable more patient-centric trials. Design, Setting, and Participants: The Bloomberg New Economy International Cancer Coalition, composed of patient advocacy, industry, government regulator, and academic medical center representatives, developed a survey directed to global biopharmaceutical companies of the coalition from October 1 through December 31, 2022, with a focus on registrational clinical trials. The data for this survey study were analyzed between January 1 and 31, 2023. Exposure: Adoption of decentralized clinical trial technologies. Main Outcomes and Measures: The survey measured (1) outcomes of different remote monitoring and data collection technologies on patient centricity, (2) adoption of these technologies in oncology and all therapeutic areas, and (3) barriers and facilitators to adoption using descriptive statistics. Results: All 8 invited coalition companies completed the survey, representing 33% of the oncology market by revenues in 2021. Across nearly all technologies, adoption in oncology trials lags that of all trials. In the current state, electronic diaries and electronic clinical outcome assessments are the most used technology, with a mean (SD) of 56% (19%) and 51% (29%) adoption for all trials and oncology trials, respectively, whereas visits within local physician networks is the least adopted at a mean (SD) of 12% (18%) and 7% (9%), respectively. Looking forward, the difference between the current and aspired adoption rate in 5 years for oncology is large, with respondents expecting a 40% or greater absolute adoption increase in 8 of the 11 technologies surveyed. Furthermore, digitally enabled recruitment, local imaging capabilities, and local physician networks were identified as technologies that could be most effective for improving patient centricity in the long term. Conclusions and Relevance: These findings may help to galvanize momentum toward greater adoption of enabling technologies to support a new paradigm of trials that are more accessible, less burdensome, and more inclusive.

Validity and reliability of the Foot Care Scale for Older Diabetics in Turkish society.

BACKGROUND: Older patients with diabetes have behaviors and practices in foot care that differ from those of younger patients. There is a need for a tool to assess the foot care of patients with diabetes who are over the age of 65 years. PURPOSE: This study aimed to assess the validity and reliability of the long and short versions of the Foot Care Scale for Older Diabetics (FCS-OD) in Turkish society. METHODS: A total of 172 older patients with diabetes participated in the study. Language validity, content validity, construct validity, Cronbach's alpha coefficient value, item-scale correlations, parallel form reliability, and test-retest correlations were used to assess the Turkish version of FCS-OD. A patient identification form, FCS-OD, Diabetic Foot Care Self-Efficacy Scale, and Foot Self-Care Behavior Scale were used for data collection. RESULTS: Turkish versions of the FCS-OD, both short and long versions, were at acceptable levels regarding their content (content validity index: 0.964, 0.975) and construct (factor loadings > 0.4). Both versions of the scale showed a high level of internal consistency as demonstrated by Cronbach's alpha values (short version, 0.802; long version, 0.905) and item-scale correlations (> 0.3). Both versions of the scale were stable over time and compared to the parallel forms. CONCLUSIONS: Both versions of the scale were found to be valid and reliable for Turkish society with regard to psychometric properties.

The National Healthcare Safety Network's digital quality measures: CDC's automated measures for surveillance of patient safety.

OBJECTIVE: This article presents the National Healthcare Safety Network (NHSN)'s approach to automation for public health surveillance using digital quality measures (dQMs) via an open-source tool (NHSNLink) and piloting of this approach using real-world data in a newly established collaborative program (NHSNCoLab). The approach leverages Health Level Seven Fast Healthcare Interoperability Resources (FHIR) application programming interfaces to improve data collection and reporting for public health and patient safety beginning with common, clinically significant, and preventable patient harms, such as medication-related hypoglycemia, healthcare facility-onset Clostridioides difficile infection, and healthcare-associated venous thromboembolism. CONCLUSIONS: The NHSN's FHIR dQMs hold the promise of minimizing the burden of reporting, improving accuracy, quality, and validity of data collected by NHSN, and increasing speed and efficiency of public health surveillance.

[Registration of national health registry-based research]. / Registrering av forskning baserad på nationella hälsoregister.

In medical research as a whole, frequent inaccurate or biased findings are of international concern. One measure against reporting biases is study registration before the start of data collection (preregistration), preferably together with the statistical analysis plan. This meta-research study systematically evaluated registration of Swedish observational research based on national health registries. In a random sample of registry-based observational studies published 2010-2022, very few were preregistered with a publicly available analysis plan (<1 procent). Ideas from the meta-research literature can be leveraged to strengthen the brand of Swedish registry-based observational studies and counteract reporting bias.

On investigating drivers' attention allocation during partially-automated driving.

The use of partially-automated systems require drivers to supervise the system functioning and resume manual control whenever necessary. Yet literature on vehicle automation show that drivers may spend more time looking away from the road when the partially-automated system is operational. In this study we answer the question of whether this pattern is a manifestation of inattentional blindness or, more dangerously, it is also accompanied by a greater attentional processing of the driving scene. Participants drove a simulated vehicle in manual or partially-automated mode. Fixations were recorded by means of a head-mounted eye-tracker. A surprise two-alternative forced-choice recognition task was administered at the end of the data collection whereby participants were quizzed on the presence of roadside billboards that they encountered during the two drives. Data showed that participants were more likely to fixate and recognize billboards when the automated system was operational. Furthermore, whereas fixations toward billboards decreased toward the end of the automated drive, the performance in the recognition task did not suffer. Based on these findings, we hypothesize that the use of the partially-automated driving system may result in an increase in attention allocation toward peripheral objects in the road scene which is detrimental to the drivers' ability to supervise the automated system and resume manual control of the vehicle.

Virtual Interviews and the Pediatric Emergency Medicine Match Geography: A National Survey.

Introduction: Virtual interviews (VI) are now a permanent part of pediatric emergency medicine (PEM) recruitment, especially given the cost and equity advantages. Yet inability to visit programs in person can impact decision-making, leading applicants to apply to more programs. Moreover, the cost advantages of VI may encourage applicants to apply to programs farther away than they might otherwise have been willing or able to travel. This could create unnecessary strain on programs. We conducted this study to determine whether PEM fellowship applicants would apply to a larger number of programs and in different geographic patterns with VI (2020 and 2021) as compared to in-person interviews (2018 and 2019). Methods: We conducted an anonymous national survey of all PEM fellows comparing two cohorts: current fellows who interviewed inperson (applied in 2018/2019) and fellows who underwent VIs in 2020/2021 (current fellows and those recently matched in 2021). The study took place in March-April 2022. Questions focused on geographic considerations during interviews and the match. We used descriptive statistics, chi-square and t-tests for analysis. Results: Overall response rate was 42% (231/550); 32% (n = 74) interviewed in person and 68% (n = 157) virtually. Fellows applied to a median of 4/6 geographic regions (interquartile range 2, 5). Most applied for fellowship both in the same region as residency (216, 93%) and outside (192, 83%). Only the Pacific region saw a statistically significant increase in applicants during VI (59.9% vs 43.2%, P = 0.02). There was no statistical difference in the number of programs applied to during in-person vs VI (mean difference (95% confidence interval 0.72, -2.8 - 4.2). A majority matched in their preferred state both during VI (60.4%) and in-person interviews (65.7%). The difference was not statistically significant (P = 0.45). Conclusion: While more PEM fellowship applicants applied outside the geographic area where their residency was and to the Pacific region, there was no overall increase in the number of programs or geographic areas PEM applicants applied to during VI as compared to in-person interview seasons. As this was the first two years of VI, ongoing data collection will further identify trends and the impactof VI.