Demonstrating the data integrity of routinely collected healthcare systems data for clinical trials (DEDICaTe): A proof-of-concept study.

Introduction/aims: Healthcare systems data (also known as real-world or routinely collected health data) could transform the conduct of clinical trials. Demonstrating integrity and provenance of these data is critical for clinical trials, to enable their use where appropriate and avoid duplication using scarce trial resources. Building on previous work, this proof-of-concept study used a data intelligence tool, the "Central Metastore," to provide metadata and lineage information of nationally held data. Methods: The feasibility of NHS England's Central Metastore to capture detailed records of the origins, processes, and methods that produce four datasets was assessed. These were England's Hospital Episode Statistics (Admitted Patient Care, Outpatients, Critical Care) and the Civil Registration of Deaths (England and Wales). The process comprised: information gathering; information ingestion using the tool; and auto-generation of lineage diagrams/content to show data integrity. A guidance document to standardise this process was developed. Results/Discussion: The tool can ingest, store and display data provenance in sufficient detail to support trust and transparency in using these datasets for trials. The slowest step was information gathering from multiple sources, so consistency in record-keeping is essential.

Researchers' experience in data collection with families of femicide victims.

OBJECTIVES: to describe researchers' experience in collecting data from families of femicide victims. METHODS: this descriptive, qualitative study took the form of an experience report and was conducted in Manaus, Amazonas, Brazil. It involved documentary consultation, training researchers, scheduling and conducting interviews, and using a field diary to record the researchers' perceptions and experiences. RESULTS: the descriptions and photographs of the crime scene were both distressing and impactful for the researchers. The mementos of the victims (including clothing, objects, and childhood photos) shown by their families were deeply moving. Identifying with these experiences facilitated listening to the stories told by the relatives. It was essential to maintain a non-judgmental attitude, acknowledge the loss, provide support for the suffering, and demonstrate a willingness to help. FINAL CONSIDERATIONS: the experience encompassed both theoretical and methodological aspects that were planned and executed in data collection, fostering the development of skills and sensitivity towards the cases. Beyond knowledge and preparation, researchers are expected to exhibit ethical conduct and empathetic capacity.

Method for Using Voicemail and Email for Qualitative Data Collection Among Nurses.

BACKGROUND: Memorializing nurses' experiences during the COVID-19 pandemic had the potential to allow scientists and policymakers to learn about the impact on the nursing profession and health care systems. Yet, nurses are considered a difficult population to recruit for research. OBJECTIVE: To describe an innovative qualitative data collection method for capturing current practice experiences among nurses working during the COVID-19 pandemic. METHODS: Guerilla theory served as the theoretical framework. Utilizing a qualitative descriptive design, a telephone voicemail messaging system was developed to capture nurses' experiences. RESULTS: Nurses were recruited with convenience and snowball sampling via social media and state listservs. The telephone voicemail messaging system, Twilio, was used. After listening to the recording of the consent form, the participants shared their experiences by leaving a voice message where they answered the prompt, "Tell us about your experiences working during the COVID-19 pandemic." Seventy voicemails were included, and the voicemails were transcribed. After a nurse shared their experience via an email sent to the research team, emails were added to the data collection; 16 emails were received. Transcripts and emails were uploaded to the qualitative data analysis software program, Dedoose, and coded by 2 researchers using content analysis. Main themes were derived and discussed among the research team. CONCLUSION: Allowing participants multiple modes of expressing their experiences promote inclusivity in data collection. Further development and standardization of this method is needed for future research.

A Framework to Support the Standardized Management of Real-World Data.

Relying on our experience on the development of data registration and management systems for clinical and biological data coming from patients with hematological malignancies, as well as on the design of strategies for data collection and analysis to support multi-center, clinical association studies, we designed a framework for the standardized collection and transformation of clinically relevant real-world data into evidence, to meet the challenges of gathering biomedical data collected during daily clinical practice in order to promote basic and clinical research.

Guidance of development, validation, and evaluation of algorithms for populating health status in observational studies of routinely collected data (DEVELOP-RCD).

BACKGROUND: In recent years, there has been a growing trend in the utilization of observational studies that make use of routinely collected healthcare data (RCD). These studies rely on algorithms to identify specific health conditions (e.g. diabetes or sepsis) for statistical analyses. However, there has been substantial variation in the algorithm development and validation, leading to frequently suboptimal performance and posing a significant threat to the validity of study findings. Unfortunately, these issues are often overlooked. METHODS: We systematically developed guidance for the development, validation, and evaluation of algorithms designed to identify health status (DEVELOP-RCD). Our initial efforts involved conducting both a narrative review and a systematic review of published studies on the concepts and methodological issues related to algorithm development, validation, and evaluation. Subsequently, we conducted an empirical study on an algorithm for identifying sepsis. Based on these findings, we formulated specific workflow and recommendations for algorithm development, validation, and evaluation within the guidance. Finally, the guidance underwent independent review by a panel of 20 external experts who then convened a consensus meeting to finalize it. RESULTS: A standardized workflow for algorithm development, validation, and evaluation was established. Guided by specific health status considerations, the workflow comprises four integrated steps: assessing an existing algorithm's suitability for the target health status; developing a new algorithm using recommended methods; validating the algorithm using prescribed performance measures; and evaluating the impact of the algorithm on study results. Additionally, 13 good practice recommendations were formulated with detailed explanations. Furthermore, a practical study on sepsis identification was included to demonstrate the application of this guidance. CONCLUSIONS: The establishment of guidance is intended to aid researchers and clinicians in the appropriate and accurate development and application of algorithms for identifying health status from RCD. This guidance has the potential to enhance the credibility of findings from observational studies involving RCD.

Metadata for Data dIscoverability aNd Study rEplicability in obseRVAtional Studies (MINERVA): Development and Pilot of a Metadata List and Catalogue in Europe.

PURPOSE: Metadata for data dIscoverability aNd study rEplicability in obseRVAtional studies (MINERVA), a European Medicines Agency-funded project (EUPAS39322), defined a set of metadata to describe real-world data sources (RWDSs) and piloted metadata collection in a prototype catalogue to assist investigators from data source discoverability through study conduct. METHODS: A list of metadata was created from a review of existing metadata catalogues and recommendations, structured interviews, a stakeholder survey, and a technical workshop. The prototype was designed to comply with the FAIR principles (findable, accessible, interoperable, reusable), using MOLGENIS software. Metadata collection was piloted by 15 data access partners (DAPs) from across Europe. RESULTS: A total of 442 metadata variables were defined in six domains: institutions (organizations connected to a data source); data banks (data collections sustained by an organization); data sources (collections of linkable data banks covering a common underlying population); studies; networks (of institutions); and common data models (CDMs). A total of 26 institutions were recorded in the prototype. Each DAP populated the metadata of one data source and its selected data banks. The number of data banks varied by data source; the most common data banks were hospital administrative records and pharmacy dispensation records (10 data sources each). Quantitative metadata were successfully extracted from three data sources conforming to different CDMs and entered into the prototype. CONCLUSIONS: A metadata list was finalized, a prototype was successfully populated, and a good practice guide was developed. Setting up and maintaining a metadata catalogue on RWDSs will require substantial effort to support discoverability of data sources and reproducibility of studies in Europe.

Improving Collection and Analysis of Overall Survival Data.

Advances in anticancer therapies have provided crucial benefits for millions of patients who are living long and fulfilling lives. Although these successes should be celebrated, there is certainly room to continue improving cancer care. Increased long-term survival presents additional challenges for determining whether new therapies further extend patients' lives through clinical trials, commonly known as the gold standard endpoint of overall survival (OS). As a result, an increasing reliance is observed on earlier efficacy endpoints, which may or may not correlate with OS, to continue the timely pace of translating innovation into novel therapies available for patients. Even when not powered as an efficacy endpoint, OS remains a critical indication of safety for regulatory decisions and is a key aspect of the FDA's Project Endpoint. Unfortunately, in the pursuit of earlier endpoints, many registrational clinical trials lack adequate planning, collection, and analysis of OS data, which complicates interpretation of a net clinical benefit or harm. This article shares best practices, proposes novel statistical methodologies, and provides detailed recommendations to improve the rigor of using OS data to inform benefit-risk assessments, including incorporating the following in clinical trials intending to demonstrate the safety and effectiveness of cancer therapy: prospective collection of OS data, establishment of fit-for-purpose definitions of OS detriment, and prespecification of analysis plans for using OS data to evaluate for potential harm. These improvements hold promise to help regulators, patients, and providers better understand the benefits and risks of novel therapies.

Evaluation of software technical quality for collecting data from patients under palliative care.

OBJECTIVES: to evaluate software technical quality for collecting data from patients under palliative care. METHODS: this is methodological technology evaluation research, according to the technical standard International Organization for Standardization/International Electrotechnical Commission 25040-2011, developed from August 2021 to August 2023. Eight nurses and eight information technology professionals participated as judges, who evaluated six quality characteristics and 23 subcharacteristics. Items that reached a percentage of agreement greater than 70% were considered suitable. RESULTS: the characteristics evaluated by nurses/information technology professionals received the following percentages of agreement, respectively: functional suitability (94%-84%); reliability (100-70%); usability (89.9-66.8%); performance efficiency (95.8%-86.1%); compatibility (95.8-79.6%); and safety (96%-83.4%). CONCLUSIONS: the software was considered suitable in quality evaluation to offer support to nurses in collecting patient data under palliative care, with the potential to operationalize the first Nursing Process stage.

Collecting routine and timely cancer stage at diagnosis by implementing a cancer staging tiered framework: the Western Australian Cancer Registry experience.

BACKGROUND: Current processes collecting cancer stage data in population-based cancer registries (PBCRs) lack standardisation, resulting in difficulty utilising diverse data sources and incomplete, low-quality data. Implementing a cancer staging tiered framework aims to improve stage collection and facilitate inter-PBCR benchmarking. OBJECTIVE: Demonstrate the application of a cancer staging tiered framework in the Western Australian Cancer Staging Project to establish a standardised method for collecting cancer stage at diagnosis data in PBCRs. METHODS: The tiered framework, developed in collaboration with a Project Advisory Group and applied to breast, colorectal, and melanoma cancers, provides business rules - procedures for stage collection. Tier 1 represents the highest staging level, involving complete American Joint Committee on Cancer (AJCC) tumour-node-metastasis (TNM) data collection and other critical staging information. Tier 2 (registry-derived stage) relies on supplementary data, including hospital admission data, to make assumptions based on data availability. Tier 3 (pathology stage) solely uses pathology reports. FINDINGS: The tiered framework promotes flexible utilisation of staging data, recognising various levels of data completeness. Tier 1 is suitable for all purposes, including clinical and epidemiological applications. Tiers 2 and 3 are recommended for epidemiological analysis alone. Lower tiers provide valuable insights into disease patterns, risk factors, and overall disease burden for public health planning and policy decisions. Capture of staging at each tier depends on data availability, with potential shifts to higher tiers as new data sources are acquired. CONCLUSIONS: The tiered framework offers a dynamic approach for PBCRs to record stage at diagnosis, promoting consistency in population-level staging data and enabling practical use for benchmarking across jurisdictions, public health planning, policy development, epidemiological analyses, and assessing cancer outcomes. Evolution with staging classifications and data variable changes will futureproof the tiered framework. Its adaptability fosters continuous refinement of data collection processes and encourages improvements in data quality.

The Icelandic Prevention Model Evaluation Framework and Implementation Integrity and Consistency Assessment.

The Icelandic Prevention Model (IPM) follows a systematic but flexible process of community capacity building, data collection, analysis, dissemination, and community-engaged decision-making to guide the data-informed selection, prioritization, and implementation of intervention strategies in preventing adolescent substance use. This paper describes two new evaluation tools intended to assess the: 1) integrity of IPM implementation, and 2) unique aspects of IPM implementation in different community contexts. These evaluation tools include a: 1) five-phase IPM Evaluation Framework for Assessing Value Across Communities, Cultures, and Outcomes (IPM-EF); and 2) 10-Step IPM Implementation Integrity and Consistency Assessment (IPM-IICA) that utilizes both quantitative (scored) and qualitative (narrative) data elements to characterize implementation integrity and consistency at both community coalition and school community levels. The IPM-EF includes five phases. Phase 1: Describe the Intervention Context; Phase 2a: Document the Extent to Which the 10 Steps of the IPM were Implemented (using the IPM-IICA scored); Phase 2b: Document the Unique Community-Specific Methods Used within the 10 Steps of the IPM to Tailor Local Intervention Delivery (using the IPM-IICA narrative); Phase 3: Measure Changes in Community Risk and Protective Factors; Phase 4: Measure the Outcomes Associated with the IPM; and Phase 5: Investigate Multiple Full Cycles Over Time.

Protocol for the Evaluation of Data-Related Processes and Challenges at Level 1-4 Trauma Centers in Arkansas: A Mixed-Methods Case Study.

Background: In the following manuscript, we describe the detailed protocol for a mixed-methods, observational case study conducted to identify and evaluate existing data-related processes and challenges currently faced by trauma centers in a rural state. The data will be utilized to assess the impact of these challenges on registry data collection. Methods: The study relies on a series of interviews and observations to collect data from trauma registry staff at level 1-4 trauma centers across the state of Arkansas. A think-aloud protocol will be used to facilitate observations to gather keystroke-level modeling data and insight into site processes and workflows for collecting and submitting data to the Arkansas Trauma Registry. Informal, semi-structured interviews will follow the observation period to assess the participant's perspective on current processes, potential barriers to data collection or submission to the registry, and recommendations for improvement. Each session will be recorded, and de-identified transcripts and session notes will be used for analysis. Keystroke level modeling data derived from observations will be extracted and analyzed quantitatively to determine time spent performing end-to-end registry-related activities. Qualitative data from interviews will be reviewed and coded by 2 independent reviewers following a thematic analysis methodology. Each set of codes will then be adjudicated by the reviewers using a consensus-driven approach to extrapolate the final set of themes. Discussion: We will utilize a mixed methods approach to understand existing processes and barriers to data collection for the Arkansas Trauma Registry. Anticipated results will provide a baseline measure of the data collection and submission processes at various trauma centers across the state. We aim to assess strengths and limitations of existing processes and identify existing barriers to interoperability. These results will provide first-hand knowledge on existing practices for the trauma registry use case and will provide quantifiable data that can be utilized in future research to measure outcomes of future process improvement efforts. The potential implications of this study can form the basis for identifying potential solutions for streamlining data collection, exchange, and utilization of trauma registry data for clinical practice, public health, and clinical and translational research.

Recommendations for data collection in cohort studies of preterm born individuals - The RECAP Preterm Core Dataset.

BACKGROUND: Preterm birth (before 37 completed weeks of gestation) is associated with an increased risk of adverse health and developmental outcomes relative to birth at term. Existing guidelines for data collection in cohort studies of individuals born preterm are either limited in scope, have not been developed using formal consensus methodology, or did not involve a range of stakeholders in their development. Recommendations meeting these criteria would facilitate data pooling and harmonisation across studies. OBJECTIVES: To develop a Core Dataset for use in longitudinal cohort studies of individuals born preterm. METHODS: This work was carried out as part of the RECAP Preterm project. A systematic review of variables included in existing core outcome sets was combined with a scoping exercise conducted with experts on preterm birth. The results were used to generate a draft core dataset. A modified Delphi process was implemented using two stages with three rounds each. Three stakeholder groups participated: RECAP Preterm project partners; external experts in the field; people with lived experience of preterm birth. The Delphi used a 9-point Likert scale. Higher values indicated greater importance for inclusion. Participants also suggested additional variables they considered important for inclusion which were voted on in later rounds. RESULTS: An initial list of 140 data items was generated. Ninety-six participants across 22 countries participated in the Delphi, of which 29% were individuals with lived experience of preterm birth. Consensus was reached on 160 data items covering Antenatal and Birth Information, Neonatal Care, Mortality, Administrative Information, Organisational Level Information, Socio-economic and Demographic information, Physical Health, Education and Learning, Neurodevelopmental Outcomes, Social, Lifestyle and Leisure, Healthcare Utilisation and Quality of Life. CONCLUSIONS: This core dataset includes 160 data items covering antenatal care through outcomes in adulthood. Its use will guide data collection in new studies and facilitate pooling and harmonisation of existing data internationally.

SHEA position statement on pandemic preparedness for policymakers: pandemic data collection, maintenance, and release.

The Society for Healthcare Epidemiology in America (SHEA) strongly supports modernization of data collection processes and the creation of publicly available data repositories that include a wide variety of data elements and mechanisms for securely storing both cleaned and uncleaned data sets that can be curated as clinical and research needs arise. These elements can be used for clinical research and quality monitoring and to evaluate the impacts of different policies on different outcomes. Achieving these goals will require dedicated, sustained and long-term funding to support data science teams and the creation of central data repositories that include data sets that can be "linked" via a variety of different mechanisms and also data sets that include institutional and state and local policies and procedures. A team-based approach to data science is strongly encouraged and supported to achieve the goal of a sustainable, adaptable national shared data resource.

Replication of a local record keeping method for collecting road crash data in low resource settings: lessons from Bangladesh and Nepal.

BACKGROUND: Police road crash and injury data in low-income and middle-income countries are known to under-report crashes, fatalities and injuries, especially for vulnerable road users. Local record keepers, who are members of the public, can be engaged to provide an additional source of crash and injury data. METHODS: This paper compares the application of a local record keeper method to capture road crash and injury data in Bangladesh and Nepal, assesses the quality of the data collected and evaluates the replicability and value of the methodology using a framework developed to evaluate the impact of being a local record keeper. OUTCOME: Application in research studies in both Bangladesh and Nepal found the local record keeper methodology provided high-quality and complete data compared with local police records. The methodology was flexible enough to adapt to project and context differences. The evaluation framework enabled the identification of the challenges and unexpected benefits realised in each study. This led to the development of an 11-step process for conducting road crash data collection using local record keepers, which is presented to facilitate replication in other settings. CONCLUSION: Data collected by local record keepers are a flexible and replicable method to understand the strengths and limitations of existing police data, adding to the evidence base and informing local and national decision-making. The method may create additional benefits for data collectors and communities, help design and assess road safety interventions and support advocacy for improved routine police data.

Improving quality of care by standardising patient data collection in electronic medical records in an oncology department in Spain.

BACKGROUND: Evaluation of quality of care in oncology is key in ensuring patients receive adequate treatment. American Society of Clinical Oncology's (ASCO) Quality Oncology Practice Initiative (QOPI) Certification Program (QCP) is an international initiative that evaluates quality of care in outpatient oncology practices. METHODS: We retrospectively reviewed free-text electronic medical records from patients with breast cancer (BR), colorectal cancer (CRC) or non-small cell lung cancer (NSCLC). In a baseline measurement, high scores were obtained for the nine disease-specific measures of QCP Track (2021 version had 26 measures); thus, they were not further analysed. We evaluated two sets of measures: the remaining 17 QCP Track measures, as well as these plus other 17 measures selected by us (combined measures). Review of data from 58 patients (26 BR; 18 CRC; 14 NSCLC) seen in June 2021 revealed low overall quality scores (OQS)-below ASCO's 75% threshold-for QCP Track measures (46%) and combined measures (58%). We developed a plan to improve OQS and monitored the impact of the intervention by abstracting data at subsequent time points. RESULTS: We evaluated potential causes for the low OQS and developed a plan to improve it over time by educating oncologists at our hospital on the importance of improving collection of measures and highlighting the goal of applying for QOPI certification. We conducted seven plan-do-study-act cycles and evaluated the scores at seven subsequent data abstraction time points from November 2021 to December 2022, reviewing 404 patients (199 BR; 114 CRC; 91 NSCLC). All measures were improved. Four months after the intervention, OQS surpassed the quality threshold and was maintained for 10 months until the end of the study (range, 78-87% for QCP Track measures; 78-86% for combined measures). CONCLUSIONS: We developed an easy-to-implement intervention that achieved a fast improvement in OQS, enabling our Medical Oncology Department to aim for QOPI certification.