Informatics infrastructure in a rural pediatric clinical trials network: Matching specific clinical research needs with best practices and industry guidelines.

Children have historically been underrepresented in randomized controlled trials and multi-center studies. This is particularly true for children who reside in rural and underserved areas. Conducting multi-center trials in rural areas presents unique informatics challenges. These challenges call for increased attention towards informatics infrastructure and the need for development and application of sound informatics approaches to the collection, processing, and management of data for clinical studies. By modifying existing local infrastructure and utilizing open source tools, we have been able to successfully deploy a multi-site data coordinating and operations center. We report our implementation decisions for data collection and management for the IDeA States Pediatric Clinical Trial Network (ISPCTN) based on the functionality needed for the ISPCTN, our synthesis of the extant literature in data collection and management methodology, and Good Clinical Data Management Practices.

A Rule-Based Data Quality Assessment System for Electronic Health Record Data.

OBJECTIVE: Rule-based data quality assessment in health care facilities was explored through compilation, implementation, and evaluation of 63,397 data quality rules in a single-center case study to assess the ability of rules-based data quality assessment to identify data errors of importance to physicians and system owners. METHODS: We applied a design science framework to design, demonstrate, test, and evaluate a scalable framework with which data quality rules can be managed and used in health care facilities for data quality assessment and monitoring. RESULTS: We identified 63,397 rules partitioned into 28 logic templates. A total of 819,683 discrepancies were identified by 4.5% of the rules. Nine out of 11 participating clinical and operational leaders indicated that the rules identified data quality problems and articulated next steps that they wanted to take based on the reported information. DISCUSSION: The combined rule template and knowledge table approach makes governance and maintenance of otherwise large rule sets manageable. Identified challenges to rule-based data quality monitoring included the lack of curated and maintained knowledge sources relevant to data error detection and lack of organizational resources to support clinical and operational leaders with investigation and characterization of data errors and pursuit of corrective and preventative actions. Limitations of our study included implementation within a single center and dependence of the results on the implemented rule set. CONCLUSION: This study demonstrates a scalable framework (up to 63,397 rules) with which data quality rules can be implemented and managed in health care facilities to identify data errors. The data quality problems identified at the implementation site were important enough to prompt action requests from clinical and operational leaders.

Analysis of Professional Competencies for the Clinical Research Data Management Profession.

OBJECTIVE: This job analysis was conducted to compare, assess and refine the competencies of the clinical research data management profession. MATERIALS AND METHODS: Two questionnaires were administered in 2015 and 2018 to collect information from data managers on professional competencies, types of data managed, types of studies supported, and necessary foundational knowledge. RESULTS: In 2018 survey, 67 professional competencies were identified. Job tasks differed between early- to mid-career and mid- to late-career practitioners. A large variation in the types of studies conducted and variation in the data managed by the participants was observed. DISCUSSION: Clinical research data managers managed different types of data with variety of research settings, which indicated a need for training in methods and concepts that could be applied across therapeutic areas and types of data. CONCLUSION: The competency survey reported here serves as the foundation for the upcoming revision of the Certified Clinical Data Manager (CCDMTM) exam.

Evaluative Outcomes in Direct Extraction and Use of EHR Data in Clinical Trials.

Use of electronic health record (EHR) data in clinical trials has long been a goal for researchers. However, few demonstrations and fewer evaluative studies have been published. The variability in outcome choice and measurement hinders synthesis of the extant literature. In collaboration with a contemporaneous systematic review of EHR data use in clinical trial data collection, we analyze reported outcomes and recommend a standardized measure set for the evaluation of human safety, data quality, operational efficiency and cost of eSource solutions.

Training as an Intervention to Decrease Medical Record Abstraction Errors Multicenter Studies.

Studies often rely on medical record abstraction as a major source of data. However, data quality from medical record abstraction has long been questioned. Electronic Health Records (EHRs) potentially add variability to the abstraction process due to the complexity of navigating and locating study data within these systems. We report training for and initial quality assessment of medical record abstraction for a clinical study conducted by the IDeA States Pediatric Clinical Trials Network (ISPCTN) and the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD) Neonatal Research Network (NRN) using medical record abstraction as the primary data source. As part of overall quality assurance, study-specific training for medical record abstractors was developed and deployed during study start-up. The training consisted of a didactic session with an example case abstraction and an independent abstraction of two standardized cases. Sixty-nine site abstractors from thirty sites were trained. The training was designed to achieve an error rate for each abstractor of no greater than 4.93% with a mean of 2.53%, at study initiation. Twenty-three percent of the trainees exceeded the acceptance limit on one or both of the training test cases, supporting the need for such training. We describe lessons learned in the design and operationalization of the study-specific, medical record abstraction training program.

Using Medical Informatics to Improve Clinical Trial Operations.

The continued escalation of clinical trial costs is becoming a public health concern. During the past decade, medical research funding peaked and there is growing concern that there may be insufficient resources to test many promising medical products. Recent changes in the regulatory environment create opportunities for the use of medical informatics to improve clinical trial operations and reduce costs. We report on a Medical Informatics Europe 2016 workshop conducted during the Health - Exploring Complexity (HEC) 2016 conference. We review presentation given on Secondary Data Use, eSource, and Data Quality in Clinical Trials and report on the workshop's discussions.

Towards Data Value-Level Metadata for Clinical Studies.

While several standards for metadata describing clinical studies exist, comprehensive metadata to support traceability of data from clinical studies has not been articulated. We examine uses of metadata in clinical studies. We examine and enumerate seven sources of data value-level metadata in clinical studies inclusive of research designs across the spectrum of the National Institutes of Health definition of clinical research. The sources of metadata inform categorization in terms of metadata describing the origin of a data value, the definition of a data value, and operations to which the data value was subjected. The latter is further categorized into information about changes to a data value, movement of a data value, retrieval of a data value, and data quality checks, constraints or assessments to which the data value was subjected. The implications of tracking and managing data value-level metadata are explored.

Evaluating common data models for use with a longitudinal community registry.

OBJECTIVE: To evaluate common data models (CDMs) to determine which is best suited for sharing data from a large, longitudinal, electronic health record (EHR)-based community registry. MATERIALS AND METHODS: Four CDMs were chosen from models in use for clinical research data: Sentinel v5.0 (referred to as the Mini-Sentinel CDM in previous versions), PCORnet v3.0 (an extension of the Mini-Sentinel CDM), OMOP v5.0, and CDISC SDTM v1.4. Each model was evaluated against 11 criteria adapted from previous research. The criteria fell into six categories: content coverage, integrity, flexibility, ease of querying, standards compatibility, and ease and extent of implementation. RESULTS: The OMOP CDM accommodated the highest percentage of our data elements (76%), fared well on other requirements, and had broader terminology coverage than the other models. Sentinel and PCORnet fell short in content coverage with 37% and 48% matches respectively. Although SDTM accommodated a significant percentage of data elements (55% true matches), 45% of the data elements mapped to SDTM's extension mechanism, known as Supplemental Qualifiers, increasing the number of joins required to query the data. CONCLUSION: The OMOP CDM best met the criteria for supporting data sharing from longitudinal EHR-based studies. Conclusions may differ for other uses and associated data element sets, but the methodology reported here is easily adaptable to common data model evaluation for other uses.

Transparent reporting of data quality in distributed data networks.

INTRODUCTION: Poor data quality can be a serious threat to the validity and generalizability of clinical research findings. The growing availability of electronic administrative and clinical data is accompanied by a growing concern about the quality of these data for observational research and other analytic purposes. Currently, there are no widely accepted guidelines for reporting quality results that would enable investigators and consumers to independently determine if a data source is fit for use to support analytic inferences and reliable evidence generation. MODEL AND METHODS: We developed a conceptual model that captures the flow of data from data originator across successive data stewards and finally to the data consumer. This "data lifecycle" model illustrates how data quality issues can result in data being returned back to previous data custodians. We highlight the potential risks of poor data quality on clinical practice and research results. Because of the need to ensure transparent reporting of a data quality issues, we created a unifying data-quality reporting framework and a complementary set of 20 data-quality reporting recommendations for studies that use observational clinical and administrative data for secondary data analysis. We obtained stakeholder input on the perceived value of each recommendation by soliciting public comments via two face-to-face meetings of informatics and comparative-effectiveness investigators, through multiple public webinars targeted to the health services research community, and with an open access online wiki. RECOMMENDATIONS: Our recommendations propose reporting on both general and analysis-specific data quality features. The goals of these recommendations are to improve the reporting of data quality measures for studies that use observational clinical and administrative data, to ensure transparency and consistency in computing data quality measures, and to facilitate best practices and trust in the new clinical discoveries based on secondary use of observational data.