Taming mental-health-focused popular literature: a crazy idea?

Providing tailored and easily accessible health information for mental health clinicians and patients can be enabled through Information Technology and Communications (ITC). The literature is mixed regarding the quality, utility and accessibility of health information in the popular press for this purpose. However, there is consensus that mental health information in the popular press is readily available, easily comprehended by patients, and is continually updated. We report the process by which mental-health-focused articles in the popular press are identified, screened, and disseminated to a large network of doctoral level psychologists (the PsyUSA network). We analyze 4-year article distribution and access data, and conclude that the distribution of mental-health-related popular press articles prompted article access. We leverage this experience to formulate a model for direct access to clinician-vetted mental-health-related popular press through a curated web based archive.

Design and implementation of an institutional case report form library.

BACKGROUND: Case report forms (CRFs) are used to collect data in clinical research. Case report form development represents a significant part of the clinical trial process and can affect study success. Libraries of CRFs can preserve the organizational knowledge and expertise invested in CRF development and expedite the sharing of such knowledge. Although CRF libraries have been advocated, there have been no published accounts reporting institutional experiences with creating and using them. PURPOSE: We sought to enhance an existing institutional CRF library by improving information indexing and accessibility. We describe this CRF library and discuss challenges encountered in its development and implementation, as well as future directions for continued work in this area. METHODS: We transformed an existing but underused and poorly accessible CRF library into a resource capable of supporting and expediting clinical and translational investigation at our institution by (1) expanding access to the entire institution; (2) adding more form attributes for improved information retrieval; and (3) creating a formal information curation and maintenance process. An open-source content management system, Plone (Plone.org), served as the platform for our CRF library. RESULTS: We report results from these three processes. Over the course of this project, the size of the CRF library increased from 160 CRFs comprising an estimated total of 17,000 pages, to 177 CRFs totaling 1.5 gigabytes. Eighty-two of these CRFs are now available to researchers across our institution; 95 CRFs remain within a contractual confidentiality window (usually 5 years from database lock) and are not available to users outside of the Duke Clinical Research Institute (DCRI). Conservative estimates suggest that the library supports an average of 37 investigators per month. The resources needed to curate and maintain the CRF library require less than 10% of the effort of one full-time equivalent employee. LIMITATIONS: Although we succeeded in expanding use of the CRF library, creating awareness of such institutional resources among investigators and research teams remains challenging and requires additional efforts to overcome. Institutions that have not achieved a critical mass of attractive research resources or effective dissemination mechanisms may encounter persistent difficulty attracting researchers to use institutional resources. Further, a useful CRF library requires both an initial investment of resources for development, as well as ongoing maintenance once it is established. CONCLUSIONS: CRF libraries can be established and made broadly available to institutional researchers. Curation - that is, indexing newly added forms - is required. Such a resource provides knowledge management capacity for institutions until standards and software are available to support widespread exchange of data and form definitions.

Defining a framework for health information technology evaluation.

Governments and providers are investing in health information technologies with little evidence as to their ultimate value. We present a conceptual framework that can be used by hospitals, clinics, and health care systems to evaluate their health information technologies. The framework contains three dimensions that collectively define generic evaluation types. When these types are combined with contextual considerations, they define specific evaluation problems. The first dimension, domain, determines whether the evaluation will address the information intervention or its outcomes. The second dimension, mechanism, identifies the specific components of the new information technology and/or its health care system that will be the subject of the evaluation study. And, the third dimension, timing, determines whether the evaluation occurs before or after the health information technology is implemented. Answers to these questions define a set of evaluation types each with generic sets of evaluation questions, study designs, data collection requirements, and analytic methods. When these types are combined with details of the evaluation context, they define specific evaluation problems.

Economic analysis of centralized vs. decentralized electronic data capture in multi-center clinical studies.

BACKGROUND: New data management models are emerging in multi-center clinical studies. We evaluated the incremental costs associated with decentralized vs. centralized models. METHODS: We developed clinical research network economic models to evaluate three data management models: centralized, decentralized with local software, and decentralized with shared database. Descriptive information from three clinical research studies served as inputs for these models. MAIN OUTCOME MEASURES: The primary outcome was total data management costs. Secondary outcomes included: data management costs for sites, local data centers, and central coordinating centers. RESULTS: Both decentralized models were more costly than the centralized model for each clinical research study: the decentralized with local software model was the most expensive. Decreasing the number of local data centers and case book pages reduced cost differentials between models. CONCLUSION: Decentralized vs. centralized data management in multi-center clinical research studies is associated with increases in data management costs.

A system dynamics analysis determining willingness to wait and pay for the implementation of data standards in clinical research.

BACKGROUND: Industry standards provide rigorous descriptions of required data presentation, with the aim of ensuring compatibility across different clinical studies. However despite their crucial importance, these standards are often not used as expected in the development of clinical research. The reasons for this lack of compliance could be related to the high cost and time-intensive nature of the process of data standards implementation. The objective of this study was to evaluate the value of the extra time and cost required for different levels of data standardisation and the likelihood of researchers to comply with these levels. Since we believe that the cost and time necessary for the implementation of data standards can change over time, System Dynamics (SD) analysis was used to investigate how these variables interact and influence the adoption of data standards by clinical researchers. METHODS: Three levels of data standards implementation were defined through focus group discussion involving four clinical research investigators. Ten Brazilian and eighteen American investigators responded to an online questionnaire which presented possible standards implementation scenarios, with respondents asked to choose one of two options available in each scenario. A random effects ordered probit model was used to estimate the effect of cost and time on investigators' willingness to adhere to data standards. The SD model was used to demonstrate the relationship between degrees of data standardisation and subsequent variation in cost and time required to start the associated study. RESULTS: A preference for low cost and rapid implementation times was observed, with investigators more likely to incur costs than to accept a time delay in project start-up. SD analysis indicated that although initially extra time and cost are necessary for clinical study standardisation, there is a decrease in both over time. CONCLUSIONS: Future studies should explore ways of creating mechanisms which decrease the time and cost associated with standardisation processes. In addition, the fact that the costs and time necessary for data standards implementation decrease with time should be made known to the wider research community. Policy makers should attempt to match their data standardisation policies better with the expectations of researchers.

Operationalization of the UFuRT methodology for usability analysis in the clinical research data management domain.

Data management software applications specifically designed for the clinical research environment are increasingly available from commercial vendors and open-source communities, however, general-purpose spreadsheets remain widely employed in clinical research data management (CRDM). The suitability of spreadsheets for this use is controversial, and no formal comparative usability evaluations have been performed. We report on an application of the UFuRT (user, function, representation, and task (analyses) methodology to create a domain-specific process for usability evaluation. We demonstrate this process in an evaluation of differences in usability between a spreadsheet program (Microsoft Excel) and a commercially available clinical research data management system (Phase Forward Clintrial). Through this domain-specific operationalization of UFuRT methodology, we successfully identified usability differences and quantified task and cost differences, while differentiating these from socio-technical aspects. UFuRT can be generalized to other domains.

Implementing Single Source: the STARBRITE proof-of-concept study.

OBJECTIVE: Inefficiencies in clinical trial data collection cause delays, increase costs, and may reduce clinician participation in medical research. In this proof-of-concept study, we examine the feasibility of using point-of-care data capture for both the medical record and clinical research in the setting of a working clinical trial. We hypothesized that by doing so, we could increase reuse of patient data, eliminate redundant data entry, and minimize disruption to clinic workflow. DESIGN: We developed and used a point-of-care electronic data capture system to record data during patient visits. The standards-based system was used for clinical research and to generate the clinic note for the medical record. The system worked in parallel with data collection procedures already in place for an ongoing multicenter clinical trial. Our system was iteratively designed after analyzing case report forms and clinic notes, and observing clinic workflow patterns and business procedures. Existing data standards from CDISC and HL7 were used for database insertion and clinical document exchange. RESULTS: Our system was successfully integrated into the clinic environment and used in two live test cases without disrupting existing workflow. Analyses performed during system design yielded detailed information on practical issues affecting implementation of systems that automatically extract, store, and reuse healthcare data. CONCLUSION: Although subject to the limitations of a small feasibility study, our study demonstrates that electronic patient data can be reused for prospective multicenter clinical research and patient care, and demonstrates a need for further development of therapeutic area standards that can facilitate researcher use of healthcare data.

Data standard ≠ data quality.

The relationship between data quality and data standards has not been clearly articulated. While some directly state that data standards increase data quality, others claim the opposite. Depending on the type of data standard and the aspects of data quality considered, both arguments may in fact be correct. We deconstruct a typology of data standards and ap ply a dimensional definition of data quality to clearly articulate the relationship between the two, providing a framework for data quality planning.

A comprehensive framework for data quality assessment in CER.

The panel addresses the urgent need to ensure that comparative effectiveness research (CER) findings derived from diverse and distributed data sources are based on credible, high-quality data; and that the methods used to assess and report data quality are consistent, comprehensive, and available to data consumers. The panel consists of representatives from four teams leveraging electronic clinical data for CER, patient centered outcomes research (PCOR), and quality improvement (QI) and seeks to change the current paradigm where data quality assessment (DQA) is performed "behind the scenes" using one-off project specific methods. The panelists will present their process of harmonizing existing models for describing and measuring clinical data quality and will describe a comprehensive integrated framework for assessing and reporting DQA findings. The collaborative project is supported by the Electronic Data Methods (EDM) Forum, a three-year grant from the Agency for Healthcare Research and Quality (AHRQ) to facilitate learning and foster collaboration across a set of CER, PCOR, and QI projects designed to build infrastructure and methods for collecting and analyzing prospective data from electronic clinical data .

Electronic health records based phenotyping in next-generation clinical trials: a perspective from the NIH Health Care Systems Collaboratory.

Widespread sharing of data from electronic health records and patient-reported outcomes can strengthen the national capacity for conducting cost-effective clinical trials and allow research to be embedded within routine care delivery. While pragmatic clinical trials (PCTs) have been performed for decades, they now can draw on rich sources of clinical and operational data that are continuously fed back to inform research and practice. The Health Care Systems Collaboratory program, initiated by the NIH Common Fund in 2012, engages healthcare systems as partners in discussing and promoting activities, tools, and strategies for supporting active participation in PCTs. The NIH Collaboratory consists of seven demonstration projects, and seven problem-specific working group 'Cores', aimed at leveraging the data captured in heterogeneous 'real-world' environments for research, thereby improving the efficiency, relevance, and generalizability of trials. Here, we introduce the Collaboratory, focusing on its Phenotype, Data Standards, and Data Quality Core, and present early observations from researchers implementing PCTs within large healthcare systems. We also identify gaps in knowledge and present an informatics research agenda that includes identifying methods for the definition and appropriate application of phenotypes in diverse healthcare settings, and methods for validating both the definition and execution of electronic health records based phenotypes.

Research management team (RMT): a model for research support services at Duke University.

Collecting and managing data for clinical and translational research presents significant challenges for clinical and translational researchers, many of whom lack needed access to data management expertise, methods, and tools. At many institutions, funding constraints result in differential levels of research informatics support among investigators. In addition, the lack of widely shared models and ontologies for clinical research informatics and health information technology hampers the accurate assessment of investigators' needs and complicates the efficient allocation of crucial resources for research projects, ultimately affecting the quality and reliability of research. In this paper, we present a model for providing flexible, cost-efficient institutional support for clinical and translational research data management and informatics, the research management team, and describe our initial experiences with deploying this model at our institution.

The MURDOCK Study: a long-term initiative for disease reclassification through advanced biomarker discovery and integration with electronic health records.

BACKGROUND: Facing critically low return per dollar invested on clinical research and clinical care, the American biomedical enterprise is in need of a significant transformation. A confluence of high-throughput "omic" technologies and increasing adoption of the electronic health record has fueled excitement for a new paradigm for biomedical research and practice. The ability to simultaneously measure thousands of molecular variables and assess their relationships with clinical data collected during the course of care could enable reclassification of disease not only by gross phenotypic observation but according to underlying molecular mechanism and influence of social determinants.In turn, this reclassification could enable development of targeted therapeutic interventions as well as disease prevention strategies at the individual and population levels. METHODS/DESIGN: The MURDOCK Study consists of distinct project "horizons" or stages. Horizon 1 entailed the generation and analysis of molecular data for existing large,clinically well-annotated cohorts in four disease areas. Horizon 1.5 involves creating and maintaining a 50,000-person,community volunteer registry for biomarker signature validation and prospective studies, including integration of environmental and social data. Horizon 2 leverages and prospectively recruits Horizon 1.5 volunteers, and extends the study to additional disease areas of interest. Horizon 3 will expand the study through regional, national,and international partnerships. DISCUSSION: The MURDOCK Study embodies a new model of team science investigation and represents a significant resource for translational research. The study team invites inquiries to form new collaborations to exploit the rich resources provided by these biospecimens and associated study data.

The Measurement to Understand Reclassification of Disease of Cabarrus/Kannapolis (MURDOCK) Study Community Registry and Biorepository.

Current understanding of chronic diseases is based on crude clinical characterization, imaging studies, and laboratory testing that has evolved over decades. The Measurement to Understand Reclassification of Disease of Cabarrus/Kannapolis (MURDOCK) Study is a multi-tiered, longitudinal study designed to enable classification of chronic diseases using clinically annotated biospecimen collections, -omic technologies, electronic health records, and standard epidemiological methods. We expect that detailed molecular classification will improve mechanistic understanding of chronic diseases, augmenting discovery and testing of new treatments, and allowing refined selection of prevention and treatment strategies. The MURDOCK Study Community Registry and Biorepository will serve as a bridge for validation of initial exploratory studies, a platform for future prospective studies in targeted populations, and a resource of both data (analytical and clinical) and samples for cross-registry meta-analyses and comparative population studies. Participation of local health care providers and the Cabarrus County/Kannapolis, NC, community will facilitate future medical research and provide the opportunity to educate and inform the public about genomic research, actively engaging them in shaping the future of medical discovery and treatment of chronic diseases. We present the rationale and study design for the MURDOCK Community Registry and Biorepository and baseline characteristics of the first 6000 participants.

Impact of the Patient-Reported Outcomes Management Information System (PROMIS) upon the design and operation of multi-center clinical trials: a qualitative research study.

New technologies may be required to integrate the National Institutes of Health's Patient Reported Outcome Management Information System (PROMIS) into multi-center clinical trials. To better understand this need, we identified likely PROMIS reporting formats, developed a multi-center clinical trial process model, and identified gaps between current capabilities and those necessary for PROMIS. These results were evaluated by key trial constituencies. Issues reported by principal investigators fell into two categories: acceptance by key regulators and the scientific community, and usability for researchers and clinicians. Issues reported by the coordinating center, participating sites, and study subjects were those faced when integrating new technologies into existing clinical trial systems. We then defined elements of a PROMIS Tool Kit required for integrating PROMIS into a multi-center clinical trial environment. The requirements identified in this study serve as a framework for future investigators in the design, development, implementation, and operation of PROMIS Tool Kit technologies.

Distributed cognition artifacts on clinical research data collection forms.

Medical record abstraction, a primary mode of data collection in secondary data use, is associated with high error rates. Cognitive factors have not been studied as a possible explanation for medical record abstraction errors. We employed the theory of distributed representation and representational analysis to systematically evaluate cognitive demands in medical record abstraction and the extent of external cognitive support employed in a sample of clinical research data collection forms.We show that the cognitive load required for abstraction in 61% of the sampled data elements was high, exceedingly so in 9%. Further, the data collection forms did not support external cognition for the most complex data elements. High working memory demands are a possible explanation for the association of data errors with data elements requiring abstractor interpretation, comparison, mapping or calculation. The representational analysis used here can be used to identify data elements with high cognitive demands.

Can prospective usability evaluation predict data errors?

Increasing amounts of clinical research data are collected by manual data entry into electronic source systems and directly from research subjects. For this manual entered source data, common methods of data cleaning such as post-entry identification and resolution of discrepancies and double data entry are not feasible. However data accuracy rates achieved without these mechanisms may be higher than desired for a particular research use. We evaluated a heuristic usability method for utility as a tool to independently and prospectively identify data collection form questions associated with data errors. The method evaluated had a promising sensitivity of 64% and a specificity of 67%. The method was used as described in the literature for usability with no further adaptations or specialization for predicting data errors. We conclude that usability evaluation methodology should be further investigated for use in data quality assurance.

The human studies database project: federating human studies design data using the ontology of clinical research.

Human studies, encompassing interventional and observational studies, are the most important source of evidence for advancing our understanding of health, disease, and treatment options. To promote discovery, the design and results of these studies should be made machine-readable for large-scale data mining, synthesis, and re-analysis. The Human Studies Database Project aims to define and implement an informatics infrastructure for institutions to share the design of their human studies. We have developed the Ontology of Clinical Research (OCRe) to model study features such as design type, interventions, and outcomes to support scientific query and analysis. We are using OCRe as the reference semantics for federated data sharing of human studies over caGrid, and are piloting this implementation with several Clinical and Translational Science Award (CTSA) institutions.

What can we learn from a decade of database audits? The Duke Clinical Research Institute experience, 1997--2006.

BACKGROUND: Despite a pressing and well-documented need for better sharing of information on clinical trials data quality assurance methods, many research organizations remain reluctant to publish descriptions of and results from their internal auditing and quality assessment methods. PURPOSE: We present findings from a review of a decade of internal data quality audits performed at the Duke Clinical Research Institute, a large academic research organization that conducts data management for a diverse array of clinical studies, both academic and industry-sponsored. In so doing, we hope to stimulate discussions that could benefit the wider clinical research enterprise by providing insight into methods of optimizing data collection and cleaning, ultimately helping patients and furthering essential research. METHODS: We present our audit methodologies, including sampling methods, audit logistics, sample sizes, counting rules used for error rate calculations, and characteristics of audited trials. We also present database error rates as computed according to two analytical methods, which we address in detail, and discuss the advantages and drawbacks of two auditing methods used during this 10-year period. RESULTS: Our review of the DCRI audit program indicates that higher data quality may be achieved from a series of small audits throughout the trial rather than through a single large database audit at database lock. We found that error rates trended upward from year to year in the period characterized by traditional audits performed at database lock (1997-2000), but consistently trended downward after periodic statistical process control type audits were instituted (2001-2006). These increases in data quality were also associated with cost savings in auditing, estimated at 1000 h per year, or the efforts of one-half of a full time equivalent (FTE). LIMITATIONS: Our findings are drawn from retrospective analyses and are not the result of controlled experiments, and may therefore be subject to unanticipated confounding. In addition, the scope and type of audits we examine here are specific to our institution, and our results may not be broadly generalizable. CONCLUSIONS: Use of statistical process control methodologies may afford advantages over more traditional auditing methods, and further research will be necessary to confirm the reliability and usability of such techniques. We believe that open and candid discussion of data quality assurance issues among academic and clinical research organizations will ultimately benefit the entire research community in the coming era of increased data sharing and re-use.

A centralized informatics infrastructure for the National Institute on Drug Abuse Clinical Trials Network.

BACKGROUND: Clinical trial networks (CTNs) were created to provide a sustaining infrastructure for the conduct of multisite clinical trials. As such, they must withstand changes in membership. Centralization of infrastructure including knowledge management, portfolio management, information management, process automation, work policies, and procedures in clinical research networks facilitates consistency and ultimately research. PURPOSE: In 2005, the National Institute on Drug Abuse (NIDA) CTN transitioned from a distributed data management model to a centralized informatics infrastructure to support the network's trial activities and administration. We describe the centralized informatics infrastructure and discuss our challenges to inform others considering such an endeavor. METHODS: During the migration of a clinical trial network from a decentralized to a centralized data center model, descriptive data were captured and are presented here to assess the impact of centralization. RESULTS: We present the framework for the informatics infrastructure and evaluative metrics. The network has decreased the time from last patient-last visit to database lock from an average of 7.6 months to 2.8 months. The average database error rate decreased from 0.8% to 0.2%, with a corresponding decrease in the interquartile range from 0.04%-1.0% before centralization to 0.01-0.27% after centralization. Centralization has provided the CTN with integrated trial status reporting and the first standards-based public data share. A preliminary cost-benefit analysis showed a 50% reduction in data management cost per study participant over the life of a trial. LIMITATIONS: A single clinical trial network comprising addiction researchers and community treatment programs was assessed. The findings may not be applicable to other research settings. CONCLUSIONS: The identified informatics components provide the information and infrastructure needed for our clinical trial network. Post centralization data management operations are more efficient and less costly, with higher data quality.

Development and evaluation of a study design typology for human research.

A systematic classification of study designs would be useful for researchers, systematic reviewers, readers, and research administrators, among others. As part of the Human Studies Database Project, we developed the Study Design Typology to standardize the classification of study designs in human research. We then performed a multiple observer masked evaluation of active research protocols in four institutions according to a standardized protocol. Thirty-five protocols were classified by three reviewers each into one of nine high-level study designs for interventional and observational research (e.g., N-of-1, Parallel Group, Case Crossover). Rater classification agreement was moderately high for the 35 protocols (Fleiss' kappa = 0.442) and higher still for the 23 quantitative studies (Fleiss' kappa = 0.463). We conclude that our typology shows initial promise for reliably distinguishing study design types for quantitative human research.