How do quantitative studies involving people with dementia report experiences of standardised data collection? A narrative synthesis of NIHR published studies.

BACKGROUND: People with dementia are routinely included as research participants in trials and other quantitative studies in which they are invited to respond to standardised measures. This paper reviews the reporting of standardised data collection from people with dementia in reports published in the National Institute for Health and Care Research (NIHR) Journals Library. The aim was to understand how the administration of standardised, self-report measures with people with dementia is reported in NIHR monographs and what could be learnt from this about the feasibility and acceptability of data collection approaches for future studies. METHODS: This was a systematic review with narrative synthesis. Broad search terms (Dementia OR Alzheimer*) were used to search the NIHR Journals Library website in December 2021. All studies that used (or intended to use) standardised measures to collect research data directly from people with dementia were eligible for inclusion. Information was extracted (where reported) on the process of data collection, dementia severity, levels of missing data and the experiences and reflections of those involved. RESULTS: Searches returned 42 records, from which 17 reports were assessed as eligible for inclusion, containing 22 studies. Response rates from participants with dementia in these studies varied considerably and appeared to be related to dementia severity and place of residence. Little information was reported on the process of data collection or the reasons for missing data, and most studies did not report the experiences of participants or those administering the measures. However, there was an indication from two studies that standardised data collection could provoke emotional distress in some participants with dementia. CONCLUSIONS: Through this review we identified both variation in levels of missing data and gaps in reporting which make it difficult to ascertain the reasons for this variation. We also identified potential risks to the well-being of participants with dementia which may be associated with the content of standardised measures and the context of data collection. Open reporting of and reflection upon data collection processes and the experiences of people involved is essential to ensure both the success of future data collection and the wellbeing of study participants. TRIAL REGISTRATION: Registered with Research on Research https://ror-hub.org/study/2905/ .

The tension between ethics and rigor when using Amazon MTurk for eating disorder research: Response to commentaries on Burnette et al. (2021).

We respond to commentaries on our 2021 paper "Concerns and recommendations for using Amazon MTurk for eating disorder research." The commentators raised many thoughtful and nuanced points regarding data validity and ethical means of online data collection. We echo concerns about the ethics of recruiting via platforms such as MTurk, and highlight tensions between recommendations for ethical data collection and ensuring data integrity. Especially, we highlight the consistent finding that MTurk workers display elevated (often remarkably so) rates of psychopathology, and argue such findings merit further scrutiny to ensure both data are valid and workers not exploited.

Measuring All-Cause Mortality With the Census Numident File.

OBJECTIVES: To assess the quality of population-level US mortality data in the US Census Bureau Numerical Identification file (Numident) and describe the details of the mortality information as well as the novel person-level linkages available when using the Census Numident. METHODS: We compared all-cause mortality in the Census Numident to published vital statistics from the Centers for Disease Control and Prevention. We provide detailed information on the linkage of the Census Numident to other Census Bureau survey, administrative, and economic data. RESULTS: Death counts in the Census Numident are similar to those from published mortality vital statistics. Yearly comparisons show that the Census Numident captures more deaths since 1997, and coverage is slightly lower going back in time. Weekly estimates show similar trends from both data sets. CONCLUSIONS: The Census Numident is a high-quality and timely source of data to study all-cause mortality. The Census Bureau makes available a vast and rich set of restricted-use, individual-level data linked to the Census Numident for researchers to use. PUBLIC HEALTH IMPLICATIONS: The Census Numident linked to data available from the Census Bureau provides infrastructure for doing evidence-based public health policy research on mortality.

How COVID-19 lockdown and reopening affected daily steps: evidence based on 164,630 person-days of prospectively collected data from Shanghai, China.

BACKGROUND: COVID-19 lockdowns may lead to physical inactivity, a major risk factor for non-communicable diseases. This study aims to determine: 1) the trajectory in daily step counts before, during and after the lockdown in China, and 2) the characteristics associated with the trajectories. METHODS: From December 2019 to July 2020, smartphone-based step counts were continuously collected in 815 Chinese adults residing in Shanghai over 202 days across three phases: before, during, and after the lockdown. Participant characteristics were reported, and height, weight and body composition measured before the lockdown. A 'sharp' regression discontinuity design with cluster robust standard errors was used to test the effect of the lockdown and reopening on daily steps and a linear mixed model was used to examine the characteristics associated with trajectories during the observed period. RESULTS: Based on 164,630 person-days of data, we found a sharp decline in daily step counts upon the lockdown (24/01/2020) by an average of 3796 (SE = 88) steps, followed by a significant trend of increase by 34 steps/day (SE = 2.5; p < .001) until the end of the lockdown (22/03/2020). This increasing trend continued into the reopening phase at a slower rate of 5 steps per day (SE = 2.3; p = 0.029). Those who were older, married, university educated, insufficiently active, had an 'at risk' body composition, and those in the control group, were slower at recovering step counts during the lockdown, and those who were older, married, without university education and with an 'at risk' body composition recovered step counts at a slower pace after the reopening. CONCLUSIONS: Despite later increases in step counts, COVID-19 lockdown led to a sustained period of reduced physical activity, which may have adverse health implications. Governments and health professionals around the world should continue to encourage and facilitate physical activity during the pandemic.

Barriers and enablers to routine register data collection for newborns and mothers: EN-BIRTH multi-country validation study.

BACKGROUND: Policymakers need regular high-quality coverage data on care around the time of birth to accelerate progress for ending preventable maternal and newborn deaths and stillbirths. With increasing facility births, routine Health Management Information System (HMIS) data have potential to track coverage. Identifying barriers and enablers faced by frontline health workers recording HMIS source data in registers is important to improve data for use. METHODS: The EN-BIRTH study was a mixed-methods observational study in five hospitals in Bangladesh, Nepal and Tanzania to assess measurement validity for selected Every Newborn coverage indicators. We described data elements required in labour ward registers to track these indicators. To evaluate barriers and enablers for correct recording of data in registers, we designed three interview tools: a) semi-structured in-depth interview (IDI) guide b) semi-structured focus group discussion (FGD) guide, and c) checklist assessing care-to-documentation. We interviewed two groups of respondents (January 2018-March 2019): hospital nurse-midwives and doctors who fill ward registers after birth (n = 40 IDI and n = 5 FGD); and data collectors (n = 65). Qualitative data were analysed thematically by categorising pre-identified codes. Common emerging themes of barriers or enablers across all five hospitals were identified relating to three conceptual framework categories. RESULTS: Similar themes emerged as both barriers and enablers. First, register design was recognised as crucial, yet perceived as complex, and not always standardised for necessary data elements. Second, register filling was performed by over-stretched nurse-midwives with variable training, limited supervision, and availability of logistical resources. Documentation complexity across parallel documents was time-consuming and delayed because of low staff numbers. Complete data were valued more than correct data. Third, use of register data included clinical handover and monthly reporting, but little feedback was given from data users. CONCLUSION: Health workers invest major time recording register data for maternal and newborn core health indicators. Improving data quality requires standardised register designs streamlined to capture only necessary data elements. Consistent implementation processes are also needed. Two-way feedback between HMIS levels is critical to improve performance and accurately track progress towards agreed health goals.

Issues with data and analyses: Errors, underlying themes, and potential solutions.

Some aspects of science, taken at the broadest level, are universal in empirical research. These include collecting, analyzing, and reporting data. In each of these aspects, errors can and do occur. In this work, we first discuss the importance of focusing on statistical and data errors to continually improve the practice of science. We then describe underlying themes of the types of errors and postulate contributing factors. To do so, we describe a case series of relatively severe data and statistical errors coupled with surveys of some types of errors to better characterize the magnitude, frequency, and trends. Having examined these errors, we then discuss the consequences of specific errors or classes of errors. Finally, given the extracted themes, we discuss methodological, cultural, and system-level approaches to reducing the frequency of commonly observed errors. These approaches will plausibly contribute to the self-critical, self-correcting, ever-evolving practice of science, and ultimately to furthering knowledge.

Discovering New Trends & Connections: Current Applications of Biomedical Text Mining.

The explosive growth of digital information in recent years has amplified the information overload experienced by today's health-care professionals. In particular, the wide variety of unstructured text makes it difficult for researchers to find meaningful data without spending a considerable amount of time reading. Text mining can be used to facilitate better discoverability and analysis, and aid researchers in identifying critical trends and connections. This column will introduce key text-mining terms, recent use cases of biomedical text mining, and current applications for this technology in medical libraries.

The accuracy of using administrative healthcare data to identify epilepsy cases: A systematic review of validation studies.

Our objective was to undertake a systematic review ascertaining the accuracy of using administrative healthcare data to identify epilepsy cases. We searched MEDLINE and Embase from 01/01/1975 to 03/07/2018 for studies evaluating the diagnostic accuracy of routinely collected healthcare data in identifying epilepsy cases. Any disease coding system in use since the International Classification of Diseases, Ninth Revision (ICD-9) was permissible. Two authors independently screened studies, extracted data, and quality-assessed studies. We assessed positive predictive value (PPV), sensitivity, negative predictive value (NPV), and specificity. The primary analysis was a narrative synthesis of review findings. Thirty studies were included, published between 1989 and 2018. Risks of bias were low, high, and unclear in 4, 14, and 12 studies, respectively. Coding systems included ICD-9, ICD-10, and Read Codes, with or without antiepileptic drugs (AEDs). PPVs included ranges of 5.2%-100% (Canada), 32.7%-96.0% (USA), 47.0%-100% (UK), and 37.0%-88.0% (Norway). Sensitivities included ranges of 22.2%-99.7% (Canada), 12.2%-97.3% (USA), and 79.0%-94.0% (UK). Nineteen studies contained at least one algorithm with a PPV >80%. Sixteen studies contained at least one algorithm with a sensitivity >80%. PPV was highest in algorithms consisting of disease codes (ICD-10 G40-41, ICD-9 345) in combination with one or more AEDs. The addition of symptom codes to this (ICD-10 R56; ICD-9 780.3, 780.39) lowered PPV. Sensitivity was highest in algorithms consisting of symptom codes with one or more AEDs. Although using AEDs alone achieved high sensitivities, the associated PPVs were low. Most NPVs and specificities were >90%. We conclude that it is reasonable to use administrative data to identify people with epilepsy (PWE) in epidemiological research. Studies prioritizing high PPVs should focus on combining disease codes with AEDs. Studies prioritizing high sensitivities should focus on combining symptom codes with AEDs. We caution against the use of AEDs alone to identify PWE.

Can cancer researchers accurately judge whether preclinical reports will reproduce?

There is vigorous debate about the reproducibility of research findings in cancer biology. Whether scientists can accurately assess which experiments will reproduce original findings is important to determining the pace at which science self-corrects. We collected forecasts from basic and preclinical cancer researchers on the first 6 replication studies conducted by the Reproducibility Project: Cancer Biology (RP:CB) to assess the accuracy of expert judgments on specific replication outcomes. On average, researchers forecasted a 75% probability of replicating the statistical significance and a 50% probability of replicating the effect size, yet none of these studies successfully replicated on either criterion (for the 5 studies with results reported). Accuracy was related to expertise: experts with higher h-indices were more accurate, whereas experts with more topic-specific expertise were less accurate. Our findings suggest that experts, especially those with specialized knowledge, were overconfident about the RP:CB replicating individual experiments within published reports; researcher optimism likely reflects a combination of overestimating the validity of original studies and underestimating the difficulties of repeating their methodologies.

The association between interpregnancy interval and severe maternal morbidities using revised national birth certificate data: A probabilistic bias analysis.

BACKGROUND: Severe maternal morbidity continues to be on the rise in the US. Short birth spacing is a modifiable risk factor associated with maternal morbidity, yet few studies have examined this association, possibly due to few available data sources to examine these rare events. OBJECTIVE: To examine the association between interpregnancy interval (IPI) and severe maternal morbidity using near-national birth certificate data and account for known under-reporting using probabilistic bias analysis. METHODS: We used revised 2014-2017 birth certificate data, restricting to resident women with a non-first-born singleton birth. We examined the following: (a) maternal blood transfusion, (b) admission to intensive care unit (ICU), (c) uterine rupture (among women with a prior caesarean delivery) and (d) third- or fourth-degree perineal laceration (among vaginal deliveries) by IPI categories (<6, 6-11, 12-17, 18-23, 24-59 and 60+ months). Risk ratios and 95% confidence intervals were estimated using log-binomial regression, adjusting for select maternal characteristics. Probabilistic bias analyses were performed. RESULTS: Compared with IPI 18 to 23 months, adjusted models revealed that the risk of maternal transfusion followed a U-shaped curve with IPI, while risk of ICU admission and perineal laceration increased with longer IPI. Risk of uterine rupture was highest among IPI <6 months. With the exception of maternal transfusion, these findings persisted regardless of the extent or type of misclassification examined in bias analyses. CONCLUSIONS: Associations between IPI and maternal morbidity varied by outcome, even after adjusting for misclassification of SMM. Differences across maternal health outcomes should be considered when counselling and making recommendations regarding optimal birth spacing.

Abortion Reporting in the United States: An Assessment of Three National Fertility Surveys.

Despite its frequency, abortion remains a highly sensitive, stigmatized, and difficult-to-measure behavior. We present estimates of abortion underreporting for three of the most commonly used national fertility surveys in the United States: the National Survey of Family Growth, the National Longitudinal Survey of Youth 1997, and the National Longitudinal Study of Adolescent to Adult Health. Numbers of abortions reported in each survey were compared with external abortion counts obtained from a census of all U.S. abortion providers, with adjustments for comparable respondent ages and periods of each data source. We examined the influence of survey design factors, including survey mode, sampling frame, and length of recall, on abortion underreporting. We used Monte Carlo simulations to estimate potential measurement biases in relationships between abortion and other variables. Underreporting of abortion in the United States compromises the ability to study abortion-and, consequently, almost any pregnancy-related experience-using national fertility surveys.

Resolving the Tension Between Exploration and Confirmation in Preclinical Biomedical Research.

Confirmation through competent replication is a founding principle of modern science. However, biomedical researchers are rewarded for innovation, and not for confirmation, and confirmatory research is often stigmatized as unoriginal and as a consequence faces barriers to publication. As a result, the current biomedical literature is dominated by exploration, which to complicate matters further is often disguised as confirmation. Only recently scientists and the public have begun to realize that high-profile research results in biomedicine can often not be replicated. Consequently, confirmation has become central stage in the quest to safeguard the robustness of research findings. Research which is pushing the boundaries of or challenges what is currently known must necessarily result in a plethora of false positive results. Thus, since discovery, the driving force of scientific progress, is unavoidably linked to high false positive rates and cannot support confirmatory inference, dedicated confirmatory investigation is needed for pivotal results. In this chapter I will argue that the tension between the two modes of research, exploration and confirmation, can be resolved if we conceptually and practically separate them. I will discuss the idiosyncrasies of exploratory and confirmatory studies, with a focus on the specific features of their design, analysis, and interpretation.

An Approach to Addressing Multiple Imputation Model Uncertainty Using Bayesian Model Averaging.

This paper considers the problem of imputation model uncertainty in the context of missing data problems. We argue that so-called "Bayesianly proper" approaches to multiple imputation, although correctly accounting for uncertainty in imputation model parameters, ignore the uncertainty in the imputation model itself. We address imputation model uncertainty by implementing Bayesian model averaging as part of the imputation process. Bayesian model averaging accounts for both model and parameter uncertainty, and thus we argue is fully Bayesianly proper. We apply Bayesian model averaging to multiple imputation under the fully conditional specification approach. An extensive simulation study is conducted comparing our Bayesian model averaging approach against normal theory-based Bayesian imputation not accounting for model uncertainty. Across almost all conditions of the simulation study, the results reveal the extent of model uncertainty in multiple imputation and a consistent advantage to our Bayesian model averaging approach over normal-theory multiple imputation under missing-at-random and missing-completely-at random in terms of Kullback-Liebler divergence and mean squared prediction error. A small case study is also presented. Directions for future research are discussed.

Causes of variability in latent phenotypes of childhood wheeze.

BACKGROUND: Latent class analysis (LCA) has been used extensively to identify (latent) phenotypes of childhood wheezing. However, the number and trajectory of discovered phenotypes differed substantially between studies. OBJECTIVE: We sought to investigate sources of variability affecting the classification of phenotypes, identify key time points for data collection to understand wheeze heterogeneity, and ascertain the association of childhood wheeze phenotypes with asthma and lung function in adulthood. METHODS: We used LCA to derive wheeze phenotypes among 3167 participants in the ALSPAC cohort who had complete information on current wheeze recorded at 14 time points from birth to age 16½ years. We examined the effects of sample size and data collection age and intervals on the results and identified time points. We examined the associations of derived phenotypes with asthma and lung function at age 23 to 24 years. RESULTS: A relatively large sample size (>2000) underestimated the number of phenotypes under some conditions (eg, number of time points <11). Increasing the number of data points resulted in an increase in the optimal number of phenotypes, but an identical number of randomly selected follow-up points led to different solutions. A variable selection algorithm identified 8 informative time points (months 18, 42, 57, 81, 91, 140, 157, and 166). The proportion of asthmatic patients at age 23 to 24 years differed between phenotypes, whereas lung function was lower among persistent wheezers. CONCLUSIONS: Sample size, frequency, and timing of data collection have a major influence on the number and type of wheeze phenotypes identified by using LCA in longitudinal data.

Nurses' Perceptions of Implant Barcode Scanning in Surgical Services.

The US Food and Drug Administration's 2013 Unique Device Identification System Rule requires manufacturers to label devices with unique identifiers. Implantable devices are now shipped with unique identifiers, and many electronic health records have fields to incorporate them. Health policy changes have prompted hospital systems to assess implementation of implant barcode scanning systems to capture unique device identifiers. Project aims were to assess predictors of operating room nurses' acceptance of a new implant barcode scanning system, describe operating room nurses' perceptions of the system value, and identify operating room nurses' perceived gaps in system implementation. An online survey was disseminated to operating room nurses, and focus groups were conducted with orthopedic operating room nurses in an academic medical center that had recently implemented an implant barcode scanning system in surgical services. Predictors of barcode scanning acceptance included perceived usefulness for patient care, perceived ease of use, and perceived usefulness (self). Nurses perceived the system to be more accurate and valuable for patient safety. Perceived gaps in system implementation related to communication, completeness of the system, consistency in process, and training. Understanding nurse perceptions of new barcode scanning systems and engaging them in the implementation process are key areas for success and optimization of these systems.

Improving Prediction of Fall Risk Using Electronic Health Record Data With Various Types and Sources at Multiple Times.

Inpatient falls are among the most common adverse events threatening patient safety. Although many studies have developed predictive models for fall risk, there are some drawbacks. First, most previous studies have relied on an incident-reporting system alone to identify fall events. Thus, it has been found that falls are more likely to be underreported. Second, there has been a controversy on how to select accurate representative values for patient status data across multiple times and various data sources in electronic health records. Given this background, this study used nurses' progress notes as a complementary data source to detect fall events. In addition, we developed criteria including coverage, currency, and granularity in order to integrate electronic health records data documented at multiple times in various data types and sources. Based on this methodology, we developed three models, logistic regression, Cox proportional hazard regression, and decision tree, to predict risk of patient falls and evaluate the predictive performance of these models by comparing the results to results from the Hendrich II Fall Risk Model. The findings of this study will be used in a clinical decision support system to predict risk of falling and provide evidence-based tailored recommendations in the future.

Methods and Processes Used to Collect Pressure Injury Risk and Prevention Measures in the National Database of Nursing Quality Indicators® (NDNQI®).

BACKGROUND: Previous research found that reliability estimates for chart-extracted quality of care data vary. PURPOSE: The purpose was to examine methods and processes used to gather data on the National Database of Nursing Quality Indicators (NDNQI) pressure injury (PI) risk and prevention measures to identify factors that may influence their reliability. METHODS: Study participants (N = 120) from 36 hospitals completed a 35-item online survey. Included were the NDNQI PI Survey Team member with the most experience and/or skill in patient record review from each hospital (n = 36) and 84 other NDNQI PI Survey Team members. RESULTS: In general, participants followed NDNQI PI data collection guidelines. However, deviations were noted such as 60 (50%) participants collected PI data on units where they work, and 92 (76.7%) determined whether moisture management was performed by direct observation of patients rather than chart documentation. CONCLUSIONS: Findings provide insight on how to improve the reliability of hospital-acquired PI risk and prevention measures that includes clarification of the data collection guidelines.

The data deficit for asthma emergency presentations might surprise you: how RAHDaR addresses the data chasm.

INTRODUCTION: National and state-based minimum data sets remain inadequate in providing a complete representation of emergency presentations, especially among paediatric asthma presentations. Thus, the aim of the study was to identify if a deficit exists in current emergency paediatric asthma hospital presentation datasets and how this may inform an understanding of childhood asthma in Victoria Methods: This retrospective cross-sectional study examined emergency hospital presentation data between 1 February 2017 and 31 January 2019. All paediatric (0-14 years) emergency asthma presentation data were collected from nine hospitals in south-western Victoria, Australia, using the Rural Acute Hospital Data Register (RAHDaR), which gathers both Victorian Emergency Minimum Dataset (VEMD) data from larger government hospitals, and non-VEMD data from smaller, more rural institutions. RESULTS: Of the 854 emergency presentations identified for children with asthma aged 0-14 years, 540 (63.2%) were managed initially at larger government-reporting hospitals. A total of 314 (36.8%) emergency presentations were initially managed at emergency facilities, such as urgent care centres. Overall, it was found that a total 278 (32.5%) of all emergency presentations did not appear in current government datasets. CONCLUSION: The RAHDaR database, a complete register of data, captures all emergency presentations in south-western Victoria and highlights as much as a 33% deficit in the data currently available to inform asthma resource initiatives including policy development, funding allocation, prevention and management initiatives in Victoria. More accurate data from sources such as RAHDaR are essential to fill the now-evident data chasm.

The nuts and bolts of evaluating science communication activities.

Since 2008 there has been a focus on fostering a culture of public engagement in higher education plus an impact agenda that demands scientists provide evidence of how their work, including their science communication, is making a difference. Good science communication takes a significant amount of time to plan and deliver so how can you improve what you are doing and demonstrate if you are having an impact? The answer is to evaluate. Effective evaluation needs to be planned so this paper takes you step by step through the evaluation process, illustrated using specific examples.