Are the Risk of Generalizability Biases Generalizable? A Meta-Epidemiological Study.

Background: Preliminary studies (e.g., pilot/feasibility studies) can result in misleading evidence that an intervention is ready to be evaluated in a large-scale trial when it is not. Risk of Generalizability Biases (RGBs, a set of external validity biases) represent study features that influence estimates of effectiveness, often inflating estimates in preliminary studies which are not replicated in larger-scale trials. While RGBs have been empirically established in interventions targeting obesity, the extent to which RGBs generalize to other health areas is unknown. Understanding the relevance of RGBs across health behavior intervention research can inform organized efforts to reduce their prevalence. Purpose: The purpose of our study was to examine whether RGBs generalize outside of obesity-related interventions. Methods: A systematic review identified health behavior interventions across four behaviors unrelated to obesity that follow a similar intervention development framework of preliminary studies informing larger-scale trials (i.e., tobacco use disorder, alcohol use disorder, interpersonal violence, and behaviors related to increased sexually transmitted infections). To be included, published interventions had to be tested in a preliminary study followed by testing in a larger trial (the two studies thus comprising a study pair). We extracted health-related outcomes and coded the presence/absence of RGBs. We used meta-regression models to estimate the impact of RGBs on the change in standardized mean difference (ΔSMD) between the preliminary study and larger trial. Results: We identified sixty-nine study pairs, of which forty-seven were eligible for inclusion in the analysis (k = 156 effects), with RGBs identified for each behavior. For pairs where the RGB was present in the preliminary study but removed in the larger trial the treatment effect decreased by an average of ΔSMD=-0.38 (range - 0.69 to -0.21). This provides evidence of larger drop in effectiveness for studies containing RGBs relative to study pairs with no RGBs present (treatment effect decreased by an average of ΔSMD =-0.24, range - 0.19 to -0.27). Conclusion: RGBs may be associated with higher effect estimates across diverse areas of health intervention research. These findings suggest commonalities shared across health behavior intervention fields may facilitate introduction of RGBs within preliminary studies, rather than RGBs being isolated to a single health behavior field.

Development and Calibration of a PATCH Device for Monitoring Children's Heart Rate and Acceleration.

INTRODUCTION: Current wearables that collect heart rate and acceleration were not designed for children and/or do not allow access to raw signals, making them fundamentally unverifiable. This study describes the creation and calibration of an open-source multichannel platform (PATCH) designed to measure heart rate and acceleration in children ages 3-8 yr. METHODS: Children (N = 63; mean age, 6.3 yr) participated in a 45-min protocol ranging in intensities from sedentary to vigorous activity. Actiheart-5 was used as a comparison measure. We calculated mean bias, mean absolute error (MAE) mean absolute percent error (MA%E), Pearson correlations, and Lin's concordance correlation coefficient (CCC). RESULTS: Mean bias between PATCH and Actiheart heart rate was 2.26 bpm, MAE was 6.67 bpm, and M%E was 5.99%. The correlation between PATCH and Actiheart heart rate was 0.89, and CCC was 0.88. For acceleration, mean bias was 1.16 mg and MAE was 12.24 mg. The correlation between PATCH and Actiheart was 0.96, and CCC was 0.95. CONCLUSIONS: The PATCH demonstrated clinically acceptable accuracies to measure heart rate and acceleration compared with a research-grade device.

A Sliding Scale Signal Quality Metric of Photoplethysmography Applicable to Measuring Heart Rate across Clinical Contexts with Chest Mounting as a Case Study.

Photoplethysmography (PPG) signal quality as a proxy for accuracy in heart rate (HR) measurement is useful in various public health contexts, ranging from short-term clinical diagnostics to free-living health behavior surveillance studies that inform public health policy. Each context has a different tolerance for acceptable signal quality, and it is reductive to expect a single threshold to meet the needs across all contexts. In this study, we propose two different metrics as sliding scales of PPG signal quality and assess their association with accuracy of HR measures compared to a ground truth electrocardiogram (ECG) measurement. METHODS: We used two publicly available PPG datasets (BUT PPG and Troika) to test if our signal quality metrics could identify poor signal quality compared to gold standard visual inspection. To aid interpretation of the sliding scale metrics, we used ROC curves and Kappa values to calculate guideline cut points and evaluate agreement, respectively. We then used the Troika dataset and an original dataset of PPG data collected from the chest to examine the association between continuous metrics of signal quality and HR accuracy. PPG-based HR estimates were compared with reference HR estimates using the mean absolute error (MAE) and the root-mean-square error (RMSE). Point biserial correlations were used to examine the association between binary signal quality and HR error metrics (MAE and RMSE). RESULTS: ROC analysis from the BUT PPG data revealed that the AUC was 0.758 (95% CI 0.624 to 0.892) for signal quality metrics of STD-width and 0.741 (95% CI 0.589 to 0.883) for self-consistency. There was a significant correlation between criterion poor signal quality and signal quality metrics in both Troika and originally collected data. Signal quality was highly correlated with HR accuracy (MAE and RMSE, respectively) between PPG and ground truth ECG. CONCLUSION: This proof-of-concept work demonstrates an effective approach for assessing signal quality and demonstrates the effect of poor signal quality on HR measurement. Our continuous signal quality metrics allow estimations of uncertainties in other emergent metrics, such as energy expenditure that relies on multiple independent biometrics. This open-source approach increases the availability and applicability of our work in public health settings.

Measuring Microtemporal Processes Underlying Preschoolers' Screen Use and Behavioral Health: Protocol for the Tots and Tech Study.

BACKGROUND: Excessive screen time is associated with poor health and behavioral outcomes in children. However, research on screen time use has been hindered by methodological limitations, including retrospective reports of usual screen time and lack of momentary etiologic processes occurring within each day. OBJECTIVE: This study is designed to assess the feasibility and utility of a comprehensive multibehavior protocol to measure the digital media use and screen time context among a racially and economically diverse sample of preschoolers and their families. This paper describes the recruitment, data collection, and analytical protocols for the Tots and Tech study. METHODS: The Tots and Tech study is a longitudinal, observational study of 100 dyads: caregivers and their preschool-age children (aged 3-5 years). Both caregivers and children will wear an Axivity AX3 accelerometer (Axivity Ltd) for 30 days to assess their physical activity, sedentary behavior, and sleep. Caregivers will complete ecological momentary assessments (EMAs) for 1 week to measure child behavioral problems, caregiver stress, and child screen time. RESULTS: The Tots and Tech study was funded in March 2020. This study maintains rolling recruitment, with each dyad on their own assessment schedule, depending on the time of enrollment. Enrollment was scheduled to take place between September 2020 and May 2022. We aim to enroll 100 caregiver-child dyads. The Tots and Tech outcome paper is expected to be published in 2022. CONCLUSIONS: The Tots and Tech study attempts to overcome previous methodological limitations by using objective measures of screen time, physical activity, sedentary behavior, and sleep behaviors with contextual factors measured by EMA. The results will be used to evaluate the feasibility and utility of a comprehensive multibehavior protocol using objective measures of mobile screen time and accelerometry in conjunction with EMA among caregiver-child dyads. Future observational and intervention studies will be able to use this study protocol to better measure screen time and its context. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/36240.

Feasibility of Measuring Screen Time, Activity, and Context Among Families With Preschoolers: Intensive Longitudinal Pilot Study.

BACKGROUND: Digital media has made screen time more available across multiple contexts, but our understanding of the ways children and families use digital media has lagged behind the rapid adoption of this technology. OBJECTIVE: This study evaluated the feasibility of an intensive longitudinal data collection protocol to objectively measure digital media use, physical activity, sleep, sedentary behavior, and socioemotional context among caregiver-child dyads. This paper also describes preliminary convergent validity of ecological momentary assessment (EMA) measures and preliminary agreement between caregiver self-reported phone use and phone use collected from passive mobile sensing. METHODS: Caregivers and their preschool-aged child (3-5 years) were recruited to complete a 30-day assessment protocol. Within 30-days, caregivers completed 7 days of EMA to measure child behavior problems and caregiver stress. Caregivers and children wore an Axivity AX3 (Newcastle Upon Tyne) accelerometer to assess physical activity, sedentary behavior, and sleep. Phone use was assessed via passive mobile sensing; we used Chronicle for Android users and screenshots of iOS screen time metrics for iOS users. Participants were invited to complete a second 14-day protocol approximately 3-12 months after their first assessment. We used Pearson correlations to examine preliminary convergent validity between validated questionnaire measures of caregiver psychological functioning, child behavior, and EMA items. Root mean square errors were computed to examine the preliminary agreement between caregiver self-reported phone use and objective phone use. RESULTS: Of 110 consenting participants, 105 completed all protocols (105/110, 95.5% retention rate). Compliance was defined a priori as completing ≥70%-75% of each protocol task. There were high compliance rates for passive mobile sensing for both Android (38/40, 95%) and iOS (64/65, 98%). EMA compliance was high (105/105, 100%), but fewer caregivers and children were compliant with accelerometry (62/99, 63% and 40/100, 40%, respectively). Average daily phone use was 383.4 (SD 157.0) minutes for Android users and 354.7 (SD 137.6) minutes for iOS users. There was poor agreement between objective and caregiver self-reported phone use; root mean square errors were 157.1 and 81.4 for Android and iOS users, respectively. Among families who completed the first assessment, 91 re-enrolled to complete the protocol a second time, approximately 7 months later (91/105, 86.7% retention rate). CONCLUSIONS: It is feasible to collect intensive longitudinal data on objective digital media use simultaneously with accelerometry and EMA from an economically and racially diverse sample of families with preschool-aged children. The high compliance and retention of the study sample are encouraging signs that these methods of intensive longitudinal data collection can be completed in a longitudinal cohort study. The lack of agreement between self-reported and objectively measured mobile phone use highlights the need for additional research using objective methods to measure digital media use. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-36240.

Measurement of Physical Activity with Wrist-Worn ActiGraph GT3X+ in Older Women.

Higher wear compliance has been seen with wrist placed accelerometers versus hip placed. Performance of wrist placed ActiGraph GT3X+ accelerometer (GT3X+, ActiGraph LLC, Pensacola, FL) in assessing physical activity (PA) remains unclear. PURPOSE: This study examined GT3X+'s performance in measuring PA energy expenditure (PAEE) and classifying PA intensity in older women. METHODS: Women [n = 89, age = 65.6 (4.3)] wore GT3X+ and SenseWear Armband Mini (SWAM, BodyMedia Inc. Pittsburgh, PA) for 2 weeks. Concurrently, doubly labeled water (DLW) determined total daily energy expenditure (TDEE). Resting energy expenditure (REE) was determined by Indirect Calorimetry. Data was processed using manufacturer-provided software. Bivariate correlations, Intra Class Correlations, and Bland-Altman plots were performed to evaluate agreement between GT3X+ and criterion measures for sedentary time, light and moderate-to-vigorous PA (determined by SWAM) and PAEE (determined by SWAM and by DLW and REE). Epoch-by-epoch analysis evaluated discrepancy and agreement of PA intensity classification between GT3X+ and SWAM. RESULTS: For PAEE, GT3X+ showed moderate correlations with criterion measures (r = 0.413, 0.400 with SWAM; r = 0.564, 0.501 with DLW and REE), but Bland-Altman plots showed large variability. When estimating time spent in PA intensity, GT3X+ underestimated sedentary time and overestimated PA intensity compared to SWAM. During epoch-by-epoch analysis, GT3X+ misclassified light intensity PA as moderate-to-vigorous PA 72% of the time. Counts per minute showed strong correlations with criterion measures (r = 0.68, 0.625 for SWAM and DLW and REE respectively). CONCLUSION: Current equations and cut points do not provide accurate measures of PA with wrist-worn GT3X+ in older women.

Evaluation of attractive toxic sugar bait (ATSB)-Barrier for control of vector and nuisance mosquitoes and its effect on non-target organisms in sub-tropical environments in Florida.

The efficacy of attractive toxic sugar baits (ATSB) with the active ingredient eugenol, an Environmental Protection Agency exempt compound, was evaluated against vector and nuisance mosquitoes in both laboratory and field studies. In the laboratory, eugenol combined in attractive sugar bait (ASB) solution provided high levels of mortality for Aedes aegypti, Culex quinquefasciatus, and Anopheles quadrimaculatus. Field studies demonstrated significant control: >70% reduction for Aedes atlanticus, Aedes. infirmatus, and Culex nigripalpus and >50% reduction for Anopheles crucians, Uranotaenia sapphirina, Culiseta melanura, and Culex erraticus three weeks post ATSB application. Furthermore, non-target feeding of six insect orders, Hymenoptera, Lepidoptera, Coleoptera, Diptera, Hemiptera, and Orthoptera, was evaluated in the field after application of a dyed-ASB to flowering and non-flowering vegetation. ASB feeding (staining) was determined by dissecting the guts and searching for food dye with a dissecting microscope. The potential impact of ATSB on non-targets, applied on green non-flowering vegetation was low for all non-target groups (0.9%). However, application of the ASB to flowering vegetation resulted in significant staining of the non-target insect orders. This highlights the need for application guidelines to reduce non-target effects. No mortality was observed in laboratory studies with predatory non-targets, spiders, praying mantis, or ground beetles, after feeding for three days on mosquitoes engorged on ATSB. Overall, our laboratory and field studies support the use of eugenol as an active ingredient for controlling important vector and nuisance mosquitoes when used as an ATSB toxin. This is the first study demonstrating effective control of anophelines in non-arid environments which suggest that even in highly competitive sugar rich environments this method could be used for control of malaria in Latin American countries.