Enhancing recall in automated record screening: A resampling algorithm.

Literature screening is the process of identifying all relevant records from a pool of candidate paper records in systematic review, meta-analysis, and other research synthesis tasks. This process is time consuming, expensive, and prone to human error. Screening prioritization methods attempt to help reviewers identify most relevant records while only screening a proportion of candidate records with high priority. In previous studies, screening prioritization is often referred to as automatic literature screening or automatic literature identification. Numerous screening prioritization methods have been proposed in recent years. However, there is a lack of screening prioritization methods with reliable performance. Our objective is to develop a screening prioritization algorithm with reliable performance for practical use, for example, an algorithm that guarantees an 80% chance of identifying at least 80 % of the relevant records. Based on a target-based method proposed in Cormack and Grossman, we propose a screening prioritization algorithm using sampling with replacement. The algorithm is a wrapper algorithm that can work with any current screening prioritization algorithm to guarantee the performance. We prove, with mathematics and probability theory, that the algorithm guarantees the performance. We also run numeric experiments to test the performance of our algorithm when applied in practice. The numeric experiment results show this algorithm achieve reliable performance under different circumstances. The proposed screening prioritization algorithm can be reliably used in real world research synthesis tasks.

Why Meta-Analyses of Growth Mindset and Other Interventions Should Follow Best Practices for Examining Heterogeneity: Commentary on Macnamara and Burgoyne (2023) and Burnette et al. (2023).

Meta-analysts often ask a yes-or-no question: Is there an intervention effect or not? This traditional, all-or-nothing thinking stands in contrast with current best practice in meta-analysis, which calls for a heterogeneity-attuned approach (i.e., focused on the extent to which effects vary across procedures, participant groups, or contexts). This heterogeneity-attuned approach allows researchers to understand where effects are weaker or stronger and reveals mechanisms. The current article builds on a rare opportunity to compare two recent meta-analyses that examined the same literature (growth mindset interventions) but used different methods and reached different conclusions. One meta-analysis used a traditional approach (Macnamara and Burgoyne, in press), which aggregated effect sizes for each study before combining them and examined moderators one-by-one by splitting the data into small subgroups. The second meta-analysis (Burnette et al., in press) modeled the variation of effects within studies-across subgroups and outcomes-and applied modern, multi-level meta-regression methods. The former concluded that growth mindset effects are biased, but the latter yielded nuanced conclusions consistent with theoretical predictions. We explain why the practices followed by the latter meta-analysis were more in line with best practices for analyzing large and heterogeneous literatures. Further, an exploratory re-analysis of the data showed that applying the modern, heterogeneity-attuned methods from Burnette et al. (in press) to the dataset employed by Macnamara and Burgoyne (in press) confirmed Burnette et al.'s conclusions; namely, that there was a meaningful, significant effect of growth mindset in focal (at-risk) groups. This article concludes that heterogeneity-attuned meta-analysis is important both for advancing theory and for avoiding the boom-or-bust cycle that plagues too much of psychological science.

Effect sizes in ANCOVA and difference-in-differences designs.

It is common practice in both randomized and quasi-experiments to adjust for baseline characteristics when estimating the average effect of an intervention. The inclusion of a pre-test, for example, can reduce both the standard error of this estimate and-in non-randomized designs-its bias. At the same time, it is also standard to report the effect of an intervention in standardized effect size units, thereby making it comparable to other interventions and studies. Curiously, the estimation of this effect size, including covariate adjustment, has received little attention. In this article, we provide a framework for defining effect sizes in designs with a pre-test (e.g., difference-in-differences and analysis of covariance) and propose estimators of those effect sizes. The estimators and approximations to their sampling distributions are evaluated using a simulation study and then demonstrated using an example from published data.

Accuracy of self-assessment in gastrointestinal endoscopy: a systematic review and meta-analysis.

BACKGROUND: Assessment is necessary to ensure both attainment and maintenance of competency in gastrointestinal (GI) endoscopy, and this can be accomplished through self-assessment. We conducted a systematic review with meta-analysis to evaluate the accuracy of self-assessment among GI endoscopists. METHODS: This was an individual participant data meta-analysis of studies that investigated self-assessment of endoscopic competency. We performed a systematic search of the following databases: Ovid MEDLINE, Ovid EMBASE, Wiley Cochrane CENTRAL, and ProQuest Education Resources Information Center. We included studies if they were primary investigations of self-assessment accuracy in GI endoscopy that used statistical analyses to determine accuracy. We conducted a meta-analysis of studies using a limits of agreement (LoA) approach to meta-analysis of Bland-Altman studies. RESULTS: After removing duplicate entries, we screened 7138 records. After full-text review, we included 16 studies for qualitative analysis and three for meta-analysis. In the meta-analysis, we found that the LoA were wide (-41.0 % to 34.0 %) and beyond the clinically acceptable difference. Subgroup analyses found that both novice and intermediate endoscopists had wide LoA (-45.0 % to 35.1 % and -54.7 % to 46.5 %, respectively) and expert endoscopists had narrow LoA (-14.2 % to 21.4 %). CONCLUSIONS: GI endoscopists are inaccurate in self-assessment of their endoscopic competency. Subgroup analyses demonstrated that novice and intermediate endoscopists were inaccurate, while expert endoscopists have accurate self-assessment. While we advise against the sole use of self-assessment among novice and intermediate endoscopists, expert endoscopists may wish to integrate it into their practice.

Case Report: A Case Study on the Neurodevelopmental Profile of a Child With Pallister-Killian Syndrome and His Unaffected Twin.

Pallister-Killian syndrome is an uncommon genetic disorder that has broad developmental and multisystemic effects. While medical complications are widely reported throughout the literature, research on the neurodevelopmental profile has been limited. Case reports make up the majority of the few existing studies regarding the neurodevelopmental phenotype associated with this disorder. The current case report describes a 3-year-old male with Pallister-Killian syndrome (AF), reports the neurodevelopmental evaluation of his unaffected twin brother (MF), and outlines the results of an optical imaging study on both boys. AF presents with severe developmental delays, however, he ambulates with support and engages in conversation using his communication device. Most severely impaired was AF's speech and expressive language, with childhood apraxia of speech (CAS) as a possible explanation for these severe deficits. MF, the sibling, demonstrated neurotypical abilities and often advanced scores for his age. Both subjects completed a functional near-infrared spectroscopy (fNIRS) study, revealing decreased temporal and frontal lobe function in AF and typical functioning in MF. This case report expands on the existing literature on PKS by describing variances in fraternal twin presentation and novel reporting on fNIRS findings in both boys.

Accuracy and Precision of Energy Expenditure, Heart Rate, and Steps Measured by Combined-Sensing Fitbits Against Reference Measures: Systematic Review and Meta-analysis.

BACKGROUND: Although it is widely recognized that physical activity is an important determinant of health, assessing this complex behavior is a considerable challenge. OBJECTIVE: The purpose of this systematic review and meta-analysis is to examine, quantify, and report the current state of evidence for the validity of energy expenditure, heart rate, and steps measured by recent combined-sensing Fitbits. METHODS: We conducted a systematic review and Bland-Altman meta-analysis of validation studies of combined-sensing Fitbits against reference measures of energy expenditure, heart rate, and steps. RESULTS: A total of 52 studies were included in the systematic review. Among the 52 studies, 41 (79%) were included in the meta-analysis, representing 203 individual comparisons between Fitbit devices and a criterion measure (ie, n=117, 57.6% for heart rate; n=49, 24.1% for energy expenditure; and n=37, 18.2% for steps). Overall, most authors of the included studies concluded that recent Fitbit models underestimate heart rate, energy expenditure, and steps compared with criterion measures. These independent conclusions aligned with the results of the pooled meta-analyses showing an average underestimation of -2.99 beats per minute (k comparison=74), -2.77 kcal per minute (k comparison=29), and -3.11 steps per minute (k comparison=19), respectively, of the Fitbit compared with the criterion measure (results obtained after removing the high risk of bias studies; population limit of agreements for heart rate, energy expenditure, and steps: -23.99 to 18.01, -12.75 to 7.41, and -13.07 to 6.86, respectively). CONCLUSIONS: Fitbit devices are likely to underestimate heart rate, energy expenditure, and steps. The estimation of these measurements varied by the quality of the study, age of the participants, type of activities, and the model of Fitbit. The qualitative conclusions of most studies aligned with the results of the meta-analysis. Although the expected level of accuracy might vary from one context to another, this underestimation can be acceptable, on average, for steps and heart rate. However, the measurement of energy expenditure may be inaccurate for some research purposes.

A machine learning analysis of the relationship of demographics and social gathering attendance from 41 countries during pandemic.

Knowing who to target with certain messages is the prerequisite of efficient public health campaigns during pandemics. Using the COVID-19 pandemic situation, we explored which facets of the society-defined by age, gender, income, and education levels-are the most likely to visit social gatherings and aggravate the spread of a disease. Analyzing the reported behavior of 87,169 individuals from 41 countries, we found that in the majority of the countries, the proportion of social gathering-goers was higher in male than female, younger than older, lower-educated than higher educated, and low-income than high-income subgroups of the populations. However, the data showed noteworthy heterogeneity between the countries warranting against generalizing from one country to another. The analysis also revealed that relative to other demographic factors, income was the strongest predictor of avoidance of social gatherings followed by age, education, and gender. Although the observed strength of these associations was relatively small, we argue that incorporating demographic-based segmentation into public health campaigns can increase the efficiency of campaigns with an important caveat: the exploration of these associations needs to be done on a country level before using the information to target populations in behavior change interventions.

Factors Determining the Agreement between Aerobic Threshold and Point of Maximal Fat Oxidation: Follow-Up on a Systematic Review and Meta-Analysis on Association.

Regular exercise at the intensity matching maximal fat oxidation (FATmax) has been proposed as a key element in both athletes and clinical populations when aiming to enhance the body's ability to oxidize fat. In order to allow a more standardized and tailored training approach, the connection between FATmax and the individual aerobic thresholds (AerT) has been examined. Although recent findings strongly suggest that a relationship exists between these two intensities, correlation alone is not sufficient to confirm that the intensities necessarily coincide and that the error between the two measures is small. Thus, this systematic review and meta-analysis aim to examine the agreement levels between the exercise intensities matching FATmax and AerT by pooling limits of agreement in a function of three parameters: (i) the average difference, (ii) the average within-study variation, and (iii) the variation in bias across studies, and to examine the influence of clinical and methodological inter- and intra-study differences on agreement levels. This study was registered with PROSPERO (CRD42021239351) and ClinicalTrials (NCT03789045). PubMed and Google Scholar were searched for studies examining FATmax and AerT connection. Overall, 12 studies with forty-five effect sizes and a total of 774 subjects fulfilled the inclusion criteria. The ROBIS tool for risk of bias assessment was used to determine the quality of included studies. In conclusion, the overall 95% limits of agreement of the differences between FATmax and AerT exercise intensities were larger than the a priori determined acceptable agreement due to the large variance caused by clinical and methodological differences among the studies. Therefore, we recommend that future studies follow a strict standardization of data collection and analysis of FATmax- and AerT-related outcomes.

Meta-analysis with Robust Variance Estimation: Expanding the Range of Working Models.

In prevention science and related fields, large meta-analyses are common, and these analyses often involve dependent effect size estimates. Robust variance estimation (RVE) methods provide a way to include all dependent effect sizes in a single meta-regression model, even when the exact form of the dependence is unknown. RVE uses a working model of the dependence structure, but the two currently available working models are limited to each describing a single type of dependence. Drawing on flexible tools from multilevel and multivariate meta-analysis, this paper describes an expanded range of working models, along with accompanying estimation methods, which offer potential benefits in terms of better capturing the types of data structures that occur in practice and, under some circumstances, improving the efficiency of meta-regression estimates. We describe how the methods can be implemented using existing software (the "metafor" and "clubSandwich" packages for R), illustrate the proposed approach in a meta-analysis of randomized trials on the effects of brief alcohol interventions for adolescents and young adults, and report findings from a simulation study evaluating the performance of the new methods.

Wellness in the Schools: A Lunch Intervention Increases Fruit and Vegetable Consumption.

Wellness in the Schools (WITS) is a national non-profit organization partnering with public schools to provide healthy, scratch cooked, less processed meals (called an Alternative Menu), and active recess. This study examined the effects of WITS programming on school lunch consumption, including fruit and vegetable intake, in second and third grade students in New York City public schools serving a high proportion of students from low-income households. The intervention was evaluated with a quasi-experimental, controlled design with 14 elementary schools (7 that had initiated WITS programming in fall 2015 and were designated as intervention schools, and 7 matched Control schools). School lunch consumption was assessed by anonymous observation using the System of Observational Cafeteria Assessment of Foods Eaten (SOCAFE) tool in the fall of 2015 (Time 0, early intervention) and the spring of 2016 (Time 1) and 2017 (Time 2). There were no baseline data. Data were also collected on the types of entrées served in the months of October, January, and April during the two school years of the study. Across time points, and relative to students in the Control schools, students in WITS schools ate more fruits and vegetables (units = cups): Time 0: Control 0.18 vs. WITS 0.28; Time 1: Control 0.25 vs. WITS 0.31; and Time 2: Control 0.19 vs. WITS 0.27; p < 0.001. They also had more fruits and vegetables (cups) on their trays, which included more vegetables from the salad bar. However, students in the WITS schools ate fewer entrées (grain and protein) and drank less milk than students in the Control schools. Compared to the Control schools, WITS schools offered more homestyle entrées and fewer finger foods and sandwich entrees, i.e., less processed food. Students in WITS schools who received the Alternative menu and all of the WITS programming at all data collection time points selected and consumed more fruits and vegetables. Replication studies with randomized designs and true baseline data are needed to confirm these findings and to identify avenues for strengthening the effects of the program on other school lunch components.

Behavioural science is unlikely to change the world without a heterogeneity revolution.

In the past decade, behavioural science has gained influence in policymaking but suffered a crisis of confidence in the replicability of its findings. Here, we describe a nascent heterogeneity revolution that we believe these twin historical trends have triggered. This revolution will be defined by the recognition that most treatment effects are heterogeneous, so the variation in effect estimates across studies that defines the replication crisis is to be expected as long as heterogeneous effects are studied without a systematic approach to sampling and moderation. When studied systematically, heterogeneity can be leveraged to build more complete theories of causal mechanism that could inform nuanced and dependable guidance to policymakers. We recommend investment in shared research infrastructure to make it feasible to study behavioural interventions in heterogeneous and generalizable samples, and suggest low-cost steps researchers can take immediately to avoid being misled by heterogeneity and begin to learn from it instead.

Neurocognitive development and capabilities in boys with 49,XXXXY syndrome.

49,XXXXY was previously associated with profound to severe intellectual deficits. However, prior research papers on the cognitive profiles of this population were confounded by small samples sizes, wide age spreads, and incomplete histories of testosterone replacement therapy. This study is the first comprehensive, international investigation of the neurocognitive aspects of 49,XXXXY, and the potential effects of biological treatment on this profile. Sixty-seven boys from infancy to 11 years of age were enrolled in this longitudinal study, with the majority of boys postnatally diagnosed though chromosomal analysis. These boys received a comprehensive neurocognitive evaluation tailored to specific language-based deficits and cognitive challenges. Results revealed higher neurocognitive capacities, both verbally and nonverbally, than previously reported in this disorder. Infant boys with 49,XXXXY who received early hormonal therapy (EHT) had significantly higher scores on the cognitive domain of the Bayley Scales of Infant Development than untreated infants (p = .013). In addition, treated school-aged participants had significantly better scaled scores than untreated boys in form completion (p = .042), a task that requires deductive reasoning, on nonverbal testing on the Leiter International Performance Scales. This study indicates greater cognitive capacities with a wide range of abilities in the child with 49,XXXXY, thus warranting further investigation to identify and understand the critical influences on the etiology and the variability of those capacities.

Accuracy and precision of zero-heat-flux temperature measurements with the 3M™ Bair Hugger™ Temperature Monitoring System: a systematic review and meta-analysis.

Zero-heat-flux thermometers provide clinicians with the ability to continuously and non-invasively monitor body temperature. These devices are increasingly being used to substitute for more invasive core temperature measurements during surgery and in critical care. The aim of this review was to determine the accuracy and precision of zero-heat-flux temperature measurements from the 3M™ Bair Hugger™ Temperature Monitoring System. Medline and EMBASE were searched for studies that reported on a measurement of core or peripheral temperature that coincided with a measurement from the zero-heat-flux device. Study selection and quality assessment was performed independently using the Revised Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2). The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach was used to summarize the strength of the evidence. Pooled estimates of the mean bias and limits of agreement with outer 95% confidence intervals (population limits of agreement) were calculated. Sixteen studies were included. The primary meta-analysis of zero-heat-flux versus core temperature consisted of 22 comparisons from 16 individual studies. Data from 952 participants with 314,137 paired measurements were included. The pooled estimate for the mean bias was 0.03 °C. Population limits of agreement, which take into consideration the between-study heterogeneity and sampling error, were wide, spanning from - 0.93 to 0.98 °C. The GRADE evidence quality rating was downgraded to moderate due to concerns about study limitations. Population limits of agreement for the sensitivity analysis restricted to studies rated as having low risk of bias across all the domains of the QUADAS-2 were similar to the primary analysis. The range of uncertainty in the accuracy of a thermometer should be taken into account when using this device to inform clinical decision-making. Clinicians should therefore consider the potential that a temperature measurement from a 3M™ Bair Hugger™ Temperature Monitoring System could be as much as 1 °C higher or lower than core temperature. Use of this device may not be appropriate in situations where a difference in temperature of less than 1 °C is important to detect.

Measurable reflection in simulation: A pilot study.

PURPOSE: Debriefing is considered a cornerstone in the success of simulated learning practices. For debriefing to be productive and meaningful, reflection of the learners is essential. Measuring reflection during debriefing has proven to be difficult. This study aimed to validate a reflection rubric based on Kolb's Theory of Experiential Learning and integrate the assessment tool into simulation debriefing practice. METHODS: This research was a non-experimental mixed-methods, sequential explanatory design. Validation for the rubric was accomplished by expert content review and an observation process. Students participated in a standardized patient simulation (SPS) portraying a patient with bipolar disorder. The SPS debriefings were recorded and thematically coded to determine the quality and level of reflection of the students according to the following rubric criteria based on Kolb's theory: (1) Concrete Experience, (2) Reflective Observation, (3) Abstract Conceptualism, and (4) Active Experimentation. A descriptive statistical analysis of student responses was applied. RESULTS: The Debriefing Reflection Rubric was successful in determining the level of reflection in the dental hygiene students; further descriptive results were evaluated by a thematic analysis of student responses (N = 173). Student responses demonstrated reflective observations and concrete experiences (n = 112); however, dental hygiene students demonstrated limited responses (n = 61) to abstract conceptualism and connecting their experience to future goals and experimentation. CONCLUSION: The Debriefing Reflection Rubric, based on Kolb's Theory of Experiential Learning, was successful in measuring a learner's level and quality of reflection during a simulation debriefing.

Why Physicians Prescribe Prophylactic Seizure Medications after Intracerebral Hemorrhage: An Adaptive Conjoint Analysis.

BACKGROUND: Seizures are a morbid complication of intracerebral hemorrhage (ICH) and increase the risk for herniation, status epilepticus, and worse patient outcomes. Prophylactic levetiracetam is administered to approximately 40% of patients with ICH. It is unclear which patients are consciously selected for treatment by physicians. We sought to determine how patients are selected for treatment with prophylactic levetiracetam after ICH. METHODS: We administered an adaptive conjoint analysis using decision making software to an NIH Stroke Trials Network Working Group. The adaptive conjoint analysis determines the most influential attributes for making a decision in an iterative, algorithm-driven process. We asked respondents which would most influence a decision to administer prophylactic levetiracetam. The attributes and their levels were taken from published phenotypes associated with prophylactic seizure medications and the likelihood of seizures after ICH: hematoma location (lobar or basal ganglia), hematoma volume (<=10 mL or >10 mL), level of consciousness (Glasgow Coma Scale 5-12 or Glasgow Coma Scale 13-15), age (<65 or ≥65 years), and race (White or Caucasian or Black/African American). The algorithm terminated when the attributes were ranked from most to least influential. RESULTS: The study sample included 27 respondents who completed the adaptive conjoint analysis out of 42 who responded to the survey with a mean age of 43.4 ± 9.4 years. The attribute with the greatest weight was hematoma location (30%), followed by reduced level of consciousness (24%), hematoma volume (19%), race (14%), and age (13%). Ranks of attributes were different (P < .001). CONCLUSIONS: The decision to administer prophylactic levetiracetam to patients with ICH is driven by lobar hematoma location and depressed level of consciousness. Future research on prophylactic seizure medication could focus on patients most likely to receive it.

A review of the intriguing interaction between testosterone and neurocognitive development in males with 47,XXY.

PURPOSE OF REVIEW: Although 47,XXY (Klinefelter syndrome) was first discovered more than 50 years ago, there have been limited comprehensive studies on this disorder. The present review explains the study of neurodevelopmental dysfunction and the impact of testosterone replacement at specific junctions in the life of males with 47,XXY. The intricate relationship between testosterone, neurodevelopment, health, and well being warrants an in-depth investigation in order to achieve optimal outcomes. RECENT FINDINGS: Current literature suggests that the implementation of biological treatment has a positive impact on numerous areas of neurodevelopment. Further research is needed to determine ideal dosage, timing, and frequency of biological treatment for efficacy and safety of the child with 47,XXY. SUMMARY: As noninvasive prenatal screening has detected increasing numbers of fetuses with 47,XXY, parents may benefit from both prenatal and postnatal counseling, including the latest innovative biological treatment, that may further optimize the child's outcome, especially when coupled with targeted early intervention services.

A national experiment reveals where a growth mindset improves achievement.

A global priority for the behavioural sciences is to develop cost-effective, scalable interventions that could improve the academic outcomes of adolescents at a population level, but no such interventions have so far been evaluated in a population-generalizable sample. Here we show that a short (less than one hour), online growth mindset intervention-which teaches that intellectual abilities can be developed-improved grades among lower-achieving students and increased overall enrolment to advanced mathematics courses in a nationally representative sample of students in secondary education in the United States. Notably, the study identified school contexts that sustained the effects of the growth mindset intervention: the intervention changed grades when peer norms aligned with the messages of the intervention. Confidence in the conclusions of this study comes from independent data collection and processing, pre-registration of analyses, and corroboration of results by a blinded Bayesian analysis.

Current practices in meta-regression in psychology, education, and medicine.

Having surveyed the history and methods of meta-regression in a previous paper, in this paper, we review which and how meta-regression methods are applied in recent research syntheses. To do so, we reviewed studies published in 2016 across four leading research synthesis journals: Psychological Bulletin, the Journal of Applied Psychology, Review of Educational Research, and the Cochrane Library. We find that the best practices defined in the previous review are rarely carried out in practice. In light of the identified discrepancies, we consider how to move forward, first by identifying areas where further methods development is needed to address persistent problems in the field and second by discussing how to more effectively disseminate points of methodological consensus.

School Lunch Environmental Factors Impacting Fruit and Vegetable Consumption.

OBJECTIVE: Assess impact of school lunch environmental factors on fruit and vegetable (F&V) consumption in second and third grade students. DESIGN: Cross-sectional observations in 1 school year. PARTICIPANTS: Students from 14 elementary schools in 4 New York City boroughs (n = 877 student-tray observations). MAIN OUTCOME MEASURE(S): Dependent variables were F&V consumption collected by visual observation. Independent variables included school lunch environmental factors, and individual-level and school-level demographics. ANALYSIS: Hierarchical linear modeling was used with F&V consumption as the outcome variable, and relevant independent variables included in each model. RESULTS: Slicing or precutting of fruits and having lunch after recess were positively associated (P < .05) with .163- and .080-cup higher fruit consumption across all students, respectively. Preplating of vegetables on lunch trays, having 2 or more vegetable options, and having lunch after recess were positively associated (P < .05) with .024-, .009-, and .007-cup higher vegetable consumption across all students, respectively. CONCLUSIONS AND IMPLICATIONS: Although there was a small increase in intake, results of the study support that some school lunch environmental factors affect children's F&V consumption, with some factors leading to more impactful increases than others. Slicing of fruits seems most promising in leading to greater fruit consumption and should be further tested.