Going virtual: mixed methods evaluation of online versus in-person learning in the NIH mixed methods research training program retreat.

BACKGROUND: Despite the central role of mixed methods in health research, studies evaluating online methods training in the health sciences are nonexistent. The focused goal was to evaluate online training by comparing the self-rated skills of scholars who experienced an in-person retreat to scholars in an online retreat in specific domains of mixed methods research for the health sciences from 2015-2023. METHODS: The authors administered a scholar Mixed Methods Skills Self-Assessment instrument based on an educational competency scale that included domains on: "research questions," "design/approach," "sampling," "analysis," and "dissemination" to participants of the Mixed Methods Research Training Program for the Health Sciences (MMRTP). Self-ratings on confidence on domains were compared before and after retreat participation within cohorts who attended in person (n = 73) or online (n = 57) as well as comparing across in-person to online cohorts. Responses to open-ended questions about experiences with the retreat were analyzed. RESULTS: Scholars in an interactive program to improve mixed methods skills reported significantly increased confidence in ability to define or explain concepts and in ability to apply the concepts to practical problems, whether the program was attended in-person or synchronously online. Scholars in the online retreat had self-rated skill improvements as good or better than scholars who participated in person. With the possible exception of networking, scholars found the online format was associated with advantages such as accessibility and reduced burden of travel and finding childcare. No differences in difficulty of learning concepts was described. CONCLUSIONS: Keeping in mind that the retreat is only one component of the MMRTP, this study provides evidence that mixed methods training online was associated with the same increases in self-rated skills as persons attending online and can be a key component to increasing the capacity for mixed methods research in the health sciences.

Network-Based Delivery and Sustainment of Evidence-Based Prevention in Community-Clinical Partnerships Addressing Health Equity: A Qualitative Exploration.

Background: Increased delivery of evidence-based preventive services can improve population health and increase health equity. Community-clinical partnerships offer particular promise, but delivery and sustainment of preventive services through these systems face several challenges related to service integration and collaboration. We used a social network analysis perspective to explore (a) the range of contributions made by community-clinical partnership network members to support the delivery of evidence-based preventive services and (b) important influences on the ability of these partnerships to sustain service delivery. Methods: Data come from an implementation evaluation of the Prevention and Wellness Trust Fund initiative, which supported nine Massachusetts communities to coordinate delivery of evidence-based prevention and address inequities in hypertension, pediatric asthma, falls among older adults, or tobacco use. In 2016, we conducted semi-structured interviews with (a) leadership teams representing nine community-level partnerships and (b) practitioners from four high-implementation partnerships (n = 23). We managed data using NVivo11 and utilized a framework analysis approach. Results: Key network contributions for delivery of evidence-based preventive services included creating referrals, delivering services, providing links to community members, and administration and leadership. Less emphasized contributions included wraparound services, technical assistance, and venue provision. Implementers from high-implementation partnerships also highlighted contributions such as program adaptation, creating buy-in, and sharing information to improve service delivery. Expected drivers of program sustainability included the ability to develop a business case, ongoing network facilitation, technology support, continued integrated action, and sufficient staffing to maintain programming. Conclusion: The study highlights the need to take a long-term, infrastructure-focused approach when designing community-clinical partnerships. Strategic partnership composition, including identifying sources of necessary network contributions, in conjunction with efforts from the outset to link systems, align effort, and build a long-term funding structure can support the required coordinated action around preventive services needed to improve health equity.

Cold-related Florida manatee mortality in relation to air and water temperatures.

Many tropical and subtropical species are sensitive to sudden temperature changes, especially drops in temperature. During winters 2009-2010 and 2010-2011, unusually cold temperatures occurred in many parts of Florida, USA, resulting in increased mortality of Florida manatees, sea turtles, fish, corals, and other species. The Florida manatee, in particular, is highly susceptible to cold stress and death when water temperatures drop below 20°C. We sought to characterize the magnitude and timing of reports of cold-related manatee carcasses in relation to fluctuations in water and air temperatures in central-east and central-west Florida during the six winters from 2008 to 2014. We used a generalized linear model to predict counts of manatee carcasses with a cold-related cause of death reported over 7-day bins in relation to various short-term (two weeks or less) and cumulative (incrementally summed from the start of the winter) heating-degree-day effects (HDD; < 20°C) and a categorical winter variable. Using water temperature data, the top-ranked model in both regions included a short-term temperature effect (14-day HDD sum) that preceded increases in reports of cold-related manatee carcasses by 7 days. Cumulative exposure to cold weather over the winter amplified effects on mortality in the central-east region. Quantifying the relationship between cold events and manatee mortality helps us prepare for rescue and salvage operations when extremely cold weather is forecast. This is especially important because anticipated loss or degradation of warm-water refuges due to human activities and sea level rise could potentially impact the manatee population in the future. These methods could also be applied to other species susceptible to cold-related mortality.

Mixed methods grant applications in the health sciences: An analysis of reviewer comments.

Our aim was to understand how reviewers appraise mixed methods research by analyzing reviewer comments for grant applications submitted primarily to the National Institutes of Health. We requested scholars and consultants in the Mixed Methods Research Training Program (MMRTP) for the Health Sciences to send us summary statements from their mixed methods grant applications and obtained 40 summary statements of funded (40%) and unfunded (60%) mixed methods grant applications. We conducted a document analysis using a coding rubric based on the NIH Best Practices for Mixed Methods Research in the Health Sciences and allowed inductive codes to emerge. Reviewers favorably appraised mixed methods applications demonstrating coherence among aims and research design elements, detailed methods, plans for mixed methods integration, and the use of theoretical models. Reviewers identified weaknesses in mixed methods applications that lacked methodological details or rationales, had a high participant burden, and failed to delineate investigator roles. Successful mixed methods applications convey assumptions behind the methods chosen to accomplish specific aims and clearly detail the procedures to be taken. Investigators planning to use mixed methods should remember that reviewers are looking for both points of view.

Accuracy, precision, and error in age estimation of Florida manatees using growth layer groups in earbones.

Ages of Florida manatees (Trichechus manatus latirostris) can be estimated by counting annual growth layer groups (GLGs) in the periotic dome portion of the tympanoperiotic complex of their earbones. The Florida Fish and Wildlife Conservation Commission manages an archive of more than 8,700 Florida manatee earbones collected from salvaged carcasses from 1989 to 2017. Our goal was to comprehensively evaluate techniques used to estimate age, given this large sample size and changes to processing protocols and earbone readers over time. We developed new standards for estimating ages from earbones, involving two independent readers to obtain measurements of within- and between-reader precision. To quantify accuracy, precision, and error, 111 earbones from manatees with approximately known ages (first known as calves: "KAC") and 69 earbones from manatees with minimum known ages ("MKA," based on photo-identification sighting histories) were processed, and their ages were estimated. There was greater precision within readers (coefficient of variation, CV: 2.4-8.5%) than between readers (CV: 13.1-13.3%). The median of age estimates fell within the true age range for 63.1% of KAC cases and was at least the sighting duration for 75.0% of MKA cases. Age estimates were generally unbiased, as indicated by an average raw error ± SD of -0.05 ± 3.05 years for the KAC group. The absolute error (i.e., absolute value of raw error) of the KAC data set averaged 1.75 ± 2.50 years. Accuracy decreased and error increased with increasing known age, especially for animals over 15 years old, whose ages were mostly underestimated due to increasing levels of resorption (the process of bone turnover that obscures GLGs). Understanding the degree of uncertainty in age estimates will help us assess the utility of age data in manatee population models. We emphasize the importance of standardizing and routinely reviewing age estimation and processing protocols to ensure that age data remain consistent and reliable.

Process Evaluation of a Retreat for Scholars in the First Cohort: The NIH Mixed Methods Research Training Program for the Health Sciences.

The Mixed Methods Research Training Program for the Health Sciences aims to enlarge the national pool of trained investigators in mixed methods and improve the quality of grant applications to the NIH. Selected scholars are assigned a consulting team, participate in webinars, and attend an annual "retreat" focused on learning mixed methods through application to their research. Our paper summarizes the process evaluation of the retreat. Scholars identified strengths in small interactive groups to discuss individual projects and the opportunity to apply learning. Scholars wanted further opportunity to discuss individual projects, understanding interventions and mixed methods, and finding collaborators. Our findings will be useful to leaders developing workshops or similar programs at the faculty level.

A Mixed Methods Approach to Evaluate Partnerships and Implementation of the Massachusetts Prevention and Wellness Trust Fund.

Background: Strong partnerships are critical to integrate evidence-based prevention interventions within clinical and community-based settings, offering multilevel and sustainable solutions to complex health issues. As part of Massachusetts' 2012 health reform, The Prevention and Wellness Trust Fund (PWTF) funded nine local partnerships throughout the state to address hypertension, pediatric asthma, falls among older adults, and tobacco use. The initiative was designed to improve health outcomes through prevention and disease management strategies and reduce healthcare costs. Purpose: Describe the mixed-methods study design for investigating PWTF implementation. Methods: The Consolidated Framework for Implementation Research guided the development of this evaluation. First, the study team conducted semi-structured qualitative interviews with leaders from each of nine partnerships to document partnership development and function, intervention adaptation and delivery, and the influence of contextual factors on implementation. The interview findings were used to develop a quantitative survey to assess the implementation experiences of 172 staff from clinical and community-based settings and a social network analysis to assess changes in the relationships among 72 PWTF partner organizations. The quantitative survey data on ratings of perceived implementation success were used to purposively select 24 staff for interviews to explore the most successful experiences of implementing evidence-based interventions for each of the four conditions. Conclusions: This mixed-methods approach for evaluation of implementation of evidence-based prevention interventions by PWTF partnerships can help decision-makers set future priorities for implementing and assessing clinical-community partnerships focused on prevention.

Skills Development and Academic Productivity of Scholars in the NIH Mixed Methods Research Training Program for the Health Sciences.

The Mixed Methods Research Training Program (MMRTP) for the Health Sciences is a mentoring-based program to train faculty in mixed methods research. We administered a Mixed Methods Skills Self-Assessment instrument with domains of "research questions," "design/approach," "sampling," "analysis," and "dissemination." For each item (i.e., skill), we requested three ratings on a 5-point Likert-format scale drawn from an educational competency ratings scale: "My ability to define/explain," "My ability to apply to practical problems," and "Extent to which I need to improve my skill." To assess productivity, we administered a survey with questions related to grants funded, grants submitted, publications, presentations, instances of serving as an institutional resource for mixed methods, and other comments. The results showed that 29 scholars in the first two cohorts represented a diverse set of disciplines and research topics. Although scholars expressed a strong interest in learning mixed methods skills, they came into the program with limited professional experiences with mixed methods. Scholars reported statistically significantly increased confidence in ability to define or explain concepts and in ability to apply the concepts to practical problems. Only practical applications of case studies and ethical principles of research did not show statistically significant improvement after the retreat. Scholars reported substantial productivity in mixed methods and described leadership in mixed methods at their institutions. Participation in an interactive program statistically significantly improved the confidence of scholars. The MMRTP holds promise to bridge the gap between complex research questions in the health sciences and investigators suitably trained in mixed methods.

Development of a Self-Rated Mixed Methods Skills Assessment: The National Institutes of Health Mixed Methods Research Training Program for the Health Sciences.

INTRODUCTION: Demand for training in mixed methods is high, with little research on faculty development or assessment in mixed methods. We describe the development of a self-rated mixed methods skills assessment and provide validity evidence. The instrument taps six research domains: "Research question," "Design/approach," "Sampling," "Data collection," "Analysis," and "Dissemination." Respondents are asked to rate their ability to define or explain concepts of mixed methods under each domain, their ability to apply the concepts to problems, and the extent to which they need to improve. METHODS: We administered the questionnaire to 145 faculty and students using an internet survey. We analyzed descriptive statistics and performance characteristics of the questionnaire using the Cronbach alpha to assess reliability and an analysis of variance that compared a mixed methods experience index with assessment scores to assess criterion relatedness. RESULTS: Internal consistency reliability was high for the total set of items (0.95) and adequate (≥0.71) for all but one subscale. Consistent with establishing criterion validity, respondents who had more professional experiences with mixed methods (eg, published a mixed methods article) rated themselves as more skilled, which was statistically significant across the research domains. DISCUSSION: This self-rated mixed methods assessment instrument may be a useful tool to assess skills in mixed methods for training programs. It can be applied widely at the graduate and faculty level. For the learner, assessment may lead to enhanced motivation to learn and training focused on self-identified needs. For faculty, the assessment may improve curriculum and course content planning.

Ensuring an optimal environment for peer education in South African schools: Goals, systems, standards and policy options for effective learning.

Peer education has long been seen as a key health promotion strategy and an important tool in preventing HIV infection. In South African schools, it is currently one of the strategies employed to do so. Based on both a recent research study of peer education across 35 schools and drawing on multiple previous studies in South Africa, this paper examines the key elements of peer education that contribute to its effectiveness and asks how this aligns with current educational and health policies. From this research, it summarises and proposes shared goals and aims, minimum standards of implementation and reflects on the necessary infrastructure required for peer education to be effective. In light of these findings, it offers policy recommendations regarding who should be doing peer education and the status peer education should have in a school's formal programme.

Influence of Manatees' Diving on Their Risk of Collision with Watercraft.

Watercraft pose a threat to endangered Florida manatees (Trichechus manatus latirostris). Mortality from watercraft collisions has adversely impacted the manatee population's growth rate, therefore reducing this threat is an important management goal. To assess factors that contribute to the risk of watercraft strikes to manatees, we studied the diving behavior of nine manatees carrying GPS tags and time-depth recorders in Tampa Bay, Florida, during winters 2002-2006. We applied a Bayesian formulation of generalized linear mixed models to depth data to model the probability (Pt) that manatees would be no deeper than 1.25 m from the water's surface as a function of behavioral and habitat covariates. Manatees above this threshold were considered to be within striking depth of a watercraft. Seventy-eight percent of depth records (individual range 62-86%) were within striking depth (mean = 1.09 m, max = 16.20 m), illustrating how vulnerable manatees are to strikes. In some circumstances manatees made consecutive dives to the bottom while traveling, even in areas >14 m, possibly to conserve energy. This is the first documentation of potential cost-efficient diving behavior in manatees. Manatees were at higher risk of being within striking depth in shallow water (<0.91 m), over seagrass, at night, and while stationary or moving slowly; they were less likely to be within striking depth when ≤50 m from a charted waterway. In shallow water the probability of a manatee being within striking depth was 0.96 (CI = 0.93-0.98) and decreased as water depth increased. The probability was greater over seagrass (Pt = 0.96, CI = 0.93-0.98) than over other substrates (Pt = 0.73, CI = 0.58-0.84). Quantitative approaches to assessing risk can improve the effectiveness of manatee conservation measures by helping identify areas for protection.

Measuring change in vulnerable adolescents: findings from a peer education evaluation in South Africa.

INTRODUCTION: In the context of poverty and HIV and AIDS, peer education is thought to be capable of providing vulnerable youth with psychosocial support as well as information and decision-making skills otherwise limited by scarce social and material resources. As a preventative education intervention method, peer education is a strategy aimed at norms and peer group influences that affect health behaviours and attitudes. However, too few evaluations of peer-led programmes are available, and they frequently fail to reflect real differences between those who have been recipients of peer education and those who have not. This article reports on an evaluation of a pilot peer-led intervention, entitled Vhutshilo, implemented on principles agreed upon through a collaborative effort in South Africa by the Harvard School of Public Health and the Centre for the Support of Peer Education (the Rutanang collaboration). Vhutshilo targeted vulnerable adolescents aged 14-16 years living in some of South Africa's under-resourced communities. METHODOLOGY: The research design was a mixed-method (qualitative and quantitative), longitudinal, quasi-experimental evaluation. Tools used included a quantitative survey questionnaire (n = 183) and semi-structured interviews (n = 32) with beneficiaries of peer education. Surveys were administered twice for beneficiaries of peer education (n = 73), immediately after completion of the programme (post-test) and 4 months later (delayed post-test), and once for control group members (n = 110). The three main methodological limitations in this study were the use of a once-off control group assessment as the baseline for comparison, without a pre-test, due to timing and resource constraints; a small sample size (n = 183), which reduced the statistical power of the evaluation; and the unavailability of existing tested survey questions to measure the impact of peer education and its role in behaviour change. FINDINGS: This article reports on the difficulties of designing a comprehensive evaluation within time and financial constraints, critically evaluates survey design with multi-item indicators, and discusses six statistically significant changes observed in Vhutshilo participants out of a 92-point survey. Youth struggling with poor quality education and living in economically fraught contexts with little social support, nonetheless, showed evidence of having greater knowledge of support networks and an expanded emotional repertoire by the end of the Vhutshilo programme, and 4 months later. At both individual and group level, many with low socio-economic status showed great improvement with regard to programme indicator scores. CONCLUSION: For the poorest adolescents, especially those living in the rural parts of South Africa, peer education has the potential to change future orientation, attitudes and knowledge regarding HIV and AIDS, including an intolerance for multiple concurrent partnerships. When well organised and properly supported, peer education programmes (and the Vhutshilo curriculum, in particular) provide vulnerable youth with opportunities to develop psychosocial skills and informational resources that contribute to the changing of norms, attitudes and behaviours. However, the article also flags the need for effective peer education evaluations that take into account limited financial resources and that possess tested indicators of programme effectiveness.

Advancing the science of community-level interventions.

Community interventions are complex social processes that need to move beyond single interventions and outcomes at individual levels of short-term change. A scientific paradigm is emerging that supports collaborative, multilevel, culturally situated community interventions aimed at creating sustainable community-level impact. This paradigm is rooted in a deep history of ecological and collaborative thinking across public health, psychology, anthropology, and other fields of social science. The new paradigm makes a number of primary assertions that affect conceptualization of health issues, intervention design, and intervention evaluation. To elaborate the paradigm and advance the science of community intervention, we offer suggestions for promoting a scientific agenda, developing collaborations among professionals and communities, and examining the culture of science.

Evaluation of adrenocortical function in Florida manatees (Trichechus manatus latirostris).

The study objectives were to determine the predominant manatee glucocorticoid; validate assays to measure this glucocorticoid and adrenocorticotropic hormone (ACTH); determine diagnostic thresholds to distinguish physiological vs. pathological concentrations; identify differences associated with sex, age class, female reproductive status, capture time, and lactate; and determine the best methods for manatee biologists and clinicians to diagnose stress. Cortisol is the predominant manatee glucocorticoid. IMMULITE 1000 assays for cortisol and ACTH were validated. Precision yielded intra- and inter-assay coefficients of variation for serum cortisol: ≤23.5 and ≤16.7%; and ACTH: ≤6.9 and ≤8.5%. Accuracy resulted in a mean adjusted R(2)≥0.87 for serum cortisol and ≥0.96 for ACTH. Assay analytical sensitivities for cortisol (0.1 µg/dl) and ACTH (10.0 pg/ml) were verified. Methods were highly correlated with another IMMULITE 1000 for serum cortisol (r=0.97) and ACTH (r=0.98). There was no significant variation in cortisol or ACTH with sex or age class and no correlation with female progesterone concentrations. Cortisol concentrations were highest in unhealthy manatees, chronically stressed by disease or injury. ACTH was greatest in healthy free-ranging or short-term rehabilitating individuals, peracutely stressed by capture and handling. Cortisol concentrations ≥1.0 µg/dl were diagnostic of chronic stress; ACTH concentrations ≥87.5 pg/ml were diagnostic of peracute stress. In healthy long-term captive manatees, cortisol (0.4±0.2 µg/dl) and ACTH (47.7±15.9 pg/ml) concentrations were lower than healthy free-ranging, short-term rehabilitated or unhealthy manatees. Capture time was not significantly correlated with cortisol; ACTH correlation was borderline significant. Cortisol and ACTH were positively correlated with lactate.

Hematology of healthy Florida manatees (Trichechus manatus).

BACKGROUND: Hematologic analysis is an important tool in evaluating the general health status of free-ranging manatees and in the diagnosis and monitoring of rehabilitating animals. OBJECTIVES: The purpose of this study was to evaluate diagnostically important hematologic analytes in healthy manatees (Trichechus manatus) and to assess variations with respect to location (free ranging vs captive), age class (small calves, large calves, subadults, and adults), and gender. METHODS: Blood was collected from 55 free-ranging and 63 captive healthy manatees. Most analytes were measured using a CELL-DYN 3500R; automated reticulocytes were measured with an ADVIA 120. Standard manual methods were used for differential leukocyte counts, reticulocyte and Heinz body counts, and plasma protein and fibrinogen concentrations. RESULTS: Rouleaux, slight polychromasia, stomatocytosis, and low numbers of schistocytes and nucleated RBCs (NRBCs) were seen often in stained blood films. Manual reticulocyte counts were higher than automated reticulocyte counts. Heinz bodies were present in erythrocytes of most manatees. Compared with free-ranging manatees, captive animals had slightly lower MCV, MCH, and eosinophil counts and slightly higher heterophil and NRBC counts, and fibrinogen concentration. Total leukocyte, heterophil, and monocyte counts tended to be lower in adults than in younger animals. Small calves tended to have higher reticulocyte counts and NRBC counts than older animals. CONCLUSIONS: Hematologic findings were generally similar between captive and free-ranging manatees. Higher manual reticulocyte counts suggest the ADVIA detects only reticulocytes containing large amounts of RNA. Higher reticulocyte and NRBC counts in young calves probably reflect an increased rate of erythropoiesis compared with older animals.

Clinical biochemistry in healthy manatees (Trichechus manatus latirostris).

Florida manatees (Trichechus manatus latirostris) are endangered aquatic mammals living in coastal and riverine waterways of Florida and adjacent states. Serum or plasma biochemical analyses are important tools in evaluating the health of free-ranging and captive manatees. The purpose of this study was to measure diagnostically important analytes in the plasma of healthy manatees and to determine whether there was significant variation with respect to location (free-ranging versus captive), age class (small calves, large calves, subadults, adults), and gender. No significant differences in plasma sodium, potassium, bilirubin, glucose, alanine aminotransferase, or creatine kinase were found among these classes of animals. Compared to free-ranging manatees, captive animals had significantly lower mean concentrations of plasma chloride, phosphate, magnesium, triglycerides, anion gap, and lactate. Captive manatees had significantly higher mean values of total CO2, calcium, urea, creatinine, alkaline phosphatase, gamma-glutamyltransferase, total protein, albumin, and albumin/globulin ratio than did free-ranging animals. Differences in the environments of these two groups, including diet, temperature, salinity, and stress, might account for some of these results. The higher plasma lactate and anion gap concentrations and lower total CO2 concentrations of free-ranging manatees were probably due to greater exertion during capture, but the lack of elevated plasma creatine kinase activity relative to captive animals indicates that there was no serious muscle injury associated with capture. Plasma phosphate decreased and total globulins increased with age. Plasma cholesterol and triglyceride concentrations were highest in small calves. Plasma aspartate aminotransferase was higher in large calves than in adults and subadults, and the albumin/ globulin ratio was higher in subadults than in adults. Plasma total CO2 was higher and chloride was slightly lower in females than in males.