Permutation-based multiple testing corrections for P $$ P $$ -values and confidence intervals for cluster randomized trials.

In this article, we derive and compare methods to derive P-values and sets of confidence intervals with strong control of the family-wise error rates and coverage for estimates of treatment effects in cluster randomized trials with multiple outcomes. There are few methods for P-value corrections and deriving confidence intervals, limiting their application in this setting. We discuss the methods of Bonferroni, Holm, and Romano and Wolf and adapt them to cluster randomized trial inference using permutation-based methods with different test statistics. We develop a novel search procedure for confidence set limits using permutation tests to produce a set of confidence intervals under each method of correction. We conduct a simulation-based study to compare family-wise error rates, coverage of confidence sets, and the efficiency of each procedure in comparison to no correction using both model-based standard errors and permutation tests. We show that the Romano-Wolf type procedure has nominal error rates and coverage under non-independent correlation structures and is more efficient than the other methods in a simulation-based study. We also compare results from the analysis of a real-world trial.

Design and analysis of three-arm parallel cluster randomized trials with small numbers of clusters.

In this article, we review and evaluate a number of methods used in the design and analysis of small three-arm parallel cluster randomized trials. We conduct a simulation-based study to evaluate restricted randomization methods including covariate-constrained randomization and a novel method for matched-group cluster randomization. We also evaluate the appropriate modelling of the data and small sample inferential methods for a variety of treatment effects relevant to three-arm trials. Our results indicate that small-sample corrections are required for high (0.05) but not low (0.001) values of the intraclass correlation coefficient and their performance can depend on trial design, number of clusters, and the nature of the hypothesis being tested. The Satterthwaite correction generally performed best at an ICC of 0.05 with a nominal type I error rate for single-period trials, and in trials with repeated measures type I error rates were between 0.04 and 0.06. Restricted randomization methods produce little benefit in trials with repeated measures but in trials with single post-intervention design can provide relatively large gains in power when compared to the most unbalanced possible allocations. Matched-group randomization improves power but is not as effective as covariate-constrained randomization. For model-based analysis, adjusting for fewer covariates than were used in a restricted randomization process under any design can produce non-nominal type I error rates and reductions in power. Where comparisons to two-arm cluster trials are possible, the performance of the methods is qualitatively very similar.

Development of a video-observation method for examining doctors' clinical and interpersonal skills in a hospital outpatient clinic in Ibadan, Oyo State, Nigeria.

BACKGROUND: Improving the quality of primary healthcare provision is a key goal in low-and middle-income countries (LMICs). However, to develop effective quality improvement interventions, we first need to be able to accurately measure the quality of care. The methods most commonly used to measure the technical quality of care all have some key limitations in LMICs settings. Video-observation is appealing but has not yet been used in this context. We examine preliminary feasibility and acceptability of video-observation for assessing physician quality in a hospital outpatients' department in Nigeria. We also develop measurement procedures and examine measurement characteristics. METHODS: Cross-sectional study at a large tertiary care hospital in Ibadan, Nigeria. Consecutive physician-patient consultations with adults and children under five seeking outpatient care were video-recorded. We also conducted brief interviews with participating physicians to gain feedback on our approach. Video-recordings were double-coded by two medically trained researchers, independent of the study team and each other, using an explicit checklist of key processes of care that we developed, from which we derived a process quality score. We also elicited a global quality rating from reviewers. RESULTS: We analysed 142 physician-patient consultations. The median process score given by both coders was 100 %. The modal overall rating category was 'above standard' (or 4 on a scale of 1-5). Coders agreed on which rating to assign only 44 % of the time (weighted Cohen's kappa = 0.26). We found in three-level hierarchical modelling that the majority of variance in process scores was explained by coder disagreement. A very high correlation of 0.90 was found between the global quality rating and process quality score across all encounters. Participating physicians liked our approach, despite initial reservations about being observed. CONCLUSIONS: Video-observation is feasible and acceptable in this setting, and the quality of consultations was high. However, we found that rater agreement is low but comparable to other modalities that involve expert clinician judgements about quality of care including in-person direct observation and case note review. We suggest ways to improve scoring consistency including careful rater selection and improved design of the measurement procedure for the process score.

Randomised evaluation of government health programmes does present a challenge to standard research ethics frameworks.

In a recent issue of Journal of Medical Ethics (JME), we discussed the ethical review of evaluations of interventions that would occur whether or not the evaluation was taking place. We concluded that standard research ethics frameworks including the Ottawa Statement, which requires justification for all aspects of an intervention and its roll-out, were a poor guide in this area. We proposed that a consideration of researcher responsibility, based on the consequences of the research taking place, would be a more appropriate way delineate the scope of research ethics review. Weijer and Taljaard present a counterargument to our proposal, which we address in this reply. They claim that a focus on researcher responsibility will weaken the protection of research participants and link it to 'unethical research' and a 'government experimenting on its own people'. However, the moral responsibility of researchers is defined in terms of the consequences of the research on human welfare and harm, not in opposition to it. Weijer and Taljaard argue that researchers must justify what they are studying whether or not they have any control over it and that governments must justify their programmes, including by demonstrating equipoise, to a research ethics committee if they implement them in a randomised way. We strongly disagree that this is a defensible way to define the scope of research ethics review and argue that this provides no further protections to research participants beyond what we propose, but places a potential barrier to learning from government programmes.

Revising ethical guidance for the evaluation of programmes and interventions not initiated by researchers.

Public health and service delivery programmes, interventions and policies (collectively, 'programmes') are typically developed and implemented for the primary purpose of effecting change rather than generating knowledge. Nonetheless, evaluations of these programmes may produce valuable learning that helps determine effectiveness and costs as well as informing design and implementation of future programmes. Such studies might be termed 'opportunistic evaluations', since they are responsive to emergent opportunities rather than being studies of interventions that are initiated or designed by researchers. However, current ethical guidance and registration procedures make little allowance for scenarios where researchers have played no role in the development or implementation of a programme, but nevertheless plan to conduct a prospective evaluation. We explore the limitations of the guidance and procedures with respect to opportunistic evaluations, providing a number of examples. We propose that one key missing distinction in current guidance is moral responsibility: researchers can only be held accountable for those aspects of a study over which they have control. We argue that requiring researchers to justify an intervention, programme or policy that would occur regardless of their involvement prevents or hinders research in the public interest without providing any further protections to research participants. We recommend that trial consent and ethics procedures allow for a clear separation of responsibilities for the intervention and the evaluation.

Improving the health and welfare of people who live in slums.

In the first paper in this Series we assessed theoretical and empirical evidence and concluded that the health of people living in slums is a function not only of poverty but of intimately shared physical and social environments. In this paper we extend the theory of so-called neighbourhood effects. Slums offer high returns on investment because beneficial effects are shared across many people in densely populated neighbourhoods. Neighbourhood effects also help explain how and why the benefits of interventions vary between slum and non-slum spaces and between slums. We build on this spatial concept of slums to argue that, in all low-income and-middle-income countries, census tracts should henceforth be designated slum or non-slum both to inform local policy and as the basis for research surveys that build on censuses. We argue that slum health should be promoted as a topic of enquiry alongside poverty and health.

The history, geography, and sociology of slums and the health problems of people who live in slums.

Massive slums have become major features of cities in many low-income and middle-income countries. Here, in the first in a Series of two papers, we discuss why slums are unhealthy places with especially high risks of infection and injury. We show that children are especially vulnerable, and that the combination of malnutrition and recurrent diarrhoea leads to stunted growth and longer-term effects on cognitive development. We find that the scientific literature on slum health is underdeveloped in comparison to urban health, and poverty and health. This shortcoming is important because health is affected by factors arising from the shared physical and social environment, which have effects beyond those of poverty alone. In the second paper we will consider what can be done to improve health and make recommendations for the development of slum health as a field of study.

Statin use and all-cause mortality in people living with HIV: a systematic review and meta-analysis.

BACKGROUND: It is unknown whether statin use among people living with HIV results in a reduction in all-cause mortality. We aimed to evaluate the effect of statin use on all-cause mortality among people living with HIV. METHODS: We conducted comprehensive literature searches of Medline, Embase, CINAHL, the Cochrane Library, and cross-references up to April 2018. We included randomised, quasi-randomised trials and prospective cohort studies that examined the association between statin use and cardio-protective and mortality outcomes among people living with HIV. Two reviewers independently abstracted the data. Hazard ratios (HRs) were pooled using empirical Bayesian random-effect meta-analysis. A number of sensitivity analyses were conducted. RESULTS: We included seven studies with a total of 35,708 participants. The percentage of participants on statins across the studies ranged from 8 to 35%. Where reported, the percentage of participants with hypertension ranged from 14 to 35% and 7 to 10% had been diagnosed with diabetes mellitus. Statin use was associated with a 33% reduction in all-cause mortality (pooled HR = 0.67, 95% Credible Interval 0.39 to 0.96). The probability that statin use conferred a moderate mortality benefit (i.e. decreased risk of mortality of at least 25%, HR ≤ 0.75) was 71.5%. Down-weighting and excluding the lower quality studies resulted in a more conservative estimate of the pooled HR. CONCLUSION: Statin use appears to confer moderate mortality benefits in people living with HIV.

A randomised Trial of Autologous Blood products, leukocyte and platelet-rich fibrin (L-PRF), to promote ulcer healing in LEprosy: The TABLE trial.

INTRODUCTION: Autologous blood products like Platelet Rich Plasma (PRP) and Leukocyte and Platelets Rich Fibrin (L-PRF) have been used for many years across many types of skin ulcers. However, the effectiveness of autologous blood products on wound healing is not well established. METHODS: We evaluated the 'second generation' autologous product- Leukocyte and Platelet- Rich Fibrin (L-PRF). Our trial was undertaken on patients suffering from neuropathic leprosy ulcers at the Anandaban hospital which serves the entire country of Nepal. We conducted a 1:1 (n = 130) individually randomised trial of L-PRF (intervention) vs. normal saline dressing (control) to compare rate of healing and time to complete healing. Rate of healing was estimated using blind assessments of ulcer areas based on three different measurement methods. Time to complete healing was measured by the local unblinded clinicians and by blind assessment of ulcer images. RESULTS: The point estimates for both outcomes were favourable to L-PRF but the effect sizes were small. Unadjusted mean differences (intervention vs control) in mean daily healing rates (cm2) were respectively 0.012 (95% confidence interval 0.001 to 0.023, p = 0.027); 0.016 (0.004 to 0.027, p = 0.008) and 0.005 (-0.005 to 0.016, p = 0.313) across the three measurement methods. Time to complete healing at 42 days yielded Hazard Ratios (unadjusted) of 1.3 (0.8 to 2.1, p = 0.300) assessed by unblinded local clinicians and 1.2 (0.7 to 2.0, p = 0.462) on blind assessment. CONCLUSION: Any benefit from L-PRF appears insufficient to justify routine use in care of neuropathic ulcers in leprosy. TRIAL REGISTRATION: ISRCTN14933421. Date of trial registration: 16 June 2020.

Evaluation of combinatorial optimisation algorithms for c-optimal experimental designs with correlated observations.

We show how combinatorial optimisation algorithms can be applied to the problem of identifying c-optimal experimental designs when there may be correlation between and within experimental units and evaluate the performance of relevant algorithms. We assume the data generating process is a generalised linear mixed model and show that the c-optimal design criterion is a monotone supermodular function amenable to a set of simple minimisation algorithms. We evaluate the performance of three relevant algorithms: the local search, the greedy search, and the reverse greedy search. We show that the local and reverse greedy searches provide comparable performance with the worst design outputs having variance <10% greater than the best design, across a range of covariance structures. We show that these algorithms perform as well or better than multiplicative methods that generate weights to place on experimental units. We extend these algorithms to identifying modle-robust c-optimal designs. Supplementary Information: The online version contains supplementary material available at 10.1007/s11222-023-10280-w.

Prevalence of plantar ulcer and its risk factors in leprosy: a systematic review and meta-analysis.

BACKGROUND: Plantar ulcers are a leading complication of leprosy that requires frequent visits to hospital and is associated with stigma. The extent of burden of ulcers in leprosy and its risk factors are scant impeding the development of targeted interventions to prevent and promote healing of ulcers. The aim of this review is to generate evidence on the prevalence of plantar ulcer and its risk factors in leprosy. METHODS: Databases (Medline, Embase, Web of Science, CINAHL, BVS), conference abstracts and reference lists were searched for eligible studies. Studies were included that reported a point prevalence of plantar ulcer and/or its "risk factors" associated with development of ulcers (either causatively or predictively), including individual level, disease related and bio-mechanical factors. We followed PRISMA guidelines for this review. Random-effects meta-analysis was undertaken to estimate the pooled point prevalence of ulcers. Reported risk factors in included studies were narratively synthesised. This review is registered in PROSPERO: CRD42022316726. RESULTS: Overall, 15 studies (8 for prevalence of ulcer and 7 for risk factors) met the inclusion criteria. The pooled point prevalence of ulcer was 34% (95% CIs: 21%, 46%) and 7% (95% CIs: 4%, 11%) among those with foot anaesthesia and among all people affected by leprosy, respectively. Risk factors for developing ulcers included: unable to feel 10 g of monofilament on sensory testing, pronated/hyper-pronated foot, foot with peak plantar pressure, foot with severe deformities, and those with lower education and the unemployed. CONCLUSIONS: The prevalence of plantar ulceration in leprosy is as high as 34% among those with loss of sensation in the feet. However, the incidence and recurrence rates of ulceration are least reported. The inability to feel 10 g of monofilament appears to be a strong predictor of those at risk of developing ulcers. However, there is a paucity of evidence on identifying those at risk of developing plantar ulcers in leprosy. Prospective studies are needed to estimate the incidence of ulcers. Identifying individuals at risk of ulcers will help design targeted interventions to minimize risk factors, prevent ulcers and promote ulcer healing.

Optimal study designs for cluster randomised trials: An overview of methods and results.

There are multiple possible cluster randomised trial designs that vary in when the clusters cross between control and intervention states, when observations are made within clusters, and how many observations are made at each time point. Identifying the most efficient study design is complex though, owing to the correlation between observations within clusters and over time. In this article, we present a review of statistical and computational methods for identifying optimal cluster randomised trial designs. We also adapt methods from the experimental design literature for experimental designs with correlated observations to the cluster trial context. We identify three broad classes of methods: using exact formulae for the treatment effect estimator variance for specific models to derive algorithms or weights for cluster sequences; generalised methods for estimating weights for experimental units; and, combinatorial optimisation algorithms to select an optimal subset of experimental units. We also discuss methods for rounding experimental weights, extensions to non-Gaussian models, and robust optimality. We present results from multiple cluster trial examples that compare the different methods, including determination of the optimal allocation of clusters across a set of cluster sequences and selecting the optimal number of single observations to make in each cluster-period for both Gaussian and non-Gaussian models, and including exchangeable and exponential decay covariance structures.

Autologous blood products: Leucocyte and Platelets Rich Fibrin (L-PRF) and Platelets Rich Plasma (PRP) gel to promote cutaneous ulcer healing - a systematic review.

OBJECTIVE: To summarise evidence on the effectiveness of Platelet-Rich Plasma (PRP) gel and Leucocyte and Platelet Rich Fibrin (L-PRF) gel as agents promoting ulcer healing compared with the standard wound dressing techniques alone. DESIGN: Systematic review. ELIGIBILITY CRITERIA: Individual patient randomised controlled trials on skin ulcers of all types excluding traumatic lesions.Intervention group: treatment with topical application of L-PRF gel or PRP gel to the wound surface. CONTROL GROUP: treatment with standard skin ulcer care using normal saline, normgel or hydrogel dressings. INFORMATION SOURCES: Medline (Ovid), Excerpta Medica Database (EMBASE), Scopus, Cumulative Index to Nursing and Allied Health Literature (CINAHL) and Web of Science and manual search of studies from previous systematic reviews and meta-analyses. The papers published from 1946 to 2022 with no restriction on geography and language were included. The last date of the search was performed on 29 August 2022. DATA EXTRACTION AND SYNTHESIS: Independent reviewers identified eligible studies, extracted data, assessed risk of bias using V.2 of the Cochrane risk-of-bias tool for randomised trials tool and assessed certainty of evidence by using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach. MAIN OUTCOME MEASURES: Time to complete healing, proportion healed at a given time and rate of healing. RESULTS: Seven studies met the inclusion criteria, five using PRP gel and two using L-PRF gel. One study showed a better proportion of complete healing, three reported reduced meantime to complete healing and five showed improved rate of healing per unit of time in the intervention group. The risk of bias was high across all studies with one exception and the GRADE showed very low certainty of evidence. CONCLUSION: The findings show potential for better outcomes in the intervention; however, the evidence remains inconclusive highlighting a large research gap in ulcer treatment and warrant better-designed clinical trials. PROSPERO REGISTRATION NUMBER: CRD42022352418.

Safety and upscaling of remote consulting for long-term conditions in primary health care in Nigeria and Tanzania (REaCH trials): stepped-wedge trials of training, mobile data allowance, and implementation.

BACKGROUND: In-person health care poses risks to health workers and patients during pandemics. Remote consultations can mitigate these risks. The REaCH intervention comprised training and mobile data allowance provision for mobile phones to support remotely delivered primary care in Africa compared with no training and mobile data allowance. The aim of this study was to estimate the effects of REaCH among adults with non-communicable diseases on remote and face-to-face consultation rates, patient safety, and trustworthiness of consultations. METHODS: In these two independent stepped-wedge cluster randomised controlled trials, we enrolled 20 primary care clusters in each of two settings (Oyo State, Nigeria, and Morogoro Region, Tanzania). Eligible clusters had 100 or more patients with diabetes, hypertension, and cardiovascular or pulmonary disease employing five health workers. Clusters were computer-randomised to one of ten (Nigeria) or one of seven (Tanzania) sequences to receive the REaCH intervention. Only outcome assessors were masked. Primary outcomes were consultation, prescription, and investigation rates, and trustworthiness collected monthly for 12 months (Nigeria) and 9 months (Tanzania) from open cohorts. Ten randomly sampled consulting patients per cluster-month completed patient reported outcome measures. This trial was registered with ISRCTN, ISRCTN17941313. FINDINGS: Overall, 40 clusters comprising 8776 (Nigeria) and 3246 (Tanzania) patients' open cohort data were analysed (6377 [72·7%] of 8776 females in Nigeria, and 2235 [68·9%] of 3246 females in Tanzania). The mean age of the participants was 55·3 years (SD 13·9) in Nigeria and 59·2 years (14·2) in Tanzania. In Nigeria, no evidence of change in face-to-face consulting rate was observed (rate ratio [RR] 1·06, 95% CI 0·98 to 1·09; p=0·16); however, remote consultations increased four-fold (4·44, 1·34 to >10; p=0·01). In Tanzania, face-to-face (0·94, 0·61 to 1·67; p=0·99) and remote consulting rates (1·17, 0·56 to 5·57; p=0·39) were unchanged. There was no evidence of difference in prescribing rates (Nigeria: 1·05, 0·60 to 1·14; p=0·23; Tanzania: 0·92, 0·60 to 1·67; p=0·97), investigation rates (Nigeria: 1·06, 0·23 to 2·12; p=0·49; Tanzania: 1·15, 0·35 to 1·64; 0·58) or trustworthiness scores (Nigeria: mean difference 0·05, 95% CI -0·45 to 0·42; p=0·89; Tanzania: 0·07, -0·15 to 0·76; p=0·70). INTERPRETATION: REaCH can be implemented and could improve intervention versus control health-care access. Remote consultations appear safe and trustworthy, supporting universal health coverage. FUNDING: The UK Research and Innovation Collective Fund. TRANSLATIONS: For the Swahili and Yoruba translations of the abstract see Supplementary Materials section.

Evaluation of an organisational-level monetary incentive to promote the health and wellbeing of workers in small and medium-sized enterprises: A mixed-methods cluster randomised controlled trial.

We conducted an independent evaluation on the effectiveness of an organisational-level monetary incentive to encourage small and medium-sized enterprises (SMEs) to improve employees' health and wellbeing. This was A mixed-methods cluster randomised trial with four arms: high monetary incentive, low monetary incentive, and two no monetary incentive controls (with or without baseline measurements to examine 'reactivity' The consequence of particpant awareness of being studied, and potential impact on participant behavior effects). SMEs with 10-250 staff based in West Midlands, England were eligible. We randomly selected up to 15 employees at baseline and 11 months post-intervention. We elicited employee perceptions of employers' actions to improve health and wellbeing; and employees' self-reported health behaviours and wellbeing. We also interviewed employers and obtained qualitative data. One hundred and fifty-two SMEs were recruited. Baseline assessments were conducted in 85 SMEs in three arms, and endline assessments in 100 SMEs across all four arms. The percentage of employees perceiving "positive action" by their employer increased after intervention (5 percentage points, pp [95% Credible Interval -3, 21] and 3pp [-9, 17], in models for high and low incentive groups). Across six secondary questions about specific issues the results were strongly and consistently positive, especially for the high incentive. This was consistent with qualitative data and quantitative employer interviews. However, there was no evidence of any impact on employee health behaviour or wellbeing outcomes, nor evidence of 'reactivity'. An organisational intervention (a monetary incentive) changed employee perceptions of employer behaviour but did not translate into changes in employees' self-reports of their own health behaviours or wellbeing. Trial registration: AEARCTR-0003420, registration date: 17.10.2018, retrospectively registered (delays in contracts and identfying a suitable trial registry). The authors confirm that there are no ongoing and related trials for this intervention.

Protocol for a parallel group, two-arm, superiority cluster randomised trial to evaluate a community-level complementary-food safety and hygiene and nutrition intervention in Mali: the MaaCiwara study (version 1.3; 10 November 2022).

BACKGROUND: Diarrhoeal disease remains a significant cause of morbidity and mortality among the under-fives in many low- and middle-income countries. Changes to food safety practices and feeding methods around the weaning period, alongside improved nutrition, may significantly reduce the risk of disease and improve development for infants. We describe a protocol for a cluster randomised trial to evaluate the effectiveness of a multi-faceted community-based educational intervention that aims to improve food safety and hygiene behaviours and enhance child nutrition. METHODS: We describe a mixed-methods, parallel group, two-arm, superiority cluster randomised controlled trial with baseline measures. One hundred twenty clusters comprising small urban and rural communities will be recruited in equal numbers and randomly allocated in a 1:1 ratio to either treatment or control arms. The community intervention will be focussed around an ideal mother concept involving all community members during campaign days with dramatic arts and pledging, and follow-up home visits. Participants will be mother-child dyads (27 per cluster period) with children aged 6 to 36 months. Data collection will comprise a day of observation and interviews with each participating mother-child pair and will take place at baseline and 4 and 15 months post-intervention. The primary analysis will estimate the effectiveness of the intervention on changes to complementary-food safety and preparation behaviours, food and water contamination, and diarrhoea. Secondary outcomes include maternal autonomy, enteric infection, nutrition, child anthropometry, and development scores. A additional structural equation analysis will be conducted to examine the causal relationships between the different outcomes. Qualitative and health economic analyses including process evaluation will be done. CONCLUSIONS: The trial will provide evidence on the effectiveness of community-based behavioural change interventions designed to reduce the burden of diarrhoeal disease in the under-fives and how effectiveness varies across different contexts. TRIAL REGISTRATION: ISRCTN14390796. Registration date December 13, 2021.

Evaluations of water, sanitation and hygiene interventions should not use diarrhoea as (primary) outcome.

Water, sanitation and hygiene interventions have been the subject of cluster trials of unprecedented size, scale and cost in recent years. However, the question 'what works in water, sanitation, hygiene (WASH)?' remains poorly understood. Evaluations of community interventions to prevent infectious disease typically use lab-confirmed infection as a primary outcome; however, WASH trials mostly use reported diarrhoea. While diarrhoea is a significant source of morbidity, it is subjected to significant misclassification error with respect to enteric infection due to the existence of non-infectious diarrhoea and asymptomatic infection. We show how this may lead to bias of estimated effects of interventions from WASH trials towards no effect. The problem is further compounded by other biases in the measurement process. Alongside testing for infection of the gut, an examination of the causal assumptions underlying WASH interventions present several other reliable alternative and complementary measurements and outcomes. Contemporary guidance on the evaluation of complex interventions requires researchers to take a broad view of the causal effects of an intervention across a system. Reported diarrhoea can fail to even be a reliable measure of changes to gastrointestinal health and so should not be used as a primary outcome if we are to progress our knowledge of what works in WASH.