False individual patient data and zombie randomised controlled trials submitted to Anaesthesia.

Concerned that studies contain false data, I analysed the baseline summary data of randomised controlled trials when they were submitted to Anaesthesia from February 2017 to March 2020. I categorised trials with false data as 'zombie' if I thought that the trial was fatally flawed. I analysed 526 submitted trials: 73 (14%) had false data and 43 (8%) I categorised zombie. Individual patient data increased detection of false data and categorisation of trials as zombie compared with trials without individual patient data: 67/153 (44%) false vs. 6/373 (2%) false; and 40/153 (26%) zombie vs. 3/373 (1%) zombie, respectively. The analysis of individual patient data was independently associated with false data (odds ratio (95% credible interval) 47 (17-144); p = 1.3 × 10-12 ) and zombie trials (odds ratio (95% credible interval) 79 (19-384); p = 5.6 × 10-9 ). Authors from five countries submitted the majority of trials: China 96 (18%); South Korea 87 (17%); India 44 (8%); Japan 35 (7%); and Egypt 32 (6%). I identified trials with false data and in turn categorised trials zombie for: 27/56 (48%) and 20/56 (36%) Chinese trials; 7/22 (32%) and 1/22 (5%) South Korean trials; 8/13 (62%) and 6/13 (46%) Indian trials; 2/11 (18%) and 2/11 (18%) Japanese trials; and 9/10 (90%) and 7/10 (70%) Egyptian trials, respectively. The review of individual patient data of submitted randomised controlled trials revealed false data in 44%. I think journals should assume that all submitted papers are potentially flawed and editors should review individual patient data before publishing randomised controlled trials.

Risks to healthcare workers following tracheal intubation of patients with COVID-19: a prospective international multicentre cohort study.

Healthcare workers involved in aerosol-generating procedures, such as tracheal intubation, may be at elevated risk of acquiring COVID-19. However, the magnitude of this risk is unknown. We conducted a prospective international multicentre cohort study recruiting healthcare workers participating in tracheal intubation of patients with suspected or confirmed COVID-19. Information on tracheal intubation episodes, personal protective equipment use and subsequent provider health status was collected via self-reporting. The primary endpoint was the incidence of laboratory-confirmed COVID-19 diagnosis or new symptoms requiring self-isolation or hospitalisation after a tracheal intubation episode. Cox regression analysis examined associations between the primary endpoint and healthcare worker characteristics, procedure-related factors and personal protective equipment use. Between 23 March and 2 June 2020, 1718 healthcare workers from 503 hospitals in 17 countries reported 5148 tracheal intubation episodes. The overall incidence of the primary endpoint was 10.7% over a median (IQR [range]) follow-up of 32 (18-48 [0-116]) days. The cumulative incidence within 7, 14 and 21 days of the first tracheal intubation episode was 3.6%, 6.1% and 8.5%, respectively. The risk of the primary endpoint varied by country and was higher in women, but was not associated with other factors. Around 1 in 10 healthcare workers involved in tracheal intubation of patients with suspected or confirmed COVID-19 subsequently reported a COVID-19 outcome. This has human resource implications for institutional capacity to deliver essential healthcare services, and wider societal implications for COVID-19 transmission.

Risk prediction models for major surgery: composing a new tune.

In this paper I explain why I think that most of the models that predict postoperative mortality should not be used when we're talking to patients about postoperative survival. Available models are isolated in time (from survival in the present) and space (from survival outside hospital). We know a lot about survival outside hospitals, with sufficient detail that we can discriminate between a man born in 1975 vs. 1976, or a woman aged 64 years vs. 65 years. We can use survival outside hospitals to inform what we do in hospital. I use my own survival to contrast with the survival of people older or younger than me. I will use my survival to illustrate how I might expect my mortality hazard to temporarily change when I have a scheduled operation (total hip replacement) and when I'm unwell and have an operation (for a fractured femoral neck). People live longer and longer and we operate on people older and older. We are also intervening earlier in progressive diseases, knowing that people are living long enough to experience harm from their progression. There is an evolving conflict between operating on older people and operating on younger people. Who has most to gain from the operation and who has most to gain from peri-operative critical care? Do we prioritise on reducing death now, in patients with relatively short life expectancies, or do we invest in the long-term survival of patients with relatively low rates of dying now? This conundrum is not informed by current risk models, with their focus on one to three postoperative months: we need to know survival outside hospital to gauge the value of what we do in hospital.

Evidence for compromised data integrity in studies of liberal peri-operative inspired oxygen.

In 2016 the World Health Organization recommended intra-operative ventilation with 80% inspired oxygen to reduce surgical site infection rates, based upon a meta-analysis of 15 randomised controlled trials, of which two were by Mario Schietroma's research group. Five trials by this group have been retracted for duplication, plagiarism, statistical error and lack of ethical approval. We analysed 40 papers by this group: 24 randomised controlled trials (5064 participants) and 16 observational studies (1847 patients). There was evidence that data integrity was compromised in 38 out of the 40 analysed papers. The distribution of baseline characteristics in randomised controlled trials was unlikely, p = 1.5 × 10-8 : continuous variables within trials were heterogeneous, p = 1.9 × 10-9 , and categorical variables were homogeneous, p = 8.5 × 10-20 . Effects of interventions varied less than expected between studies: for categorical variables, for instance postoperative wound infection, p < 1 × 10-7 , and for continuous variables, for instance HLA-DR concentration, p = 0.00001. Of 184 calculable p values, for baseline variables or results, 179 (98%) were incorrect, ranging from three orders of magnitude too small to 10 orders of magnitude too large. Twenty-one graphs occurred 81 times in 23 out of 40 papers. Liberal peri-operative oxygen did not reduce surgical site infection in a meta-analysis of 20 trials that excluded seven trials by Mario Schietroma and colleagues (odds ratio (95%CI) 0.89 (0.73-1.08); p = 0.23). An update by the World Health Organization has now excluded trials of liberal oxygen by Schietroma's group, four of which have not been retracted. We conclude that Mario Schietroma's work should not inform practice until investigated.

Using predicted 30 day mortality to plan postoperative colorectal surgery care: a cohort study.

BACKGROUND: Preoperative identification of high-risk surgical patients might help to reduce postoperative morbidity and mortality. Using a patient's predicted 30 day mortality to plan postoperative high-dependency unit (HDU) care after elective colorectal surgery might be associated with reduced postoperative morbidity. METHODS: The 30 day postoperative mortality was predicted for 504 elective colorectal surgical patients in a preoperative clinic. The prediction was used to determine postoperative surgical ward or HDU care. Those with a predicted 30 day mortality of 1-3% mortality, and thus deemed at intermediate risk, had either planned HDU care (n=68) or planned ward care (n=139). The main outcome measures were emergency laparotomy and unplanned critical care admission. RESULTS: There were more emergency laparotomies and unplanned critical care admissions in patients with a predicted 30 day mortality of 1-3% who went to an HDU after surgery compared with patients who went to a ward: 0 vs 14 (10%), P=0.0056 and 0 vs 22 (16%), P=0.0002, respectively. CONCLUSIONS: Planned postoperative critical care was associated with a lower rate of complications after elective colorectal surgery.

Data fabrication and other reasons for non-random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals.

Randomised, controlled trials have been retracted after publication because of data fabrication and inadequate ethical approval. Fabricated data have included baseline variables, for instance, age, height or weight. Statistical tests can determine the probability of the distribution of means, given their standard deviation and the number of participants in each group. Randomised, controlled trials have been retracted after the data distributions have been calculated as improbable. Most retracted trials have been written by anaesthetists and published by specialist anaesthetic journals. I wanted to explore whether the distribution of baseline data in trials was consistent with the expected distribution. I wanted to determine whether trials retracted after publication had distributions different to trials that have not been retracted. I wanted to determine whether data distributions in trials published in specialist anaesthetic journals have been different to distributions in non-specialist medical journals. I analysed the distribution of 72,261 means of 29,789 variables in 5087 randomised, controlled trials published in eight journals between January 2000 and December 2015: Anaesthesia (399); Anesthesia and Analgesia (1288); Anesthesiology (541); British Journal of Anaesthesia (618); Canadian Journal of Anesthesia (384); European Journal of Anaesthesiology (404); Journal of the American Medical Association (518) and New England Journal of Medicine (935). I chose these journals as I had electronic access to the full text. Trial p values were distorted by an excess of baseline means that were similar and an excess that were dissimilar: 763/5015 (15.2%) trials that had not been retracted from publication had p values that were within 0.05 of 0 or 1 (expected 10%), that is, a 5.2% excess, p = 1.2 × 10-7 . The p values of 31/72 (43%) trials that had been retracted after publication were within 0.05 of 0 or 1, a rate different to that for unretracted trials, p = 1.03 × 10-10 . The difference between the distributions of these two subgroups was confirmed by comparison of their overall distributions, p = 5.3 × 10-15 . Each journal exhibited the same abnormal distribution of baseline means. There was no difference in distributions of baseline means for 1453 trials in non-anaesthetic journals and 3634 trials in anaesthetic journals, p = 0.30. The rate of retractions from JAMA and NEJM, 6/1453 or 1 in 242, was one-quarter the rate from the six anaesthetic journals, 66/3634 or 1 in 55, relative risk (99%CI) 0.23 (0.08-0.68), p = 0.00022. A probability threshold of 1 in 10,000 identified 8/72 (11%) retracted trials (7 by Fujii et al.) and 82/5015 (1.6%) unretracted trials. Some p values were so extreme that the baseline data could not be correct: for instance, for 43/5015 unretracted trials the probability was less than 1 in 1015 (equivalent to one drop of water in 20,000 Olympic-sized swimming pools). A probability threshold of 1 in 100 for two or more trials by the same author identified three authors of retracted trials (Boldt, Fujii and Reuben) and 21 first or corresponding authors of 65 unretracted trials. Fraud, unintentional error, correlation, stratified allocation and poor methodology might have contributed to the excess of randomised, controlled trials with similar or dissimilar means, a pattern that was common to all the surveyed journals. It is likely that this work will lead to the identification, correction and retraction of hitherto unretracted randomised, controlled trials.

Evidence for non-random sampling in randomised, controlled trials by Yuhji Saitoh.

A large number of randomised trials authored by Yoshitaka Fujii have been retracted, in part as a consequence of a previous analysis finding a very low probability of random sampling. Dr Yuhji Saitoh co-authored 34 of those trials and he was corresponding author for eight of them. We found a number of additional randomised, controlled trials that included baseline data, with Saitoh as corresponding author, that Fujii did not co-author. We used Monte Carlo simulations to analyse the baseline data from 32 relevant trials in total as well as an outcome (muscle twitch recovery ratios) reported in several. We also compared a series of muscle twitch recovery graphs appearing in a number of Saitoh's publications. The baseline data in 14/32 randomised, controlled trials had p < 0.01, of which seven p values were < 0.001. Eight trials reported four ratios of the time for the return of muscle activity after neuromuscular blockade, the distributions of which were homogeneous: the p values for the observed Q statistics were 0.0055, 0.031, 0.016 and 0.0071. Comparison of graphs revealed multiple coincident or near-coincident curves across a large number of publications, a finding also inconsistent with random sampling. Combining the continuous and categorical probabilities of the 32 included trials, we found a very low likelihood of random sampling: p = 1.27 × 10-8 (1 in 100,000,000). The high probability of non-random sampling and the repetition of lines in multiple graphs suggest that further scrutiny of Saitoh's work is warranted.

Simulations of the effects of scheduled abdominal aortic aneurysm repair on survival.

I simulated survival with and without scheduled repair of abdominal aortic aneurysms with different diameters in different populations. The results imply that scheduled repair should be determined by the combination of a patient's monthly mortality hazard and aneurysm diameter. The median survival of some patients will be extended by the scheduled repair of aneurysms smaller than 55 mm, whereas the median survival of other patients will be curtailed by repair of any aneurysm. The results also suggest that, on average, surveillance is futile: the effect of scheduled aneurysm repair on an individual's median survival did not change but the cohort effect diminished as patients died during surveillance. The results of the U.K. Small Aneurysm Study were reproduced in simulation and are compatible with the repair of aneurysms smaller than 55 mm diameter. Epidemiological simulations suggest that past randomised controlled trials underestimate the effect of aneurysm repair today.

Validation of long-term survival prediction for scheduled abdominal aortic aneurysm repair with an independent calculator using only pre-operative variables.

We observed survival after scheduled repair of abdominal aortic aneurysm in 1096 patients for a median (IQR [range]) of 3.0 (1.5-5.8 [0-15]) years: 943 patients had complete data, 250 of whom died. We compared discrimination and calibration of an external model with the Kaplan-Meier model generated from the study data. Integrated Brier misclassification scores for both models at 1-5 postoperative years were 0.04, 0.08, 0.11, 0.13 and 0.16, respectively. Harrel's concordance index at 1-5 postoperative years was 0.73, 0.71, 0.68, 0.67 and 0.66, respectively. Groups with median 5-year predicted mortality of 40% (n = 251), 18% (n = 414) and 8% (n = 164) had lower observed mortality than 114 patients with 70% predicted mortality, hazard ratio (95% CI): 0.58 (0.37-0.76), p = 0.0031; 0.30 (0.19-0.48), p = 1.7 × 10(-12) and 0.19 (0.13-0.27), p = 1.3 × 10(-10) , respectively, test for trend p = 5.6 × 10(-15) . Survival predicted by the external calculator was similar to the Kaplan-Meier estimate.

Calculating the probability of random sampling for continuous variables in submitted or published randomised controlled trials.

In a previous paper, one of the authors (JBC) used a chi-squared method to analyse the means (SD) of baseline variables, such as height or weight, from randomised controlled trials by Fujii et al., concluding that the probabilities that the reported distributions arose by chance were infinitesimally small. Subsequent testing of that chi-squared method, using simulation, suggested that the method was incorrect. This paper corrects the chi-squared method and tests its performance and the performance of Monte Carlo simulations and ANOVA to analyse the probability of random sampling. The corrected chi-squared method and ANOVA method became inaccurate when applied to means that were reported imprecisely. Monte Carlo simulations confirmed that baseline data from 158 randomised controlled trials by Fujii et al. were different to those from 329 trials published by other authors and that the distribution of Fujii et al.'s data were different to the expected distribution, both p < 10(-16) . The number of Fujii randomised controlled trials with unlikely distributions was less with Monte Carlo simulation than with the 2012 chi-squared method: 102 vs 117 trials with p < 0.05; 60 vs 86 for p < 0.01; 30 vs 56 for p < 0.001; and 12 vs 24 for p < 0.00001, respectively. The Monte Carlo analysis nevertheless confirmed the original conclusion that the distribution of the data presented by Fujii et al. was extremely unlikely to have arisen from observed data. The Monte Carlo analysis may be an appropriate screening tool to check for non-random (i.e. unreliable) data in randomised controlled trials submitted to journals.

Pre-operative co-morbidity and postoperative survival in the elderly: beyond one lunar orbit.

Mortality is a good measure of killing, but it is a poor measure of cure, palliation or the maintenance of function. Nevertheless, it has remained the primary metric of hospital care for 200 years. This article discusses the factors that contribute to mortality risk and survival trajectories, as well as the increasing recognition that surgery kills for months after the last suture is tied. This article discusses how disparate factors can usefully combine to generate an 'elderly' group with a monthly mortality in excess of 1% and a median life expectancy less than 3.5 years. A downloadable spreadsheet is provided that combines risk factors to generate mortality risks and their associated survival curves, emphasising the importance of looking beyond one postoperative month.

Case-controlled study of critical care or surgical ward care after elective open colorectal surgery.

BACKGROUND: Evidence for the benefit of critical care after surgery is limited. This study assessed the value of immediate admission to the critical care unit (CCU) after open colorectal surgery. METHODS: Patients aged over 45 years were screened with a cardiopulmonary exercise test to determine their anaerobic threshold. Less fit patients defined by an anaerobic threshold below 11 ml oxygen per kg per min were assigned to either critical care or surgical ward care. Those with an anaerobic threshold of 11 ml oxygen per kg per min or above were assigned to ward care. The outcome measure was the number of cardiac events. RESULTS: Of 153 patients who underwent exercise testing, 55 had an anaerobic threshold of at least 11 ml oxygen per kg per min (ward care) and 98 had a threshold of less than 11 ml oxygen per kg per min, of whom 39 were allocated to ward care and 51 to critical care. Median length of CCU stay was 31 (range 5-46) h. More cardiac events occurred in patients allocated to ward care (7 of 39) than in those allocated to critical care (0 of 51): absolute difference 18 (95 per cent confidence interval 10 to 26) per cent (P = 0·002). There were no cardiac events in patients with an anaerobic threshold of 11 ml oxygen per kg per min or higher. CONCLUSION: Patients with an anaerobic threshold of at least 11 ml oxygen per kg per min and those with a threshold below 11 ml oxygen per kg per min managed in the CCU had fewer cardiac events.

Assessing fitness, predicting outcome, and the missing axis.

Summary This article discusses how to estimate the risk of postoperative death, an outcome that affects an important minority of patients in the month(s) after scheduled surgery. In addition, it reflects on our inability to characterize the effects of surgery on quality of life. This outcome is of primary concern to the much more numerous survivors but is absent from all graphs of postoperative survival: it is the missing axis. The calculations discussed in the article are available online at https://sites.google.com/site/informrisk/.