The impact of imputation quality on machine learning classifiers for datasets with missing values.

BACKGROUND: Classifying samples in incomplete datasets is a common aim for machine learning practitioners, but is non-trivial. Missing data is found in most real-world datasets and these missing values are typically imputed using established methods, followed by classification of the now complete samples. The focus of the machine learning researcher is to optimise the classifier's performance. METHODS: We utilise three simulated and three real-world clinical datasets with different feature types and missingness patterns. Initially, we evaluate how the downstream classifier performance depends on the choice of classifier and imputation methods. We employ ANOVA to quantitatively evaluate how the choice of missingness rate, imputation method, and classifier method influences the performance. Additionally, we compare commonly used methods for assessing imputation quality and introduce a class of discrepancy scores based on the sliced Wasserstein distance. We also assess the stability of the imputations and the interpretability of model built on the imputed data. RESULTS: The performance of the classifier is most affected by the percentage of missingness in the test data, with a considerable performance decline observed as the test missingness rate increases. We also show that the commonly used measures for assessing imputation quality tend to lead to imputed data which poorly matches the underlying data distribution, whereas our new class of discrepancy scores performs much better on this measure. Furthermore, we show that the interpretability of classifier models trained using poorly imputed data is compromised. CONCLUSIONS: It is imperative to consider the quality of the imputation when performing downstream classification as the effects on the classifier can be considerable.

Many artificial intelligence (AI) methods aim to classify samples of data into groups, e.g., patients with disease vs. those without. This often requires datasets to be complete, i.e., that all data has been collected for all samples. However, in clinical practice this is often not the case and some data can be missing. One solution is to 'complete' the dataset using a technique called imputation to replace those missing values. However, assessing how well the imputation method performs is challenging. In this work, we demonstrate why people should care about imputation, develop a new method for assessing imputation quality, and demonstrate that if we build AI models on poorly imputed data, the model can give different results to those we would hope for. Our findings may improve the utility and quality of AI models in the clinic.

Common methodological pitfalls in ICI pneumonitis risk prediction studies.

Background: Pneumonitis is one of the most common adverse events induced by the use of immune checkpoint inhibitors (ICI), accounting for a 20% of all ICI-associated deaths. Despite numerous efforts to identify risk factors and develop predictive models, there is no clinically deployed risk prediction model for patient risk stratification or for guiding subsequent monitoring. We believe this is due to systemic suboptimal approaches in study designs and methodologies in the literature. The nature and prevalence of different methodological approaches has not been thoroughly examined in prior systematic reviews. Methods: The PubMed, medRxiv and bioRxiv databases were used to identify studies that aimed at risk factor discovery and/or risk prediction model development for ICI-induced pneumonitis (ICI pneumonitis). Studies were then analysed to identify common methodological pitfalls and their contribution to the risk of bias, assessed using the QUIPS and PROBAST tools. Results: There were 51 manuscripts eligible for the review, with Japan-based studies over-represented, being nearly half (24/51) of all papers considered. Only 2/51 studies had a low risk of bias overall. Common bias-inducing practices included unclear diagnostic method or potential misdiagnosis, lack of multiple testing correction, the use of univariate analysis for selecting features for multivariable analysis, discretization of continuous variables, and inappropriate handling of missing values. Results from the risk model development studies were also likely to have been overoptimistic due to lack of holdout sets. Conclusions: Studies with low risk of bias in their methodology are lacking in the existing literature. High-quality risk factor identification and risk model development studies are urgently required by the community to give the best chance of them progressing into a clinically deployable risk prediction model. Recommendations and alternative approaches for reducing the risk of bias were also discussed to guide future studies.

Effectiveness of international border control measures during the COVID-19 pandemic: a narrative synthesis of published systematic reviews.

The effectiveness of international border control measures during the COVID-19 pandemic is not well understood. Using a narrative synthesis approach to published systematic reviews, we synthesized the evidence from both modelling and observational studies on the effects of border control measures on domestic transmission of the virus. We find that symptomatic screening measures were not particularly effective, but that diagnostic-based screening methods were more effective at identifying infected travellers. Targeted travel restrictions levied against travellers from Wuhan were likely temporarily effective but insufficient to stop the exportation of the virus to the rest of the world. Quarantine of inbound travellers was also likely effective at reducing transmission, but only with relatively long quarantine periods, and came with important economic and social effects. There is little evidence that most travel restrictions, including border closure and those implemented to stop the introduction of new variants of concern, were particularly effective. Border control measures played an important role in former elimination locations but only when coupled with strong domestic public health measures. In future outbreaks, if border control measures are to be adopted, they should be seen as part of a broader strategy that includes other non-pharmaceutical interventions. This article is part of the theme issue 'The effectiveness of non-pharmaceutical interventions on the COVID-19 pandemic: the evidence'.

A pipeline to further enhance quality, integrity and reusability of the NCCID clinical data.

The National COVID-19 Chest Imaging Database (NCCID) is a centralized UK database of thoracic imaging and corresponding clinical data. It is made available by the National Health Service Artificial Intelligence (NHS AI) Lab to support the development of machine learning tools focused on Coronavirus Disease 2019 (COVID-19). A bespoke cleaning pipeline for NCCID, developed by the NHSx, was introduced in 2021. We present an extension to the original cleaning pipeline for the clinical data of the database. It has been adjusted to correct additional systematic inconsistencies in the raw data such as patient sex, oxygen levels and date values. The most important changes will be discussed in this paper, whilst the code and further explanations are made publicly available on GitLab. The suggested cleaning will allow global users to work with more consistent data for the development of machine learning tools without being an expert. In addition, it highlights some of the challenges when working with clinical multi-center data and includes recommendations for similar future initiatives.

Effect of pelleting on the enzymatic digestibility of corn stover.

Pelleting of lignocellulosic biomass to improve its transportation, storage and handling impacts subsequent processing and conversion. This work reports the role of high moisture pelleting in the enzymatic digestibility of corn stover prior to pretreatment, together with associated substrate characteristics. Pelleting increases the digestibility of unpretreated corn stover, from 8.2 to 15.5% glucan conversion, at 5% solid loading using 1 FPU Cellic® CTec2 per g solids. Compositional analysis indicates that loose and pelleted corn stover have similar non-dissolvable compositions, although their extractives are different. Enzymatic hydrolysis of corn stover after size reduction to normalize particle sizes and removal of extractives confirms that pelleting improves corn stover digestibility. Such differences may be explained by the decreased particle size, improved substrate accessibility, and hydrolysis of cross-linking structures induced by pelleting. These findings are useful for the development of processing schemes for sustainable and efficient use of lignocellulose.

On Krylov methods for large-scale CBCT reconstruction.

Krylov subspace methods are a powerful family of iterative solvers for linear systems of equations, which are commonly used for inverse problems due to their intrinsic regularization properties. Moreover, these methods are naturally suited to solve large-scale problems, as they only require matrix-vector products with the system matrix (and its adjoint) to compute approximate solutions, and they display a very fast convergence. Even if this class of methods has been widely researched and studied in the numerical linear algebra community, its use in applied medical physics and applied engineering is still very limited. e.g. in realistic large-scale computed tomography (CT) problems, and more specifically in cone beam CT (CBCT). This work attempts to breach this gap by providing a general framework for the most relevant Krylov subspace methods applied to 3D CT problems, including the most well-known Krylov solvers for non-square systems (CGLS, LSQR, LSMR), possibly in combination with Tikhonov regularization, and methods that incorporate total variation regularization. This is provided within an open source framework: the tomographic iterative GPU-based reconstruction toolbox, with the idea of promoting accessibility and reproducibility of the results for the algorithms presented. Finally, numerical results in synthetic and real-world 3D CT applications (medical CBCT andµ-CT datasets) are provided to showcase and compare the different Krylov subspace methods presented in the paper, as well as their suitability for different kinds of problems.

Screening method for Enzyme-based liquefaction of corn stover pellets at high solids.

Liquefaction of high solid loadings of unpretreated corn stover pellets has been demonstrated with rheology of the resulting slurries enabling mixing and movement within biorefinery bioreactors. However, some forms of pelleted stover do not readily liquefy, so it is important to screen out lots of unsuitable pellets before processing is initiated. This work reports a laboratory assay that rapidly assesses whether pellets have the potential for enzyme-based liquefaction at high solids loadings. Twenty-eight pelleted corn stover (harvested at the same time and location) were analyzed using 20 mL enzyme solutions (3 FPU cellulase/ g biomass) at 30 % w/v solids loading. Imaging together with measurement of reducing sugars were performed over 24-hours. Some samples formed concentrated slurries of 300 mg/mL (dry basis) in the small-scale assay, which was later confirmed in an agitated bioreactor. Also, the laboratory assay showed potential for optimizing enzyme formulations that could be employed for slurry formation.

New strategy for liquefying corn stover pellets.

The movement of solid material into and between unit operations within a biorefinery is a bottleneck in reaching design capacity, with formation of biomass slurries needed to introduce feedstock. Corn stover slurries have been achieved from dilute acid, pretreated materials resulting in slurry concentrations of up to about 150 g/L, above which flowability is compromised. We report a new strategy to liquefy corn stover at higher solids concentration (300 g/L) by initially cooking it with the enzyme mimetic maleic acid at 40 mM and 150 °C. This is followed by 6 h of enzymatic modification at 1 FPU (2.2 mg protein)/g solids, resulting in a yield stress of 171 Pa after 6 h and 58 Pa in 48 h compared to 6806 Pa for untreated stover. Mimetic treatment of corn stover pellets minimizes the inhibitory effect of xylo-oligomers on hydrolytic enzymes. This strategy allows for the delivery of solid lignocellulosic slurry into a pretreatment reactor by pumping, improving operability of a biorefinery.

Aß-induced vulnerability propagates via the brain's default mode network.

The link between brain amyloid-ß (Aß), metabolism, and dementia symptoms remains a pressing question in Alzheimer's disease. Here, using positron emission tomography ([18F]florbetapir tracer for Aß and [18F]FDG tracer for glucose metabolism) with a novel analytical framework, we found that Aß aggregation within the brain's default mode network leads to regional hypometabolism in distant but functionally connected brain regions. Moreover, we found that an interaction between this hypometabolism with overlapping Aß aggregation is associated with subsequent cognitive decline. These results were also observed in transgenic Aß rats that do not form neurofibrillary tangles, which support these findings as an independent mechanism of cognitive deterioration. These results suggest a model in which distant Aß induces regional metabolic vulnerability, whereas the interaction between local Aß with a vulnerable environment drives the clinical progression of dementia.

Publisher Correction: Accurate autocorrelation modeling substantially improves fMRI reliability.

The original HTML version of this Article had an incorrect Published online date of 25 December 2019; it should have been 21 March 2019. This has been corrected in the HTML version of the Article. The PDF version was correct from the time of publication.

Accurate autocorrelation modeling substantially improves fMRI reliability.

Given the recent controversies in some neuroimaging statistical methods, we compare the most frequently used functional Magnetic Resonance Imaging (fMRI) analysis packages: AFNI, FSL and SPM, with regard to temporal autocorrelation modeling. This process, sometimes known as pre-whitening, is conducted in virtually all task fMRI studies. Here, we employ eleven datasets containing 980 scans corresponding to different fMRI protocols and subject populations. We found that autocorrelation modeling in AFNI, although imperfect, performed much better than the autocorrelation modeling of FSL and SPM. The presence of residual autocorrelated noise in FSL and SPM leads to heavily confounded first level results, particularly for low-frequency experimental designs. SPM's alternative pre-whitening method, FAST, performed better than SPM's default. The reliability of task fMRI studies could be improved with more accurate autocorrelation modeling. We recommend that fMRI analysis packages provide diagnostic plots to make users aware of any pre-whitening problems.

A Functional Approach to Deconvolve Dynamic Neuroimaging Data.

Positron emission tomography (PET) is an imaging technique which can be used to investigate chemical changes in human biological processes such as cancer development or neurochemical reactions. Most dynamic PET scans are currently analyzed based on the assumption that linear first-order kinetics can be used to adequately describe the system under observation. However, there has recently been strong evidence that this is not the case. To provide an analysis of PET data which is free from this compartmental assumption, we propose a nonparametric deconvolution and analysis model for dynamic PET data based on functional principal component analysis. This yields flexibility in the possible deconvolved functions while still performing well when a linear compartmental model setup is the true data generating mechanism. As the deconvolution needs to be performed on only a relative small number of basis functions rather than voxel by voxel in the entire three-dimensional volume, the methodology is both robust to typical brain imaging noise levels while also being computationally efficient. The new methodology is investigated through simulations in both one-dimensional functions and 2D images and also applied to a neuroimaging study whose goal is the quantification of opioid receptor concentration in the brain.

Monotonicity of fitness landscapes and mutation rate control.

A common view in evolutionary biology is that mutation rates are minimised. However, studies in combinatorial optimisation and search have shown a clear advantage of using variable mutation rates as a control parameter to optimise the performance of evolutionary algorithms. Much biological theory in this area is based on Ronald Fisher's work, who used Euclidean geometry to study the relation between mutation size and expected fitness of the offspring in infinite phenotypic spaces. Here we reconsider this theory based on the alternative geometry of discrete and finite spaces of DNA sequences. First, we consider the geometric case of fitness being isomorphic to distance from an optimum, and show how problems of optimal mutation rate control can be solved exactly or approximately depending on additional constraints of the problem. Then we consider the general case of fitness communicating only partial information about the distance. We define weak monotonicity of fitness landscapes and prove that this property holds in all landscapes that are continuous and open at the optimum. This theoretical result motivates our hypothesis that optimal mutation rate functions in such landscapes will increase when fitness decreases in some neighbourhood of an optimum, resembling the control functions derived in the geometric case. We test this hypothesis experimentally by analysing approximately optimal mutation rate control functions in 115 complete landscapes of binding scores between DNA sequences and transcription factors. Our findings support the hypothesis and find that the increase of mutation rate is more rapid in landscapes that are less monotonic (more rugged). We discuss the relevance of these findings to living organisms.

Degradation of phenolic compounds by the lignocellulose deconstructing thermoacidophilic bacterium Alicyclobacillus Acidocaldarius.

Alicyclobacillus acidocaldarius, a thermoacidophilic bacterium, has a repertoire of thermo- and acid-stable enzymes that deconstruct lignocellulosic compounds. The work presented here describes the ability of A. acidocaldarius to reduce the concentration of the phenolic compounds: phenol, ferulic acid, ρ-coumaric acid and sinapinic acid during growth conditions. The extent and rate of the removal of these compounds were significantly increased by the presence of micro-molar copper concentrations, suggesting activity by copper oxidases that have been identified in the genome of A. acidocaldarius. Substrate removal kinetics was first order for phenol, ferulic acid, ρ-coumaric acid and sinapinic acid in the presence of 50 µM copper sulfate. In addition, laccase enzyme assays of cellular protein fractions suggested significant activity on a lignin analog between the temperatures of 45 and 90 °C. This work shows the potential for A. acidocaldarius to degrade phenolic compounds, demonstrating potential relevance to biofuel production and other industrial processes.

Estimating the population local wavelet spectrum with application to non-stationary functional magnetic resonance imaging time series.

Functional magnetic resonance imaging (fMRI) is a dynamic four-dimensional imaging modality. However, in almost all fMRI analyses, the time series elements of this data are assumed to be second-order stationary. In this paper, we examine, using time series spectral methods, whether such stationary assumptions can be made and whether estimates of non-stationarity can be used to gain understanding into fMRI experiments. A non-stationary version of replicated stationary time series analysis is proposed that takes into account the replicated time series that are available from nearby voxels in a region of interest (ROI). These are used to investigate non-stationarities in both the ROI itself and the variations within the ROI. The proposed techniques are applied to simulated data and to an anxiety-inducing fMRI experiment.

Adaptive Multivariate Global Testing.

We present a methodology for dealing with recent challenges in testing global hypotheses using multivariate observations. The proposed tests target situations, often arising in emerging applications of neuroimaging, where the sample size n is relatively small compared with the observations' dimension K. We employ adaptive designs allowing for sequential modifications of the test statistics adapting to accumulated data. The adaptations are optimal in the sense of maximizing the predictive power of the test at each interim analysis while still controlling the Type I error. Optimality is obtained by a general result applicable to typical adaptive design settings. Further, we prove that the potentially high-dimensional design space of the tests can be reduced to a low-dimensional projection space enabling us to perform simpler power analysis studies, including comparisons to alternative tests. We illustrate the substantial improvement in efficiency that the proposed tests can make over standard tests, especially in the case of n smaller or slightly larger than K. The methods are also studied empirically using both simulated data and data from an EEG study, where the use of prior knowledge substantially increases the power of the test. Supplementary materials for this article are available online.

Mutation rate plasticity in rifampicin resistance depends on Escherichia coli cell-cell interactions.

Variation of mutation rate at a particular site in a particular genotype, in other words mutation rate plasticity (MRP), can be caused by stress or ageing. However, mutation rate control by other factors is less well characterized. Here we show that in wild-type Escherichia coli (K-12 and B strains), the mutation rate to rifampicin resistance is plastic and inversely related to population density: lowering density can increase mutation rates at least threefold. This MRP is genetically switchable, dependent on the quorum-sensing gene luxS--specifically its role in the activated methyl cycle--and is socially mediated via cell-cell interactions. Although we identify an inverse association of mutation rate with fitness under some circumstances, we find no functional link with stress-induced mutagenesis. Our experimental manipulation of mutation rates via the social environment raises the possibility that such manipulation occurs in nature and could be exploited medically.

Reprint of "Pharmacokinetic modelling of the anti-malarial drug artesunate and its active metabolite dihydroartemisinin".

A four compartment mechanistic mathematical model is developed for the pharmacokinetics of the commonly used anti-malarial drug artesunate and its principle metabolite dihydroartemisinin following oral administration of artesunate. The model is structurally unidentifiable unless additional constraints are imposed. Combinations of mechanistically derived constraints are considered to assess their effects on structural identifiability and on model fits. Certain combinations of the constraints give rise to locally or globally identifiable model structures. Initial validation of the model under various combinations of the constraints leading to identifiable model structures was performed against a dataset of artesunate and dihydroartemisinin concentration-time profiles of 19 malaria patients. When all the discussed constraints were imposed on the model, the resulting globally identifiable model structure was found to fit reasonably well to those patients with normal drug absorption profiles. However, there is wide variability in the fitted parameters and further investigation is warranted.

Where antibiotic resistance mutations meet quorum-sensing.

We do not need to rehearse the grim story of the global rise of antibiotic resistant microbes. But what if it were possible to control the rate with which antibiotic resistance evolves by de novo mutation? It seems that some bacteria may already do exactly that: they modify the rate at which they mutate to antibiotic resistance dependent on their biological environment. In our recent study [Krasovec, et al. Nat. Commun. (2014), 5, 3742] we find that this modification depends on the density of the bacterial population and cell-cell interactions (rather than, for instance, the level of stress). Specifically, the wild-type strains of Escherichia coli we used will, in minimal glucose media, modify their rate of mutation to rifampicin resistance according to the density of wild-type cells. Intriguingly, the higher the density, the lower the mutation rate (Figure 1). Why this novel density-dependent 'mutation rate plasticity' (DD-MRP) occurs is a question at several levels. Answers are currently fragmentary, but involve the quorum-sensing gene luxS and its role in the activated methyl cycle.

Pharmacokinetic modelling of the anti-malarial drug artesunate and its active metabolite dihydroartemisinin.

A four compartment mechanistic mathematical model is developed for the pharmacokinetics of the commonly used anti-malarial drug artesunate and its principle metabolite dihydroartemisinin following oral administration of artesunate. The model is structurally unidentifiable unless additional constraints are imposed. Combinations of mechanistically derived constraints are considered to assess their effects on structural identifiability and on model fits. Certain combinations of the constraints give rise to locally or globally identifiable model structures. Initial validation of the model under various combinations of the constraints leading to identifiable model structures was performed against a dataset of artesunate and dihydroartemisinin concentration-time profiles of 19 malaria patients. When all the discussed constraints were imposed on the model, the resulting globally identifiable model structure was found to fit reasonably well to those patients with normal drug absorption profiles. However, there is wide variability in the fitted parameters and further investigation is warranted.