Q-Learning to navigate turbulence without a map.

We consider the problem of olfactory searches in a turbulent environment. We focus on agents that respond solely to odor stimuli, with no access to spatial perception nor prior information about the odor location. We ask whether navigation strategies to a target can be learned robustly within a sequential decision making framework. We develop a reinforcement learning algorithm using a small set of interpretable olfactory states and train it with realistic turbulent odor cues. By introducing a temporal memory, we demonstrate that two salient features of odor traces, discretized in few olfactory states, are sufficient to learn navigation in a realistic odor plume. Performance is dictated by the sparse nature of turbulent plumes. An optimal memory exists which ignores blanks within the plume and activates a recovery strategy outside the plume. We obtain the best performance by letting agents learn their recovery strategy and show that it is mostly casting cross wind, similar to behavior observed in flying insects. The optimal strategy is robust to substantial changes in the odor plumes, suggesting minor parameter tuning may be sufficient to adapt to different environments.

Transplantation Induces Profound Changes in the Transcriptional Asset of Hematopoietic Stem Cells: Identification of Specific Signatures Using Machine Learning Techniques.

During the phase of proliferation needed for hematopoietic reconstitution following transplantation, hematopoietic stem/progenitor cells (HSPC) must express genes involved in stem cell self-renewal. We investigated the expression of genes relevant for self-renewal and expansion of HSPC (operationally defined as CD34+ cells) in steady state and after transplantation. Specifically, we evaluated the expression of ninety-one genes that were analyzed by real-time PCR in CD34+ cells isolated from (i) 12 samples from umbilical cord blood (UCB); (ii) 15 samples from bone marrow healthy donors; (iii) 13 samples from bone marrow after umbilical cord blood transplant (UCBT); and (iv) 29 samples from patients after transplantation with adult hematopoietic cells. The results show that transplanted CD34+ cells from adult cells acquire an asset very different from transplanted CD34+ cells from cord blood. Multivariate machine learning analysis (MMLA) showed that four specific gene signatures can be obtained by comparing the four types of CD34+ cells. In several, but not all cases, transplanted HSPC from UCB overexpress reprogramming genes. However, these remarkable changes do not alter the commitment to hematopoietic lineage. Overall, these results reveal undisclosed aspects of transplantation biology.

The hidden information in patient-reported outcomes and clinician-assessed outcomes: multiple sclerosis as a proof of concept of a machine learning approach.

Machine learning (ML) applied to patient-reported (PROs) and clinical-assessed outcomes (CAOs) could favour a more predictive and personalized medicine. Our aim was to confirm the important role of applying ML to PROs and CAOs of people with relapsing-remitting (RR) and secondary progressive (SP) form of multiple sclerosis (MS), to promptly identifying information useful to predict disease progression. For our analysis, a dataset of 3398 evaluations from 810 persons with MS (PwMS) was adopted. Three steps were provided: course classification; extraction of the most relevant predictors at the next time point; prediction if the patient will experience the transition from RR to SP at the next time point. The Current Course Assignment (CCA) step correctly assigned the current MS course with an accuracy of about 86.0%. The MS course at the next time point can be predicted using the predictors selected in CCA. PROs/CAOs Evolution Prediction (PEP) followed by Future Course Assignment (FCA) was able to foresee the course at the next time point with an accuracy of 82.6%. Our results suggest that PROs and CAOs could help the clinician decision-making in their practice.

Data-driven strategies for robust forecast of continuous glucose monitoring time-series.

Over the past decade, continuous glucose monitoring (CGM) has proven to be a very resourceful tool for diabetes management. To date, CGM devices are employed for both retrospective and online applications. Their use allows to better describe the patients' pathology as well as to achieve a better control of patients' level of glycemia. The analysis of CGM sensor data makes possible to observe a wide range of metrics, such as the glycemic variability during the day or the amount of time spent below or above certain glycemic thresholds. However, due to the high variability of the glycemic signals among sensors and individuals, CGM data analysis is a non-trivial task. Standard signal filtering solutions fall short when an appropriate model personalization is not applied. State-of-the-art data-driven strategies for online CGM forecasting rely upon the use of recursive filters. Each time a new sample is collected, such models need to adjust their parameters in order to predict the next glycemic level. In this paper we aim at demonstrating that the problem of online CGM forecasting can be successfully tackled by personalized machine learning models, that do not need to recursively update their parameters.

Enhancing Interpretability of Gene Signatures with Prior Biological Knowledge.

Biological interpretability is a key requirement for the output of microarray data analysis pipelines. The most used pipeline first identifies a gene signature from the acquired measurements and then uses gene enrichment analysis as a tool for functionally characterizing the obtained results. Recently Knowledge Driven Variable Selection (KDVS), an alternative approach which performs both steps at the same time, has been proposed. In this paper, we assess the effectiveness of KDVS against standard approaches on a Parkinson's Disease (PD) dataset. The presented quantitative analysis is made possible by the construction of a reference list of genes and gene groups associated to PD. Our work shows that KDVS is much more effective than the standard approach in enhancing the interpretability of the obtained results.

Genome instability model of metastatic neuroblastoma tumorigenesis by a dictionary learning algorithm.

BACKGROUND: Metastatic neuroblastoma (NB) occurs in pediatric patients as stage 4S or stage 4 and it is characterized by heterogeneous clinical behavior associated with diverse genotypes. Tumors of stage 4 contain several structural copy number aberrations (CNAs) rarely found in stage 4S. To date, the NB tumorigenesis is not still elucidated, although it is evident that genomic instability plays a critical role in the genesis of the tumor. Here we propose a mathematical approach to decipher genomic data and we provide a new model of NB metastatic tumorigenesis. METHOD: We elucidate NB tumorigenesis using Enhanced Fused Lasso Latent Feature Model (E-FLLat) modeling the array comparative chromosome hybridization (aCGH) data of 190 metastatic NBs (63 stage 4S and 127 stage 4). This model for aCGH segmentation, based on the minimization of functional dictionary learning (DL), combines several penalties tailored to the specificities of aCGH data. In DL, the original signal is approximated by a linear weighted combination of atoms: the elements of the learned dictionary. RESULTS: The hierarchical structures for stage 4S shows at the first level of the oncogenetic tree several whole chromosome gains except to the unbalanced gains of 17q, 2p and 2q. Conversely, the high CNA complexity found in stage 4 tumors, requires two different trees. Both stage 4 oncogenetic trees are marked diverged, up to five sublevels and the 17q gain is the most common event at the first level (2/3 nodes). Moreover the 11q deletion, one of the major unfavorable marker of disease progression, occurs before 3p loss indicating that critical chromosome aberrations appear at early stages of tumorigenesis. Finally, we also observed a significant (p = 0.025) association between patient age and chromosome loss in stage 4 cases. CONCLUSION: These results led us to propose a genome instability progressive model in which NB cells initiate with a DNA synthesis uncoupled from cell division, that leads to stage 4S tumors, primarily characterized by numerical aberrations, or stage 4 tumors with high levels of genome instability resulting in complex chromosome rearrangements associated with high tumor aggressiveness and rapid disease progression.

Online space-variant background modeling with sparse coding.

In this paper, we propose a sparse coding approach to background modeling. The obtained model is based on dictionaries which we learn and keep up to date as new data are provided by a video camera. We observe that, without dynamic events, video frames may be seen as noisy data belonging to the background. Over time, such background is subject to local and global changes due to variable illumination conditions, camera jitter, stable scene changes, and intermittent motion of background objects. To capture the locality of some changes, we propose a space-variant analysis where we learn a dictionary of atoms for each image patch, the size of which depends on the background variability. At run time, each patch is represented by a linear combination of the atoms learnt online. A change is detected when the atoms are not sufficient to provide an appropriate representation, and stable changes over time trigger an update of the current dictionary. Even if the overall procedure is carried out at a coarse level, a pixel-wise segmentation can be obtained by comparing the atoms with the patch corresponding to the dynamic event. Experiments on benchmarks indicate that the proposed method achieves very good performances on a variety of scenarios. An assessment on long video streams confirms our method incorporates periodical changes, as the ones caused by variations in natural illumination. The model, fully data driven, is suitable as a main component of a change detection system.

Advances in dynamic modeling of colorectal cancer signaling-network regions, a path toward targeted therapies.

The interconnected network of pathways downstream of the TGFß, WNT and EGF-families of receptor ligands play an important role in colorectal cancer pathogenesis.We studied and implemented dynamic simulations of multiple downstream pathways and described the section of the signaling network considered as a Molecular Interaction Map (MIM). Our simulations used Ordinary Differential Equations (ODEs), which involved 447 reactants and their interactions.Starting from an initial "physiologic condition", the model can be adapted to simulate individual pathologic cancer conditions implementing alterations/mutations in relevant onco-proteins. We verified some salient model predictions using the mutated colorectal cancer lines HCT116 and HT29. We measured the amount of MYC and CCND1 mRNAs and AKT and ERK phosphorylated proteins, in response to individual or combination onco-protein inhibitor treatments. Experimental and simulation results were well correlated. Recent independently published results were also predicted by our model.Even in the presence of an approximate and incomplete signaling network information, a predictive dynamic modeling seems already possible. An important long term road seems to be open and can be pursued further, by incremental steps, toward even larger and better parameterized MIMs. Personalized treatment strategies with rational associations of signaling-proteins inhibitors, could become a realistic goal.

A machine learning pipeline for multiple sclerosis course detection from clinical scales and patient reported outcomes.

In this work we present a machine learning pipeline for the detection of multiple sclerosis course from a collection of inexpensive and non-invasive measures such as clinical scales and patient-reported outcomes. The proposed analysis is conducted on a dataset coming from a clinical study comprising 457 patients affected by multiple sclerosis. The 91 collected variables describe patients mobility, fatigue, cognitive performance, emotional status, bladder continence and quality of life. A preliminary data exploration phase suggests that the group of patients diagnosed as Relapsing-Remitting can be isolated from other clinical courses. Supervised learning algorithms are then applied to perform feature selection and course classification. Our results confirm that clinical scales and patient-reported outcomes can be used to classify Relapsing-Remitting patients.

Alternating proximal regularized dictionary learning.

We present an algorithm for dictionary learning that is based on the alternating proximal algorithm studied by Attouch, Bolte, Redont, and Soubeyran (2010), coupled with a reliable and efficient dual algorithm for computation of the related proximity operators. This algorithm is suitable for a general dictionary learning model composed of a Bregman-type data fit term that accounts for the goodness of the representation and several convex penalization terms on the coefficients and atoms, explaining the prior knowledge at hand. As Attouch et al. recently proved, an alternating proximal scheme ensures better convergence properties than the simpler alternating minimization. We take care of the issue of inexactness in the computation of the involved proximity operators, giving a sound stopping criterion for the dual inner algorithm, which keeps under control the related errors, unavoidable for such a complex penalty terms, providing ultimately an overall effective procedure. Thanks to the generality of the proposed framework, we give an application in the context of genome-wide data understanding, revising the model proposed by Nowak, Hastie, Pollack, and Tibshirani (2011). The aim is to extract latent features (atoms) and perform segmentation on array-based comparative genomic hybridization (aCGH) data. We improve several important aspects that increase the quality and interpretability of the results. We show the effectiveness of the proposed model with two experiments on synthetic data, which highlight the enhancements over the original model.

Unsupervised tissue segmentation from dynamic contrast-enhanced magnetic resonance imaging.

OBJECTIVE: Design, implement, and validate an unsupervised method for tissue segmentation from dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). METHODS: For each DCE-MRI acquisition, after a spatial registration phase, the time-varying intensity of each voxel is represented as a sparse linear combination of adaptive basis signals. Both the basis signals and the sparse coefficients are learned by minimizing a functional consisting of a data fidelity term and a sparsity inducing penalty. Tissue segmentation is then obtained by applying a standard clustering algorithm to the computed representation. RESULTS: Quantitative estimates on two real data sets are presented. In the first case, the overlap with expert annotation measured with the DICE metric is nearly 90% and thus 5% more accurate than state-of-the-art techniques. In the second case, assessment of the correlation between quantitative scores, obtained by the proposed method against imagery manually annotated by two experts, achieved a Pearson coefficient of 0.83 and 0.87, and a Spearman coefficient of 0.83 and 0.71, respectively. CONCLUSIONS: The sparse representation of DCE MRI signals obtained by means of adaptive dictionary learning techniques appears to be well-suited for unsupervised tissue segmentation and applicable to different clinical contexts with little effort.

Dictionary learning improves subtyping of breast cancer aCGH data.

The advent of Comparative Genomic Hybridization (CGH) data led to the development of new mathematical models and computational methods to automatically infer chromosomal alterations. In this work we tackle a standard clustering problem exploiting the good representation properties of a novel method based on dictionary learning. The identified dictionary atoms, which show co-occuring shared alterations among samples, can be easily interpreted by domain experts. We compare a state-of-the-art approach with an original method on a breast cancer dataset.

Molecular fingerprinting reflects different histotypes and brain region in low grade gliomas.

BACKGROUND: Paediatric low-grade gliomas (LGGs) encompass a heterogeneous set of tumours of different histologies, site of lesion, age and gender distribution, growth potential, morphological features, tendency to progression and clinical course. Among LGGs, Pilocytic astrocytomas (PAs) are the most common central nervous system (CNS) tumours in children. They are typically well-circumscribed, classified as grade I by the World Health Organization (WHO), but recurrence or progressive disease occurs in about 10-20% of cases. Despite radiological and neuropathological features deemed as classic are acknowledged, PA may present a bewildering variety of microscopic features. Indeed, tumours containing both neoplastic ganglion and astrocytic cells occur at a lower frequency. METHODS: Gene expression profiling on 40 primary LGGs including PAs and mixed glial-neuronal tumours comprising gangliogliomas (GG) and desmoplastic infantile gangliogliomas (DIG) using Affymetrix array platform was performed. A biologically validated machine learning workflow for the identification of microarray-based gene signatures was devised. The method is based on a sparsity inducing regularization algorithm l1l2 that selects relevant variables and takes into account their correlation. The most significant genetic signatures emerging from gene-chip analysis were confirmed and validated by qPCR. RESULTS: We identified an expression signature composed by a biologically validated list of 15 genes, able to distinguish infratentorial from supratentorial LGGs. In addition, a specific molecular fingerprinting distinguishes the supratentorial PAs from those originating in the posterior fossa. Lastly, within supratentorial tumours, we also identified a gene expression pattern composed by neurogenesis, cell motility and cell growth genes which dichotomize mixed glial-neuronal tumours versus PAs. Our results reinforce previous observations about aberrant activation of the mitogen-activated protein kinase (MAPK) pathway in LGGs, but still point to an active involvement of TGF-beta signaling pathway in the PA development and pick out some hitherto unreported genes worthy of further investigation for the mixed glial-neuronal tumours. CONCLUSIONS: The identification of a brain region-specific gene signature suggests that LGGs, with similar pathological features but located at different sites, may be distinguishable on the basis of cancer genetics. Molecular fingerprinting seems to be able to better sub-classify such morphologically heterogeneous tumours and it is remarkable that mixed glial-neuronal tumours are strikingly separated from PAs.

Knowledge Driven Variable Selection (KDVS) - a new approach to enrichment analysis of gene signatures obtained from high-throughput data.

BACKGROUND: High-throughput (HT) technologies provide huge amount of gene expression data that can be used to identify biomarkers useful in the clinical practice. The most frequently used approaches first select a set of genes (i.e. gene signature) able to characterize differences between two or more phenotypical conditions, and then provide a functional assessment of the selected genes with an a posteriori enrichment analysis, based on biological knowledge. However, this approach comes with some drawbacks. First, gene selection procedure often requires tunable parameters that affect the outcome, typically producing many false hits. Second, a posteriori enrichment analysis is based on mapping between biological concepts and gene expression measurements, which is hard to compute because of constant changes in biological knowledge and genome analysis. Third, such mapping is typically used in the assessment of the coverage of gene signature by biological concepts, that is either score-based or requires tunable parameters as well, limiting its power. RESULTS: We present Knowledge Driven Variable Selection (KDVS), a framework that uses a priori biological knowledge in HT data analysis. The expression data matrix is transformed, according to prior knowledge, into smaller matrices, easier to analyze and to interpret from both computational and biological viewpoints. Therefore KDVS, unlike most approaches, does not exclude a priori any function or process potentially relevant for the biological question under investigation. Differently from the standard approach where gene selection and functional assessment are applied independently, KDVS embeds these two steps into a unified statistical framework, decreasing the variability derived from the threshold-dependent selection, the mapping to the biological concepts, and the signature coverage. We present three case studies to assess the usefulness of the method. CONCLUSIONS: We showed that KDVS not only enables the selection of known biological functionalities with accuracy, but also identification of new ones. An efficient implementation of KDVS was devised to obtain results in a fast and robust way. Computing time is drastically reduced by the effective use of distributed resources. Finally, integrated visualization techniques immediately increase the interpretability of results. Overall, KDVS approach can be considered as a viable alternative to enrichment-based approaches.

Parameter space exploration within dynamic simulations of signaling networks.

We started offering an introduction to very basic aspects of molecular biology, for the reader coming from computer sciences, information technology, mathematics. Similarly we offered a minimum of information about pathways and networks in graph theory, for a reader coming from the bio-medical sector. At the crossover about the two different types of expertise, we offered some definition about Systems Biology. The core of the article deals with a Molecular Interaction Map (MIM), a network of biochemical interactions involved in a small signaling-network sub-region relevant in breast cancer. We explored robustness/sensitivity to random perturbations. It turns out that our MIM is a non-isomorphic directed graph. For non physiological directions of propagation of the signal the network is quite resistant to perturbations. The opposite happens for biologically significant directions of signal propagation. In these cases we can have no signal attenuation, and even signal amplification. Signal propagation along a given pathway is highly unidirectional, with the exception of signal-feedbacks, that again have a specific biological role and significance. In conclusion, even a relatively small network like our present MIM reveals the preponderance of specific biological functions over unspecific isomorphic behaviors. This is perhaps the consequence of hundreds of millions of years of biological evolution.

SVS: data and knowledge integration in computational biology.

In this paper we present a framework for structured variable selection (SVS). The main concept of the proposed schema is to take a step towards the integration of two different aspects of data mining: database and machine learning perspective. The framework is flexible enough to use not only microarray data, but other high-throughput data of choice (e.g. from mass spectrometry, microarray, next generation sequencing). Moreover, the feature selection phase incorporates prior biological knowledge in a modular way from various repositories and is ready to host different statistical learning techniques. We present a proof of concept of SVS, illustrating some implementation details and describing current results on high-throughput microarray data.

Identification of multiple hypoxia signatures in neuroblastoma cell lines by l1-l2 regularization and data reduction.

Hypoxia is a condition of low oxygen tension occurring in the tumor and negatively correlated with the progression of the disease. We studied the gene expression profiles of nine neuroblastoma cell lines grown under hypoxic conditions to define gene signatures that characterize hypoxic neuroblastoma. The l(1)-l(2) regularization applied to the entire transcriptome identified a single signature of 11 probesets discriminating the hypoxic state. We demonstrate that new hypoxia signatures, with similar discriminatory power, can be generated by a prior knowledge-based filtering in which a much smaller number of probesets, characterizing hypoxia-related biochemical pathways, are analyzed. l(1)-l(2) regularization identified novel and robust hypoxia signatures within apoptosis, glycolysis, and oxidative phosphorylation Gene Ontology classes. We conclude that the filtering approach overcomes the noisy nature of the microarray data and allows generating robust signatures suitable for biomarker discovery and patients risk assessment in a fraction of computer time.

A biology-driven approach identifies the hypoxia gene signature as a predictor of the outcome of neuroblastoma patients.

BACKGROUND: Hypoxia is a condition of low oxygen tension occurring in the tumor microenvironment and it is related to poor prognosis in human cancer. To examine the relationship between hypoxia and neuroblastoma, we generated and tested an in vitro derived hypoxia gene signature for its ability to predict patients' outcome. RESULTS: We obtained the gene expression profile of 11 hypoxic neuroblastoma cell lines and we derived a robust 62 probesets signature (NB-hypo) taking advantage of the strong discriminating power of the l1-l2 feature selection technique combined with the analysis of differential gene expression. We profiled gene expression of the tumors of 88 neuroblastoma patients and divided them according to the NB-hypo expression values by K-means clustering. The NB-hypo successfully stratifies the neuroblastoma patients into good and poor prognosis groups. Multivariate Cox analysis revealed that the NB-hypo is a significant independent predictor after controlling for commonly used risk factors including the amplification of MYCN oncogene. NB-hypo increases the resolution of the MYCN stratification by dividing patients with MYCN not amplified tumors in good and poor outcome suggesting that hypoxia is associated with the aggressiveness of neuroblastoma tumor independently from MYCN amplification. CONCLUSIONS: Our results demonstrate that the NB-hypo is a novel and independent prognostic factor for neuroblastoma and support the view that hypoxia is negatively correlated with tumors' outcome. We show the power of the biology-driven approach in defining hypoxia as a critical molecular program in neuroblastoma and the potential for improvement in the current criteria for risk stratification.

Dynamic contrast-enhanced magnetic resonance imaging in the assessment of disease activity in patients with juvenile idiopathic arthritis.

OBJECTIVE: To determine the capability and reliability of dynamic contrast-enhanced MRI (DCE-MRI) in the assessment of disease activity in juvenile idiopathic arthritis (JIA). METHODS: DCE-MRI of the clinically more affected wrist or hip joints was undertaken in 21 patients, coupled with standard clinical assessment and biochemical analysis. Synovial inflammation was assessed by computing the maximum level of synovial enhancement (ME), the maximum rate of enhancement (MV) and the rate of early enhancement (REE) from the enhancement curves generated from region of interest independently delineated by two readers in the area of the ME. Correlations between dynamic parameters and clinical measures of disease activity, and static MRI synovitis score were investigated. RESULTS: In patients with wrist arthritis, REE correlated with the wrist swelling score (r(s) = 0.72), ESR (r(s) = 0.69), pain assessment scale (r(s) = 0.63) and childhood HAQ (r(s) = 0.60). In patients with hip arthritis, ME correlated with the hip limitation of motion (r(s) = 0.69). Static MRI synovitis score based on post-gadolinium enhancement correlated with MV (r(s) = 0.63) in patients with wrist arthritis and with ME (r = 0.68) in those with hip arthritis. The inter-reader agreement assessed by intra-class correlation coefficient (ICC) for ME, MV and REE (ICC = 0.98, 0.97 and 0.84, respectively) was excellent. CONCLUSIONS: DCE-MRI represents a promising method for the assessment of disease activity in JIA, especially in patients with wrist arthritis. As far as we know, this study is the first to demonstrate the feasibility, reliability and construct validity of DCE-MRI in JIA. These results should be confirmed in large-scale longitudinal studies in view of its further application in therapeutic decision making and in clinical trials.

The l1-l2 regularization framework unmasks the hypoxia signature hidden in the transcriptome of a set of heterogeneous neuroblastoma cell lines.

BACKGROUND: Gene expression signatures are clusters of genes discriminating different statuses of the cells and their definition is critical for understanding the molecular bases of diseases. The identification of a gene signature is complicated by the high dimensional nature of the data and by the genetic heterogeneity of the responding cells. The l1-l2 regularization is an embedded feature selection technique that fulfills all the desirable properties of a variable selection algorithm and has the potential to generate a specific signature even in biologically complex settings. We studied the application of this algorithm to detect the signature characterizing the transcriptional response of neuroblastoma tumor cell lines to hypoxia, a condition of low oxygen tension that occurs in the tumor microenvironment. RESULTS: We determined the gene expression profile of 9 neuroblastoma cell lines cultured under normoxic and hypoxic conditions. We studied a heterogeneous set of neuroblastoma cell lines to mimic the in vivo situation and to test the robustness and validity of the l1-l2 regularization with double optimization. Analysis by hierarchical, spectral, and k-means clustering or supervised approach based on t-test analysis divided the cell lines on the bases of genetic differences. However, the disturbance of this strong transcriptional response completely masked the detection of the more subtle response to hypoxia. Different results were obtained when we applied the l1-l2 regularization framework. The algorithm distinguished the normoxic and hypoxic statuses defining signatures comprising 3 to 38 probesets, with a leave-one-out error of 17%. A consensus hypoxia signature was established setting the frequency score at 50% and the correlation parameter epsilon equal to 100. This signature is composed by 11 probesets representing 8 well characterized genes known to be modulated by hypoxia. CONCLUSION: We demonstrate that l1-l2 regularization outperforms more conventional approaches allowing the identification and definition of a gene expression signature under complex experimental conditions. The l1-l2 regularization and the cross validation generates an unbiased and objective output with a low classification error. We feel that the application of this algorithm to tumor biology will be instrumental to analyze gene expression signatures hidden in the transcriptome that, like hypoxia, may be major determinant of the course of the disease.