Seroprevalence of severe acute respiratory syndrome coronavirus 2 in Slovenia: results of two rounds of a nationwide population study on a probability-based sample, challenges and lessons learned.

OBJECTIVES: Seroprevalence surveys provide crucial information on cumulative severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) exposure. This Slovenian nationwide population study is the first longitudinal 6-month serosurvey using probability-based samples across all age categories. METHODS: Each participant supplied two blood samples: 1316 samples in April 2020 (first round) and 1211 in October/November 2020 (second round). The first-round sera were tested using Euroimmun Anti-SARS-CoV-2 ELISA IgG (ELISA) and, because of uncertain estimates, were retested using Elecsys Anti-SARS-CoV-2 (Elecsys-N) and Elecsys Anti-SARS-CoV-2 S (Elecsys-S). The second-round sera were concomitantly tested using Elecsys-N/Elecsys-S. RESULTS: The populations of both rounds matched the overall population (n = 3000), with minor settlement type and age differences. The first-round seroprevalence corrected for the ELISA manufacturer's specificity was 2.78% (95% highest density interval [HDI] 1.81%-3.80%), corrected using pooled ELISA specificity calculated from published data 0.93% (95% CI 0.00%-2.65%), and based on Elecsys-N/Elecsys-S results 0.87% (95% HDI 0.40%-1.38%). The second-round unadjusted lower limit of seroprevalence on 11 November 2020 was 4.06% (95% HDI 2.97%-5.16%) and on 3 October 2020, unadjusted upper limit was 4.29% (95% HDI 3.18%-5.47%). CONCLUSIONS: SARS-CoV-2 seroprevalence in Slovenia increased four-fold from late April to October/November 2020, mainly due to a devastating second wave. Significant logistic/methodological challenges accompanied both rounds. The main lessons learned were a need for caution when relying on manufacturer-generated assay evaluation data, the importance of multiple manufacturer-independent assay performance assessments, the need for concomitant use of highly-specific serological assays targeting different SARS-CoV-2 proteins in serosurveys conducted in low-prevalence settings or during epidemic exponential growth and the usefulness of a Bayesian approach for overcoming complex methodological challenges.

Hands-on training about overfitting.

Overfitting is one of the critical problems in developing models by machine learning. With machine learning becoming an essential technology in computational biology, we must include training about overfitting in all courses that introduce this technology to students and practitioners. We here propose a hands-on training for overfitting that is suitable for introductory level courses and can be carried out on its own or embedded within any data science course. We use workflow-based design of machine learning pipelines, experimentation-based teaching, and hands-on approach that focuses on concepts rather than underlying mathematics. We here detail the data analysis workflows we use in training and motivate them from the viewpoint of teaching goals. Our proposed approach relies on Orange, an open-source data science toolbox that combines data visualization and machine learning, and that is tailored for education in machine learning and explorative data analysis.

Democratized image analytics by visual programming through integration of deep models and small-scale machine learning.

Analysis of biomedical images requires computational expertize that are uncommon among biomedical scientists. Deep learning approaches for image analysis provide an opportunity to develop user-friendly tools for exploratory data analysis. Here, we use the visual programming toolbox Orange ( http://orange.biolab.si ) to simplify image analysis by integrating deep-learning embedding, machine learning procedures, and data visualization. Orange supports the construction of data analysis workflows by assembling components for data preprocessing, visualization, and modeling. We equipped Orange with components that use pre-trained deep convolutional networks to profile images with vectors of features. These vectors are used in image clustering and classification in a framework that enables mining of image sets for both novel and experienced users. We demonstrate the utility of the tool in image analysis of progenitor cells in mouse bone healing, identification of developmental competence in mouse oocytes, subcellular protein localization in yeast, and developmental morphology of social amoebae.

scOrange-a tool for hands-on training of concepts from single-cell data analytics.

MOTIVATION: Single-cell RNA sequencing allows us to simultaneously profile the transcriptomes of thousands of cells and to indulge in exploring cell diversity, development and discovery of new molecular mechanisms. Analysis of scRNA data involves a combination of non-trivial steps from statistics, data visualization, bioinformatics and machine learning. Training molecular biologists in single-cell data analysis and empowering them to review and analyze their data can be challenging, both because of the complexity of the methods and the steep learning curve. RESULTS: We propose a workshop-style training in single-cell data analytics that relies on an explorative data analysis toolbox and a hands-on teaching style. The training relies on scOrange, a newly developed extension of a data mining framework that features workflow design through visual programming and interactive visualizations. Workshops with scOrange can proceed much faster than similar training methods that rely on computer programming and analysis through scripting in R or Python, allowing the trainer to cover more ground in the same time-frame. We here review the design principles of the scOrange toolbox that support such workshops and propose a syllabus for the course. We also provide examples of data analysis workflows that instructors can use during the training. AVAILABILITY AND IMPLEMENTATION: scOrange is an open-source software. The software, documentation and an emerging set of educational videos are available at http://singlecell.biolab.si.

Combinatorial algorithm for counting small induced graphs and orbits.

Graphlet analysis is an approach to network analysis that is particularly popular in bioinformatics. We show how to set up a system of linear equations that relate the orbit counts and can be used in an algorithm that is significantly faster than the existing approaches based on direct enumeration of graphlets. The approach presented in this paper presents a generalization of the currently fastest method for counting 5-node graphlets in bioinformatics. The algorithm requires existence of a vertex with certain properties; we show that such vertex exists for graphlets of arbitrary size, except for complete graphs and a cycle with four nodes, which are treated separately. Empirical analysis of running time agrees with the theoretical results.

Assessment of machine learning reliability methods for quantifying the applicability domain of QSAR regression models.

The vastness of chemical space and the relatively small coverage by experimental data recording molecular properties require us to identify subspaces, or domains, for which we can confidently apply QSAR models. The prediction of QSAR models in these domains is reliable, and potential subsequent investigations of such compounds would find that the predictions closely match the experimental values. Standard approaches in QSAR assume that predictions are more reliable for compounds that are "similar" to those in subspaces with denser experimental data. Here, we report on a study of an alternative set of techniques recently proposed in the machine learning community. These methods quantify prediction confidence through estimation of the prediction error at the point of interest. Our study includes 20 public QSAR data sets with continuous response and assesses the quality of 10 reliability scoring methods by observing their correlation with prediction error. We show that these new alternative approaches can outperform standard reliability scores that rely only on similarity to compounds in the training set. The results also indicate that the quality of reliability scoring methods is sensitive to data set characteristics and to the regression method used in QSAR. We demonstrate that at the cost of increased computational complexity these dependencies can be leveraged by integration of scores from various reliability estimation approaches. The reliability estimation techniques described in this paper have been implemented in an open source add-on package ( https://bitbucket.org/biolab/orange-reliability ) to the Orange data mining suite.

A combinatorial approach to graphlet counting.

MOTIVATION: Small-induced subgraphs called graphlets are emerging as a possible tool for exploration of global and local structure of networks and for analysis of roles of individual nodes. One of the obstacles to their wider use is the computational complexity of algorithms for their discovery and counting. RESULTS: We propose a new combinatorial method for counting graphlets and orbit signatures of network nodes. The algorithm builds a system of equations that connect counts of orbits from graphlets with up to five nodes, which allows to compute all orbit counts by enumerating just a single one. This reduces its practical time complexity in sparse graphs by an order of magnitude as compared with the existing pure enumeration-based algorithms. AVAILABILITY AND IMPLEMENTATION: Source code is available freely at http://www.biolab.si/supp/orca/orca.html.

Classification of ischaemic episodes with ST/HR diagrams.

Coronary artery disease is the developed world's premier cause of mortality and the most probable cause of myocardial ischaemia. More advanced diagnostic tests aside, in electrocardiogram (ECG) analysis it manifests itself as a ST segment deviation, targeted by both exercise ECG and ambulatory ECG. In ambulatory ECG, besides ischaemic ST segment deviation episodes there are also non-ischaemic heart rate related episodes which aggravate real ischaemia detection. We present methods to transform the features developed for the heart rate adjustment of ST segment depression in exercise ECG for use in ambulatory ECG. We use annotations provided by the Long-Term ST Database to plot the ST/HR diagrams and then estimate the overall and maximal slopes of the diagrams in the exercise and recovery phase for each ST segment deviation episode. We also estimate the angle at the extrema of the ST/HR diagrams. Statistical analysis shows that ischaemic ST segment deviation episodes have significantly steeper overall and maximal slopes than heart rate related episodes, which indicates the explored features' utility for distinguishing between the two types of episodes. This makes the proposed features very useful in automated ECG analysis.

FragViz: visualization of fragmented networks.

BACKGROUND: Researchers in systems biology use network visualization to summarize the results of their analysis. Such networks often include unconnected components, which popular network alignment algorithms place arbitrarily with respect to the rest of the network. This can lead to misinterpretations due to the proximity of otherwise unrelated elements. RESULTS: We propose a new network layout optimization technique called FragViz which can incorporate additional information on relations between unconnected network components. It uses a two-step approach by first arranging the nodes within each of the components and then placing the components so that their proximity in the network corresponds to their relatedness. In the experimental study with the leukemia gene networks we demonstrate that FragViz can obtain network layouts which are more interpretable and hold additional information that could not be exposed using classical network layout optimization algorithms. CONCLUSIONS: Network visualization relies on computational techniques for proper placement of objects under consideration. These algorithms need to be fast so that they can be incorporated in responsive interfaces required by the explorative data analysis environments. Our layout optimization technique FragViz meets these requirements and specifically addresses the visualization of fragmented networks, for which standard algorithms do not consider similarities between unconnected components. The experiments confirmed the claims on speed and accuracy of the proposed solution.

Does replication groups scoring reduce false positive rate in SNP interaction discovery?

BACKGROUND: Computational methods that infer single nucleotide polymorphism (SNP) interactions from phenotype data may uncover new biological mechanisms in non-Mendelian diseases. However, practical aspects of such analysis face many problems. Present experimental studies typically use SNP arrays with hundreds of thousands of SNPs but record only hundreds of samples. Candidate SNP pairs inferred by interaction analysis may include a high proportion of false positives. Recently, Gayan et al. (2008) proposed to reduce the number of false positives by combining results of interaction analysis performed on subsets of data (replication groups), rather than analyzing the entire data set directly. If performing as hypothesized, replication groups scoring could improve interaction analysis and also any type of feature ranking and selection procedure in systems biology. Because Gayan et al. do not compare their approach to the standard interaction analysis techniques, we here investigate if replication groups indeed reduce the number of reported false positive interactions. RESULTS: A set of simulated and false interaction-imputed experimental SNP data sets were used to compare the inference of SNP-SNP interactions by means of replication groups to the standard approach where the entire data set was directly used to score all candidate SNP pairs. In all our experiments, the inference of interactions from the entire data set (e.g. without using the replication groups) reported fewer false positives. CONCLUSIONS: With respect to the direct scoring approach the utility of replication groups does not reduce false positive rates, and may, depending on the data set, often perform worse.

Open-source tools for data mining.

With a growing volume of biomedical databases and repositories, the need to develop a set of tools to address their analysis and support knowledge discovery is becoming acute. The data mining community has developed a substantial set of techniques for computational treatment of these data. In this article, we discuss the evolution of open-source toolboxes that data mining researchers and enthusiasts have developed over the span of a few decades and review several currently available open-source data mining suites. The approaches we review are diverse in data mining methods and user interfaces and also demonstrate that the field and its tools are ready to be fully exploited in biomedical research.

Visualization-based cancer microarray data classification analysis.

MOTIVATION: Methods for analyzing cancer microarray data often face two distinct challenges: the models they infer need to perform well when classifying new tissue samples while at the same time providing an insight into the patterns and gene interactions hidden in the data. State-of-the-art supervised data mining methods often cover well only one of these aspects, motivating the development of methods where predictive models with a solid classification performance would be easily communicated to the domain expert. RESULTS: Data visualization may provide for an excellent approach to knowledge discovery and analysis of class-labeled data. We have previously developed an approach called VizRank that can score and rank point-based visualizations according to degree of separation of data instances of different class. We here extend VizRank with techniques to uncover outliers, score features (genes) and perform classification, as well as to demonstrate that the proposed approach is well suited for cancer microarray analysis. Using VizRank and radviz visualization on a set of previously published cancer microarray data sets, we were able to find simple, interpretable data projections that include only a small subset of genes yet do clearly differentiate among different cancer types. We also report that our approach to classification through visualization achieves performance that is comparable to state-of-the-art supervised data mining techniques. AVAILABILITY: VizRank and radviz are implemented as part of the Orange data mining suite (http://www.ailab.si/orange). SUPPLEMENTARY INFORMATION: Supplementary data are available from http://www.ailab.si/supp/bi-cancer.

FreeViz--an intelligent multivariate visualization approach to explorative analysis of biomedical data.

Visualization can largely improve biomedical data analysis. It plays a crucial role in explorative data analysis and may support various data mining tasks. The paper presents FreeViz, an optimization method that finds linear projection and associated scatterplot that best separates instances of different class. In a single graph, the resulting FreeViz visualization can provide a global view of the classification problem being studied, reveal interesting relations between classes and features, uncover feature interactions, and provide information about intra-class similarities. The paper gives mathematical foundations of FreeViz, and presents its utility on various biomedical data sets.

Predictive model for estimating risk of crush syndrome: a data mining approach.

BACKGROUND: There is no standard triage method for earthquake victims with crush injuries because of a scarcity of epidemiologic and quantitative data. We conducted a retrospective cohort study to develop predictive models based on clinical data for crush injury in the Kobe earthquake. METHODS: The medical records of 372 patients with crush injuries from the Kobe earthquake were retrospectively analyzed. Twenty-one risk factors were assessed with logistic regression analysis for three outcomes relating to crush syndrome. Two types of predictive triage models--initial evaluation in the field and secondary assessment at the hospital--were developed using logistic regression analysis. Classification accuracy, Brier score and area under the receiver operating characteristic curve (AUC) were used to evaluate the model. RESULTS: The initial triage model, which includes pulse rate, delayed rescue, and abnormal urine color, has an AUC of 0.73. The secondary model, which includes WBC, tachycardia, abnormal urine color, and hyperkalemia, shows an AUC of 0.76. CONCLUSIONS: These triage models may be especially useful to nondisaster experts for distinguishing earthquake victims at high risk of severe crush syndrome from those at lower risk. Application of the model may allow relief workers to better utilize limited medical and transportation resources in the aftermath of a disaster.

GenePath: from mutations to genetic networks and back.

GenePath is a web-based application for the analysis of mutant-based experiments and synthesis of genetic networks. Here, we introduce GenePath and describe a number of new approaches, including conflict resolution, handling cyclic pathways, confidence level assignment, what-if analysis and new experiment proposal. We illustrate the key concepts using data from a study of adhesion genes in Dictyostelium discoideum and show that GenePath discovered genetic interactions that were ignored in the original publication. GenePath is available at http://www.genepath.org/genepath2.

Epistasis analysis with global transcriptional phenotypes.

Classical epistasis analysis can determine the order of function of genes in pathways using morphological, biochemical and other phenotypes. It requires knowledge of the pathway's phenotypic output and a variety of experimental expertise and so is unsuitable for genome-scale analysis. Here we used microarray profiles of mutants as phenotypes for epistasis analysis. Considering genes that regulate activity of protein kinase A in Dictyostelium, we identified known and unknown epistatic relationships and reconstructed a genetic network with microarray phenotypes alone. This work shows that microarray data can provide a uniform, quantitative tool for large-scale genetic network analysis.

Microarray data mining with visual programming.

UNLABELLED: Visual programming offers an intuitive means of combining known analysis and visualization methods into powerful applications. The system presented here enables users who are not programmers to manage microarray and genomic data flow and to customize their analyses by combining common data analysis tools to fit their needs. AVAILABILITY: http://www.ailab.si/supp/bi-visprog SUPPLEMENTARY INFORMATION: http://www.ailab.si/supp/bi-visprog.

GenePath: a system for inference of genetic networks and proposal of genetic experiments.

A genetic network is a formalism that is often used in biology to represent causalities and reason about biological phenomena related to genetic regulation. We present GenePath, a computer-based system that supports the inference of genetic networks from a set of genetic experiments. Implemented in Prolog, GenePath uses abductive inference to elucidate network constraints based on background knowledge and experimental results. Additionally, it can propose genetic experiments that may further refine the discovered network and establish relations between genes that could not be related based on the original experimental data. We illustrate GenePath's approach and utility on analysis of data on aggregation and sporulation of the soil amoeba Dictyostelium discoideum.

GenePath: a system for automated construction of genetic networks from mutant data.

MOTIVATION: Genetic networks are often used in the analysis of biological phenomena. In classical genetics, they are constructed manually from experimental data on mutants. The field lacks formalism to guide such analysis, and accounting for all the data becomes complicated when large amounts of data are considered. RESULTS: We have developed GenePath, an intelligent assistant that automates the analysis of genetic data. GenePath employs expert-defined patterns to uncover gene relations from the data, and uses these relations as constraints in the search for a plausible genetic network. GenePath formalizes genetic data analysis, facilitates the consideration of all the available data in a consistent manner, and the examination of the large number of possible consequences of planned experiments. It also provides an explanation mechanism that traces every finding to the pertinent data. AVAILABILITY: GenePath can be accessed at http://genepath.org. SUPPLEMENTARY INFORMATION: Supplementary material is available at http://genepath.org/bi-.supp.