Data File Standard for Flow Cytometry, Version FCS 3.2.

FCS 3.2 is a revision of the flow cytometry data standard based on a decade of suggested improvements from the community as well as industry needs to capture instrument conditions and measurement features more precisely. The unchanged goal of the standard is to provide a uniform file format that allows files created by one type of acquisition hardware and software to be analyzed by any other type. The standard retains the overall FCS file structure and most features of previous versions, but also contains a few changes that were required to support new types of data and use cases efficiently. These changes are incompatible with existing FCS file readers. Notably, FCS 3.2 supports mixed data types to, for example, allow FCS measurements that are intrinsically integers (e.g., indices or class assignments) or measurements that are commonly captured as integers (e.g., time ticks) to be more represented as integer values, while capturing other measurements as floating-point values in the same FCS data set. In addition, keywords explicitly specifying dyes, detectors, and analytes were added to avoid having to extract those heuristically and unreliably from measurement names. Types of measurements were formalized, several keywords added, others removed, or deprecated, and various aspects of the specification were clarified. A reference implementation of the cyclic redundancy check (CRC) calculation is provided in two programming languages since a correct CRC implementation was problematic for many vendors. © 2020 International Society for Advancement of Cytometry.

ISAC Probe Tag Dictionary: Standardized Nomenclature for Detection and Visualization Labels Used in Cytometry and Microscopy Imaging.

Since the advent of microscopy imaging and flow cytometry, there has been an explosion in the number of probes, consisting of a component binding to an analyte and a detectable tag, to mark areas of interest in or on cells and tissue. Probe tags have been created to detect and/or visualize probes. Over time, these probe tags have increased in number. The expansion has resulted in arbitrarily created synonyms of probe tags used in publications and software. The synonyms are problematic for readability of publications, accuracy of text/data mining, and bridging data from multiple platforms, protocols, and databases for Big Data analysis. Development and implementation of a universal language for probe tags will ensure equivalent quality and level of data being reported or extracted for clinical/scientific evaluation as well as help connect data from many platforms. The International Society for Advancement of Cytometry Data Standards Task Force composed of academic scientists and industry hardware/software/reagent manufactures have developed recommendations for a standardized nomenclature for probe tags used in cytometry and microscopy imaging. These recommendations are shared in this technical note in the form of a Probe Tag Dictionary. © 2020 International Society for Advancement of Cytometry.

Methodology for evaluating and comparing flow cytometers: A multisite study of 23 instruments.

We demonstrate improved methods for making valid and accurate comparisons of fluorescence measurement capabilities among instruments tested at different sites and times. We designed a suite of measurements and automated data processing methods to obtain consistent objective results and applied them to a selection of 23 instruments at nine sites to provide a range of instruments as well as multiple instances of similar instruments. As far as we know, this study represents the most accurate methods and results so far demonstrated for this purpose. The first component of the study reporting improved methods for photoelectron scale (Spe) evaluations, which was published previously (Parks, El Khettabi, Chase, Hoffman, Perfetto, Spidlen, Wood, Moore, and Brinkman: Cytometry A 91 (2017) 232-249). Those results which were within themselves are not sufficient for instrument comparisons, so here, we use the Spe scale results for the 23 cytometers and combine them with additional information from the analysis suite to obtain the metrics actually needed for instrument evaluations and comparisons. We adopted what we call the 2+2SD limit of resolution as a maximally informative metric, for evaluating and comparing dye measurement sensitivity among different instruments and measurement channels. Our results demonstrate substantial differences among different classes of instruments in both dye response and detection sensitivity and some surprisingly large differences among similar instruments, even among instruments with nominally identical configurations. On some instruments, we detected defective measurement channels needing service. The system can be applied in shared resource laboratories and other facilities as an aspect of quality assurance, and accurate instrument comparisons can be valuable for selecting instruments for particular purposes and for making informed instrument acquisition decisions. An institutionally supported program could serve the cytometry community by facilitating access to materials, and analysis and maintaining an archive of results. © 2018 International Society for Advancement of Cytometry.

Evaluating flow cytometer performance with weighted quadratic least squares analysis of LED and multi-level bead data.

We developed a fully automated procedure for analyzing data from LED pulses and multilevel bead sets to evaluate backgrounds and photoelectron scales of cytometer fluorescence channels. The method improves on previous formulations by fitting a full quadratic model with appropriate weighting and by providing standard errors and peak residuals as well as the fitted parameters themselves. Here we describe the details of the methods and procedures involved and present a set of illustrations and test cases that demonstrate the consistency and reliability of the results. The automated analysis and fitting procedure is generally quite successful in providing good estimates of the Spe (statistical photoelectron) scales and backgrounds for all the fluorescence channels on instruments with good linearity. The precision of the results obtained from LED data is almost always better than that from multilevel bead data, but the bead procedure is easy to carry out and provides results good enough for most purposes. Including standard errors on the fitted parameters is important for understanding the uncertainty in the values of interest. The weighted residuals give information about how well the data fits the model, and particularly high residuals indicate bad data points. Known photoelectron scales and measurement channel backgrounds make it possible to estimate the precision of measurements at different signal levels and the effects of compensated spectral overlap on measurement quality. Combining this information with measurements of standard samples carrying dyes of biological interest, we can make accurate comparisons of dye sensitivity among different instruments. Our method is freely available through the R/Bioconductor package flowQB. © 2017 International Society for Advancement of Cytometry.

flowClean: Automated identification and removal of fluorescence anomalies in flow cytometry data.

Modern flow cytometry systems can be coupled to plate readers for high-throughput acquisition. These systems allow hundreds of samples to be analyzed in a single day. Quality control of the data remains challenging, however, and is further complicated when a large number of parameters is measured in an experiment. Our examination of 29,228 publicly available FCS files from laboratories worldwide indicates 13.7% have a fluorescence anomaly. In particular, fluorescence measurements for a sample over the collection time may not remain stable due to fluctuations in fluid dynamics; the impact of instabilities may differ between samples and among parameters. Therefore, we hypothesized that tracking cell populations (which represent a summary of all parameters) in centered log ratio space would provide a sensitive and consistent method of quality control. Here, we present flowClean, an algorithm to track subset frequency changes within a sample during acquisition, and flag time periods with fluorescence perturbations leading to the emergence of false populations. Aberrant time periods are reported as a new parameter and added to a revised data file, allowing users to easily review and exclude those events from further analysis. We apply this method to proof-of-concept datasets and also to a subset of data from a recent vaccine trial. The algorithm flags events that are suspicious by visual inspection, as well as those showing more subtle effects that might not be consistently flagged by investigators reviewing the data manually, and out-performs the current state-of-the-art. flowClean is available as an R package on Bioconductor, as a module on the free-to-use GenePattern web server, and as a plugin for FlowJo X. © 2016 International Society for Advancement of Cytometry.

ISAC's classification results file format.

Identifying homogenous sets of cell populations in flow cytometry is an important process for sorting and selecting populations of interests for further data acquisition and analysis. Many computational methods are now available to automate this process, with several algorithms partitioning cells based on high-dimensional separation versus the traditional pairwise two-dimensional visualization approach of manual gating. ISAC's classification results file format was developed to exchange the results of both manual gating and algorithmic classification approaches in a standardized way based on per event based classifications, including the potential for soft classifications expressed as the probability of an event being a member of a class. © 2014 International Society for Advancement of Cytometry.

ISAC's Gating-ML 2.0 data exchange standard for gating description.

The lack of software interoperability with respect to gating has traditionally been a bottleneck preventing the use of multiple analytical tools and reproducibility of flow cytometry data analysis by independent parties. To address this issue, ISAC developed Gating-ML, a computer file format to encode and interchange gates. Gating-ML 1.5 was adopted and published as an ISAC Candidate Recommendation in 2008. Feedback during the probationary period from implementors, including major commercial software companies, instrument vendors, and the wider community, has led to a streamlined Gating-ML 2.0. Gating-ML has been significantly simplified and therefore easier to support by software tools. To aid developers, free, open source reference implementations, compliance tests, and detailed examples are provided to stimulate further commercial adoption. ISAC has approved Gating-ML as a standard ready for deployment in the public domain and encourages its support within the community as it is at a mature stage of development having undergone extensive review and testing, under both theoretical and practical conditions.

Flow cytometry bioinformatics.

Flow cytometry bioinformatics is the application of bioinformatics to flow cytometry data, which involves storing, retrieving, organizing, and analyzing flow cytometry data using extensive computational resources and tools. Flow cytometry bioinformatics requires extensive use of and contributes to the development of techniques from computational statistics and machine learning. Flow cytometry and related methods allow the quantification of multiple independent biomarkers on large numbers of single cells. The rapid growth in the multidimensionality and throughput of flow cytometry data, particularly in the 2000s, has led to the creation of a variety of computational analysis methods, data standards, and public databases for the sharing of results. Computational methods exist to assist in the preprocessing of flow cytometry data, identifying cell populations within it, matching those cell populations across samples, and performing diagnosis and discovery using the results of previous steps. For preprocessing, this includes compensating for spectral overlap, transforming data onto scales conducive to visualization and analysis, assessing data for quality, and normalizing data across samples and experiments. For population identification, tools are available to aid traditional manual identification of populations in two-dimensional scatter plots (gating), to use dimensionality reduction to aid gating, and to find populations automatically in higher dimensional space in a variety of ways. It is also possible to characterize data in more comprehensive ways, such as the density-guided binary space partitioning technique known as probability binning, or by combinatorial gating. Finally, diagnosis using flow cytometry data can be aided by supervised learning techniques, and discovery of new cell types of biological importance by high-throughput statistical methods, as part of pipelines incorporating all of the aforementioned methods. Open standards, data, and software are also key parts of flow cytometry bioinformatics. Data standards include the widely adopted Flow Cytometry Standard (FCS) defining how data from cytometers should be stored, but also several new standards under development by the International Society for Advancement of Cytometry (ISAC) to aid in storing more detailed information about experimental design and analytical steps. Open data is slowly growing with the opening of the CytoBank database in 2010 and FlowRepository in 2012, both of which allow users to freely distribute their data, and the latter of which has been recommended as the preferred repository for MIFlowCyt-compliant data by ISAC. Open software is most widely available in the form of a suite of Bioconductor packages, but is also available for web execution on the GenePattern platform.

FCS 3.1 Implementation guidance.

The Flow Cytometry Standard (FCS) format was developed back in 1984. Since then, FCS became the standard file format supported by all flow cytometry software and hardware vendors. Over the years, updates were incorporated to adapt to technological advancements in both flow cytometry and computing technologies. However, flexibility in how data may be stored in FCS has led to implementation difficulties for instrument vendors and third party software developers. In this technical note, we are providing implementation guidance and examples related to FCS 3.1, the latest version of the standard. By publishing this text, we intend to prevent potential compatibility issues that could be faced when implementing the FCS spillover and preferred display keywords that have arisen during discussions among some implementers.

AutoSpill is a principled framework that simplifies the analysis of multichromatic flow cytometry data.

Compensating in flow cytometry is an unavoidable challenge in the data analysis of fluorescence-based flow cytometry. Even the advent of spectral cytometry cannot circumvent the spillover problem, with spectral unmixing an intrinsic part of such systems. The calculation of spillover coefficients from single-color controls has remained essentially unchanged since its inception, and is increasingly limited in its ability to deal with high-parameter flow cytometry. Here, we present AutoSpill, an alternative method for calculating spillover coefficients. The approach combines automated gating of cells, calculation of an initial spillover matrix based on robust linear regression, and iterative refinement to reduce error. Moreover, autofluorescence can be compensated out, by processing it as an endogenous dye in an unstained control. AutoSpill uses single-color controls and is compatible with common flow cytometry software. AutoSpill allows simpler and more robust workflows, while reducing the magnitude of compensation errors in high-parameter flow cytometry.

Data File Standard for Flow Cytometry, version FCS 3.1.

The flow cytometry data file standard provides the specifications needed to completely describe flow cytometry data sets within the confines of the file containing the experimental data. In 1984, the first Flow Cytometry Standard format for data files was adopted as FCS 1.0. This standard was modified in 1990 as FCS 2.0 and again in 1997 as FCS 3.0. We report here on the next generation flow cytometry standard data file format. FCS 3.1 is a minor revision based on suggested improvements from the community. The unchanged goal of the standard is to provide a uniform file format that allows files created by one type of acquisition hardware and software to be analyzed by any other type.The FCS 3.1 standard retains the basic FCS file structure and most features of previous versions of the standard. Changes included in FCS 3.1 address potential ambiguities in the previous versions and provide a more robust standard. The major changes include simplified support for international characters and improved support for storing compensation. The major additions are support for preferred display scale, a standardized way of capturing the sample volume, information about originality of the data file, and support for plate and well identification in high throughput, plate based experiments. Please see the normative version of the FCS 3.1 specification in Supporting Information for this manuscript (or at http://www.isac-net.org/ in the Current standards section) for a complete list of changes.

flowCore: a Bioconductor package for high throughput flow cytometry.

BACKGROUND: Recent advances in automation technologies have enabled the use of flow cytometry for high throughput screening, generating large complex data sets often in clinical trials or drug discovery settings. However, data management and data analysis methods have not advanced sufficiently far from the initial small-scale studies to support modeling in the presence of multiple covariates. RESULTS: We developed a set of flexible open source computational tools in the R package flowCore to facilitate the analysis of these complex data. A key component of which is having suitable data structures that support the application of similar operations to a collection of samples or a clinical cohort. In addition, our software constitutes a shared and extensible research platform that enables collaboration between bioinformaticians, computer scientists, statisticians, biologists and clinicians. This platform will foster the development of novel analytic methods for flow cytometry. CONCLUSION: The software has been applied in the analysis of various data sets and its data structures have proven to be highly efficient in capturing and organizing the analytic work flow. Finally, a number of additional Bioconductor packages successfully build on the infrastructure provided by flowCore, open new avenues for flow data analysis.

FuGEFlow: data model and markup language for flow cytometry.

BACKGROUND: Flow cytometry technology is widely used in both health care and research. The rapid expansion of flow cytometry applications has outpaced the development of data storage and analysis tools. Collaborative efforts being taken to eliminate this gap include building common vocabularies and ontologies, designing generic data models, and defining data exchange formats. The Minimum Information about a Flow Cytometry Experiment (MIFlowCyt) standard was recently adopted by the International Society for Advancement of Cytometry. This standard guides researchers on the information that should be included in peer reviewed publications, but it is insufficient for data exchange and integration between computational systems. The Functional Genomics Experiment (FuGE) formalizes common aspects of comprehensive and high throughput experiments across different biological technologies. We have extended FuGE object model to accommodate flow cytometry data and metadata. METHODS: We used the MagicDraw modelling tool to design a UML model (Flow-OM) according to the FuGE extension guidelines and the AndroMDA toolkit to transform the model to a markup language (Flow-ML). We mapped each MIFlowCyt term to either an existing FuGE class or to a new FuGEFlow class. The development environment was validated by comparing the official FuGE XSD to the schema we generated from the FuGE object model using our configuration. After the Flow-OM model was completed, the final version of the Flow-ML was generated and validated against an example MIFlowCyt compliant experiment description. RESULTS: The extension of FuGE for flow cytometry has resulted in a generic FuGE-compliant data model (FuGEFlow), which accommodates and links together all information required by MIFlowCyt. The FuGEFlow model can be used to build software and databases using FuGE software toolkits to facilitate automated exchange and manipulation of potentially large flow cytometry experimental data sets. Additional project documentation, including reusable design patterns and a guide for setting up a development environment, was contributed back to the FuGE project. CONCLUSION: We have shown that an extension of FuGE can be used to transform minimum information requirements in natural language to markup language in XML. Extending FuGE required significant effort, but in our experiences the benefits outweighed the costs. The FuGEFlow is expected to play a central role in describing flow cytometry experiments and ultimately facilitating data exchange including public flow cytometry repositories currently under development.

Automated optimized parameters for T-distributed stochastic neighbor embedding improve visualization and analysis of large datasets.

Accurate and comprehensive extraction of information from high-dimensional single cell datasets necessitates faithful visualizations to assess biological populations. A state-of-the-art algorithm for non-linear dimension reduction, t-SNE, requires multiple heuristics and fails to produce clear representations of datasets when millions of cells are projected. We develop opt-SNE, an automated toolkit for t-SNE parameter selection that utilizes Kullback-Leibler divergence evaluation in real time to tailor the early exaggeration and overall number of gradient descent iterations in a dataset-specific manner. The precise calibration of early exaggeration together with opt-SNE adjustment of gradient descent learning rate dramatically improves computation time and enables high-quality visualization of large cytometry and transcriptomics datasets, overcoming limitations of analysis tools with hard-coded parameters that often produce poorly resolved or misleading maps of fluorescent and mass cytometry data. In summary, opt-SNE enables superior data resolution in t-SNE space and thereby more accurate data interpretation.

Gating-ML: XML-based gating descriptions in flow cytometry.

The lack of software interoperability with respect to gating due to lack of a standardized mechanism for data exchange has traditionally been a bottleneck, preventing reproducibility of flow cytometry (FCM) data analysis and the usage of multiple analytical tools. To facilitate interoperability among FCM data analysis tools, members of the International Society for the Advancement of Cytometry (ISAC) Data Standards Task Force (DSTF) have developed an XML-based mechanism to formally describe gates (Gating-ML). Gating-ML, an open specification for encoding gating, data transformations and compensation, has been adopted by the ISAC DSTF as a Candidate Recommendation. Gating-ML can facilitate exchange of gating descriptions the same way that FCS facilitated for exchange of raw FCM data. Its adoption will open new collaborative opportunities as well as possibilities for advanced analyses and methods development. The ISAC DSTF is satisfied that the standard addresses the requirements for a gating exchange standard.

MIFlowCyt: the minimum information about a Flow Cytometry Experiment.

A fundamental tenet of scientific research is that published results are open to independent validation and refutation. Minimum data standards aid data providers, users, and publishers by providing a specification of what is required to unambiguously interpret experimental findings. Here, we present the Minimum Information about a Flow Cytometry Experiment (MIFlowCyt) standard, stating the minimum information required to report flow cytometry (FCM) experiments. We brought together a cross-disciplinary international collaborative group of bioinformaticians, computational statisticians, software developers, instrument manufacturers, and clinical and basic research scientists to develop the standard. The standard was subsequently vetted by the International Society for Advancement of Cytometry (ISAC) Data Standards Task Force, Standards Committee, membership, and Council. The MIFlowCyt standard includes recommendations about descriptions of the specimens and reagents included in the FCM experiment, the configuration of the instrument used to perform the assays, and the data processing approaches used to interpret the primary output data. MIFlowCyt has been adopted as a standard by ISAC, representing the FCM scientific community including scientists as well as software and hardware manufacturers. Adoptionof MIFlowCyt by the scientific and publishing communities will facilitate third-party understanding and reuse of FCM data.

Use FlowRepository to share your clinical data upon study publication.

A fundamental tenet of scientific research is that published results including underlying data should be open to independent validation and refutation. Data sharing encourages collaboration, facilitates quality and reduces redundancy in data production. Authors submitting manuscripts to several journals have already adopted the habit of sharing their underlying flow cytometry data by deposition to FlowRepository-a data repository that is jointly supported by the International Society for Advancement of Cytometry, the International Clinical Cytometry Society and the European Society for Clinical Cell Analysis. De-identification is required for publishing data from clinical studies and we discuss ways to satisfy data sharing requirements and patient privacy requirements simultaneously. Scientific communities in the fields of microarray, proteomics, and sequencing have been benefiting from reuse and re-exploration of data in public repositories for over decade. We believe it is time that clinicians follow suit and that de-identified clinical data also become routinely available along with published cytometry-based findings. © 2016 International Clinical Cytometry Society.

Data standards for flow cytometry.

Flow cytometry (FCM) is an analytical tool widely used for cancer and HIV/AIDS research, and treatment, stem cell manipulation and detecting microorganisms in environmental samples. Current data standards do not capture the full scope of FCM experiments and there is a demand for software tools that can assist in the exploration and analysis of large FCM datasets. We are implementing a standardized approach to capturing, analyzing, and disseminating FCM data that will facilitate both more complex analyses and analysis of datasets that could not previously be efficiently studied. Initial work has focused on developing a community-based guideline for recording and reporting the details of FCM experiments. Open source software tools that implement this standard are being created, with an emphasis on facilitating reproducible and extensible data analyses. As well, tools for electronic collaboration will assist the integrated access and comprehension of experiments to empower users to collaborate on FCM analyses. This coordinated, joint development of bioinformatics standards and software tools for FCM data analysis has the potential to greatly facilitate both basic and clinical research--impacting a notably diverse range of medical and environmental research areas.