Development of a Cell-Based SARS-CoV-2 Pseudovirus Neutralization Assay Using Imaging and Flow Cytometry Analysis.

COVID-19 is an ongoing, global pandemic caused by the novel, highly infectious SARS-CoV-2 virus. Efforts to mitigate the effects of SARS-CoV-2, such as mass vaccination and development of monoclonal therapeutics, require precise measurements of correlative, functional neutralizing antibodies that block virus infection. The development of rapid, safe, and easy-to-use neutralization assays is essential for faster diagnosis and treatment. Here, we developed a vesicular stomatitis virus (VSV)-based neutralization assay with two readout methods, imaging and flow cytometry, that were capable of quantifying varying degrees of neutralization in patient serum samples. We tested two different spike-pseudoviruses and conducted a time-course assay at multiple multiplicities of infection (MOIs) to optimize the assay workflow. The results of this assay correlate with the results of previously developed serology and surrogate neutralization assays. The two pseudovirus readout methods produced similar values of 50% neutralization titer values. Harvest-free in situ readouts for live-cell imaging and high-throughput analysis results for flow cytometry can provide unique capabilities for fast evaluation of neutralization, which is critical for the mitigation of future pandemics.

Technical framework for enabling high quality measurements in new approach methodologies (NAMs).

New approach methodologies (NAMs) are in vitro, in chemico, and in silico or computational approaches that can potentially be used to reduce animal testing. For NAMs that require laboratory experiments, it is critical that they provide consistent and reliable results. While guidance has been provided on improving the reproducibility of NAMs that require laboratory experiments, there is not yet an overarching technical framework that details how to add measurement quality features into a protocol. In this manuscript, we discuss such a framework and provide a step-by-step process describing how to refine a protocol using basic quality tools. The steps in this framework include 1) conceptual analysis of sources of technical variability in the assay, 2) within-laboratory evaluation of assay performance, 3) statistical data analysis, and 4) determination of method transferability (if needed). While each of these steps has discrete components, they are all inter-related, and insights from any step can influence the others. Following the steps in this framework can help reveal the advantages and limitations of different choices during the design of an assay such as which in-process control measurements to include and how many replicates to use for each control measurement and for each test substance. Overall, the use of this technical framework can support optimizing NAM reproducibility, thereby supporting meeting research and regulatory needs.

Development of a 96-Well Electrophilic Allergen Screening Assay for Skin Sensitization Using a Measurement Science Approach.

The Electrophilic Allergen Screening Assay (EASA) has emerged as a promising in chemico method to detect the first key event in the adverse outcome pathway (AOP) for skin sensitization. This assay functions by assessing the depletion of one of two probe molecules (4-nitrobenzenethiol (NBT) and pyridoxylamine (PDA)) in the presence of a test compound (TC). The initial development of EASA utilized a cuvette format resulting in multiple measurement challenges such as low throughput and the inability to include adequate control measurements. In this study, we describe the redesign of EASA into a 96-well plate format that incorporates in-process control measurements to quantify key sources of variability each time the assay is run. The data from the analysis of 67 TCs using the 96-well format had 77% concordance with animal data from the local lymph node assay (LLNA), a result consistent with that for the direct peptide reactivity assay (DPRA), an OECD test guideline (442C) protein binding assay. Overall, the measurement science approach described here provides steps during assay development that can be taken to increase confidence of in chemico assays by attempting to fully characterize the sources of variability and potential biases and incorporate in-process control measurements into the assay.

A Sensitive and Controlled Data-Independent Acquisition Method for Proteomic Analysis of Cell Therapies.

Mass spectrometry (MS)-based proteomic measurements are uniquely poised to impact the development of cell and gene therapies. With the adoption of rigorous instrumental performance qualifications (PQs), large-scale proteomics can move from a research to a manufacturing control tool. Especially suited, data-independent acquisition (DIA) approaches have distinctive qualities to extend multiattribute method (MAM) principles to characterize the proteome of cell therapies. Here, we describe the development of a DIA method for the sensitive identification and quantification of proteins on a Q-TOF instrument. Using the improved acquisition parameters, we defined a control strategy and highlighted some metrics to improve the reproducibility of SWATH acquisition-based proteomic measurements. Finally, we applied the method to analyze the proteome of Jurkat cells that here serves as a model for human T-cells. Raw and processed data were deposited in PRIDE (PXD029780).

Characteristics to consider when selecting a positive control material for an in vitro assay

The use of in vitro assays to inform decision-making requires robust and reproducible results across studies, laboratories, and time. Experiments using positive control materials are an integral component of an assay procedure to demonstrate the extent to which the measurement system is performing as expected. This paper reviews ten characteristics that should be considered when selecting a positive control material for an in vitro assay: 1) the biological mechanism of action, 2) ease of preparation, 3) chemical purity, 4) verifiable physical properties, 5) stability, 6) ability to generate responses spanning the dynamic range of the assay, 7) technical or biological interference, 8) commercial availability, 9) user toxicity, and 10) disposability. Examples and a case study of the monocyte activation test are provided to demonstrate the application of these characteristics for identification and selection of potential positive control materials. Because specific positive control materials are often written into testing standards for in vitro assays, selection of the positive control material based on these characteristics can aid in ensuring the long-term relevance and usability of these standards.

Cause-and-Effect Analysis as a Tool To Improve the Reproducibility of Nanobioassays: Four Case Studies.

One of the challenges in using in vitro data to understand the potential risks of engineered nanomaterials (ENMs) is that results often differ or are even contradictory among studies. While it is recognized that numerous factors can influence results produced by nanobioassays, there has not yet been a consistently used conceptual framework to identify key sources of variability in these assays. In this paper, we use cause-and-effect analysis to systematically describe sources of variability in four key in vitro nanobioassays: the 2',7'-dichlorofluorescein assay, an enzyme-linked immunosorbent assay for measuring interleukin-8, a flow cytometry assay (Annexin V/propidium iodide), and the Comet assay. These assays measure end points that can occur in cells impacted by ENMs through oxidative stress, a principle mechanism for ENM toxicity. The results from this analysis identify control measurements to test for potential artifacts or biases that could occur during conduct of these assays with ENMs. Cause-and-effect analysis also reveals additional measurements that could be performed either in preliminary experiments or each time the assay is run to increase confidence in the assay results and their reproducibility within and among laboratories. The approach applied here with these four assays can be used to support the development of a broad range of nanobioassays.

Mass Measurements of Focal Adhesions in Single Cells Using High Resolution Surface Plasmon Resonance Microscopy.

Surface plasmon resonance microscopy (SPRM) is a powerful label-free imaging technique with spatial resolution approaching the optical diffraction limit. The high sensitivity of SPRM to small changes in index of refraction at an interface allows imaging of dynamic protein structures within a cell. Visualization of subcellular features, such as focal adhesions (FAs), can be performed on live cells using a high numerical aperture objective lens with a digital light projector to precisely position the incident angle of the excitation light. Within the cell-substrate region of the SPRM image, punctate regions of high contrast are putatively identified as the cellular FAs. Optical parameter analysis is achieved by application of the Fresnel model to the SPRM data and resulting refractive index measurements are used to calculate protein density and mass. FAs are known to be regions of high protein density that reside at the cell-substratum interface. Comparing SPRM with fluorescence images of antibody stained for vinculin, a component in FAs, reveals similar measurements of FA size. In addition, a positive correlation between FA size and protein density is revealed by SPRM. Comparing SPRM images for two cell types reveals a distinct difference in the protein density and mass of their respective FAs. Application of SPRM to quantify mass can greatly aid monitoring basic processes that control FA mass and growth and contribute to accurate models that describe cell-extracellular interactions.

How measurement science can improve confidence in research results.

The current push for rigor and reproducibility is driven by a desire for confidence in research results. Here, we suggest a framework for a systematic process, based on consensus principles of measurement science, to guide researchers and reviewers in assessing, documenting, and mitigating the sources of uncertainty in a study. All study results have associated ambiguities that are not always clarified by simply establishing reproducibility. By explicitly considering sources of uncertainty, noting aspects of the experimental system that are difficult to characterize quantitatively, and proposing alternative interpretations, the researcher provides information that enhances comparability and reproducibility.

Summary of the National Institute of Standards and Technology and US Food And Drug Administration cell counting workshop: Sharing practices in cell counting measurements.

The emergence of cell-based therapeutics has increased the need for high-quality, robust and validated measurements for cell characterization. Cell count, being one of the most fundamental measures for cell-based therapeutics, now requires increased levels of measurement confidence. The National Institute of Standards and Technology (NIST) and the US Food and Drug Administration (FDA) jointly hosted a workshop focused on cell counting in April 2017 entitled "NIST-FDA Cell Counting Workshop: Sharing Practices in Cell Counting Measurements." The focus of the workshop was on approaches for selecting, designing and validating cell counting methods and overcoming gaps in obtaining sufficient measurement assurance for cell counting. Key workshop discussion points, representing approximately 50 subject matter experts from industry, academia and government agencies, are summarized here. A key conclusion is the need to design the most appropriate cell counting method, including control/measurement assurance strategies, for a specific counting purposes. There remains a need for documentary standards for streamlining the process to develop, qualify and validate cell counting measurements as well as community-driven efforts to develop new or improved biological and non-biological reference materials.

Agglomeration of Escherichia coli with Positively Charged Nanoparticles Can Lead to Artifacts in a Standard Caenorhabditis elegans Toxicity Assay.

The increased use and incorporation of engineered nanoparticles (ENPs) in consumer products requires a robust assessment of their potential environmental implications. However, a lack of standardized methods for nanotoxicity testing has yielded results that are sometimes contradictory. Standard ecotoxicity assays may work appropriately for some ENPs with minimal modification but produce artifactual results for others. Therefore, understanding the robustness of assays for a range of ENPs is critical. In this study, we evaluated the performance of a standard Caenorhabditis elegans ( C. elegans) toxicity assay containing an Escherichia coli ( E. coli) food supply with silicon, polystyrene, and gold ENPs with different charged coatings and sizes. Of all the ENPs tested, only those with a positively charged coating caused growth inhibition. However, the positively charged ENPs were observed to heteroagglomerate with E. coli cells, suggesting that the ENPs impacted the ability of nematodes to feed, leading to a false positive toxic effect on C. elegans growth and reproduction. When the ENPs were tested in two alternate C. elegans assays that did not contain E. coli, we found greatly reduced toxicity of ENPs. This study illustrates a key unexpected artifact that may occur during nanotoxicity assays.

Counting Caenorhabditis elegans: Protocol Optimization and Applications for Population Growth and Toxicity Studies in Liquid Medium.

The nematode Caenorhabditis elegans is used extensively in molecular, toxicological and genetics research. However, standardized methods for counting nematodes in liquid culture do not exist despite the wide use of nematodes and need for accurate measurements. Herein, we provide a simple and affordable counting protocol developed to maximize count accuracy and minimize variability in liquid nematode culture. Sources of variability in the counting process were identified and tested in 14 separate experiments. Three variables resulted in significant effects on nematode count: shaking of the culture, priming of pipette tips, and sampling location within a microcentrifuge tube. Between-operator variability did not have a statistically significant effect on counts, even among differently-skilled operators. The protocol was used to assess population growth rates of nematodes in two different but common liquid growth media: axenic modified Caenorhabditis elegans Habitation and Reproduction medium (mCeHR) and S-basal complete. In mCeHR, nematode populations doubled daily for 10 d. S-basal complete populations initially doubled every 12 h, but slowed within 7 d. We also detected a statistically significant difference between embryo-to-hatchling incubation period of 5 d in mCeHR compared to 4 d in S-basal complete. The developed counting method for Caenorhabditis elegans reduces variability and allows for rigorous and reliable experimentation.

Evaluating the quality of a cell counting measurement process via a dilution series experimental design.

BACKGROUND AIMS: Cell counting measurements are critical in the research, development and manufacturing of cell-based products, yet determining cell quantity with accuracy and precision remains a challenge. Validating and evaluating a cell counting measurement process can be difficult because of the lack of appropriate reference material. Here we describe an experimental design and statistical analysis approach to evaluate the quality of a cell counting measurement process in the absence of appropriate reference materials or reference methods. METHODS: The experimental design is based on a dilution series study with replicate samples and observations as well as measurement process controls. The statistical analysis evaluates the precision and proportionality of the cell counting measurement process and can be used to compare the quality of two or more counting methods. As an illustration of this approach, cell counting measurement processes (automated and manual methods) were compared for a human mesenchymal stromal cell (hMSC) preparation. RESULTS: For the hMSC preparation investigated, results indicated that the automated method performed better than the manual counting methods in terms of precision and proportionality. DISCUSSION: By conducting well controlled dilution series experimental designs coupled with appropriate statistical analysis, quantitative indicators of repeatability and proportionality can be calculated to provide an assessment of cell counting measurement quality. This approach does not rely on the use of a reference material or comparison to "gold standard" methods known to have limited assurance of accuracy and precision. The approach presented here may help the selection, optimization, and/or validation of a cell counting measurement process.

A novel measure and significance testing in data analysis of cell image segmentation.

BACKGROUND: Cell image segmentation (CIS) is an essential part of quantitative imaging of biological cells. Designing a performance measure and conducting significance testing are critical for evaluating and comparing the CIS algorithms for image-based cell assays in cytometry. Many measures and methods have been proposed and implemented to evaluate segmentation methods. However, computing the standard errors (SE) of the measures and their correlation coefficient is not described, and thus the statistical significance of performance differences between CIS algorithms cannot be assessed. RESULTS: We propose the total error rate (TER), a novel performance measure for segmenting all cells in the supervised evaluation. The TER statistically aggregates all misclassification error rates (MER) by taking cell sizes as weights. The MERs are for segmenting each single cell in the population. The TER is fully supported by the pairwise comparisons of MERs using 106 manually segmented ground-truth cells with different sizes and seven CIS algorithms taken from ImageJ. Further, the SE and 95% confidence interval (CI) of TER are computed based on the SE of MER that is calculated using the bootstrap method. An algorithm for computing the correlation coefficient of TERs between two CIS algorithms is also provided. Hence, the 95% CI error bars can be used to classify CIS algorithms. The SEs of TERs and their correlation coefficient can be employed to conduct the hypothesis testing, while the CIs overlap, to determine the statistical significance of the performance differences between CIS algorithms. CONCLUSIONS: A novel measure TER of CIS is proposed. The TER's SEs and correlation coefficient are computed. Thereafter, CIS algorithms can be evaluated and compared statistically by conducting the significance testing.

Separation, Sizing, and Quantitation of Engineered Nanoparticles in an Organism Model Using Inductively Coupled Plasma Mass Spectrometry and Image Analysis.

For environmental studies assessing uptake of orally ingested engineered nanoparticles (ENPs), a key step in ensuring accurate quantification of ingested ENPs is efficient separation of the organism from ENPs that are either nonspecifically adsorbed to the organism and/or suspended in the dispersion following exposure. Here, we measure the uptake of 30 and 60 nm gold nanoparticles (AuNPs) by the nematode, Caenorhabditis elegans, using a sucrose density gradient centrifugation protocol to remove noningested AuNPs. Both conventional inductively coupled plasma mass spectrometry (ICP-MS) and single particle (sp)ICP-MS are utilized to measure the total mass and size distribution, respectively, of ingested AuNPs. Scanning electron microscopy/energy-dispersive X-ray spectroscopy (SEM/EDS) imaging confirmed that traditional nematode washing procedures were ineffective at removing excess suspended and/or adsorbed AuNPs after exposure. Water rinsing procedures had AuNP removal efficiencies ranging from 57 to 97% and 22 to 83%, while the sucrose density gradient procedure had removal efficiencies of 100 and 93 to 98%, respectively, for the 30 and 60 nm AuNP exposure conditions. Quantification of total Au uptake was performed following acidic digestion of nonexposed and Au-exposed nematodes, whereas an alkaline digestion procedure was optimized for the liberation of ingested AuNPs for spICP-MS characterization. Size distributions and particle number concentrations were determined for AuNPs ingested by nematodes with corresponding confirmation of nematode uptake via high-pressure freezing/freeze substitution resin preparation and large-area SEM imaging. Methods for the separation and in vivo quantification of ENPs in multicellular organisms will facilitate robust studies of ENP uptake, biotransformation, and hazard assessment in the environment.

Toward achieving harmonization in a nano-cytotoxicity assay measurement through an interlaboratory comparison study.

Development of reliable cell-based nanotoxicology assays is important for evaluation of potentially hazardous engineered nanomaterials. Challenges to producing a reliable assay protocol include working with nanoparticle dispersions and living cell lines, and the potential for nano-related interference effects. Here we demonstrate the use of a 96-well plate design with several measurement controls and an interlaboratory comparison study involving five laboratories to characterize the robustness of a nanocytotoxicity MTS cell viability assay based on the A549 cell line. The consensus EC50 values were 22.1 mg/L (95% confidence intervals 16.9 mg/L to 27.2 mg/L) and 52.6 mg/L (44.1 mg/L to 62.6 mg/L) for positively charged polystyrene nanoparticles for the serum-free and serum conditions, respectively, and 49.7 µmol/L (47.5 µmol/L to 51.5 µmol/L) and 77.0 µmol/L (54.3 µmol/L to 99.4 µmol/L) for positive chemical control cadmium sulfate for the serum-free and serum conditions, respectively. Results from the measurement controls can be used to evaluate the sources of variability and their relative magnitudes within and between laboratories. This information revealed steps of the protocol that may need to be modified to improve the overall robustness and precision. The results suggest that protocol details such as cell line ID, media exchange, cell handling, and nanoparticle dispersion are critical to ensure protocol robustness and comparability of nanocytotoxicity assay results. The combination of system control measurements and interlaboratory comparison data yielded insights that would not have been available by either approach by itself.

Surface plasmon resonance microscopy: Achieving a quantitative optical response.

Surface plasmon resonance (SPR) imaging allows real-time label-free imaging based on index of refraction and changes in index of refraction at an interface. Optical parameter analysis is achieved by application of the Fresnel model to SPR data typically taken by an instrument in a prism based figuration. We carry out SPR imaging on a microscope by launching light into a sample and collecting reflected light through a high numerical aperture microscope objective. The SPR microscope enables spatial resolution that approaches the diffraction limit and has a dynamic range that allows detection of subnanometer to submicrometer changes in thickness of biological material at a surface. However, unambiguous quantitative interpretation of SPR changes using the microscope system could not be achieved using the Fresnel model because of polarization dependent attenuation and optical aberration that occurs in the high numerical aperture objective. To overcome this problem, we demonstrate a model to correct for polarization diattenuation and optical aberrations in the SPR data and develop a procedure to calibrate reflectivity to index of refraction values. The calibration and correction strategy for quantitative analysis was validated by comparing the known indices of refraction of bulk materials with corrected SPR data interpreted with the Fresnel model. Subsequently, we applied our SPR microscopy method to evaluate the index of refraction for a series of polymer microspheres in aqueous media and validated the quality of the measurement with quantitative phase microscopy.

Strategies for Achieving Measurement Assurance for Cell Therapy Products.

UNLABELLED: The cell therapy industry has identified the inability to reliably characterize cells as possibly its greatest challenge and has called for standards and reference materials to provide assurance for measurements of cell properties. The challenges in characterization of cell therapy products can be largely addressed with systematic approaches for assessing sources of uncertainty and improving confidence in key measurements. This article presents the many strategies that can be used to ensure measurement confidence and discusses them in terms of how they can be applied to characterization of cell therapy products. SIGNIFICANCE: Application of these strategies to cell measurements will help to establish qualified assays for cell characterization, which may help streamline regulatory approval and enable more efficient development of cell therapy products.

Large-scale time-lapse microscopy of Oct4 expression in human embryonic stem cell colonies.

Identification and quantification of the characteristics of stem cell preparations is critical for understanding stem cell biology and for the development and manufacturing of stem cell based therapies. We have developed image analysis and visualization software that allows effective use of time-lapse microscopy to provide spatial and dynamic information from large numbers of human embryonic stem cell colonies. To achieve statistically relevant sampling, we examined >680 colonies from 3 different preparations of cells over 5days each, generating a total experimental dataset of 0.9 terabyte (TB). The 0.5 Giga-pixel images at each time point were represented by multi-resolution pyramids and visualized using the Deep Zoom Javascript library extended to support viewing Giga-pixel images over time and extracting data on individual colonies. We present a methodology that enables quantification of variations in nominally-identical preparations and between colonies, correlation of colony characteristics with Oct4 expression, and identification of rare events.