Size and Refractive Index Determination of Subwavelength Particles and Air Bubbles by Holographic Nanoparticle Tracking Analysis.

Determination of size and refractive index (RI) of dispersed unlabeled subwavelength particles is of growing interest in several fields, including biotechnology, wastewater monitoring, and nanobubble preparations. Conventionally, the size distribution of such samples is determined via the Brownian motion of the particles, but simultaneous determination of their RI remains challenging. This work demonstrates nanoparticle tracking analysis (NTA) in an off-axis digital holographic microscope (DHM) enabling determination of both particle size and RI of individual subwavelength particles from the combined information about size and optical phase shift. The potential of the method to separate particle populations is demonstrated by analyzing a mixture of three types of dielectric particles within a narrow size range, where conventional NTA methods based on Brownian motion alone would fail. Using this approach, the phase shift allowed individual populations of dielectric beads overlapping in either size or RI to be clearly distinguished and quantified with respect to these properties. The method was furthermore applied for analysis of surfactant-stabilized micro- and nanobubbles, with RI lower than that of water. Since bubbles induce a phase shift of opposite sign to that of solid particles, they were easily distinguished from similarly sized solid particles made up of undissolved surfactant. Surprisingly, the dependence of the phase shift on bubble size indicates that only those with 0.15-0.20 µm radius were individual bubbles, whereas larger bubbles were actually clusters of bubbles. This label-free means to quantify multiple parameters of suspended individual submicrometer particles offers a crucial complement to current characterization strategies, suggesting broad applicability for a wide range of nanoparticle systems.

Nanoalignment by critical Casimir torques.

The manipulation of microscopic objects requires precise and controllable forces and torques. Recent advances have led to the use of critical Casimir forces as a powerful tool, which can be finely tuned through the temperature of the environment and the chemical properties of the involved objects. For example, these forces have been used to self-organize ensembles of particles and to counteract stiction caused by Casimir-Liftshitz forces. However, until now, the potential of critical Casimir torques has been largely unexplored. Here, we demonstrate that critical Casimir torques can efficiently control the alignment of microscopic objects on nanopatterned substrates. We show experimentally and corroborate with theoretical calculations and Monte Carlo simulations that circular patterns on a substrate can stabilize the position and orientation of microscopic disks. By making the patterns elliptical, such microdisks can be subject to a torque which flips them upright while simultaneously allowing for more accurate control of the microdisk position. More complex patterns can selectively trap 2D-chiral particles and generate particle motion similar to non-equilibrium Brownian ratchets. These findings provide new opportunities for nanotechnological applications requiring precise positioning and orientation of microscopic objects.

Roadmap on Deep Learning for Microscopy.

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Microplankton life histories revealed by holographic microscopy and deep learning.

The marine microbial food web plays a central role in the global carbon cycle. However, our mechanistic understanding of the ocean is biased toward its larger constituents, while rates and biomass fluxes in the microbial food web are mainly inferred from indirect measurements and ensemble averages. Yet, resolution at the level of the individual microplankton is required to advance our understanding of the microbial food web. Here, we demonstrate that, by combining holographic microscopy with deep learning, we can follow microplanktons throughout their lifespan, continuously measuring their three-dimensional position and dry mass. The deep-learning algorithms circumvent the computationally intensive processing of holographic data and allow rapid measurements over extended time periods. This permits us to reliably estimate growth rates, both in terms of dry mass increase and cell divisions, as well as to measure trophic interactions between species such as predation events. The individual resolution provides information about selectivity, individual feeding rates, and handling times for individual microplanktons. The method is particularly useful to detail the rates and routes of organic matter transfer in micro-zooplankton, the most important and least known group of primary consumers in the oceans. Studying individual interactions in idealized small systems provides insights that help us understand microbial food webs and ultimately larger-scale processes. We exemplify this by detailed descriptions of micro-zooplankton feeding events, cell divisions, and long-term monitoring of single cells from division to division.

Picture a glass of seawater. It looks clear and empty, but in reality, it contains one hundred million bacteria, about one hundred thousand other single-celled organisms, and a few microscopic animals. In fact, the majority of life in the ocean is microscopic and we know relatively little about it. Nevertheless, these microbes have a major impact on our lives. Microscopic algae known as phytoplankton, for example, produce half of the oxygen we breathe. For animals, birds and other large organisms in the ocean, we have a good understanding of who eats who and where the material ends up. However, for phytoplankton and other microbes, we depend on bulk measurements and averages of large groups. Bachimanchi et al. developed a method to follow individual microbes living in seawater and to observe how they move, grow, consume each other and reproduce. The team combined holographic microscopy with artificial intelligence to follow multiple planktons, diatoms and other microbes throughout their life span and continuously measured their three-dimensional location and mass. This made it possible to estimate how fast the organisms were growing and moving, and to observe what they ate. The experiments revealed new insights into how micro-zooplankton, diatoms and other microbes in the ocean interact with each other. This new method may be useful for researchers who would like to track the movements and whereabouts of microscopic planktons, bacteria or other microbes for extended periods of time. It is also a rapid method for counting, sizing, and weighing cells in suspension. The hardware used in this method is relatively cheap, and Bachimanchi et al. have shared all the computer code with examples and demonstrations in a public database to enable other researchers to use it.

Single-shot self-supervised object detection in microscopy.

Object detection is a fundamental task in digital microscopy, where machine learning has made great strides in overcoming the limitations of classical approaches. The training of state-of-the-art machine-learning methods almost universally relies on vast amounts of labeled experimental data or the ability to numerically simulate realistic datasets. However, experimental data are often challenging to label and cannot be easily reproduced numerically. Here, we propose a deep-learning method, named LodeSTAR (Localization and detection from Symmetries, Translations And Rotations), that learns to detect microscopic objects with sub-pixel accuracy from a single unlabeled experimental image by exploiting the inherent roto-translational symmetries of this task. We demonstrate that LodeSTAR outperforms traditional methods in terms of accuracy, also when analyzing challenging experimental data containing densely packed cells or noisy backgrounds. Furthermore, by exploiting additional symmetries we show that LodeSTAR can measure other properties, e.g., vertical position and polarizability in holographic microscopy.

Fast and Accurate Nanoparticle Characterization Using Deep-Learning-Enhanced Off-Axis Holography.

Characterization of suspended nanoparticles in their native environment plays a central role in a wide range of fields, from medical diagnostics and nanoparticle-enhanced drug delivery to nanosafety and environmental nanopollution assessment. Standard optical approaches for nanoparticle sizing assess the size via the diffusion constant and, as a consequence, require long trajectories and that the medium has a known and uniform viscosity. However, in most biological applications, only short trajectories are available, while simultaneously, the medium viscosity is unknown and tends to display spatiotemporal variations. In this work, we demonstrate a label-free method to quantify not only size but also refractive index of individual subwavelength particles using 2 orders of magnitude shorter trajectories than required by standard methods and without prior knowledge about the physicochemical properties of the medium. We achieved this by developing a weighted average convolutional neural network to analyze holographic images of single particles, which was successfully applied to distinguish and quantify both size and refractive index of subwavelength silica and polystyrene particles without prior knowledge of solute viscosity or refractive index. We further demonstrate how these features make it possible to temporally resolve aggregation dynamics of 31 nm polystyrene nanoparticles, revealing previously unobserved time-resolved dynamics of the monomer number and fractal dimension of individual subwavelength aggregates.

Active droploids.

Active matter comprises self-driven units, such as bacteria and synthetic microswimmers, that can spontaneously form complex patterns and assemble into functional microdevices. These processes are possible thanks to the out-of-equilibrium nature of active-matter systems, fueled by a one-way free-energy flow from the environment into the system. Here, we take the next step in the evolution of active matter by realizing a two-way coupling between active particles and their environment, where active particles act back on the environment giving rise to the formation of superstructures. In experiments and simulations we observe that, under light-illumination, colloidal particles and their near-critical environment create mutually-coupled co-evolving structures. These structures unify in the form of active superstructures featuring a droplet shape and a colloidal engine inducing self-propulsion. We call them active droploids-a portmanteau of droplet and colloids. Our results provide a pathway to create active superstructures through environmental feedback.

Extracting quantitative biological information from bright-field cell images using deep learning.

Quantitative analysis of cell structures is essential for biomedical and pharmaceutical research. The standard imaging approach relies on fluorescence microscopy, where cell structures of interest are labeled by chemical staining techniques. However, these techniques are often invasive and sometimes even toxic to the cells, in addition to being time consuming, labor intensive, and expensive. Here, we introduce an alternative deep-learning-powered approach based on the analysis of bright-field images by a conditional generative adversarial neural network (cGAN). We show that this is a robust and fast-converging approach to generate virtually stained images from the bright-field images and, in subsequent downstream analyses, to quantify the properties of cell structures. Specifically, we train a cGAN to virtually stain lipid droplets, cytoplasm, and nuclei using bright-field images of human stem-cell-derived fat cells (adipocytes), which are of particular interest for nanomedicine and vaccine development. Subsequently, we use these virtually stained images to extract quantitative measures about these cell structures. Generating virtually stained fluorescence images is less invasive, less expensive, and more reproducible than standard chemical staining; furthermore, it frees up the fluorescence microscopy channels for other analytical probes, thus increasing the amount of information that can be extracted from each cell. To make this deep-learning-powered approach readily available for other users, we provide a Python software package, which can be easily personalized and optimized for specific virtual-staining and cell-profiling applications.