Roadmap on Deep Learning for Microscopy.

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Label-free nanofluidic scattering microscopy of size and mass of single diffusing molecules and nanoparticles.

Label-free characterization of single biomolecules aims to complement fluorescence microscopy in situations where labeling compromises data interpretation, is technically challenging or even impossible. However, existing methods require the investigated species to bind to a surface to be visible, thereby leaving a large fraction of analytes undetected. Here, we present nanofluidic scattering microscopy (NSM), which overcomes these limitations by enabling label-free, real-time imaging of single biomolecules diffusing inside a nanofluidic channel. NSM facilitates accurate determination of molecular weight from the measured optical contrast and of the hydrodynamic radius from the measured diffusivity, from which information about the conformational state can be inferred. Furthermore, we demonstrate its applicability to the analysis of a complex biofluid, using conditioned cell culture medium containing extracellular vesicles as an example. We foresee the application of NSM to monitor conformational changes, aggregation and interactions of single biomolecules, and to analyze single-cell secretomes.

Nanoplasmonic-Nanofluidic Single-Molecule Biosensors for Ultrasmall Sample Volumes.

Detection of small amounts of biological compounds is of ever-increasing importance but also remains an experimental challenge. In this context, plasmonic nanoparticles have emerged as strong contenders enabling label-free optical sensing with single-molecule resolution. However, the performance of a plasmonic single-molecule biosensor is not only dependent on its ability to detect a molecule but equally importantly on its efficiency to transport it to the binding site. Here, we present a theoretical study of the impact of downscaling fluidic structures decorated with plasmonic nanoparticles from conventional microfluidics to nanofluidics. We find that for ultrasmall picolitre sample volumes, nanofluidics enables unprecedented binding characteristics inaccessible with conventional microfluidic devices, and that both detection times and number of detected binding events can be improved by several orders of magnitude. Therefore, we propose nanoplasmonic-nanofluidic biosensing platforms as an efficient tool that paves the way for label-free single-molecule detection from ultrasmall volumes, such as single cells.

Analyte transport to micro- and nano-plasmonic structures.

The study of optical affinity biosensors based on plasmonic nanostructures has received significant attention in recent years. The sensing surfaces of these biosensors have complex architectures, often composed of localized regions of high sensitivity (electromagnetic hot spots) dispersed along a dielectric substrate having little to no sensitivity. Under conditions such that the sensitive regions are selectively functionalized and the remaining regions passivated, the rate of analyte capture (and thus the sensing performance) will have a strong dependence on the nanoplasmonic architecture. Outside of a few recent studies, there has been little discussion on how changes to a nanoplasmonic architecture will affect the rate of analyte transport. We recently proposed an analytical model to predict transport to such complex architectures; however, those results were based on numerical simulation and to date, have only been partially verified. In this study we measure the characteristics of analyte transport across a wide range of plasmonic structures, varying both in the composition of their base plasmonic element (microwires, nanodisks, and nanorods) and the packing density of such elements. We functionalized each structure with nucleic acid-based bioreceptors, where for each structure we used analyte/receptor sequences as to maintain a Damköhler number close to unity. This method allows to extract both kinetic (in the form of association and dissociation constants) and analyte transport parameters (in the form of a mass transfer coefficient) from sensorgrams taken from each substrate. We show that, despite having large differences in optical characteristics, measured rates of analyte transport for all plasmonic structures match very well to predictions using our previously proposed model. These results highlight that, along with optical characteristics, analyte transport plays a large role in the overall sensing performance of a nanoplasmonic biosensor.

High-performance biosensor exploiting a light guidance in sparse arrays of metal nanoparticles.

We introduce a new approach to plasmonic biosensing with superior biosensing properties based on spectroscopy of an electromagnetic mode guided by a monolayer of sparsely distributed colloidal plasmonic nanoparticles. The theoretical prediction of optical and sensing performance is confirmed by an experimental study in which adsorption of biomolecules on the sensor surface is studied. An unprecedentedly high figure of merit related to surface refractive index changes (FOMS) is demonstrated for distances of the biomolecules from the sensor surface up to 30 nm, which makes this approach a promising candidate for localized biosensing.

Biomolecular charges influence the response of surface plasmon resonance biosensors through electronic and ionic mechanisms.

Surface plasmon resonance (SPR) biosensors have become an important label-free optical biomolecular sensing technology and a "gold standard" for retrieving information on the kinetics of biomolecular interactions. Even though biomolecules typically contain an abundance of easily ionizable chemical groups, there is a gap in understanding of whether (and how) the electrostatic charge of a biomolecular system influences the SPR biosensor response. In this work we show that negative static charge present in a biomolecular layer on the surface of an SPR sensor results in significant SPR spectral shifts, and we identify two major mechanisms responsible for such shifts: 1) the formation of an electrical double layer (ionic mechanism), and 2) changes in the electron density at the surface of a metal (electronic mechanism). We show that under low ionic strength conditions, the electronic mechanism is dominant and the SPR wavelength shift is linearly proportional to the surface concentration of biomolecular charges. At high ionic strength conditions, both electric and ionic mechanisms contribute to the SPR wavelength shift. Using the electronic mechanism, we estimated the pKa of surface-bound carboxylic groups and the relative concentration of the carboxyl-terminated alkanethiols in a binary self-assembled monolayer of alkanethiols. The reported sensitivity of SPR to surface charge is especially important in the context of biomolecular sensing. Moreover, it provides an avenue for the application of SPR sensors for fast, label-free determination of the net charge of a biomolecular coating, which is of interest in material science, surface chemistry, electrochemistry, and other fields.

Nanoplasmonic Ruler for Measuring Separation Distance between Supported Lipid Bilayers and Oxide Surfaces.

Unraveling the details of how supported lipid bilayers (SLBs) are coupled to oxide surfaces is experimentally challenging, and there is an outstanding need to develop highly surface-sensitive measurement strategies to determine SLB separation distances. Indeed, subtle variations in separation distance can be associated with significant differences in bilayer-substrate interaction energy. Herein, we report a nanoplasmonic ruler strategy to measure the absolute separation distance between SLBs and oxide surfaces. A localized surface plasmon resonance (LSPR) sensor was employed to track SLB formation onto titania- and silica-coated gold nanodisk arrays. To interpret measurement data, an analytical model relating the LSPR measurement response to bilayer-substrate separation distance was developed based on finite-difference time-domain (FDTD) simulations and theoretical calculations. The results indicate that there is a larger separation distance between SLBs and titania surfaces than silica surfaces, and the trend was consistent across three tested lipid compositions. We discuss these findings within the context of the interfacial forces underpinning bilayer-substrate interactions, and the nanoplasmonic ruler strategy provides the first direct experimental evidence comparing SLB separation distances on titania and silica surfaces.

Nanoplasmonic ruler to measure lipid vesicle deformation.

A nanoplasmonic ruler method is presented in order to measure the deformation of adsorbed, nm-scale lipid vesicles on solid supports. It is demonstrated that single adsorbed vesicles undergo greater deformation on silicon oxide over titanium oxide, offering direct experimental evidence to support membrane tension-based theoretical models of supported lipid bilayer formation.

Enhancing sensitivity of surface plasmon resonance biosensors by functionalized gold nanoparticles: size matters.

We study how the size of spherical gold nanoparticles (AuNPs) influences their ability to enhance the response of optical biosensors based on surface plasmon resonance (SPR). We present a theoretical model that relates the enhancement generated by the AuNPs to their composition, size, and concentration, thus allowing for accurate predictions regarding the SPR sensor response to various AuNPs. The effect of the AuNP size is also investigated experimentally using an SPR biosensor for the detection of carcinoembryonic antigen (CEA) in which AuNPs covered with neutravidin (N-AuNPs) are used in the last step of a sandwich assay to enhance the sensor response to biotinylated secondary antibody against CEA. The experimental data are in excellent agreement with the results of the theoretical analysis. We demonstrate that the sensor response enhancement generated by the N-AuNPs is determined by (i) the sensor sensitivity to N-AuNP surface density (Sσ) and (ii) the ability of the N-AuNPs to bind to the functionalized surface of the sensor. Our results indicate that, while Sσ increases with the size of the N-AuNP, the ability of the functionalized surface of the sensor to bind the N-AuNPs is affected by steric effects and decreases with the size of N-AuNP.

Functionalized ultra-low fouling carboxy- and hydroxy-functional surface platforms: functionalization capacity, biorecognition capability and resistance to fouling from undiluted biological media.

The non-specific binding of non-target species to functionalized surfaces of biosensors continues to be challenge for biosensing in real-world media. Three different low-fouling and functionalizable surface platforms were employed to study the effect of functionalization on fouling resistance from several types of undiluted media including blood plasma and food media. The surface platforms investigated in this work included two polymer brushes: hydroxy-functional poly(2-hydroxyethyl methacrylate) (pHEMA) and carboxy-functional poly(carboxybetaine acrylamide) (pCBAA), and a standard OEG-based carboxy-functional alkanethiolate self-assembled monolayer (AT-SAM). The wet and dry polymer brushes were analyzed by AFM, ellipsometry, FT-IRRAS, and surface plasmon resonance (SPR). The surfaces were functionalized by the covalent attachment of antibodies, streptavidin, and oligonucleotides and the binding and biorecognition characteristics of the coatings were compared. We found that functionalization did not substantially affect the ultra-low fouling properties of pCBAA (plasma fouling of ~20 ng/cm(2)), a finding in contrast with pHEMA that completely lost its resistance to fouling after the activation of hydroxyl groups. Blocking a functionalized AT-SAM covalently with BSA decreased fouling down to the level comparable to unblocked pCBAA. However, the biorecognition capability of blocked functionalized AT-SAM was poor in comparison with functionalized pCBAA. Limits of detection of Escherichia coli O157:H7 in undiluted milk were determined to be 6×10(4), 8×10(5), and 6×10(5) cells/ml for pCBAA, pHEMA, and AT-SAM-blocked, respectively. Effect of analyte size on biorecognition activity of functionalized coatings was investigated and it was shown that the best performance in terms of overall fouling resistance and biorecognition capability is provided by pCBAA.

Sensing properties of lattice resonances of 2D metal nanoparticle arrays: an analytical model.

Theoretical study of sensing properties of lattice resonances supported by arrays of gold nanoparticles expressed in terms of the figure of merit (FOM) is reported. Analytical expressions for the FOM for surface and bulk refractive index changes are derived to establish the relationship between the sensing performance and design parameters and to allow for the design of nanoparticle arrays with optimal sensing performance. It is demonstrated that lattice resonances exhibit about two orders of magnitude higher bulk FOM than localized surface plasmon (LSP) resonance and that the surface FOM provided by lattice resonances and LSP resonances are comparable.

Configuration-controlled Au nanocluster arrays on inverse micelle nano-patterns: versatile platforms for SERS and SPR sensors.

Nanopatterned 2-dimensional Au nanocluster arrays with controlled configuration are fabricated onto reconstructed nanoporous poly(styrene-block-vinylpyridine) inverse micelle monolayer films. Near-field coupling of localized surface plasmons is studied and compared for disordered and ordered core-centered Au NC arrays. Differences in evolution of the absorption band and field enhancement upon Au nanoparticle adsorption are shown. The experimental results are found to be in good agreement with theoretical studies based on the finite-difference time-domain method and rigorous coupled-wave analysis. The realized Au nanopatterns are exploited as substrates for surface-enhanced Raman scattering and integrated into Kretschmann-type SPR sensors, based on which unprecedented SPR-coupling-type sensors are demonstrated.

Theoretical analysis of a fiber optic surface plasmon resonance sensor utilizing a Bragg grating.

A rigorous theoretical analysis of a fiber optic surface plasmon resonance sensor is presented. The sensor is based on the spectroscopy of mixed surface plasmon--fiber cladding modes excited by the fundamental mode of an optical fiber via a Bragg grating formed in the fiber core. The transmission spectrum is calculated by means of the Coupled Mode Theory. The modal structure is theoretically analyzed using a 3-D method based on a field expansion approach for matching the field continuity at the boundary of the layers. The theoretical analysis revealed a series of narrow transmission dips associated with the coupling of the fundamental mode to the mixed surface plasmon--fiber cladding modes. The sensitivity of these dips to changes in the refractive index of the analyte is calculated. Moreover, the refractive index resolution of the sensor was estimated to be better than 2 x 10(-6) RIU.