Integrated optical device for Structured Illumination Microscopy.

Structured Illumination Microscopy (SIM) is a key technology for high resolution and super-resolution imaging of biological cells and molecules. The spread of portable and easy-to-align SIM systems requires the development of novel methods to generate a light pattern and to shift it across the field of view of the microscope. Here we show a miniaturized chip that incorporates optical waveguides, splitters, and phase shifters, to generate a 2D structured illumination pattern suitable for SIM microscopy. The chip creates three point-sources, coherent and controlled in phase, without the need for further alignment. Placed in the pupil of a microscope's objective, the three sources generate a hexagonal illumination pattern on the sample, which is spatially translated thanks to thermal phase shifters. We validate and use the chip, upgrading a commercial inverted fluorescence microscope to a SIM setup and we image biological sample slides, extending the resolution of the microscope.

Robust deep learning optical autofocus system applied to automated multiwell plate single molecule localization microscopy.

We presenta robust, long-range optical autofocus system for microscopy utilizing machine learning. This can be useful for experiments with long image data acquisition times that may be impacted by defocusing resulting from drift of components, for example due to changes in temperature or mechanical drift. It is also useful for automated slide scanning or multiwell plate imaging where the sample(s) to be imaged may not be in the same horizontal plane throughout the image data acquisition. To address the impact of (thermal or mechanical) fluctuations over time in the optical autofocus system itself, we utilize a convolutional neural network (CNN) that is trained over multiple days to account for such fluctuations. To address the trade-off between axial precision and range of the autofocus, we implement orthogonal optical readouts with separate CNN training data, thereby achieving an accuracy well within the 600 nm depth of field of our 1.3 numerical aperture objective lens over a defocus range of up to approximately +/-100 µm. We characterize the performance of this autofocus system and demonstrate its application to automated multiwell plate single molecule localization microscopy.

Many microscopy experiments involve extended imaging of samples over timescales from minutes to days, during which the microscope can 'drift' out of focus. When imaging at high magnification, the depth of field is of the order of one micron and so the imaging system should keep the sample in the focal plane of the microscope objective lens to this precision. Unfortunately, temperature changes in the laboratory can cause thermal expansion of microscope components that can move the focal plane by more than a micron and such changes can occur on a timescale of minutes. This is a particular issue for super-resolved microscopy experiments using single molecule localization microscopy (SMLM) techniques, for which 1000s of images are acquired, and for automated imaging of multiple samples in multiwell plates. It is possible to maintain the sample in the focal plane focus position by either automatically moving the sample or adjusting the imaging system, for example by moving the objective lens. This is called 'autofocus' and is frequently achieved by reflecting a light beam from the microscope coverslip and measuring its position of beam profile as a function of defocus of the microscope. The correcting adjustment is then usually calculated analytically but there is recent interest in using machine learning techniques to determine the required focussing adjustment. Here, we present a system that uses a neural network to determine the required defocus correcting adjustment from camera images of a laser beam that is reflected from the coverslip. Unfortunately, this approach will only work when the microscope is in the same condition as it was when the neural network was trained - and this can be compromised by the same drift of the optical system that causes the defocus needing to be corrected. We show, however, that by training a neural network over an extended period, for example 10 days, this approach can 'learn' about the optical system drifts and provide the required autofocus function. We also show that an optical system utilizing a rectangular slit can make two measurements of the defocus simultaneously, with one measurement being optimized for high accuracy over a limited range (±10 µm) near focus and the other providing lower accuracy but over a much longer range (±100 µm). This robust autofocus system is suitable for automated super-resolved microscopy of arrays of samples in a multiwell plate using SMLM, for which an experiment routinely lasts more than 5 h.

GPU-accelerated real-time reconstruction in Python of three-dimensional datasets from structured illumination microscopy with hexagonal patterns.

We present a structured illumination microscopy system that projects a hexagonal pattern by the interference among three coherent beams, suitable for implementation in a light-sheet geometry. Seven images acquired as the illumination pattern is shifted laterally can be processed to produce a super-resolved image that surpasses the diffraction-limited resolution by a factor of over 2 in an exemplar light-sheet arrangement. Three methods of processing data are discussed depending on whether the raw images are available in groups of seven, individually in a stream or as a larger batch representing a three-dimensional stack. We show that imaging axially moving samples can introduce artefacts, visible as fine structures in the processed images. However, these artefacts are easily removed by a filtering operation carried out as part of the batch processing algorithm for three-dimensional stacks. The reconstruction algorithms implemented in Python include specific optimizations for calculation on a graphics processing unit and we demonstrate its operation on experimental data of static objects and on simulated data of moving objects. We show that the software can process over 239 input raw frames per second at 512 × 512 pixels, generating over 34 super-resolved frames per second at 1024 × 1024 pixels. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

Improving axial resolution in Structured Illumination Microscopy using deep learning.

Structured Illumination Microscopy (SIM) is a widespread methodology to image live and fixed biological structures smaller than the diffraction limits of conventional optical microscopy. Using recent advances in image up-scaling through deep learning models, we demonstrate a method to reconstruct 3D SIM image stacks with twice the axial resolution attainable through conventional SIM reconstructions. We further demonstrate our method is robust to noise and evaluate it against two-point cases and axial gratings. Finally, we discuss potential adaptions of the method to further improve resolution. This article is part of the Theo Murphy meeting issue 'Super-resolution structured illumination microscopy (part 1)'.

All-optical crosstalk-free manipulation and readout of Chronos-expressing neurons.

All optical neurophysiology allows manipulation and readout of neural network activity with single-cell spatial resolution and millisecond temporal resolution. Neurons can be made to express proteins that actuate transmembrane currents upon light absorption, enabling optical control of membrane potential and action potential signalling. In addition, neurons can be genetically or synthetically labelled with fluorescent reporters of changes in intracellular calcium concentration or membrane potential. Thus, to optically manipulate and readout neural activity in parallel, two spectra are involved: the action spectrum of the actuator, and the absorption spectrum of the fluorescent reporter. Due to overlap in these spectra, previous all-optical neurophysiology paradigms have been hindered by spurious activation of neuronal activity caused by the readout light. Here, we pair the blue-green absorbing optogenetic actuator, Chronos, with a deep red-emitting fluorescent calcium reporter CaSiR-1. We show that cultured Chinese hamster ovary cells transfected with Chronos do not exhibit transmembrane currents when illuminated with wavelengths and intensities suitable for exciting one-photon CaSiR-1 fluorescence. We then demonstrate crosstalk-free, high signal-to-noise ratio CaSiR-1 red fluorescence imaging at 100 frames s-1 of Chronos-mediated calcium transients evoked in neurons with blue light pulses at rates up to 20 Hz. These results indicate that the spectral separation between red light excited fluorophores, excited efficiently at or above 640 nm, with blue-green absorbing opsins such as Chronos, is sufficient to avoid spurious opsin actuation by the imaging wavelengths and therefore enable crosstalk-free all-optical neuronal manipulation and readout.

Semi-random multicore fibre design for adaptive multiphoton endoscopy.

This paper reports the development, modelling and application of a semi-random multicore fibre (MCF) design for adaptive multiphoton endoscopy. The MCF was constructed from 55 sub-units, each comprising 7 single mode cores, in a hexagonally close-packed lattice where each sub-unit had a random angular orientation. The resulting fibre had 385 single mode cores and was double-clad for proximal detection of multiphoton excited fluorescence. The random orientation of each sub-unit in the fibre reduces the symmetry of the positions of the cores in the MCF, reducing the intensity of higher diffracted orders away from the central focal spot formed at the distal tip of the fibre and increasing the maximum size of object that can be imaged. The performance of the MCF was demonstrated by imaging fluorescently labelled beads with both distal and proximal fluorescence detection and pollen grains with distal fluorescence detection. We estimate that the number of independent resolution elements in the final image - measured as the half-maximum area of the two-photon point spread function divided by the area imaged - to be ~3200.

Adaptive multiphoton endomicroscopy through a dynamically deformed multicore optical fiber using proximal detection.

This paper demonstrates multiphoton excited fluorescence imaging through a polarisation maintaining multicore fiber (PM-MCF) while the fiber is dynamically deformed using all-proximal detection. Single-shot proximal measurement of the relative optical path lengths of all the cores of the PM-MCF in double pass is achieved using a Mach-Zehnder interferometer read out by a scientific CMOS camera operating at 416 Hz. A non-linear least squares fitting procedure is then employed to determine the deformation-induced lateral shift of the excitation spot at the distal tip of the PM-MCF. An experimental validation of this approach is presented that compares the proximally measured deformation-induced lateral shift in focal spot position to an independent distally measured ground truth. The proximal measurement of deformation-induced shift in focal spot position is applied to correct for deformation-induced shifts in focal spot position during raster-scanning multiphoton excited fluorescence imaging.

Optically assembled droplet interface bilayer (OptiDIB) networks from cell-sized microdroplets.

We report a new platform technology to systematically assemble droplet interface bilayer (DIB) networks in user-defined 3D architectures from cell-sized droplets using optical tweezers. Our OptiDIB platform is the first demonstration of optical trapping to precisely construct 3D DIB networks, paving the way for the development of a new generation of modular bio-systems.

Sonic hedgehog multimerization: a self-organizing event driven by post-translational modifications?

Sonic hedgehog (Shh) is a morphogen active during vertebrate development and tissue homeostasis in adulthood. Dysregulation of the Shh signalling pathway is known to incite carcinogenesis. Due to the highly lipophilic nature of this protein imparted by two post-translational modifications, Shh's method of transit through the aqueous extracellular milieu has been a long-standing conundrum, prompting the proposition of numerous hypotheses to explain the manner of its displacement from the surface of the producing cell. Detection of high molecular-weight complexes of Shh in the intercellular environment has indicated that the protein achieves this by accumulating into multimeric structures prior to release from producing cells. The mechanism of assembly of the multimers, however, has hitherto remained mysterious and contentious. Here, with the aid of high-resolution optical imaging and post-translational modification mutants of Shh, we show that the C-terminal cholesterol and the N-terminal palmitate adducts contribute to the assembly of large multimers and regulate their shape. Moreover, we show that small Shh multimers are produced in the absence of any lipid modifications. Based on an assessment of the distribution of various dimensional characteristics of individual Shh clusters, in parallel with deductions about the kinetics of release of the protein from the producing cells, we conclude that multimerization is driven by self-assembly underpinned by the law of mass action. We speculate that the lipid modifications augment the size of the multimolecular complexes through prolonging their association with the exoplasmic membrane.

Correction Approach for Delta Function Convolution Model Fitting of Fluorescence Decay Data in the Case of a Monoexponential Reference Fluorophore.

A correction is proposed to the Delta function convolution method (DFCM) for fitting a multiexponential decay model to time-resolved fluorescence decay data using a monoexponential reference fluorophore. A theoretical analysis of the discretised DFCM multiexponential decay function shows the presence an extra exponential decay term with the same lifetime as the reference fluorophore that we denote as the residual reference component. This extra decay component arises as a result of the discretised convolution of one of the two terms in the modified model function required by the DFCM. The effect of the residual reference component becomes more pronounced when the fluorescence lifetime of the reference is longer than all of the individual components of the specimen under inspection and when the temporal sampling interval is not negligible compared to the quantity (τR (-1) - τ(-1))(-1), where τR and τ are the fluorescence lifetimes of the reference and the specimen respectively. It is shown that the unwanted residual reference component results in systematic errors when fitting simulated data and that these errors are not present when the proposed correction is applied. The correction is also verified using real data obtained from experiment.

Experimental proof of concept of nanoparticle-assisted STED.

We imaged core-shell nanoparticles, consisting of a dye-doped silica core covered with a layer of gold, with a stimulated emission depletion, fluorescence lifetime imaging (STED-FLIM) microscope. Because of the field enhancement provided by the localized surface plasmon resonance of the gold shell, we demonstrate a reduction of the STED depletion power required to obtain resolution improvement by a factor of 4. This validates the concept of nanoparticle-assisted STED (NP-STED), where hybrid dye-plasmonic nanoparticles are used as labels for STED in order to decrease the depletion powers required for subwavelength imaging.

Analysis of DNA binding and nucleotide flipping kinetics using two-color two-photon fluorescence lifetime imaging microscopy.

Uracil DNA glycosylase plays a key role in DNA maintenance via base excision repair. Its role is to bind to DNA, locate unwanted uracil, and remove it using a base flipping mechanism. To date, kinetic analysis of this complex process has been achieved using stopped-flow analysis but, due to limitations in instrumental dead-times, discrimination of the "binding" and "base flipping" steps is compromised. Herein we present a novel approach for analyzing base flipping using a microfluidic mixer and two-color two-photon (2c2p) fluorescence lifetime imaging microscopy (FLIM). We demonstrate that 2c2p FLIM can simultaneously monitor binding and base flipping kinetics within the continuous flow microfluidic mixer, with results showing good agreement with computational fluid dynamics simulations.

Remodelling of cortical actin where lytic granules dock at natural killer cell immune synapses revealed by super-resolution microscopy.

Natural Killer (NK) cells are innate immune cells that secrete lytic granules to directly kill virus-infected or transformed cells across an immune synapse. However, a major gap in understanding this process is in establishing how lytic granules pass through the mesh of cortical actin known to underlie the NK cell membrane. Research has been hampered by the resolution of conventional light microscopy, which is too low to resolve cortical actin during lytic granule secretion. Here we use two high-resolution imaging techniques to probe the synaptic organisation of NK cell receptors and filamentous (F)-actin. A combination of optical tweezers and live cell confocal microscopy reveals that microclusters of NKG2D assemble into a ring-shaped structure at the centre of intercellular synapses, where Vav1 and Grb2 also accumulate. Within this ring-shaped organisation of NK cell proteins, lytic granules accumulate for secretion. Using 3D-structured illumination microscopy (3D-SIM) to gain super-resolution of ~100 nm, cortical actin was detected in a central region of the NK cell synapse irrespective of whether activating or inhibitory signals dominate. Strikingly, the periodicity of the cortical actin mesh increased in specific domains at the synapse when the NK cell was activated. Two-colour super-resolution imaging revealed that lytic granules docked precisely in these domains which were also proximal to where the microtubule-organising centre (MTOC) polarised. Together, these data demonstrate that remodelling of the cortical actin mesh occurs at the central region of the cytolytic NK cell immune synapse. This is likely to occur for other types of cell secretion and also emphasises the importance of emerging super-resolution imaging technology for revealing new biology.

Absolute quantification of protein copy number using a single-molecule-sensitive microarray.

We report the use of a microfluidic microarray incorporating single molecule detection for the absolute quantification of protein copy number in solution. In this paper we demonstrate protocols which enable calibration free detection for two protein detection assays. An EGFP protein assay has a limit of detection of <30 EGFP proteins in a microfluidic analysis chamber (limited by non-specific background binding), with a measured limit of linearity of approximately 6 × 10(6) molecules of analyte in the analysis chamber and a dynamic range of >5 orders of magnitude in protein concentration. An antibody sandwich assay was used to detect unlabelled human tumour suppressor protein p53 with a limit of detection of approximately 21 p53 proteins and a dynamic range of >3 orders of magnitude. We show that these protocols can be used to calibrate data retrospectively to determine the absolute protein copy number at the single cell level in two human cancer cell lines.

The grab-and-drop protocol: a novel strategy for membrane protein isolation and reconstitution from single cells.

We present a rapid and robust technique for the sampling of membrane-associated proteins from the surface of a single, live cell and their subsequent deposition onto a solid-supported lipid bilayer. As a proof of principle, this method has been used to extract green fluorescent protein (EGFP) labelled K-ras proteins located at the inner leaflet of the plasma membrane of colon carcinoma cells and to transfer them to an S-layer supported lipid bilayer system. The technique is non-destructive, meaning that both the cell and proteins are intact after the sampling operation, offering the potential for repeated measurements of the same cell of interest. This system provides the ideal tool for the investigation of cellular heterogeneity, as well as a platform for the investigation of rare cell types such as circulating tumour cells.

Double pass, common path method for arbitrary polarization control using a ferroelectric liquid crystal spatial light modulator.

We present a method for arbitrary control of the polarization of a light beam. Our method uses two holograms on a binary ferroelectric liquid crystal spatial light modulator (FLCSLM), and so has the potential to allow polarization state switching at kilohertz rates. Unlike previous methods that achieve polarization control using FLCSLMs, our method is common path and requires only the simplest optical components. For this reason, the method is very easy to setup, align, and maintain. In addition, it has the ability to modulate unpolarized input light. We demonstrate the formation of radially, azimuthally, and circularly polarized beams by imaging their focal spots formed at low numerical aperture.

Differential modes of DNA binding by mismatch uracil DNA glycosylase from Escherichia coli: implications for abasic lesion processing and enzyme communication in the base excision repair pathway.

Mismatch uracil DNA glycosylase (Mug) from Escherichia coli is an initiating enzyme in the base-excision repair pathway. As with other DNA glycosylases, the abasic product is potentially more harmful than the initial lesion. Since Mug is known to bind its product tightly, inhibiting enzyme turnover, understanding how Mug binds DNA is of significance when considering how Mug interacts with downstream enzymes in the base-excision repair pathway. We have demonstrated differential binding modes of Mug between its substrate and abasic DNA product using both band shift and fluorescence anisotropy assays. Mug binds its product cooperatively, and a stoichiometric analysis of DNA binding, catalytic activity and salt-dependence indicates that dimer formation is of functional significance in both catalytic activity and product binding. This is the first report of cooperativity in the uracil DNA glycosylase superfamily of enzymes, and forms the basis of product inhibition in Mug. It therefore provides a new perspective on abasic site protection and the findings are discussed in the context of downstream lesion processing and enzyme communication in the base excision repair pathway.

Structured-light-sheet imaging in an integrated optofluidic platform.

Heterogeneity investigation at the single-cell level reveals morphological and phenotypic characteristics in cell populations. In clinical research, heterogeneity has important implications in the correct detection and interpretation of prognostic markers and in the analysis of patient-derived material. Among single-cell analysis, imaging flow cytometry allows combining information retrieved by single cell images with the throughput of fluidic platforms. Nevertheless, these techniques might fail in a comprehensive heterogeneity evaluation because of limited image resolution and bidimensional analysis. Light sheet fluorescence microscopy opened new ways to study in 3D the complexity of cellular functionality in samples ranging from single-cells to micro-tissues, with remarkably fast acquisition and low photo-toxicity. In addition, structured illumination microscopy has been applied to single-cell studies enhancing the resolution of imaging beyond the conventional diffraction limit. The combination of these techniques in a microfluidic environment, which permits automatic sample delivery and translation, would allow exhaustive investigation of cellular heterogeneity with high throughput image acquisition at high resolution. Here we propose an integrated optofluidic platform capable of performing structured light sheet imaging flow cytometry (SLS-IFC). The system encompasses a multicolor directional coupler equipped with a thermo-optic phase shifter, cylindrical lenses and a microfluidic network to generate and shift a patterned light sheet within a microchannel. The absence of moving parts allows a stable alignment and an automated fluorescence signal acquisition during the sample flow. The platform enables 3D imaging of an entire cell in about 1 s with a resolution enhancement capable of revealing sub-cellular features and sub-diffraction limit details.

Application of ultrafast gold luminescence to measuring the instrument response function for multispectral multiphoton fluorescence lifetime imaging.

When performing multiphoton fluorescence lifetime imaging in multiple spectral emission channels, an instrument response function must be acquired in each channel if accurate measurements of complex fluorescence decays are to be performed. Although this can be achieved using the reference reconvolution technique, it is difficult to identify suitable fluorophores with a mono-exponential fluorescence decay across a broad emission spectrum. We present a solution to this problem by measuring the IRF using the ultrafast luminescence from gold nanorods. We show that ultrafast gold nanorod luminescence allows the IRF to be directly obtained in multiple spectral channels simultaneously across a wide spectral range. We validate this approach by presenting an analysis of multispectral autofluorescence FLIM data obtained from human skin ex vivo.

Interferometry using binary holograms without high order diffraction effects.

We describe a technique for a phase-stepping interferometer based on programmable binary phase holograms, particularly useful for optical testing of aspheric or free-form surfaces. It is well-known that binary holograms can be used to generate reference surfaces for interferometry, but a major problem is that cross talk from higher diffraction orders and aliasing can reduce the fidelity of the system. Here, we propose a new encoding technique which improves the accuracy of the technique and demonstrate its implementation using a binary liquid crystal spatial light modulator.