Deep learning in single-molecule imaging and analysis: recent advances and prospects.

Single-molecule microscopy is advantageous in characterizing heterogeneous dynamics at the molecular level. However, there are several challenges that currently hinder the wide application of single molecule imaging in bio-chemical studies, including how to perform single-molecule measurements efficiently with minimal run-to-run variations, how to analyze weak single-molecule signals efficiently and accurately without the influence of human bias, and how to extract complete information about dynamics of interest from single-molecule data. As a new class of computer algorithms that simulate the human brain to extract data features, deep learning networks excel in task parallelism and model generalization, and are well-suited for handling nonlinear functions and extracting weak features, which provide a promising approach for single-molecule experiment automation and data processing. In this perspective, we will highlight recent advances in the application of deep learning to single-molecule studies, discuss how deep learning has been used to address the challenges in the field as well as the pitfalls of existing applications, and outline the directions for future development.

Effect of water-soluble fullerenes on macrophage surface ultrastructure revealed by scanning ion conductance microscopy.

C60-fullerenes have unique potential in antiviral, drug delivery, photodynamic therapy and other biomedical applications. However, little is known about their effects on macrophage surface morphology and ultrastructure. Here by using contact-free scanning ion conductance microscopy (SICM), we investigated the effects of two water-soluble fullerenes on the surface ultrastructure and function of macrophages. The results showed that these fullerenes would be a promising phagocytosis inhibitor and SICM would be an excellent tool to study the morphological information of adhesive and fragile samples.

Coaxial illumination module of the stimulated-emission-depletion nanoscope.

Stimulated-emission-depletion (STED) nanoscope achieves super-resolution imaging by using a donut-shaped depletion beam to darken the fluorophores around the excitation spot. As an important factor determining the resolution of imaging, the coaxiality between the excitation and the depletion beam is required to be maintained at the nanoscale, which is often degraded by various interference such as ambient vibration and temperatures etc. Here, we propose a specially designed STED illumination module to guarantee the coaxiality between the two beams while modulating the phase of the depletion beam. This STED illumination module can realize phase modulation, polarization adjustment, pulse delay and two beams coaxial at the same time. With the experiments, the module can guarantee the two beams are stably coaxial for a long time. We imaged fluorescence particles with diameter 40 nm and got images of 40 nm full width at half maximum. Adjacent microfilaments at 80 nm being clearly distinguished with our STED nonoscope demonstrates that it could be well applied to biological samples.

Surface-Engineered Gold Nanoclusters for Stimulated Emission Depletion and Correlated Light and Electron Microscopy Imaging.

Stimulated emission depletion (STED) nanoscopy is an emerging super-resolution imaging platform for the study of the cellular structure. Developing suitable fluorescent probes of small size, good photostability, and easy functionalization is still in demand. Herein, we introduce a new type of surface-engineered gold nanoclusters (Au NCs) that are ultrasmall (1.7 nm) and ultrabright (QY = 60%) for STED bioimaging. A rigid shell formed by l-arginine (l-Arg) and 6-aza-2-thiothymine (ATT) on the Au NC surface enables not only its strong fluorescence in aqueous solution but also its easy chemical modification for specific biomolecule labeling. Au NCs show remarkable performance as STED nanoprobes, including high depletion efficiency, good photobleaching resistance, and low saturation intensity. Super-resolution imaging has been achieved with these Au NCs, and targeted nanoscopic imaging of cellular tubulin has been demonstrated. Moreover, the circular structure of lysosomes in live cells has been revealed. As a Au NC is also an ideal probe for electron microscopy, dual imaging of Aß42 aggregates with the single labeling probe of Au NCs has been realized in correlative light and electron microscopy (CLEM). This work reports, for the first time, the application of Au NCs as a novel probe in STED and CLEM imaging. With their excellent properties, Au NCs show promising potential for nanoscale bioimaging.

Fluorescence live cell imaging revealed wogonin targets mitochondria.

Scutellaria baicalensis is one of the widely used Chinese traditional medicines, and wogonin is one of major active components in it. However, the mechanism of action of wogonin has largely remained unclear. In this work, we designed a fluorescent probe, namely ATTO565-WGN, by conjugating wogonin with the fluorophore ATTO565 based on Mannich reaction via a flexible chain linker. In vitro assays verified that the ATTO565-WGN conjugate has a similar anti-proliferative activity to wogonin against human A549 and HeLa cancer cell lines. Combining co-localization and competition studies, confocal fluorescence imaging clearly demonstrated that the fluorescent wogonin probe predominantly located in mitochondrial area of living cells, indicating that wogonin acts at mitochondrion to exert its pharmacological functions. Significantly, the conjugated ATTO565 fluorophore conferred the wogonin probe STED (Stimulated Emission Depletion) feature, enabling STED fluorescence living cell imaging with a 55 nm of ultrahigh spatial resolution. This will greatly beneficial for the in situ investigation of interactions between wogonin and biological targets at the finely organized and dynamic mitochondria of living cells. Moreover, this work also provides novel insights into rational design of mitochondrion targeting fluorescence probes for ultrahigh resolution living cell imaging.

Exploiting the robust network structure of zein/low-acyl gellan gum nanocomplexes to create Pickering emulsion gels with favorable properties.

Zein/low-acyl gellan gum (GG) composite particles (ZGPs) were fabricated to stabilize Pickering emulsions (termed "ZGPEs"). The wettability of ZGPs was manipulated simply by adjusting the concentration of GG. The effects of GG concentration, oil fraction and pH on ZGPEs were systematically evaluated by confocal laser scanning microscopy (CLSM), cryo-scanning electron microscopy (cryo-SEM), dynamic light scattering technique, stimulated emission depletion (STED) nanoscopy and rheology. The results showed that ZGPEs exhibited robust colloidal properties and distinct advantage over other previously reported zein-polysaccharide-based Pickering emulsions. CLSM, STED and cryo-SEM analyses revealed that the network structures formed by GG and ZGPs at the continuous phase and oil-water interface were the main contributors to the emulsion's characteristics. This study provides insights into the fabrication of food-grade Pickering emulsions with distinct characteristics that impart favorable properties to various foods and bioactive delivery systems.

Highly Efficient Photochromic Tungsten Oxide@PNIPAM Composite Spheres with a Fast Response.

Coloration efficiency and a fast response are important in developing materials for optical switching. A novel, highly efficient photochromic tungsten oxide@poly(N-isopropylacrylamide) (PNIPAM) hybrid sphere is reported, whose colors can be rapidly converted between yellow and blue under different lights. The color change can be seen clearly even if the tungsten oxide content in the hybrid sphere is very low, exhibiting outstanding coloration efficiency of tungsten oxide. A photochromic mechanism is proposed in which the amide group in PNIPAM spheres participates in electron injection and the transition of valence states between W5+ and W6+ in the photochromic process. The interaction between tungsten oxide and PNIPAM plays a key role in enhancing the coloration efficiency of tungsten oxide and accelerating the switchable speed of color transformation, which is very useful in developing new photochromic materials. These hybrid spheres can be used in rewritable record displays and have wide potential applications in controlling energy transmittance in smart windows or in detecting UV light in optical sensors.

Analyzing protein dynamics from fluorescence intensity traces using unsupervised deep learning network.

We propose an unsupervised deep learning network to analyze the dynamics of membrane proteins from the fluorescence intensity traces. This system was trained in an unsupervised manner with the raw experimental time traces and synthesized ones, so neither predefined state number nor pre-labelling were required. With the bidirectional Long Short-Term Memory (biLSTM) networks as the hidden layers, both the past and future context can be used fully to improve the prediction results and can even extract information from the noise distribution. The method was validated with the synthetic dataset and the experimental dataset of monomeric fluorophore Cy5, and then applied to extract the membrane protein interaction dynamics from experimental data successfully.

Fluorescent Polymer Dot-Based Multicolor Stimulated Emission Depletion Nanoscopy with a Single Laser Beam Pair for Cellular Tracking.

Stimulated emission depletion (STED) nanoscopy provides subdiffraction resolution while preserving the benefits of fluorescence confocal microscopy in live-cell imaging. However, there are several challenges for multicolor STED nanoscopy, including sophisticated microscopy architectures, fast photobleaching, and cross talk of fluorescent probes. Here, we introduce two types of nanoscale fluorescent semiconducting polymer dots (Pdots) with different emission wavelengths: CNPPV (poly[2-methoxy-5-(2-ethylhexyloxy)-1,4-(1-cyanovinylene-1,4-phenylene)]) Pdots and PDFDP (poly[{9,9-dihexyl-2,7-bis(1-cyanovinylene)fluorene}-alt-co-{2,5-bis (N,N'-diphenylamino)-1,4-phenylene}]) Pdots, for dual-color STED bioimaging and cellular tracking. Besides bright fluorescence, strong photostability, and easy bioconjugation, these Pdots have large Stokes shifts, which make it possible to share both excitation and depletion beams, thus requiring only a single pair of laser beams for the dual-color STED imaging. Long-term tracking of cellular organelles by the Pdots has been achieved in living cells, and the dynamic interaction of endosomes derived from clathrin-mediated and caveolae-mediated endocytic pathways has been monitored for the first time to propose their interaction models. These results demonstrate the promise of Pdots as excellent probes for live-cell multicolor STED nanoscopy.

Analysis of the Diffusivity Change from Single-Molecule Trajectories on Living Cells.

With the wide application of live-cell single-molecule imaging and tracking of biomolecules at work, deriving diffusion state changes of individual molecules is of particular interest as these changes reflect molecular oligomerization or interaction with other cellular components and thus correlate with functional changes. We have developed a Rayleigh mixture distribution-based hidden Markov model method to analyze time-lapse diffusivity change of single molecules, especially membrane proteins, with unknown dynamic states in living cells. With this method, the diffusion parameters, including diffusion state number, state transition probability, diffusion coefficient, and state mixture ratio, can be extracted from the single-molecule diffusion trajectories accurately via easy computation. The validity of our method has been demonstrated with not only experiments on synthetic trajectories but also single-molecule fluorescence imaging data of two typical membrane receptors. Our method offers a new analytical tool for the investigation of molecular interaction kinetics at the single-molecule level.

Effects of pectin polydispersity on zein/pectin composite nanoparticles (ZAPs) as high internal-phase Pickering emulsion stabilizers.

In the present study, the properties of two apple sourced-pectin (AP-1 and AP-2) were comparative studied, and their influence on the formation of high internal-phase Pickering emulsions (HIPPEs) was investigated. Results showed that AP-2 has lower polydispersity index (PDI = 2.51) than AP-1. Zein/AP-2 complex nanoparticles (ZAPs-2) was able to stabilize 80% oil-phase to form HIPPEs, while ZAPs-1 failed to remain stable at same oil fraction. After correlating GPC (Gel Permeation Chromatography) results of pectins with their emulsion behavior, pectin PDI was found to play an important role in HIPPEs formation. Storage experiments and rheological properties analysis showed that HIPPEs exerted excellent stability and plasticity. Besides, super-resolution microscopy (including cryo-SEM and STED nanoscopy) depicted an intuitive interface structure of HIPPEs. These findings may contribute some basis to manipulating emulsion performance by adjusting pectin properties, as well as to further understanding the behavior of ZAPs at O/W interface.

Automated Stoichiometry Analysis of Single-Molecule Fluorescence Imaging Traces via Deep Learning.

The stoichiometry of protein complexes is precisely regulated in cells and is fundamental to protein function. Singe-molecule fluorescence imaging based photobleaching event counting is a new approach for protein stoichiometry determination under physiological conditions. Due to the interference of the high noise level and photoblinking events, accurately extracting real bleaching steps from single-molecule fluorescence traces is still a challenging task. Here, we develop a novel method of using convolutional and long-short-term memory deep learning neural network (CLDNN) for photobleaching event counting. We design the  convolutional layers to accurately extract features of steplike photobleaching drops and long-short-term memory (LSTM) recurrent layers to distinguish between photobleaching and photoblinking events. Compared with traditional algorithms, CLDNN shows higher accuracy with at least 2 orders of magnitude improvement of efficiency, and it does not require user-specified parameters. We have verified our CLDNN method using experimental data from imaging of single dye-labeled molecules in vitro and epidermal growth factor receptors (EGFR) on cells. Our CLDNN method is expected to provide a new strategy to stoichiometry study and time series analysis in chemistry.

Quantitative Characterization of the Membrane Dynamics of Newly Delivered TGF-ß Receptors by Single-Molecule Imaging.

The dynamics and stoichiometry of receptors newly delivered on the plasma membrane play a vital role in cell signal transduction, yet knowledge of this process is limited because of the lack of suitable analytical methods. Here we developed a new strategy that combines single-molecule imaging (SMI) and fluorescence recovery after photobleaching (FRAP), named FRAP-SMI, to monitor and quantify individual newly delivered and inserted transmembrane receptors on plasma membranes of living cells. Transforming-growth-factor-ß type II receptor (TßRII), a typical serine/threoninekinase receptor, was studied with this method. We first eliminated the fluorescence signals from the pre-existing EGFP-labeled TßRII molecules on the plasma membrane, and then we recorded the individual newly appeared TßRII-GFP by total-internal-reflection fluorescence imaging. The fluorescence-intensity distributions, photobleaching steps, and diffusion rates of the single TßRII-GFP molecules were analyzed. We reported, for the first time, that TßRII was transported to the plasma membrane mainly in the monomeric form in both resting and TGF-ß1stimulated cells. This strongly supported our former discovery that TßRII could exist as a monomer on the cell membrane. We also found that ligand stimulation resulted in enhanced delivery rates and prolonged membrane-association times for the TßRII molecules. On the basis of these observations, we proposed a mechanism of TGF-ß1-induced TßRII dimerization for receptor activation. Our method provides a useful tool for the real-time quantification of the spatial arrangement, mobility, and oligomerization of cell-surface proteins in living cells, thus providing a better understanding of cell signaling.

Nanoscale Distribution of Transforming Growth Factor Receptor on Post-Golgi Vesicle Revealed by Super-resolution Microscopy.

Transforming growth factor-ß (TGF-ß) type II receptor (TßRII) plays a critical role in the initiation of TGF-ß signaling pathway; therefore, the study of its synthesis and transport processes is of great important. In this work, we achieved super-resolution imaging of a new type of TßRII-containing post-Golgi vesicle by our home-built stimulated emission depletion (STED) microscope. We visualized the ring-shaped structure of these vesicles containing newly synthesized TßRII in the cytoplasm and characterized their size distribution from 300 to 1000 nm. These vesicles could be swollen by chloroquine treatment. Further investigation revealed that TßRII formed clusters on the outer ring of the post-Golgi vesicles. This study offers new information on the intracellular transportation of TGF-ß receptors for better understanding its signaling process.

Single-Molecule Imaging Reveals the Activation Dynamics of Intracellular Protein Smad3 on Cell Membrane.

Smad3 is an intracellular protein that plays a key role in propagating transforming growth factor ß (TGF-ß) signals from cell membrane to nucleus. However whether the transient process of Smad3 activation occurs on cell membrane and how it is regulated remains elusive. Using advanced live-cell single-molecule fluorescence microscopy to image and track fluorescent protein-labeled Smad3, we observed and quantified, for the first time, the dynamics of individual Smad3 molecules docking to and activation on the cell membrane. It was found that Smad3 docked to cell membrane in both unstimulated and stimulated cells, but with different diffusion rates and dissociation kinetics. The change in its membrane docking dynamics can be used to study the activation of Smad3. Our results reveal that Smad3 binds with type I TGF-ß receptor (TRI) even in unstimulated cells. Its activation is regulated by TRI phosphorylation but independent of receptor endocytosis. This study offers new information on TGF-ß/Smad signaling, as well as a new approach to investigate the activation of intracellular signaling proteins for a better understanding of their functions in signal transduction.

Single-molecule imaging reveals the stoichiometry change of ß2-adrenergic receptors by a pharmacological biased ligand.

The stoichiometry of the ß2-adrenergic receptor (ß2AR) was determined using single-molecule fluorescence imaging in living cells. The results showed that ß2AR mainly existed as monomers under physiological conditions and exhibited ß-arrestin-dependent dimerization upon stimulation with the pharmacological biased ligand carvedilol. The association of ß2AR dimerization with biased signalling is revealed.

Study of the interactions between endolysin and bacterial peptidoglycan on S. aureus by dynamic force spectroscopy.

The cell wall binding domain (CBD) of bacteriophage lysins can recognize target bacteria with extraordinary specificity through binding to bacterial peptidoglycan, thus it is a promising new probe to identify the corresponding bacterial pathogen. In this work, we used atomic force microscopy (AFM) based single-molecule force spectroscopy to investigate the interaction between the CBD of lysin PlyV12 (PlyV12C) and pathogenic bacterium Staphylococcus aureus (S. aureus). The binding forces of PlyV12C with S. aureus have been measured, and the dissociation process of their binding complex has been characterized. Furthermore, we compared the interactions of PlyV12C-S. aureus and antibody-S. aureus. It is revealed that PlyV12C has a comparable affinity to bacterial peptidoglycans as that of the S. aureus antibody. The results provide new information on the binding properties of lysin CBD with bacterium, and the application of lysin CBD in bacterium detection.

Mammalian actin-binding protein 1/HIP-55 is essential for the scission of clathrin-coated pits by regulating dynamin-actin interaction.

Actin and dynamin work cooperatively to drive the invagination and scission of clathrin-coated pits (CCPs). However, little is known about the mechanism that orchestrates the spatiotemporal recruitment of dynamin and actin. Here, we have identified the mammalian actin-binding protein 1 (mAbp1; also called HIP-55 or SH3P7), which could bind to clathrin, actin, as well as dynamin, as an adaptor that links the dynamic recruitment of dynamin and actin for the scission of CCPs. Live-cell imaging reveals that mAbp1 is specifically recruited at a late stage of the long-lived CCPs. mAbp1 knockdown impaired CCP scission by reducing dynamin recruitment at the plasma membrane. However, actin disruption remarkably eliminates mAbp1 recruitment and thus dynamin recruitment. These data suggest that by binding to both clathrin and F-actin, mAbp1 is specifically recruited at a late stage of CCP formation, which subsequently recruits dynamin to CCPs.

Analysis of the steps in single-molecule photobleaching traces by using the hidden markov model and maximum-likelihood clustering.

The step analysis of single-molecule photobleaching data offers a new approach for studying protein stoichiometry under physiological conditions. As such, it is important to develop suitable algorithms that can accurately extract the step events from the noisy single-molecule data. Herein, we report a HMM method that combines maximum-likelihood clustering for initializing the emission-probability distribution of the HMMs with an extended silhouette clustering criterion for estimating the state number of single molecules. In this way, the limitations of standard HMM in terms of processing typical single-molecule data with a short sequence are overcome. By using this method, the number and time points of the step events are automatically determined, without the introduction of any subjectivity. Simulation experiments on the experimental photobleaching data indicate that our method is very effective and robust in the analysis of single-molecule fluorescence photobleaching curves if the signal/noise ratio is larger than 2:1. This method was employed for processing photobleaching data that were obtained from single-molecule fluorescence imaging of transforming growth factor typeII receptors on a cell surface. This method is also expected to be applicable to the analysis of other stepwise events.

Single-molecule dynamics of site-specific labeled transforming growth factor type II receptors on living cells.

We achieved single-molecule imaging and tracking of the transforming growth factor type II receptor (TßRII) that was labeled by an organic dye via a genetically encoded unnatural amino acid (UAA) and the copper-free click chemistry. The stoichiometry, mobility and dimerization kinetics of individual TßRII molecules were determined.