Deep learning analysis of endometrial histology as a promising tool to predict the chance of pregnancy after frozen embryo transfers.

PURPOSE: Endometrial histology on hematoxylin and eosin (H&E)-stained preparations provides information associated with receptivity. However, traditional histological examination by Noyes' dating method is of limited value as it is prone to subjectivity and is not well correlated with fertility status or pregnancy outcome. This study aims to mitigate the weaknesses of Noyes' dating by analyzing endometrial histology through deep learning (DL) algorithm to predict the chance of pregnancy. METHODS: Endometrial biopsies were taken during the window of receptivity from healthy volunteers in natural menstrual cycles (group A) and infertile patients undergoing mock artificial cycles (group B). H&E staining was performed followed by whole slide image scanning for DL analysis. RESULTS: In a proof-of-concept trial to differentiate group A (n=24) vs. B (n=37), a DL-based binary classifier was trained, cross-validated, and achieved 100% for accuracy. Patients in group B underwent subsequent frozen-thawed embryo transfers (FETs) and were further categorized into "pregnant (n=15)" or "non-pregnant (n=18)" sub-groups based on the outcomes. In the following trial to predict pregnancy outcome in group B, the DL-based binary classifier yielded 77.8% for accuracy. Its performance was further validated by an accuracy of 75% in a "held-out" test set where patients had euploid embryo transfers. Furthermore, the DL model identified histo-characteristics including stromal edema, glandular secretion, and endometrial vascularity as important features related to pregnancy prediction. CONCLUSIONS: DL-based endometrial histology analysis demonstrated its feasibility and robustness in pregnancy prediction for patients undergoing FETs, indicating its value as a prognostic tool in fertility treatment.

Multiplexed In Situ Protein Profiling with High-Performance Cleavable Fluorescent Tyramide.

Understanding the composition, function and regulation of complex cellular systems requires tools that quantify the expression of multiple proteins at their native cellular context. Here, we report a highly sensitive and accurate protein in situ profiling approach using off-the-shelf antibodies and cleavable fluorescent tyramide (CFT). In each cycle of this method, protein targets are stained with horseradish peroxidase (HRP) conjugated antibodies and CFT. Subsequently, the fluorophores are efficiently cleaved by mild chemical reagents, which simultaneously deactivate HRP. Through reiterative cycles of protein staining, fluorescence imaging, fluorophore cleavage, and HRP deactivation, multiplexed protein quantification in single cells in situ can be achieved. We designed and synthesized the high-performance CFT, and demonstrated that over 95% of the staining signals can be erased by mild chemical reagents while preserving the integrity of the epitopes on protein targets. Applying this method, we explored the protein expression heterogeneity and correlation in a group of genetically identical cells. With the high signal removal efficiency, this approach also enables us to accurately profile proteins in formalin-fixed paraffin-embedded (FFPE) tissues in the order of low to high and also high to low expression levels.

Highly Multiplexed Single-Cell In Situ RNA and DNA Analysis by Consecutive Hybridization.

The ability to comprehensively profile nucleic acids in individual cells in their natural spatial contexts is essential to advance our understanding of biology and medicine. Here, we report a novel method for spatial transcriptomics and genomics analysis. In this method, every nucleic acid molecule is detected as a fluorescent spot at its natural cellular location throughout the cycles of consecutive fluorescence in situ hybridization (C-FISH). In each C-FISH cycle, fluorescent oligonucleotide probes hybridize to the probes applied in the previous cycle, and also introduce the binding sites for the next cycle probes. With reiterative cycles of hybridization, imaging and photobleaching, the identities of the varied nucleic acids are determined by their unique color sequences. To demonstrate the feasibility of this method, we show that transcripts or genomic loci in single cells can be unambiguously quantified with 2 fluorophores and 16 C-FISH cycles or with 3 fluorophores and 9 C-FISH cycles. Without any error correction, the error rates obtained using the raw data are close to zero. These results indicate that C-FISH potentially enables tens of thousands (216 = 65,536 or 39 = 19,683) of different transcripts or genomic loci to be precisely profiled in individual cells in situ.

Highly Multiplexed Single-Cell Protein Analysis.

Single-cell proteomic analysis is crucial to advance our understanding of normal physiology and disease pathogenesis. The comprehensive protein profiling in individual cells of a heterogeneous sample can provide new insights into many important biological issues, such as the regulation of inter- and intracellular signaling pathways or the varied cellular compositions of normal and diseased tissues. With highly multiplexed molecular imaging of many different protein biomarkers in patient biopsies, diseases can be accurately diagnosed to guide the selection of the ideal treatment. In this Minireview, we will describe the recent technological advances of single-cell proteomic assays, discuss their advantages and limitations, highlight their applications in biology and precision medicine, and present the current challenges and potential solutions.

Highly Multiplexed Single-Cell In†Situ Protein Analysis with Cleavable Fluorescent Antibodies.

Limitations on the number of proteins that can be quantified in single cells in situ impede advances in our deep understanding of normal cell physiology and disease pathogenesis. Herein, we present a highly multiplexed single-cell in situ protein analysis approach that is based on chemically cleavable fluorescent antibodies. In this method, antibodies tethered to fluorophores through a novel azide-based cleavable linker are utilized to detect their protein targets. After fluorescence imaging and data storage, the fluorophores coupled to the antibodies are efficiently cleaved without loss of protein target antigenicity. Upon continuous cycles of target recognition, fluorescence imaging, and fluorophore cleavage, this approach has the potential to quantify over 100 different proteins in individual cells at optical resolution. This single-cell in situ protein profiling technology will have wide applications in signaling network analysis, molecular diagnosis, and cellular targeted therapies.

GeoNet++: Iterative Geometric Neural Network with Edge-Aware Refinement for Joint Depth and Surface Normal Estimation.

In this paper, we propose a geometric neural network with edge-aware refinement (GeoNet++) to jointly predict both depth and surface normal maps from a single image. Building on top of two-stream CNNs, GeoNet++ captures the geometric relationships between depth and surface normals with the proposed depth-to-normal and normal-to-depth modules. In particular, the "depth-to-normal" module exploits the least square solution of estimating surface normals from depth to improve their quality, while the "normal-to-depth" module refines the depth map based on the constraints on surface normals through kernel regression. Boundary information is exploited via an edge-aware refinement module. GeoNet++ effectively predicts depth and surface normals with high 3D consistency and sharp boundaries resulting in better reconstructed 3D scenes. Note that GeoNet++ is generic and can be used in other depth/normal prediction frameworks to improve 3D reconstruction quality and pixel-wise accuracy of depth and surface normals. Furthermore, we propose a new 3D geometric metric (3DGM) for evaluating depth prediction in 3D. In contrast to current metrics that focus on evaluating pixel-wise error/accuracy, 3DGM measures whether the predicted depth can reconstruct high quality 3D surface normals. This is a more natural metric for many 3D application domains. Our experiments on NYUD-V2 [1] and KITTI [2] datasets verify that GeoNet++ produces fine boundary details and the predicted depth can be used to reconstruct high quality 3D surfaces.

Structure-Coherent Deep Feature Learning for Robust Face Alignment.

In this paper, we propose a structure-coherent deep feature learning method for face alignment. Unlike most existing face alignment methods which overlook the facial structure cues, we explicitly exploit the relation among facial landmarks to make the detector robust to hard cases such as occlusion and large pose. Specifically, we leverage a landmark-graph relational network to enforce the structural relationships among landmarks. We consider the facial landmarks as structural graph nodes and carefully design the neighborhood to passing features among the most related nodes. Our method dynamically adapts the weights of node neighborhood to eliminate distracted information from noisy nodes, such as occluded landmark point. Moreover, different from most previous works which only tend to penalize the landmarks absolute position during the training, we propose a relative location loss to enhance the information of relative location of landmarks. This relative location supervision further regularizes the facial structure. Our approach considers the interactions among facial landmarks and can be easily implemented on top of any convolutional backbone to boost the performance. Extensive experiments on three popular benchmarks, including WFLW, COFW and 300W, demonstrate the effectiveness of the proposed method. In particular, due to explicit structure modeling, our approach is especially robust to challenging cases resulting in impressive low failure rate on COFW and WFLW datasets. The model and code are publicly available at https://github.com/BeierZhu/Sturcture-Coherency-Face-Alignment.

Highly Sensitive and Multiplexed In-Situ Protein Profiling with Cleavable Fluorescent Streptavidin.

The ability to perform highly sensitive and multiplexed in-situ protein analysis is crucial to advance our understanding of normal physiology and disease pathogenesis. To achieve this goal, we here develop an approach using cleavable biotin-conjugated antibodies and cleavable fluorescent streptavidin (CFS). In this approach, protein targets are first recognized by the cleavable biotin-labeled antibodies. Subsequently, CFS is applied to stain the protein targets. Though layer-by-layer signal amplification using cleavable biotin-conjugated orthogonal antibodies and CSF, the protein detection sensitivity can be enhanced at least 10-fold, compared with the current in-situ proteomics methods. After imaging, the fluorophore and the biotin unbound to streptavidin are removed by chemical cleavage. The leftover streptavidin is blocked by biotin. Upon reiterative analysis cycles, a large number of different proteins with a wide range of expression levels can be profiled in individual cells at the optical resolution. Applying this approach, we have demonstrated that multiple proteins are unambiguously detected in the same set of cells, regardless of the protein analysis order. We have also shown that this method can be successfully applied to quantify proteins in formalin-fixed paraffin-embedded (FFPE) tissues.

Highly Sensitive and Multiplexed Protein Imaging With Cleavable Fluorescent Tyramide Reveals Human Neuronal Heterogeneity.

The ability to comprehensively profile proteins in intact tissues in situ is crucial for our understanding of health and disease. However, the existing methods suffer from low sensitivity and limited sample throughput. To address these issues, here we present a highly sensitive and multiplexed in situ protein analysis approach using cleavable fluorescent tyramide and off-the-shelf antibodies. Compared with the current methods, this approach enhances the detection sensitivity and reduces the imaging time by 1-2 orders of magnitude, and can potentially detect hundreds of proteins in intact tissues at the optical resolution. Applying this approach, we studied protein expression heterogeneity in a population of genetically identical cells, and performed protein expression correlation analysis to identify co-regulated proteins. We also profiled >6,000 neurons in a human formalin-fixed paraffin-embedded (FFPE) hippocampus tissue. By partitioning these neurons into varied cell clusters based on their multiplexed protein expression profiles, we observed different sub-regions of the hippocampus consist of neurons from distinct clusters.

Properties of Rhodotorula gracilis D-amino acid oxidase immobilized on magnetic beads through his-tag.

D-amino acid oxidase catalyzes one of the key steps in the production of semisynthetic cephalosporins. We expressed and purified recombinant Rhodotorula gracilis D-amino acid oxidase with C-terminal his-tags. This engineered enzyme was immobilized onto Ni(2+)-chelated nitrilotriacetic acid magnetic beads through the interaction between his-tag and Ni(2+). The kinetic constants, storage properties, and the reusability of the immobilized d-amino acid oxidase were determined. The effects of temperature, pH, and hydrogen peroxide on the activity of immobilized d-amino acid oxidase were also studied. The highest activity recovery was 75%. Thermal stability was improved after immobilization; the relative activity of the immobilized enzyme was 56% whereas the free enzyme was completely inactivated after incubation at 50 degrees C for 1 h. In the presence of 10 mM hydrogen peroxide, the immobilized enzyme did not show a rapid loss of activity during the first 2 h of incubation, which was observed in the case of the free enzyme; the residual activity of the immobilized enzyme after 9 h was 72% compared with 22% of the free form. The long-term storage stability was improved; the residual activity of the immobilized enzyme was 74% compared with 20% of the free enzyme when stored at room temperature for 10 d. The immobilized form retained 37% of its initial activity after 20 consecutive reaction cycles.

Highly multiplexed single-cell in situ RNA and DNA analysis with bioorthogonal cleavable fluorescent oligonucleotides.

The ability to profile transcripts and genomic loci comprehensively in single cells in situ is essential to advance our understanding of normal physiology and disease pathogenesis. Here we report a highly multiplexed single-cell in situ RNA and DNA analysis approach using bioorthogonal cleavable fluorescent oligonucleotides. In this approach, oligonucleotides tethered to fluorophores through an azide-based cleavable linker are used to detect their nucleic acids targets by in situ hybridization. After fluorescence imaging, the fluorophores in the whole specimen are efficiently cleaved in 30 minutes without loss of RNA or DNA integrity. Through reiterative cycles of hybridization, imaging, and cleavage, this method has the potential to quantify hundreds to thousands of different RNA species or genomic loci in single cells in situ at the single-molecule sensitivity. Applying this approach, we demonstrate that different nucleic acids can be detected in each hybridization cycle by multi-color staining, and at least ten continuous hybridization cycles can be carried out in the same specimen. We also show that the integrated single-cell in situ analysis of DNA, RNA and protein can be achieved using cleavable fluorescent oligonucleotides combined with cleavable fluorescent antibodies. This highly multiplexed imaging platform will have wide applications in systems biology and biomedical research.

Personal object discovery in first-person videos.

People know and care for personal objects, which can be different for individuals. Automatically discovering personal objects is thus of great practical importance. We, in this paper, pursue this task with wearable cameras based on the common sense that personal objects generally company us in various scenes. With this clue, we exploit a new object-scene distribution for robust detection. Two technical challenges involved in estimating this distribution, i.e., scene extraction and unsupervised object discovery, are tackled. For scene extraction, we learn the latent representation instead of simply selecting a few frames from the videos. In object discovery, we build an interaction model to select frame-level objects and use nonparametric Bayesian clustering. Experiments verify the usefulness of our approach.