Precise immunofluorescence canceling for highly multiplexed imaging to capture specific cell states.

Cell states are regulated by the response of signaling pathways to receptor ligand-binding and intercellular interactions. High-resolution imaging has been attempted to explore the dynamics of these processes and, recently, multiplexed imaging has profiled cell states by achieving a comprehensive acquisition of spatial protein information from cells. However, the specificity of antibodies is still compromised when visualizing activated signals. Here, we develop Precise Emission Canceling Antibodies (PECAbs) that have cleavable fluorescent labeling. PECAbs enable high-specificity sequential imaging using hundreds of antibodies, allowing for reconstruction of the spatiotemporal dynamics of signaling pathways. Additionally, combining this approach with seq-smFISH can effectively classify cells and identify their signal activation states in human tissue. Overall, the PECAb system can serve as a comprehensive platform for analyzing complex cell processes.

Development of an automatic surgical planning system for high tibial osteotomy using artificial intelligence.

BACKGROUND: This study proposed an automatic surgical planning system for high tibial osteotomy (HTO) using deep learning-based artificial intelligence and validated its accuracy. The system simulates osteotomy and measures lower-limb alignment parameters in pre- and post-osteotomy simulations. METHODS: A total of 107 whole-leg standing radiographs were obtained from 107 patients who underwent HTO. First, the system detected anatomical landmarks on radiographs. Then, it simulated osteotomy and automatically measured five parameters in pre- and post-osteotomy simulation (hip knee angle [HKA], weight-bearing line ratio [WBL ratio], mechanical lateral distal femoral angle [mLDFA], mechanical medial proximal tibial angle [mMPTA], and mechanical lateral distal tibial angle [mLDTA]). The accuracy of the measured parameters was validated by comparing them with the ground truth (GT) values given by two orthopaedic surgeons. RESULTS: All absolute errors of the system were within 1.5° or 1.5%. All inter-rater correlation confidence (ICC) values between the system and GT showed good reliability (>0.80). Excellent reliability was observed in the HKA (0.99) and WBL ratios (>0.99) for the pre-osteotomy simulation. The intra-rater difference of the system exhibited excellent reliability with an ICC value of 1.00 for all lower-limb alignment parameters in pre- and post-osteotomy simulations. In addition, the measurement time per radiograph (0.24 s) was considerably shorter than that of an orthopaedic surgeon (118 s). CONCLUSION: The proposed system is practically applicable because it can measure lower-limb alignment parameters accurately and quickly in pre- and post-osteotomy simulations. The system has potential applications in surgical planning systems.

Artificial intelligence quantifying endoscopic severity of ulcerative colitis in gradation scale.

OBJECTIVES: Existing endoscopic scores for ulcerative colitis (UC) objectively categorize disease severity based on the presence or absence of endoscopic findings; therefore, it may not reflect the range of clinical severity within each category. However, inflammatory bowel disease (IBD) expert endoscopists categorize the severity and diagnose the overall impression of the degree of inflammation. This study aimed to develop an artificial intelligence (AI) system that can accurately represent the assessment of the endoscopic severity of UC by IBD expert endoscopists. METHODS: A ranking-convolutional neural network (ranking-CNN) was trained using comparative information on the UC severity of 13,826 pairs of endoscopic images created by IBD expert endoscopists. Using the trained ranking-CNN, the UC Endoscopic Gradation Scale (UCEGS) was used to express severity. Correlation coefficients were calculated to ensure that there were no inconsistencies in assessments of severity made using UCEGS diagnosed by the AI and the Mayo Endoscopic Subscore, and the correlation coefficients of the mean for test images assessed using UCEGS by four IBD expert endoscopists and the AI. RESULTS: Spearman's correlation coefficient between the UCEGS diagnosed by AI and Mayo Endoscopic Subscore was approximately 0.89. The correlation coefficients between IBD expert endoscopists and the AI of the evaluation results were all higher than 0.95 (P < 0.01). CONCLUSIONS: The AI developed here can diagnose UC severity endoscopically similar to IBD expert endoscopists.

Transcriptomic Interpretation on Explainable AI-Guided Intuition Uncovers Premonitory Reactions of Disordering Fate in Persimmon Fruit.

Deep neural network (DNN) techniques, as an advanced machine learning framework, have allowed various image diagnoses in plants, which often achieve better prediction performance than human experts in each specific field. Notwithstanding, in plant biology, the application of DNNs is still mostly limited to rapid and effective phenotyping. The recent development of explainable CNN frameworks has allowed visualization of the features in the prediction by a convolutional neural network (CNN), which potentially contributes to the understanding of physiological mechanisms in objective phenotypes. In this study, we propose an integration of explainable CNN and transcriptomic approach to make a physiological interpretation of a fruit internal disorder in persimmon, rapid over-softening. We constructed CNN models to accurately predict the fate to be rapid softening in persimmon cv. Soshu, only with photo images. The explainable CNNs, such as Gradient-weighted Class Activation Mapping (Grad-Class Activation Mapping (CAM)) and guided Grad-CAM, visualized specific featured regions relevant to the prediction of rapid softening, which would correspond to the premonitory symptoms in a fruit. Transcriptomic analyses to compare the featured regions of the predicted rapid-softening and control fruits suggested that rapid softening is triggered by precocious ethylene signal-dependent cell wall modification, despite exhibiting no direct phenotypic changes. Further transcriptomic comparison between the featured and non-featured regions in the predicted rapid-softening fruit suggested that premonitory symptoms reflected hypoxia and the related stress signals finally to induce ethylene signals. These results would provide a good example for the collaboration of image analysis and omics approaches in plant physiology, which uncovered a novel aspect of fruit premonitory reactions in the rapid-softening fate.

Development of a chub mackerel with less-aggressive fry stage by genome editing of arginine vasotocin receptor V1a2.

Genome editing is a technology that can remarkably accelerate crop and animal breeding via artificial induction of desired traits with high accuracy. This study aimed to develop a chub mackerel variety with reduced aggression using an experimental system that enables efficient egg collection and genome editing. Sexual maturation and control of spawning season and time were technologically facilitated by controlling the photoperiod and water temperature of the rearing tank. In addition, appropriate low-temperature treatment conditions for delaying cleavage, shape of the glass capillary, and injection site were examined in detail in order to develop an efficient and robust microinjection system for the study. An arginine vasotocin receptor V1a2 (V1a2) knockout (KO) strain of chub mackerel was developed in order to reduce the frequency of cannibalistic behavior at the fry stage. Video data analysis using bioimage informatics quantified the frequency of aggressive behavior, indicating a significant 46% reduction (P = 0.0229) in the frequency of cannibalistic behavior than in wild type. Furthermore, in the V1a2 KO strain, the frequency of collisions with the wall and oxygen consumption also decreased. Overall, the manageable and calm phenotype reported here can potentially contribute to the development of a stable and sustainable marine product.

Development of a simultaneous electrorotation device with microwells for monitoring the rotation rates of multiple single cells upon chemical stimulation.

Here, we described a unique simultaneous electrorotation (ROT) device for monitoring the rotation rate of Jurkat cells via chemical stimulation without fluorescent labeling and an algorithm for estimating cell rotation rates. The device comprised two pairs of interdigitated array electrodes that were stacked orthogonally through a 20 µm-thick insulating layer with rectangular microwells. Four microelectrodes (two were patterned on the bottom of the microwells and the other two on the insulating layer) were arranged on each side of the rectangular microwells. The cells, which were trapped in the microwells, underwent ROT when AC voltages were applied to the four microelectrodes to generate a rotating electric field. These microwells maintained the cells even in fluid flows. Thereafter, the ROT rates of the trapped cells were estimated and monitored during the stimulation. We demonstrated the feasibility of estimating the chemical efficiency of cells by monitoring the ROT rates of the cells. After introducing a Jurkat cell suspension into the device, the cells were subjected to ROT by applying an AC signal. Further, the rotating cells were chemically stimulated by adding an ionomycin (a calcium ionophore)-containing aliquot. The ROT rate of the ionomycin-stimulated cells decreased gradually to 90% of the initial rate after 30 s. The ROT rate was reduced by an increase in membrane capacitance. Thus, our device enabled the simultaneous chemical stimulation-induced monitoring of the alterations in the membrane capacitances of many cells without fluorescent labeling.

Deep learning-based automatic-bone-destruction-evaluation system using contextual information from other joints.

BACKGROUND: X-ray images are commonly used to assess the bone destruction of rheumatoid arthritis. The purpose of this study is to propose an automatic-bone-destruction-evaluation system fully utilizing deep neural networks (DNN). This system detects all target joints of the modified Sharp/van der Heijde score (SHS) from a hand X-ray image. It then classifies every target joint as intact (SHS = 0) or non-intact (SHS ≥ 1). METHODS: We used 226 hand X-ray images of 40 rheumatoid arthritis patients. As for detection, we used a DNN model called DeepLabCut. As for classification, we built four classification models that classify the detected joint as intact or non-intact. The first model classifies each joint independently, whereas the second model does it while comparing the same contralateral joint. The third model compares the same joint group (e.g., the proximal interphalangeal joints) of one hand and the fourth model compares the same joint group of both hands. We evaluated DeepLabCut's detection performance and classification models' performances. The classification models' performances were compared to three orthopedic surgeons. RESULTS: Detection rates for all the target joints were 98.0% and 97.3% for erosion and joint space narrowing (JSN). Among the four classification models, the model that compares the same contralateral joint showed the best F-measure (0.70, 0.81) and area under the curve of the precision-recall curve (PR-AUC) (0.73, 0.85) regarding erosion and JSN. As for erosion, the F-measure and PR-AUC of this model were better than the best of the orthopedic surgeons. CONCLUSIONS: The proposed system was useful. All the target joints were detected with high accuracy. The classification model that compared the same contralateral joint showed better performance than the orthopedic surgeons regarding erosion.

Genome-wide cis-decoding for expression design in tomato using cistrome data and explainable deep learning.

In the evolutionary history of plants, variation in cis-regulatory elements (CREs) resulting in diversification of gene expression has played a central role in driving the evolution of lineage-specific traits. However, it is difficult to predict expression behaviors from CRE patterns to properly harness them, mainly because the biological processes are complex. In this study, we used cistrome datasets and explainable convolutional neural network (CNN) frameworks to predict genome-wide expression patterns in tomato (Solanum lycopersicum) fruit from the DNA sequences in gene regulatory regions. By fixing the effects of trans-acting factors using single cell-type spatiotemporal transcriptome data for the response variables, we developed a prediction model for crucial expression patterns in the initiation of tomato fruit ripening. Feature visualization of the CNNs identified nucleotide residues critical to the objective expression pattern in each gene, and their effects were validated experimentally in ripening tomato fruit. This cis-decoding framework will not only contribute to the understanding of the regulatory networks derived from CREs and transcription factor interactions, but also provides a flexible means of designing alleles for optimized expression.

Tricellulin secures the epithelial barrier at tricellular junctions by interacting with actomyosin.

The epithelial cell sheet functions as a barrier to prevent invasion of pathogens. It is necessary to eliminate intercellular gaps not only at bicellular junctions, but also at tricellular contacts, where three cells meet, to maintain epithelial barrier function. To that end, tight junctions between adjacent cells must associate as closely as possible, particularly at tricellular contacts. Tricellulin is an integral component of tricellular tight junctions (tTJs), but the molecular mechanism of its contribution to the epithelial barrier function remains unclear. In this study, we revealed that tricellulin contributes to barrier formation by regulating actomyosin organization at tricellular junctions. Furthermore, we identified α-catenin, which is thought to function only at adherens junctions, as a novel binding partner of tricellulin. α-catenin bridges tricellulin attachment to the bicellular actin cables that are anchored end-on at tricellular junctions. Thus, tricellulin mobilizes actomyosin contractility to close the lateral gap between the TJ strands of the three proximate cells that converge on tricellular junctions.

Discovery of anti-inflammatory physiological peptides that promote tissue repair by reinforcing epithelial barrier formation.

Epithelial barriers that prevent dehydration and pathogen invasion are established by tight junctions (TJs), and their disruption leads to various inflammatory diseases and tissue destruction. However, a therapeutic strategy to overcome TJ disruption in diseases has not been established because of the lack of clinically applicable TJ-inducing molecules. Here, we found TJ-inducing peptides (JIPs) in mice and humans that corresponded to 35 to 42 residue peptides of the C terminus of alpha 1-antitrypsin (A1AT), an acute-phase anti-inflammatory protein. JIPs were inserted into the plasma membrane of epithelial cells, which promoted TJ formation by directly activating the heterotrimeric G protein G13. In a mouse intestinal epithelial injury model established by dextran sodium sulfate, mouse or human JIP administration restored TJ integrity and strongly prevented colitis. Our study has revealed TJ-inducing anti-inflammatory physiological peptides that play a critical role in tissue repair and proposes a previously unidentified therapeutic strategy for TJ-disrupted diseases.

An empirical survey of data augmentation for time series classification with neural networks.

In recent times, deep artificial neural networks have achieved many successes in pattern recognition. Part of this success can be attributed to the reliance on big data to increase generalization. However, in the field of time series recognition, many datasets are often very small. One method of addressing this problem is through the use of data augmentation. In this paper, we survey data augmentation techniques for time series and their application to time series classification with neural networks. We propose a taxonomy and outline the four families in time series data augmentation, including transformation-based methods, pattern mixing, generative models, and decomposition methods. Furthermore, we empirically evaluate 12 time series data augmentation methods on 128 time series classification datasets with six different types of neural networks. Through the results, we are able to analyze the characteristics, advantages and disadvantages, and recommendations of each data augmentation method. This survey aims to help in the selection of time series data augmentation for neural network applications.

Soft and self constrained clustering for group-based labeling.

When using deep neural networks in medical image classification tasks, it is mandatory to prepare a large-scale labeled image set, and this often requires significant effort by medical experts. One strategy to reduce the labeling cost is group-based labeling, where image samples are clustered and then a label is attached to each cluster. The efficiency of this strategy depends on the purity of the clusters. Constrained clustering is an effective way to improve the purity of the clusters if we can give appropriate must-links and cannot-links as constraints. However, for medical image clustering, the conventional constrained clustering methods encounter two issues. The first issue is that constraints are not always appropriate due to the gap between semantic and visual similarities. The second issue is that attaching constraints requires extra effort from medical experts. To deal with the first issue, we propose a novel soft-constrained clustering method, which has the ability to ignore inappropriate constraints. To deal with the second issue, we propose a self-constrained clustering method that utilizes prior knowledge about the target images to set the constraints automatically. Experiments with the endoscopic image datasets demonstrated that the proposed methods give clustering results with higher purity.

STIM-Orai1 signaling regulates fluidity of cytoplasm during membrane blebbing.

The cytoplasm in mammalian cells is considered homogeneous. In this study, we report that the cytoplasmic fluidity is regulated in the blebbing cells; the cytoplasm of rapidly expanding membrane blebs is more disordered than the cytoplasm of retracting blebs. The increase of cytoplasmic fluidity in the expanding bleb is caused by a sharp rise in the calcium concentration. The STIM-Orai1 pathway regulates this rapid and restricted increase of calcium in the expanding blebs. Conversely, activated ERM protein binds to Orai1 to inhibit the store-operated calcium entry in retracting blebs, which results in decreased in cytoplasmic calcium, rapid reassembly of the actin cortex.

Explainable Deep Learning Reproduces a 'Professional Eye' on the Diagnosis of Internal Disorders in Persimmon Fruit.

Recent rapid progress in deep neural network techniques has allowed recognition and classification of various objects, often exceeding the performance of the human eye. In plant biology and crop sciences, some deep neural network frameworks have been applied mainly for effective and rapid phenotyping. In this study, beyond simple optimizations of phenotyping, we propose an application of deep neural networks to make an image-based internal disorder diagnosis that is hard even for experts, and to visualize the reasons behind each diagnosis to provide biological interpretations. Here, we exemplified classification of calyx-end cracking in persimmon fruit by using five convolutional neural network models with various layer structures and examined potential analytical options involved in the diagnostic qualities. With 3,173 visible RGB images from the fruit apex side, the neural networks successfully made the binary classification of each degree of disorder, with up to 90% accuracy. Furthermore, feature visualizations, such as Grad-CAM and LRP, visualize the regions of the image that contribute to the diagnosis. They suggest that specific patterns of color unevenness, such as in the fruit peripheral area, can be indexes of calyx-end cracking. These results not only provided novel insights into indexes of fruit internal disorders but also proposed the potential applicability of deep neural networks in plant biology.

Guided neural style transfer for shape stylization.

Designing logos, typefaces, and other decorated shapes can require professional skills. In this paper, we aim to produce new and unique decorated shapes by stylizing ordinary shapes with machine learning. Specifically, we combined parametric and non-parametric neural style transfer algorithms to transfer both local and global features. Furthermore, we introduced a distance-based guiding to the neural style transfer process, so that only the foreground shape will be decorated. Lastly, qualitative evaluation and ablation studies are provided to demonstrate the usefulness of the proposed method.

Coordinated changes in cell membrane and cytoplasm during maturation of apoptotic bleb.

Apoptotic cells form membrane blebs, but little is known about how the formation and dynamics of membrane blebs are regulated. The size of blebs gradually increases during the progression of apoptosis, eventually forming large extracellular vesicles called apoptotic bodies that have immune-modulating activities. In this study, we investigated the molecular mechanism involved in the differentiation of blebs into apoptotic blebs by comparing the dynamics of the bleb formed during cell migration and the bleb formed during apoptosis. We revealed that the enhanced activity of ROCK1 is required for the formation of small blebs in the early phase of apoptosis, which leads to the physical disruption of nuclear membrane and the degradation of Lamin A. In the late phase of apoptosis, the loss of asymmetry in phospholipids distribution caused the enlargement of blebs, which enabled translocation of damage-associated molecular patterns to the bleb cytoplasm and maturation of functional apoptotic blebs. Thus, changes in cell membrane dynamics are closely linked to cytoplasmic changes during apoptotic bleb formation.

Automatic Generation of Typographic Font From Small Font Subset.

The automated generation of fonts containing a large number of characters is in high demand. For example, a typical Japanese font requires over 1000 characters. Unfortunately, professional typographers create the majority of fonts, resulting in significant financial and time investments for font generation. The main contribution of this article is the development of a method that automatically generates a target typographic font containing thousands of characters, from a small subset of character images in the target font. We generate characters other than the subset so that a complete font is obtained. We propose a novel font generation method with the capability to deal with various fonts, including a font composed of distinctive strokes, which are difficult for existing methods to handle. We demonstrated the proposed method by generating 2965 characters in 47 fonts. Moreover, objective and subjective evaluations verified that the generated characters are similar to the original characters.

Endoscopic Image Clustering with Temporal Ordering Information Based on Dynamic Programming.

In this paper, we propose a clustering method with temporal ordering information for endoscopic image sequences. It is difficult to collect a sufficient amount of endoscopic image datasets to train machine learning techniques by manual labeling. The clustering of endoscopic images leads to group-based labeling, which is useful for reducing the cost of dataset construction. Therefore, in this paper, we propose a clustering method where the property of endoscopic image sequences is fully utilized. For the proposed method, a deep neural network was used to extract features from endoscopic images, and clustering with temporal ordering information was solved by dynamic programming. In the experiments, we clustered the esophagogastroduodenoscopy images. From the results, we confirmed that the performance was improved by using the sequential property.

Scribbles for Metric Learning in Histological Image Segmentation.

Segmentation is a fundamental process in biomedical image analysis that enables various types of analysis. Segmenting organs in histological microscopy images is problematic because the boundaries between regions are ambiguous, the images have various appearances, and the amount of training data is limited. To address these difficulties, supervised learning methods (e.g., convolutional neural networking (CNN)) are insufficient to predict regions accurately because they usually require a large amount of training data to learn the various appearances. In this paper, we propose a semi-automatic segmentation method that effectively uses scribble annotations for metric learning. Deep discriminative metric learning re-trains the representation of the feature space so that the distances between the samples with the same class labels are reduced, while those between ones with different class labels are enlarged. It makes pixel classification easy. Evaluation of the proposed method in a heart region segmentation task demonstrated that it performed better than three other methods.