Deep learning for quantifying spatial patterning and formation process of early differentiated human-induced pluripotent stem cells with micropattern images.

Micropatterning is reliable method for quantifying pluripotency of human-induced pluripotent stem cells (hiPSCs) that differentiate to form a spatial pattern of sorted, ordered and nonoverlapped three germ layers on the micropattern. In this study, we propose a deep learning method to quantify spatial patterning of the germ layers in the early differentiation stage of hiPSCs using micropattern images. We propose decoding and encoding U-net structures learning labelled Hoechst (DNA-stained) hiPSC regions with corresponding Hoechst and bright-field micropattern images to segment hiPSCs on Hoechst or bright-field images. We also propose a U-net structure to extract extraembryonic regions on a micropattern, and an algorithm to compares intensities of the fluorescence images staining respective germ-layer cells and extract their regions. The proposed method thus can quantify the pluripotency of a hiPSC line with spatial patterning including cell numbers, areas and distributions of germ-layer and extraembryonic cells on a micropattern, and reveal the formation process of hiPSCs and germ layers in the early differentiation stage by segmenting live-cell bright-field images. In our assay, the cell-number accuracy achieved 86% and 85%, and the cell region accuracy 89% and 81% for segmenting Hoechst and bright-field micropattern images, respectively. Applications to micropattern images of multiple hiPSC lines, micropattern sizes, groups of markers, living and fixed cells show the proposed method can be expected to be a useful protocol and tool to quantify pluripotency of a new hiPSC line before providing it to the scientific community.

Development of a visible to 1600 nm hyperspectral imaging rigid-scope system using supercontinuum light and an acousto-optic tunable filter.

In this study, we developed a rigid-scope system that can perform hyperspectral imaging (HSI) between visible and 1600 nm wavelengths using a supercontinuum light source and an acousto-optic tunable filter to emit specific wavelengths. The system optical performance was verified, and the classification ability was investigated. Consequently, it was demonstrated that HSI (490-1600 nm) could be performed. In addition, seven different targets could be classified by the neural network with an accuracy of 99.6%, recall of 93.7%, and specificity of 99.1% when the wavelength range of over 1000 nm (OTN) was extracted from HSI data as train data.

Assessing Molecular Localization of Symbiont Microalgae in Coral Branches Through Mass Spectrometry Imaging.

Reef-building corals are a fundamental pillar of coral reef ecosystems in tropical and subtropical shallow environments. Corals harbor symbiotic dinoflagellates belonging to the family Symbiodiniaceae, commonly known as zooxanthellae. Extensive research has been conducted on this symbiotic relationship, yet the fundamental information about the distribution and localization of Symbiodiniaceae cells in corals is still limited. This information is crucial to understanding the mechanism underlying the metabolite exchange between corals and their algal symbionts, as well as the metabolic flow within holobionts. To examine the distribution of Symbiodiniaceae cells within corals, in this study, we used fluorescence imaging and matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MS-Imaging) on branches of the Acropora tenuis coral. We successfully prepared frozen sections of the coral for molecular imaging without fixing or decalcifying the coral branches. By combining the results of MS-Imaging with that of the fluorescence imaging, we determined that the algal Symbiodiniaceae symbionts were not only localized in the tentacle and surface region of the coral branches but also inhabited the in inner parts. Therefore, the molecular imaging technique used in this study could be valuable to further investigate the molecular dynamics between corals and their symbionts.

Quantitative Analyses for Early Tempo-spatial Patterning of Differentiated Human Induced Pluripotent Stem Cells on Micropatterns using Time-lapse Bright-field Microscopy Images.

Three germ layer formation on micropatterns are extremely useful for quantitative analysis of hiPSC (human induced pluripotent stem cells) pluripotency. Spatial patterns of stem cells differentiated on the micropatterns will be formed from about 24 hours after differentiation induction and usually quantitated near 48 hours. To delineate the germ layer formation process, temporal changes in spatial patterning of germ layers should be analyzed by noninvasive microscopy. This study proposed a series of image processing methods combined with a U-net automatic segmentation to segment differentiated hiPSCs captured by bright-field microscopy. High segmentation accuracy (83.3%) for the test bright-field images compared with their concurrent Hoechst images (85%) was achieved. Tempo-spatial patterning and formation process of germ layers on the micropatterns can be visualized and quantified by segmenting time-lapse bright-field microscopy images using our method.

Quantitative Analysis of Differentiation Activity for Mouse Embryonic Stem Cells by Deep Learning for Cell Center Detection using Three-Dimensional Confocal Fluorescence Microscopy Images.

Accurate single cell segmentation provides means to monitor the behavior of single cell within a population of cells. Time-lapse fluorescence images are used to reveal heterogeneous nature of single mouse embryonic stem cell (ESC) colony and monitor fluctuations of the cell states. Automatic quantification of speed and status shifts of the ESCs depends on accurate single cell segmentation that is used to calculate the 3D center of every cell and track this cell for the quantification. This study proposes a new 3D U-net to accurately detect center of each single cell in 3D confocal images. The dimension of input 3D images to the U-net is flexible so that multiple center detections from different image directions can be implemented simultaneously to improve the center detection accuracy. This study showed that our method can improve accuracy for cell center detection and thus the quantification for ESC speeds and status shifts.

In situ FRET measurement of cellular tension using conventional confocal laser microscopy in newly established reporter mice expressing actinin tension sensor.

FRET-based sensors are utilized for real-time measurements of cellular tension. However, transfection of the sensor gene shows low efficacy and is only effective for a short period. Reporter mice expressing such sensors have been developed, but sensor fluorescence has not been measured successfully using conventional confocal microscopy. Therefore, methods for spatiotemporal measurement of cellular tension in vivo or ex vivo are still limited. We established a reporter mouse line expressing FRET-based actinin tension sensors consisting of EGFP as the donor and mCherry as the acceptor and whose FRET ratio change is observable with confocal microscopy. Tension-induced changes in FRET signals were monitored in the aorta and tail tendon fascicles, as well as aortic smooth muscle cells isolated from these mice. The pattern of FRET changes was distinctive, depending on tissue type. Indeed, aortic smooth muscle cells exhibit different sensitivity to macroscopic tensile strain in situ and in an isolated state. This mouse strain will enable novel types of biomechanical investigations of cell functions in important physiological events.

Evaluating the identification of the extent of gastric cancer by over-1000 nm near-infrared hyperspectral imaging using surgical specimens.

Significance: Determining the extent of gastric cancer (GC) is necessary for evaluating the gastrectomy margin for GC. Additionally, determining the extent of the GC that is not exposed to the mucosal surface remains difficult. However, near-infrared (NIR) can penetrate mucosal tissues highly efficiently. Aim: We investigated the ability of near-infrared hyperspectral imaging (NIR-HSI) to identify GC areas, including exposed and unexposed using surgical specimens, and explored the identifiable characteristics of the GC. Approach: Our study examined 10 patients with diagnosed GC who underwent surgery between 2020 and 2021. Specimen images were captured using NIR-HSI. For the specimens, the exposed area was defined as an area wherein the cancer was exposed on the surface, the unexposed area as an area wherein the cancer was present although the surface was covered by normal tissue, and the normal area as an area wherein the cancer was absent. We estimated the GC (including the exposed and unexposed areas) and normal areas using a support vector machine, which is a machine-learning method for classification. The prediction accuracy of the GC region in every area and normal region was evaluated. Additionally, the tumor thicknesses of the GC were pathologically measured, and their differences in identifiable and unidentifiable areas were compared using NIR-HSI. Results: The average prediction accuracy of the GC regions combined with both areas was 77.2%; with exposed and unexposed areas was 79.7% and 68.5%, respectively; and with normal regions was 79.7%. Additionally, the areas identified as cancerous had a tumor thickness of >2 mm. Conclusions: NIR-HSI identified the GC regions with high rates. As a feature, the exposed and unexposed areas with tumor thicknesses of >2 mm were identified using NIR-HSI.

Computer-aided demarcation of early gastric cancer: a pilot comparative study with endoscopists.

BACKGROUND: Precise area diagnosis of early gastric cancer (EGC) is critical for reliable endoscopic resection. Computer-aided diagnosis (CAD) shows strong potential for detecting EGC and reducing cancer-care disparities caused by differences in endoscopists' skills. To be used in clinical practice, CAD should enable both the detection and the demarcation of lesions. This study proposes a scheme for the detection and delineation of EGC under white-light endoscopy and validates its performance using 1-year consecutive cases. METHODS: Only 300 endoscopic images randomly selected from 68 consecutive cases were used for training a convolutional neural network. All cases were treated with endoscopic submucosal dissection, enabling the accumulation of a training dataset in which the extent of lesions was precisely determined. For validation, 462 cancer images and 396 normal images from 137 consecutive cases were used. From the validation results, 38 randomly selected images were compared with those delineated by six endoscopists. RESULTS: Successful detections of EGC in 387 cancer images (83.8%) and the absence of lesions in 307 normal images (77.5%) were achieved. Positive and negative predictive values were 81.3% and 80.4%, respectively. Successful detection was achieved in 130 cases (94.9%). We achieved precise demarcation of EGC with a mean intersection over union of 66.5%, showing the extent of lesions with a smooth boundary; the results were comparable to those achieved by specialists. CONCLUSIONS: Our scheme, validated using 1-year consecutive cases, shows potential for demarcating EGC. Its performance matched that of specialists; it might therefore be suitable for clinical use in the future.

Human induced pluripotent stem cell formation and morphology prediction during reprogramming with time-lapse bright-field microscopy images using deep learning methods.

BACKGROUND AND OBJECTIVE: Human induced pluripotent stem cells (hiPSCs) represent an ideal source for patient specific cell-based regenerative medicine; however, efficiency of hiPSC formation from reprogramming cells is low. We use several deep-learning results from time-lapse brightfield microscopy images during culture, to early detect the cells potentially reprogramming into hiPSCs and predict the colony morphology of these cells for improving efficiency of culturing a new hiPSC line. METHODS: Sets of time-lapse bright-field images are taken to track reprogramming process of CD34+ cells biologically identified as just beginning reprogramming. Prior the experiment, 9 classes of templates with distinct cell features clipped from microscopy images at various reprogramming stages are used to train a CNN model. The CNN is then used to classify a microscopy image as probability images of these classes. Probability images of some class are used to train a densely connected convolutional network for extracting regions of this class on a microscopy image. A U-net is trained to segment cells on the time-lapse images in early reprogramming stage during culture. The segmented cells are classified by the extracted regions to count various types of cells appearing in the early reprogramming stage for predicting the identified cells potentially forming hiPSCs. The probability images of hiPSC classes are also used to train a spatiotemporal RNN for predicting the future hiPSC colony morphology of the potential cells. RESULTS: Experimental results show the prediction (before 7 days after of beginning of the reprogramming) achieved 0.8 accuracy, and 66% of the identified cells under different culture conditions, predicted as forming, finally formed hiPSCs. The predicted hiPSC images and extracted colonies on the images show the prediction for future 1.5 days achieved high accuracy of hiPSC colony areas and image similarity. CONCLUSIONS: Our study proposes a method using several deep learning models to efficiently select the reprogramming cells possibly forming hiPSCs and to predict the shapes of growing hiPSC colonies. The proposed method is expected to improve the efficiency when establishing a new hiPSC line culture.

A covalently linked probe to monitor local membrane properties surrounding plasma membrane proteins.

Functional membrane proteins in the plasma membrane are suggested to have specific membrane environments that play important roles to maintain and regulate their function. However, the local membrane environments of membrane proteins remain largely unexplored due to the lack of available techniques. We have developed a method to probe the local membrane environment surrounding membrane proteins in the plasma membrane by covalently tethering a solvatochromic, environment-sensitive dye, Nile Red, to a GPI-anchored protein and the insulin receptor through a flexible linker. The fluidity of the membrane environment of the GPI-anchored protein depended upon the saturation of the acyl chains of the lipid anchor. The local environment of the insulin receptor was distinct from the average plasma membrane fluidity and was quite dynamic and heterogeneous. Upon addition of insulin, the local membrane environment surrounding the receptor specifically increased in fluidity in an insulin receptor-kinase dependent manner and on the distance between the dye and the receptor.

Prediction of Human Induced Pluripotent Stem Cell Formation Based on Deep Learning Analyses Using Time-lapse Brightfield Microscopy Images.

We use deep learning methods to predict human induced pluripotent stem cell (hiPSC) formation using time-lapse brightfield microscopy images taken from a cell identified as the beginning of entered into the reprogramming process. A U-net is used to segment cells and a CNN is used to classify the segmented cells into eight types of cells during the reprogramming and hiPSC formation based on cellular morphology on the microscopy images. The numbers of respective types of cells in cell clusters before the hiPSC formation stage are used to predict if hiPSC regions can be well formed lately. Experimental results show good prediction by the criteria using the numbers of different cells in the clusters. Time-series images with respective types of classified cells can be used to visualize and quantitatively analyze the growth and transition among dispersed cells not in cell clusters, various types of cells in the clusters before the hiPSC formation stage and hiPSC cells.

U-net structures for segmentation of single mouse embryonic stem cells using three-dimensional confocal microscopy images.

Cell segmentation at a single cell resolution is required to provide insights for basic biology and application study. However, there are issues of low signal-to-noise ratio, weak fluorescence response, and insufficient resolution along the image stacking direction in 3D confocal images (volume). It has been difficult to segment out single cells from close or contacted cells in a cell volume using image processing methods or together with geometric processing methods. Recently, 3D deep learning methods have been used to avoid tedious parameter settings in the image and geometric processing, but still not easy to segment out close or contacted single cells. This paper proposes a 2D U-net to segment cell regions in high accuracy and computing performance. Better 3D cell images and single cell segmentation for close or contacted cells are achieved by combining a 3D U-net to detect the centers of single cells in the volume.

Modeling the musculoskeletal system of an insect thorax for flapping flight.

Indirect actuation of the wings via thoracic deformation is a unique mechanism widely observed in flying insect species. The physical properties of the thorax have been intensively studied in terms of their ability to efficiently generate wingbeats. The basic mechanism of indirect wing actuation is generally explained as a lever model on a cross-sectional plane, where the dorsoventral movement of the mesonotum (dorsal exoskeleton of the mesothorax) generated by contractions of indirect muscles actuates the wing. However, the model considers the mesonotum as an ideal flat plane, whereas the mesonotum is hemispherical and becomes locally deformed during flight. Furthermore, the conventional model is two-dimensional; therefore, three-dimensional wing kinematics by indirect muscles have not been studied to date. In this study, we develop structural models of the mesonotum and mesothorax of the hawkmothAgrius convolvuli, reconstructed from serial cross-sectional images. External forces are applied to the models to mimic muscle contraction, and mesonotum deformation and wing trajectories are analyzed using finite element analysis. We find that applying longitudinal strain to the mesonotum to mimic strain by depressor muscle contraction reproduces local deformation comparable to that of the thorax during flight. Furthermore, the phase difference of the forces applied to the depressor and elevator muscles changes the wing trajectory from a figure eight to a circle, which is qualitatively consistent with the tethered flight experiment. These results indicate that the local deformation of the mesonotum due to its morphology and the thoracic deformation via indirect power muscles can modulate three-dimensional wing trajectories.

A comprehensible machine learning tool to differentially diagnose idiopathic pulmonary fibrosis from other chronic interstitial lung diseases.

BACKGROUND AND OBJECTIVE: Idiopathic pulmonary fibrosis (IPF) has poor prognosis, and the multidisciplinary diagnostic agreement is low. Moreover, surgical lung biopsies pose comorbidity risks. Therefore, using data from non-invasive tests usually employed to assess interstitial lung diseases (ILDs), we aimed to develop an automated algorithm combining deep learning and machine learning that would be capable of detecting and differentiating IPF from other ILDs. METHODS: We retrospectively analysed consecutive patients presenting with ILD between April 2007 and July 2017. Deep learning was used for semantic image segmentation of HRCT based on the corresponding labelled images. A diagnostic algorithm was then trained using the semantic results and non-invasive findings. Diagnostic accuracy was assessed using five-fold cross-validation. RESULTS: In total, 646,800 HRCT images and the corresponding labelled images were acquired from 1068 patients with ILD, of whom 42.7% had IPF. The average segmentation accuracy was 96.1%. The machine learning algorithm had an average diagnostic accuracy of 83.6%, with high sensitivity, specificity and kappa coefficient values (80.7%, 85.8% and 0.665, respectively). Using Cox hazard analysis, IPF diagnosed using this algorithm was a significant prognostic factor (hazard ratio, 2.593; 95% CI, 2.069-3.250; p < 0.001). Diagnostic accuracy was good even in patients with usual interstitial pneumonia patterns on HRCT and those with surgical lung biopsies. CONCLUSION: Using data from non-invasive examinations, the combined deep learning and machine learning algorithm accurately, easily and quickly diagnosed IPF in a population with various ILDs.

Contributing role of mitochondrial energy metabolism on platelet adhesion, activation and thrombus formation under blood flow conditions.

Platelets have an active energy metabolism mediated by mitochondria. However, the role of mitochondria in platelet adhesion, activation, and thrombus formation under blood flow conditions remains to be elucidated. Blood specimens were obtained from healthy adult volunteers. The consumption of glucose molecules by platelets was measured after 24 hours. Platelet adhesion, activation, and thrombus formation on collagen fibrils and immobilized von Willebrand factor (VWF) at a wall shear rate of 1,500 s-1 were detected by fluorescence microscopy with an ultrafast laser confocal unit in the presence or absence of mitochondrial functional inhibitors of carbonyl cyanide 4-(trifluoromethoxy) phenylhydrazone (FCCP), antimycin A, and oligomycin. Consumption of glucose molecules within the first 24 h of 4.21 × 10-15 ± 4.46 x 10-15 (n = 6) increased to 13.82 × 10-15 ± 3.46 x 10-15 (n = 4) in the presence of FCCP, 12.11 × 10-15 ± 2.33 x 10-15 (n = 4) in the presence of antimycin A, and 11.87 × 10-15 ± 3.56 x 10-15 (n = 4) in the presence of oligomycin (p < .05). These mitochondrial functional blockers did not influence both surface area coverage by platelets and the 3-dimensional size of platelet thrombi formed on the collagen fibrils. However, a rapid increase in the intracellular calcium ion concentration ([Ca2+]i) upon adhering on immobilized VWF decreased significantly from 405.5 ± 86.2 nM in control to 198.0 ± 79.2 nM in the presence of FCCP (p < .005). A similar decrease in the rapid increase in ([Ca2+]i) was observed in the presence of antimycin A and oligomycin. Mitochondrial function is necessary for platelet activation represented by a rapid increase in [Ca2+]i after platelet adhesion on VWF. However, the influence could not be detected as changes in platelet adhesion or 3-dimensional growth of platelet thrombi on collagen fibrils.

Influence of the difference in refractive index on the interface of an object and the surroundings in near-infrared fluorescence tomography.

The refraction of fluorescence from the inside of a sample at the surface results in fluctuations in fluorescence computed tomography (CT). We evaluated the influence of the difference in refractive index (RI) between the sample body and the surroundings on fluorescence CT results. The brightest fluorescent point is away from the correct point on the tomograms owing to the refraction. The speculated position is determined as the exact point if the RI ratio ranges between 0.97 and 1.03 by immersing the body in an RI matching liquid. The results can help in experimental settings of fluorescence CT for acquiring three-dimensional positional information.

Detecting colon polyps in endoscopic images using artificial intelligence constructed with automated collection of annotated images from an endoscopy reporting system.

BACKGROUND: Artificial intelligence (AI) has made considerable progress in image recognition, especially in the analysis of endoscopic images. The availability of large-scale annotated datasets has contributed to the recent progress in this field. Datasets of high-quality annotated endoscopic images are widely available, particularly in Japan. A system for collecting annotated data reported daily could aid in accumulating a significant number of high-quality annotated datasets. AIM: We assessed the validity of using daily annotated endoscopic images in a constructed reporting system for a prototype AI model for polyp detection. METHODS: We constructed an automated collection system for daily annotated datasets from an endoscopy reporting system. The key images were selected and annotated for each case only during daily practice, not to be performed retrospectively. We automatically extracted annotated endoscopic images of diminutive colon polyps that had been diagnosed (study period March-September 2018) using the keywords of diagnostic information, and additionally collect the normal colon images. The collected dataset was devised into training and validation to build and evaluate the AI system. The detection model was developed using a deep learning algorithm, RetinaNet. RESULTS: The automated system collected endoscopic images (47,391) from colonoscopies (745), and extracted key colon polyp images (1356) with localized annotations. The sensitivity, specificity, and accuracy of our AI model were 97.0%, 97.7%, and 97.3% (n = 300), respectively. CONCLUSION: The automated system enabled the development of a high-performance colon polyp detector using images in endoscopy reporting system without the efforts of retrospective annotation works.

Visualization and quantitative analyses for mouse embryonic stem cell tracking by manipulating hierarchical data structures using time-lapse confocal microscopy images.

We present a cell tracking method for time-lapse confocal microscopy (3D) images that uses dynamic hierarchical data structures to assist cell and colony segmentation and tracking. During the segmentation, the cell and colony numbers and their geometric data are recorded for each 3D image set. In tracking, the colony correspondences between neighboring frames of time-lapse 3D images are first computed using the recorded colony centers. Then, cell correspondences in the correspondent colonies are computed using the recorded cell centers. The examples show the proposed cell tracking method can achieve high tracking accuracy for time-lapse 3D images of undifferentiated but self-renewing mouse embryonic stem (mES) cells where the number and mobility of ES cells in a cell colony may change suddenly by a colony merging or splitting, and cell proliferation or death. The geometric data in the hierarchical data structures also help the visualization and quantitation of the cell shapes and mobility.

High Resolution U-Net for Quantitatively Analyzing Early Spatial Patterning of Human Induced Pluripotent Stem Cells on Micropatterns.

Human induced pluripotent stem cells (hiPSCs) can differentiate into three germ layer cells, i.e. ectoderm, mesoderm and endoderm, on micropatterned chips in highly synchronous and reproducible manners. The cells are confined within the chip, expanding two-dimensionally as almost in the form of monolayer, thus to be ideal for serving quantitative analysis of their pluripotency. We present a new U-Net (MP-UNet) structure for cell segmentation of early spatial patterning of hiPSCs on micropattern chips using Hoechst fluorescence images. In this structure, the encoding/decoding layers can be dynamically adjusted to extract sufficient image features and be flexible to image sizes. Dice and weight loss functions are designed to identify slight difference in low signal-to-noise ratio, high boundary-to-area ratio and compacted cell images. Several sizes of Hoechst images were tested to show MP-UNet can achieve high accuracy in cell regions and number counting for various sizes of micropattern chips, thus to be excellent quantitative tool for early spatial patterning of hiPSCs.

Distributed sensory coding by cerebellar complex spikes in units of cortical segments.

Sensory processing is essential for motor control. Climbing fibers from the inferior olive transmit sensory signals to Purkinje cells, but how the signals are represented in the cerebellar cortex remains elusive. To examine the olivocerebellar organization of the mouse brain, we perform quantitative Ca2+ imaging to measure complex spikes (CSs) evoked by climbing fiber inputs over the entire dorsal surface of the cerebellum simultaneously. The surface is divided into approximately 200 segments, each composed of ∼100 Purkinje cells that fire CSs synchronously. Our in vivo imaging reveals that, although stimulation of four limb muscles individually elicits similar global CS responses across nearly all segments, the timing and location of a stimulus are derived by Bayesian inference from coordinated activation and inactivation of multiple segments on a single trial basis. We propose that the cerebellum performs segment-based, distributed-population coding that represents the conditional probability of sensory events.