Whole-brain imaging with single-cell resolution using chemical cocktails and computational analysis.

Systems-level identification and analysis of cellular circuits in the brain will require the development of whole-brain imaging with single-cell resolution. To this end, we performed comprehensive chemical screening to develop a whole-brain clearing and imaging method, termed CUBIC (clear, unobstructed brain imaging cocktails and computational analysis). CUBIC is a simple and efficient method involving the immersion of brain samples in chemical mixtures containing aminoalcohols, which enables rapid whole-brain imaging with single-photon excitation microscopy. CUBIC is applicable to multicolor imaging of fluorescent proteins or immunostained samples in adult brains and is scalable from a primate brain to subcellular structures. We also developed a whole-brain cell-nuclear counterstaining protocol and a computational image analysis pipeline that, together with CUBIC reagents, enable the visualization and quantification of neural activities induced by environmental stimulation. CUBIC enables time-course expression profiling of whole adult brains with single-cell resolution.

Development of a visible to 1600 nm hyperspectral imaging rigid-scope system using supercontinuum light and an acousto-optic tunable filter.

In this study, we developed a rigid-scope system that can perform hyperspectral imaging (HSI) between visible and 1600 nm wavelengths using a supercontinuum light source and an acousto-optic tunable filter to emit specific wavelengths. The system optical performance was verified, and the classification ability was investigated. Consequently, it was demonstrated that HSI (490-1600 nm) could be performed. In addition, seven different targets could be classified by the neural network with an accuracy of 99.6%, recall of 93.7%, and specificity of 99.1% when the wavelength range of over 1000 nm (OTN) was extracted from HSI data as train data.

Deep learning for quantifying spatial patterning and formation process of early differentiated human-induced pluripotent stem cells with micropattern images.

Micropatterning is reliable method for quantifying pluripotency of human-induced pluripotent stem cells (hiPSCs) that differentiate to form a spatial pattern of sorted, ordered and nonoverlapped three germ layers on the micropattern. In this study, we propose a deep learning method to quantify spatial patterning of the germ layers in the early differentiation stage of hiPSCs using micropattern images. We propose decoding and encoding U-net structures learning labelled Hoechst (DNA-stained) hiPSC regions with corresponding Hoechst and bright-field micropattern images to segment hiPSCs on Hoechst or bright-field images. We also propose a U-net structure to extract extraembryonic regions on a micropattern, and an algorithm to compares intensities of the fluorescence images staining respective germ-layer cells and extract their regions. The proposed method thus can quantify the pluripotency of a hiPSC line with spatial patterning including cell numbers, areas and distributions of germ-layer and extraembryonic cells on a micropattern, and reveal the formation process of hiPSCs and germ layers in the early differentiation stage by segmenting live-cell bright-field images. In our assay, the cell-number accuracy achieved 86% and 85%, and the cell region accuracy 89% and 81% for segmenting Hoechst and bright-field micropattern images, respectively. Applications to micropattern images of multiple hiPSC lines, micropattern sizes, groups of markers, living and fixed cells show the proposed method can be expected to be a useful protocol and tool to quantify pluripotency of a new hiPSC line before providing it to the scientific community.

A comprehensible machine learning tool to differentially diagnose idiopathic pulmonary fibrosis from other chronic interstitial lung diseases.

BACKGROUND AND OBJECTIVE: Idiopathic pulmonary fibrosis (IPF) has poor prognosis, and the multidisciplinary diagnostic agreement is low. Moreover, surgical lung biopsies pose comorbidity risks. Therefore, using data from non-invasive tests usually employed to assess interstitial lung diseases (ILDs), we aimed to develop an automated algorithm combining deep learning and machine learning that would be capable of detecting and differentiating IPF from other ILDs. METHODS: We retrospectively analysed consecutive patients presenting with ILD between April 2007 and July 2017. Deep learning was used for semantic image segmentation of HRCT based on the corresponding labelled images. A diagnostic algorithm was then trained using the semantic results and non-invasive findings. Diagnostic accuracy was assessed using five-fold cross-validation. RESULTS: In total, 646,800 HRCT images and the corresponding labelled images were acquired from 1068 patients with ILD, of whom 42.7% had IPF. The average segmentation accuracy was 96.1%. The machine learning algorithm had an average diagnostic accuracy of 83.6%, with high sensitivity, specificity and kappa coefficient values (80.7%, 85.8% and 0.665, respectively). Using Cox hazard analysis, IPF diagnosed using this algorithm was a significant prognostic factor (hazard ratio, 2.593; 95% CI, 2.069-3.250; p < 0.001). Diagnostic accuracy was good even in patients with usual interstitial pneumonia patterns on HRCT and those with surgical lung biopsies. CONCLUSION: Using data from non-invasive examinations, the combined deep learning and machine learning algorithm accurately, easily and quickly diagnosed IPF in a population with various ILDs.

Contributing role of mitochondrial energy metabolism on platelet adhesion, activation and thrombus formation under blood flow conditions.

Platelets have an active energy metabolism mediated by mitochondria. However, the role of mitochondria in platelet adhesion, activation, and thrombus formation under blood flow conditions remains to be elucidated. Blood specimens were obtained from healthy adult volunteers. The consumption of glucose molecules by platelets was measured after 24 hours. Platelet adhesion, activation, and thrombus formation on collagen fibrils and immobilized von Willebrand factor (VWF) at a wall shear rate of 1,500 s-1 were detected by fluorescence microscopy with an ultrafast laser confocal unit in the presence or absence of mitochondrial functional inhibitors of carbonyl cyanide 4-(trifluoromethoxy) phenylhydrazone (FCCP), antimycin A, and oligomycin. Consumption of glucose molecules within the first 24 h of 4.21 × 10-15 ± 4.46 x 10-15 (n = 6) increased to 13.82 × 10-15 ± 3.46 x 10-15 (n = 4) in the presence of FCCP, 12.11 × 10-15 ± 2.33 x 10-15 (n = 4) in the presence of antimycin A, and 11.87 × 10-15 ± 3.56 x 10-15 (n = 4) in the presence of oligomycin (p < .05). These mitochondrial functional blockers did not influence both surface area coverage by platelets and the 3-dimensional size of platelet thrombi formed on the collagen fibrils. However, a rapid increase in the intracellular calcium ion concentration ([Ca2+]i) upon adhering on immobilized VWF decreased significantly from 405.5 ± 86.2 nM in control to 198.0 ± 79.2 nM in the presence of FCCP (p < .005). A similar decrease in the rapid increase in ([Ca2+]i) was observed in the presence of antimycin A and oligomycin. Mitochondrial function is necessary for platelet activation represented by a rapid increase in [Ca2+]i after platelet adhesion on VWF. However, the influence could not be detected as changes in platelet adhesion or 3-dimensional growth of platelet thrombi on collagen fibrils.

Influence of the difference in refractive index on the interface of an object and the surroundings in near-infrared fluorescence tomography.

The refraction of fluorescence from the inside of a sample at the surface results in fluctuations in fluorescence computed tomography (CT). We evaluated the influence of the difference in refractive index (RI) between the sample body and the surroundings on fluorescence CT results. The brightest fluorescent point is away from the correct point on the tomograms owing to the refraction. The speculated position is determined as the exact point if the RI ratio ranges between 0.97 and 1.03 by immersing the body in an RI matching liquid. The results can help in experimental settings of fluorescence CT for acquiring three-dimensional positional information.

Detecting colon polyps in endoscopic images using artificial intelligence constructed with automated collection of annotated images from an endoscopy reporting system.

BACKGROUND: Artificial intelligence (AI) has made considerable progress in image recognition, especially in the analysis of endoscopic images. The availability of large-scale annotated datasets has contributed to the recent progress in this field. Datasets of high-quality annotated endoscopic images are widely available, particularly in Japan. A system for collecting annotated data reported daily could aid in accumulating a significant number of high-quality annotated datasets. AIM: We assessed the validity of using daily annotated endoscopic images in a constructed reporting system for a prototype AI model for polyp detection. METHODS: We constructed an automated collection system for daily annotated datasets from an endoscopy reporting system. The key images were selected and annotated for each case only during daily practice, not to be performed retrospectively. We automatically extracted annotated endoscopic images of diminutive colon polyps that had been diagnosed (study period March-September 2018) using the keywords of diagnostic information, and additionally collect the normal colon images. The collected dataset was devised into training and validation to build and evaluate the AI system. The detection model was developed using a deep learning algorithm, RetinaNet. RESULTS: The automated system collected endoscopic images (47,391) from colonoscopies (745), and extracted key colon polyp images (1356) with localized annotations. The sensitivity, specificity, and accuracy of our AI model were 97.0%, 97.7%, and 97.3% (n = 300), respectively. CONCLUSION: The automated system enabled the development of a high-performance colon polyp detector using images in endoscopy reporting system without the efforts of retrospective annotation works.

Accurate and fast mitotic detection using an anchor-free method based on full-scale connection with recurrent deep layer aggregation in 4D microscopy images.

BACKGROUND: To effectively detect and investigate various cell-related diseases, it is essential to understand cell behaviour. The ability to detection mitotic cells is a fundamental step in diagnosing cell-related diseases. Convolutional neural networks (CNNs) have been successfully applied to object detection tasks, however, when applied to mitotic cell detection, most existing methods generate high false-positive rates due to the complex characteristics that differentiate normal cells from mitotic cells. Cell size and orientation variations in each stage make detecting mitotic cells difficult in 2D approaches. Therefore, effective extraction of the spatial and temporal features from mitotic data is an important and challenging task. The computational time required for detection is another major concern for mitotic detection in 4D microscopic images. RESULTS: In this paper, we propose a backbone feature extraction network named full scale connected recurrent deep layer aggregation (RDLA++) for anchor-free mitotic detection. We utilize a 2.5D method that includes 3D spatial information extracted from several 2D images from neighbouring slices that form a multi-stream input. CONCLUSIONS: Our proposed technique addresses the scale variation problem and can efficiently extract spatial and temporal features from 4D microscopic images, resulting in improved detection accuracy and reduced computation time compared with those of other state-of-the-art methods.

A metabolic reaction-diffusion model for PKCα translocation via PIP2 hydrolysis in an endothelial cell.

Hydrolysis of the phospholipid phosphatidylinositol 4,5-bisphosphate (PIP2) at the cell membrane induces the release of inositol 1,4,5-trisphosphate (IP3) into the cytoplasm and diffusion of diacylglycerol (DAG) through the membrane, respectively. Release of IP3 subsequently increases Ca2+ levels in the cytoplasm, which results in activation of protein kinase C α (PKCα) by Ca2+ and DAG, and finally the translocation of PKCα from the cytoplasm to the membrane. In this study, we developed a metabolic reaction-diffusion framework to simulate PKCα translocation via PIP2 hydrolysis in an endothelial cell. A three-dimensional cell model, divided into membrane and cytoplasm domains, was reconstructed from confocal microscopy images. The associated metabolic reactions were divided into their corresponding domain; PIP2 hydrolysis at the membrane domain resulted in DAG diffusion at the membrane domain and IP3 release into the cytoplasm domain. In the cytoplasm domain, Ca2+ was released from the endoplasmic reticulum, and IP3, Ca2+, and PKCα diffused through the cytoplasm. PKCα bound Ca2+ at, and diffused through, the cytoplasm, and was finally activated by binding with DAG at the membrane. Using our model, we analyzed IP3 and DAG dynamics, Ca2+ waves, and PKCα translocation in response to a microscopic stimulus. We found a qualitative agreement between our simulation results and our experimental results obtained by live-cell imaging. Interestingly, our results suggest that PKCα translocation is dominated by DAG dynamics. This three-dimensional reaction-diffusion mathematical framework could be used to investigate the link between PKCα activation in a cell and cell function.

ViBrism DB: an interactive search and viewer platform for 2D/3D anatomical images of gene expression and co-expression networks.

Understanding anatomical structures and biological functions based on gene expression is critical in a systemic approach to address the complexity of the mammalian brain, where >25 000 genes are expressed in a precise manner. Co-expressed genes are thought to regulate cell type- or region-specific brain functions. Thus, well-designed data acquisition and visualization systems for profiling combinatorial gene expression in relation to anatomical structures are crucial. To this purpose, using our techniques of microtomy-based gene expression measurements and WebGL-based visualization programs, we mapped spatial expression densities of genome-wide transcripts to the 3D coordinates of mouse brains at four post-natal stages, and built a database, ViBrism DB (http://vibrism.neuroinf.jp/). With the DB platform, users can access a total of 172 022 expression maps of transcripts, including coding, non-coding and lncRNAs in the whole context of 3D magnetic resonance (MR) images. Co-expression of transcripts is represented in the image space and in topological network graphs. In situ hybridization images and anatomical area maps are browsable in the same space of 3D expression maps using a new browser-based 2D/3D viewer, BAH viewer. Created images are shareable using URLs, including scene-setting parameters. The DB has multiple links and is expandable by community activity.

Over 1000 nm Near-Infrared Multispectral Imaging System for Laparoscopic In Vivo Imaging.

In this study, a laparoscopic imaging device and a light source able to select wavelengths by bandpass filters were developed to perform multispectral imaging (MSI) using over 1000 nm near-infrared (OTN-NIR) on regions under a laparoscope. Subsequently, MSI (wavelengths: 1000-1400 nm) was performed using the built device on nine live mice before and after tumor implantation. The normal and tumor pixels captured within the mice were used as teaching data sets, and the tumor-implanted mice data were classified using a neural network applied following a leave-one-out cross-validation procedure. The system provided a specificity of 89.5%, a sensitivity of 53.5%, and an accuracy of 87.8% for subcutaneous tumor discrimination. Aggregated true-positive (TP) pixels were confirmed in all tumor-implanted mice, which indicated that the laparoscopic OTN-NIR MSI could potentially be applied in vivo for classifying target lesions such as cancer in deep tissues.

Assessment of the deformation of the outer nuclear layer in the Epiretinal membrane using spectral-domain optical coherence tomography.

BACKGROUND: We aimed to investigate the deformation of the outer nuclear layer using optical coherence tomography in patients with epiretinal membrane (ERM) and its relationship with metamorphopsia. METHODS: Thirty-nine eyes from 39 patients with ERM were included in the study. Patients with the subtypes of pseudo macula hole and lamellar hole were excluded. Twenty-one fellow eyes without macular disease were included as normal controls. Forty-nine B-scan images were obtained in the range of 20 degrees around the macula using SD-OCT. The outer nuclear layer (ONL) was evaluated as a three-dimensional image (3D-ONL) reconstructed using the distance between the ONL and retinal pigment epithelium (RPE) line. The deformation of the ONL was figured at the reference plane and evaluation plane (ONL-B). The characteristic parameters of the ONL-B were defined as circularity, area ratio, and axis ratio. The correlations between these parameters and visual acuity and MCHART score ratio (MH/MV) were then evaluated. RESULTS: ONL height was significantly higher in ERM patients than in normal controls (54.1 ± 5.3 µm and 84.1 ± 12.9 µm, respectively; P < 0.001). In ERM patients, the MV score was 0.53 ± 0.50, the MH score was 0.71 ± 0.61, and the distance from the RPE line to the ONL-B was 153.5 ± 13.5 µm. The axis of the ONL-B in normal controls and ERM patients was - 6.25 ± 21.8 and - 1.28 ± 29.1, respectively, which indicates that the ONL is horizontally long in both normal individuals and ERM patients. The circularity and area ratio were significantly smaller in ERM patients than in normal controls. In all ERM patients, MH/MV had a significant correlation with axis (r = - 0.29, p = 0.034), circularity (r = - 0.28, p = 0.044), and area ratio (r = - 0.47, p = 0.001). Moreover, we found that the correlation was more significant if the subjects had an axis of the ONL within ±10 degrees (n = 16); the correlations of MH/MV with axis (r = - 0.29, p = 0.034), circularity (r = - 0.53, p = 0.021), and area ratio were more significant (r = - 0.78, P < 0.0001). CONCLUSION: The ONL is horizontally long in normal individuals and ERM patients. The direction of metamorphopsia is correlated with the direction of ONL deformation.

Three-dimensional model of intracellular and intercellular Ca2+ waves propagation in endothelial cells.

Intracellular and intercellular Ca2+ waves play key roles in cellular functions, and focal stimulation triggers Ca2+ wave propagation from stimulation points to neighboring cells, involving localized metabolism reactions and specific diffusion processes. Among these, inositol 1,4,5-trisphosphate (IP3) is produced at membranes and diffuses into the cytoplasm to release Ca2+ from endoplasmic reticulum (ER). In this study, we developed a three-dimensional (3D) simulation model for intercellular and intracellular Ca2+ waves in endothelial cells (ECs). 3D model of 2 cells was reconstructed from confocal microscopic images and was connected via gap junctions. Cells have membrane and cytoplasm domains, and metabolic reactions were divided into each domain. Finally, the intracellular and intercellular Ca2+ wave propagations were induced using microscopic stimulation and were compared between numerical simulations and experiments. The experiments showed that initial sharp increases in intracellular Ca2+ occurred approximately 0.3 s after application of stimuli. In addition, Ca2+ wave speeds remained constant in cells, with intracellular and intercellular speeds of approximately 35 and 15 µm/s, respectively. Simulations indicated initial increases in Ca2+ concentrations at points of stimulation, and these were then propagated across stimulated and neighboring cells. In particular, initial rapid increases in intracellular Ca2+ were delayed and subsequent intracellular and intercellular Ca2+ wave speeds were approximately 25 and 12 µm/s, respectively. Simulation results were in agreement with those from cell culture experiments, indicating the utility of our 3D model for investigations of intracellular and intercellular messaging in ECs.

Prediction of open urinary tract in laparoscopic partial nephrectomy by virtual resection plane visualization.

BACKGROUND: The purpose of this study is presenting a method to predict the presence of an open urinary tract and the position of the opening in laparoscopic partial nephrectomy from three dimensional (3D) computed tomography (CT) images by using novel image segmentation and visualization techniques. METHODS: From CT images of patients who underwent laparoscopic partial nephrectomy, 3D regions of the kidney, urinary tract, and tumor were segmented. For each patient, multiple virtual resection planes of the kidney with different surgical margins (1 mm to 5 mm, every 1 mm) were generated and the presence of an open urinary tract and the position of the opening were predicted from the images. RESULTS: We compared the predictions with actual operations in 5 cases by using recorded video of the operations and operative notes. In terms of the presence of an open urinary tract, agreement of the predictions and the intraoperative results was obtained in all patients. The expected positions of the openings were close to those in the actual operations. CONCLUSIONS: We have developed a method to virtually visualize the resection plane of laparoscopic partial nephrectomy. Image segmentation methods used in this study were precise and effective. The comparison indicated that our method accurately predicted the presence of an open urinary tract and the position of the opening and provided useful preoperative information.

Assessing Molecular Localization of Symbiont Microalgae in Coral Branches Through Mass Spectrometry Imaging.

Reef-building corals are a fundamental pillar of coral reef ecosystems in tropical and subtropical shallow environments. Corals harbor symbiotic dinoflagellates belonging to the family Symbiodiniaceae, commonly known as zooxanthellae. Extensive research has been conducted on this symbiotic relationship, yet the fundamental information about the distribution and localization of Symbiodiniaceae cells in corals is still limited. This information is crucial to understanding the mechanism underlying the metabolite exchange between corals and their algal symbionts, as well as the metabolic flow within holobionts. To examine the distribution of Symbiodiniaceae cells within corals, in this study, we used fluorescence imaging and matrix-assisted laser desorption/ionization (MALDI) mass spectrometry imaging (MS-Imaging) on branches of the Acropora tenuis coral. We successfully prepared frozen sections of the coral for molecular imaging without fixing or decalcifying the coral branches. By combining the results of MS-Imaging with that of the fluorescence imaging, we determined that the algal Symbiodiniaceae symbionts were not only localized in the tentacle and surface region of the coral branches but also inhabited the in inner parts. Therefore, the molecular imaging technique used in this study could be valuable to further investigate the molecular dynamics between corals and their symbionts.

The molecular mechanism of apoptosis upon caspase-8 activation: quantitative experimental validation of a mathematical model.

Caspase-8 (CASP8) is a cysteine protease that plays a pivotal role in the extrinsic apoptotic signaling pathway via death receptors. The kinetics, dynamics, and selectivity with which the pathway transmits apoptotic signals to downstream molecules upon CASP8 activation are not fully understood. We have developed a system for using high-sensitivity FRET-based biosensors to monitor the protease activity of CASP8 and its downstream effector, caspase-3, in living single cells. Using this system, we systematically investigated the caspase cascade by regulating the magnitude of extrinsic signals received by the cell. Furthermore, we determined the molar concentration of five caspases and Bid required for hierarchical transmission of apoptotic signals in a HeLa cell. Based on these quantitative experimental data, we validated a mathematical model suitable for estimation of the kinetics and dynamics of caspases, which predicts the minimal concentration of CASP8 required to act as an initiator. Consequently, we found that less than 1% of the total CASP8 proteins are sufficient to set the apoptotic program in motion if activated. Taken together, our findings demonstrate the precise cascade of CASP8-mediated apoptotic signals through the extrinsic pathway.

Human induced pluripotent stem cell formation and morphology prediction during reprogramming with time-lapse bright-field microscopy images using deep learning methods.

BACKGROUND AND OBJECTIVE: Human induced pluripotent stem cells (hiPSCs) represent an ideal source for patient specific cell-based regenerative medicine; however, efficiency of hiPSC formation from reprogramming cells is low. We use several deep-learning results from time-lapse brightfield microscopy images during culture, to early detect the cells potentially reprogramming into hiPSCs and predict the colony morphology of these cells for improving efficiency of culturing a new hiPSC line. METHODS: Sets of time-lapse bright-field images are taken to track reprogramming process of CD34+ cells biologically identified as just beginning reprogramming. Prior the experiment, 9 classes of templates with distinct cell features clipped from microscopy images at various reprogramming stages are used to train a CNN model. The CNN is then used to classify a microscopy image as probability images of these classes. Probability images of some class are used to train a densely connected convolutional network for extracting regions of this class on a microscopy image. A U-net is trained to segment cells on the time-lapse images in early reprogramming stage during culture. The segmented cells are classified by the extracted regions to count various types of cells appearing in the early reprogramming stage for predicting the identified cells potentially forming hiPSCs. The probability images of hiPSC classes are also used to train a spatiotemporal RNN for predicting the future hiPSC colony morphology of the potential cells. RESULTS: Experimental results show the prediction (before 7 days after of beginning of the reprogramming) achieved 0.8 accuracy, and 66% of the identified cells under different culture conditions, predicted as forming, finally formed hiPSCs. The predicted hiPSC images and extracted colonies on the images show the prediction for future 1.5 days achieved high accuracy of hiPSC colony areas and image similarity. CONCLUSIONS: Our study proposes a method using several deep learning models to efficiently select the reprogramming cells possibly forming hiPSCs and to predict the shapes of growing hiPSC colonies. The proposed method is expected to improve the efficiency when establishing a new hiPSC line culture.

Quantitative Analyses for Early Tempo-spatial Patterning of Differentiated Human Induced Pluripotent Stem Cells on Micropatterns using Time-lapse Bright-field Microscopy Images.

Three germ layer formation on micropatterns are extremely useful for quantitative analysis of hiPSC (human induced pluripotent stem cells) pluripotency. Spatial patterns of stem cells differentiated on the micropatterns will be formed from about 24 hours after differentiation induction and usually quantitated near 48 hours. To delineate the germ layer formation process, temporal changes in spatial patterning of germ layers should be analyzed by noninvasive microscopy. This study proposed a series of image processing methods combined with a U-net automatic segmentation to segment differentiated hiPSCs captured by bright-field microscopy. High segmentation accuracy (83.3%) for the test bright-field images compared with their concurrent Hoechst images (85%) was achieved. Tempo-spatial patterning and formation process of germ layers on the micropatterns can be visualized and quantified by segmenting time-lapse bright-field microscopy images using our method.

Quantitative Analysis of Differentiation Activity for Mouse Embryonic Stem Cells by Deep Learning for Cell Center Detection using Three-Dimensional Confocal Fluorescence Microscopy Images.

Accurate single cell segmentation provides means to monitor the behavior of single cell within a population of cells. Time-lapse fluorescence images are used to reveal heterogeneous nature of single mouse embryonic stem cell (ESC) colony and monitor fluctuations of the cell states. Automatic quantification of speed and status shifts of the ESCs depends on accurate single cell segmentation that is used to calculate the 3D center of every cell and track this cell for the quantification. This study proposes a new 3D U-net to accurately detect center of each single cell in 3D confocal images. The dimension of input 3D images to the U-net is flexible so that multiple center detections from different image directions can be implemented simultaneously to improve the center detection accuracy. This study showed that our method can improve accuracy for cell center detection and thus the quantification for ESC speeds and status shifts.