Deep UV-excited fluorescence microscopy installed with CycleGAN-assisted image translation enhances precise detection of lymph node metastasis towards rapid intraoperative diagnosis.

Rapid and precise intraoperative diagnosing systems are required for improving surgical outcomes and patient prognosis. Because of the poor quality and time-intensive process of the prevalent frozen section procedure, various intraoperative diagnostic imaging systems have been explored. Microscopy with ultraviolet surface excitation (MUSE) is an inexpensive, maintenance-free, and rapid imaging technique that yields images like thin-sectioned samples without sectioning. However, pathologists find it nearly impossible to assign diagnostic labels to MUSE images of unfixed specimens; thus, AI for intraoperative diagnosis cannot be trained in a supervised learning manner. In this study, we propose a deep-learning pipeline model for lymph node metastasis detection, in which CycleGAN translate MUSE images of unfixed lymph nodes to formalin-fixed paraffin-embedded (FFPE) sample, and diagnostic prediction is performed using deep convolutional neural network trained on FFPE sample images. Our pipeline yielded an average accuracy of 84.6% when using each of the three deep convolutional neural networks, which is a 18.3% increase over the classification-only model without CycleGAN. The modality translation to FFPE sample images using CycleGAN can be applied to various intraoperative diagnostic imaging systems and eliminate the difficulty for pathologists in labeling new modality images in clinical sites. We anticipate our pipeline to be a starting point for accurate rapid intraoperative diagnostic systems for new imaging modalities, leading to healthcare quality improvement.

Application of deep learning as an ancillary diagnostic tool for thyroid FNA cytology.

BACKGROUND: Several studies have used artificial intelligence (AI) to analyze cytology images, but AI has yet to be adopted in clinical practice. The objective of this study was to demonstrate the accuracy of AI-based image analysis for thyroid fine-needle aspiration cytology (FNAC) and to propose its application in clinical practice. METHODS: In total, 148,395 microscopic images of FNAC were obtained from 393 thyroid nodules to train and validate the data, and EfficientNetV2-L was used as the image-classification model. The 35 nodules that were classified as atypia of undetermined significance (AUS) were predicted using AI training. RESULTS: The precision-recall area under the curve (PR AUC) was >0.95, except for poorly differentiated thyroid carcinoma (PR AUC = 0.49) and medullary thyroid carcinoma (PR AUC = 0.91). Poorly differentiated thyroid carcinoma had the lowest recall (35.4%) and was difficult to distinguish from papillary thyroid carcinoma, medullary thyroid carcinoma, and follicular thyroid carcinoma. Follicular adenomas and follicular thyroid carcinomas were distinguished from each other by 86.7% and 93.9% recall, respectively. For two-dimensional mapping of the data using t-distributed stochastic neighbor embedding, the lymphomas, follicular adenomas, and anaplastic thyroid carcinomas were divided into three, two, and two groups, respectively. Analysis of the AUS nodules showed 94.7% sensitivity, 14.4% specificity, 56.3% positive predictive value, and 66.7% negative predictive value. CONCLUSIONS: The authors developed an AI-based approach to analyze thyroid FNAC cases encountered in routine practice. This analysis could be useful for the clinical management of AUS and follicular neoplasm nodules (e.g., an online AI platform for thyroid cytology consultations).

Automated diagnosis of optical coherence tomography imaging on plaque vulnerability and its relation to clinical outcomes in coronary artery disease.

This study sought to develop a deep learning-based diagnostic algorithm for plaque vulnerability by analyzing intravascular optical coherence tomography (OCT) images and to investigate the relation between AI-plaque vulnerability and clinical outcomes in patients with coronary artery disease (CAD). A total of 1791 study patients who underwent OCT examinations were recruited from a multicenter clinical database, and the OCT images were first labeled as either normal, a stable plaque, or a vulnerable plaque by expert cardiologists. A DenseNet-121-based deep learning algorithm for plaque characterization was developed by training with 44,947 prelabeled OCT images, and demonstrated excellent differentiation among normal, stable plaques, and vulnerable plaques. Patients who were diagnosed with vulnerable plaques by the algorithm had a significantly higher rate of both events from the OCT-observed segments and clinical events than the patients with normal and stable plaque (log-rank p < 0.001). On the multivariate logistic regression analyses, the OCT diagnosis of a vulnerable plaque by the algorithm was independently associated with both types of events (p = 0.047 and p < 0.001, respectively). The AI analysis of intracoronary OCT imaging can assist cardiologists in diagnosing plaque vulnerability and identifying CAD patients with a high probability of occurrence of future clinical events.

Diagnostic performance for pulmonary adenocarcinoma on CT: comparison of radiologists with and without three-dimensional convolutional neural network.

OBJECTIVES: To compare diagnostic performance for pulmonary invasive adenocarcinoma among radiologists with and without three-dimensional convolutional neural network (3D-CNN). METHODS: Enrolled were 285 patients with adenocarcinoma in situ (AIS, n = 75), minimally invasive adenocarcinoma (MIA, n = 58), and invasive adenocarcinoma (IVA, n = 152). A 3D-CNN model was constructed with seven convolution-pooling and two max-pooling layers and fully connected layers, in which batch normalization, residual connection, and global average pooling were used. Only the flipping process was performed for augmentation. The output layer comprised two nodes for two conditions (AIS/MIA and IVA) according to prognosis. Diagnostic performance of the 3D-CNN model in 285 patients was calculated using nested 10-fold cross-validation. In 90 of 285 patients, results from each radiologist (R1, R2, and R3; with 9, 14, and 26 years of experience, respectively) with and without the 3D-CNN model were statistically compared. RESULTS: Without the 3D-CNN model, accuracy, sensitivity, and specificity of the radiologists were as follows: R1, 70.0%, 52.1%, and 90.5%; R2, 72.2%, 75%, and 69%; and R3, 74.4%, 89.6%, and 57.1%, respectively. With the 3D-CNN model, accuracy, sensitivity, and specificity of the radiologists were as follows: R1, 72.2%, 77.1%, and 66.7%; R2, 74.4%, 85.4%, and 61.9%; and R3, 74.4%, 93.8%, and 52.4%, respectively. Diagnostic performance of each radiologist with and without the 3D-CNN model had no significant difference (p > 0.88), but the accuracy of R1 and R2 was significantly higher with than without the 3D-CNN model (p < 0.01). CONCLUSIONS: The 3D-CNN model can support a less-experienced radiologist to improve diagnostic accuracy for pulmonary invasive adenocarcinoma without deteriorating any diagnostic performances. KEY POINTS: • The 3D-CNN model is a non-invasive method for predicting pulmonary invasive adenocarcinoma in CT images with high sensitivity. • Diagnostic accuracy by a less-experienced radiologist was better with the 3D-CNN model than without the model.

Following Embryonic Stem Cells, Their Differentiated Progeny, and Cell-State Changes During iPS Reprogramming by Raman Spectroscopy.

Monitoring cell-state transition in pluripotent cells is invaluable for application and basic research. In this study, we demonstrate the pertinence of noninvasive, label-free Raman spectroscopy to monitor and characterize the cell-state transition of mouse stem cells undergoing reprogramming. Using an isogenic cell line of mouse stem cells, reprogramming from neuronal cells was performed, and we showcase a comparative analysis of living single-cell spectral data of the original stem cells, their neuronal progenitors, and reprogrammed cells. Neural network, regression models, and ratiometric analyses were used to discriminate the cell states and extract several important biomarkers specific to differentiation or reprogramming. Our results indicated that the Raman spectrum allowed us to build a low-dimensional space allowing us to monitor and characterize the dynamics of cell-state transition at a single-cell level, scattered in heterogeneous populations. The ability of monitoring pluripotency by Raman spectroscopy and distinguishing differences between ES and reprogrammed cells is also discussed.

Red-Fluorescent Pt Nanoclusters for Detecting and Imaging HER2 in Breast Cancer Cells.

Overexpression of human epidermal growth factor receptor 2 (HER2) is associated with more frequent cancer recurrence and metastasis. Sensitive sensing of HER2 in living breast cancer cells is crucial in the early stages of cancer and to further understand its role in cells. Biomedical imaging has become an indispensable tool in the fields of early cancer diagnosis and therapy. In this study, we designed and synthesized platinum (Pt) nanocluster bionanoprobes with red emission (Ex/Em = 535/630 nm) for fluorescence imaging of HER2. Our Pt nanoclusters, which were synthesized using polyamidoamine (PAMAM) dendrimer and preequilibration, exhibited approximately 1% quantum yield and possessed low cytotoxicity, ultrasmall size, and excellent photostability. Furthermore, combined with ProteinA as an adapter protein, we developed Pt bionanoprobes with minimal nonspecific binding and utilized them as fluorescent probes for highly sensitive optical imaging of HER2 at the cellular level. More importantly, molecular probes with long-wavelength emission have allowed visualization of deep anatomical features because of enhanced tissue penetration and a decrease in background noise from tissue scattering. Our Pt nanoclusters are promising fluorescent probes for biomedical applications.

Improvement of nerve imaging speed with coherent anti-Stokes Raman scattering rigid endoscope using deep-learning noise reduction.

A coherent anti-Stokes Raman scattering (CARS) rigid endoscope was developed to visualize peripheral nerves without labeling for nerve-sparing endoscopic surgery. The developed CARS endoscope had a problem with low imaging speed, i.e. low imaging rate. In this study, we demonstrate that noise reduction with deep learning boosts the nerve imaging speed with CARS endoscopy. We employ fine-tuning and ensemble learning and compare deep learning models with three different architectures. In the fine-tuning strategy, deep learning models are pre-trained with CARS microscopy nerve images and retrained with CARS endoscopy nerve images to compensate for the small dataset of CARS endoscopy images. We propose using the equivalent imaging rate (EIR) as a new evaluation metric for quantitatively and directly assessing the imaging rate improvement by deep learning models. The highest EIR of the deep learning model was 7.0 images/min, which was 5 times higher than that of the raw endoscopic image of 1.4 images/min. We believe that the improvement of the nerve imaging speed will open up the possibility of reducing postoperative dysfunction by intraoperative nerve identification.

Nerve Segmentation with Deep Learning from Label-Free Endoscopic Images Obtained Using Coherent Anti-Stokes Raman Scattering.

Semantic segmentation with deep learning to extract nerves from label-free endoscopic images obtained using coherent anti-Stokes Raman scattering (CARS) for nerve-sparing surgery is described. We developed a CARS rigid endoscope in order to identify the exact location of peripheral nerves in surgery. Myelinated nerves are visualized with a CARS lipid signal in a label-free manner. Because the lipid distribution includes other tissues as well as nerves, nerve segmentation is required to achieve nerve-sparing surgery. We propose using U-Net with a VGG16 encoder as a deep learning model and pre-training with fluorescence images, which visualize the lipid distribution similar to CARS images, before fine-tuning with a small dataset of CARS endoscopy images. For nerve segmentation, we used 24 CARS and 1,818 fluorescence nerve images of three rabbit prostates. We achieved label-free nerve segmentation with a mean accuracy of 0.962 and an F 1 value of 0.860. Pre-training on fluorescence images significantly improved the performance of nerve segmentation in terms of the mean accuracy and F 1 value ( p < 0 . 05 ). Nerve segmentation of label-free endoscopic images will allow for safer endoscopic surgery, while reducing dysfunction and improving prognosis after surgery.

Convolutional Neural Network Can Recognize Drug Resistance of Single Cancer Cells.

It is known that single or isolated tumor cells enter cancer patients' circulatory systems. These circulating tumor cells (CTCs) are thought to be an effective tool for diagnosing cancer malignancy. However, handling CTC samples and evaluating CTC sequence analysis results are challenging. Recently, the convolutional neural network (CNN) model, a type of deep learning model, has been increasingly adopted for medical image analyses. However, it is controversial whether cell characteristics can be identified at the single-cell level by using machine learning methods. This study intends to verify whether an AI system could classify the sensitivity of anticancer drugs, based on cell morphology during culture. We constructed a CNN based on the VGG16 model that could predict the efficiency of antitumor drugs at the single-cell level. The machine learning revealed that our model could identify the effects of antitumor drugs with ~0.80 accuracies. Our results show that, in the future, realizing precision medicine to identify effective antitumor drugs for individual patients may be possible by extracting CTCs from blood and performing classification by using an AI system.

Deep-UV excitation fluorescence microscopy for detection of lymph node metastasis using deep neural network.

Deep-UV (DUV) excitation fluorescence microscopy has potential to provide rapid diagnosis with simple technique comparing to conventional histopathology based on hematoxylin and eosin (H&E) staining. We established a fluorescent staining protocol for DUV excitation fluorescence imaging that has enabled clear discrimination of nucleoplasm, nucleolus, and cytoplasm. Fluorescence images of metastasis-positive/-negative lymph nodes of gastric cancer patients were used for patch-based training with a deep neural network (DNN) based on Inception-v3 architecture. The performance on small patches of the fluorescence images was comparable with that of H&E images. Gradient-weighted class activation mapping analysis revealed the areas where the trained model identified metastatic lesions in the images containing cancer cells. We extended the method to large-size image analysis enabling accurate detection of metastatic lesions. We discuss usefulness of DUV excitation fluorescence imaging with the aid of DNN analysis, which is promising for assisting pathologists in assessment of lymph node metastasis.

Excitation of erbium-doped nanoparticles in 1550-nm wavelength region for deep tissue imaging with reduced degradation of spatial resolution.

Rare-earth-doped nanoparticles are one of the emerging probes for bioimaging due to their visible-to-near-infrared (NIR) upconversion emission via sequential single-photon absorption at NIR wavelengths. The NIR-excited upconversion property and high photostability make this probe appealing for deep tissue imaging. So far, upconversion nanoparticles include ytterbium ions (Yb3 + ) codoped with other rare earth ions, such as erbium (Er3 + ) and thulium (Tm3 + ). In these types of upconversion nanoparticles, through energy transfer from Yb3 + excited with continuous wave light at a wavelength of 980 nm, upconversion emission of the other rare earth dopants is induced. We have found that the use of the excitation of Er3 + in the 1550-nm wavelength region allows us to perform deep tissue imaging with reduced degradation of spatial resolution. In this excitation­emission process, three and four photons of 1550-nm light are sequentially absorbed, and Er3 + emits photons in the 550- and 660-nm wavelength regions. We demonstrate that, compared with the case using 980-nm wavelength excitation, the use of 1550-nm light enables us to moderate degradation of spatial resolution in deep tissue imaging due to the lower light scattering coefficient compared with 980-nm light. We also demonstrate that live cell imaging is feasible with this 1550 nm excitation.

Application of deep learning (3-dimensional convolutional neural network) for the prediction of pathological invasiveness in lung adenocarcinoma: A preliminary study.

To compare results for radiological prediction of pathological invasiveness in lung adenocarcinoma between radiologists and a deep learning (DL) system.Ninety patients (50 men, 40 women; mean age, 66 years; range, 40-88 years) who underwent pre-operative chest computed tomography (CT) with 0.625-mm slice thickness were included in this retrospective study. Twenty-four cases of adenocarcinoma in situ (AIS), 20 cases of minimally invasive adenocarcinoma (MIA), and 46 cases of invasive adenocarcinoma (IVA) were pathologically diagnosed. Three radiologists of different levels of experience diagnosed each nodule by using previously documented CT findings to predict pathological invasiveness. DL was structured using a 3-dimensional (3D) convolutional neural network (3D-CNN) constructed with 2 successive pairs of convolution and max-pooling layers, and 2 fully connected layers. The output layer comprises 3 nodes to recognize the 3 conditions of adenocarcinoma (AIS, MIA, and IVA) or 2 nodes for 2 conditions (AIS and MIA/IVA). Results from DL and the 3 radiologists were statistically compared.No significant differences in pathological diagnostic accuracy rates were seen between DL and the 3 radiologists (P >.11). Receiver operating characteristic analysis demonstrated that area under the curve for DL (0.712) was almost the same as that for the radiologist with extensive experience (0.714; P = .98). Compared with the consensus results from radiologists, DL offered significantly inferior sensitivity (P = .0005), but significantly superior specificity (P = .02).Despite the small training data set, diagnostic performance of DL was almost the same as the radiologist with extensive experience. In particular, DL provided higher specificity than radiologists.

[Medical Application of Artificial Intelligence/Deep Learning].

Deep learning is a subset of the medical application of artificial intelligence. Its significant results are garnering attention, particularly in radiographic image interpretation, pathological diagnosis, gene analysis, and prediction of cancer recurrence. In this study, we summarize the concept of deep learning. The human body structure, from the molecule to physical functions, is a complex system. Deep learning is a new way to analyze its complex systems. An essential point of the analysis is the categorization of obstacles. To a certain extent, deep learning approximates a doctor's cognition.

Graphical classification of DNA sequences of HLA alleles by deep learning.

Alleles of human leukocyte antigen (HLA)-A DNAs are classified and expressed graphically by using artificial intelligence "Deep Learning (Stacked autoencoder)". Nucleotide sequence data corresponding to the length of 822 bp, collected from the Immuno Polymorphism Database, were compressed to 2-dimensional representation and were plotted. Profiles of the two-dimensional plots indicate that the alleles can be classified as clusters are formed. The two-dimensional plot of HLA-A DNAs gives a clear outlook for characterizing the various alleles.

Classification of C2C12 cells at differentiation by convolutional neural network of deep learning using phase contrast images.

In the field of regenerative medicine, tremendous numbers of cells are necessary for tissue/organ regeneration. Today automatic cell-culturing system has been developed. The next step is constructing a non-invasive method to monitor the conditions of cells automatically. As an image analysis method, convolutional neural network (CNN), one of the deep learning method, is approaching human recognition level. We constructed and applied the CNN algorithm for automatic cellular differentiation recognition of myogenic C2C12 cell line. Phase-contrast images of cultured C2C12 are prepared as input dataset. In differentiation process from myoblasts to myotubes, cellular morphology changes from round shape to elongated tubular shape due to fusion of the cells. CNN abstract the features of the shape of the cells and classify the cells depending on the culturing days from when differentiation is induced. Changes in cellular shape depending on the number of days of culture (Day 0, Day 3, Day 6) are classified with 91.3% accuracy. Image analysis with CNN has a potential to realize regenerative medicine industry.

Measurement of DNA Length Changes Upon CpG Hypermethylation by Microfluidic Molecular Stretching.

Abnormal DNA methylation in CpG-rich promoters is recognized as a distinct molecular feature of precursor lesions to cancer. Such unintended methylation can occur during in vitro differentiation of stem cells. It takes place in a subset of genes during the differentiation or expansion of stem cell derivatives under general culture conditions, which may need to be monitored in future cell transplantation studies. Here we demonstrate a microfluidic device for investigating morphological length changes in DNA methylation. Arrayed polymer chains of single DNA molecules were fluorescently observed by parallel trapping and stretching in the microfluidic channel. This observational study revealed that the shortened DNA length is due to the increased rigidity of the methylated DNA molecule. The trapping rate of the device for DNA molecules was substantially unaffected by changes in the CpG methylation.

Multispectral Emissions of Lanthanide-Doped Gadolinium Oxide Nanophosphors for Cathodoluminescence and Near-Infrared Upconversion/Downconversion Imaging.

Comprehensive imaging of a biological individual can be achieved by utilizing the variation in spatial resolution, the scale of cathodoluminescence (CL), and near-infrared (NIR), as favored by imaging probe Gd2O3 co-doped lanthanide nanophosphors (NPPs). A series of Gd2O3:Ln3+/Yb3+ (Ln3+: Tm3+, Ho3+, Er3+) NPPs with multispectral emission are prepared by the sol-gel method. The NPPs show a wide range of emissions spanning from the visible to the NIR region under 980 nm excitation. The dependence of the upconverting (UC)/downconverting (DC) emission intensity on the dopant ratio is investigated. The optimum ratios of dopants obtained for emissions in the NIR regions at 810 nm, 1200 nm, and 1530 nm are applied to produce nanoparticles by the homogeneous precipitation (HP) method. The nanoparticles produced from the HP method are used to investigate the dual NIR and CL imaging modalities. The results indicate the possibility of using Gd2O3 co-doped Ln3+/Yb3+ (Ln3+: Tm3+, Ho3+, Er3+) in correlation with NIR and CL imaging. The use of Gd2O3 promises an extension of the object dimension to the whole-body level by employing magnetic resonance imaging (MRI).

Rare-earth-doped nanophosphors for multicolor cathodoluminescence nanobioimaging using scanning transmission electron microscopy.

We describe rare-earth-doped nanophosphors (RE-NPs) for biological imaging using cathodoluminescence(CL) microscopy based on scanning transmission electron microscopy (STEM). We report the first demonstration of multicolor CL nanobioimaging using STEM with nanophosphors. The CL spectra of the synthesized nanophosphors (Y2O3∶Eu, Y2O3∶Tb) were sufficiently narrow to be distinguished. From CL images of RE-NPs on an elastic carbon-coated copper grid, the spatial resolution was beyond the diffraction limit of light.Y2O3∶Tb and Y2O3∶Eu RE-NPs showed a remarkable resistance against electron beam exposure even at high acceleration voltage (80 kV) and retained a CL intensity of more than 97% compared with the initial intensity for 1 min. In biological CL imaging with STEM, heavy-metal-stained cell sections containing the RE-NPs were prepared,and both the CL images of RE-NPs and cellular structures, such as mitochondria, were clearly observed from STEM images with high contrast. The cellular CL imaging using RE-NPs also had high spatial resolution even though heavy-metal-stained cells are normally regarded as highly scattering media. Moreover, since theRE-NPs exhibit photoluminescence (PL) excited by UV light, they are useful for multimodal correlative imaging using CL and PL.

Laser-targeted photofabrication of gold nanoparticles inside cells.

Nanoparticle manipulation is of increasing interest, since they can report single molecule-level measurements of the cellular environment. Until now, however, intracellular nanoparticle locations have been essentially uncontrollable. Here we show that by infusing a gold ion solution, focused laser light-induced photoreduction allows in situ fabrication of gold nanoparticles at precise locations. The resulting particles are pure gold nanocrystals, distributed throughout the laser focus at sizes ranging from 2 to 20 nm, and remain in place even after removing the gold solution. We demonstrate the spatial control by scanning a laser beam to write characters in gold inside a cell. Plasmonically enhanced molecular signals could be detected from nanoparticles, allowing their use as nano-chemical probes at targeted locations inside the cell, with intracellular molecular feedback. Such light-based control of the intracellular particle generation reaction also offers avenues for in situ plasmonic device creation in organic targets, and may eventually link optical and electron microscopy.