Performance changes due to differences among annotating radiologists for training data in computerized lesion detection.

PURPOSE: The quality and bias of annotations by annotators (e.g., radiologists) affect the performance changes in computer-aided detection (CAD) software using machine learning. We hypothesized that the difference in the years of experience in image interpretation among radiologists contributes to annotation variability. In this study, we focused on how the performance of CAD software changes with retraining by incorporating cases annotated by radiologists with varying experience. METHODS: We used two types of CAD software for lung nodule detection in chest computed tomography images and cerebral aneurysm detection in magnetic resonance angiography images. Twelve radiologists with different years of experience independently annotated the lesions, and the performance changes were investigated by repeating the retraining of the CAD software twice, with the addition of cases annotated by each radiologist. Additionally, we investigated the effects of retraining using integrated annotations from multiple radiologists. RESULTS: The performance of the CAD software after retraining differed among annotating radiologists. In some cases, the performance was degraded compared to that of the initial software. Retraining using integrated annotations showed different performance trends depending on the target CAD software, notably in cerebral aneurysm detection, where the performance decreased compared to using annotations from a single radiologist. CONCLUSIONS: Although the performance of the CAD software after retraining varied among the annotating radiologists, no direct correlation with their experience was found. The performance trends differed according to the type of CAD software used when integrated annotations from multiple radiologists were used.

Detection support of lesions in patients with prostate cancer using 18 F -PSMA 1007 PET/CT.

PURPOSE: This study proposes a detection support system for primary and metastatic lesions of prostate cancer using 18 F -PSMA 1007 positron emission tomography/computed tomography (PET/CT) images with non-image information, including patient metadata and location information of an input slice image. METHODS: A convolutional neural network with condition generators and feature-wise linear modulation (FiLM) layers was employed to allow input of not only PET/CT images but also non-image information, namely, Gleason score, flag of pre- or post-prostatectomy, and normalized z-coordinate of an input slice. We explored the insertion position of the FiLM layers to optimize the conditioning of the network using non-image information. RESULTS: 18 F -PSMA 1007 PET/CT images were collected from 163 patients with prostate cancer and applied to the proposed system in a threefold cross-validation manner to evaluate the performance. The proposed system achieved a Dice score of 0.5732 (per case) and sensitivity of 0.8200 (per lesion), which are 3.87 and 4.16 points higher than the network without non-image information. CONCLUSION: This study demonstrated the effectiveness of the use of non-image information, including metadata of the patient and location information of the input slice image, in the detection of prostate cancer from 18 F -PSMA 1007 PET/CT images. Improvement in the sensitivity of inactive and small lesions remains a future challenge.

Attention induction for a CT volume classification of COVID-19.

PURPOSE: This study proposes a method to draw attention toward the specific radiological findings of coronavirus disease 2019 (COVID-19) in CT images, such as bilaterality of ground glass opacity (GGO) and/or consolidation, in order to improve the classification accuracy of input CT images. METHODS: We propose an induction mask that combines a similarity and a bilateral mask. A similarity mask guides attention to regions with similar appearances, and a bilateral mask induces attention to the opposite side of the lung to capture bilaterally distributed lesions. An induction mask for pleural effusion is also proposed in this study. ResNet18 with nonlocal blocks was trained by minimizing the loss function defined by the induction mask. RESULTS: The four-class classification accuracy of the CT images of 1504 cases was 0.6443, where class 1 was the typical appearance of COVID-19 pneumonia, class 2 was the indeterminate appearance of COVID-19 pneumonia, class 3 was the atypical appearance of COVID-19 pneumonia, and class 4 was negative for pneumonia. The four classes were divided into two subgroups. The accuracy of COVID-19 and pneumonia classifications was evaluated, which were 0.8205 and 0.8604, respectively. The accuracy of the four-class and COVID-19 classifications improved when attention was paid to pleural effusion. CONCLUSION: The proposed attention induction method was effective for the classification of CT images of COVID-19 patients. Improvement of the classification accuracy of class 3 by focusing on features specific to the class remains a topic for future work.

Classification and visual explanation for COVID-19 pneumonia from CT images using triple learning.

This study presents a novel framework for classifying and visualizing pneumonia induced by COVID-19 from CT images. Although many image classification methods using deep learning have been proposed, in the case of medical image fields, standard classification methods are unable to be used in some cases because the medical images that belong to the same category vary depending on the progression of the symptoms and the size of the inflamed area. In addition, it is essential that the models used be transparent and explainable, allowing health care providers to trust the models and avoid mistakes. In this study, we propose a classification method using contrastive learning and an attention mechanism. Contrastive learning is able to close the distance for images of the same category and generate a better feature space for classification. An attention mechanism is able to emphasize an important area in the image and visualize the location related to classification. Through experiments conducted on two-types of classification using a three-fold cross validation, we confirmed that the classification accuracy was significantly improved; in addition, a detailed visual explanation was achieved comparison with conventional methods.

Distributed sensory coding by cerebellar complex spikes in units of cortical segments.

Sensory processing is essential for motor control. Climbing fibers from the inferior olive transmit sensory signals to Purkinje cells, but how the signals are represented in the cerebellar cortex remains elusive. To examine the olivocerebellar organization of the mouse brain, we perform quantitative Ca2+ imaging to measure complex spikes (CSs) evoked by climbing fiber inputs over the entire dorsal surface of the cerebellum simultaneously. The surface is divided into approximately 200 segments, each composed of ∼100 Purkinje cells that fire CSs synchronously. Our in vivo imaging reveals that, although stimulation of four limb muscles individually elicits similar global CS responses across nearly all segments, the timing and location of a stimulus are derived by Bayesian inference from coordinated activation and inactivation of multiple segments on a single trial basis. We propose that the cerebellum performs segment-based, distributed-population coding that represents the conditional probability of sensory events.

Extraction of metastasis hotspots in a whole-body bone scintigram based on bilateral asymmetry.

PURPOSE: A hotspot of bone metastatic lesion in a whole-body bone scintigram is often observed as left-right asymmetry. The purpose of this study is to present a network to evaluate bilateral difference of a whole-body bone scintigram, and to subsequently integrate it with our previous network that extracts the hotspot from a pair of anterior and posterior images. METHODS: Input of the proposed network is a pair of scintigrams that are the original one and the flipped version with respect to body axis. The paired scintigrams are processed by a butterfly-type network (BtrflyNet). Subsequently, the output of the network is combined with the output of another BtrflyNet for a pair of anterior and posterior scintigrams by employing a convolutional layer optimized using training images. RESULTS: We evaluated the performance of the combined networks, which comprised two BtrflyNets followed by a convolutional layer for integration, in terms of accuracy of hotspot extraction using 1330 bone scintigrams of 665 patients with prostate cancer. A threefold cross-validation experiment showed that the number of false positive regions was reduced from 4.30 to 2.13 for anterior and 4.71 to 2.62 for posterior scintigrams on average compared with our previous model. CONCLUSIONS: This study presented a network for hotspot extraction of bone metastatic lesion that evaluates bilateral difference of a whole-body bone scintigram. When combining the network with the previous network that extracts the hotspot from a pair of anterior and posterior scintigrams, the false positives were reduced by nearly half compared to our previous model.

Classification of large-scale image database of various skin diseases using deep learning.

PURPOSE: The purpose of this study was to develop a deep learning-based computer-aided diagnosis system for skin disease classification using photographic images of patients. The targets are 59 skin diseases, including localized and diffuse diseases captured by photographic cameras, resulting in highly diverse images in terms of the appearance of the diseases or photographic conditions. METHODS: ResNet-18 is used as a baseline model for classification and is reinforced by metric learning to boost generalization in classification by avoiding the overfitting of the training data and increasing the reliability of CADx for dermatologists. Patient-wise classification is performed by aggregating the inference vectors of all the input patient images. RESULTS: The experiment using 70,196 images of 13,038 patients demonstrated that classification accuracy was significantly improved by both metric learning and aggregation, resulting in patient accuracies of 0.579 for Top-1, 0.793 for Top-3, and 0.863 for Top-5. The McNemar test showed that the improvements achieved by the proposed method were statistically significant. CONCLUSION: This study presents a deep learning-based classification of 59 skin diseases using multiple photographic images of a patient. The experimental results demonstrated that the proposed classification reinforced by metric learning and aggregation of multiple input images was effective in the classification of patients with diverse skin diseases and imaging conditions.

Spatiotemporal Statistical Shape Model for Temporal Shape Change Analysis of Adult Brain.

BACKGROUND: This study presents a novel method of constructing a spatiotemporal statistical shape model (st-SSM) for adult brain. St-SSM is an extension of statistical shape model (SSM) in the temporal domain which will represent the statistical variability of shape as well as the temporal change of statistical variance with respect to time. AIMS: Expectation-Maximization (EM) based weighted principal component analysis (WPCA) using a temporal weight function is applied where the eigenvalues of each data are estimated by Estep using temporal eigenvectors, and M-step updates Eigenvectors in order to maximize the variance. Both E and M-step are iterated until updating vectors reaches the convergence point. A weight parameter for each subject is allocated in accordance with the subject's age to calculate the weighted variance. A Gaussian function is utilized to define the weight function. The center of the function is a time point while the variance is a predefined parameter. METHODS: The proposed method constructs adult brain st-SSM by changing the time point between minimum to maximum age range with a small interval. Here, the eigenvectors changes with aging. The feature vector of representing adult brain shape is extracted through a level set algorithm. To validate the method, this study employed 103 adult subjects (age: 22 to 93 y.o. with Mean ± SD = 59.32±16.89) from OASIS database. st-SSM was constructed for time point 40 to 90 with a step of 2. RESULTS: We calculated the temporal deformation change between two-time points and evaluated the corresponding difference to investigate the influence of analysis parameter. An application of the proposed model is also introduced which involves Alzheimer's disease (AD) identification utilizing support vector machine. CONCLUSION: In this study, st-SSM based adult brain shape feature extraction and classification techniques are introduced to classify between normal and AD subject as an application.

Correction to: Automated measurement of bone scan index from a whole-body bone scintigram.

The article Automated measurement of bone scan index from a whole-body bone scintigram.

Automated measurement of bone scan index from a whole-body bone scintigram.

PURPOSE: We propose a deep learning-based image interpretation system for skeleton segmentation and extraction of hot spots of bone metastatic lesion from a whole-body bone scintigram followed by automated measurement of a bone scan index (BSI), which will be clinically useful. METHODS: The proposed system employs butterfly-type networks (BtrflyNets) for skeleton segmentation and extraction of hot spots of bone metastatic lesions, in which a pair of anterior and posterior images are processed simultaneously. BSI is then measured using the segmented bones and extracted hot spots. To further improve the networks, deep supervision (DSV) and residual learning technologies were introduced. RESULTS: We evaluated the performance of the proposed system using 246 bone scintigrams of prostate cancer in terms of accuracy of skeleton segmentation, hot spot extraction, and BSI measurement, as well as computational cost. In a threefold cross-validation experiment, the best performance was achieved by BtrflyNet with DSV for skeleton segmentation and BtrflyNet with residual blocks. The cross-correlation between the measured and true BSI was 0.9337, and the computational time for a case was 112.0 s. CONCLUSION: We proposed a deep learning-based BSI measurement system for a whole-body bone scintigram and proved its effectiveness by threefold cross-validation study using 246 whole-body bone scintigrams. The automatically measured BSI and computational time for a case are deemed clinically acceptable and reliable.

Detection of pancreatic tumor cell nuclei via a hyperspectral analysis of pathological slides based on stain spectra.

Hyperspectral imaging (HSI) provides more detailed information than red-green-blue (RGB) imaging, and therefore has potential applications in computer-aided pathological diagnosis. This study aimed to develop a pattern recognition method based on HSI, called hyperspectral analysis of pathological slides based on stain spectrum (HAPSS), to detect cancers in hematoxylin and eosin-stained pathological slides of pancreatic tumors. The samples, comprising hyperspectral cubes of 420-750 nm, were harvested for HSI and tissue microarray (TMA) analysis. As a result of conducting HAPSS experiments with a support vector machine (SVM) classifier, we obtained maximal accuracy of 94%, a 14% improvement over the widely used RGB images. Thus, HAPSS is a suitable method to automatically detect tumors in pathological slides of the pancreas.

Robust 3D image reconstruction of pancreatic cancer tumors from histopathological images with different stains and its quantitative performance evaluation.

PURPOSE: Histopathological imaging is widely used for the analysis and diagnosis of multiple diseases. Several methods have been proposed for the 3D reconstruction of pathological images, captured from thin sections of a given specimen, which get nonlinearly deformed due to the preparation process. The majority of the available methods for registering such images use the degree of matching of adjacent images as the criteria for registration, which can result in unnatural deformations of the anatomical structures. Moreover, most methods assume that the same staining is used for all images, when in fact multiple staining is usually applied in order to enhance different structures in the images. METHODS: This paper proposes a non-rigid 3D reconstruction method based on the assumption that internal structures on the original tissue must be smooth and continuous. Landmarks are detected along anatomical structures using template matching based on normalized cross-correlation (NCC), forming jagged shape trajectories that traverse several slices. The registration process smooths out these trajectories and deforms the images accordingly. Artifacts are automatically handled by using the confidence of the NCC in order to reject unreliable landmarks. RESULTS: The proposed method was applied to a large series of histological sections from the pancreas of a KPC mouse. Some portions were dyed primarily with HE stain, while others were dyed alternately with HE, CK19, MT and Ki67 stains. A new evaluation method is proposed to quantitatively evaluate the smoothness and isotropy of the obtained reconstructions, both for single and multiple staining. CONCLUSIONS: The experimental results show that the proposed method produces smooth and nearly isotropic 3D reconstructions of pathological images with either single or multiple stains. From these reconstructions, microanatomical structures enhanced by different stains can be simultaneously observed.

Level set distribution model of nested structures using logarithmic transformation.

In this study, we propose a method for constructing a multishape statistical shape model (SSM) for nested structures such that each is a subset or superset of another. The proposed method has potential application to any pair of shapes with an inclusive relationship. These types of shapes are often found in anatomy, such as the brain surface and ventricles. The main contribution of this paper is to introduce a new shape representation called log-transformed level set function (LT-LSF), which has a vector space structure that preserves the correct inclusive relationship of the nested shape. In addition, our method is applicable to an arbitrary number of nested shapes. We demonstrate the effectiveness of the proposed shape representation by modeling the anatomy of human embryos, including the brain, ventricles, and choroid plexus volumes. The performance of the SSM was evaluated in terms of generalization and specificity ability. Additionally, we measured leakage criteria to assess the ability to preserve inclusive relationships. A quantitative comparison of our SSM with conventional multishape SSMs demonstrates the superiority of the proposed method.

HoTPiG: a novel graph-based 3-D image feature set and its applications to computer-assisted detection of cerebral aneurysms and lung nodules.

PURPOSE: A novel image feature set named histogram of triangular paths in graph (HoTPiG) is presented. The purpose of this study is to evaluate the feasibility of the proposed HoTPiG feature set through two clinical computer-aided detection tasks: nodule detection in lung CT images and aneurysm detection in head MR angiography images. METHODS: The HoTPiG feature set is calculated from an undirected graph structure derived from a binarized volume. The features are derived from a 3-D histogram in which each bin represents a triplet of shortest path distances between the target node and all possible node pairs near the target node. First, the vessel structure is extracted from CT/MR volumes. Then, a graph structure is extracted using an 18-neighbor rule. Using this graph, a HoTPiG feature vector is calculated at every foreground voxel. After explicit feature mapping with an exponential-χ2 kernel, each voxel is judged by a linear support vector machine classifier. The proposed method was evaluated using 300 CT and 300 MR datasets. RESULTS: The proposed method successfully detected lung nodules and cerebral aneurysms. The sensitivity was about 80% when the number of false positives was three per case for both applications. CONCLUSIONS: The HoTPiG image feature set was presented, and its high general versatility was shown through two medical lesion detection applications.

Liver segmentation from low-radiation-dose pediatric computed tomography using patient-specific, statistical modeling.

PURPOSE: The pediatric computed tomography (CT) volume is acquired at a low dose because radiation is harmful to young children. Consequently, the pediatric CT volume has lower signal-to-noise ratio, which makes organ segmentation difficult. In this paper, we propose a liver segmentation algorithm for pediatric CT scan using a patient-specific level set distribution model (LSDM). METHODS: The patient-specific LSDM was constructed using a conditional LSDM (C-LSDM) conditioned on age. Furthermore, a patient-specific probabilistic atlas (PA) was generated using the model, which became a priori to the maximum a posteriori-based segmentation. The patient-specific PA generation by the C-LSDM using kernel density estimation was quicker than the conventional PA generation method using random numbers, and also, it was more accurate as it did not include any approximations. RESULTS: The liver segmentation algorithm was tested on 42 CT volumes of children aged between 2 weeks and 7 years. In the proposed method, the calculation time of the PA was about 9 s for the single Gaussian method, while it was 337 s for the conventional PA generation method using random numbers. Furthermore, using the kernel density estimation, median and 25%/75% tile of the generalized Dice similarity index between the PA and the correct liver region were found to be 0.3443 and 0.3191/0.3595. The Dice similarity index in the segmentation was 0.8821 and 0.8545/0.9085, which are higher than those obtained by the conventional method, and requires lower computational cost. CONCLUSION: We proposed a method to quickly and accurately generate a PA, combined with C-LSDM using kernel density estimation, which enabled efficient calculation and improved segmentation accuracy.

Critical Growth Processes for the Midfacial Morphogenesis in the Early Prenatal Period.

BACKGROUND: Congenital midfacial hypoplasia often requires intensive treatments and is a typical condition for the Binder phenotype and syndromic craniosynostosis. The growth trait of the midfacial skeleton during the early fetal period has been assumed to be critical for such an anomaly. However, previous embryological studies using 2-dimensional analyses and specimens during the late fetal period have not been sufficient to reveal it. OBJECTIVE: To understand the morphogenesis of the midfacial skeleton in the early fetal period via 3-dimensional quantification of the growth trait and investigation of the developmental association between the growth centers and midface. METHODS: Magnetic resonance images were obtained from 60 human fetuses during the early fetal period. Three-dimensional shape changes in the craniofacial skeleton along growth were quantified and visualized using geometric morphometrics. Subsequently, the degree of development was computed. Furthermore, the developmental association between the growth centers and the midfacial skeleton was statistically investigated and visualized. RESULTS: The zygoma expanded drastically in the anterolateral dimension, and the lateral part of the maxilla developed forward until approximately 13 weeks of gestation. The growth centers such as the nasal septum and anterior portion of the sphenoid were highly associated with the forward growth of the midfacial skeleton (RV = 0.589; P < .001). CONCLUSIONS: The development of the midface, especially of the zygoma, before 13 weeks of gestation played an essential role in the midfacial development. Moreover, the growth centers had a strong association with midfacial forward growth before birth.

Fast approximation for joint optimization of segmentation, shape, and location priors, and its application in gallbladder segmentation.

PURPOSE: This paper addresses joint optimization for segmentation and shape priors, including translation, to overcome inter-subject variability in the location of an organ. Because a simple extension of the previous exact optimization method is too computationally complex, we propose a fast approximation for optimization. The effectiveness of the proposed approximation is validated in the context of gallbladder segmentation from a non-contrast computed tomography (CT) volume. METHODS: After spatial standardization and estimation of the posterior probability of the target organ, simultaneous optimization of the segmentation, shape, and location priors is performed using a branch-and-bound method. Fast approximation is achieved by combining sampling in the eigenshape space to reduce the number of shape priors and an efficient computational technique for evaluating the lower bound. RESULTS: Performance was evaluated using threefold cross-validation of 27 CT volumes. Optimization in terms of translation of the shape prior significantly improved segmentation performance. The proposed method achieved a result of 0.623 on the Jaccard index in gallbladder segmentation, which is comparable to that of state-of-the-art methods. The computational efficiency of the algorithm is confirmed to be good enough to allow execution on a personal computer. CONCLUSIONS: Joint optimization of the segmentation, shape, and location priors was proposed, and it proved to be effective in gallbladder segmentation with high computational efficiency.