Synthesizing Audio from Tongue Motion During Speech Using Tagged MRI Via Transformer.

Investigating the relationship between internal tissue point motion of the tongue and oropharyngeal muscle deformation measured from tagged MRI and intelligible speech can aid in advancing speech motor control theories and developing novel treatment methods for speech related-disorders. However, elucidating the relationship between these two sources of information is challenging, due in part to the disparity in data structure between spatiotemporal motion fields (i.e., 4D motion fields) and one-dimensional audio waveforms. In this work, we present an efficient encoder-decoder translation network for exploring the predictive information inherent in 4D motion fields via 2D spectrograms as a surrogate of the audio data. Specifically, our encoder is based on 3D convolutional spatial modeling and transformer-based temporal modeling. The extracted features are processed by an asymmetric 2D convolution decoder to generate spectrograms that correspond to 4D motion fields. Furthermore, we incorporate a generative adversarial training approach into our framework to further improve synthesis quality on our generated spectrograms. We experiment on 63 paired motion field sequences and speech waveforms, demonstrating that our framework enables the generation of clear audio waveforms from a sequence of motion fields. Thus, our framework has the potential to improve our understanding of the relationship between these two modalities and inform the development of treatments for speech disorders.

Evaluation of trans- and cis-4-[18F]fluorogabapentin for brain PET imaging.

Gabapentin, a selective ligand for the α2δ subunit of voltage-dependent calcium channels, is an anticonvulsant medication used in the treatment of neuropathic pain, epilepsy and other neurological conditions. We recently described two radiofluorinated derivatives of gabapentin (trans-4-[18F]fluorogabapentin, [18F]tGBP4F, and cis-4-[18F]fluorogabapentin, [18F]cGBP4F) and showed that these compounds accumulate in the injured nerves in a rodent model of neuropathic pain. Given the use of gabapentin in brain diseases, here we investigate whether these radiofluorinated derivatives of gabapentin can be used for imaging α2δ receptors in the brain. Specifically, we developed automated radiosynthesis methods for [18F]tGBP4F and [18F]cGBP4F and conducted dynamic PET imaging in adult rhesus macaques with and without preadministration of pharmacological doses of gabapentin. Both radiotracers showed very high metabolic stability, negligible plasma protein binding and slow accumulation in the brain. [18F]tGBP4F, the isomer with higher binding affinity, showed low brain uptake and could not be displaced whereas [18F]cGBP4F showed moderate brain uptake and could be partially displaced. Kinetic modeling of brain regional time-activity curves using a metabolite-corrected arterial input function shows that a 1-tissue compartment model accurately fits the data. Graphical analysis using Logan or multilinear analysis 1 produced similar results as compartmental modeling indicating robust quantification. This study advances our understanding of how gabapentinoids work and provides an important advancement towards imaging α2δ receptors in the brain.

Radiosynthesis automation, non-human primate biodistribution and dosimetry of K + channel tracer [ 11 C]3MeO4AP.

Purpose: 4-Aminopyridine (4AP) is a medication for the symptomatic treatment of multiple sclerosis. Several 4AP-based PET tracers have been developed for imaging demyelination. In preclinical studies, [ 11 C]3MeO4AP has shown promise due to its high brain permeability, high metabolic stability, high plasma availability, and high in vivo binding affinity. To prepare for the translation to human studies, we developed a cGMP-compliant automated radiosynthesis protocol and evaluated the whole-body biodistribution and radiation dosimetry of [ 11 C]3MeO4AP in non-human primates (NHPs). Methods: Automated radiosynthesis was carried out using a GE TRACERlab FX-C Pro synthesis module. One male and one female adult rhesus macaques were used in the study. A high-resolution CT from cranial vertex to knee was acquired. PET data were collected using a dynamic acquisition protocol with 4 bed positions and 13 passes over a total scan time of ∼150 minutes. Based on the CT and PET images, volumes of interest (VOIs) were manually drawn for selected organs. Non-decay corrected time-activity curves (TACs) were extracted for each VOI. Radiation dosimetry and effective dose were calculated from the integrated TACs using OLINDA software. Results: Fully automated radiosynthesis of [ 11 C]3MeO4AP was achieved with 7.3 ± 1.2 % (n = 4) of non-decay corrected radiochemical yield within 38 min of synthesis and purification time. [ 11 C]3MeO4AP distributed quickly throughout the body and into the brain. The organs with highest dose were the kidneys. The average effective dose of [ 11 C]3MeO4AP was 4.27 ± 0.57 µSv/MBq. No significant changes in vital signs were observed during the scan. Conclusion: The cGMP compliant automated radiosynthesis of [ 11 C]3MeO4AP was developed. The whole-body biodistribution and radiation dosimetry of [ 11 C]3MeO4AP was successfully evaluated in NHPs. [ 11 C]3MeO4AP shows lower average effective dose than [ 18 F]3F4AP and similar average effective dose as other carbon-11 tracers.

Posterior estimation using deep learning: a simulation study of compartmental modeling in dynamic positron emission tomography.

BACKGROUND: In medical imaging, images are usually treated as deterministic, while their uncertainties are largely underexplored. PURPOSE: This work aims at using deep learning to efficiently estimate posterior distributions of imaging parameters, which in turn can be used to derive the most probable parameters as well as their uncertainties. METHODS: Our deep learning-based approaches are based on a variational Bayesian inference framework, which is implemented using two different deep neural networks based on conditional variational auto-encoder (CVAE), CVAE-dual-encoder, and CVAE-dual-decoder. The conventional CVAE framework, that is, CVAE-vanilla, can be regarded as a simplified case of these two neural networks. We applied these approaches to a simulation study of dynamic brain PET imaging using a reference region-based kinetic model. RESULTS: In the simulation study, we estimated posterior distributions of PET kinetic parameters given a measurement of the time-activity curve. Our proposed CVAE-dual-encoder and CVAE-dual-decoder yield results that are in good agreement with the asymptotically unbiased posterior distributions sampled by Markov Chain Monte Carlo (MCMC). The CVAE-vanilla can also be used for estimating posterior distributions, although it has an inferior performance to both CVAE-dual-encoder and CVAE-dual-decoder. CONCLUSIONS: We have evaluated the performance of our deep learning approaches for estimating posterior distributions in dynamic brain PET. Our deep learning approaches yield posterior distributions, which are in good agreement with unbiased distributions estimated by MCMC. All these neural networks have different characteristics and can be chosen by the user for specific applications. The proposed methods are general and can be adapted to other problems.

Joint spectral quantification of MR spectroscopic imaging using linear tangent space alignment-based manifold learning.

PURPOSE: To develop a manifold learning-based method that leverages the intrinsic low-dimensional structure of MR Spectroscopic Imaging (MRSI) signals for joint spectral quantification. METHODS: A linear tangent space alignment (LTSA) model was proposed to represent MRSI signals. In the proposed model, the signals of each metabolite were represented using a subspace model and the local coordinates of the subspaces were aligned to the global coordinates of the underlying low-dimensional manifold via linear transform. With the basis functions of the subspaces predetermined via quantum mechanics simulations, the global coordinates and the matrices for the local-to-global coordinate alignment were estimated by fitting the proposed LTSA model to noisy MRSI data with a spatial smoothness constraint on the global coordinates and a sparsity constraint on the matrices. RESULTS: The performance of the proposed method was validated using numerical simulation data and in vivo proton-MRSI experimental data acquired on healthy volunteers at 3T. The results of the proposed method were compared with the QUEST method and the subspace-based method. In all the compared cases, the proposed method achieved superior performance over the QUEST and the subspace-based methods both qualitatively in terms of noise and artifacts in the estimated metabolite concentration maps, and quantitatively in terms of spectral quantification accuracy measured by normalized root mean square errors. CONCLUSION: Joint spectral quantification using linear tangent space alignment-based manifold learning improves the accuracy of MRSI spectral quantification.

Kinetic evaluation and assessment of longitudinal changes in reference region and extracerebral [18F]MK-6240 PET uptake.

[18F]MK-6240 meningeal/extracerebral off-target binding may impact tau quantification. We examined the kinetics and longitudinal changes of extracerebral and reference regions. [18F]MK-6240 PET was performed in 24 cognitively-normal and eight cognitively-impaired subjects, with arterial samples in 13 subjects. Follow-up scans at 6.1 ± 0.5 (n = 25) and 13.3 ± 0.9 (n = 16) months were acquired. Extracerebral and reference region (cerebellar gray matter (CerGM)-based, cerebral white matter (WM), pons) uptake were evaluated using standardized uptake values (SUV90-110), spectral analysis, and distribution volume. Longitudinal changes in SUV90-110 were examined. The impact of reference region on target region outcomes, partial volume correction (PVC) and regional erosion were evaluated. Eroded WM and pons showed lower variability, lower extracerebral contamination, and lower longitudinal changes than CerGM-based regions. CerGM-based regions resulted larger cross-sectional effect sizes for group differentiation. Extracerebral signal was high in 50% of subjects and exhibited irreversible kinetics and nonsignificant longitudinal changes over one-year but was highly variable at subject-level. PVC resulted in higher variability in reference region uptake and longitudinal changes. Our results suggest that eroded CerGM may be preferred for cross-sectional, whilst eroded WM or pons may be preferred for longitudinal [18F]MK-6240 studies. For CerGM, erosion was necessary (preferred over PVC) to address the heterogenous nature of extracerebral signal.

Noise-Robust Sleep Staging via Adversarial Training With an Auxiliary Model.

Deep learning (DL)-based automatic sleep staging approaches have attracted much attention recently due in part to their outstanding accuracy. At the testing stage, however, the performance of these approaches is likely to be degraded, when applied in different testing environments, because of the problem of domain shift. This is because while a pre-trained model is typically trained on noise-free electroencephalogram (EEG) signals acquired from accurate medical equipment, deployment is carried out on consumer-level devices with undesirable noise. To alleviate this challenge, in this work, we propose an efficient training approach that is robust against unseen arbitrary noise. In particular, we propose to generate the worst-case input perturbations by means of adversarial transformation in an auxiliary model, to learn a wide range of input perturbations and thereby to improve reliability. Our approach is based on two separate training models: (i) an auxiliary model to generate adversarial noise and (ii) a target network to incorporate the noise signal to enhance robustness. Furthermore, we exploit novel class-wise robustness during the training of the target network to represent different robustness patterns of each sleep stage. Our experimental results demonstrated that our approach improved sleep staging performance on healthy controls, in the presence of moderate to severe noise levels, compared with competing methods. Our approach was able to effectively train and deploy a DL model to handle different types of noise, including adversarial, Gaussian, and shot noise.

PET imaging studies to investigate functional expression of mGluR2 using [11C]mG2P001.

Metabotropic glutamate receptor 2 (mGluR2) has been extensively studied for the treatment of various neurological and psychiatric disorders. Understanding of the mGluR2 function is pivotal in supporting the drug discovery targeting mGluR2. Herein, the positive allosteric modulation of mGluR2 was investigated via the in vivo positron emission tomography (PET) imaging using 2-((4-(2-[11C]methoxy-4-(trifluoromethyl)phenyl)piperidin-1-yl)methyl)-1-methyl-1H-imidazo[4,5-b]pyridine ([11C]mG2P001). Distinct from the orthosteric compounds, pretreatment with the unlabeled mG2P001, a potent mGluR2 positive allosteric modulator (PAM), resulted in a significant increase instead of decrease of the [11C]mG2P001 accumulation in rat brain detected by PET imaging. Subsequent in vitro studies with [3H]mG2P001 revealed the cooperative binding mechanism of mG2P001 with glutamate and its pharmacological effect that contributed to the enhanced binding of [3H]mG2P001 in transfected CHO cells expressing mGluR2. The in vivo PET imaging and quantitative analysis of [11C]mG2P001 in non-human primates (NHPs) further validated the characteristics of [11C]mG2P001 as an imaging ligand for mGluR2. Self-blocking studies in primates enhanced accumulation of [11C]mG2P001. Altogether, these studies show that [11C]mG2P001 is a sensitive biomarker for mGluR2 expression and the binding is affected by the tissue glutamate concentration.

Manifold Learning via Linear Tangent Space Alignment (LTSA) for Accelerated Dynamic MRI With Sparse Sampling.

The spatial resolution and temporal frame-rate of dynamic magnetic resonance imaging (MRI) can be improved by reconstructing images from sparsely sampled k -space data with mathematical modeling of the underlying spatiotemporal signals. These models include sparsity models, linear subspace models, and non-linear manifold models. This work presents a novel linear tangent space alignment (LTSA) model-based framework that exploits the intrinsic low-dimensional manifold structure of dynamic images for accelerated dynamic MRI. The performance of the proposed method was evaluated and compared to state-of-the-art methods using numerical simulation studies as well as 2D and 3D in vivo cardiac imaging experiments. The proposed method achieved the best performance in image reconstruction among all the compared methods. The proposed method could prove useful for accelerating many MRI applications, including dynamic MRI, multi-parametric MRI, and MR spectroscopic imaging.

Automation of generative adversarial network-based synthetic data-augmentation for maximizing the diagnostic performance with paranasal imaging.

Thus far, there have been no reported specific rules for systematically determining the appropriate augmented sample size to optimize model performance when conducting data augmentation. In this paper, we report on the feasibility of synthetic data augmentation using generative adversarial networks (GAN) by proposing an automation pipeline to find the optimal multiple of data augmentation to achieve the best deep learning-based diagnostic performance in a limited dataset. We used Waters' view radiographs for patients diagnosed with chronic sinusitis to demonstrate the method developed herein. We demonstrate that our approach produces significantly better diagnostic performance parameters than models trained using conventional data augmentation. The deep learning method proposed in this study could be implemented to assist radiologists in improving their diagnosis. Researchers and industry workers could overcome the lack of training data by employing our proposed automation pipeline approach in GAN-based synthetic data augmentation. This is anticipated to provide new means to overcome the shortage of graphic data for algorithm training.

Variational Inference for Quantifying Inter-observer Variability in Segmentation of Anatomical Structures.

Lesions or organ boundaries visible through medical imaging data are often ambiguous, thus resulting in significant variations in multi-reader delineations, i.e., the source of aleatoric uncertainty. In particular, quantifying the inter-observer variability of manual annotations with Magnetic Resonance (MR) Imaging data plays a crucial role in establishing a reference standard for various diagnosis and treatment tasks. Most segmentation methods, however, simply model a mapping from an image to its single segmentation map and do not take the disagreement of annotators into consideration. In order to account for inter-observer variability, without sacrificing accuracy, we propose a novel variational inference framework to model the distribution of plausible segmentation maps, given a specific MR image, which explicitly represents the multi-reader variability. Specifically, we resort to a latent vector to encode the multi-reader variability and counteract the inherent information loss in the imaging data. Then, we apply a variational autoencoder network and optimize its evidence lower bound (ELBO) to efficiently approximate the distribution of the segmentation map, given an MR image. Experimental results, carried out with the QUBIQ brain growth MRI segmentation datasets with seven annotators, demonstrate the effectiveness of our approach.

SELF-SEMANTIC CONTOUR ADAPTATION FOR CROSS MODALITY BRAIN TUMOR SEGMENTATION.

Unsupervised domain adaptation (UDA) between two significantly disparate domains to learn high-level semantic alignment is a crucial yet challenging task. To this end, in this work, we propose exploiting low-level edge information to facilitate the adaptation as a precursor task, which has a small cross-domain gap, compared with semantic segmentation. The precise contour then provides spatial information to guide the semantic adaptation. More specifically, we propose a multi-task framework to learn a contouring adaptation network along with a semantic segmentation adaptation network, which takes both magnetic resonance imaging (MRI) slice and its initial edge map as input. These two networks are jointly trained with source domain labels, and the feature and edge map level adversarial learning is carried out for cross-domain alignment. In addition, self-entropy minimization is incorporated to further enhance segmentation performance. We evaluated our framework on the BraTS2018 database for cross-modality segmentation of brain tumors, showing the validity and superiority of our approach, compared with competing methods.

Subtype-Aware Dynamic Unsupervised Domain Adaptation.

Unsupervised domain adaptation (UDA) has been successfully applied to transfer knowledge from a labeled source domain to target domains without their labels. Recently introduced transferable prototypical networks (TPNs) further address class-wise conditional alignment. In TPN, while the closeness of class centers between source and target domains is explicitly enforced in a latent space, the underlying fine-grained subtype structure and the cross-domain within-class compactness have not been fully investigated. To counter this, we propose a new approach to adaptively perform a fine-grained subtype-aware alignment to improve the performance in the target domain without the subtype label in both domains. The insight of our approach is that the unlabeled subtypes in a class have the local proximity within a subtype while exhibiting disparate characteristics because of different conditional and label shifts. Specifically, we propose to simultaneously enforce subtype-wise compactness and class-wise separation, by utilizing intermediate pseudo-labels. In addition, we systematically investigate various scenarios with and without prior knowledge of subtype numbers and propose to exploit the underlying subtype structure. Furthermore, a dynamic queue framework is developed to evolve the subtype cluster centroids steadily using an alternative processing scheme. Experimental results, carried out with multiview congenital heart disease data and VisDA and DomainNet, show the effectiveness and validity of our subtype-aware UDA, compared with state-of-the-art UDA methods.

Brain MR Atlas Construction Using Symmetric Deep Neural Inpainting.

Modeling statistical properties of anatomical structures using magnetic resonance imaging is essential for revealing common information of a target population and unique properties of specific subjects. In brain imaging, a statistical brain atlas is often constructed using a number of healthy subjects. When tumors are present, however, it is difficult to either provide a common space for various subjects or align their imaging data due to the unpredictable distribution of lesions. Here we propose a deep learning-based image inpainting method to replace the tumor regions with normal tissue intensities using only a patient population. Our framework has three major innovations: 1) incompletely distributed datasets with random tumor locations can be used for training; 2) irregularly-shaped tumor regions are properly learned, identified, and corrected; and 3) a symmetry constraint between the two brain hemispheres is applied to regularize inpainted regions. Henceforth, regular atlas construction and image registration methods can be applied using inpainted data to obtain tissue deformation, thereby achieving group-specific statistical atlases and patient-to-atlas registration. Our framework was tested using the public database from the Multimodal Brain Tumor Segmentation challenge. Results showed increased similarity scores as well as reduced reconstruction errors compared with three existing image inpainting methods. Patient-to-atlas registration also yielded better results with improved normalized cross-correlation and mutual information and a reduced amount of deformation over the tumor regions.

Tagged-MRI Sequence to Audio Synthesis via Self Residual Attention Guided Heterogeneous Translator.

Understanding the underlying relationship between tongue and oropharyngeal muscle deformation seen in tagged-MRI and intelligible speech plays an important role in advancing speech motor control theories and treatment of speech related-disorders. Because of their heterogeneous representations, however, direct mapping between the two modalities-i.e., two-dimensional (mid-sagittal slice) plus time tagged-MRI sequence and its corresponding one-dimensional waveform-is not straightforward. Instead, we resort to two-dimensional spectrograms as an intermediate representation, which contains both pitch and resonance, from which to develop an end-to-end deep learning framework to translate from a sequence of tagged-MRI to its corresponding audio waveform with limited dataset size. Our framework is based on a novel fully convolutional asymmetry translator with guidance of a self residual attention strategy to specifically exploit the moving muscular structures during speech. In addition, we leverage a pairwise correlation of the samples with the same utterances with a latent space representation disentanglement strategy. Furthermore, we incorporate an adversarial training approach with generative adversarial networks to offer improved realism on our generated spectrograms. Our experimental results, carried out with a total of 63 tagged-MRI sequences alongside speech acoustics, showed that our framework enabled the generation of clear audio waveforms from a sequence of tagged-MRI, surpassing competing methods. Thus, our framework provides the great potential to help better understand the relationship between the two modalities.

VoxelHop: Successive Subspace Learning for ALS Disease Classification Using Structural MRI.

Deep learning has great potential for accurate detection and classification of diseases with medical imaging data, but the performance is often limited by the number of training datasets and memory requirements. In addition, many deep learning models are considered a "black-box," thereby often limiting their adoption in clinical applications. To address this, we present a successive subspace learning model, termed VoxelHop, for accurate classification of Amyotrophic Lateral Sclerosis (ALS) using T2-weighted structural MRI data. Compared with popular convolutional neural network (CNN) architectures, VoxelHop has modular and transparent structures with fewer parameters without any backpropagation, so it is well-suited to small dataset size and 3D imaging data. Our VoxelHop has four key components, including (1) sequential expansion of near-to-far neighborhood for multi-channel 3D data; (2) subspace approximation for unsupervised dimension reduction; (3) label-assisted regression for supervised dimension reduction; and (4) concatenation of features and classification between controls and patients. Our experimental results demonstrate that our framework using a total of 20 controls and 26 patients achieves an accuracy of 93.48 % and an AUC score of 0.9394 in differentiating patients from controls, even with a relatively small number of datasets, showing its robustness and effectiveness. Our thorough evaluations also show its validity and superiority to the state-of-the-art 3D CNN classification approaches. Our framework can easily be generalized to other classification tasks using different imaging modalities.

Adapting Off-the-Shelf Source Segmenter for Target Medical Image Segmentation.

Unsupervised domain adaptation (UDA) aims to transfer knowledge learned from a labeled source domain to an unlabeled and unseen target domain, which is usually trained on data from both domains. Access to the source domain data at the adaptation stage, however, is often limited, due to data storage or privacy issues. To alleviate this, in this work, we target source free UDA for segmentation, and propose to adapt an "off-the-shelf" segmentation model pre-trained in the source domain to the target domain, with an adaptive batch-wise normalization statistics adaptation framework. Specifically, the domain-specific low-order batch statistics, i.e., mean and variance, are gradually adapted with an exponential momentum decay scheme, while the consistency of domain shareable high-order batch statistics, i.e., scaling and shifting parameters, is explicitly enforced by our optimization objective. The transferability of each channel is adaptively measured first from which to balance the contribution of each channel. Moreover, the proposed source free UDA framework is orthogonal to unsupervised learning methods, e.g., self-entropy minimization, which can thus be simply added on top of our framework. Extensive experiments on the BraTS 2018 database show that our source free UDA framework outperformed existing source-relaxed UDA methods for the cross-subtype UDA segmentation task and yielded comparable results for the cross-modality UDA segmentation task, compared with a supervised UDA methods with the source data.

Generative Self-training for Cross-domain Unsupervised Tagged-to-Cine MRI Synthesis.

Self-training based unsupervised domain adaptation (UDA) has shown great potential to address the problem of domain shift, when applying a trained deep learning model in a source domain to unlabeled target domains. However, while the self-training UDA has demonstrated its effectiveness on discriminative tasks, such as classification and segmentation, via the reliable pseudo-label selection based on the softmax discrete histogram, the self-training UDA for generative tasks, such as image synthesis, is not fully investigated. In this work, we propose a novel generative self-training (GST) UDA framework with continuous value prediction and regression objective for cross-domain image synthesis. Specifically, we propose to filter the pseudo-label with an uncertainty mask, and quantify the predictive confidence of generated images with practical variational Bayes learning. The fast test-time adaptation is achieved by a round-based alternative optimization scheme. We validated our framework on the tagged-to-cine magnetic resonance imaging (MRI) synthesis problem, where datasets in the source and target domains were acquired from different scanners or centers. Extensive validations were carried out to verify our framework against popular adversarial training UDA methods. Results show that our GST, with tagged MRI of test subjects in new target domains, improved the synthesis quality by a large margin, compared with the adversarial training UDA methods.