Ultra high frequency ultrasound enables real-time visualization of blood supply from chorioallantoic membrane to human autosomal dominant polycystic kidney tissue.

Ultra high frequency (UHF) ultrasound enables the visualization of very small structures that cannot be detected by conventional ultrasound. The utilization of UHF imaging as a new imaging technique for the 3D-in-vivo chorioallantoic membrane (CAM) model can facilitate new insights into tissue perfusion and survival. Therefore, human renal cystic tissue was grafted onto the CAM and examined using UHF ultrasound imaging. Due to the unprecedented resolution of UHF ultrasound, it was possible to visualize microvessels, their development, and the formation of anastomoses. This enabled the observation of anastomoses between human and chicken vessels only 12 h after transplantation. These observations were validated by 3D reconstructions from a light sheet microscopy image stack, indocyanine green angiography, and histological analysis. Contrary to the assumption that the nutrient supply of the human cystic tissue and the gas exchange happens through diffusion from CAM vessels, this study shows that the vasculature of the human cystic tissue is directly connected to the blood vessels of the CAM and perfusion is established within a short period. Therefore, this in-vivo model combined with UHF imaging appears to be the ideal platform for studying the effects of intravenously applied therapeutics to inhibit renal cyst growth.

Modeling the Temperature-Dependent Size Change of Polydisperse Nano-objects using a Deep Generative Model.

Modern microscopy techniques can be used to investigate soft nano-objects at the nanometer scale. However, time-consuming microscopy measurements combined with low numbers of observable polydisperse objects often limit the statistics. We propose a method for identifying the most representative objects from their respective point clouds. These point cloud data are obtained, for example, through the localization of single emitters in super-resolution fluorescence microscopy. External stimuli, such as temperature, can cause changes in the shape and properties of adaptive objects. Due to the demanding and time-consuming nature of super-resolution microscopy experiments, only a limited number of temperature steps can be performed. Therefore, we propose a deep generative model that learns the underlying point distribution of temperature-dependent microgels, enabling the reliable generation of unlimited samples with an arbitrary number of localizations. Our method greatly cuts down the data collection effort across diverse experimental conditions, proving invaluable for soft condensed matter studies.

A revised conceptual framework for mouse vomeronasal pumping and stimulus sampling.

The physiological performance of any sensory organ is determined by its anatomy and physical properties. Consequently, complex sensory structures with elaborate features have evolved to optimize stimulus detection. Understanding these structures and their physical nature forms the basis for mechanistic insights into sensory function. Despite its crucial role as a sensor for pheromones and other behaviorally instructive chemical cues, the vomeronasal organ (VNO) remains a poorly characterized mammalian sensory structure. Fundamental principles of its physico-mechanical function, including basic aspects of stimulus sampling, remain poorly explored. Here, we revisit the classical vasomotor pump hypothesis of vomeronasal stimulus uptake. Using advanced anatomical, histological, and physiological methods, we demonstrate that large parts of the lateral mouse VNO are composed of smooth muscle. Vomeronasal smooth muscle tissue comprises two subsets of fibers with distinct topography, structure, excitation-contraction coupling, and, ultimately, contractile properties. Specifically, contractions of a large population of noradrenaline-sensitive cells mediate both transverse and longitudinal lumen expansion, whereas cholinergic stimulation targets an adluminal group of smooth muscle fibers. The latter run parallel to the VNO's rostro-caudal axis and are ideally situated to mediate antagonistic longitudinal constriction of the lumen. This newly discovered arrangement implies a novel mode of function. Single-cell transcriptomics and pharmacological profiling reveal the receptor subtypes involved. Finally, 2D/3D tomography provides non-invasive insight into the intact VNO's anatomy and mechanics, enables measurement of luminal fluid volume, and allows an assessment of relative volume change upon noradrenergic stimulation. Together, we propose a revised conceptual framework for mouse vomeronasal pumping and, thus, stimulus sampling.

Advances in medical image analysis with vision Transformers: A comprehensive review.

The remarkable performance of the Transformer architecture in natural language processing has recently also triggered broad interest in Computer Vision. Among other merits, Transformers are witnessed as capable of learning long-range dependencies and spatial correlations, which is a clear advantage over convolutional neural networks (CNNs), which have been the de facto standard in Computer Vision problems so far. Thus, Transformers have become an integral part of modern medical image analysis. In this review, we provide an encyclopedic review of the applications of Transformers in medical imaging. Specifically, we present a systematic and thorough review of relevant recent Transformer literature for different medical image analysis tasks, including classification, segmentation, detection, registration, synthesis, and clinical report generation. For each of these applications, we investigate the novelty, strengths and weaknesses of the different proposed strategies and develop taxonomies highlighting key properties and contributions. Further, if applicable, we outline current benchmarks on different datasets. Finally, we summarize key challenges and discuss different future research directions. In addition, we have provided cited papers with their corresponding implementations in https://github.com/mindflow-institue/Awesome-Transformer.

A Deep Learning-based in silico Framework for Optimization on Retinal Prosthetic Stimulation.

We propose a neural network-based framework to optimize the perceptions simulated by the in silico retinal implant model pulse2percept. The overall pipeline consists of a trainable encoder, a pre-trained retinal implant model and a pre-trained evaluator. The encoder is a U-Net, which takes the original image and outputs the stimulus. The pre-trained retinal implant model is also a U-Net, which is trained to mimic the biomimetic perceptual model implemented in pulse2percept. The evaluator is a shallow VGG classifier, which is trained with original images. Based on 10,000 test images from the MNIST dataset, we show that the convolutional neural network-based encoder performs significantly better than the trivial downsampling approach, yielding a boost in the weighted F1-Score by 36.17% in the pre-trained classifier with 6×10 electrodes. With this fully neural network-based encoder, the quality of the downstream perceptions can be fine-tuned using gradient descent in an end-to-end fashion.

Diffusion MRI anomaly detection in glioma patients.

Diffusion-MRI (dMRI) measures molecular diffusion, which allows to characterize microstructural properties of the human brain. Gliomas strongly alter these microstructural properties. Delineation of brain tumors currently mainly relies on conventional MRI-techniques, which are, however, known to underestimate tumor volumes in diffusely infiltrating glioma. We hypothesized that dMRI is well suited for tumor delineation, and developed two different deep-learning approaches. The first diffusion-anomaly detection architecture is a denoising autoencoder, the second consists of a reconstruction and a discrimination network. Each model was exclusively trained on non-annotated dMRI of healthy subjects, and then applied on glioma patients' data. To validate these models, a state-of-the-art supervised tumor segmentation network was modified to generate groundtruth tumor volumes based on structural MRI. Compared to groundtruth segmentations, a dice score of 0.67 ± 0.2 was obtained. Further inspecting mismatches between diffusion-anomalous regions and groundtruth segmentations revealed, that these colocalized with lesions delineated only later on in structural MRI follow-up data, which were not visible at the initial time of recording. Anomaly-detection methods are suitable for tumor delineation in dMRI acquisitions, and may further enhance brain-imaging analysis by detection of occult tumor infiltration in glioma patients, which could improve prognostication of disease evolution and tumor treatment strategies.

Towards substitution of invasive telemetry: An integrated home cage concept for unobtrusive monitoring of objective physiological parameters in rodents.

This study presents a novel concept for a smart home cage design, tools, and software used to monitor the physiological parameters of mice and rats in animal-based experiments. The proposed system focuses on monitoring key clinical parameters, including heart rate, respiratory rate, and body temperature, and can also assess activity and circadian rhythm. As the basis of the smart home cage system, an in-depth analysis of the requirements was performed, including camera positioning, imaging system types, resolution, frame rates, external illumination, video acquisition, data storage, and synchronization. Two different camera perspectives were considered, and specific camera models, including two near-infrared and two thermal cameras, were selected to meet the requirements. The developed specifications, hardware models, and software are freely available via GitHub. During the first testing phase, the system demonstrated the potential of extracting vital parameters such as respiratory and heart rate. This technology has the potential to reduce the need for implantable sensors while providing reliable and accurate physiological data, leading to refinement and improvement in laboratory animal care.

Validation of deep learning techniques for quality augmentation in diffusion MRI for clinical studies.

The objective of this study is to evaluate the efficacy of deep learning (DL) techniques in improving the quality of diffusion MRI (dMRI) data in clinical applications. The study aims to determine whether the use of artificial intelligence (AI) methods in medical images may result in the loss of critical clinical information and/or the appearance of false information. To assess this, the focus was on the angular resolution of dMRI and a clinical trial was conducted on migraine, specifically between episodic and chronic migraine patients. The number of gradient directions had an impact on white matter analysis results, with statistically significant differences between groups being drastically reduced when using 21 gradient directions instead of the original 61. Fourteen teams from different institutions were tasked to use DL to enhance three diffusion metrics (FA, AD and MD) calculated from data acquired with 21 gradient directions and a b-value of 1000 s/mm2. The goal was to produce results that were comparable to those calculated from 61 gradient directions. The results were evaluated using both standard image quality metrics and Tract-Based Spatial Statistics (TBSS) to compare episodic and chronic migraine patients. The study results suggest that while most DL techniques improved the ability to detect statistical differences between groups, they also led to an increase in false positive. The results showed that there was a constant growth rate of false positives linearly proportional to the new true positives, which highlights the risk of generalization of AI-based tasks when assessing diverse clinical cohorts and training using data from a single group. The methods also showed divergent performance when replicating the original distribution of the data and some exhibited significant bias. In conclusion, extreme caution should be exercised when using AI methods for harmonization or synthesis in clinical studies when processing heterogeneous data in clinical studies, as important information may be altered, even when global metrics such as structural similarity or peak signal-to-noise ratio appear to suggest otherwise.

Diffusion models in medical imaging: A comprehensive survey.

Denoising diffusion models, a class of generative models, have garnered immense interest lately in various deep-learning problems. A diffusion probabilistic model defines a forward diffusion stage where the input data is gradually perturbed over several steps by adding Gaussian noise and then learns to reverse the diffusion process to retrieve the desired noise-free data from noisy data samples. Diffusion models are widely appreciated for their strong mode coverage and quality of the generated samples in spite of their known computational burdens. Capitalizing on the advances in computer vision, the field of medical imaging has also observed a growing interest in diffusion models. With the aim of helping the researcher navigate this profusion, this survey intends to provide a comprehensive overview of diffusion models in the discipline of medical imaging. Specifically, we start with an introduction to the solid theoretical foundation and fundamental concepts behind diffusion models and the three generic diffusion modeling frameworks, namely, diffusion probabilistic models, noise-conditioned score networks, and stochastic differential equations. Then, we provide a systematic taxonomy of diffusion models in the medical domain and propose a multi-perspective categorization based on their application, imaging modality, organ of interest, and algorithms. To this end, we cover extensive applications of diffusion models in the medical domain, including image-to-image translation, reconstruction, registration, classification, segmentation, denoising, 2/3D generation, anomaly detection, and other medically-related challenges. Furthermore, we emphasize the practical use case of some selected approaches, and then we discuss the limitations of the diffusion models in the medical domain and propose several directions to fulfill the demands of this field. Finally, we gather the overviewed studies with their available open-source implementations at our GitHub.1 We aim to update the relevant latest papers within it regularly.

M cell maturation and cDC activation determine the onset of adaptive immune priming in the neonatal Peyer's patch.

Early-life immune development is critical to long-term host health. However, the mechanisms that determine the pace of postnatal immune maturation are not fully resolved. Here, we analyzed mononuclear phagocytes (MNPs) in small intestinal Peyer's patches (PPs), the primary inductive site of intestinal immunity. Conventional type 1 and 2 dendritic cells (cDC1 and cDC2) and RORgt+ antigen-presenting cells (RORgt+ APC) exhibited significant age-dependent changes in subset composition, tissue distribution, and reduced cell maturation, subsequently resulting in a lack in CD4+ T cell priming during the postnatal period. Microbial cues contributed but could not fully explain the discrepancies in MNP maturation. Type I interferon (IFN) accelerated MNP maturation but IFN signaling did not represent the physiological stimulus. Instead, follicle-associated epithelium (FAE) M cell differentiation was required and sufficient to drive postweaning PP MNP maturation. Together, our results highlight the role of FAE M cell differentiation and MNP maturation in postnatal immune development.

Unsupervised many-to-many stain translation for histological image augmentation to improve classification accuracy.

Background: Deep learning tasks, which require large numbers of images, are widely applied in digital pathology. This poses challenges especially for supervised tasks since manual image annotation is an expensive and laborious process. This situation deteriorates even more in the case of a large variability of images. Coping with this problem requires methods such as image augmentation and synthetic image generation. In this regard, unsupervised stain translation via GANs has gained much attention recently, but a separate network must be trained for each pair of source and target domains. This work enables unsupervised many-to-many translation of histopathological stains with a single network while seeking to maintain the shape and structure of the tissues. Methods: StarGAN-v2 is adapted for unsupervised many-to-many stain translation of histopathology images of breast tissues. An edge detector is incorporated to motivate the network to maintain the shape and structure of the tissues and to have an edge-preserving translation. Additionally, a subjective test is conducted on medical and technical experts in the field of digital pathology to evaluate the quality of generated images and to verify that they are indistinguishable from real images. As a proof of concept, breast cancer classifiers are trained with and without the generated images to quantify the effect of image augmentation using the synthetized images on classification accuracy. Results: The results show that adding an edge detector helps to improve the quality of translated images and to preserve the general structure of tissues. Quality control and subjective tests on our medical and technical experts show that the real and artificial images cannot be distinguished, thereby confirming that the synthetic images are technically plausible. Moreover, this research shows that, by augmenting the training dataset with the outputs of the proposed stain translation method, the accuracy of breast cancer classifier with ResNet-50 and VGG-16 improves by 8.0% and 9.3%, respectively. Conclusions: This research indicates that a translation from an arbitrary source stain to other stains can be performed effectively within the proposed framework. The generated images are realistic and could be employed to train deep neural networks to improve their performance and cope with the problem of insufficient numbers of annotated images.

SegPC-2021: A challenge & dataset on segmentation of Multiple Myeloma plasma cells from microscopic images.

Multiple Myeloma (MM) is an emerging ailment of global concern. Its diagnosis at the early stages is critical for recovery. Therefore, efforts are underway to produce digital pathology tools with human-level intelligence that are efficient, scalable, accessible, and cost-effective. Following the trend, a medical imaging challenge on "Segmentation of Multiple Myeloma Plasma Cells in Microscopic Images (SegPC-2021)" was organized at the IEEE International Symposium on Biomedical Imaging (ISBI), 2021, France. The challenge addressed the problem of cell segmentation in microscopic images captured from the slides prepared from the bone marrow aspirate of patients diagnosed with Multiple Myeloma. The challenge released a total of 775 images with 690 and 85 images of sizes 2040×1536 and 1920×2560 pixels, respectively, captured from two different (microscope and camera) setups. The participants had to segment the plasma cells with a separate label on each cell's nucleus and cytoplasm. This problem comprises many challenges, including a reduced color contrast between the cytoplasm and the background, and the clustering of cells with a feeble boundary separation of individual cells. To our knowledge, the SegPC-2021 challenge dataset is the largest publicly available annotated data on plasma cell segmentation in MM so far. The challenge targets a semi-automated tool to ensure the supervision of medical experts. It was conducted for a span of five months, from November 2020 to April 2021. Initially, the data was shared with 696 people from 52 teams, of which 41 teams submitted the results of their models on the evaluation portal in the validation phase. Similarly, 20 teams qualified for the last round, of which 16 teams submitted the results in the final test phase. All the top-5 teams employed DL-based approaches, and the best mIoU obtained on the final test set of 277 microscopic images was 0.9389. All these five models have been analyzed and discussed in detail. This challenge task is a step towards the target of creating an automated MM diagnostic tool.

Postoperative Severity Assessment in Sheep.

INTRODUCTION: Sheep are frequently used in translational surgical orthopedic studies. Naturally, a good pain management is mandatory for animal welfare, although it is also important with regard to data quality. However, methods for adequate severity assessment, especially considering pain, are rather rare regarding large animal models. Therefore, in the present study, accompanying a surgical pilot study, telemetry and the Sheep Grimace Scale (SGS) were used in addition to clinical scoring for severity assessment after surgical interventions in sheep. METHODS: Telemetric devices were implanted in a first surgery subcutaneously into four German black-headed mutton ewes (4-5 years, 77-115 kg). After 3-4 weeks of recovery, sheep underwent tendon ablation of the left M. infraspinatus. Clinical scoring and video recordings for SGS analysis were performed after both surgeries, and the heart rate (HR) and general activity were monitored by telemetry. RESULTS: Immediately after surgery, clinical score and HR were slightly increased, and activity was decreased in individual sheep after both surgeries. The SGS mildly elevated directly after transmitter implantation but increased to higher levels after tendon ablation immediately after surgery and on the following day. CONCLUSION: In summary, SGS- and telemetry-derived data were suitable to detect postoperative pain in sheep with the potential to improve individual pain recognition and postoperative management, which consequently contributes to refinement.

Self-supervised Semantic Segmentation: Consistency over Transformation.

Accurate medical image segmentation is of utmost importance for enabling automated clinical decision procedures. However, prevailing supervised deep learning approaches for medical image segmentation encounter significant challenges due to their heavy dependence on extensive labeled training data. To tackle this issue, we propose a novel self-supervised algorithm, S3-Net, which integrates a robust framework based on the proposed Inception Large Kernel Attention (I-LKA) modules. This architectural enhancement makes it possible to comprehensively capture contextual information while preserving local intricacies, thereby enabling precise semantic segmentation. Furthermore, considering that lesions in medical images often exhibit deformations, we leverage deformable convolution as an integral component to effectively capture and delineate lesion deformations for superior object boundary definition. Additionally, our self-supervised strategy emphasizes the acquisition of invariance to affine transformations, which is commonly encountered in medical scenarios. This emphasis on robustness with respect to geometric distortions significantly enhances the model's ability to accurately model and handle such distortions. To enforce spatial consistency and promote the grouping of spatially connected image pixels with similar feature representations, we introduce a spatial consistency loss term. This aids the network in effectively capturing the relationships among neighboring pixels and enhancing the overall segmentation quality. The S3-Net approach iteratively learns pixel-level feature representations for image content clustering in an end-to-end manner. Our experimental results on skin lesion and lung organ segmentation tasks show the superior performance of our method compared to the SOTA approaches.

Laplacian-Former: Overcoming the Limitations of Vision Transformers in Local Texture Detection.

Vision Transformer (ViT) models have demonstrated a breakthrough in a wide range of computer vision tasks. However, compared to the Convolutional Neural Network (CNN) models, it has been observed that the ViT models struggle to capture high-frequency components of images, which can limit their ability to detect local textures and edge information. As abnormalities in human tissue, such as tumors and lesions, may greatly vary in structure, texture, and shape, high-frequency information such as texture is crucial for effective semantic segmentation tasks. To address this limitation in ViT models, we propose a new technique, Laplacian-Former, that enhances the self-attention map by adaptively re-calibrating the frequency information in a Laplacian pyramid. More specifically, our proposed method utilizes a dual attention mechanism via efficient attention and frequency attention while the efficient attention mechanism reduces the complexity of self-attention to linear while producing the same output, selectively intensifying the contribution of shape and texture features. Furthermore, we introduce a novel efficient enhancement multi-scale bridge that effectively transfers spatial information from the encoder to the decoder while preserving the fundamental features. We demonstrate the efficacy of Laplacian-former on multi-organ and skin lesion segmentation tasks with +1.87% and +0.76% dice scores compared to SOTA approaches, respectively. Our implementation is publically available at GitHub.

Improving unsupervised stain-to-stain translation using self-supervision and meta-learning.

Background: In digital pathology, many image analysis tasks are challenged by the need for large and time-consuming manual data annotations to cope with various sources of variability in the image domain. Unsupervised domain adaptation based on image-to-image translation is gaining importance in this field by addressing variabilities without the manual overhead. Here, we tackle the variation of different histological stains by unsupervised stain-to-stain translation to enable a stain-independent applicability of a deep learning segmentation model. Methods: We use CycleGANs for stain-to-stain translation in kidney histopathology, and propose two novel approaches to improve translational effectivity. First, we integrate a prior segmentation network into the CycleGAN for a self-supervised, application-oriented optimization of translation through semantic guidance, and second, we incorporate extra channels to the translation output to implicitly separate artificial meta-information otherwise encoded for tackling underdetermined reconstructions. Results: The latter showed partially superior performances to the unmodified CycleGAN, but the former performed best in all stains providing instance-level Dice scores ranging between 78% and 92% for most kidney structures, such as glomeruli, tubules, and veins. However, CycleGANs showed only limited performance in the translation of other structures, e.g. arteries. Our study also found somewhat lower performance for all structures in all stains when compared to segmentation in the original stain. Conclusions: Our study suggests that with current unsupervised technologies, it seems unlikely to produce "generally" applicable simulated stains.

Large-scale extraction of interpretable features provides new insights into kidney histopathology - A proof-of-concept study.

Whole slide images contain a magnitude of quantitative information that may not be fully explored in qualitative visual assessments. We propose: (1) a novel pipeline for extracting a comprehensive set of visual features, which are detectable by a pathologist, as well as sub-visual features, which are not discernible by human experts and (2) perform detailed analyses on renal images from mice with experimental unilateral ureteral obstruction. An important criterion for these features is that they are easy to interpret, as opposed to features obtained from neural networks. We extract and compare features from pathological and healthy control kidneys to learn how the compartments (glomerulus, Bowman's capsule, tubule, interstitium, artery, and arterial lumen) are affected by the pathology. We define feature selection methods to extract the most informative and discriminative features. We perform statistical analyses to understand the relation of the extracted features, both individually, and in combinations, with tissue morphology and pathology. Particularly for the presented case-study, we highlight features that are affected in each compartment. With this, prior biological knowledge, such as the increase in interstitial nuclei, is confirmed and presented in a quantitative way, alongside with novel findings, like color and intensity changes in glomeruli and Bowman's capsule. The proposed approach is therefore an important step towards quantitative, reproducible, and rater-independent analysis in histopathology.

High-throughput phenotyping of nematode cysts.

The beet cyst nematode Heterodera schachtii is a plant pest responsible for crop loss on a global scale. Here, we introduce a high-throughput system based on computer vision that allows quantifying beet cyst nematode infestation and measuring phenotypic traits of cysts. After recording microscopic images of soil sample extracts in a standardized setting, an instance segmentation algorithm serves to detect nematode cysts in these images. In an evaluation using both ground truth samples with known cyst numbers and manually annotated images, the computer vision approach produced accurate nematode cyst counts, as well as accurate cyst segmentations. Based on such segmentations, cyst features could be computed that served to reveal phenotypical differences between nematode populations in different soils and in populations observed before and after the sugar beet planting period. The computer vision approach enables not only fast and precise cyst counting, but also phenotyping of cyst features under different conditions, providing the basis for high-throughput applications in agriculture and plant breeding research. Source code and annotated image data sets are freely available for scientific use.

Learning to Segment Fine Structures Under Image-Level Supervision With an Application to Nematode Segmentation.

Image segmentation models trained only with image-level labels have become increasingly popular as they require significantly less annotation effort than models trained with scribble, bounding box or pixel-wise annotations. While methods utilizing image-level labels achieve promising performance for the segmentation of larger-scale objects, they perform less well for the fine structures frequently encountered in biological images. In order to address this performance gap, we propose a deep network architecture based on two key principles, Global Weighted Pooling (GWP) and segmentation refinement by low-level image cues, that, together, make segmentation of fine structures possible. We apply our segmentation method to image datasets containing such fine structures, nematodes (worms + eggs) and nematode cysts immersed in organic debris objects, which is an application scenario encountered in automated soil sample screening. Supervised only with image-level labels, our approach achieves Dice coefficients of 79.72% and 58.51 % for nematode and nematode cyst segmentation, respectively.