Implicit neural representations in light microscopy.

Three-dimensional stacks acquired with confocal or two-photon microscopy are crucial for studying neuroanatomy. However, high-resolution image stacks acquired at multiple depths are time-consuming and susceptible to photobleaching. In vivo microscopy is further prone to motion artifacts. In this work, we suggest that deep neural networks with sine activation functions encoding implicit neural representations (SIRENs) are suitable for predicting intermediate planes and correcting motion artifacts, addressing the aforementioned shortcomings. We show that we can accurately estimate intermediate planes across multiple micrometers and fully automatically and unsupervised estimate a motion-corrected denoised picture. We show that noise statistics can be affected by SIRENs, however, rescued by a downstream denoising neural network, shown exemplarily with the recovery of dendritic spines. We believe that the application of these technologies will facilitate more efficient acquisition and superior post-processing in the future.

DeepD3, an open framework for automated quantification of dendritic spines.

Dendritic spines are the seat of most excitatory synapses in the brain, and a cellular structure considered central to learning, memory, and activity-dependent plasticity. The quantification of dendritic spines from light microscopy data is usually performed by humans in a painstaking and error-prone process. We found that human-to-human variability is substantial (inter-rater reliability 82.2±6.4%), raising concerns about the reproducibility of experiments and the validity of using human-annotated 'ground truth' as an evaluation method for computational approaches of spine identification. To address this, we present DeepD3, an open deep learning-based framework to robustly quantify dendritic spines in microscopy data in a fully automated fashion. DeepD3's neural networks have been trained on data from different sources and experimental conditions, annotated and segmented by multiple experts and they offer precise quantification of dendrites and dendritic spines. Importantly, these networks were validated in a number of datasets on varying acquisition modalities, species, anatomical locations and fluorescent indicators. The entire DeepD3 open framework, including the fully segmented training data, a benchmark that multiple experts have annotated, and the DeepD3 model zoo is fully available, addressing the lack of openly available datasets of dendritic spines while offering a ready-to-use, flexible, transparent, and reproducible spine quantification method.

A Generative Method for a Laryngeal Biosignal.

The Glottal Area Waveform (GAW) is an important component in quantitative clinical voice assessment, providing valuable insights into vocal fold function. In this study, we introduce a novel method employing Variational Autoencoders (VAEs) to generate synthetic GAWs. Our approach enables the creation of synthetic GAWs that closely replicate real-world data, offering a versatile tool for researchers and clinicians. We elucidate the process of manipulating the VAE latent space using the Glottal Opening Vector (GlOVe). The GlOVe allows precise control over the synthetic closure and opening of the vocal folds. By utilizing the GlOVe, we generate synthetic laryngeal biosignals. These biosignals accurately reflect vocal fold behavior, allowing for the emulation of realistic glottal opening changes. This manipulation extends to the introduction of arbitrary oscillations in the vocal folds, closely resembling real vocal fold oscillations. The range of factor coefficient values enables the generation of diverse biosignals with varying frequencies and amplitudes. Our results demonstrate that this approach yields highly accurate laryngeal biosignals, with the Normalized Mean Absolute Error values for various frequencies ranging from 9.6 â‹… 10-3 to 1.20 â‹… 10-2 for different experimented frequencies, alongside a remarkable training effectiveness, reflected in reductions of up to approximately 89.52% in key loss components. This proposed method may have implications for downstream speech synthesis and phonetics research, offering the potential for advanced and natural-sounding speech technologies.

Machine Learning-assisted immunophenotyping of peripheral blood identifies innate immune cells as best predictor of response to induction chemo-immunotherapy in head and neck squamous cell carcinoma - knowledge obtained from the CheckRad-CD8 trial.

PURPOSE: Individual prediction of treatment response is crucial for personalized treatment in multimodal approaches against head-and-neck squamous cell carcinoma (HNSCC). So far, no reliable predictive parameters for treatment schemes containing immunotherapy have been identified. This study aims to predict treatment response to induction chemo-immunotherapy based on the peripheral blood immune status in patients with locally advanced HNSCC. METHODS: The peripheral blood immune phenotype was assessed in whole blood samples in patients treated in the phase II CheckRad-CD8 trial as part of the pre-planned translational research program. Blood samples were analyzed by multicolor flow cytometry before (T1) and after (T2) induction chemo-immunotherapy with cisplatin/docetaxel/durvalumab/tremelimumab. Machine Learning techniques were used to predict pathological complete response (pCR) after induction therapy. RESULTS: The tested classifier methods (LDA, SVM, LR, RF, DT, and XGBoost) allowed a distinct prediction of pCR. Highest accuracy was achieved with a low number of features represented as principal components. Immune parameters obtained from the absolute difference (lT2-T1l) allowed the best prediction of pCR. In general, less than 30 parameters and at most 10 principal components were needed for highly accurate predictions. Across several datasets, cells of the innate immune system such as polymorphonuclear cells, monocytes, and plasmacytoid dendritic cells are most prominent. CONCLUSIONS: Our analyses imply that alterations of the innate immune cell distribution in the peripheral blood following induction chemo-immuno-therapy is highly predictive for pCR in HNSCC.

The modified Polsby-Popper score, a novel quantitative histomorphological biomarker and its potential to predict lymph node positivity and cancer-specific survival in oral tongue squamous cell carcinoma.

BACKGROUND: The significance of different histological spreading patterns of tumor tissue in oral tongue squamous cell carcinoma (TSCC) is well known. Our aim was to construct a numeric parameter on a continuous scale, that is, the modified Polsby-Popper (MPP) score, to describe the aggressiveness of tumor growth and infiltration, with the potential to analyze hematoxylin and eosin-stained whole slide images (WSIs) in an automated manner. We investigated the application of the MPP score in predicting survival and cervical lymph node metastases as well as in determining patients at risk in the context of different surgical margin scenarios. METHODS: We developed a semiautomated image analysis pipeline to detect areas belonging to the tumor tissue compartment. Perimeter and area measurements of all detected tissue regions were derived, and a specific mathematical formula was applied to reflect the perimeter/area ratio in a comparable, observer-independent manner across digitized WSIs. We demonstrated the plausibility of the MPP score by correlating it with well-established clinicopathologic parameters. We then performed survival analysis to assess the relevance of the MPP score, with an emphasis on different surgical margin scenarios. Machine learning models were developed to assess the relevance of the MPP score in predicting survival and occult cervical nodal metastases. RESULTS: The MPP score was associated with unfavorable tumor growth and infiltration patterns, the presence of lymph node metastases, the extracapsular spread of tumor cells, and higher tumor thickness. Higher MPP scores were associated with worse overall survival (OS) and tongue carcinoma-specific survival (TCSS), both when assessing all pT-categories and pT1-pT2 categories only; moreover, higher MPP scores were associated with a significantly worse TCSS in cases where a cancer-free surgical margin of <5 mm could be achieved on the main surgical specimen. This discriminatory capacity remained constant when examining pT1-pT2 categories only. Importantly, the MPP score could successfully define cases at risk in terms of metastatic disease in pT1-pT2 cancer where tumor thickness failed to exhibit a significant predictive value. Machine learning (ML) models incorporating the MPP score could predict the 5-year TCSS efficiently. Furthermore, we demonstrated that machine learning models that predict occult cervical lymph node involvement can benefit from including the MPP score. CONCLUSIONS: We introduced an objective, quantifiable, and observer-independent parameter, the MPP score, representing the aggressiveness of tumor growth and infiltration in TSCC. We showed its prognostic relevance especially in pT1-pT2 category TSCC, and its possible use in ML models predicting TCSS and occult lymph node metastases.

Explainable convolutional neural networks for assessing head and neck cancer histopathology.

PURPOSE: Although neural networks have shown remarkable performance in medical image analysis, their translation into clinical practice remains difficult due to their lack of interpretability. An emerging field that addresses this problem is Explainable AI. METHODS: Here, we aimed to investigate the ability of Convolutional Neural Networks (CNNs) to classify head and neck cancer histopathology. To this end, we manually annotated 101 histopathological slides of locally advanced head and neck squamous cell carcinoma. We trained a CNN to classify tumor and non-tumor tissue, and another CNN to semantically segment four classes - tumor, non-tumor, non-specified tissue, and background. We applied Explainable AI techniques, namely Grad-CAM and HR-CAM, to both networks and explored important features that contributed to their decisions. RESULTS: The classification network achieved an accuracy of 89.9% on previously unseen data. Our segmentation network achieved a class-averaged Intersection over Union score of 0.690, and 0.782 for tumor tissue in particular. Explainable AI methods demonstrated that both networks rely on features agreeing with the pathologist's expert opinion. CONCLUSION: Our work suggests that CNNs can predict head and neck cancer with high accuracy. Especially if accompanied by visual explanations, CNNs seem promising for assisting pathologists in the assessment of cancer sections.

GlottisNetV2: Temporal Glottal Midline Detection Using Deep Convolutional Neural Networks.

High-speed videoendoscopy is a major tool for quantitative laryngology. Glottis segmentation and glottal midline detection are crucial for computing vocal fold-specific, quantitative parameters. However, fully automated solutions show limited clinical applicability. Especially unbiased glottal midline detection remains a challenging problem. We developed a multitask deep neural network for glottis segmentation and glottal midline detection. We used techniques from pose estimation to estimate the anterior and posterior points in endoscopy images. Neural networks were set up in TensorFlow/Keras and trained and evaluated with the BAGLS dataset. We found that a dual decoder deep neural network termed GlottisNetV2 outperforms the previously proposed GlottisNet in terms of MAPE on the test dataset (1.85% to 6.3%) while converging faster. Using various hyperparameter tunings, we allow fast and directed training. Using temporal variant data on an additional data set designed for this task, we can improve the median prediction accuracy from 2.1% to 1.76% when using 12 consecutive frames and additional temporal filtering. We found that temporal glottal midline detection using a dual decoder architecture together with keypoint estimation allows accurate midline prediction. We show that our proposed architecture allows stable and reliable glottal midline predictions ready for clinical use and analysis of symmetry measures.

Pixel-Based Swallow Measurements: Correcting Nonsquare Pixels.

PURPOSE: This research note illustrates the effects of video data with nonsquare pixels on the pixel-based measures obtained from videofluoroscopic swallow studies (VFSS). METHOD: Six pixel-based distance and area measures were obtained from two different videoflouroscopic study units; both yielding videos with nonsquare pixels with different pixel aspect ratios (PARs). The swallowing measures were obtained from the original VFSS videos and from the videos after their pixels were squared. RESULTS: The results demonstrated significant multivariate effects both in video type (original vs. squared) and in the interaction between video type and sample (two video recordings of different patients, different PARs, and opposing tilt angles of the external reference). A wide range of variabilities was observed on the pixel-based measures between original and squared videos with the percent deviation ranging from 0.1% to 9.1% with the maximum effect size of 7.43. CONCLUSIONS: This research note demonstrates the effect of disregarding PAR to distance and area pixel-based parameters. In addition, we present a multilevel roadmap to prevent possible measurement errors that could occur. At the planning stage, the PAR of video source should be identified, and, at the analyses stage, video data should be prescaled prior to analysis with PAR-unaware software. No methodology in prior absolute or relative pixel-based studies reports adjustment to the PAR prior to measurements nor identify the PAR as a possible source of variation within the literature. Addressing PAR will improve the precision and stability of pixel-based VFSS findings and improve comparability within and across clinical and research settings. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.21957134.

Long-term performance assessment of fully automatic biomedical glottis segmentation at the point of care.

Deep Learning has a large impact on medical image analysis and lately has been adopted for clinical use at the point of care. However, there is only a small number of reports of long-term studies that show the performance of deep neural networks (DNNs) in such an environment. In this study, we measured the long-term performance of a clinically optimized DNN for laryngeal glottis segmentation. We have collected the video footage for two years from an AI-powered laryngeal high-speed videoendoscopy imaging system and found that the footage image quality is stable across time. Next, we determined the DNN segmentation performance on lossy and lossless compressed data revealing that only 9% of recordings contain segmentation artifacts. We found that lossy and lossless compression is on par for glottis segmentation, however, lossless compression provides significantly superior image quality. Lastly, we employed continual learning strategies to continuously incorporate new data into the DNN to remove the aforementioned segmentation artifacts. With modest manual intervention, we were able to largely alleviate these segmentation artifacts by up to 81%. We believe that our suggested deep learning-enhanced laryngeal imaging platform consistently provides clinically sound results, and together with our proposed continual learning scheme will have a long-lasting impact on the future of laryngeal imaging.

A single latent channel is sufficient for biomedical glottis segmentation.

Glottis segmentation is a crucial step to quantify endoscopic footage in laryngeal high-speed videoendoscopy. Recent advances in deep neural networks for glottis segmentation allow for a fully automatic workflow. However, exact knowledge of integral parts of these deep segmentation networks remains unknown, and understanding the inner workings is crucial for acceptance in clinical practice. Here, we show that a single latent channel as a bottleneck layer is sufficient for glottal area segmentation using systematic ablations. We further demonstrate that the latent space is an abstraction of the glottal area segmentation relying on three spatially defined pixel subtypes allowing for a transparent interpretation. We further provide evidence that the latent space is highly correlated with the glottal area waveform, can be encoded with four bits, and decoded using lean decoders while maintaining a high reconstruction accuracy. Our findings suggest that glottis segmentation is a task that can be highly optimized to gain very efficient and explainable deep neural networks, important for application in the clinic. In the future, we believe that online deep learning-assisted monitoring is a game-changer in laryngeal examinations.