Optimization-based decoding of Imaging Spatial Transcriptomics data.

MOTIVATION: Imaging Spatial Transcriptomics techniques characterize gene expression in cells in their native context by imaging barcoded probes for mRNA with single molecule resolution. However, the need to acquire many rounds of high-magnification imaging data limits the throughput and impact of existing methods. RESULTS: We describe the Joint Sparse method for Imaging Transcriptomics, an algorithm for decoding lower magnification Imaging Spatial Transcriptomics data than that used in standard experimental workflows. Joint Sparse method for Imaging Transcriptomics incorporates codebook knowledge and sparsity assumptions into an optimization problem, which is less reliant on well separated optical signals than current pipelines. Using experimental data obtained by performing Multiplexed Error-Robust Fluorescence in situ Hybridization on tissue from mouse brain, we demonstrate that Joint Sparse method for Imaging Transcriptomics enables improved throughput and recovery performance over standard decoding methods. AVAILABILITY AND IMPLEMENTATION: Software implementation of JSIT, together with example files, is available at https://github.com/jpbryan13/JSIT.

Communication-efficient federated learning.

Federated learning (FL) enables edge devices, such as Internet of Things devices (e.g., sensors), servers, and institutions (e.g., hospitals), to collaboratively train a machine learning (ML) model without sharing their private data. FL requires devices to exchange their ML parameters iteratively, and thus the time it requires to jointly learn a reliable model depends not only on the number of training steps but also on the ML parameter transmission time per step. In practice, FL parameter transmissions are often carried out by a multitude of participating devices over resource-limited communication networks, for example, wireless networks with limited bandwidth and power. Therefore, the repeated FL parameter transmission from edge devices induces a notable delay, which can be larger than the ML model training time by orders of magnitude. Hence, communication delay constitutes a major bottleneck in FL. Here, a communication-efficient FL framework is proposed to jointly improve the FL convergence time and the training loss. In this framework, a probabilistic device selection scheme is designed such that the devices that can significantly improve the convergence speed and training loss have higher probabilities of being selected for ML model transmission. To further reduce the FL convergence time, a quantization method is proposed to reduce the volume of the model parameters exchanged among devices, and an efficient wireless resource allocation scheme is developed. Simulation results show that the proposed FL framework can improve the identification accuracy and convergence time by up to 3.6% and 87% compared to standard FL.

cSPARCOM: Multi-detector reconstruction by confocal super-resolution correlation microscopy.

Image scanning microscopy (ISM), an upgraded successor of the ubiquitous confocal microscope, facilitates up to two-fold improvement in lateral resolution, and has become an indispensable element in the toolbox of the bio-imaging community. Recently, super-resolution optical fluctuation image scanning microscopy (SOFISM) integrated the analysis of intensity-fluctuations information into the basic ISM architecture, to enhance its resolving power. Both of these techniques typically rely on pixel-reassignment as a fundamental processing step, in which the parallax of different detector elements to the sample is compensated by laterally shifting the point spread function (PSF). Here, we propose an alternative analysis approach, based on the recent high-performing sparsity-based super-resolution correlation microscopy (SPARCOM) method. Through measurements of DNA origami nano-rulers and fixed cells labeled with organic dye, we experimentally show that confocal SPARCOM (cSPARCOM), which circumvents pixel-reassignment altogether, provides enhanced resolution compared to pixel-reassigned based analysis. Thus, cSPARCOM further promotes the effectiveness of ISM, and particularly that of correlation based ISM implementations such as SOFISM, where the PSF deviates significantly from spatial invariance.

COVID-19 classification of X-ray images using deep neural networks.

OBJECTIVES: In the midst of the coronavirus disease 2019 (COVID-19) outbreak, chest X-ray (CXR) imaging is playing an important role in diagnosis and monitoring of patients with COVID-19. We propose a deep learning model for detection of COVID-19 from CXRs, as well as a tool for retrieving similar patients according to the model's results on their CXRs. For training and evaluating our model, we collected CXRs from inpatients hospitalized in four different hospitals. METHODS: In this retrospective study, 1384 frontal CXRs, of COVID-19 confirmed patients imaged between March and August 2020, and 1024 matching CXRs of non-COVID patients imaged before the pandemic, were collected and used to build a deep learning classifier for detecting patients positive for COVID-19. The classifier consists of an ensemble of pre-trained deep neural networks (DNNS), specifically, ReNet34, ReNet50¸ ReNet152, and vgg16, and is enhanced by data augmentation and lung segmentation. We further implemented a nearest-neighbors algorithm that uses DNN-based image embeddings to retrieve the images most similar to a given image. RESULTS: Our model achieved accuracy of 90.3%, (95% CI: 86.3-93.7%) specificity of 90% (95% CI: 84.3-94%), and sensitivity of 90.5% (95% CI: 85-94%) on a test dataset comprising 15% (350/2326) of the original images. The AUC of the ROC curve is 0.96 (95% CI: 0.93-0.97). CONCLUSION: We provide deep learning models, trained and evaluated on CXRs that can assist medical efforts and reduce medical staff workload in handling COVID-19. KEY POINTS: • A machine learning model was able to detect chest X-ray (CXR) images of patients tested positive for COVID-19 with accuracy and detection rate above 90%. • A tool was created for finding existing CXR images with imaging characteristics most similar to a given CXR, according to the model's image embeddings.

Deep Task-Based Quantization.

Quantizers play a critical role in digital signal processing systems. Recent works have shown that the performance of acquiring multiple analog signals using scalar analog-to-digital converters (ADCs) can be significantly improved by processing the signals prior to quantization. However, the design of such hybrid quantizers is quite complex, and their implementation requires complete knowledge of the statistical model of the analog signal. In this work we design data-driven task-oriented quantization systems with scalar ADCs, which determine their analog-to-digital mapping using deep learning tools. These mappings are designed to facilitate the task of recovering underlying information from the quantized signals. By using deep learning, we circumvent the need to explicitly recover the system model and to find the proper quantization rule for it. Our main target application is multiple-input multiple-output (MIMO) communication receivers, which simultaneously acquire a set of analog signals, and are commonly subject to constraints on the number of bits. Our results indicate that, in a MIMO channel estimation setup, the proposed deep task-bask quantizer is capable of approaching the optimal performance limits dictated by indirect rate-distortion theory, achievable using vector quantizers and requiring complete knowledge of the underlying statistical model. Furthermore, for a symbol detection scenario, it is demonstrated that the proposed approach can realize reliable bit-efficient hybrid MIMO receivers capable of setting their quantization rule in light of the task.

Learned SPARCOM: unfolded deep super-resolution microscopy.

The use of photo-activated fluorescent molecules to create long sequences of low emitter-density diffraction-limited images enables high-precision emitter localization, but at the cost of low temporal resolution. We suggest combining SPARCOM, a recent high-performing classical method, with model-based deep learning, using the algorithm unfolding approach, to design a compact neural network incorporating domain knowledge. Our results show that we can obtain super-resolution imaging from a small number of high emitter density frames without knowledge of the optical system and across different test sets using the proposed learned SPARCOM (LSPARCOM) network. We believe LSPARCOM can pave the way to interpretable, efficient live-cell imaging in many settings, and find broad use in single molecule localization microscopy of biological structures.

Enhancing the Kramers-Kronig receiver via dispersion-based spatial diversity.

We report a scheme for reconstructing the complex envelope of an optical signal from two decorrelated measurements of its intensity. The decorrelation is achieved by splitting the received optical signal into two copies, and by dispersing one of the copies prior to photo detection. The reconstructed complex-valued signal is obtained by means of an iterative algorithm that requires only a few tens of iterations. The starting point of the search procedure is produced by Kramers-Kronig (KK) reconstruction. With this procedure, the continuous-wave tone that accompanies the received signal is reduced by 5 dB to 6 dB compared to the requirement of a KK receiver alone.

Sparsity-based super-resolution microscopy from correlation information.

For more than a century, the wavelength of light was considered to be a fundamental limit on the spatial resolution of optical imaging. Particularly in light microscopy, this limit, known as Abbe's diffraction limit, places a fundamental constraint on the ability to image sub-cellular organelles with high resolution. However, modern microscopy techniques such as STED, PALM, and STORM, manage to recover sub-wavelength information, by relying on fluorescence imaging. Specifically, PALM/STORM acquire large sequences of fluorescence images from molecules attached to the organelles within the imaged specimen, such that in each frame only a small set of fluorophores are active. The position of each fluorophore can be found accurately in each frame, and the image is recovered by superimposing the points from all frames. The resulting grainy image is subsequently smoothed to produce the final super-resolved image with a resolution of tens of nano-meters. However, because PALM/STORM rely on many (>10,000) exposures, they suffer from poor temporal resolution. To address that, super-resolution optical fluctuation imaging (SOFI) was shown to produce sub-diffraction images with increased temporal resolution, by allowing for higher fluorophore density and exploiting the temporal statistics of the emissions. However, the improved temporal resolution of SOFI comes at the expense of its spatial resolution, which is not as high as that of PALM/STORM. Here, we present a new method called SPARCOM: sparsity-based super-resolution correlation microscopy, which combines a shorter integration time than previously reported with spatial resolution comparable to PALM and STORM. SPARCOM relies on sparsity in the correlation domain, exploiting the sparse distribution of fluorescent molecules and the lack of correlation between different emitters. We demonstrate our technique in simulations and in experiments and provide comparisons to state-of-the-art high density methods.

Sparsity-based super-resolution microscopy from correlation information: erratum.

Two references on sparse deconvolution of high-density fluorescence microscopy images are added to [Opt. Express26(14), 18238 (2018)].

Sparsity-Based Super Resolution for SEM Images.

The scanning electron microscope (SEM) is an electron microscope that produces an image of a sample by scanning it with a focused beam of electrons. The electrons interact with the atoms in the sample, which emit secondary electrons that contain information about the surface topography and composition. The sample is scanned by the electron beam point by point, until an image of the surface is formed. Since its invention in 1942, the capabilities of SEMs have become paramount in the discovery and understanding of the nanometer world, and today it is extensively used for both research and in industry. In principle, SEMs can achieve resolution better than one nanometer. However, for many applications, working at subnanometer resolution implies an exceedingly large number of scanning points. For exactly this reason, the SEM diagnostics of microelectronic chips is performed either at high resolution (HR) over a small area or at low resolution (LR) while capturing a larger portion of the chip. Here, we employ sparse coding and dictionary learning to algorithmically enhance low-resolution SEM images of microelectronic chips-up to the level of the HR images acquired by slow SEM scans, while considerably reducing the noise. Our methodology consists of two steps: an offline stage of learning a joint dictionary from a sequence of LR and HR images of the same region in the chip, followed by a fast-online super-resolution step where the resolution of a new LR image is enhanced. We provide several examples with typical chips used in the microelectronics industry, as well as a statistical study on arbitrary images with characteristic structural features. Conceptually, our method works well when the images have similar characteristics, as microelectronics chips do. This work demonstrates that employing sparsity concepts can greatly improve the performance of SEM, thereby considerably increasing the scanning throughput without compromising on analysis quality and resolution.

WINDOW: wideband demodulator for optical waveforms.

Optical communication systems, which operate at very high rates, are often limited by the sampling rate bottleneck. The optical wideband regime may exceed analog to digital converters (ADCs) front-end bandwidth. Multi-channel sampling approaches, such as interleaved ADCs, also known as multicoset sampling in some contexts, have been proposed to sample the wideband signal using several channels. Each channel samples below the Nyquist rate such that the overall sampling rate is preserved. However, this scheme suffers from two practical limitations that make its implementation difficult. First, the inherent anti-aliasing filter of the samplers distorts the wideband signal. Second, it requires accurate time shifts on the order of the signal's Nyquist rate, which are challenging to maintain. In this work, we propose an alternative multi-channel sampling scheme, the wideband demodulator for optical waveforms (WINDOW), based on analog RF demodulation, where each channel aliases the spectrum using a periodic mixing function before integration and sampling. We show that intentionally using the inherent ADC filter to perform integration increases the signal to noise ratio (SNR). We demonstrate both theoretically and through numerical experiments that our system outperforms interleaved sampling in terms of signal recovery and symbol estimation in the presence of both thermal and quantization noise but is slightly less robust to timing jitter. The main contribution of this work is the application of RF demodulation concepts proposed in the context of sub-Nyquist sampling, e.g. random demodulator and modulated wideband converter, to optical communication signals in the Nyquist regime. We develop a sampling scheme that presents an alternative for optical links where thermal noise in the receiver is the bottleneck.

Photoacoustic imaging beyond the acoustic diffraction-limit with dynamic speckle illumination and sparse joint support recovery.

In deep tissue photoacoustic imaging the spatial resolution is inherently limited by the acoustic wavelength. Recently, it was demonstrated that it is possible to surpass the acoustic diffraction limit by analyzing fluctuations in a set of photoacoustic images obtained under unknown speckle illumination patterns. Here, we purpose an approach to boost reconstruction fidelity and resolution, while reducing the number of acquired images by utilizing a compressed sensing computational reconstruction framework. The approach takes into account prior knowledge of the system response and sparsity of the target structure. We provide proof of principle experiments of the approach and demonstrate that improved performance is obtained when both speckle fluctuations and object priors are used. We numerically study the expected performance as a function of the measurement's signal to noise ratio and sample spatial-sparsity. The presented reconstruction framework can be applied to analyze existing photoacoustic experimental data sets containing dynamic fluctuations.

Interpretable Neural Networks for Video Separation: Deep Unfolding RPCA With Foreground Masking.

We present two deep unfolding neural networks for the simultaneous tasks of background subtraction and foreground detection in video. Unlike conventional neural networks based on deep feature extraction, we incorporate domain-knowledge models by considering a masked variation of the robust principal component analysis problem (RPCA). With this approach, we separate video clips into low-rank and sparse components, respectively corresponding to the backgrounds and foreground masks indicating the presence of moving objects. Our models, coined ROMAN-S and ROMAN-R, map the iterations of two alternating direction of multipliers methods (ADMM) to trainable convolutional layers, and the proximal operators are mapped to non-linear activation functions with trainable thresholds. This approach leads to lightweight networks with enhanced interpretability that can be trained on limited data. In ROMAN-S, the correlation in time of successive binary masks is controlled with side-information based on l1 - l1 minimization. ROMAN-R enhances the foreground detection by learning a dictionary of atoms to represent the moving foreground in a high-dimensional feature space and by using reweighted- l1 - l1 minimization. Experiments are conducted on both synthetic and real video datasets, for which we also include an analysis of the generalization to unseen clips. Comparisons are made with existing deep unfolding RPCA neural networks, which do not use a mask formulation for the foreground, and with a 3D U-Net baseline. Results show that our proposed models outperform other deep unfolding networks, as well as the untrained optimization algorithms. ROMAN-R, in particular, is competitive with the U-Net baseline for foreground detection, with the additional advantage of providing video backgrounds and requiring substantially fewer training parameters and smaller training sets.

Detecting bone lesions in X-ray under diverse acquisition conditions.

Purpose: The diagnosis of primary bone tumors is challenging as the initial complaints are often non-specific. The early detection of bone cancer is crucial for a favorable prognosis. Incidentally, lesions may be found on radiographs obtained for other reasons. However, these early indications are often missed. We propose an automatic algorithm to detect bone lesions in conventional radiographs to facilitate early diagnosis. Detecting lesions in such radiographs is challenging. First, the prevalence of bone cancer is very low; any method must show high precision to avoid a prohibitive number of false alarms. Second, radiographs taken in health maintenance organizations (HMOs) or emergency departments (EDs) suffer from inherent diversity due to different X-ray machines, technicians, and imaging protocols. This diversity poses a major challenge to any automatic analysis method. Approach: We propose training an off-the-shelf object detection algorithm to detect lesions in radiographs. The novelty of our approach stems from a dedicated preprocessing stage that directly addresses the diversity of the data. The preprocessing consists of self-supervised region-of-interest detection using vision transformer (ViT), and a foreground-based histogram equalization for contrast enhancement to relevant regions only. Results: We evaluate our method via a retrospective study that analyzes bone tumors on radiographs acquired from January 2003 to December 2018 under diverse acquisition protocols. Our method obtains 82.43% sensitivity at a 1.5% false-positive rate and surpasses existing preprocessing methods. For lesion detection, our method achieves 82.5% accuracy and an IoU of 0.69. Conclusions: The proposed preprocessing method enables effectively coping with the inherent diversity of radiographs acquired in HMOs and EDs.

Efficient coherent diffractive imaging for sparsely varying objects.

We demonstrate an efficient phase-retrieval method, allowing the recovery of features of dynamic objects that are sparsely varying--i.e. when the difference between two consecutive frames is small. The method uses redundancy in information between similar consecutive frames to recover the features more robustly than current phase-retrieval methods, and necessitates a considerably smaller number of measurements. Both of these features directly lead to shorter acquisition time, paving the way to coherent diffractive imaging of fast dynamic processes. Numerical simulations show a possible 100-fold improvement in temporal resolution over existing methods.

Sparsity-based super-resolution and phase-retrieval in waveguide arrays.

We present a scheme for recovering the complex input field launched into a waveguide array, from partial measurements of its output intensity, given advance knowledge that the input is sparse. In spite of the fact that in general the inversion problem is ill-conditioned, we demonstrate experimentally and in simulations that the prior knowledge of sparsity helps overcome the loss of information. Our method is based on GESPAR, a recently proposed efficient phase retrieval algorithm. Possible applications include optical interconnects and quantum state tomography, and the ideas are extendable to other multiple input and multiple output (MIMO) communication schemes.

Real-time change detection of steady-state evoked potentials.

Steady-state evoked potentials (SSEP) are the electrical activity recorded from the scalp in response to high-rate sensory stimulation. SSEP consist of a constituent frequency component matching the stimulation rate, whose amplitude and phase remain constant with time and are sensitive to functional changes in the stimulated sensory system. Monitoring SSEP during neurosurgical procedures allows identification of an emerging impairment early enough before the damage becomes permanent. In routine practice, SSEP are extracted by averaging of the EEG recordings, allowing detection of neurological changes within approximately a minute. As an alternative to the relatively slow-responding empirical averaging, we present an algorithm that detects changes in the SSEP within seconds. Our system alerts when changes in the SSEP are detected by applying a two-step Generalized Likelihood Ratio Test (GLRT) on the unaveraged EEG recordings. This approach outperforms conventional detection and provides the monitor with a statistical measure of the likelihood that a change occurred, thus enhancing its sensitivity and reliability. The system's performance is analyzed using Monte Carlo simulations and tested on real EEG data recorded under coma.

Sparsity-Based Multi-Person Non-Contact Vital Signs Monitoring via FMCW Radar.

Non-contact technology for monitoring the vital signs of multiple individuals, such as respiration and heartbeat, has been investigated in recent years due to the rising cardiopulmonary morbidity, the risk of disease transmission, and the heavy burden on medical staff. Frequency-modulated continuous wave (FMCW) radars have shown great promise in meeting these needs, even using a single-input-single-output (SISO) setup. However, contemporary techniques for non-contact vital signs monitoring (NCVSM) via SISO FMCW radar, are based on simplistic models and present difficulties in coping with noisy environments containing multiple objects. In this work, we first develop an extended model for multi-person NCVSM via SISO FMCW radar. Then, by utilizing the sparse nature of the modeled signals in conjunction with human-typical cardiopulmonary features, we present accurate localization and NCVSM of multiple individuals in a cluttered scenario, even with only a single channel. Specifically, we provide a joint-sparse recovery mechanism to localize people and develop a robust method for NCVSM called Vital Signs-based Dictionary Recovery (VSDR), which uses a dictionary-based approach to search for the rates of respiration and heartbeat over high-resolution grids corresponding to human cardiopulmonary activity. The advantages of our method are illustrated through examples that combine the proposed model with in-vivo data of 30 individuals. We demonstrate accurate human localization in a noisy scenario that includes both static and vibrating objects and show that our VSDR approach outperforms existing NCVSM techniques based on several statistical metrics. The findings support the widespread use of FMCW radars with the proposed algorithms in healthcare.

Ultrasound Signal Processing: From Models to Deep Learning.

Medical ultrasound imaging relies heavily on high-quality signal processing to provide reliable and interpretable image reconstructions. Conventionally, reconstruction algorithms have been derived from physical principles. These algorithms rely on assumptions and approximations of the underlying measurement model, limiting image quality in settings where these assumptions break down. Conversely, more sophisticated solutions based on statistical modeling or careful parameter tuning or derived from increased model complexity can be sensitive to different environments. Recently, deep learning-based methods, which are optimized in a data-driven fashion, have gained popularity. These model-agnostic techniques often rely on generic model structures and require vast training data to converge to a robust solution. A relatively new paradigm combines the power of the two: leveraging data-driven deep learning and exploiting domain knowledge. These model-based solutions yield high robustness and require fewer parameters and training data than conventional neural networks. In this work we provide an overview of these techniques from the recent literature and discuss a wide variety of ultrasound applications. We aim to inspire the reader to perform further research in this area and to address the opportunities within the field of ultrasound signal processing. We conclude with a future perspective on model-based deep learning techniques for medical ultrasound.

Memory-efficient low-compute segmentation algorithms for bladder-monitoring smart ultrasound devices.

Post-operative urinary retention is a medical condition where patients cannot urinate despite having a full bladder. Ultrasound imaging of the bladder is used to estimate urine volume for early diagnosis and management of urine retention. Moreover, the use of bladder ultrasound can reduce the need for an indwelling urinary catheter and the risk of catheter-associated urinary tract infection. Wearable ultrasound devices combined with machine-learning based bladder volume estimation algorithms reduce the burdens of nurses in hospital settings and improve outpatient care. However, existing algorithms are memory and computation intensive, thereby demanding the use of expensive GPUs. In this paper, we develop and validate a low-compute memory-efficient deep learning model for accurate bladder region segmentation and urine volume calculation. B-mode ultrasound bladder images of 360 patients were divided into training and validation sets; another 74 patients were used as the test dataset. Our 1-bit quantized models with 4-bits and 6-bits skip connections achieved an accuracy within [Formula: see text] and [Formula: see text], respectively, of a full precision state-of-the-art neural network (NN) without any floating-point operations and with an [Formula: see text] and [Formula: see text] reduction in memory requirements to fit under 150 kB. The means and standard deviations of the volume estimation errors, relative to estimates from ground-truth clinician annotations, were [Formula: see text] ml and [Formula: see text] ml, respectively. This lightweight NN can be easily integrated on the wearable ultrasound device for automated and continuous monitoring of urine volume. Our approach can potentially be extended to other clinical applications, such as monitoring blood pressure and fetal heart rate.