A survey of Transformer applications for histopathological image analysis: New developments and future directions.

Transformers have been widely used in many computer vision challenges and have shown the capability of producing better results than convolutional neural networks (CNNs). Taking advantage of capturing long-range contextual information and learning more complex relations in the image data, Transformers have been used and applied to histopathological image processing tasks. In this survey, we make an effort to present a thorough analysis of the uses of Transformers in histopathological image analysis, covering several topics, from the newly built Transformer models to unresolved challenges. To be more precise, we first begin by outlining the fundamental principles of the attention mechanism included in Transformer models and other key frameworks. Second, we analyze Transformer-based applications in the histopathological imaging domain and provide a thorough evaluation of more than 100 research publications across different downstream tasks to cover the most recent innovations, including survival analysis and prediction, segmentation, classification, detection, and representation. Within this survey work, we also compare the performance of CNN-based techniques to Transformers based on recently published papers, highlight major challenges, and provide interesting future research directions. Despite the outstanding performance of the Transformer-based architectures in a number of papers reviewed in this survey, we anticipate that further improvements and exploration of Transformers in the histopathological imaging domain are still required in the future. We hope that this survey paper will give readers in this field of study a thorough understanding of Transformer-based techniques in histopathological image analysis, and an up-to-date paper list summary will be provided at https://github.com/S-domain/Survey-Paper .

Multi-Scale FPGA-Based Infrared Image Enhancement by Using RGF and CLAHE.

Infrared sensors capture thermal radiation emitted by objects. They can operate in all weather conditions and are thus employed in fields such as military surveillance, autonomous driving, and medical diagnostics. However, infrared imagery poses challenges such as low contrast and indistinct textures due to the long wavelength of infrared radiation and susceptibility to interference. In addition, complex enhancement algorithms make real-time processing challenging. To address these problems and improve visual quality, in this paper, we propose a multi-scale FPGA-based method for real-time enhancement of infrared images by using rolling guidance filter (RGF) and contrast-limited adaptive histogram equalization (CLAHE). Specifically, the original image is first decomposed into various scales of detail layers and a base layer using RGF. Secondly, we fuse detail layers of diverse scales, then enhance the detail information by using gain coefficients and employ CLAHE to improve the contrast of the base layer. Thirdly, we fuse the detail layers and base layer to obtain the image with global details of the input image. Finally, the proposed algorithm is implemented on an FPGA using advanced high-level synthesis tools. Comprehensive testing of our proposed method on the AXU15EG board demonstrates its effectiveness in significantly improving image contrast and enhancing detail information. At the same time, real-time enhancement at a speed of 147 FPS is achieved for infrared images with a resolution of 640 × 480.

A Cost-Efficient High-Speed VLSI Architecture for Spiking Convolutional Neural Network Inference Using Time-Step Binary Spike Maps.

Neuromorphic hardware systems have been gaining ever-increasing focus in many embedded applications as they use a brain-inspired, energy-efficient spiking neural network (SNN) model that closely mimics the human cortex mechanism by communicating and processing sensory information via spatiotemporally sparse spikes. In this paper, we fully leverage the characteristics of spiking convolution neural network (SCNN), and propose a scalable, cost-efficient, and high-speed VLSI architecture to accelerate deep SCNN inference for real-time low-cost embedded scenarios. We leverage the snapshot of binary spike maps at each time-step, to decompose the SCNN operations into a series of regular and simple time-step CNN-like processing to reduce hardware resource consumption. Moreover, our hardware architecture achieves high throughput by employing a pixel stream processing mechanism and fine-grained data pipelines. Our Zynq-7045 FPGA prototype reached a high processing speed of 1250 frames/s and high recognition accuracies on the MNIST and Fashion-MNIST image datasets, demonstrating the plausibility of our SCNN hardware architecture for many embedded applications.

A High-Speed Low-Cost VLSI System Capable of On-Chip Online Learning for Dynamic Vision Sensor Data Classification.

This paper proposes a high-speed low-cost VLSI system capable of on-chip online learning for classifying address-event representation (AER) streams from dynamic vision sensor (DVS) retina chips. The proposed system executes a lightweight statistic algorithm based on simple binary features extracted from AER streams and a Random Ferns classifier to classify these features. The proposed system's characteristics of multi-level pipelines and parallel processing circuits achieves a high throughput up to 1 spike event per clock cycle for AER data processing. Thanks to the nature of the lightweight algorithm, our hardware system is realized in a low-cost memory-centric paradigm. In addition, the system is capable of on-chip online learning to flexibly adapt to different in-situ application scenarios. The extra overheads for on-chip learning in terms of time and resource consumption are quite low, as the training procedure of the Random Ferns is quite simple, requiring few auxiliary learning circuits. An FPGA prototype of the proposed VLSI system was implemented with 9.5~96.7% memory consumption and <11% computational and logic resources on a Xilinx Zynq-7045 chip platform. It was running at a clock frequency of 100 MHz and achieved a peak processing throughput up to 100 Meps (Mega events per second), with an estimated power consumption of 690 mW leading to a high energy efficiency of 145 Meps/W or 145 event/µJ. We tested the prototype system on MNIST-DVS, Poker-DVS, and Posture-DVS datasets, and obtained classification accuracies of 77.9%, 99.4% and 99.3%, respectively. Compared to prior works, our VLSI system achieves higher processing speeds, higher computing efficiency, comparable accuracy, and lower resource costs.

MorphBungee: A 65-nm 7.2-mm2 27-µJ/image Digital Edge Neuromorphic Chip with On-Chip 802-frame/s Multi-Layer Spiking Neural Network Learning.

This paper presents a digital edge neuromorphic spiking neural network (SNN) processor chip for a variety of edge intelligent cognitive applications. This processor allows high-speed, high-accuracy and fully on-chip spike-timing-based multi-layer SNN learning. It is characteristic of hierarchical multi-core architecture, event-driven processing paradigm, meta-crossbar for efficient spike communication, and hybrid and reconfigurable parallelism. A prototype chip occupying an active silicon area of 7.2 mm2 was fabricated using a 65-nm 1P9M CMOS process. when running a 256-256-256-256-200 4-layer fully-connected SNN on downscaled 16 × 16 MNIST images. it typically achieved a high-speed throughput of 802 and 2270 frames/s for on-chip learning and inference, respectively, with a relatively low power dissipation of around 61 mW at a 100 MHz clock rate under a 1.0V core power supply, Our on-chip learning results in comparably high visual recognition accuracies of 96.06%, 83.38%, 84.53%, 99.22% and 100% on the MNIST, Fashion-MNIST, ETH-80, Yale-10 and ORL-10 datasets, respectively. In addition, we have successfully applied our neuromorphic chip to demonstrate high-resolution satellite cloud image segmentation and non-visual tasks including olfactory classification and textural news categorization. These results indicate that our neuromorphic chip is suitable for various intelligent edge systems under restricted cost, energy and latency budgets while requiring in-situ self-adaptative learning capability.

InfraNet: Accurate forehead temperature measurement framework for people in the wild with monocular thermal infrared camera.

During an epidemic, accurate human temperature screening based on neural networks for disease surveillance is important and challenging. Existing distant human forehead temperature measuring device usually adopts a dual-camera system using paired RGB and thermal infrared images to conduct face detection and temperature measurement. Since the facial RGB image may undermine people's privacy, we designed a monocular thermal system and proposed an effective framework called the InfraNet to measure and calibrate forehead temperature of people in the wild. To address the challenge of temperature floating, the InfraNet calibrates the subject's temperature with one's physical depth and horizontal offset predicted by a single infrared image. Our InfraNet framework mainly consists of three parts: face detection subnet, depth and horizontal offset estimation subnet and temperature calibration subnet. The temperature calibration performance can be improved with the help of spatial regularization term concentrating on predicting precise depth and horizontal offset of people. Besides, we collected a large-scale infrared image dataset in the both lab and wild scenarios, including 8,215 thermal infrared images. Experiments on our wild dataset demonstrated that the InfraNet achieved 91.6% high accuracy of distant multi-subject temperature measurement on average under the standard temperature threshold of strict 0.3°C.

TripleBrain: A Compact Neuromorphic Hardware Core With Fast On-Chip Self-Organizing and Reinforcement Spike-Timing Dependent Plasticity.

Human brain cortex acts as a rich inspiration source for constructing efficient artificial cognitive systems. In this paper, we investigate to incorporate multiple brain-inspired computing paradigms for compact, fast and high-accuracy neuromorphic hardware implementation. We propose the TripleBrain hardware core that tightly combines three common brain-inspired factors: the spike-based processing and plasticity, the self-organizing map (SOM) mechanism and the reinforcement learning scheme, to improve object recognition accuracy and processing throughput, while keeping low resource costs. The proposed hardware core is fully event-driven to mitigate unnecessary operations, and enables various on-chip learning rules (including the proposed SOM-STDP & R-STDP rule and the R-SOM-STDP rule regarded as the two variants of our TripleBrain learning rule) with different accuracy-latency tradeoffs to satisfy user requirements. An FPGA prototype of the neuromorphic core was implemented and elaborately tested. It realized high-speed learning (1349 frame/s) and inference (2698 frame/s), and obtained comparably high recognition accuracies of 95.10%, 80.89%, 100%, 94.94%, 82.32%, 100% and 97.93% on the MNIST, ETH-80, ORL-10, Yale-10, N-MNIST, Poker-DVS and Posture-DVS datasets, respectively, while only consuming 4146 (7.59%) slices, 32 (3.56%) DSPs and 131 (24.04%) Block RAMs on a Xilinx Zynq-7045 FPGA chip. Our neuromorphic core is very attractive for real-time resource-limited edge intelligent systems.

SWAV: a web-based visualization browser for sliding window analysis.

Sliding window analysis has been extensively applied in evolutionary biology. With the development of the high-throughput DNA sequencing of organisms at the population level, an application that is dedicated to visualizing population genetic test statistics at the genomic level is needed. We have developed the sliding window analysis viewer (SWAV), which is a web-based program that can be used to integrate, view and browse test statistics and perform genome annotation. In addition to browsing, SAV can mark, generate and customize statistical images and search by sequence alignment, position or gene name. These features facilitate the effectiveness of sliding window analysis. As an example application, yeast and silkworm resequencing data are analyzed with SWAV. The SWAV package, user manual and usage demo are available at http://swav.popgenetics.net.

A Noise-Reduced Light-to-Frequency Converter for Sub-0.1% Perfusion Index Blood SpO[Formula: see text] Sensing.

To improve the SpO 2 sensing system performance for hypoperfusion (low perfusion index) applications, this paper proposes a low-noise light-to-frequency converter scheme from two aspects. First, a low-noise photocurrent buffer is proposed by reducing the amplifier noise floor with a transconductance-boost ( gm-boost) circuit structure. Second, a digital processing unit of pulse-frequency-duty-cycle modulation is proposed to minimize the quantization noise in the following timer by limiting the maximum output frequency. The proposed light-to-frequency sensor chip is designed and fabricated with a 0.35- µm CMOS process. The overall chip area is 1 × 0.9 mm 2 and the typical total current consumption is about 1.8 mA from a 3.3-V power supply at room temperature. The measurement results prove the proposed functionality of output pulse duty cycle modulation, while the SNR of a typical 10-kHz output frequency is 59 dB with about 9-dB improvement when compared with the previous design. Among them, 2-3 dB SNR improvement stems from the gm-boosting and the rest comes from the layout design. In-system experimental results show that the minimum measurable PI using the proposed blood SpO 2 sensor could be as low as 0.06% with 2-percentage-point error of SpO 2. The proposed chip is suitable for portable low-power high-performance blood oximeter devices especially for hypoperfusion applications.

A Deep Learning Approach for Targeted Contrast-Enhanced Ultrasound Based Prostate Cancer Detection.

The important role of angiogenesis in cancer development has driven many researchers to investigate the prospects of noninvasive cancer diagnosis based on the technology of contrast-enhanced ultrasound (CEUS) imaging. This paper presents a deep learning framework to detect prostate cancer in the sequential CEUS images. The proposed method uniformly extracts features from both the spatial and the temporal dimensions by performing three-dimensional convolution operations, which captures the dynamic information of the perfusion process encoded in multiple adjacent frames for prostate cancer detection. The deep learning models were trained and validated against expert delineations over the CEUS images recorded using two types of contrast agents, i.e., the anti-PSMA based agent targeted to prostate cancer cells and the non-targeted blank agent. Experiments showed that the deep learning method achieved over 91 percent specificity and 90 percent average accuracy over the targeted CEUS images for prostate cancer detection, which was superior ( ) than previously reported approaches and implementations.

Alterations in patients with major depressive disorder before and after electroconvulsive therapy measured by fractional amplitude of low-frequency fluctuations (fALFF).

BACKGROUND: Electroconvulsive therapy (ECT) is an important treatment option for patients with major depressive disorder (MDD). However, the mechanisms of ECT in MDD are still unclear. METHODS: Twenty-four patients with severe MDD and 14 healthy controls were enrolled in this study. Eight ECT sessions were conducted for MDD patients using brief-pulse square-wave signal at bitemporal locations. To investigate the regional cerebral blood flow in MDD patients before and after ECT treatments by resting-state functional magnetic resonance imaging (rs-fMRI), the patients were scanned twice (before the first ECT and after the eighth ECT) for data acquisition. Afterward, we adopted fractional amplitude of low-frequency fluctuations (fALFF) to assess the alterations of regional brain activity. RESULTS: Compared with healthy controls, the fALFF in the cerebellum lobe, parahippocampal gyrus, fusiform gyrus, anterior cingulate gyrus, and thalamus in MDD patients before ECT (pre-ECT) was significantly increased. In another comparison, the fALFF in the cerebellum anterior lobe, fusiform gyrus, insula, parahippocampal gyrus, middle frontal gyrus, and inferior frontal gyrus in pre-ECT patients was significantly greater than the post-ECT fALFF. LIMITATIONS: Only two rs-fMRI scans were conducted at predefined times: before the first and after the eighth ECT treatment. More scans during the ECT sessions would yield more information. In addition, the sample size in this study was limited. The number of control subjects was relatively small. A larger number of subjects would produce more robust findings. CONCLUSIONS: The fALFF of both healthy controls and post-ECT patients in cerebellum anterior lobe, fusiform gyrus, and parahippocampal gyrus is significantly lower than the fALFF of pre-ECT patients. This finding demonstrates that ECT treatment is effective on these brain areas in MDD patients.

A Low Power and Fast Tracking Light-to-Frequency Converter With Adaptive Power Scaling for Blood SpO 2 Sensing.

This paper presents a monolithic low power and fast tracking light-to-frequency converter for blood SpO 2 sensing. Normally, the tracking speed and the power consumption are two contradictory characteristics. However, different gain-bandwidth specifications for various ambient light intensities allow the dynamic optimization of the power consumption according to the light intensity. In this paper, the amplifier power consumption is adaptively scaled by the generated light-intensity-positively-correlated control voltage. Thus, the chip total power consumption at low light intensity is significantly decreased. Moreover, the proposed adaptive power scaling is achieved with a continuous analog domain, which does not introduce extra switching noise. The proposed light-to-frequency sensor chip is fabricated by using 0.35  µm CMOS technology with a die area of 1 × 0.9 mm 2. The measurement results show that the pulse light response for any light intensity is no longer than two new output square-wave cycles. The maximum total current consumption is 1.9 mA from a 3.3 V supply voltage, which can be adaptively scaled down to only 0.7 mA if the output frequency is about 25 KHz or lower. The minimum operational supply voltage of the proposed sensor chip is 2.5 V in the temperature range of -25 to 80  °C with 4 KV ESD level (human-body model).

DANoC: An Efficient Algorithm and Hardware Codesign of Deep Neural Networks on Chip.

Deep neural networks (NNs) are the state-of-the-art models for understanding the content of images and videos. However, implementing deep NNs in embedded systems is a challenging task, e.g., a typical deep belief network could exhaust gigabytes of memory and result in bandwidth and computational bottlenecks. To address this challenge, this paper presents an algorithm and hardware codesign for efficient deep neural computation. A hardware-oriented deep learning algorithm, named the deep adaptive network, is proposed to explore the sparsity of neural connections. By adaptively removing the majority of neural connections and robustly representing the reserved connections using binary integers, the proposed algorithm could save up to 99.9% memory utility and computational resources without undermining classification accuracy. An efficient sparse-mapping-memory-based hardware architecture is proposed to fully take advantage of the algorithmic optimization. Different from traditional Von Neumann architecture, the deep-adaptive network on chip (DANoC) brings communication and computation in close proximity to avoid power-hungry parameter transfers between on-board memory and on-chip computational units. Experiments over different image classification benchmarks show that the DANoC system achieves competitively high accuracy and efficiency comparing with the state-of-the-art approaches.

A Spatial-Temporal Method to Detect Global Influenza Epidemics Using Heterogeneous Data Collected from the Internet.

The 2009 influenza pandemic teaches us how fast the influenza virus could spread globally within a short period of time. To address the challenge of timely global influenza surveillance, this paper presents a spatial-temporal method that incorporates heterogeneous data collected from the Internet to detect influenza epidemics in real time. Specifically, the influenza morbidity data, the influenza-related Google query data and news data, and the international air transportation data are integrated in a multivariate hidden Markov model, which is designed to describe the intrinsic temporal-geographical correlation of influenza transmission for surveillance purpose. Respective models are built for 106 countries and regions in the world. Despite that the WHO morbidity data are not always available for most countries, the proposed method achieves 90.26 to 97.10 percent accuracy on average for real-time detection of global influenza epidemics during the period from January 2005 to December 2015. Moreover, experiment shows that, the proposed method could even predict an influenza epidemic before it occurs with 89.20 percent accuracy on average. Timely international surveillance results may help the authorities to prevent and control the influenza disease at the early stage of a global influenza pandemic.

A multichannel decision-level fusion method for T wave alternans detection.

Sudden cardiac death (SCD) is one of the most prominent causes of death among patients with cardiac diseases. Since ventricular arrhythmia is the main cause of SCD and it can be predicted by T wave alternans (TWA), the detection of TWA in the body-surface electrocardiograph (ECG) plays an important role in the prevention of SCD. But due to the multi-source nature of TWA, the nonlinear propagation through thorax, and the effects of the strong noises, the information from different channels is uncertain and competitive with each other. As a result, the single-channel decision is one-sided while the multichannel decision is difficult to reach a consensus on. In this paper, a novel multichannel decision-level fusion method based on the Dezert-Smarandache Theory is proposed to address this issue. Due to the redistribution mechanism for highly competitive information, higher detection accuracy and robustness are achieved. It also shows promise to low-cost instruments and portable applications by reducing demands for the synchronous sampling. Experiments on the real records from the Physikalisch-Technische Bundesanstalt diagnostic ECG database indicate that the performance of the proposed method improves by 12%-20% compared with the one-dimensional decision method based on the periodic component analysis.

Monitoring epidemic alert levels by analyzing Internet search volume.

The prevention of infectious diseases is a global health priority area. The early detection of possible epidemics is the first and important defense line against infectious diseases. However, conventional surveillance systems, e.g., the Centers for Disease Control and Prevention (CDC), rely on clinical data. The CDC publishes the surveillance results weeks after epidemic outbreaks. To improve the early detection of epidemic outbreaks, we designed a syndromic surveillance system to predict the epidemic trends based on disease-related Google search volume. Specifically, we first represented the epidemic trend with multiple alert levels to reduce the noise level. Then, we predicted the epidemic alert levels using a continuous density HMM, which incorporated the intrinsic characteristic of the disease transmission for alert level estimation. Respective models are built to monitor both national and regional epidemic alert levels of the U.S. The proposed system can provide real-time surveillance results, which are weeks before the CDC's reports. This paper focusses on monitoring the infectious disease in the U.S., however, we believe similar approach may be used to monitor epidemics for the developing countries as well.

Robust suppression of nonstationary power-line interference in electrocardiogram signals.

It is a challenge to suppress time-varying power-line interference (PLI) with various levels in electrocardiogram (ECG) signals. Most previous attempts of tracking and suppressing the nonstationary PLI signal are based on the least-squares (LS) algorithm. This makes these methods susceptible to QRS complex in suppressing a low-level PLI signal which is frequently coupled in battery-operated ECG equipment. To address the limitation of LS-based methods, this study presents a robust PLI suppression system based on a robust extension of the Kalman filter. In addition, we used an improved version of empirical mode decomposition to further attenuate the QRS complex. Experiments show that our system could effectively suppress the PLI while preserving meaningful ECG components at various interference levels.

Tuberculosis surveillance by analyzing Google trends.

Tuberculosis (TB) is a major global health concern, causing nearly ten million new cases and over one million deaths every year. The early detection of possible epidemic is the first and important defense line against tuberculosis. However, traditional surveillance approaches, e.g. US Centers for Disease Control and Prevention (CDC), publish the TB morbidity surveillance results on a quarterly basis, with months of reporting lag. Moreover, in some developing countries, where most infections occur, there may not be enough medical resources to build traditional surveillance systems. To improve early detection of tuberculosis outbreaks, we developed a syndromic approach to estimate the actual number of TB cases using Google Search Volume. Specifically, the search volume of nineteen TB-related terms, obtained from January 2004 to April 2009, were examined for surveillance purpose. Contemporary TB surveillance data were extracted from the CDCs reports to build and evaluate the syndromic system. We estimate the actual TB occurrences using a non-stationary dynamic system. Respective models are built to monitor both national-level and state-level TB activities. The surveillance results of the syndromic system can be updated every day, which is twelve weeks ahead of CDCs reports.