Sharing massive biomedical data at magnitudes lower bandwidth using implicit neural function.

Efficient storage and sharing of massive biomedical data would open up their wide accessibility to different institutions and disciplines. However, compressors tailored for natural photos/videos are rapidly limited for biomedical data, while emerging deep learning-based methods demand huge training data and are difficult to generalize. Here, we propose to conduct Biomedical data compRession with Implicit nEural Function (BRIEF) by representing the target data with compact neural networks, which are data specific and thus have no generalization issues. Benefiting from the strong representation capability of implicit neural function, BRIEF achieves 2[Formula: see text]3 orders of magnitude compression on diverse biomedical data at significantly higher fidelity than existing techniques. Besides, BRIEF is of consistent performance across the whole data volume, and supports customized spatially varying fidelity. BRIEF's multifold advantageous features also serve reliable downstream tasks at low bandwidth. Our approach will facilitate low-bandwidth data sharing and promote collaboration and progress in the biomedical field.

Comparative analysis of optional hunting behavior in Cricetinae hamsters using the data compression approach.

Research into the hunting behavior in members of the Cricetidae family offers an opportunity to reveal what changes in the predatory behavioral sequences occur when a rodent species shifts from an omnivorous to a predatory lifestyle. The study tests the following hypotheses: are there phylogenetic differences in the divergence of species' predatory lifestyles in hamsters or do ecological factors lead to shaping their hunting behavior? We applied the data compression approach for performing comparative analysis of hunting patterns as biological "texts." The study presents a comparative analysis of hunting behaviors in five Cricetinae species, focusing on the new data obtained for the desert hamster Phodopus roborovskii whose behavior has never been studied before. The hunting behavior of P. roborovskii appeared to be the most variable one. In contrast, behavioral sequences in P. campbelli and Allocricetulus curtatus display more significant order and predictability of behavior during hunting. Optional hunting behavior in the most ancient species P. roborovskii displayed similarities with obligate patterns in "young" Allocricetulus species. It thus turned out to be the most advanced hunter among members of the Phodopus genus. Differences in hunting sequences among Phodopus representatives suggest that the hunting behavior of these species, despite its optional mode, was subject to selection during species splitting within the genus. These results did not reveal the role played by phylogenetic differences in the divergence of species' predatory lifestyles. They suggested that ecological conditions are the main factors in speciation of the hunting behavior in hamsters.

Variational Autoencoders for Network Lifetime Enhancement in Wireless Sensors.

Wireless sensor networks (WSNs) are structured for monitoring an area with distributed sensors and built-in batteries. However, most of their battery energy is consumed during the data transmission process. In recent years, several methodologies, like routing optimization, topology control, and sleep scheduling algorithms, have been introduced to improve the energy efficiency of WSNs. This study introduces a novel method based on a deep learning approach that utilizes variational autoencoders (VAEs) to improve the energy efficiency of WSNs by compressing transmission data. The VAE approach is customized in this work for compressing WSN data by retaining its important features. This is achieved by analyzing the statistical structure of the sensor data rather than providing a fixed-size latent representation. The performance of the proposed model is verified using a MATLAB simulation platform, integrating a pre-trained variational autoencoder model with openly available wireless sensor data. The performance of the proposed model is found to be satisfactory in comparison to traditional methods, like the compressed sensing technique, lightweight temporal compression, and the autoencoder, in terms of having an average compression rate of 1.5572. The WSN simulation also indicates that the VAE-incorporated architecture attains a maximum network lifetime of 1491 s and suggests that VAE could be used for compression-based transmission using WSNs, as its reconstruction rate is 0.9902, which is better than results from all the other techniques.

A Survey on Data Compression Techniques for Automotive LiDAR Point Clouds.

In the evolving landscape of autonomous driving technology, Light Detection and Ranging (LiDAR) sensors have emerged as a pivotal instrument for enhancing environmental perception. They can offer precise, high-resolution, real-time 3D representations around a vehicle, and the ability for long-range measurements under low-light conditions. However, these advantages come at the cost of the large volume of data generated by the sensor, leading to several challenges in transmission, processing, and storage operations, which can be currently mitigated by employing data compression techniques to the point cloud. This article presents a survey of existing methods used to compress point cloud data for automotive LiDAR sensors. It presents a comprehensive taxonomy that categorizes these approaches into four main groups, comparing and discussing them across several important metrics.

Efficient Data Management in Agricultural IoT: Compression, Security, and MQTT Protocol Analysis.

The integration of Internet of Things (IoT) technology into agriculture has revolutionized farming practices by using connected devices and sensors to optimize processes and facilitate sustainable execution. Because most IoT devices have limited resources, the vital requirement to efficiently manage data traffic while ensuring data security in agricultural IoT solutions creates several challenges. Therefore, it is important to study the data amount that IoT protocols generate for resource-constrained devices, as it has a direct impact on the device performance and overall usability of the IoT solution. In this paper, we present a comprehensive study that focuses on optimizing data transmission in agricultural IoT solutions with the use of compression algorithms and secure technologies. Through experimentation and analysis, we evaluate different approaches to minimize data traffic while protecting sensitive agricultural data. Our results highlight the effectiveness of compression algorithms, especially Huffman coding, in reducing data size and optimizing resource usage. In addition, the integration of encryption techniques, such as AES, provides the security of the transmitted data without incurring significant overhead. By assessing different communication scenarios, we identify the most efficient approach, a combination of Huffman encoding and AES encryption, to strike a balance between data security and transmission efficiency.

A Data Compression Method for Wellbore Stability Monitoring Based on Deep Autoencoder.

The compression method for wellbore trajectory data is crucial for monitoring wellbore stability. However, classical methods like methods based on Huffman coding, compressed sensing, and Differential Pulse Code Modulation (DPCM) suffer from low real-time performance, low compression ratios, and large errors between the reconstructed data and the source data. To address these issues, a new compression method is proposed, leveraging a deep autoencoder for the first time to significantly improve the compression ratio. Additionally, the method reduces error by compressing and transmitting residual data from the feature extraction process using quantization coding and Huffman coding. Furthermore, a mean filter based on the optimal standard deviation threshold is applied to further minimize error. Experimental results show that the proposed method achieves an average compression ratio of 4.05 for inclination and azimuth data; compared to the DPCM method, it is improved by 118.54%. Meanwhile, the average mean square error of the proposed method is 76.88, which is decreased by 82.46% when compared to the DPCM method. Ablation studies confirm the effectiveness of the proposed improvements. These findings highlight the efficacy of the proposed method in enhancing wellbore stability monitoring performance.

Building Test Batteries Based on Analyzing Random Number Generator Tests within the Framework of Algorithmic Information Theory.

The problem of testing random number generators is considered and a new method for comparing the power of different statistical tests is proposed. It is based on the definitions of random sequence developed in the framework of algorithmic information theory and allows comparing the power of different tests in some cases when the available methods of mathematical statistics do not distinguish between tests. In particular, it is shown that tests based on data compression methods using dictionaries should be included in test batteries.

PMFFRC: a large-scale genomic short reads compression optimizer via memory modeling and redundant clustering.

BACKGROUND: Genomic sequencing reads compressors are essential for balancing high-throughput sequencing short reads generation speed, large-scale genomic data sharing, and infrastructure storage expenditure. However, most existing short reads compressors rarely utilize big-memory systems and duplicative information between diverse sequencing files to achieve a higher compression ratio for conserving reads data storage space. RESULTS: We employ compression ratio as the optimization objective and propose a large-scale genomic sequencing short reads data compression optimizer, named PMFFRC, through novelty memory modeling and redundant reads clustering technologies. By cascading PMFFRC, in 982 GB fastq format sequencing data, with 274 GB and 3.3 billion short reads, the state-of-the-art and reference-free compressors HARC, SPRING, Mstcom, and FastqCLS achieve 77.89%, 77.56%, 73.51%, and 29.36% average maximum compression ratio gains, respectively. PMFFRC saves 39.41%, 41.62%, 40.99%, and 20.19% of storage space sizes compared with the four unoptimized compressors. CONCLUSIONS: PMFFRC rational usage big-memory of compression server, effectively saving the sequencing reads data storage space sizes, which relieves the basic storage facilities costs and community sharing transmitting overhead. Our work furnishes a novel solution for improving sequencing reads compression and saving storage space. The proposed PMFFRC algorithm is packaged in a same-name Linux toolkit, available un-limited at https://github.com/fahaihi/PMFFRC .

Supervised topological data analysis for MALDI mass spectrometry imaging applications.

BACKGROUND: Matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) displays significant potential for applications in cancer research, especially in tumor typing and subtyping. Lung cancer is the primary cause of tumor-related deaths, where the most lethal entities are adenocarcinoma (ADC) and squamous cell carcinoma (SqCC). Distinguishing between these two common subtypes is crucial for therapy decisions and successful patient management. RESULTS: We propose a new algebraic topological framework, which obtains intrinsic information from MALDI data and transforms it to reflect topological persistence. Our framework offers two main advantages. Firstly, topological persistence aids in distinguishing the signal from noise. Secondly, it compresses the MALDI data, saving storage space and optimizes computational time for subsequent classification tasks. We present an algorithm that efficiently implements our topological framework, relying on a single tuning parameter. Afterwards, logistic regression and random forest classifiers are employed on the extracted persistence features, thereby accomplishing an automated tumor (sub-)typing process. To demonstrate the competitiveness of our proposed framework, we conduct experiments on a real-world MALDI dataset using cross-validation. Furthermore, we showcase the effectiveness of the single denoising parameter by evaluating its performance on synthetic MALDI images with varying levels of noise. CONCLUSION: Our empirical experiments demonstrate that the proposed algebraic topological framework successfully captures and leverages the intrinsic spectral information from MALDI data, leading to competitive results in classifying lung cancer subtypes. Moreover, the framework's ability to be fine-tuned for denoising highlights its versatility and potential for enhancing data analysis in MALDI applications.

Dimensional reduction in complex living systems: Where, why, and how.

The unprecedented prowess of measurement techniques provides a detailed, multi-scale look into the depths of living systems. Understanding these avalanches of high-dimensional data-by distilling underlying principles and mechanisms-necessitates dimensional reduction. We propose that living systems achieve exquisite dimensional reduction, originating from their capacity to learn, through evolution and phenotypic plasticity, the relevant aspects of a non-random, smooth physical reality. We explain how geometric insights by mathematicians allow one to identify these genuine hallmarks of life and distinguish them from universal properties of generic data sets. We illustrate these principles in a concrete example of protein evolution, suggesting a simple general recipe that can be applied to understand other biological systems.

Compression Performance Analysis of Experimental Holographic Data Coding Systems.

It is challenging to find a proper way to compress computer-generated holography (CGH) data owing to their huge data requirements and characteristics. This study proposes CGH data coding systems with high-efficiency video coding (HEVC), three-dimensional extensions of HEVC (3D-HEVC), and video-based point cloud compression (V-PCC) codecs. In the proposed system, we implemented a procedure for codec usage and format conversion and evaluated the objective and subjective results to analyze the performance of the three coding systems. We discuss the relative advantages and disadvantages of the three coding systems with respect to their coding efficiency and reconstruction results. Our analysis concluded that 3D-HEVC and V-PCC are potential solutions for compressing red, green, blue, and depth (RGBD)-sourced CGH data.

Unmanned Aerial Vehicle-Based Compressed Data Acquisition for Environmental Monitoring in WSNs.

With the increasing concerns for the environment, the amount of the data monitored by wireless sensor networks (WSNs) is becoming larger and the energy required for data transmission is greater. However, sensor nodes have limited storage capacity and battery power. The WSNs are faced with the challenge of handling larger data volumes while minimizing energy consumption for transmission. To address this issue, this paper employs data compression technology to eliminate redundant information in the environmental data, thereby reducing energy consumption of sensor nodes. Additionally, an unmanned aerial vehicle (UAV)-assisted compressed data acquisition algorithm is put forward. In this algorithm, compressive sensing (CS) is introduced to decrease the amount of data in the network and the UAV serves as a mobile aerial base station for efficient data gathering. Based on CS theory, the UAV selectively collects measurements from a subset of sensor nodes along a route planned using the optimized greedy algorithm with variation and insertion strategies. Once the UAV returns, the sink node reconstructs sensory data from these measurements using the reconstruction algorithms. Extensive experiments are conducted to verify the performance of this algorithm. Experimental results show that the proposed algorithm has lower energy consumption compared to other approaches. Furthermore, we employ different data reconstruction algorithms to recover data and discover that the data can be better reconstructed in a shorter time.

Design of Edge-IoMT Network Architecture with Weight-Based Scheduling.

Population health monitoring based on the Internet of Medical Things (IoMT) is becoming an important application trend healthcare improvement. This work aims to develop an autonomous network architecture, collecting sensor data with a cluster topology, forwarding information through relay nodes, and applying edge computing and transmission scheduling for network scalability and operational efficiency. The proposed distributed network architecture incorporates data compression technologies and effective scheduling algorithms for handling the transmission scheduling of various physiological signals. Compared to existing scheduling mechanisms, the experimental results depict the network performance and show that in analyzing the delay and jitter, the proposed WFQ-based algorithms have reduced the delay and jitter ratio by about 40% and 19.47% compared to LLQ with priority queueing scheme, respectively. The experimental results also demonstrate that the proposed network topology is more effective than the direct path transmission approach in terms of energy consumption, which suggests that the proposed network architecture may improve the development of medical applications with body area networks such that the goal of self-organizing population health monitoring can be achieved.

Deep Learning-Based Adaptive Compression and Anomaly Detection for Smart B5G Use Cases Operation.

The evolution towards next-generation Beyond 5G (B5G) networks will require not only innovation in transport technologies but also the adoption of smarter, more efficient operations of the use cases that are foreseen to be the high consumers of network resources in the next decades. Among different B5G use cases, the Digital Twin (DT) has been identified as a key high bandwidth-demanding use case. The creation and operation of a DT require the continuous collection of an enormous and widely distributed amount of sensor telemetry data which can overwhelm the transport layer. Therefore, the reduction in such transported telemetry data is an essential objective of smart use case operation. Moreover, deep telemetry data analysis, i.e., anomaly detection, can be executed in a hierarchical way to reduce the processing needed to perform such analysis in a centralized way. In this paper, we propose a smart management system consisting of a hierarchical architecture for telemetry sensor data analysis using deep autoencoders (AEs). The system contains AE-based methods for the adaptive compression of telemetry time series data using pools of AEs (called AAC), as well as for anomaly detection in single (called SS-AD) and multiple (called MS-AGD) sensor streams. Numerical results using experimental telemetry data show compression ratios of up to 64% with reconstruction errors of less than 1%, clearly improving upon the benchmark state-of-the-art methods. In addition, fast and accurate anomaly detection is demonstrated for both single and multiple-sensor scenarios. Finally, a great reduction in transport network capacity resources of 50% and more is obtained by smart use case operation for distributed DT scenarios.

Context-Aware Lossless and Lossy Compression of Radio Frequency Signals.

We propose an algorithm based on linear prediction that can perform both the lossless and near-lossless compression of RF signals. The proposed algorithm is coupled with two signal detection methods to determine the presence of relevant signals and apply varying levels of loss as needed. The first method uses spectrum sensing techniques, while the second one takes advantage of the error computed in each iteration of the Levinson-Durbin algorithm. These algorithms have been integrated as a new pre-processing stage into FAPEC, a data compressor first designed for space missions. We test the lossless algorithm using two different datasets. The first one was obtained from OPS-SAT, an ESA CubeSat, while the second one was obtained using a SDRplay RSPdx in Barcelona, Spain. The results show that our approach achieves compression ratios that are 23% better than gzip (on average) and very similar to those of FLAC, but at higher speeds. We also assess the performance of our signal detectors using the second dataset. We show that high ratios can be achieved thanks to the lossy compression of the segments without any relevant signal.

Study of the Impact of Data Compression on the Energy Consumption Required for Data Transmission in a Microcontroller-Based System.

As the number of Internet of Things (IoT) devices continues to rise dramatically each day, the data generated and transmitted by them follow similar trends. Given that a significant portion of these embedded devices operate on battery power, energy conservation becomes a crucial factor in their design. This paper aims to investigate the impact of data compression on the energy consumption required for data transmission. To achieve this goal, we conduct a comprehensive study using various transmission modules in a severely resource-limited microcontroller-based system designed for battery power. Our study evaluates the performance of several compression algorithms, conducting a detailed analysis of computational and memory complexity, along with performance metrics. The primary finding of our study is that by carefully selecting an algorithm for compressing different types of data before transmission, a significant amount of energy can be saved. Moreover, our investigation demonstrates that for a battery-powered embedded device transmitting sensor data based on the STM32F411CE microcontroller, the recommended transmission module is the nRF24L01+ board, as it requires the least amount of energy to transmit one byte of data. This module is most effective when combined with the LZ78 algorithm for optimal energy and time efficiency. In the case of image data, our findings indicate that the use of the JPEG algorithm for compression yields the best results. Overall, our research underscores the importance of selecting appropriate compression algorithms tailored to specific data types, contributing to enhanced energy efficiency in IoT devices.

Over the Limits of Traditional Sampling: Advantages and Issues of AICs for Measurement Instrumentation.

Data acquisition systems have shown the need of wideband spectrum monitoring for many years. This paper describes and discusses a recently proposed architecture aimed at acquiring efficiently wideband signals, named the Analog-to-Information Converter (AIC). AIC framework and working principle implementing the sub-Nyquist sampling are analyzed in general terms. Attention is specifically focused on the idea of exploiting the condition of the signals that, despite their large bandwidth, have a small information content in the frequency domain. However, as clarified in the paper, employing AICs in measurement instrumentation necessarily entails their characterization, through the analysis of their building blocks and the corresponding non-idealities, in order to improve the signal reconstruction.

Improved Recurrence Plots Compression Distance by Learning Parameter for Video Compression Quality.

As the Internet-of-Things is deployed widely, many time-series data are generated everyday. Thus, classifying time-series automatically has become important. Compression-based pattern recognition has attracted attention, because it can analyze various data universally with few model parameters. RPCD (Recurrent Plots Compression Distance) is known as a compression-based time-series classification method. First, RPCD transforms time-series data into an image called "Recurrent Plots (RP)". Then, the distance between two time-series data is determined as the dissimilarity between their RPs. Here, the dissimilarity between two images is computed from the file size, when an MPEG-1 encoder compresses the video, which serializes the two images in order. In this paper, by analyzing the RPCD, we give an important insight that the quality parameter for the MPEG-1 encoding that controls the resolution of compressed videos influences the classification performance very much. We also show that the optimal parameter value depends extremely on the dataset to be classified: Interestingly, the optimal value for one dataset can make the RPCD fall behind a naive random classifier for another dataset. Supported by these insights, we propose an improved version of RPCD named qRPCD, which searches the optimal parameter value by means of cross-validation. Experimentally, qRPCD works superiorly to the original RPCD by about 4% in terms of classification accuracy.

Unconditionally Secure Ciphers with a Short Key for a Source with Unknown Statistics.

We consider the problem of constructing an unconditionally secure cipher with a short key for the case where the probability distribution of encrypted messages is unknown. Note that unconditional security means that an adversary with no computational constraints can only obtain a negligible amount of information ("leakage") about an encrypted message (without knowing the key). Here, we consider the case of a priori (partially) unknown message source statistics. More specifically, the message source probability distribution belongs to a given family of distributions. We propose an unconditionally secure cipher for this case. As an example, one can consider constructing a single cipher for texts written in any of the languages of the European Union. That is, the message to be encrypted could be written in any of these languages.

The Cauchy Distribution in Information Theory.

The Gaussian law reigns supreme in the information theory of analog random variables. This paper showcases a number of information theoretic results which find elegant counterparts for Cauchy distributions. New concepts such as that of equivalent pairs of probability measures and the strength of real-valued random variables are introduced here and shown to be of particular relevance to Cauchy distributions.