Recording of elapsed time and temporal information about biological events using Cas9.

DNA has not been utilized to record temporal information, although DNA has been used to record biological information and to compute mathematical problems. Here, we found that indel generation by Cas9 and guide RNA can occur at steady rates, in contrast to typical dynamic biological reactions, and the accumulated indel frequency can be a function of time. By measuring indel frequencies, we developed a method for recording and measuring absolute time periods over hours to weeks in mammalian cells. These time-recordings were conducted in several cell types, with different promoters and delivery vectors for Cas9, and in both cultured cells and cells of living mice. As applications, we recorded the duration of chemical exposure and the lengths of elapsed time since the onset of biological events (e.g., heat exposure and inflammation). We propose that our systems could serve as synthetic "DNA clocks."

A Novel Electrical Equipment Status Diagnosis Method Based on Super-Resolution Reconstruction and Logical Reasoning.

The accurate detection of electrical equipment states and faults is crucial for the reliable operation of such equipment and for maintaining the health of the overall power system. The state of power equipment can be effectively monitored through deep learning-based visual inspection methods, which provide essential information for diagnosing and predicting equipment failures. However, there are significant challenges: on the one hand, electrical equipment typically operates in complex environments, thus resulting in captured images that contain environmental noise, which significantly reduces the accuracy of state recognition based on visual perception. This, in turn, affects the comprehensiveness of the power system's situational awareness. On the other hand, visual perception is limited to obtaining the appearance characteristics of the equipment. The lack of logical reasoning makes it difficult for purely visual analysis to conduct a deeper analysis and diagnosis of the complex equipment state. Therefore, to address these two issues, we first designed an image super-resolution reconstruction method based on the Generative Adversarial Network (GAN) to filter environmental noise. Then, the pixel information is analyzed using a deep learning-based method to obtain the spatial feature of the equipment. Finally, by constructing the logic diagram for electrical equipment clusters, we propose an interpretable fault diagnosis method that integrates the spatial features and temporal states of the electrical equipment. To verify the effectiveness of the proposed algorithm, extensive experiments are conducted on six datasets. The results demonstrate that the proposed method can achieve high accuracy in diagnosing electrical equipment faults.

Diverse Metrics for Robust LBS Privacy: Distance, Semantics, and Temporal Factors.

Addressing inherent limitations in distinguishing metrics relying solely on Euclidean distance, especially within the context of geo-indistinguishability (Geo-I) as a protection mechanism for location-based service (LBS) privacy, this paper introduces an innovative and comprehensive metric. Our proposed metric not only incorporates geographical information but also integrates semantic, temporal, and query data, serving as a powerful tool to foster semantic diversity, ensure high servifice similarity, and promote spatial dispersion. We extensively evaluate our technique by constructing a comprehensive metric for Dongcheng District, Beijing, using road network data obtained through the OSMNX package and semantic and temporal information acquired through Gaode Map. This holistic approach proves highly effective in mitigating adversarial attacks based on background knowledge. Compared with existing methods, our proposed protection mechanism showcases a minimum 50% reduction in service quality and an increase of at least 0.3 times in adversarial attack error using a real-world dataset from Geolife. The simulation results underscore the efficacy of our protection mechanism in significantly enhancing user privacy compared to existing methodologies in the LBS location privacy-protection framework. This adjustment more fully reflects the authors' preference while maintaining clarity about the role of Geo-I as a protection mechanism within the broader framework of LBS location privacy protection.

Collaborative Joint Perception and Prediction for Autonomous Driving.

Collaboration among road agents, such as connected autonomous vehicles and roadside units, enhances driving performance by enabling the exchange of valuable information. However, existing collaboration methods predominantly focus on perception tasks and rely on single-frame static information sharing, which limits the effective exchange of temporal data and hinders broader applications of collaboration. To address this challenge, we propose CoPnP, a novel collaborative joint perception and prediction system, whose core innovation is to realize multi-frame spatial-temporal information sharing. To achieve effective and communication-efficient information sharing, two novel designs are proposed: (1) a task-oriented spatial-temporal information-refinement model, which filters redundant and noisy multi-frame features into concise representations; (2) a spatial-temporal importance-aware feature-fusion model, which comprehensively fuses features from various agents. The proposed CoPnP expands the benefits of collaboration among road agents to the joint perception and prediction task. The experimental results demonstrate that CoPnP outperforms existing state-of-the-art collaboration methods, achieving a significant performance-communication trade-off and yielding up to 11.51%/10.34% Intersection over union and 12.31%/10.96% video panoptic quality gains over single-agent PnP on the OPV2V/V2XSet datasets.

Link Prediction in Dynamic Social Networks Combining Entropy, Causality, and a Graph Convolutional Network Model.

Link prediction is recognized as a crucial means to analyze dynamic social networks, revealing the principles of social relationship evolution. However, the complex topology and temporal evolution characteristics of dynamic social networks pose significant research challenges. This study introduces an innovative fusion framework that incorporates entropy, causality, and a GCN model, focusing specifically on link prediction in dynamic social networks. Firstly, the framework preprocesses the raw data, extracting and recording timestamp information between interactions. It then introduces the concept of "Temporal Information Entropy (TIE)", integrating it into the Node2Vec algorithm's random walk to generate initial feature vectors for nodes in the graph. A causality analysis model is subsequently applied for secondary processing of the generated feature vectors. Following this, an equal dataset is constructed by adjusting the ratio of positive and negative samples. Lastly, a dedicated GCN model is used for model training. Through extensive experimentation in multiple real social networks, the framework proposed in this study demonstrated a better performance than other methods in key evaluation indicators such as precision, recall, F1 score, and accuracy. This study provides a fresh perspective for understanding and predicting link dynamics in social networks and has significant practical value.

Integration of Sequencing and Epidemiologic Data for Surveillance of Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) Infections in a Tertiary-Care Hospital.

BACKGROUND: The ongoing coronavirus disease 2019 pandemic significantly burdens hospitals and other healthcare facilities. Therefore, understanding the entry and transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is critical for effective prevention and preparedness measures. We performed surveillance and analysis of testing and transmission of SARS-CoV-2 infections in a tertiary-care hospital in Germany during the second and third pandemic waves in fall/winter 2020. METHODS: Between calendar week 41 in 2020 and calendar week 1 in 2021, 40%, of all positive patient and staff samples (284 total) were subjected to full-length viral genome sequencing. Clusters were defined based on similar genotypes indicating common sources of infection. We integrated phylogenetic, spatial, and temporal metadata to detect nosocomial infections and outbreaks, uncover transmission chains, and evaluate containment measures' effectiveness. RESULTS: Epidemiologic data and contact tracing readily recognize most healthcare-associated (HA) patient infections. However, sequencing data reveal that temporally preceding index cases and transmission routes can be missed using epidemiologic methods, resulting in delayed interventions and serially linked outbreaks being counted as independent events. While hospital-associated transmissions were significantly elevated at a moderate rate of community transmission during the second wave, systematic testing and high vaccination rates among staff have led to a substantial decrease in HA infections at the end of the second/beginning of the third wave despite high community transmissions. CONCLUSIONS: While epidemiologic analysis is critical for immediate containment of HA SARS-CoV-2 outbreaks, integration of genomic surveillance revealed weaknesses in identifying staff contacts. Our study underscores the importance of high testing frequency and genomic surveillance to detect, contain and prevent SARS-CoV-2-associated infections in healthcare settings.

Prediction of Shooting Events in Soccer Videos Using Complete Bipartite Graphs and Players' Spatial-Temporal Relations.

In soccer, quantitatively evaluating the performance of players and teams is essential to improve tactical coaching and players' decision-making abilities. To achieve this, some methods use predicted probabilities of shoot event occurrences to quantify player performances, but conventional shoot prediction models have not performed well and have failed to consider the reliability of the event probability. This paper proposes a novel method that effectively utilizes players' spatio-temporal relations and prediction uncertainty to predict shoot event occurrences with greater accuracy and robustness. Specifically, we represent players' relations as a complete bipartite graph, which effectively incorporates soccer domain knowledge, and capture latent features by applying a graph convolutional recurrent neural network (GCRNN) to the constructed graph. Our model utilizes a Bayesian neural network to predict the probability of shoot event occurrence, considering spatio-temporal relations between players and prediction uncertainty. In our experiments, we confirmed that the proposed method outperformed several other methods in terms of prediction performance, and we found that considering players' distances significantly affects the prediction accuracy.

Improving Multiple Pedestrian Tracking in Crowded Scenes with Hierarchical Association.

Recently, advances in detection and re-identification techniques have significantly boosted tracking-by-detection-based multi-pedestrian tracking (MPT) methods and made MPT a great success in most easy scenes. Several very recent works point out that the two-step scheme of first detection and then tracking is problematic and propose using the bounding box regression head of an object detector to realize data association. In this tracking-by-regression paradigm, the regressor directly predicts each pedestrian's location in the current frame according to its previous position. However, when the scene is crowded and pedestrians are close to each other, the small and partially occluded targets are easily missed. In this paper, we follow this pattern and design a hierarchical association strategy to obtain better performance in crowded scenes. To be specific, at the first association, the regressor is used to estimate the positions of obvious pedestrians. At the second association, we employ a history-aware mask to filter out the already occupied regions implicitly and look carefully at the remaining regions to find out the ignored pedestrians during the first association. We integrate the hierarchical association in a learning framework and directly infer the occluded and small pedestrians in an end-to-end way. We conduct extensive pedestrian tracking experiments on three public pedestrian tracking benchmarks from less crowded to crowded scenes, demonstrating the proposed strategy's effectiveness in crowded scenes.

Temporal information extraction with the scalable cross-sentence context for electronic health records.

Temporal information is essential for accurate understanding of medical information hidden in electronic health record texts. In the absence of temporal information, it is even impossible to distinguish whether the mentioned symptom is a current condition or past medical history. Hence, identifying the relationship between medical events and document creation time (DCT) is a critical component for medical language comprehension, which can link the mentioned medical information to the time dimension by marking temporal tags. Existing natural language processing (NLP) systems are typically based on the sentence where the medical event is located to extract the DCT relationship. Inevitably, the limited textual context can be insufficient as it is difficult to contain adequate document information. Introducing the surrounding sentences into models is a fitting way to enrich the information. However, in addition to document information, the added context can also bring noise to confuse the models. For effective utilization of the context, we design the DCDR (Dynamic Context and Dynamic Representation) model. Our model consists of two modules, i.e. the dynamic context mechanism and dynamic representation mechanism. The dynamic context mechanism is employed to bring the related texts into our model via the sliding windows and a scoring calculation. For the dynamic representation mechanism, a modified dynamic routing algorithm is adopted to filter the noise and generate an integrated representation for the whole context. Besides, the mentioned medical information is led into the routing process to enhance the dynamic representation module. The experiments show that our proposed model achieves improvement over existing models and achieves an F-score of 85.7% on the commonly used THYME corpus.

MEST: An Action Recognition Network with Motion Encoder and Spatio-Temporal Module.

As a sub-field of video content analysis, action recognition has received extensive attention in recent years, which aims to recognize human actions in videos. Compared with a single image, video has a temporal dimension. Therefore, it is of great significance to extract the spatio-temporal information from videos for action recognition. In this paper, an efficient network to extract spatio-temporal information with relatively low computational load (dubbed MEST) is proposed. Firstly, a motion encoder to capture short-term motion cues between consecutive frames is developed, followed by a channel-wise spatio-temporal module to model long-term feature information. Moreover, the weight standardization method is applied to the convolution layers followed by batch normalization layers to expedite the training process and facilitate convergence. Experiments are conducted on five public datasets of action recognition, Something-Something-V1 and -V2, Jester, UCF101 and HMDB51, where MEST exhibits competitive performance compared to other popular methods. The results demonstrate the effectiveness of our network in terms of accuracy, computational cost and network scales.

Tiny Vehicle Detection for Mid-to-High Altitude UAV Images Based on Visual Attention and Spatial-Temporal Information.

Mid-to-high altitude Unmanned Aerial Vehicle (UAV) imagery can provide important remote sensing information between satellite and low altitude platforms, and vehicle detection in mid-to-high altitude UAV images plays a crucial role in land monitoring and disaster relief. However, the high background complexity of images and limited pixels of objects challenge the performance of tiny vehicle detection. Traditional methods suffer from poor adaptation ability to complex backgrounds, while deep neural networks (DNNs) have inherent defects in feature extraction of tiny objects with finite pixels. To address the issue above, this paper puts forward a vehicle detection method combining the DNNs-based and traditional methods for mid-to-high altitude UAV images. We first employ the deep segmentation network to exploit the co-occurrence of the road and vehicles, then detect tiny vehicles based on visual attention mechanism with spatial-temporal constraint information. Experimental results show that the proposed method achieves effective detection of tiny vehicles in complex backgrounds. In addition, ablation experiments are performed to inspect the effectiveness of each component, and comparative experiments on tinier objects are carried out to prove the superior generalization performance of our method in detecting vehicles with a limited size of 5 × 5 pixels or less.

Adaptive Reservation of Network Resources According to Video Classification Scenes.

Video quality evaluation needs a combined approach that includes subjective and objective metrics, testing, and monitoring of the network. This paper deals with the novel approach of mapping quality of service (QoS) to quality of experience (QoE) using QoE metrics to determine user satisfaction limits, and applying QoS tools to provide the minimum QoE expected by users. Our aim was to connect objective estimations of video quality with the subjective estimations. A comprehensive tool for the estimation of the subjective evaluation is proposed. This new idea is based on the evaluation and marking of video sequences using the sentinel flag derived from spatial information (SI) and temporal information (TI) in individual video frames. The authors of this paper created a video database for quality evaluation, and derived SI and TI from each video sequence for classifying the scenes. Video scenes from the database were evaluated by objective and subjective assessment. Based on the results, a new model for prediction of subjective quality is defined and presented in this paper. This quality is predicted using an artificial neural network based on the objective evaluation and the type of video sequences defined by qualitative parameters such as resolution, compression standard, and bitstream. Furthermore, the authors created an optimum mapping function to define the threshold for the variable bitrate setting based on the flag in the video, determining the type of scene in the proposed model. This function allows one to allocate a bitrate dynamically for a particular segment of the scene and maintains the desired quality. Our proposed model can help video service providers with the increasing the comfort of the end users. The variable bitstream ensures consistent video quality and customer satisfaction, while network resources are used effectively. The proposed model can also predict the appropriate bitrate based on the required quality of video sequences, defined using either objective or subjective assessment.

Extraction of temporal relations from clinical free text: A systematic review of current approaches.

BACKGROUND: Temporal relations between clinical events play an important role in clinical assessment and decision making. Extracting such relations from free text data is a challenging task because it lies on between medical natural language processing, temporal representation and temporal reasoning. OBJECTIVES: To survey existing methods for extracting temporal relations (TLINKs) between events from clinical free text in English; to establish the state-of-the-art in this field; and to identify outstanding methodological challenges. METHODS: A systematic search in PubMed and the DBLP computer science bibliography was conducted for studies published between January 2006 and December 2018. The relevant studies were identified by examining the titles and abstracts. Then, the full text of selected studies was analyzed in depth and information were collected on TLINK tasks, TLINK types, data sources, features selection, methods used, and reported performance. RESULTS: A total of 2834 publications were identified for title and abstract screening. Of these publications, 51 studies were selected. Thirty-two studies used machine learning approaches, 15 studies used a hybrid approaches, and only four studies used a rule-based approach. The majority of studies use publicly available corpora: THYME (28 studies) and the i2b2 corpus (17 studies). CONCLUSION: The performance of TLINK extraction methods ranges widely depending on relation types and events (e.g. from 32% to 87% F-score for identifying relations between clinical events and document creation time). A small set of TLINKs (before, after, overlap and contains) has been widely studied with relatively good performance, whereas other types of TLINK (e.g., started by, finished by, precedes) are rarely studied and remain challenging. Machine learning classifiers (such as Support Vector Machine and Conditional Random Fields) and Deep Neural Networks were among the best performing methods for extracting TLINKs, but nearly all the work has been carried out and tested on two publicly available corpora only. The field would benefit from the availability of more publicly available, high-quality, annotated clinical text corpora.

Temporal Super-resolution of Ultrasound Imaging Using Matrix Completion.

The temporal super-resolution of the dynamic ultrasound imaging, a means to observe rapid heart movements, is considered an important subject in medical diagnosis of cardiac conditions. Here, a new technique based on the acquisition scheme using the matrix completion (MC) theory is offered for the temporal super-resolution of the two-dimensional (2D) and three-dimensional (3D) ultrasound imaging. MC mentions the problem of completing a low-rank matrix when only a subset of its elements can be observed. Here, the lower scan lines are acquired. Whereby, the proposed method uses temporal and spatial information of the radio frequency (RF) image sequences for the reconstruction of skipped RF lines. This is performed using the construction of the MC images and then reconstruction of them by the MC theory. The results of the proposed method are compared with the compressive sensing (CS) reconstruction methods. The qualitative and quantitative evaluations of 2D and 3D data demonstrate that in the proposed method, which uses the spatial and temporal relation of RF images and the MC theory, the reconstruction is more accurate, and the reconstruction error is lower. The computational complexity of this method is very low. It also does not require hardware adjustments. Therefore, it can be easily implemented in current ultrasound-imaging devices with the frame-rate enhancement. For instance, the frame rate up to two times the original sequence is feasible using the proposed methods, while root mean square error is decreased by about 35% and 30% for 2D and 3D data, respectively, compared with the CS reconstruction method.

Parallel group ICA+ICA: Joint estimation of linked functional network variability and structural covariation with application to schizophrenia.

There is growing evidence that rather than using a single brain imaging modality to study its association with physiological or symptomatic features, the field is paying more attention to fusion of multimodal information. However, most current multimodal fusion approaches that incorporate functional magnetic resonance imaging (fMRI) are restricted to second-level 3D features, rather than the original 4D fMRI data. This trade-off is that the valuable temporal information is not utilized during the fusion step. Here we are motivated to propose a novel approach called "parallel group ICA+ICA" that incorporates temporal fMRI information from group independent component analysis (GICA) into a parallel independent component analysis (ICA) framework, aiming to enable direct fusion of first-level fMRI features with other modalities (e.g., structural MRI), which thus can detect linked functional network variability and structural covariations. Simulation results show that the proposed method yields accurate intermodality linkage detection regardless of whether it is strong or weak. When applied to real data, we identified one pair of significantly associated fMRI-sMRI components that show group difference between schizophrenia and controls in both modalities, and this linkage can be replicated in an independent cohort. Finally, multiple cognitive domain scores can be predicted by the features identified in the linked component pair by our proposed method. We also show these multimodal brain features can predict multiple cognitive scores in an independent cohort. Overall, results demonstrate the ability of parallel GICA+ICA to estimate joint information from 4D and 3D data without discarding much of the available information up front, and the potential for using this approach to identify imaging biomarkers to study brain disorders.

Visible-Light Camera Sensor-Based Presentation Attack Detection for Face Recognition by Combining Spatial and Temporal Information.

Face-based biometric recognition systems that can recognize human faces are widely employed in places such as airports, immigration offices, and companies, and applications such as mobile phones. However, the security of this recognition method can be compromised by attackers (unauthorized persons), who might bypass the recognition system using artificial facial images. In addition, most previous studies on face presentation attack detection have only utilized spatial information. To address this problem, we propose a visible-light camera sensor-based presentation attack detection that is based on both spatial and temporal information, using the deep features extracted by a stacked convolutional neural network (CNN)-recurrent neural network (RNN) along with handcrafted features. Through experiments using two public datasets, we demonstrate that the temporal information is sufficient for detecting attacks using face images. In addition, it is established that the handcrafted image features efficiently enhance the detection performance of deep features, and the proposed method outperforms previous methods.

Deep Spatial-Temporal Joint Feature Representation for Video Object Detection.

With the development of deep neural networks, many object detection frameworks have shown great success in the fields of smart surveillance, self-driving cars, and facial recognition. However, the data sources are usually videos, and the object detection frameworks are mostly established on still images and only use the spatial information, which means that the feature consistency cannot be ensured because the training procedure loses temporal information. To address these problems, we propose a single, fully-convolutional neural network-based object detection framework that involves temporal information by using Siamese networks. In the training procedure, first, the prediction network combines the multiscale feature map to handle objects of various sizes. Second, we introduce a correlation loss by using the Siamese network, which provides neighboring frame features. This correlation loss represents object co-occurrences across time to aid the consistent feature generation. Since the correlation loss should use the information of the track ID and detection label, our video object detection network has been evaluated on the large-scale ImageNet VID dataset where it achieves a 69.5% mean average precision (mAP).

Age-Related Neural Oscillation Patterns During the Processing of Temporally Manipulated Speech.

This EEG-study aims to investigate age-related differences in the neural oscillation patterns during the processing of temporally modulated speech. Viewing from a lifespan perspective, we recorded the electroencephalogram (EEG) data of three age samples: young adults, middle-aged adults and older adults. Stimuli consisted of temporally degraded sentences in Swedish-a language unfamiliar to all participants. We found age-related differences in phonetic pattern matching when participants were presented with envelope-degraded sentences, whereas no such age-effect was observed in the processing of fine-structure-degraded sentences. Irrespective of age, during speech processing the EEG data revealed a relationship between envelope information and the theta band (4-8 Hz) activity. Additionally, an association between fine-structure information and the gamma band (30-48 Hz) activity was found. No interaction, however, was found between acoustic manipulation of stimuli and age. Importantly, our main finding was paralleled by an overall enhanced power in older adults in high frequencies (gamma: 30-48 Hz). This occurred irrespective of condition. For the most part, this result is in line with the Asymmetric Sampling in Time framework (Poeppel in Speech Commun 41:245-255, 2003), which assumes an isomorphic correspondence between frequency modulations in neurophysiological patterns and acoustic oscillations in spoken language. We conclude that speech-specific neural networks show strong stability over adulthood, despite initial processes of cortical degeneration indicated by enhanced gamma power. The results of our study therefore confirm the concept that sensory and cognitive processes undergo multidirectional trajectories within the context of healthy aging.

A new algorithmic approach for the extraction of temporal associations from clinical narratives with an application to medical product safety surveillance reports.

The sheer volume of textual information that needs to be reviewed and analyzed in many clinical settings requires the automated retrieval of key clinical and temporal information. The existing natural language processing systems are often challenged by the low quality of clinical texts and do not demonstrate the required performance. In this study, we focus on medical product safety report narratives and investigate the association of the clinical events with appropriate time information. We developed a novel algorithm for tagging and extracting temporal information from the narratives, and associating it with related events. The proposed algorithm minimizes the performance dependency on text quality by relying only on shallow syntactic information and primitive properties of the extracted event and time entities. We demonstrated the effectiveness of the proposed algorithm by evaluating its tagging and time assignment capabilities on 140 randomly selected reports from the US Vaccine Adverse Event Reporting System (VAERS) and the FDA (Food and Drug Administration) Adverse Event Reporting System (FAERS). We compared the performance of our tagger with the SUTime and HeidelTime taggers, and our algorithm's event-time associations with the Temporal Awareness and Reasoning Systems for Question Interpretation (TARSQI). We further evaluated the ability of our algorithm to correctly identify the time information for the events in the 2012 Informatics for Integrating Biology and the Bedside (i2b2) Challenge corpus. For the time tagging task, our algorithm performed better than the SUTime and the HeidelTime taggers (F-measure in VAERS and FAERS: Our algorithm: 0.86 and 0.88, SUTime: 0.77 and 0.74, and HeidelTime 0.75 and 0.42, respectively). In the event-time association task, our algorithm assigned an inappropriate timestamp for 25% of the events, while the TARSQI toolkit demonstrated a considerably lower performance, assigning inappropriate timestamps in 61.5% of the same events. Our algorithm also supported the correct calculation of 69% of the event relations to the section time in the i2b2 testing set.

Long-period rhythmic synchronous firing in a scale-free network.

Stimulus information is encoded in the spatial-temporal structures of external inputs to the neural system. The ability to extract the temporal information of inputs is fundamental to brain function. It has been found that the neural system can memorize temporal intervals of visual inputs in the order of seconds. Here we investigate whether the intrinsic dynamics of a large-size neural circuit alone can achieve this goal. The network models we consider have scale-free topology and the property that hub neurons are difficult to be activated. The latter is implemented by either including abundant electrical synapses between neurons or considering chemical synapses whose efficacy decreases with the connectivity of the postsynaptic neuron. We find that hub neurons trigger synchronous firing across the network, loops formed by low-degree neurons determine the rhythm of synchronous firing, and the hardness of exciting hub neurons avoids epileptic firing of the network. Our model successfully reproduces the experimentally observed rhythmic synchronous firing with long periods and supports the notion that the neural system can process temporal information through the dynamics of local circuits in a distributed way.