SGORNN: Combining scalar gates and orthogonal constraints in recurrent networks.

Recurrent Neural Network (RNN) models have been applied in different domains, producing high accuracies on time-dependent data. However, RNNs have long suffered from exploding gradients during training, mainly due to their recurrent process. In this context, we propose a variant of the scalar gated FastRNN architecture, called Scalar Gated Orthogonal Recurrent Neural Networks (SGORNN). SGORNN utilizes orthogonal matrices at the recurrent step. Our experiments evaluate SGORNN using two recently proposed orthogonal parametrizations for the recurrent weights of an RNN. We present a constraint on the scalar gates of SGORNN, which is easily enforced at training time to provide a probabilistic generalization gap which grows linearly with the length of sequences processed. Next, we provide bounds on the gradients of SGORNN to show the impossibility of exponentially exploding gradients through time. Our experimental results on the addition problem confirm that our combination of orthogonal and scalar gated RNNs are able to outperform other orthogonal RNNs and LSTM on long sequences. We further evaluate SGORNN on the HAR-2 classification task, where it improves upon the accuracy of several models using far fewer parameters than standard RNNs. Finally, we evaluate SGORNN on the Penn Treebank word-level language modeling task, where it again outperforms its related architectures and shows comparable performance to LSTM using far less parameters. Overall, SGORNN shows higher representation capacity than the other orthogonal RNNs tested, suffers from less overfitting than other models in our experiments, benefits from a decrease in parameter count, and alleviates exploding gradients during backpropagation through time.

A Semi-Supervised Methodology for Fishing Activity Detection Using the Geometry behind the Trajectory of Multiple Vessels.

Automatic Identification System (AIS) messages are useful for tracking vessel activity across oceans worldwide using radio links and satellite transceivers. Such data play a significant role in tracking vessel activity and mapping mobility patterns such as those found during fishing activities. Accordingly, this paper proposes a geometric-driven semi-supervised approach for fishing activity detection from AIS data. Through the proposed methodology, it is shown how to explore the information included in the messages to extract features describing the geometry of the vessel route. To this end, we leverage the unsupervised nature of cluster analysis to label the trajectory geometry, highlighting changes in the vessel's moving pattern, which tends to indicate fishing activity. The labels obtained by the proposed unsupervised approach are used to detect fishing activities, which we approach as a time-series classification task. We propose a solution using recurrent neural networks on AIS data streams with roughly 87% of the overall F-score on the whole trajectories of 50 different unseen fishing vessels. Such results are accompanied by a broad benchmark study assessing the performance of different Recurrent Neural Network (RNN) architectures. In conclusion, this work contributes by proposing a thorough process that includes data preparation, labeling, data modeling, and model validation. Therefore, we present a novel solution for mobility pattern detection that relies upon unfolding the geometry observed in the trajectory.