Assessing Perceptual Load and Cognitive Load by Fixation-Related Information of Eye Movements.

Assessing mental workload is imperative for avoiding unintended negative consequences in critical situations such as driving and piloting. To evaluate mental workload, measures of eye movements have been adopted, but unequivocal results remain elusive, especially those related to fixation-related parameters. We aimed to resolve the discrepancy of previous results by differentiating two kinds of mental workload (perceptual load and cognitive load) and manipulated them independently using a modified video game. We found opposite effects of the two kinds of mental workload on fixation-related parameters: shorter fixation durations and more fixations when participants played an episode with high (vs. low) perceptual load, and longer fixation durations and fewer fixations when they played an episode with high (vs. low) cognitive load. Such opposite effects were in line with the load theory and demonstrated that fixation-related parameters can be used to index mental workload at different (perceptual and cognitive) stages of mental processing.

Utilising Low Complexity CNNs to Lift Non-Local Redundancies in Video Coding.

Digital media is ubiquitous and produced in ever-growing quantities. This necessitates a constant evolution of compression techniques, especially for video, in order to maintain efficient storage and transmission. In this work, we aim at exploiting non-local redundancies in video data that remain difficult to erase for conventional video codecs We design convolutional neural networks with a particular emphasis on low memory and computational footprint. The parameters of those networks are trained on the fly, at encoding time, to predict the residual signal from the decoded video signal. After the training process has converged, the parameters are compressed and signalled as part of the code of the underlying video codec. The method can be applied to any existing video codec to increase coding gains while its low computational footprint allows for an application under resource-constrained conditions. Building on top of High Efficiency Video Coding, we achieve coding gains similar to those of pretrained denoising CNNs while only requiring about 1% of their computational complexity Through extensive experiments, we provide insights into the effectiveness of our network design decisions. In addition, we demonstrate that our algorithm delivers stable performance under conditions met in practical video compression: our algorithm performs without significant performance loss on very long random access segments (up to 256 frames) and with moderate performance drops can even be applied to single frames in high-resolution low delay settings.

Feasible and Robust Optimization Framework for Auxiliary Information Refinement in Spatially-Varying Image Enhancement.

In content-based image processing, the precise inference of auxiliary information dominates various image enhancement applications. Given the rough auxiliary information provided by users or inference algorithms, a common scenario is to refine it with respect to the image content. Quadratic Laplacian regularization is generally used as the refinement framework because of the availability of closed-form solutions. However, solving the resultant large linear system imposes a great burden on commodity computing hardware systems in the form of computational time and memory consumption, so efficient computing algorithms without losing precision are required, especially for large images. In this paper, we first analyze the geometric nature of the quadratic Laplacian regularization associated with the algebraic property of the corresponding linear system, which clarifies the essential issues causing ineffective solutions for conventional optimization algorithms. Correspondingly, we propose an optimization scheme that is capable of approaching the closed-form solution in an efficient manner using existing fast local filters, and we perform a spectral analysis to validate the robustness of this method in severe conditions. Finally, experimental results show that the proposed scheme is more feasible for large input images and is more robust to obtain the effective refinement than conventional algorithms.

Perceptual quality-regulable video coding system with region-based rate control scheme.

In this paper, we discuss a region-based perceptual quality-regulable H.264 video encoder system that we developed. The ability to adjust the quality of specific regions of a source video to a predefined level of quality is an essential technique for region-based video applications. We use the structural similarity index as the quality metric for distortion-quantization modeling and develop a bit allocation and rate control scheme for enhancing regional perceptual quality. Exploiting the relationship between the reconstructed macroblock and the best predicted macroblock from mode decision, a novel quantization parameter prediction method is built and used to achieve the target video quality of the processed macroblock. Experimental results show that the system model has only 0.013 quality error in average. Moreover, the proposed region-based rate control system can encode video well under a bitrate constraint with a 0.1% bitrate error in average. For the situation of the low bitrate constraint, the proposed system can encode video with a 0.5% bit error rate in average and enhance the quality of the target regions.