Appearance-based Gaze Estimation with Deep Learning: A Review and Benchmark.

Human gaze provides valuable information on human focus and intentions, making it a crucial area of research. Recently, deep learning has revolutionized appearance-based gaze estimation. However, due to the unique features of gaze estimation research, such as the unfair comparison between 2D gaze positions and 3D gaze vectors and the different pre-processing and post-processing methods, there is a lack of a definitive guideline for developing deep learning-based gaze estimation algorithms. In this paper, we present a systematic review of the appearance-based gaze estimation methods using deep learning. Firstly, we survey the existing gaze estimation algorithms along the typical gaze estimation pipeline: deep feature extraction, deep learning model design, personal calibration and platforms. Secondly, to fairly compare the performance of different approaches, we summarize the data pre-processing and post-processing methods, including face/eye detection, data rectification, 2D/3D gaze conversion and gaze origin conversion. Finally, we set up a comprehensive benchmark for deep learning-based gaze estimation. We characterize all the public datasets and provide the source code of typical gaze estimation algorithms. This paper serves not only as a reference to develop deep learning-based gaze estimation methods, but also a guideline for future gaze estimation research. The project web page can be found at https://phi-ai.buaa.edu.cn/Gazehub/.

A deep image classification model based on prior feature knowledge embedding and application in medical diagnosis.

Aiming at the problem of image classification with insignificant morphological structural features, strong target correlation, and low signal-to-noise ratio, combined with prior feature knowledge embedding, a deep learning method based on ResNet and Radial Basis Probabilistic Neural Network (RBPNN) is proposed model. Taking ResNet50 as a visual modeling network, it uses feature pyramid and self-attention mechanism to extract appearance and semantic features of images at multiple scales, and associate and enhance local and global features. Taking into account the diversity of category features, channel cosine similarity attention and dynamic C-means clustering algorithms are used to select representative sample features in different category of sample subsets to implicitly express prior category feature knowledge, and use them as the kernel centers of radial basis probability neurons (RBPN) to realize the embedding of diverse prior feature knowledge. In the RBPNN pattern aggregation layer, the outputs of RBPN are selectively summed according to the category of the kernel center, that is, the subcategory features are combined into category features, and finally the image classification is implemented based on Softmax. The functional module of the proposed method is designed specifically for image characteristics, which can highlight the significance of local and structural features of the image, form a non-convex decision-making area, and reduce the requirements for the completeness of the sample set. Applying the proposed method to medical image classification, experiments were conducted based on the brain tumor MRI image classification public dataset and the actual cardiac ultrasound image dataset, and the accuracy rate reached 85.82% and 83.92% respectively. Compared with the three mainstream image classification models, the performance indicators of this method have been significantly improved.

Can flood resilience of green-grey-blue system cope with future uncertainty?

Urban flooding is becoming a great global concern due to growing cities, while climate change and urbanization may pose daunting challenges to both environment and humans. The integrated green-grey-blue (IGGB) system has gained interests worldwide to mitigate flood issues, however, how IGGB system acts in urban flood resilience and whether it can address future uncertainties have not been fully understood. In this study, a new framework, which combined an evaluation index system and coupling model, was constructed to quantify urban flood resilience (FR) and its responses to future uncertainties. The results showed that higher FR upstream than downstream; however, upstream FR declined approximately twice as much as downstream when faced with climate change and urbanization. Generally, climate change appeared to have a greater impact on urban flood resilience than urbanization, resulting to 3.20%-4.28% and 2.08%-4.09% FR reduction, respectively. The IGGB system could greatly improve robustness against future uncertainty, due to the fact that the IGGB without low impact development facilities (LIDs) was about 2 times in FR decline compared with IGGB with LIDs. The increase of LIDs proportion could diminish the impact of climate change, which shifted the dominant factor affecting FR from the interaction between urbanization and climate change to urbanization. Notably, a threshold of 13% construction land increase was quantified, beyond which negative effects of rainfall become dominant again. The results could guide IGGB design and urban flooding management in other similar regions.

Regulating the electronic structure of MoO2/Mo2C/C by heterostructure and oxygen vacancies for boosting lithium storage kinetics.

The electronic structure regulation of electrode materials can improve the ion/electron kinetics, which is beneficial to the cyclic performance and rate capability for lithium ion batteries (LIBs). Herein, we propose a facile strategy to achieve a MoO2/Mo2C/C heterostructure with abundant oxygen vacancies. Density functional theory calculations indicate that the heterostructure of MoO2/Mo2C/C can significantly promote the Li+/charge transfer and reduce the Li adsorption energy, and the abundant oxygen vacancies in MoO2/Mo2C/C can improve the intrinsic electronic conductivity and reduce the Li+ diffusion barrier. Benefiting from the multiscale coordinated regulation, the obtained MoO2/Mo2C/C film exhibits outstanding high rate capability (454.7 mA h g-1 at 5 A g-1) and remarkable cyclic performance (retaining 569 mA h g-1 over 1000 cycles at 2 A g-1). The insightful findings in this study can shed light on the behavior of the electron/ion structure regulation by the heterostructure and oxygen vacancies, which can guide future studies on designing other electrode materials with high-performance lithium-ion storage.

Gaze Estimation by Exploring Two-Eye Asymmetry.

Eye gaze estimation is increasingly demanded by recent intelligent systems to facilitate a range of interactive applications. Unfortunately, learning the highly complicated regression from a single eye image to the gaze direction is not trivial. Thus, the problem is yet to be solved efficiently. Inspired by the two-eye asymmetry as two eyes of the same person may appear uneven, we propose the face-based asymmetric regression-evaluation network (FARE-Net) to optimize the gaze estimation results by considering the difference between left and right eyes. The proposed method includes one face-based asymmetric regression network (FAR-Net) and one evaluation network (E-Net). The FAR-Net predicts 3D gaze directions for both eyes and is trained with the asymmetric mechanism, which asymmetrically weights and sums the loss generated by two-eye gaze directions. With the asymmetric mechanism, the FAR-Net utilizes the eyes that can achieve high performance to optimize network. The E-Net learns the reliabilities of two eyes to balance the learning of the asymmetric mechanism and symmetric mechanism. Our FARENet achieves leading performances on MPIIGaze, EyeDiap and RT-Gene datasets. Additionally, we investigate the effectiveness of FARE-Net by analyzing the distribution of errors and ablation study.

[An experimental study on the regulation of expression of Th1/Th2 cytokines by allergen vaccine atomization inhalation in patients with asthma].

OBJECTIVE: To explore the role of specific immunotherapy treatment (SIT) with allergen vaccine (AV) atomization inhalation in regulation of the expression of Th1/Th1 cytokines (IL-2 and IL-4) by T lymphocyte and serum IgE level in asthmas. METHOD: Ninety cases with asthma in remission were randomly divided into three groups and each group contained thirty patients who received SIT respectively with atomization inhalation (A group), sublingual contained (B group) and hypodermic injection (C group). 32 healthy subjects were as control group. Before and after treatment, the expression of IL-2 and IL-4, total IgE level were measured by radio-immune. The three SIT treatments were evaluated. RESULT: Before SIT treatment, compared with the normal control group, the patients' serum level of Th2 cytokines IL-4 and total IgE had significantly increased, the level of Th1 cytokines IL-2 had significantly reduced. There were statistical significance between the asthma and normal control (t = 4.869, 5.598, 10.009, P < 0.01). After 12 months SIT treatment, the patients' serum IL-2 showed increased, the serum IL-4 and total IgE showed reduced, the difference was statistical significant (t = 7.303, 2.494, 2.225, P < 0.05). There was good positive correlation between the level of IL-4 and the total IgE (r = 0.264, P < 0.01), there was good negative correlation between the level of IL-2 and the total IgE (r = -0.369, P < 0.001), and there was good negative correlation between the level of IL-2 and IL-4 (r = -0.277, P < 0.01). The effects among A, B, C groups were similar (all were over 86%), without statistical significance (chi2 = 0.055, P > 0.05). CONCLUSION: The specific response to antigen may be transferred from Th2 to Th1 by SIT with AV atomization inhalation, and mediated between balance of Th1/Th2 in patients with asthma. This method is thought a direct influence on the target organ. It is safe, simple and valid and can be easily accepted by sufferers.