Learning to Assess Image Quality Like an Observer.

Human observers are the ultimate receivers and evaluators of the image visual information and have powerful perception ability of visual quality with short-term global perception and long-term regional observation. Thus, it is natural to design an image quality assessment (IQA) computational model to act like an observer for accurately predicting the human perception of image quality. Inspired by this, here, we propose a novel observer-like network (OLN) to perform IQA by jointly considering the global glimpsing information and local scanning information. Specifically, the OLN consists of a global distortion perception (GDP) module and a local distortion observation (LDO) module. The GDP module is designed to mimic the observer's global perception of image quality through performing classification of images' distortion categories and levels. Simultaneously, to simulate the human local observation behavior, the LDO module attempts to gather the long-term regional observation information of the distorted images by continuously tracing the human scanpath in the observer-like scanning manner. By leveraging the bilinear pooling layer to collaborate the short-term global perception with the long-term regional observation, our network precisely predicts the quality scores of distorted images, such as human observers. Comprehensive experiments on the public datasets powerfully demonstrate that the proposed OLN achieves state-of-the-art performance.

Learning an Invariant and Equivariant Network for Weakly Supervised Object Detection.

Weakly Supervised Object Detection (WSOD) is of increasing importance in the community of computer vision as its extensive applications and low manual cost. Most of the advanced WSOD approaches build upon an indefinite and quality-agnostic framework, leading to unstable and incomplete object detectors. This paper attributes these issues to the process of inconsistent learning for object variations and the unawareness of localization quality and constructs a novel end-to-end Invariant and Equivariant Network (IENet). It is implemented with a flexible multi-branch online refinement, to be naturally more comprehensive-perceptive against various objects. Specifically, IENet first performs label propagation from the predicted instances to their transformed ones in a progressive manner, achieving affine-invariant learning. Meanwhile, IENet also naturally utilizes rotation-equivariant learning as a pretext task and derives an instance-level rotation-equivariant branch to be aware of the localization quality. With affine-invariance learning and rotation-equivariant learning, IENet urges consistent and holistic feature learning for WSOD without additional annotations. On the challenging datasets of both natural scenes and aerial scenes, we substantially boost WSOD to new state-of-the-art performance. The codes have been released at: https://github.com/XiaoxFeng/IENet.

Learning Complementary Spatial-Temporal Transformer for Video Salient Object Detection.

Besides combining appearance and motion information, another crucial factor for video salient object detection (VSOD) is to mine spatial-temporal (ST) knowledge, including complementary long-short temporal cues and global-local spatial context from neighboring frames. However, the existing methods only explored part of them and ignored their complementarity. In this article, we propose a novel complementary ST transformer (CoSTFormer) for VSOD, which has a short-global branch and a long-local branch to aggregate complementary ST contexts. The former integrates the global context from the neighboring two frames using dense pairwise attention, while the latter is designed to fuse long-term temporal information from more consecutive frames with local attention windows. In this way, we decompose the ST context into a short-global part and a long-local part and leverage the powerful transformer to model the context relationship and learn their complementarity. To solve the contradiction between local window attention and object motion, we propose a novel flow-guided window attention (FGWA) mechanism to align the attention windows with object and camera movements. Furthermore, we deploy CoSTFormer on fused appearance and motion features, thus enabling the effective combination of all three VSOD factors. Besides, we present a pseudo video generation method to synthesize sufficient video clips from static images for training ST saliency models. Extensive experiments have verified the effectiveness of our method and illustrated that we achieve new state-of-the-art results on several benchmark datasets.

Towards Large-Scale Small Object Detection: Survey and Benchmarks.

With the rise of deep convolutional neural networks, object detection has achieved prominent advances in past years. However, such prosperity could not camouflage the unsatisfactory situation of Small Object Detection (SOD), one of the notoriously challenging tasks in computer vision, owing to the poor visual appearance and noisy representation caused by the intrinsic structure of small targets. In addition, large-scale dataset for benchmarking small object detection methods remains a bottleneck. In this paper, we first conduct a thorough review of small object detection. Then, to catalyze the development of SOD, we construct two large-scale Small Object Detection dAtasets (SODA), SODA-D and SODA-A, which focus on the Driving and Aerial scenarios respectively. SODA-D includes 24828 high-quality traffic images and 278433 instances of nine categories. For SODA-A, we harvest 2513 high resolution aerial images and annotate 872069 instances over nine classes. The proposed datasets, as we know, are the first-ever attempt to large-scale benchmarks with a vast collection of exhaustively annotated instances tailored for multi-category SOD. Finally, we evaluate the performance of mainstream methods on SODA. We expect the released benchmarks could facilitate the development of SOD and spawn more breakthroughs in this field.

Empirical study of employee loyalty and satisfaction in the mining industry using structural equation modeling.

Mining is a high-risk industry and a crucial economic driver that has a crucial role in the economies of countries worldwide. The implications of the labor market on the sustainability of the mining industry have increased the importance of sustainable human resource management at the strategic level of mining and safety management. In this article, from the perspective of management research in an energy production enterprise, we investigated the relationship between employee loyalty and employee satisfaction through a survey that targets employee loyalty, work quality, and job satisfaction and the relationship between enterprise image and switching costs. Based on service profit chain theory, we established a research model for mining employee loyalty, and 500 miners in a typical extreme mining environment in China were surveyed. The study hypotheses were tested using a structural equation model and an employee loyalty model, followed by empirical testing of the models. Employee loyalty was significantly associated with enterprise image and employee satisfaction, work quality indirectly affected loyalty through satisfaction, and the impact of switching costs on employee loyalty was not significant. We provide strong empirical evidence to help enterprises improve sustainable human resource management and regulatory policies, with important implications for safety production. Our study also provides a useful reference for further studies of sustainable human resource management in mining.

P-CNN: Part-Based Convolutional Neural Networks for Fine-Grained Visual Categorization.

This paper proposes an end-to-end fine-grained visual categorization system, termed Part-based Convolutional Neural Network (P-CNN), which consists of three modules. The first module is a Squeeze-and-Excitation (SE) block, which learns to recalibrate channel-wise feature responses by emphasizing informative channels and suppressing less useful ones. The second module is a Part Localization Network (PLN) used to locate distinctive object parts, through which a bank of convolutional filters are learned as discriminative part detectors. Thus, a group of informative parts can be discovered by convolving the feature maps with each part detector. The third module is a Part Classification Network (PCN) that has two streams. The first stream classifies each individual object part into image-level categories. The second stream concatenates part features and global feature into a joint feature for the final classification. In order to learn powerful part features and boost the joint feature capability, we propose a Duplex Focal Loss used for metric learning and part classification, which focuses on training hard examples. We further merge PLN and PCN into a unified network for an end-to-end training process via a simple training technique. Comprehensive experiments and comparisons with state-of-the-art methods on three benchmark datasets demonstrate the effectiveness of our proposed method.

Information fusion and multi-classifier system for miner fatigue recognition in plateau environments based on electrocardiography and electromyography signals.

BACKGROUND AND OBJECTIVE: Human factors are important contributors to accidents, especially human error induced by fatigue. In this study, field tests and analyses were conducted on physiological indexes extracted from electrocardiography (ECG) and electromyography (EMG) signals in miners working under the extreme conditions of a plateau environment. To provide insights into models for fatigue classification and recognition based on machine learning, multi-modal feature information fusion and miner fatigue identification based on ECG and EMG signals as physiological indicators were studied. METHODS: Fifty-five miners were randomly selected as field test subjects, and characteristic signals were extracted from 110 groups of ECG and EMG signals as the basic signals for fatigue analysis. We conducted principal component analysis (PCA) and grey relational analysis (GRA) on the measurement indicators. Support vector machine (SVM), random forest (RF) and extreme gradient boosting (XG-Boost) machine learning models were used for fatigue classification based on multi-modal information fusion. The area under the receiver operating characteristic (ROC) curve and the confusion matrix were used to evaluate the performance of the recognition models. RESULTS: The ECG and EMG signals showed obvious changes with fatigue. The results of fatigue model identification showed that PCA feature fusion was superior to GRA feature fusion for all three machine learning approaches, and XG-Boost achieved the best performance, with a recognition accuracy of 89.47%, a sensitivity and specificity of 100%, and an AUC of 0.90. The SVM model also showed good recognition performance (89.47% accuracy, AUC=0.89). The worst performance was that of the RF model, with a recognition accuracy of only 78.95%. CONCLUSIONS: This study shows that the physiological indexes of ECG and EMG exhibit obvious, regular changes with fatigue and that it is feasible to use SVM, RF and XG-Boost models for miner fatigue identification. The PCA fusion technique can improve the identification accuracy more than the GRA method. XG-Boost classification yields the best accuracy and robustness. This study can serve as a reference for clinical research on the identification of human fatigue at high altitudes and for the clinical study of acute mountain sickness and human acclimatization to high altitudes.

Psychophysiological data-driven multi-feature information fusion and recognition of miner fatigue in high-altitude and cold areas.

Fatigue-induced human error is a leading cause of accidents. The purpose of this exploratory study in China was to perform field tests to measure fatigue psychophysiological parameters, such as electrocardiography (ECG), electromyography (EMG), pulse, blood pressure, reaction time and vital capacity (VC), in miners in high-altitude and cold areas and to perform multi-feature information fusion and fatigue identification. Forty-five miners were randomly selected as subjects for a field test, and feature signals were extracted from 90 psychophysiological features as basic signals for fatigue analysis. Fatigue sensitivity indices were obtained by Pearson correlation analysis, t-test and receiver operating characteristic (ROC) curve performance evaluation. The ECG time-domain, ECG frequency-domain, EMG, VC, systolic blood pressure (SBP), and pulse were significantly different after miner fatigue. The support vector machine (SVM) and random forest (RF) techniques were used to classify and identify fatigue by information fusion and factor combination. The optimal fatigue classification factors were ECG-FD (CV Accuracy = 85.0%) and EMG (CV Accuracy = 90.0%). The optimal combination of factors was ECG-TD + ECG-FD + EMG (CV accuracy = 80.0%). Furthermore, SVM machine learning had a good recognition effect. This study shows that SVM and RF can effectively identify miner fatigue based on fatigue-related factor combinations. ECG-FD and EMG are the best indicators of fatigue, and the best performance and robustness are obtained with three-factor combination classification. This study on miner fatigue identification provides a reference for research on clinical medicine and the identification of human fatigue under high-altitude, cold and low-oxygen conditions.

Linear and nonlinear analyses of normal and fatigue heart rate variability signals for miners in high-altitude and cold areas.

BACKGROUND AND OBJECTIVE: Fatigue is an important cause of operational errors, and human errors are the main cause of accidents. This study is an exploratory study in China. Field tests were conducted on heart rate variability (HRV) parameters and physiological indicators of fatigue among miners in high-altitude, cold and low-oxygen areas. This paper studies heart activity patterns during work fatigue in miners. METHODS: Fatigue affects both the sympathetic and parasympathetic nervous systems, and it is expressed as an abnormal pattern of HRV parameters. Thirty miners were selected as subjects for a field test, and HRV was extracted from 60 groups of electrocardiography (ECG) datasets as basic signals for fatigue analysis. Then, we analyzed the HRV signals of the miners using linear (time domain and frequency domain) and nonlinear dynamics (Poincaré plot and sample entropy (SampEn)), and a Pearson's correlation coefficient analysis and t-tests were performed on the measured indices. RESULTS: The results showed that the time-domain indices (SDNN, RMSSD, SDSD, pNN50, RRn, heart rate (HR), R-wave humps (RH)) and the coefficient of variation (CV)) and the frequency-domain indices (low frequency/high frequency (LF/HF), LFnorm and HFnorm) clearly changed after fatigue. These features were selected using a Poincaré plot, sample entropy, Pearson's correlation coefficient and a t-test for further analysis. The fatigue characteristics and sensitivity parameters of miners in a high-altitude, cold and hypoxic environment were obtained. CONCLUSIONS: This study provides deep insight into the use of linear and nonlinear fatigue characteristics to effectively and reliably identify miner fatigue. Furthermore, the study provides a reference for clinical studies of acute mountain sickness in high-altitude, cold and hypoxic environments.

Optimization of sand casting performance parameters and missing data prediction.

Due to a wide range of applications, sand casting occupies an important position in modern casting practice. The main purpose of this study was to optimize the performance parameters of sand casting based on grey relational analysis and predict the missing data using back propagation (BP) neural network. First, the influence of human factors was eliminated by adopting the objective entropy weight method, which also saved manpower. The larger variation degree in the evaluation indicators, indicating that the evaluated projects had good discrimination in this regard, the larger weight should be given to these evaluation indicators. Second, the performance parameters of sand casting were optimized based on grey relational analysis, providing a reference for sand milling. The larger the grey relational degree, the closer the evaluated project was to the ideal project. Third, this paper provided a new method for determining the number of hidden neurons in a network according to the mean square error of training samples, and venting quality was predicted based on BP neural network. The relevant theory was deduced before predicting missing data, such that there will be a general understanding regarding the prediction principle of BP neural network. Fourth, to demonstrate the validity of BP neural network adopted in the process of missing data prediction, grey system theory was applied to compare the result of missing data prediction.

Mine safety assessment based on basic event importance: grey relational analysis and bow tie model.

Safety assessments are a crucial first step in preventing coal and gas outburst accidents. The main purpose of this study was to create a new accident prevention technique using a novel safety assessment method based on fault tree basic event importance, grey relational analysis and the bow tie model. The innovation of the proposed method lies in generating the composite importance of a basic event from the fundamental importance via grey relational analysis; bow tie analysis serves to reveal the most critical basic event. First, the minimal cut sets and minimal path sets of a coal and gas outburst accident are determined by fault tree analysis. The role of minimal cut and path sets is determined and the coal and gas outburst occurrence frequency is calculated accordingly. Second, the structure, probability, critical and Fussell-Vesely importance ranked basic events differently due to different aspects of the basic events as investigated. We establish a composite importance to represent single basic events and achieved new ranking results by grey relational analysis. Third, the critical basic event low permeability coefficient is analysed via bow tie model and safety measures are defined which prevent the dangerous consequences of a low permeability coefficient. An actual coal and gas outburst accident is used as a case study to test the feasibility and effectiveness of the proposed method.

Safety assessment of petrochemical enterprise using the cloud model, PHA-LOPA and the bow-tie model.

Safe production is the foundation of the normal operations of petrochemical enterprises, and it helps maintain social stability. The main purpose of this study is to prevent petrochemical enterprise accidents by proposing a composite safety assessment approach based on the cloud model, preliminary hazard analysis-layer of protection analysis (PHA-LOPA) and the bow-tie model. First, the petrochemical enterprise and its relevant indicators were evaluated based on the cloud model. Second, the quantitative effect of the uncertainty transformation on the evaluation result of the cloud model was further analysed. This mainly includes the error analysis of the numerical characteristics under the conditions of few samples and small values. Third, the critical indicators such as shock and noise can be weakened and prevented by corresponding safety measures based on PHA-LOPA and the bow-tie model. After adopting two independent protection layers, the risk levels of shock and noise decrease from 3 to 2. Then, shock and noise were analysed in depth with the bow-tie model, and the causes and consequences were identified. Moreover, corresponding safety measures were taken to prevent accidents. The case study validated the validity and feasibility of the composite safety assessment approach proposed here.

Sand casting safety assessment for foundry enterprises: fault tree analysis, Heinrich accident triangle, HAZOP-LOPA, bow tie model.

Sand casting operations, though commonplace, pose a significant threat of explosion accidents. This paper presents a novel sand casting safety assessment technique based on fault tree analysis, Heinrich accident triangle, hazard and operability-layer of protection analysis (HAZOP-LOPA) and bow tie model components. Minimal cut sets and minimal path sets are first determined based on fault tree analysis, then the frequency of sand casting explosion accidents is calculated based on the Heinrich accident triangle. Third, the risk level of venting quality can be reduced by adopting HAZOP-LOPA; the residual risk level of venting quality remains excessive even after adopting two independent protective layers. The bow tie model is then adopted to determine the causes and consequences of venting quality. Five preventative measures are imposed to enhance the venting quality of foundry sand accompanied by 16 mitigative safety measures. Our results indicate that the risk attributable to low foundry sand venting quality can be minimized via bow tie analysis.

Thermal decomposition characteristics of foundry sand for cast iron in nitrogen atmosphere.

Sand casting, currently the most popular approach to the casting production, has wide adaptability and low cost. The thermal decomposition characteristics of foundry sand for cast iron were determined for the first time in this study. Thermogravimetry was monitored by simultaneous thermal analyser to find that there was no obvious oxidation or combustion reaction in the foundry sand; the thermal decomposition degree increased as the heating rate increased. There was an obvious endothermic peak at about 846 K due to the transition of quartz from ß to α phase. A novel technique was established to calculate the starting temperature of volatile emission in determining the volatile release parameter of foundry sand for cast iron. Foundry sand does not readily evaporate because its volatile content is only about 2.68 wt% and its main components have high-temperature stability. The thermal decomposition kinetics parameters of foundry sand, namely activation energy and pre-exponential factor, were obtained under kinetics theory. The activation energy of foundry sand for cast iron was small, mainly due to the wide temperature range of thermal decomposition in the foundry sand.

Discriminative Joint-Feature Topic Model With Dual Constraints for WCE Classification.

Wireless capsule endoscopy (WCE) enables clinicians to examine the digestive tract without any surgical operations, at the cost of a large amount of images to be analyzed. The main challenge for automatic computer-aided diagnosis arises from the difficulty of robust characterization of these images. To tackle this problem, a novel discriminative joint-feature topic model (DJTM) with dual constraints is proposed to classify multiple abnormalities in WCE images. We first propose a joint-feature probabilistic latent semantic analysis (PLSA) model, where color and texture descriptors extracted from same image patches are jointly modeled with their conditional distributions. Then the proposed dual constraints: visual words importance and local image manifold are embedded into the joint-feature PLSA model simultaneously to obtain discriminative latent semantic topics. The visual word importance is proposed in our DJTM to guarantee that visual words with similar importance come from close latent topics while the local image manifold constraint enforces that images within the same category share similar latent topics. Finally, each image is characterized by distribution of latent semantic topics instead of low level features. Our proposed DJTM showed an excellent overall recognition accuracy 90.78%. Comprehensive comparison results demonstrate that our method outperforms existing multiple abnormalities classification methods for WCE images.

Revisiting Co-Saliency Detection: A Novel Approach Based on Two-Stage Multi-View Spectral Rotation Co-clustering.

With the goal of discovering the common and salient objects from the given image group, co-saliency detection has received tremendous research interest in recent years. However, as most of the existing co-saliency detection methods are performed based on the assumption that all the images in the given image group should contain co-salient objects in only one category, they can hardly be applied in practice, particularly for the large-scale image set obtained from the Internet. To address this problem, this paper revisits the co-saliency detection task and advances its development into a new phase, where the problem setting is generalized to allow the image group to contain objects in arbitrary number of categories and the algorithms need to simultaneously detect multi-class co-salient objects from such complex data. To solve this new challenge, we decompose it into two sub-problems, i.e., how to identify subgroups of relevant images and how to discover relevant co-salient objects from each subgroup, and propose a novel co-saliency detection framework to correspondingly address the two sub-problems via two-stage multi-view spectral rotation co-clustering. Comprehensive experiments on two publically available benchmarks demonstrate the effectiveness of the proposed approach. Notably, it can even outperform the state-of-the-art co-saliency detection methods, which are performed based on the image subgroups carefully separated by the human labor.

Fuzzy Bayesian Network-Bow-Tie Analysis of Gas Leakage during Biomass Gasification.

Biomass gasification technology has been rapidly developed recently. But fire and poisoning accidents caused by gas leakage restrict the development and promotion of biomass gasification. Therefore, probabilistic safety assessment (PSA) is necessary for biomass gasification system. Subsequently, Bayesian network-bow-tie (BN-bow-tie) analysis was proposed by mapping bow-tie analysis into Bayesian network (BN). Causes of gas leakage and the accidents triggered by gas leakage can be obtained by bow-tie analysis, and BN was used to confirm the critical nodes of accidents by introducing corresponding three importance measures. Meanwhile, certain occurrence probability of failure was needed in PSA. In view of the insufficient failure data of biomass gasification, the occurrence probability of failure which cannot be obtained from standard reliability data sources was confirmed by fuzzy methods based on expert judgment. An improved approach considered expert weighting to aggregate fuzzy numbers included triangular and trapezoidal numbers was proposed, and the occurrence probability of failure was obtained. Finally, safety measures were indicated based on the obtained critical nodes. The theoretical occurrence probabilities in one year of gas leakage and the accidents caused by it were reduced to 1/10.3 of the original values by these safety measures.