Weakly Supervised Violence Detection in Surveillance Video.

Automatic violence detection in video surveillance is essential for social and personal security. Monitoring the large number of surveillance cameras used in public and private areas is challenging for human operators. The manual nature of this task significantly increases the possibility of ignoring important events due to human limitations when paying attention to multiple targets at a time. Researchers have proposed several methods to detect violent events automatically to overcome this problem. So far, most previous studies have focused only on classifying short clips without performing spatial localization. In this work, we tackle this problem by proposing a weakly supervised method to detect spatially and temporarily violent actions in surveillance videos using only video-level labels. The proposed method follows a Fast-RCNN style architecture, that has been temporally extended. First, we generate spatiotemporal proposals (action tubes) leveraging pre-trained person detectors, motion appearance (dynamic images), and tracking algorithms. Then, given an input video and the action proposals, we extract spatiotemporal features using deep neural networks. Finally, a classifier based on multiple-instance learning is trained to label each action tube as violent or non-violent. We obtain similar results to the state of the art in three public databases Hockey Fight, RLVSD, and RWF-2000, achieving an accuracy of 97.3%, 92.88%, 88.7%, respectively.

Applicability of image analysis to support QbD driven development of pellets.

OBJECTIVE: The stages of preparing high drug loaded pellets were investigated using static and dynamic imaging techniques to provide a greater understanding and ease the scale up process. SIGNIFICANCE: An example of a real case laboratory and production scale quality by design (QbD) based development of pellets is demonstrated. Potential process analytical technology (PAT) approaches by dynamic image analysis (DIA) are presented in various process phases. METHODS: Pellets were prepared at laboratory and production scale (high shear granulation, extrusion/spheronization, drying, and coating). The influence of process parameters on pellet properties (aspect ratio (AR), yield, pellet size, and their distribution) was investigated using static and DIA. During coating, we focused on the coating thickness and identification of potential agglomeration. RESULTS AND CONCLUSION: The effects of kneading time, amount of water, extrusion screen plate (ESP) opening diameter and thickness on pellet properties were confirmed in accordance with literature. In terms of screw speed, spheronization speed and time, no considerable influence on pellet properties was observed in the range of studied process parameters, thereby confirming the design space. In addition to the ESP thickness and opening diameter, quality of the ESP impacts the pellet properties. Lastly, coating thickness measurements with dynamic and static image analysis were comparable and an exemplary case of in-line agglomeration detection was presented. Real-time evaluation with PATVIS APA is an effective PAT tool for the evaluation of spheronization (pellet size distribution, AR, and yield) and coating (coating thickness, agglomeration detection).

A Hybrid Network for Large-Scale Action Recognition from RGB and Depth Modalities.

The paper presents a novel hybrid network for large-scale action recognition from multiple modalities. The network is built upon the proposed weighted dynamic images. It effectively leverages the strengths of the emerging Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) based approaches to specifically address the challenges that occur in large-scale action recognition and are not fully dealt with by the state-of-the-art methods. Specifically, the proposed hybrid network consists of a CNN based component and an RNN based component. Features extracted by the two components are fused through canonical correlation analysis and then fed to a linear Support Vector Machine (SVM) for classification. The proposed network achieved state-of-the-art results on the ChaLearn LAP IsoGD, NTU RGB+D and Multi-modal & Multi-view & Interactive ( M 2 I ) datasets and outperformed existing methods by a large margin (over 10 percentage points in some cases).

An Adaptive Deep Learning Framework for Dynamic Image Classification in the Internet of Things Environment.

In the modern era of digitization, the analysis in the Internet of Things (IoT) environment demands a brisk amalgamation of domains such as high-dimension (images) data sensing technologies, robust internet connection (4 G or 5 G) and dynamic (adaptive) deep learning approaches. This is required for a broad range of indispensable intelligent applications, like intelligent healthcare systems. Dynamic image classification is one of the major areas of concern for researchers, which may take place during analysis under the IoT environment. Dynamic image classification is associated with several temporal data perturbations (such as novel class arrival and class evolution issue) which cause a massive classification deterioration in the deployed classification models and make them in-effective. Therefore, this study addresses such temporal inconsistencies (novel class arrival and class evolution issue) and proposes an adapted deep learning framework (ameliorated adaptive convolutional neural network (CNN) ensemble framework), which handles novel class arrival and class evaluation issue during dynamic image classification. The proposed framework is an improved version of previous adaptive CNN ensemble with an additional online training (OT) and online classifier update (OCU) modules. An OT module is a clustering-based approach which uses the Euclidean distance and silhouette method to determine the potential new classes, whereas, the OCU updates the weights of the existing instances of the ensemble with newly arrived samples. The proposed framework showed the desirable classification improvement under non-stationary scenarios for the benchmark (CIFAR10) and real (ISIC 2019: Skin disease) data streams. Also, the proposed framework outperformed against state-of-art shallow learning and deep learning models. The results have shown the effectiveness and proven the diversity of the proposed framework to adapt the new concept changes during dynamic image classification. In future work, the authors of this study aim to develop an IoT-enabled adaptive intelligent dermoscopy device (for dermatologists). Therefore, further improvements in classification accuracy (for real dataset) is the future concern of this study.

Apple Fruit Recognition Algorithm Based on Multi-Spectral Dynamic Image Analysis.

The ability to accurately recognize fruit on trees is a critical step in robotic harvesting. Many researchers have investigated a variety of image analysis methods based on different imaging technologies for fruit recognition. However, challenges still occur in the implementation of this goal due to various factors, especially variable light and proximal color background. In this study, images with fruit were acquired with a Forward Looking Infrared (FLIR) camera based on the Multi-Spectral Dynamic Imaging (MSX) technology. In view of its imaging mechanism, the optimal timing and shooting angle for image acquisition were pre-analyzed to obtain the maximum contrast between fruit and background. An effective algorithm was developed for locking potential fruit regions, which was based on the pseudo-color and texture information from MSX images. The algorithm was applied to 506 training and 340 evaluating images, including a variety of fruit and complex backgrounds. Recognition precision and sensitivity of these complete fruit regions were both above 92%, and those of incomplete fruit regions were not lower than 72%. The average processing time for each image was less than 1 s. The results indicated that the developed algorithm based on MSX imaging was effective for fruit recognition and could be suggested as a potential method for the automation of orchard production.

Wearable Device-Based Gait Recognition Using Angle Embedded Gait Dynamic Images and a Convolutional Neural Network.

The widespread installation of inertial sensors in smartphones and other wearable devices provides a valuable opportunity to identify people by analyzing their gait patterns, for either cooperative or non-cooperative circumstances. However, it is still a challenging task to reliably extract discriminative features for gait recognition with noisy and complex data sequences collected from casually worn wearable devices like smartphones. To cope with this problem, we propose a novel image-based gait recognition approach using the Convolutional Neural Network (CNN) without the need to manually extract discriminative features. The CNN's input image, which is encoded straightforwardly from the inertial sensor data sequences, is called Angle Embedded Gait Dynamic Image (AE-GDI). AE-GDI is a new two-dimensional representation of gait dynamics, which is invariant to rotation and translation. The performance of the proposed approach in gait authentication and gait labeling is evaluated using two datasets: (1) the McGill University dataset, which is collected under realistic conditions; and (2) the Osaka University dataset with the largest number of subjects. Experimental results show that the proposed approach achieves competitive recognition accuracy over existing approaches and provides an effective parametric solution for identification among a large number of subjects by gait patterns.

Development of videogrammetry as a tool for gas-particle fluidization research.

Many industries use fluidization of solid particles for energy efficiency or environmental friendly process development, and this paper introduces research techniques developed for investigating gas-particle systems At present there is plenty of room for refining gas-particle fluidization process. With the rapidly rising application of mathematical modelling, real time visualization of processes will be widely used for validation of those models in the near future. In presented research, photogrammetry, as a part of close range vision metrology, has been expanded to allow dynamic space and time analysis of the phase concentration distribution inside fluidization devices. A novel videogrammetry method was created with additional stochastic process analysis for detailed frequency and amplitude characteristics. Videogrammetry was used for the assessment of flow regimes, which were held in various types of fluidization apparatuses. Classic bubbling, jet-spouted and fast circulating fluidization processes were explored under the investigation. Videogrammetry is non-invasive flow regime recognition method, which enables detailed research of gas-particle fluidization phenomena. Until now, there were no comparative studies for three different types of fluidization processes with the use of one complex approach. Developed videogrammetric method consists of the flow structure visualization and dynamic image analysis. The analysed feature is the grey level of the image in time domain, and grey level signals were analysed with the use of autocorrelation function and power density function. The results are presented as images, plots and a flow map. Efficiency of the method was tested by comparison of real observed flow structures to the reconstructed flow structures and the recognition accuracy reached 92%.

The Influence of Wet Granulation Parameters on the Compaction Behavior and Tablet Strength of a Hydralazine Powder Mixture.

The aim of this paper was to describe the influence of high-shear wet granulation process parameters on tablet tensile strength and compaction behavior of a powder mixture and granules containing hydralazine. The hydralazine powder mixture and eight types of granules were compacted into tablets and evaluated using the Heckel, Kawakita and Adams analyses. The granules were created using two types of granulation liquid (distilled water and aqueous solution of polyvinylpyrrolidone), at different impeller speeds (500 and 700 rpm) and with different wet massing times (without wet massing and for 2 min). Granulation resulted in improved compressibility, reduced dustiness and narrower particle-size distribution. A significant influence of wet massing time on parameters from the Kawakita and Adams analysis was found. Wet massing time had an equally significant effect on tablet tensile strength, regardless of the granulation liquid used. Granules formed with the same wet massing time showed the same trends in tabletability graphs. Tablets created using a single-tablet press (batch compaction) and an eccentric tablet press showed opposite values of tensile strength. Tablets from granules with a higher bulk density showed lower strength during batch compaction and, conversely, higher strength during eccentric tableting.

Designing virtual reality based 3D modeling and interaction technologies for museums.

In China, museums are of great historical significance, which can greatly improve the country's cultural standards. With the advent of new media and economic times, people's behavior and way of thinking have changed, so they are less and less interested in traditional museum displays. How to create a museum moving image that meets the aesthetic and experiential requirements of the general audience has become critical. The purpose of this paper was to study the design of moving image displays using virtual reality (VR) in museums. This paper proposed a VR-based 3D modeling technology and human-computer interaction algorithm. Both of these technologies were an important part of VR technology. It can manage museums digitally and display objects clearly in two-dimensional and three-dimensional spaces. According to the experimental results of this paper, among the 80 participants, 40% were very satisfied with the exhibition hall experience of Chengde Mountain Resort Museum, and 35% were only moderately satisfied. It can be seen that most people find it very attractive to integrate VR technology into the showroom experience. Therefore, it is very important to integrate VR technology into the dynamic image display of the museum.

Modification of Polyamide 66 for a Media-Tight Hybrid Composite with Aluminum.

Metal-plastic composites are becoming increasingly important in lightweight construction. As a combination, e.g., for transmission housings in automobiles, composites made of die-cast aluminum housings and Polyamide 66 are a promising material. The interface between metal and plastic and the properties of the plastic component play an important role with regard to media tightness against transmission oil. The mechanical properties of the plastic can be matched to aluminum by glass fibers and additives. In the case of fiber-reinforced plastics, the mechanical properties depend on the fiber length and their orientation. These structural properties were investigated using computer tomography and dynamic image analysis. In addition to the mechanical properties, the thermal expansion coefficient was also investigated since a strongly different coefficient of the joining partners leads to stresses in the interface. Polyamide 66 was processed with 30 wt% glass fibers to align the mechanical and thermal expansion properties to those of aluminum. In contrast to the reinforcement additives, an impact modifier to improve the toughness of the composite, and/or a calcium stearate to exert influence on the rheological behavior of the composite, were used. The combination of the glass fibers with calcium stearate in Polyamide 66 led to high stiffnesses (11,500 MPa) and strengths (200 MPa), which were closest to those of aluminum. The coefficient of thermal expansion was found to be 6.6 × 10-6/K for the combination of Polyamide 66 with 30 wt% glass fiber and shows a low expansion exponent compared to neat Polamid 66. It was detected that the use of an impact modifier led to less orientated fibers along the injection direction, which resulted in lower modulus and strength in terms of mechanical properties.

Influence of Different Hot Runner-Systems in the Injection Molding Process on the Structural and Mechanical Properties of Regenerated Cellulose Fiber Reinforced Polypropylene.

The increasing demand for renewable raw materials and lightweight composites leads to an increasing request for natural fiber composites (NFC) in series production. In order to be able to use NFC competitively, they must also be processable with hot runner systems in injection molding series production. For this reason, the influences of two hot runner systems on the structural and mechanical properties of Polypropylene with 20 wt.% regenerated cellulose fibers (RCF) were investigated. Therefore, the material was processed into test specimens using two different hot runner systems (open and valve gate) and six different process settings. The tensile tests carried out showed very good strength for both hot runner systems, which were max. 20% below the reference specimen processed with a cold runner and, however, significantly influenced by the different parameter settings. Fiber length measurements with the dynamic image analysis showed approx. 20% lower median values of GF and 5% lower of RCF through the processing with both hot runner systems compared to the reference, although the influence of the parameter settings was small. The X-ray microtomography performed on the open hot runner samples showed the influences of the parameter settings on the fiber orientation. In summary, it was shown that RCF composites can be processed with different hot runner systems in a wide process window. Nevertheless, the specimens of the setting with the lowest applied thermal load showed the best mechanical properties for both hot runner systems. It was furthermore shown that the resulting mechanical properties of the composites are not only due to one structural property (fiber length, orientation, or thermally induced changes in fiber properties) but are based on a combination of several material- and process-related properties.

AI-based analysis of in-line process endoscope images for real-time particle size measurement in a continuous pharmaceutical milling process.

This paper presents a case study on the first in-line application of AI-based image analysis for real-time pharmaceutical particle size measurement in a continuous milling process. An AI-based imaging system, which utilises a rigid endoscope, was tested for the real-time particle size measurement of solid NaCl powder used as a model API in the range of 200-1000 µm. After creating a dataset containing annotated images of NaCl particles, it was used to train an AI model for detecting particles and measuring their size. The developed system could analyse overlapping particles without dispersing air, thus broadening its applicability. The performance of the system was evaluated by measuring pre-sifted NaCl samples with the imaging tool, after which it was installed into a continuous mill for in-line particle size measurement of a milling process. By analysing ∼100 particles/s, the system was able to accurately measure the particle size of sifted NaCl samples and detect particle size reduction when applied in the milling process. The Dv50 values and PSDs measured real-time with the AI-based system correlated well with the reference laser diffraction measurements (<6% mean absolute difference over the measured samples). The AI-based imaging system shows great potential for in-line particle size analysis, which, in line with the latest pharmaceutical QC trends, can provide valuable information for process development and control.

Influence of Processing Glass-Fiber Filled Plastics on Different Twin-Screw Extruders and Varying Screw Designs on Fiber Length and Particle Distribution.

Due to their valuable properties (low weight, and good thermal and mechanical properties), glass fiber reinforced thermoplastics are becoming increasingly important. Fiber-reinforced thermoplastics are mainly manufactured by injection molding and extrusion, whereby the extrusion compounding process is primarily used to produce fiber-filled granulates. Reproducible production of high-quality components requires a granulate in which the fiber length is even and high. However, the extrusion process leads to the fact that fiber breakages can occur during processing. To enable a significant quality enhancement, experimentally validated modeling is required. In this study, short glass fiber reinforced thermoplastics (polypropylene) were produced on two different twin-screw extruders. Therefore, the machine-specific process behavior is of major interest regarding its influence. First, the fiber length change after processing was determined by experimental investigations and then simulated with the SIGMA simulation software. By comparing the simulation and experimental tests, important insights could be gained and the effects on fiber lengths could be determined in advance. The resulting fiber lengths and distributions were different, not only for different screw configurations (SC), but also for the same screw configurations on different twin-screw extruders. This may have been due to manufacturer-specific tolerances.

In-line particle size measurement based on image analysis in a fully continuous granule manufacturing line for rapid process understanding and development.

The present paper serves as a demonstration how an in-line PAT tool can be used for rapid and efficient process development in a fully continuous powder to granule line consisting of an interconnected twin-screw wet granulator, vibrational fluid bed dryer, and a regranulating mill. A new method was investigated for the periodic in-line particle size measurement of high mass flow materials to obtain real-time particle size data of the regranulated product. The system utilises a vibratory feeder with periodically altered feeding intensity in order to temporarily reduce the mass flow of the material passing in front of the camera. This results in the drastic reduction of particle overlapping in the images, making image analysis a viable tool for the in-line particle size measurement of high mass-flow materials. To evaluate the performance of the imaging system, the effect of several milling settings and the liquid-to-solid ratio was investigated on the product's particle size in the span of a few hours. The particle sizes measured with the in-line system were in accordance with the expected trends as well as with the results of the off-line reference particle size measurements. Based on the results, the in-line imaging system can serve as a PAT tool to obtain valuable real-time information for rapid process development or quality assurance.

Comparison of 2D Optical Imaging and 3D Microtomography Shape Measurements of a Coastal Bioclastic Calcareous Sand.

This article compares measurements of particle shape parameters from three-dimensional (3D) X-ray micro-computed tomography (µCT) and two-dimensional (2D) dynamic image analysis (DIA) from the optical microscopy of a coastal bioclastic calcareous sand from Western Australia. This biogenic sand from a high energy environment consists largely of the shells and tests of marine organisms and their clasts. A significant difference was observed between the two imaging techniques for measurements of aspect ratio, convexity, and sphericity. Measured values of aspect ratio, sphericity, and convexity are larger in 2D than in 3D. Correlation analysis indicates that sphericity is correlated with convexity in both 2D and 3D. These results are attributed to inherent limitations of DIA when applied to platy sand grains and to the shape being, in part, dependent on the biology of the grain rather than a purely random clastic process, like typical siliceous sands. The statistical data has also been fitted to Johnson Bounded Distribution for the ease of future use. Overall, this research demonstrates the need for high-quality 3D microscopy when conducting a micromechanical analysis of biogenic calcareous sands.

Systematic Study of the Effects of High Shear Granulation Parameters on Process Yield, Granule Size, and Shape by Dynamic Image Analysis.

The aim of the work was to analyze the influence of process parameters of high shear granulation on the process yield and on the morphology of granules on the basis of dynamic image analysis. The amount of added granulation liquid had a significant effect on all monitored granulometric parameters and caused significant changes in the yield of the process. In regard of the shape, the most spherical granules with the smoothest surface were formed at a liquid to solid ratio of ≈1. The smallest granules were formed at an impeller speed of 700 rpm, but the granules formed at 500 rpm showed both the most desirable shape and the highest process yield. Variation in the shape factors relied not only on the process parameters, but also on the area equivalent diameter of the individual granules in the batch. A linear relationship was found between the amount of granulation liquid and the compressibility of the granules. Using response surface methodology, models for predicting the size of granules and process yield related to the amount of added liquid and the impeller speed were generated, on the basis of which the size of granules and yield can be determined with great accuracy.

Systematic study of paracetamol powder mixtures and granules tabletability: Key role of rheological properties and dynamic image analysis.

The aim of this systematic study was to analyze the granulometric and rheological behavior of tableting mixtures in relation to tabletability by single tablet and lab-scale batch compression with an eccentric tablet machine. Three mixtures containing 33, 50, and 66% of the cohesive drug paracetamol were prepared. The high compressibility of the powder mixtures caused problems with overcompaction or lamination in the single tablet compression method; due to jamming of the material during the filling of the die, the lab-scale batch compression was impossible. Using high shear granulation, the flow properties and tabletability were adjusted. A linear relationship between the span of granules and the specific energy measured by FT4 powder rheometer was detected. In parallel, a linear relationship between conditioned bulk density and the tensile strength of the tablets at lab-scale batch tableting was noted. The combination of dynamic image analysis and powder rheometry was useful for predicting the tabletability of pharmaceutical mixtures during the single tablet (design) compression and the lab-scale batch compression.

Dynamic Image for 3D MRI Image Alzheimer's Disease Classification.

We propose to apply a 2D CNN architecture to 3D MRI image Alzheimer's disease classification. Training a 3D convolutional neural network (CNN) is time-consuming and computationally expensive. We make use of approximate rank pooling to transform the 3D MRI image volume into a 2D image to use as input to a 2D CNN. We show our proposed CNN model achieves 9.5% better Alzheimer's disease classification accuracy than the baseline 3D models. We also show that our method allows for efficient training, requiring only 20% of the training time compared to 3D CNN models. The code is available online: https://github.com/UkyVision/alzheimer-project.

Videometric mass flow control: A new method for real-time measurement and feedback control of powder micro-feeding based on image analysis.

The present paper reports the first monitoring and control of ultra-low dose powder feeding using a camera image-based mass flow measurement system. Caffeine was fed via a single-screw microfeeder as a model active pharmaceutical ingredient (API). The mass, mass flow and sizes of the particles were successfully monitored in real-time by the developed videometric system consisting of a high-speed process camera coupled with an image analysis software. The system was also tested in feedback control mode to automatically reach the desired mass flow values by adjusting the feeder speed based on the mass flow measured by the image analysis system. Based on these features, the developed videometric system can serve as a multi-purpose PAT-tool and can provide valuable real-time information about the process which is indispensable for modern continuous pharmaceutical manufacturing.

Initial Experience of the Synchronized, Real-Time, Interactive, Remote Transthoracic Echocardiogram Consultation System in Rural China: Longitudinal Observational Study.

BACKGROUND: China has a vast territory, and the quality of health care services provided, especially transthoracic echocardiography (TTE), in remote regions is still low. Patients usually need to travel long distances to tertiary care centers for confirmation of a diagnosis. Considering the rapid development of high-speed communication technology, telemedicine will be a significant technology for improving the diagnosis and treatment of patients at secondary care hospitals. OBJECTIVE: This study aimed to discuss the feasibility and perceived clinical value of a synchronized, real-time, interactive, remote TTE consultation system based on cloud computing technology. METHODS: By using the cloud computing platform coupled with unique dynamic image coding and decoding and synchronization technology, multidimensional communication information in the form of voice, texts, and pictures was integrated. A remote TTE consultation system connecting Henan Provincial People's Hospital and two county-level secondary care hospitals located 300 km away was developed, which was used for consultation with 45 patients. RESULTS: This remote TTE consultation system achieved remote consultation for 45 patients. The total time for consultation was 341.31 min, and the mean time for each patient was 7.58 (SD 6.17) min. Among the 45 patients, 3 were diagnosed with congenital heart diseases (7%) and 42 were diagnosed with acquired heart diseases (93%) at the secondary care hospitals. After expert consultation, the final diagnosis was congenital heart diseases in 5 patients (11%), acquired heart disease in 34 patients (76%), and absence of heart abnormalities in 6 patients (13%). Compared with the initial diagnosis at secondary care hospitals, remote consultation using this system revealed new abnormalities in 7 patients (16%), confirmation was obtained in 6 patients (13%), and abnormalities were excluded in 6 patients (13%). The expert opinions agreed with the initial diagnosis in the remaining 26 patients (58%). In addition, several questions about rare illnesses raised by the rural doctors at the secondary care hospitals were answered. CONCLUSIONS: The synchronized real-time interactive remote TTE consultation system based on cloud computing service and unique dynamic image coding and decoding technology had high feasibility and applicability.