Research on Target Image Classification in Low-Light Night Vision.

In extremely dark conditions, low-light imaging may offer spectators a rich visual experience, which is important for both military and civic applications. However, the images taken in ultra-micro light environments usually have inherent defects such as extremely low brightness and contrast, a high noise level, and serious loss of scene details and colors, which leads to great challenges in the research of low-light image and object detection and classification. The low-light night vision image used as the study object in this work has an excessively dim overall picture and very little information about the screen's features. Three algorithms, HE, AHE, and CLAHE, were used to enhance and highlight the image. The effectiveness of these image enhancement methods is evaluated using metrics such as the peak signal-to-noise ratio and mean square error, and CLAHE was selected after comparison. The target image includes vehicles, people, license plates, and objects. The gray-level co-occurrence matrix (GLCM) was used to extract the texture features of the enhanced images, and the extracted image texture features were used as input to construct a backpropagation (BP) neural network classification model. Then, low-light image classification models were developed based on VGG16 and ResNet50 convolutional neural networks combined with low-light image enhancement algorithms. The experimental results show that the overall classification accuracy of the VGG16 convolutional neural network model is 92.1%. Compared with the BP and ResNet50 neural network models, the classification accuracy was increased by 4.5% and 2.3%, respectively, demonstrating its effectiveness in classifying low-light night vision targets.

A synthetic high-voltage power line insulator images dataset.

High-voltage power line insulators are crucial for safe and efficient electricity transmission. However, real-world image limitations, particularly regarding dirty insulator strings, delay the development of robust algorithms for insulator inspection. This dataset addresses this challenge by creating a novel synthetic high-voltage power line insulator image database. The database was created using computer-aided design softwares and a game development engine. Publicly available CAD models of high-voltage towers with the most common insulator types (polymer, glass, and porcelain) were imported into the game engine. This virtual environment allowed for the generation of a diverse dataset by manipulating virtual cameras, simulating various lighting conditions, and incorporating different backgrounds such as mountains, forests, plantation, rivers, city and deserts. The database comprises two main sets: The Image Segmentation Set, which includes 47,286 images categorized by insulator material (ceramic, polymeric, and glass) and landscape type (mountains, forests, plantation, rivers, city and deserts). Moreover, the Image Classification Set that contains 14,424 images simulating common insulator string contaminants: salt, soot, bird excrement, and clean insulators. Each contaminant category has 3,606 images divided into 1,202 images per insulator type. This synthetic database offers a valuable resource for training and evaluating machine learning algorithms for high-voltage power line insulator inspection, ultimately contributing to enhanced power grid maintenance and reliability.

A hands-on guide to use network video recorders, internet protocol cameras, and deep learning models for dynamic monitoring of trout and salmon in small streams.

This study outlines a method for using surveillance cameras and an algorithm that calls a deep learning model to generate video segments featuring salmon and trout in small streams. This automated process greatly reduces the need for human intervention in video surveillance. Furthermore, a comprehensive guide is provided on setting up and configuring surveillance equipment, along with instructions on training a deep learning model tailored to specific requirements. Access to video data and knowledge about deep learning models makes monitoring of trout and salmon dynamic and hands-on, as the collected data can be used to train and further improve deep learning models. Hopefully, this setup will encourage fisheries managers to conduct more monitoring as the equipment is relatively cheap compared with customized solutions for fish monitoring. To make effective use of the data, natural markings of the camera-captured fish can be used for individual identification. While the automated process greatly reduces the need for human intervention in video surveillance and speeds up the initial sorting and detection of fish, the manual identification of individual fish based on natural markings still requires human effort and involvement. Individual encounter data hold many potential applications, such as capture-recapture and relative abundance models, and for evaluating fish passages in streams with hydropower by spatial recaptures, that is, the same individual identified at different locations. There is much to gain by using this technique as camera captures are the better option for the fish's welfare and are less time-consuming compared with physical captures and tagging.

Utilizing Polarization Diversity in GBSAR Data-Based Object Classification.

In recent years, the development of intelligent sensor systems has experienced remarkable growth, particularly in the domain of microwave and millimeter wave sensing, thanks to the increased availability of affordable hardware components. With the development of smart Ground-Based Synthetic Aperture Radar (GBSAR) system called GBSAR-Pi, we previously explored object classification applications based on raw radar data. Building upon this foundation, in this study, we analyze the potential of utilizing polarization information to improve the performance of deep learning models based on raw GBSAR data. The data are obtained with a GBSAR operating at 24 GHz with both vertical (VV) and horizontal (HH) polarization, resulting in two matrices (VV and HH) per observed scene. We present several approaches demonstrating the integration of such data into classification models based on a modified ResNet18 architecture. We also introduce a novel Siamese architecture tailored to accommodate the dual input radar data. The results indicate that a simple concatenation method is the most promising approach and underscore the importance of considering antenna polarization and merging strategies in deep learning applications based on radar data.

EResNet-SVM: an overfitting-relieved deep learning model for recognition of plant diseases and pests.

BACKGROUND: The accurate recognition and early warning for plant diseases and pests are a prerequisite of intelligent prevention and control for plant diseases and pests. As a result of the phenotype similarity of the hazarded plant after plant diseases and pests occur, as well as the interference of the external environment, traditional deep learning models often face the overfitting problem in phenotype recognition of plant diseases and pests, which leads to not only the slow convergence speed of the network, but also low recognition accuracy. RESULTS: Motivated by the above problems, the present study proposes a deep learning model EResNet-support vector machine (SVM) to alleviate the overfitting for the recognition and classification of plant diseases and pests. First, the feature extraction capability of the model is improved by increasing feature extraction layers in the convolutional neural network. Second, the order-reduced modules are embedded and a sparsely activated function is introduced to reduce model complexity and alleviate overfitting. Finally, a classifier fused by SVM and fully connected layers are introduced to transforms the original non-linear classification problem into a linear classification problem in high-dimensional space to further alleviate the overfitting and improve the recognition accuracy of plant diseases and pests. The ablation experiments further demonstrate that the fused structure can effectively alleviate the overfitting and improve the recognition accuracy. The experimental recognition results for typical plant diseases and pests show that the proposed EResNet-SVM model has 99.30% test accuracy for eight conditions (seven plant diseases and one normal), which is 5.90% higher than the original ResNet18. Compared with the classic AlexNet, GoogLeNet, Xception, SqueezeNet and DenseNet201 models, the accuracy of the EResNet-SVM model has improved by 5.10%, 7%, 8.10%, 6.20% and 1.90%, respectively. The testing accuracy of the EResNet-SVM model for 6 insect pests is 100%, which is 3.90% higher than that of the original ResNet18 model. CONCLUSION: This research provides not only useful references for alleviating the overfitting problem in deep learning, but also a theoretical and technical support for the intelligent detection and control of plant diseases and pests. © 2024 Society of Chemical Industry.

Humans in the loop: Community science and machine learning synergies for overcoming herbarium digitization bottlenecks.

Premise: Among the slowest steps in the digitization of natural history collections is converting imaged labels into digital text. We present here a working solution to overcome this long-recognized efficiency bottleneck that leverages synergies between community science efforts and machine learning approaches. Methods: We present two new semi-automated services. The first detects and classifies typewritten, handwritten, or mixed labels from herbarium sheets. The second uses a workflow tuned for specimen labels to label text using optical character recognition (OCR). The label finder and classifier was built via humans-in-the-loop processes that utilize the community science Notes from Nature platform to develop training and validation data sets to feed into a machine learning pipeline. Results: Our results showcase a >93% success rate for finding and classifying main labels. The OCR pipeline optimizes pre-processing, multiple OCR engines, and post-processing steps, including an alignment approach borrowed from molecular systematics. This pipeline yields >4-fold reductions in errors compared to off-the-shelf open-source solutions. The OCR workflow also allows human validation using a custom Notes from Nature tool. Discussion: Our work showcases a usable set of tools for herbarium digitization including a custom-built web application that is freely accessible. Further work to better integrate these services into existing toolkits can support broad community use.

Identifying the Growth Status of Hydroponic Lettuce Based on YOLO-EfficientNet.

Hydroponic lettuce was prone to pest and disease problems after transplantation. Manual identification of the current growth status of each hydroponic lettuce not only consumed time and was prone to errors but also failed to meet the requirements of high-quality and efficient lettuce cultivation. In response to this issue, this paper proposed a method called YOLO-EfficientNet for identifying the growth status of hydroponic lettuce. Firstly, the video data of hydroponic lettuce were processed to obtain individual frame images. And 2240 images were selected from these frames as the image dataset A. Secondly, the YOLO-v8n object detection model was trained using image dataset A to detect the position of each hydroponic lettuce in the video data. After selecting the targets based on the predicted bounding boxes, 12,000 individual lettuce images were obtained by cropping, which served as image dataset B. Finally, the EfficientNet-v2s object classification model was trained using image dataset B to identify three growth statuses (Healthy, Diseases, and Pests) of hydroponic lettuce. The results showed that, after training image dataset A using the YOLO-v8n model, the accuracy and recall were consistently around 99%. After training image dataset B using the EfficientNet-v2s model, it achieved excellent scores of 95.78 for Val-acc, 94.68 for Test-acc, 96.02 for Recall, 96.32 for Precision, and 96.18 for F1-score. Thus, the method proposed in this paper had potential in the agricultural application of identifying and classifying the growth status in hydroponic lettuce.

Autonomous Detection of Humans in Off-Limits Mountain Areas.

This paper is on the autonomous detection of humans in off-limits mountains. In off-limits mountains, a human rarely exists, thus human detection is an extremely rare event. Due to the advances in artificial intelligence, object detection-classification algorithms based on a Convolution Neural Network (CNN) can be used for this application. However, considering off-limits mountains, there should be no person in general. Thus, it is not desirable to run object detection-classification algorithms continuously, since they are computationally heavy. This paper addresses a time-efficient human detector system, based on both motion detection and object classification. The proposed scheme is to run a motion detection algorithm from time to time. In the camera image, we define a feasible human space where a human can appear. Once motion is detected inside the feasible human space, one enables the object classification, only inside the bounding box where motion is detected. Since motion detection inside the feasible human space runs much faster than an object detection-classification method, the proposed approach is suitable for real-time human detection with low computational loads. As far as we know, no paper in the literature used the feasible human space, as in our paper. The outperformance of our human detector system is verified by comparing it with other state-of-the-art object detection-classification algorithms (HOG detector, YOLOv7 and YOLOv7-tiny) under experiments. This paper demonstrates that the accuracy of the proposed human detector system is comparable to other state-of-the-art algorithms, while outperforming in computational speed. Our experiments show that in environments with no humans, the proposed human detector runs 62 times faster than YOLOv7 method, while showing comparable accuracy.

Orchard monitoring based on unmanned aerial vehicles and image processing by artificial neural networks: a systematic review.

Orchard monitoring is a vital direction of scientific research and practical application for increasing fruit production in ecological conditions. Recently, due to the development of technology and the decrease in equipment cost, the use of unmanned aerial vehicles and artificial intelligence algorithms for image acquisition and processing has achieved tremendous progress in orchards monitoring. This paper highlights the new research trends in orchard monitoring, emphasizing neural networks, unmanned aerial vehicles (UAVs), and various concrete applications. For this purpose, papers on complex topics obtained by combining keywords from the field addressed were selected and analyzed. In particular, the review considered papers on the interval 2017-2022 on the use of neural networks (as an important exponent of artificial intelligence in image processing and understanding) and UAVs in orchard monitoring and production evaluation applications. Due to their complexity, the characteristics of UAV trajectories and flights in the orchard area were highlighted. The structure and implementations of the latest neural network systems used in such applications, the databases, the software, and the obtained performances are systematically analyzed. To recommend some suggestions for researchers and end users, the use of the new concepts and their implementations were surveyed in concrete applications, such as a) identification and segmentation of orchards, trees, and crowns; b) detection of tree diseases, harmful insects, and pests; c) evaluation of fruit production, and d) evaluation of development conditions. To show the necessity of this review, in the end, a comparison is made with review articles with a related theme.

YDD-SLAM: Indoor Dynamic Visual SLAM Fusing YOLOv5 with Depth Information.

Simultaneous location and mapping (SLAM) technology is key in robot autonomous navigation. Most visual SLAM (VSLAM) algorithms for dynamic environments cannot achieve sufficient positioning accuracy and real-time performance simultaneously. When the dynamic object proportion is too high, the VSLAM algorithm will collapse. To solve the above problems, this paper proposes an indoor dynamic VSLAM algorithm called YDD-SLAM based on ORB-SLAM3, which introduces the YOLOv5 object detection algorithm and integrates deep information. Firstly, the objects detected by YOLOv5 are divided into eight subcategories according to their motion characteristics and depth values. Secondly, the depth ranges of the dynamic object and potentially dynamic object in the moving state in the scene are calculated. Simultaneously, the depth value of the feature point in the detection box is compared with that of the feature point in the detection box to determine whether the point is a dynamic feature point; if it is, the dynamic feature point is eliminated. Further, multiple feature point optimization strategies were developed for VSLAM in dynamic environments. A public data set and an actual dynamic scenario were used for testing. The accuracy of the proposed algorithm was significantly improved compared to that of ORB-SLAM3. This work provides a theoretical foundation for the practical application of a dynamic VSLAM algorithm.

A Unified Efficient Deep Learning Architecture for Rapid Safety Objects Classification Using Normalized Quantization-Aware Learning.

The efficient recognition and classification of personal protective equipment are essential for ensuring the safety of personnel in complex industrial settings. Using the existing methods, manually performing macro-level classification and identification of personnel in intricate spheres is tedious, time-consuming, and inefficient. The availability of several artificial intelligence models in recent times presents a new paradigm shift in object classification and tracking in complex settings. In this study, several compact and efficient deep learning model architectures are explored, and a new efficient model is constructed by fusing the learning capabilities of the individual, efficient models for better object feature learning and optimal inferencing. The proposed model ensures rapid identification of personnel in complex working environments for appropriate safety measures. The new model construct follows the contributory learning theory whereby each fussed model brings its learned features that are then combined to obtain a more accurate and rapid model using normalized quantization-aware learning. The major contribution of the work is the introduction of a normalized quantization-aware learning strategy to fuse the features learned by each of the contributing models. During the investigation, a separable convolutional driven model was constructed as a base model, and then the various efficient architectures were combined for the rapid identification and classification of the various hardhat classes used in complex industrial settings. A remarkable rapid classification and accuracy were recorded with the new resultant model.

Near-distance raw and reconstructed ground based SAR data.

Presented data includes two datasets named RealSAR-RAW and RealSAR-IMG. The first one contains unprocessed (raw) radar data obtained using Ground Based Synthetic Aperture Radar (GBSAR), while the second one contains images reconstructed using Omega-K algorithm applied to raw data from the first set. The GBSAR system moves the radar sensor along the track to virtually extend (synthesize) the antenna aperture and provides imaging data of the area in front of the system. The used sensor was a Frequency Modulated Continuous Wave (FMCW) radar with a central frequency of 24 GHz and a 700 MHz wide bandwidth which in our case covered the observed scene in 30 steps with 1 cm step size. The measured (recorded) scenes were made on combinations of three test objects (bottles) made of different material (aluminum, glass, and plastic) in different positions. The aim was to develop a small dataset of GBSAR data useful for classification applications focused on distinguishing different materials from sparse radar data.

Dataset for detecting motorcyclists in pedestrian areas.

This paper presents a semi-automated, scalable, and homologous methodology towards IoT implemented in Python for extracting and integrating images in pedestrian and motorcyclist areas on the road for constructing a multiclass object classifier. It consists of two stages. The first stage deals with creating a non-debugged data set by acquiring images related to the semantic context previously mentioned, using an embedded device connected 24/7 via Wi-Fi to a free and public CCTV service in Medellin, Colombia. Through artificial vision techniques, and automatically performs a comparative chronological analysis to download the images observed by 80 cameras that report data asynchronously. The second stage proposes two algorithms focused on debugging the previously obtained data set. The first one facilitates the user in labeling the data set not debugged through Regions of Interest (ROI) and hotkeys. It decomposes the information in the nth image of the data set in the same dictionary to store it in a binary Pickle file. The second one is nothing more than an observer of the classification performed by the user through the first algorithm to allow the user to verify if the information contained in the Pickle file built is correct.

Advanced Stiffness Sensing through the Pincer Grasping of Soft Pneumatic Grippers.

In this study, a comprehensive approach for sensing object stiffness through the pincer grasping of soft pneumatic grippers (SPGs) is presented. This study was inspired by the haptic sensing of human hands that allows us to perceive object properties through grasping. Many researchers have tried to imitate this capability in robotic grippers. The association between gripper performance and object reaction must be determined for this purpose. However, soft pneumatic actuators (SPA), the main components of SPGs, are extremely compliant. SPA compliance makes the determination of the association challenging. Methodologically, the connection between the behaviors of grasped objects and those of SPAs was clarified. A new concept of SPA modeling was then introduced. A method for stiffness sensing through SPG pincer grasping was developed based on this connection, and demonstrated on four samples. This method was validated through compression testing on the same samples. The results indicate that the proposed method yielded similar stiffness trends with slight deviations in compression testing. A main limitation in this study was the occlusion effect, which leads to dramatic deviations when grasped objects greatly deform. This is the first study to enable stiffness sensing and SPG grasping to be carried out in the same attempt. This study makes a major contribution to research on soft robotics by progressing the role of sensing for SPG grasping and object classification by offering an efficient method for acquiring another effective class of classification input. Ultimately, the proposed framework shows promise for future applications in inspecting and classifying visually indistinguishable objects.

Implementation of Field-Programmable Gate Array Platform for Object Classification Tasks Using Spike-Based Backpropagated Deep Convolutional Spiking Neural Networks.

This paper investigates the performance of deep convolutional spiking neural networks (DCSNNs) trained using spike-based backpropagation techniques. Specifically, the study examined temporal spike sequence learning via backpropagation (TSSL-BP) and surrogate gradient descent via backpropagation (SGD-BP) as effective techniques for training DCSNNs on the field programmable gate array (FPGA) platform for object classification tasks. The primary objective of this experimental study was twofold: (i) to determine the most effective backpropagation technique, TSSL-BP or SGD-BP, for deeper spiking neural networks (SNNs) with convolution filters across various datasets; and (ii) to assess the feasibility of deploying DCSNNs trained using backpropagation techniques on low-power FPGA for inference, considering potential configuration adjustments and power requirements. The aforementioned objectives will assist in informing researchers and companies in this field regarding the limitations and unique perspectives of deploying DCSNNs on low-power FPGA devices. The study contributions have three main aspects: (i) the design of a low-power FPGA board featuring a deployable DCSNN chip suitable for object classification tasks; (ii) the inference of TSSL-BP and SGD-BP models with novel network architectures on the FPGA board for object classification tasks; and (iii) a comparative evaluation of the selected spike-based backpropagation techniques and the object classification performance of DCSNNs across multiple metrics using both public (MNIST, CIFAR10, KITTI) and private (INHA_ADAS, INHA_KLP) datasets.

All Resistive Pressure-Temperature Bimodal Sensing E-Skin for Object Classification.

Electronic skin (E-skin) with multimodal sensing ability demonstrates huge prospects in object classification by intelligent robots. However, realizing the object classification capability of E-skin faces severe challenges in multiple types of output signals. Herein, a hierarchical pressure-temperature bimodal sensing E-skin based on all resistive output signals is developed for accurate object classification, which consists of laser-induced graphene/silicone rubber (LIG/SR) pressure sensing layer and NiO temperature sensing layer. The highly conductive LIG is employed as pressure-sensitive material as well as the interdigital electrode. Benefiting from high conductivity of LIG, pressure perception exhibits an excellent sensitivity of -34.15 kPa-1 . Meanwhile, a high temperature coefficient of resistance of -3.84%°C-1 is obtained in the range of 24-40 °C. More importantly, based on only electrical resistance as the output signal, the bimodal sensing E-skin with negligible crosstalk can simultaneously achieve pressure and temperature perception. Furthermore, a smart glove based on this E-skin enables classifying various objects with different shapes, sizes, and surface temperatures, which achieves over 92% accuracy under assistance of deep learning. Consequently, the hierarchical pressure-temperature bimodal sensing E-skin demonstrates potential application in human-machine interfaces, intelligent robots, and smart prosthetics.

A Survey on Deep Learning Based Segmentation, Detection and Classification for 3D Point Clouds.

The computer vision, graphics, and machine learning research groups have given a significant amount of focus to 3D object recognition (segmentation, detection, and classification). Deep learning approaches have lately emerged as the preferred method for 3D segmentation problems as a result of their outstanding performance in 2D computer vision. As a result, many innovative approaches have been proposed and validated on multiple benchmark datasets. This study offers an in-depth assessment of the latest developments in deep learning-based 3D object recognition. We discuss the most well-known 3D object recognition models, along with evaluations of their distinctive qualities.

Review of IoT Sensor Systems Used for Monitoring the Road Infrastructure.

An intelligent transportation system is one of the fundamental goals of the smart city concept. The Internet of Things (IoT) concept is a basic instrument to digitalize and automatize the process in the intelligent transportation system. Digitalization via the IoT concept enables the automatic collection of data usable for management in the transportation system. The IoT concept includes a system of sensors, actuators, control units and computational distribution among the edge, fog and cloud layers. The study proposes a taxonomy of sensors used for monitoring tasks based on motion detection and object tracking in intelligent transportation system tasks. The sensor's taxonomy helps to categorize the sensors based on working principles, installation or maintenance methods and other categories. The sensor's categorization enables us to compare the effectiveness of each sensor's system. Monitoring tasks are analyzed, categorized, and solved in intelligent transportation systems based on a literature review and focusing on motion detection and object tracking methods. A literature survey of sensor systems used for monitoring tasks in the intelligent transportation system was performed according to sensor and monitoring task categorization. In this review, we analyzed the achieved results to measure, sense, or classify events in intelligent transportation system monitoring tasks. The review conclusions were used to propose an architecture of the universal sensor system for common monitoring tasks based on motion detection and object tracking methods in intelligent transportation tasks. The proposed architecture was built and tested for the first experimental results in the case study scenario. Finally, we propose methods that could significantly improve the results in the following research.

Mapping environmentally sustainable urban development within six US cities through object-based image change analysis of aerial orthoimagery.

The rapid expansion of cities and continuous urban population growth underscores a need for sustainable urban development. Sustainable development is that which addresses human needs, contributes to well-being, is economically viable, and utilizes natural resources at a degree sustainable by the surrounding environmental systems. Urban green spaces, green roofs, and solar panels are examples of environmentally sustainable urban development (ESUD), or development that focuses on environmental impact, but also presents the potential to achieve social and economic sustainability. The aim of this study was to map and compare amounts of ESUD c. 2010 and c. 2019 through an object-based image analysis (OBIA) approach using National Agricultural Imagery Program (NAIP) aerial orthoimagery for six mid- to large-size cities in the USA. The results of this study indicate a hybrid OBIA and manual interpretation approach applied to NAIP orthoimagery may allow for reliable mapping and areal estimation of urban green space and green roof changes in urban areas. The reliability of OBIA-only mapping and estimation of areal extents of existing green roofs, and new and existing solar panels, is inconclusive due to low mapping accuracy and coarse spatial resolution of aerial orthoimagery relative to some ESUD features. The three urban study areas in humid continental climate zones (Dfa) were estimated to have greater areal extent of new and existing urban green space and existing green roofs, but less areal extent of new green roofs and existing solar panels compared to the three study areas in humid subtropical climate zones (Cfa).

Multi-Class Classification and Multi-Output Regression of Three-Dimensional Objects Using Artificial Intelligence Applied to Digital Holographic Information.

Digital holographically sensed 3D data processing, which is useful for AI-based vision, is demonstrated. Three prominent methods of learning from datasets such as sensed holograms, computationally retrieved intensity and phase from holograms forming concatenated intensity-phase (whole information) images, and phase-only images (depth information) were utilized for the proposed multi-class classification and multi-output regression tasks of the chosen 3D objects in supervised learning. Each dataset comprised 2268 images obtained from the chosen eighteen 3D objects. The efficacy of our approaches was validated on experimentally generated digital holographic data then further quantified and compared using specific evaluation matrices. The machine learning classifiers had better AUC values for different classes on the holograms and whole information datasets compared to the CNN, whereas the CNN had a better performance on the phase-only image dataset compared to these classifiers. The MLP regressor was found to have a stable prediction in the test and validation sets with a fixed EV regression score of 0.00 compared to the CNN, the other regressors for holograms, and the phase-only image datasets, whereas the RF regressor showed a better performance in the validation set for the whole information dataset with a fixed EV regression score of 0.01 compared to the CNN and other regressors.