Inorganic Halide Perovskite Nanowires/Conjugated Polymer Heterojunction-Based Optoelectronic Synaptic Transistors for Dynamic Machine Vision.

Inspired by the retina, artificial optoelectronic synapses have groundbreaking potential for machine vision. The field-effect transistor is a crucial platform for optoelectronic synapses that is highly sensitive to external stimuli and can modulate conductivity. On the basis of the decent optical absorption, perovskite materials have been widely employed for constructing optoelectronic synaptic transistors. However, the reported optoelectronic synaptic transistors focus on the static processing of independent stimuli at different moments, while the natural visual information consists of temporal signals. Here, we report CsPbBrI2 nanowire-based optoelectronic synaptic transistors to study the dynamic responses of artificial synaptic transistors to time-varying visual information for the first time. Moreover, on the basis of the dynamic synaptic behavior, a hardware system with an accuracy of 85% is built to the trajectory of moving objects. This work offers a new way to develop artificial optoelectronic synapses for the construction of dynamic machine vision systems.

A Common Knowledge-Driven Generic Vision Inspection Framework for Adaptation to Multiple Scenarios, Tasks, and Objects.

The industrial manufacturing model is undergoing a transformation from a product-centric model to a customer-centric one. Driven by customized requirements, the complexity of products and the requirements for quality have increased, which pose a challenge to the applicability of traditional machine vision technology. Extensive research demonstrates the effectiveness of AI-based learning and image processing on specific objects or tasks, but few publications focus on the composite task of the integrated product, the traceability and improvability of methods, as well as the extraction and communication of knowledge between different scenarios or tasks. To address this problem, this paper proposes a common, knowledge-driven, generic vision inspection framework, targeted for standardizing product inspection into a process of information decoupling and adaptive metrics. Task-related object perception is planned into a multi-granularity and multi-pattern progressive alignment based on industry knowledge and structured tasks. Inspection is abstracted as a reconfigurable process of multi-sub-pattern space combination mapping and difference metric under appropriate high-level strategies and experiences. Finally, strategies for knowledge improvement and accumulation based on historical data are presented. The experiment demonstrates the process of generating a detection pipeline for complex products and continuously improving it through failure tracing and knowledge improvement. Compared to the (1.767°, 69.802 mm) and 0.883 obtained by state-of-the-art deep learning methods, the generated pipeline achieves a pose estimation ranging from (2.771°, 153.584 mm) to (1.034°, 52.308 mm) and a detection rate ranging from 0.462 to 0.927. Through verification of other imaging methods and industrial tasks, we prove that the key to adaptability lies in the mining of inherent commonalities of knowledge, multi-dimensional accumulation, and reapplication.

A Method for Measuring Shaft Diameter Based on Light Stripe Image Enhancement.

When the workpiece surface exhibits strong reflectivity, it becomes challenging to obtain accurate key measurements using non-contact, visual measurement techniques due to poor image quality. In this paper, we propose a high-precision measurement method shaft diameter based on an enhanced quality stripe image. By capturing two stripe images with different exposure times, we leverage their different characteristics. The results extracted from the low-exposure image are used to perform grayscale correction on the high-exposure image, improving the distribution of stripe grayscale and resulting in more accurate extraction results for the center points. The incorporation of different measurement positions and angles further enhanced measurement precision and robustness. Additionally, ellipse fitting is employed to derive shaft diameter. This method was applied to the profiles of different cross-sections and angles within the same shaft segment. To reduce the shape error of the shaft measurement, the average of these measurements was taken as the estimate of the average diameter for the shaft segment. In the experiments, the average shaft diameters determined by averaging elliptical estimations were compared with shaft diameters obtained using a coordinate measuring machine (CMM) the maximum error and the minimum error were respectively 18 µm and 7 µm; the average error was 11 µm; and the root mean squared error of the multiple measurement results was 10.98 µm. The measurement accuracy achieved is six times higher than that obtained from the unprocessed stripe images.

A Lightweight and Efficient Multi-Type Defect Detection Method for Transmission Lines Based on DCP-YOLOv8.

Currently, the intelligent defect detection of massive grid transmission line inspection pictures using AI image recognition technology is an efficient and popular method. Usually, there are two technical routes for the construction of defect detection algorithm models: one is to use a lightweight network, which improves the efficiency, but it can generally only target a few types of defects and may reduce the detection accuracy; the other is to use a complex network model, which improves the accuracy, and can identify multiple types of defects at the same time, but it has a large computational volume and low efficiency. To maintain the model's high detection accuracy as well as its lightweight structure, this paper proposes a lightweight and efficient multi type defect detection method for transmission lines based on DCP-YOLOv8. The method employs deformable convolution (C2f_DCNv3) to enhance the defect feature extraction capability, and designs a re-parameterized cross phase feature fusion structure (RCSP) to optimize and fuse high-level semantic features with low level spatial features, thus improving the capability of the model to recognize defects at different scales while significantly reducing the model parameters; additionally, it combines the dynamic detection head and deformable convolutional v3's detection head (DCNv3-Dyhead) to enhance the feature expression capability and the utilization of contextual information to further improve the detection accuracy. Experimental results show that on a dataset containing 20 real transmission line defects, the method increases the average accuracy (mAP@0.5) to 72.2%, an increase of 4.3%, compared with the lightest baseline YOLOv8n model; the number of model parameters is only 2.8 M, a reduction of 9.15%, and the number of processed frames per second (FPS) reaches 103, which meets the real time detection demand. In the scenario of multi type defect detection, it effectively balances detection accuracy and performance with quantitative generalizability.

A Cost-Effective Integrated Methodology for Electromagnetic Actuation via Visual Feedback.

Electromagnetic actuation can support many fields of technology, such as robotics or biomedical applications. In this context, fully understanding the system behavior and proposing a low-cost package for feedback control is challenging. Modeling the electromagnetic force is particularly tricky because it is a nonlinear function of the actuated object's position and coil's current. Measuring in real time the position of the actuated object with the precision required for accurate motion control is also nontrivial. In this study, we propose a novel, cost-effective electromagnetic set-up to achieve position control via visual feedback. We actuated vertically and under different experimental conditions a 10 mm diameter steel ball hanging on a low-stiffness spring, demonstrating good tracking performance (the position error remained within ±0.5 mm, with a negligible phase delay in the best scenarios). The experimental results confirm the feasibility of the proposed set-up, which is characterized by minimum complexity and realized with off-the-shelf and cost-effective components. For these reasons, such a contribution helps to understand and apply electromagnetic actuation even further.

Object Detection and Tracking with YOLO and the Sliding Innovation Filter.

Object detection and tracking are pivotal tasks in machine learning, particularly within the domain of computer vision technologies. Despite significant advancements in object detection frameworks, challenges persist in real-world tracking scenarios, including object interactions, occlusions, and background interference. Many algorithms have been proposed to carry out such tasks; however, most struggle to perform well in the face of disturbances and uncertain environments. This research proposes a novel approach by integrating the You Only Look Once (YOLO) architecture for object detection with a robust filter for target tracking, addressing issues of disturbances and uncertainties. The YOLO architecture, known for its real-time object detection capabilities, is employed for initial object detection and centroid location. In combination with the detection framework, the sliding innovation filter, a novel robust filter, is implemented and postulated to improve tracking reliability in the face of disturbances. Specifically, the sliding innovation filter is implemented to enhance tracking performance by estimating the optimal centroid location in each frame and updating the object's trajectory. Target tracking traditionally relies on estimation theory techniques like the Kalman filter, and the sliding innovation filter is introduced as a robust alternative particularly suitable for scenarios where a priori information about system dynamics and noise is limited. Experimental simulations in a surveillance scenario demonstrate that the sliding innovation filter-based tracking approach outperforms existing Kalman-based methods, especially in the presence of disturbances. In all, this research contributes a practical and effective approach to object detection and tracking, addressing challenges in real-world, dynamic environments. The comparative analysis with traditional filters provides practical insights, laying the groundwork for future work aimed at advancing multi-object detection and tracking capabilities in diverse applications.

Research on Multi-Hole Localization Tracking Based on a Combination of Machine Vision and Deep Learning.

In the process of industrial production, manual assembly of workpieces exists with low efficiency and high intensity, and some of the assembly process of the human body has a certain degree of danger. At the same time, traditional machine learning algorithms are difficult to adapt to the complexity of the current industrial field environment; the change in the environment will greatly affect the accuracy of the robot's work. Therefore, this paper proposes a method based on the combination of machine vision and the YOLOv5 deep learning model to obtain the disk porous localization information, after coordinate mapping by the ROS communication control robotic arm work, in order to improve the anti-interference ability of the environment and work efficiency but also reduce the danger to the human body. The system utilizes a camera to collect real-time images of targets in complex environments and, then, trains and processes them for recognition such that coordinate localization information can be obtained. This information is converted into coordinates under the robot coordinate system through hand-eye calibration, and the robot is then controlled to complete multi-hole localization and tracking by means of communication between the upper and lower computers. The results show that there is a high accuracy in the training and testing of the target object, and the control accuracy of the robotic arm is also relatively high. The method has strong anti-interference to the complex environment of industry and exhibits a certain feasibility and effectiveness. It lays a foundation for achieving the automated installation of docking disk workpieces in industrial production and also provides a more favorable choice for the production and installation of the process of screw positioning needs.

Revolutionizing Robotic Depalletizing: AI-Enhanced Parcel Detecting with Adaptive 3D Machine Vision and RGB-D Imaging for Automated Unloading.

Detecting parcels accurately and efficiently has always been a challenging task when unloading from trucks onto conveyor belts because of the diverse and complex ways in which parcels are stacked. Conventional methods struggle to quickly and accurately classify the various shapes and surface patterns of unordered parcels. In this paper, we propose a parcel-picking surface detection method based on deep learning and image processing for the efficient unloading of diverse and unordered parcels. Our goal is to develop a systematic image processing algorithm that emphasises the boundaries of parcels regardless of their shape, pattern, or layout. The core of the algorithm is the utilisation of RGB-D technology for detecting the primary boundary lines regardless of obstacles such as adhesive labels, tapes, or parcel surface patterns. For cases where detecting the boundary lines is difficult owing to narrow gaps between parcels, we propose using deep learning-based boundary line detection through the You Only Look at Coefficients (YOLACT) model. Using image segmentation techniques, the algorithm efficiently predicts boundary lines, enabling the accurate detection of irregularly sized parcels with complex surface patterns. Furthermore, even for rotated parcels, we can extract their edges through complex mathematical operations using the depth values of the specified position, enabling the detection of the wider surfaces of the rotated parcels. Finally, we validate the accuracy and real-time performance of our proposed method through various case studies, achieving mAP (50) values of 93.8% and 90.8% for randomly sized and rotationally covered boxes with diverse colours and patterns, respectively.

Low-Cost Plant-Protection Unmanned Ground Vehicle System for Variable Weeding Using Machine Vision.

This study presents a machine vision-based variable weeding system for plant- protection unmanned ground vehicles (UGVs) to address the issues of pesticide waste and environmental pollution that are readily caused by traditional spraying agricultural machinery. The system utilizes fuzzy rules to achieve adaptive modification of the Kp, Ki, and Kd adjustment parameters of the PID control algorithm and combines them with an interleaved period PWM controller to reduce the impact of nonlinear variations in water pressure on the performance of the system, and to improve the stability and control accuracy of the system. After testing various image threshold segmentation and image graying algorithms, the normalized super green algorithm (2G-R-B) and the fast iterative threshold segmentation method were adopted as the best combination. This combination effectively distinguished between the vegetation and the background, and thus improved the accuracy of the pixel extraction algorithm for vegetation distribution. The results of orthogonal testing by selected four representative spraying duty cycles-25%, 50%, 75%, and 100%-showed that the pressure variation was less than 0.05 MPa, the average spraying error was less than 2%, and the highest error was less than 5% throughout the test. Finally, the performance of the system was comprehensively evaluated through field trials. The evaluation showed that the system was able to adjust the corresponding spraying volume in real time according to the vegetation distribution under the decision-making based on machine vision algorithms, which proved the low cost and effectiveness of the designed variable weed control system.

Real-Time Detection of Microplastics Using an AI Camera.

Microplastics (MPs, size ≤ 5 mm) have emerged as a significant worldwide concern, threatening marine and freshwater ecosystems, and the lack of MP detection technologies is notable. The main goal of this research is the development of a camera sensor for the detection of MPs and measuring their size and velocity while in motion. This study introduces a novel methodology involving computer vision and artificial intelligence (AI) for the detection of MPs. Three different camera systems, including fixed-focus 2D and autofocus (2D and 3D), were implemented and compared. A YOLOv5-based object detection model was used to detect MPs in the captured image. DeepSORT was then implemented for tracking MPs through consecutive images. In real-time testing in a laboratory flume setting, the precision in MP counting was found to be 97%, and during field testing in a local river, the precision was 96%. This study provides foundational insights into utilizing AI for detecting MPs in different environmental settings, contributing to more effective efforts and strategies for managing and mitigating MP pollution.

Performance Assessment of Object Detection Models Trained with Synthetic Data: A Case Study on Electrical Equipment Detection.

This paper explores a data augmentation approach for images of rigid bodies, particularly focusing on electrical equipment and analogous industrial objects. By leveraging manufacturer-provided datasheets containing precise equipment dimensions, we employed straightforward algorithms to generate synthetic images, permitting the expansion of the training dataset from a potentially unlimited viewpoint. In scenarios lacking genuine target images, we conducted a case study using two well-known detectors, representing two machine-learning paradigms: the Viola-Jones (VJ) and You Only Look Once (YOLO) detectors, trained exclusively on datasets featuring synthetic images as the positive examples of the target equipment, namely lightning rods and potential transformers. Performances of both detectors were assessed using real images in both visible and infrared spectra. YOLO consistently demonstrates F1 scores below 26% in both spectra, while VJ's scores lie in the interval from 38% to 61%. This performance discrepancy is discussed in view of paradigms' strengths and weaknesses, whereas the relatively high scores of at least one detector are taken as empirical evidence in favor of the proposed data augmentation approach.

GC Snakes: An Efficient and Robust Segmentation Model for Hot Forging Images.

Machine vision is a desirable non-contact measurement method for hot forgings, as image segmentation has been a challenging issue in performance and robustness resulting from the diversity of working conditions for hot forgings. Thus, this paper proposes an efficient and robust active contour model and corresponding image segmentation approach for forging images, by which verification experiments are conducted to prove the performance of the segmentation method by measuring geometric parameters for forging parts. Specifically, three types of continuity parameters are defined based on the geometric continuity of equivalent grayscale surfaces for forging images; hence, a new image force and external energy functional are proposed to form a new active contour model, Geometric Continuity Snakes (GC Snakes), which is more percipient to the grayscale distribution characteristics of forging images to improve the convergence for active contour robustly; additionally, a generating strategy for initial control points for GC Snakes is proposed to compose an efficient and robust image segmentation approach. The experimental results show that the proposed GC Snakes has better segmentation performance compared with existing active contour models for forging images of different temperatures and sizes, which provides better performance and efficiency in geometric parameter measurement for hot forgings. The maximum positioning and dimension errors by GC Snakes are 0.5525 mm and 0.3868 mm, respectively, compared with errors of 0.7873 mm and 0.6868 mm by the Snakes model.

Multispectral imaging for distinguishing hybrid forest seeds of Corymbia spp. and Eucalyptus spp. from their progenitors.

In the forest industry, interspecific hybridization, such as Eucalyptus urograndis (Eucalyptus grandis × Eucalyptus urophylla) and Corymbia maculata × Corymbia torelliana, has led to the development of high-performing F1 generations. The successful breeding of these hybrids relies on verifying progenitor origins and confirming post-crossing, but conventional genotype identification methods are resource-intensive and result in seed destruction. As an alternative, multispectral imaging analysis has emerged as an efficient and non-destructive tool for seed phenotyping. This approach has demonstrated success in various crop seeds. However, identifying seed species in the context of forest seeds presents unique challenges due to their natural phenotypic variability and the striking resemblance between different species. This study evaluates the efficacy of spectral imaging analysis in distinguishing hybrid seeds of E. urograndis and C. maculata × C. torelliana from their progenitors. Four experiments were conducted: one for Corymbia spp. seeds, one for each Eucalyptus spp. batch separately, and one for pooled batches. Multispectral images were acquired at 19 wavelengths within the spectral range of 365-970 nm. Classification models based on Linear Discriminant Analysis (LDA), Random Forest (RF), and Support Vector Machine (SVM) was created using reflectance and reflectance features, combined with color, shape, and texture features, as well as nCDA transformed features. The LDA algorithm, combining all features, provided the highest accuracy, reaching 98.15% for Corymbia spp., and 92.75%, 85.38, and 86.00 for Eucalyptus batch one, two, and pooled batches, respectively. The study demonstrated the effectiveness of multispectral imaging in distinguishing hybrid seeds of Eucalyptus and Corymbia species. The seeds' spectral signature played a key role in this differentiation. This technology holds great potential for non-invasively classifying forest seeds in breeding programs.

Applying artificial intelligence to determination of legal age of majority from radiographic data.

Forensic odontologists use biological patterns to estimate chronological age for the judicial system. The age of majority is a legally significant period with a limited set of reliable oral landmarks. Currently, experts rely on the questionable development of third molars to assess whether litigants can be prosecuted as legal adults. Identification of new and novel patterns may illuminate features more dependably indicative of chronological age, which have, until now, remained unseen. Unfortunately, biased perceptions and limited cognitive capacity compromise the ability of researchers to notice new patterns. The present study demonstrates how artificial intelligence can break through identification barriers and generate new estimation modalities. A convolutional neural network was trained with 4003 panoramic-radiographs to sort subjects into 'under-18' and 'over-18' age categories. The resultant architecture identified legal adults with a high predictive accuracy equally balanced between precision, specificity and recall. Moving forward, AI-based methods could improve courtroom efficiency, stand as automated assessment methods and contribute to our understanding of biological ageing.

Application of IoT technology based on neural networks in basketball training motion capture and injury prevention.

Basketball players need to frequently engage in various physical movements during the game, which puts a certain burden on their bodies and can easily lead to various sports injuries. Therefore, it is crucial to prevent sports injuries in basketball teaching. This paper also studies basketball motion track capture. Basketball motion capture preserves the motion posture information of the target person in three-dimensional space. Because the motion capture system based on machine vision often encounters problems such as occlusion or self occlusion in the application scene, human motion capture is still a challenging problem in the current research field. This article designs a multi perspective human motion trajectory capture algorithm framework, which uses a two-dimensional human motion pose estimation algorithm based on deep learning to estimate the position distribution of human joint points on the two-dimensional image from each perspective. By combining the knowledge of camera poses from multiple perspectives, the three-dimensional spatial distribution of joint points is transformed, and the final evaluation result of the target human 3D pose is obtained. This article applies the research results of neural networks and IoT devices to basketball motion capture methods, further developing basketball motion capture systems.

The machine-vision image guided surgery system reduces fluoroscopy time, ionizing radiation and intraoperative blood loss in posterior spinal fusion for scoliosis.

PURPOSE: To determine if the novel 3D Machine-Vision Image Guided Surgery (MvIGS) (FLASH™) system can reduce intraoperative radiation exposure, while improving surgical outcomes when compared to 2D fluoroscopic navigation. METHODS: Clinical and radiographic records of 128 patients (≤ 18 years of age) who underwent posterior spinal fusion (PSF), utilising either MvIGS or 2D fluoroscopy, for severe idiopathic scoliosis were retrospectively reviewed. Operative time was analysed using the cumulative sum (CUSUM) method to evaluate the learning curve for MvIGS. RESULTS: Between 2017 and 2021, 64 patients underwent PSF using pedicle screws with 2D fluoroscopy and another 64 with the MvIGS. Age, gender, BMI, and scoliosis aetiology were comparable between the two groups. The CUSUM method estimated that the MvIGS learning curve with respect to operative time was 9 cases. This curve consisted of 2 phases: Phase 1 comprises the first 9 cases and Phase 2 the remaining 55 cases. Compared to 2D fluoroscopy, MvIGS reduced intraoperative fluoroscopy time, radiation exposure, estimated blood loss and length of stay by 53%, 62% 44%, and 21% respectively. Scoliosis curve correction was 4% higher in the MvIGS group, without any increase in operative time. CONCLUSION: MvIGS for screw insertion in PSF contributed to a significant reduction in intraoperative radiation exposure and fluoroscopy time, as well as blood loss and length of stay. The real-time feedback and ability to visualize the pedicle in 3D with MvIGS enabled greater curve correction without increasing the operative time.

Synchronous Control of a Group of Flying Robots Following a Leader UAV in an Unfamiliar Environment.

An increasing number of professional drone flights require situational awareness of aerial vehicles. Vehicles in a group of drones must be aware of their surroundings and the other group members. The amount of data to be exchanged and the total cost are skyrocketing. This paper presents an implementation and assessment of an organized drone group comprising a fully aware leader and much less expensive followers. The solution achieved a significant cost reduction by decreasing the number of sensors onboard followers and improving the organization and manageability of the group in the system. In this project, a group of quadrotor drones was evaluated. An automatically flying leader was followed by drones equipped with low-end cameras only. The followers were tasked with following ArUco markers mounted on a preceding drone. Several test tasks were designed and conducted. Finally, the presented system proved appropriate for slowly moving groups of drones.

Method of Building Detection in Optical Remote Sensing Images Based on SegFormer.

An appropriate detection network is required to extract building information in remote sensing images and to relieve the issue of poor detection effects resulting from the deficiency of detailed features. Firstly, we embed a transposed convolution sampling module fusing multiple normalization activation layers in the decoder based on the SegFormer network. This step alleviates the issue of missing feature semantics by adding holes and fillings, cascading multiple normalizations and activation layers to hold back over-fitting regularization expression and guarantee steady feature parameter classification. Secondly, the atrous spatial pyramid pooling decoding module is fused to explore multi-scale contextual information and to overcome issues such as the loss of detailed information on local buildings and the lack of long-distance information. Ablation experiments and comparison experiments are performed on the remote sensing image AISD, MBD, and WHU dataset. The robustness and validity of the improved mechanism are demonstrated by control groups of ablation experiments. In comparative experiments with the HRnet, PSPNet, U-Net, DeepLabv3+ networks, and the original detection algorithm, the mIoU of the AISD, the MBD, and the WHU dataset is enhanced by 17.68%, 30.44%, and 15.26%, respectively. The results of the experiments show that the method of this paper is superior to comparative methods such as U-Net. Furthermore, it is better for integrity detection of building edges and reduces the number of missing and false detections.

Analysis of Training Deep Learning Models for PCB Defect Detection.

Recently, many companies have introduced automated defect detection methods for defect-free PCB manufacturing. In particular, deep learning-based image understanding methods are very widely used. In this study, we present an analysis of training deep learning models to perform PCB defect detection stably. To this end, we first summarize the characteristics of industrial images, such as PCB images. Then, the factors that can cause changes (contamination and quality degradation) to the image data in the industrial field are analyzed. Subsequently, we organize defect detection methods that can be applied according to the situation and purpose of PCB defect detection. In addition, we review the characteristics of each method in detail. Our experimental results demonstrated the impact of various degradation factors, such as defect detection methods, data quality, and image contamination. Based on our overview of PCB defect detection and experiment results, we present knowledge and guidelines for correct PCB defect detection.

Classification of Fluorescently Labelled Maize Kernels Using Convolutional Neural Networks.

Accurate real-time classification of fluorescently labelled maize kernels is important for the industrial application of its advanced breeding techniques. Therefore, it is necessary to develop a real-time classification device and recognition algorithm for fluorescently labelled maize kernels. In this study, a machine vision (MV) system capable of identifying fluorescent maize kernels in real time was designed using a fluorescent protein excitation light source and a filter to achieve optimal detection. A high-precision method for identifying fluorescent maize kernels based on a YOLOv5s convolutional neural network (CNN) was developed. The kernel sorting effects of the improved YOLOv5s model, as well as other YOLO models, were analysed and compared. The results show that using a yellow LED light as an excitation light source combined with an industrial camera filter with a central wavelength of 645 nm achieves the best recognition effect for fluorescent maize kernels. Using the improved YOLOv5s algorithm can increase the recognition accuracy of fluorescent maize kernels to 96%. This study provides a feasible technical solution for the high-precision, real-time classification of fluorescent maize kernels and has universal technical value for the efficient identification and classification of various fluorescently labelled plant seeds.