Stereovision-Based Ego-Motion Estimation for Combine Harvesters.

Ego-motion estimation is a foundational capability for autonomous combine harvesters, supporting high-level functions such as navigation and harvesting. This paper presents a novel approach for estimating the motion of a combine harvester from a sequence of stereo images. The proposed method starts with tracking a set of 3D landmarks which are triangulated from stereo-matched features. Six Degree of Freedom (DoF) ego motion is obtained by minimizing the reprojection error of those landmarks on the current frame. Then, local bundle adjustment is performed to refine structure (i.e., landmark positions) and motion (i.e., keyframe poses) jointly in a sliding window. Both processes are encapsulated into a two-threaded architecture to achieve real-time performance. Our method utilizes a stereo camera, which enables estimation at true scale and easy startup of the system. Quantitative tests were performed on real agricultural scene data, comprising several different working paths, in terms of estimating accuracy and real-time performance. The experimental results demonstrated that our proposed perception system achieved favorable accuracy, outputting the pose at 10 Hz, which is sufficient for online ego-motion estimation for combine harvesters.

Development of Impurity-Detection System for Tracked Rice Combine Harvester Based on DEM and Mask R-CNN.

Impurity rate is one of the key performance indicators of the rice combine harvester and is also the main basis for parameter regulation. At present, the tracked rice combine harvester impurity rates cannot be monitored in real time. Due to the lack of parameter regulation basis, the harvest working parameters are set according to the operator's experience and not adjusted during the operation, which leads to the harvest quality fluctuating greatly in a complex environment. In this paper, an impurity-detection system, including a grain-sampling device and machine vision system, was developed. Sampling device structure and impurity extraction algorithm were studied to enhance the impurity identification accuracy. To reduce the effect of impurity occlusion on visual recognition, an infusion-type sampling device was designed. The sampling device light source form was determined based on the brightness histogram analysis of a captured image under different light irradiations. The effect of sampling device structures on impurity visualization, grain distribution, and mass flow rate was investigated by the discrete element method (DEM). The impurity recognition algorithm was proposed based on Mask R-CNN, which mainly includes an impurity feature extraction network, an ROI generation network, and a target segmentation network. The test set experiment showed that the precision rate, recall rate, average precision, and comprehensive evaluation indicator of the impurity recognition model were 92.49%, 88.63%, 81.47%, and 90.52%, respectively. The conversion between impurity pixel number and its actual mass was realized according to the pixel density calibration test and impurity rate correction factor. The bench test result showed that the designed system has a good detection accuracy of 91.15~97.26% for the five varieties. The result relative error was in a range of 5.71~11.72% between the impurity-detection system and manual method in field conditions. The impurity-detection system could be applied to tracked rice combine harvesters to provide a reference for the adjustment of operating parameters.

Toward real-time and accurate dense 3D mapping of crop fields for combine harvesters using a stereo camera.

Fast and accurate 3D scene perception is a crucial prerequisite for the autonomous navigation and harvesting of combine harvesters. However, crop field scenarios pose severe challenges for vision-based perception systems due to repetitive scenes, illumination changes and real-time constraints on embedded computing platforms. In this paper, we propose a feature-based, two-stage approach for real-time dense 3D mapping for combine harvesters. In the first stage, our approach constructs a sparse 3D map using reliable feature matching, which provides prior knowledge about the environment. In the second stage, our method formulates per-pixel disparity calculation as probabilistic inference. The key to our approach is the ability to compute dense 3D maps by combining Bayesian estimation with efficient and discriminative point cues from images, exhibiting tolerance against visual measurement uncertainties due to repetitive textures and uneven lighting in crop fields. We validate the performance of the proposed method using real crop field data, and the results demonstrate that our dense 3D maps provide detailed spatial metric information while maintaining a balance between accuracy and efficiency. This makes our approach highly valuable for online perception in combine harvesters operating with resource-limited systems.

Cutting speed follow-up adjusting system of bidirectional electric drive side cutter for rape combine harvester.

In rape combine harvester, side cutter must be equipped to cut off tangled rapeseed twigs. Inappropriate cutting speed would increase the repeated cutting and missing cutting of side cutter, which lead to serious header loss. In allusion to the problems mentioned above, bidirectional electric drive side cutter and a cutting speed follow-up adjusting system were proposed. The kinematic law of side cutter blades was analyzed. The trajectory, velocity, and acceleration of the two blades were the same, but the phase difference is π. Numerical simulation of cutting areas at different cutting speed ratios was carried out and the best cutting speed ratio was determined to be 1.1. Cutting speed follow-up adjusting system was designed based on matching relationship between combine harvester forward speed and side cutter cutting speed. Cutting speed follow-up adjusting system was designed with proportional-integral-derivative (PID) algorithm. The control parameters were determined to be Kp = 1.3, Ki = 4.3, Kd = 0.007. Simulation showed that the maximum overshoot of the system was 4.3%, steady-state error was 0.24%, and the rise time was 0.036 s. The cutting speed follow-up adjusting system was applied to the 4LZ-6T-type rape combine harvester. Experimental results showed that the side cutter cutting speed error was within 1.5%. When forward speed changed, the cutting speed response delay time was 1.5 s. The rape combine harvester header average loss was 2.96% and side cutter average loss was 0.81%. Compared to the fixed speed cutting, header loss was reduced by 14.05% and side cutter loss was reduced by 34.76%. The research can reduce the loss of rapeseed combine harvester and provide theoretical basis for the design of rapeseed combine harvester.