基于车载计算机视觉的推土机铲运作业识别方法研究

陈嘉锡

摘 要

推土机作为一种主要的土方工程机械，因其工作特点鲜明，应用场景多样，可以在土方工程的多个工序中使用。因此，推土机的施工作业生产效率极大程度的影响了土方平整工作的完成时间与施工效果，土方现场一般需指派施工员在推土机旁，凭五官感受及主观经验以监督推土机的施工作业，属于劳动密集型工作，费时费力且效果不佳。而随着技术的进步，监控所需硬件的成本不断降低，相比之下，人力成本却不断增加，由现实的工程需要及经济上的考量产生了用摄像头代替施工员“人眼”的设想，以监督推土机的施工作业。本文使用计算机视觉的方法来自动化识别推土机的铲运作业，主要包括以下工作：

首先，本文在已有的国内外研究基础上，分析了现场施工员对推土机施工作业的监督重点，结合对推土机铲运作业的特点分析，将对推土机施工作业效率的监督问题转化为铲运状态与动作的识别问题。为了获取识别所需的数据样本，设计并实现了车载视频的部署安装方案，并整体分析了推土机铲运作业时各向视角的视频数据特征，为后续推土机铲运状态及铲运动作的识别奠定了理论基础；

其次，分析论证后选定车头靠前处中部摄像头处的视频数据作为铲运状态与铲运动作识别的数据来源，其中铲运状态的识别借助于图像样本即可，铲运动作的识别因涉及到时序信息的处理，需以视频样本作为数据集，图像样本与视频样本均需做数据预处理以增强模型的鲁棒性，提升模型的泛化性能力。对于铲运状态与铲运动作的识别均选取了三个主流的深度学习算法，并通过调参寻优得到各自最优的参数组合，其中识别铲运状态效果相对最优的算法是YOLOv5,识别铲运动作效果相对最优的算法是LRCN。通过对YOLOv5算法的损失函数与回归框筛选方法进行优化，模型的mAP值分别提高了2.12%与2.18%；通过对LRCN算法的损失函数、特征提取网络，时序学习单元进行优化，模型的准确率分别提高了2.55%、1.82%、1.68%。

最后，对推土机铲运状态、铲运动作的识别均已达到较好的效果，但两算法间彼此独立，而单识别铲运状态或动作无法满足土方现场对推土机作业效率的现实监控需求，所以提出将两个独立算法耦合，以实现铲运状态与铲运动作的同步识别。

结果表明，本文提出的铲运状态与动作同步识别算法可以满足自动化识别推土机铲运作业情况的实际需求，通过提供一个可视化的过程中监督的方法，减轻了在大量的施工扬尘与噪音的现场下，施工员对土方机械施工作业的监督压力，并为推土机生产效率的提高提供了数据基础。

关键词： 推土机；铲运状态；铲运动作；深度学习；同步识别

Abstract

As a main earthmoving machinery, bulldozers can be used in multiple processes of earthmoving due to their distinctive working characteristics and diverse application scenarios. Therefore, the construction work production efficiency of bulldozers greatly affects the completion time and construction effect of earthwork leveling work. Generally, construction workers need to be assigned to the earthwork site beside the bulldozer to supervise the construction work of the bulldozer based on five senses and subjective experience, which belongs to labor. Intensive work, time consuming and ineffective. With the advancement of technology, the cost of hardware required for monitoring continues to decrease. In contrast, labor costs continue to increase. Real engineering needs and economic considerations have led to the idea of replacing the "human eye" of construction workers with cameras, to supervise the construction work of the bulldozer. This thesis uses the method of computer vision to automatically identify the scraping operations of bulldozers, mainly including the following work:

First of all, on the basis of the existing research at home and abroad, this thesis analyzes the focus of the on-site construction workers' supervision of the bulldozer construction operation, combined with the analysis of the characteristics of the bulldozer's scraping operation, and transforms the supervision problem of the bulldozer's construction operation efficiency into the scraping state. Recognition problems with actions. In order to obtain the data samples required for identification, the deployment and installation scheme of on-board video was designed and implemented, and the characteristics of video data from all directions during the bulldozer’s scraping operation were analyzed as a whole, which laid a solid foundation for the subsequent identification of the bulldozer’s scraping status and scraping action. the theoretical basis;

Secondly, after the analysis and demonstration, the video data at the middle camera in the front of the front of the car is selected as the data source for the identification of the scraping state and the shovel movement, of which the identification of the scraping state can be done with the help of image samples, and the identification of the shoveling movement is related to the processing of timing information, and the video samples need to be used as the data set, and the image samples and video samples need to be preprocessed to enhance the robustness of the model and improve the generalizability of the model. For the identification of the scraping state and the shoveling motion, three mainstream deep learning algorithms are selected, and the optimal combination of parameters is obtained by tuning the parameters, of which the algorithm that identifies the relatively optimal effect of the scraping state is YOLOv5, and the algorithm that identifies the relatively optimal effect of the shoveling motion is LRCN. By optimizing the loss function and regression box screening method of the YOLOv5 algorithm, the mAP value of the model was increased by 2.12% and 2.18%, respectively, and the accuracy of the model was improved by 2.55%, 1.82%, and 1.68% by optimizing the loss function, feature extraction network, and time series learning unit of the LRCN algorithm, respectively. Finally, the identification of the scraping status and scraping action of the bulldozer has achieved good results, but the two algorithms are independent of each other, and single identification of scraping status or action cannot meet the actual monitoring needs of the bulldozer’s working efficiency on the earthwork site. It is proposed to couple two independent algorithms to realize synchronous identification of scraping state and scraping action.

The results show that the synchronous identification algorithm of scraping status and action proposed in this thesis can meet the actual needs of automatic identification of bulldozer scraping operations. By providing a visual process supervision method, it can reduce the amount of construction dust and noise on the site. , the construction workers' supervision pressure on earthmoving machinery construction operations, and provide a data basis for the improvement of bulldozer production efficiency.

Key words: bulldozer, Shovel status, Shovel movement, Deep learning, Synchronous recognition