A deep learning fusion approach to retrieve images of People's unsafe behavior from construction sites

Fang Weili, Love Peter E. D., Luo Hanbin, Xu Shuangjie

Abstract

Retrieving unsafe behaviours from an existing digital database can provide managers and the like with the necessary information to put in place strategies to improve safety in construction. Prevailing studies have focused on developing content-based image retrieval (CBIR) approaches (e.g., color-based) to retrieve objects and ma-terials obtained from construction sites. While CBIR approaches are effective in extracting low-level features from digital images they are unable to accurately retrieve unsafe behaviours those from existing databases. To address this limitation, we develop an improved CBIR approach to retrieve unsafe behaviour images more accurately and automatically, which combines features extracted from different models. We utilise a digital database developed by Huazhong University of Science and Technology to validate the feasibility of our proposed approach. Our research demonstrates that the fusion of ResNet-101 and VGG-19 can obtain higher levels of Top-K recall and outperform the one-feature extraction method.

Keywords: Digital image, Image retrieval, Deep learning, Feature fusion, Content-based image retrieval

https://www.sciencedirect.com/science/article/pii/S2666165922000199?via%3Dihub