Vision-based method of automatically detecting construction video highlights by integrating machine tracking and CNN feature extraction

Automatic analysis of construction video footage is beneficial for project management tasks such as productivity analysis and safety control. However, construction videos are usually long in duration and only contain limited useful information to engineers, while the storage of video data from construction projects is challenging. To obtain and store useful video footage systematically and concisely, this research proposes a vision-based method to automatically generate video highlights from construction videos. The proposed approach is validated through two case studies: a gate scenario and an earthmoving scenario. In experiments, the proposed method has achieved 89.2% on precision and 93.3% on recall, which outperforms the feature-based method by 12.7% and 17.2% on precision and recall, respectively. Meanwhile, the proposed method reduces the required digital storage space by 94.6%. The proposed approach offers potential benefits to construction management in terms of significantly reducing video storage space and efficiently indexing construction video footage. © 2021 Elsevier B.V.