A deep learning-based approach for mitigating falls from height with computer vision: Convolutional neural network

Abstract

Structural supports (e.g., concrete and steel) provide engineering structures with stability by transferring loads. During the construction of an engineering structure, individuals are often prone to taking short take-cuts by traversing supports to perform their daily activities and save time. Thus, the likelihood of an individual being subjected to an injury or even killing themselves significantly increases when performing such unsafe behavior. To address this problem, we have developed an automatic computer-vision approach that utilizes a Mask Region Based Convolutional Neural Network (R-CNN) to detect individuals traversing structural supports during the construction of a project. The algorithms developed are used to: (1) automatically identify the presence of people; and (2) recognize the relationship between people and concrete/steel supports to determine their presence of them. To validate our approach, we created an extensive database of photographs of people who had traversed structural supports from a number of different constructions project to train and test the developed Mask R-CNN. The recall and precision rates for overlapping detection were found to be 90% and 75%. The results demonstrate that the developed Mask R-CNN can accurately detect people that traverse concrete/steel supports during construction. We suggest that proposed computer-vision approach that we have developed can be used by site management to automatically identify unsafe behavior and provide feedback to individuals about their likelihood of falls from heights. By recognizing unsafe behavior in real-time, appropriate actions (e.g. education) can be instantly put in place to prevent their re-occurrence.