Reliable equipment that automatically and instantly detects activities on building sites helps to lower expenses, improve operational costs, and eliminate pollution. To track resource progress and productivity, a comprehensive perspective and resource allocation of different construction-related tasks are very important. Despite the task’s importance, all of the work must be done by hand. Nonetheless, research studies have made significant progress in examining computer vision techniques. Nevertheless, it is focused on a small range of object kinds and activity classes. Furthermore, the emphasis is on identifying individual activities when an image only contains one or more objects performing that activity. This study square approach uses deep convolution neural networks to identify 22 groups of items in photos taken during various construction-level activities. The chance of things coexisting will be represented by semantic relevance, and the proximity at the positions of the image and patterns of activity will be represented by spatial relevance. It is advantageous to have the ability to automatically recognize structure activity. So, this is a potential method to manage important time when gathering information to identify construction-related activities.