This is a summary of a paper called Machine learning in construction: From shallow to deep learning published in 2021.

Introduction

The construction industry has significant potential to benefit from machine learning technologies, which can enhance site supervision, automate detection processes, and enable intelligent maintenance. Despite lagging behind other industries in performance and productivity, construction can leverage machine learning to improve efficiency and safety. However, challenges exist due to difficulties in acquiring labeled data, especially in the complex environments of construction sites.

Shallow Learning in Construction

Shallow learning algorithms have been primarily used for safety management and other applications in the construction industry. The table below summarizes various algorithms and their applications:

Algorithm	Application
Logistic Regression	Factors influencing tender success
Decision Tree	Cost prediction
Bayesian Network	Construction performance diagnostics
k-Means	Assessing rust defects in bridge painting
SVM	Dynamic safety monitoring during construction
KNN	Knowledge-sharing for severe change order disputes
AdaBoost	Decision support for selecting formwork systems
Mean Shift	Tracking site labor
Random Forest	Soil properties and metallic pipe deterioration
HMM	Probabilistic prediction of tunnel geology
Conditional Random Field	Automatic creation of semantically rich 3D building models
Markov Random Field	Improving laser image resolution for corrosion measurement
Spectral Clustering	Extracting planar patches from noisy point-cloud data

People Detection

An example of shallow learning application is a safety vest detection method that uses color pixels to detect workers on-site. Algorithms like SVM, Artificial Neural Networks (ANN), and Logistic Regression classify pixels in different color spaces. This method serves as a precursor to worker positioning, recognition, and motion analysis.

Deep Learning in Construction

Since 2017, deep learning has been widely adopted in construction for applications such as safety, road surveys, bridge inspections, and on-site operation monitoring. Countries like China are heavily investing in infrastructure development and encouraging advanced technologies in urbanization processes.

Main Categories of Deep Learning Methods

1. Feature Extraction Models: VGG-16, ResNet-50
2. Object Detection Models: Faster R-CNN, SSD, Mask R-CNN, YOLO v2, YOLO v3
3. Classification and Recognition Schemes: Utilizing traditional machine learning algorithms, specific mathematical methods, or deep learning networks based on detection results.

Computer Vision Applications

Computer vision is extensively used on construction sites. Common applications include:

• Detecting improper use of hardhats
• Safety guardrail detection in images
• Real-time detection of personal protective equipment
• Identifying non-certified work on sites
• Activity analysis of construction equipment
• Vehicle detection in dense construction environments
• Recognizing diverse construction activities
• Mitigating falls from height

Common Model Architectures: R-CNNs, Convolutional Neural Networks (CNNs), ResNet, FSSD, VGG-16, YOLO

Object Detection

Automated tools for detecting cracks or defects in infrastructure surfaces ensure effective maintenance and reduce human resource needs. These tools can promptly detect faults, guaranteeing maximum use of facilities.

Image Segmentation

Image segmentation is used to detect defects by segmenting images of infrastructure surfaces. For instance, Deeplab-v3++ has been applied for road pothole extraction, distinguishing potholes from patches using depth information.

Action Recognition

Action recognition analyzes spatial and temporal patterns to identify activities of workers or equipment on-site. Key points are detected using computer vision or sensors, and models like Long Short-Term Memory (LSTM) networks are used for training.

Language Models

Text Classification

Models like BERT and CNNs are used for:

• Classifying accident narrative texts
• Visualizing interdependence between causal variables in accidents
• Automated classification of near-miss reports

Text Mining

Text mining extracts valuable information from project documents like accident reports and logs, aiding project managers in identifying causes and patterns. Challenges include the non-repeatability of accidents and projects, which can affect model performance on new data.

Limitations and Challenges

Lack of Data

Machine learning requires large datasets for training, but acquiring labeled data in construction is challenging due to difficulties in data annotation and collection. While transfer learning can help, the lack of data remains a significant barrier.

Accuracy Risks

Some machine learning algorithms do not achieve sufficient accuracy for practical applications, such as action recognition. Errors can lead to safety threats and cost losses, making it crucial to improve algorithm performance or find alternative solutions.

Complex Environments

Construction sites have complex environments that can affect data quality, hindering the performance of machine learning algorithms.

Future Directions

The construction industry is moving towards increased automation to address labor shortages and rising costs. Machine learning, particularly deep learning, will play a crucial role in this transformation.

Emphasis on Computer Vision

Computer vision applications will continue to grow rapidly due to their maturity and consistent tasks across projects. Safety and automatic detection will become common, with deep learning playing a more significant role and shallow learning used for preprocessing.

Data Acquisition and Public Datasets

Establishing public datasets specific to construction can accelerate research and application of machine learning by allowing researchers to focus more on algorithm development. A construction-related dataset similar to ImageNet could significantly promote progress.

Integration with Construction Knowledge

Future research should aim to fully understand machine learning algorithms and combine them with construction-specific knowledge to develop dedicated deep network models tailored for the industry. The main innovation lies in using machine learning results for further judgment, rather than merely improving the algorithms themselves.