AI-driven project controls: integrated computer vision production tracking & AI-driven forecasting in building construction

Nunez Morales, Juan Diego

Permalink

https://hdl.handle.net/2142/124713 Copy

Description

Title

AI-driven project controls: integrated computer vision production tracking & AI-driven forecasting in building construction

Author(s)

Nunez Morales, Juan Diego

Issue Date

2024-04-29

Director of Research (if dissertation) or Advisor (if thesis)

Golparvar-Fard, Mani

Doctoral Committee Chair(s)

• Golparvar-Fard, Mani
• El-Rayes, Khaled
• El-Gohary, Nora
• Pena-Mora, Feniosky

Department of Study

Civil & Environmental Eng

Discipline

Civil Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Automated Progress Monitoring
• Computer Vision
• Construction Management
• Deep Learning

Abstract

The process of project performance control and progress monitoring is one of the crucial steps for successful project management applications. This process enables the opportunity to measure and analyze ongoing project operations to improve the execution cycle and increase the likelihood of project success. However, as essential as this process is acknowledged to be, current industry practices (1) lack proper data structuring for collecting and organizing partial progress data; (2) rely on manual, subjective, and low-frequent human-based progress information retrieval and interpretation; and (3) are retroactive in nature, leaving a gap for addressing foreseeable issues and productivity deviations. These limitations are exacerbated due to their tasks being often associated with non-value-adding activities, requiring the analysis of complex data structures necessary to ensure the success of a project. To address these limitations, modern computer vision-based solutions have been proposed and are now being practiced at large across the industry. These solutions are often based on processing a large collection of reality capture data (i.e., perspective and 360 photos and videos) into 3D point clouds and integrating them with BIM and schedule. These digital twins (i.e., reality models over time, twinned against 4D BIM) are processed via geometrical and appearance-based computer vision techniques to identify progress deviations. However, scaling these techniques has been hampered by (1) a lack of high Levels-of-Development (LoD) across all model disciplines in BIM, (2) low granularity of schedule activity data, and (3) a formal representation of rules-of-credit for visual progress tracking per construction activity. These solutions have also focused on only conventional project control workflows and, as such, only retroactively report progress as opposed to predicting risk for delay in a way that project teams can proactively address them in their weekly coordination meetings. To fill in these gaps in knowledge, this dissertation introduces a systemic approach and computational frameworks for integrating Computer Vision and AI-driven forecasting to detect, monitor, and predict the progress of under-construction systems in Building Construction and their associated labor productivity. The development of a series of deep-learning and autoregressive methods helps overcome the challenges of current management practices in collecting and processing daily production in construction sites while providing explainable labor productivity forecasts for better decision-making and labor management on daily jobsite operations. Specifically, the proposed approach presents a new classification system, denominated Visual States of Work-in-Progress (ViS-WP), as an extension to the ASTM Uniformat II Classification for Building Elements to determine partial states of progress of most construction elements. This system is used to develop a Transformer-based deep learning computational framework for detecting production values through semantic segmentation of 360-degree images. As an extension to computer vision-based solutions, the developed framework combines region boundary image segmentation from Large Vision Models (LVM) to provide precise object shape boundary, depth, and surface normal maps inference to aggregate pixel class probabilities based on their 3D geometry features and a custom-trained encoder-decoder model for creating class-based detection using a hierarchy-based feature inference that aggregates fine-grained class features to their coarse class equivalent. Lastly, a Multiview inference algorithm creates a probabilistic consensus of the correct coarse and fine-grained pixel class based on the shared pixel features between overlapping camera-to-3D point correspondences. Large data requirements of Transformer-based models are addressed by developing a procedural data generation algorithm to automatically collect ground-truth data of indoor construction environments based on high Level of Detail (LOD) BIMs. As a result, a large-scale synthetic dataset of construction site indoor environments denominated the Synthy Construction Dataset, is created. Upon the datapoint frequency increase of labor-productivity, a deep learning-based time series forecasting model is developed to detect trend features based on Autoregression, classifying their differentials to an associated root cause, producing explainable forecasts for proactive progress monitoring. Several studies have been carried out across commonly observable element classes in building construction to understand the validity, accuracy, and precision of the proposed framework. The obtained results are compared against other traditional detection and forecasting benchmarks, showing significant improvements in scalability across the board. Limitations and future work related to both underlying techniques, as well as the use of the system as a whole across different stages of project planning and controls, are discussed in detail.

Graduation Semester

2024-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2024 Juan Diego Nunez-Morales

Owning Collections