Autonomous vision-based inspection of RC railway bridges for rapid post-earthquake response and recovery

Narazaki, Yasutaka

Permalink

https://hdl.handle.net/2142/109502 Copy

Description

Title

Autonomous vision-based inspection of RC railway bridges for rapid post-earthquake response and recovery

Author(s)

Narazaki, Yasutaka

Issue Date

2020-11-25

Director of Research (if dissertation) or Advisor (if thesis)

Spencer Jr., Billie F

Doctoral Committee Chair(s)

Spencer Jr., Billie F

Committee Member(s)

• Dyke, Shirley J
• Golparvar-Fard, Mani
• Chowdhary, Girish

Department of Study

Civil & Environmental Eng

Discipline

Civil Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Post-earthquake inspection
• Autonomous inspection
• Railway bridges
• Reinforced concrete
• Structural component recognition
• Unmanned aerial vehicles
• Autonomous navigation
• Semantic segmentation
• Monocular depth estimation
• Structure from motion
• Fully convolutional networks
• Synthetic environment
• Machine learning
• Deep learning
• Artificial intelligence
• Computer vision
• Robotic navigation

Abstract

This research proposes a framework for autonomous vision-based inspection of reinforced concrete (RC) railway bridges to realize rapid post-earthquake response and recovery, which are crucial for minimizing the disruptions to people’s lives and business. Currently, after an earthquake, coarse preliminary structural assessment is performed based on the data immediately available (e.g. ground acceleration data). Then, a human-based visual inspection is performed to obtain detailed information about the structures with suspected damage. This procedure is often time-consuming and laborious because the preliminary assessment tends to result in conservative decisions, leaving a significant workload for the subsequent detailed inspection. For example, after the 2011 Tohoku Earthquake, the challenge of the post-earthquake inspection lead to 16-18 hours of operational shutdown of railways in the Tokyo metropolitan area, Japan, causing more than 5 million people who were unable to return home. The framework proposed in this research seeks to minimize such negative impact on people’s lives through improved structural assessment during the initial response phase, allowing rapid resumption of the operation of critical transportation infrastructure if the affected bridges are not damaged. Realizing the proposed framework requires completion of four main research tasks: (i) development of a synthetic environment for bridge structures, (ii) recognition and localization of critical structural components, (iii) autonomous vision-based unmanned aerial vehicle (UAV) navigation for image collection, and (iv) structural assessment by processing the collected images. To investigate and validate the proposed framework, appropriate testing environments that represent the target post-earthquake structural inspection scenarios are crucial. The first task, development of the synthetic environment, produces photo-realistic computer graphics models of railway bridges that can be used to generate training data for the machine learning algorithm applications and to distill and validate various algorithms. The second task, bridge component recognition, and localization are realized by fully convolutional network (FCN)-based semantic segmentation and depth prediction algorithms. The investigation begins with the bridge component recognition using a single real-world image. Based on the lessons learned from the initial work, the recognition and localization results are improved by incorporating recognition results from previous frames, as well as large-scale synthetic training data with accurate ground truth annotations. The bridge component recognition and localization results are combined with the sparse point cloud data in the third research task, autonomous vision-based UAV navigation, to plan navigation paths for the rapid structural inspection. This task is realized by developing an on-line image data processing strategy that builds three-dimensional (3D) maps of the environment, where target structural components (columns) are represented by rectangular prismatic shapes. Based on the 3D column detection results, waypoints for performing rapid structural inspection are determined. The fourth task, structural assessment using the collected images, is based on the FCN for semantic segmentation. Compared to the semantic segmentation for structural components, semantic segmentation for structural damage is challenging, because the damage often presents subtle textures, making accurate recognition difficult when the distance to the target surface is large. To address the problem, this research introduces the regions of reliable damage recognition (RRDRs), which are estimated using the depth prediction results. The output of the structural assessment unit is parsed images that contain the information of structural damage, structural components where the damage is identified, and the RRDR. The final product of this research is a prototype autonomous system for vision-based rapid post-earthquake bridge inspection. The developed system is a significant step toward the automation of the entire bridge inspection procedure in the field environment that minimizes the disruptions to people’s lives and business.

Graduation Semester

2020-12

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/109502

Copyright and License Information

Copyright 2020 Yasutaka Narazaki

Owning Collections