Introspective learning based Visual-LiDAR fusion for adaptive Simultaneous Localization and Mapping

Kedia, Shubham

Permalink

https://hdl.handle.net/2142/121406 Copy

Description

Title

Introspective learning based Visual-LiDAR fusion for adaptive Simultaneous Localization and Mapping

Author(s)

Kedia, Shubham

Issue Date

2023-06-08

Director of Research (if dissertation) or Advisor (if thesis)

Hauser, Kris

Department of Study

Mechanical Sci & Engineering

Discipline

Mechanical Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• SLAM
• Computer Vision
• Sensor Fusion
• State Estimation
• Deep Learning
• Robotics
• Optimization
• Adaptive SLAM

Abstract

This work is about developing an adaptive Visual-LiDAR Simultaneous Localization and Mapping (SLAM) algorithm. The objective is to develop a SLAM system that can adaptively negotiate LiDAR degenerate scenarios and visually challenging environments using sensor fusion. The fusion is based on the Pose Graph Optimization (PGO) technique, utilizing adaptive fusion weights predicted from a Deep Neural Network (DNN) model. The DNN model framework is inspired by introspective learning for vision systems. The DNN model is trained on a large dataset called TartanAir, which has diverse and challenging environmental conditions. The output of the model is the predicted error on the visual odometry and LiDAR odometry, which is used to compose the information matrix of the PGO. The PGO framework with weighted pose constraints from visual odometry, LiDAR odometry, and loop closure is solved using the Levenberg–Marquardt optimization algorithm. The proposed framework shows superior performance compared to the visual-only, LiDAR-only SLAM, and baseline fusion methods that were evaluated in this study.

Graduation Semester

2023-08

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Shubham Kedia

Owning Collections