University of Illinois Urbana-Champaign

Robust imitation learning from observation

Tang, Zhenyi

Content Files
TANG-THESIS-2020.pdf

Permalink

https://hdl.handle.net/2142/108132

Description

Title
Robust imitation learning from observation
Author(s)
Tang, Zhenyi
Issue Date
2020-05-11
Director of Research (if dissertation) or Advisor (if thesis)
Driggs-Campbell, Katherine
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Date of Ingest
2020-08-26T23:57:19Z
Keyword(s)
Imitation learning, imitation learning from observation, robustness
Abstract
Imitation learning, sometimes referred as learning from demonstrations, has been used in real world scenarios because of its sample efficiency and computational feasibility, such as autonomous driving and robotics control. However, imitation learning often suffers from compounding error and data mismatch, which leads to lack of robustness. Another drawback is that in traditional imitation learning, people usually assume that data for both states and actions is accessible. In reality, data about the action experts took may be more difficult to access than the data about state transitions. For example, a driving video clip shows each states (traffic signal, road condition, map navigation, etc.) the vehicle is in, but does not contain associated information about whether the driver steers left or right in this transition. To address these two issues, we propose an algorithm called Robust Imitation Learning from Observation (RILfO), that aims to provide robustness in an imitation learning from observation setting. First, we allow the agent to learn a policy given state-only demonstrations from experts. Second, we introduce an adversarial agent that aims to optimally destabilize the system by carefully engineering its loss function. We jointly train the agent and adversary so that the adversary is reinforced, and the agent explores more possibilities, thus becomes more robust to the various adversarial conditions. We experimentally test RILfO in multiple benchmark environments, compare RILfO with some baseline methods, demonstrate its robustness. We also discuss about its limitations and opportunities for future work.
Graduation Semester
2020-05
Type of Resource
Thesis
Permalink
http://hdl.handle.net/2142/108132
Copyright and License Information
Copyright 2020 Zhenyi Tang

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Computer Science
Dissertations and Theses from the Siebel School of Computer Science