Robust imitation learning from observation

Tang, Zhenyi

Permalink

https://hdl.handle.net/2142/108132 Copy

Description

Title

Robust imitation learning from observation

Author(s)

Tang, Zhenyi

Issue Date

2020-05-11

Director of Research (if dissertation) or Advisor (if thesis)

Driggs-Campbell, Katherine

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

Imitation learning, imitation learning from observation, robustness

Abstract

Imitation learning, sometimes referred as learning from demonstrations, has been used in real world scenarios because of its sample efficiency and computational feasibility, such as autonomous driving and robotics control. However, imitation learning often suffers from compounding error and data mismatch, which leads to lack of robustness. Another drawback is that in traditional imitation learning, people usually assume that data for both states and actions is accessible. In reality, data about the action experts took may be more difficult to access than the data about state transitions. For example, a driving video clip shows each states (traffic signal, road condition, map navigation, etc.) the vehicle is in, but does not contain associated information about whether the driver steers left or right in this transition. To address these two issues, we propose an algorithm called Robust Imitation Learning from Observation (RILfO), that aims to provide robustness in an imitation learning from observation setting. First, we allow the agent to learn a policy given state-only demonstrations from experts. Second, we introduce an adversarial agent that aims to optimally destabilize the system by carefully engineering its loss function. We jointly train the agent and adversary so that the adversary is reinforced, and the agent explores more possibilities, thus becomes more robust to the various adversarial conditions. We experimentally test RILfO in multiple benchmark environments, compare RILfO with some baseline methods, demonstrate its robustness. We also discuss about its limitations and opportunities for future work.

Graduation Semester

2020-05

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/108132

Copyright and License Information

Copyright 2020 Zhenyi Tang

Owning Collections