Robust model-based reinforcement learning using L1 adaptive control

Karumanchi, Sambhu Harimanas

Permalink

https://hdl.handle.net/2142/121362 Copy

Description

Title

Robust model-based reinforcement learning using L1 adaptive control

Author(s)

Karumanchi, Sambhu Harimanas

Issue Date

2023-07-17

Director of Research (if dissertation) or Advisor (if thesis)

Hovakimyan, Naira

Department of Study

Aerospace Engineering

Discipline

Aerospace Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Robust Adaptive Control
• Reinforcement Learning

Abstract

We introduce L1-MBRL, a control-theoretic augmentation scheme for Model-Based Reinforcement Learning (MBRL) algorithms. Unlike model-free approaches, MBRL algorithms learn a model of the transition function using data and use it to design a control input. Our approach approximates the transition function along each trajectory with a control-affine model to generate a control input augmentation that perturbs the input produced by the underlying MBRL policy. The perturbation produced by the L1 adaptive control is designed to enhance the robustness of the system against uncertainties. Importantly, the proposed L1 adaptive control-based learning scheme is agnostic to the choice of MBRL algorithm and can be integrated seamlessly with many model-based RL systems in practice. The method exhibits superior performance and sample efficiency on multiple MuJoCo environments, both with and without system noise, as demonstrated by our numerical simulations.

Graduation Semester

2023-08

Type of Resource

Thesis

Handle URL

https://hdl.handle.net/2142/121362

Copyright and License Information

Copyright 2023 Sambhu Harimanas Karumanchi

Owning Collections