H2 LINEARCONTROLWITHH∞ ROBUSTNESSGUARANTEE: AGAME-THEORETICAPPROACH

Zhang, Xiangyuan

Permalink

https://hdl.handle.net/2142/125045 Copy

Description

Title

H2 LINEARCONTROLWITHH∞ ROBUSTNESSGUARANTEE: AGAME-THEORETICAPPROACH

Author(s)

Zhang, Xiangyuan

Issue Date

2020-05-01

Keyword(s)

reinforcement learning (RL), Nash equilibrium, inner loop problem(NE),

Abstract

In recent years, reinforcement learning (RL) has shown promising developments in solving sequential decision-making problems as well as handling continuous control tasks. Among the success stories, many are related to policy optimization (PO) algorithms, developed in the context of constrained optimization. To address the stability and robustness of the controller as the algorithm iterates, constraints such as the H∞-norm one need to be enforced on-the-fly. Recently, Zhang et al. (2019) showed the implicit regularization and the global convergence property of PO methods for the mixed H2/H∞ design problem, a classic problem in the robust control literature. Despite the non-convex, noncoercive optimization landscape of the problem, iterates of PO methods are guaranteed to preserve the H∞-norm constraint without explicit encoding, while converging to the global optimizer. In this thesis, we demonstrate that the solution of the mixed H2/H∞ design problem can also be obtained through solving the Nash equilibrium (NE) of a sequential zerosumlinear-quadratic (LQ) game via double-loop PO methods. Specifically, we first show that the natural policy gradient algorithm can be applied to solve the inner loop problem with a fixed outer loop control policy. Then, we establish the desired stability and global convergence properties despite the non-coercive nature of the inner loop cost function. Subsequently, the outer loop problem can also be solved using the natural policy gradient algorithm, similar to the techniques presented in Zhang et al. (2019). The connection between the mixed H2/H∞ design problem and the zero-sum LQ game provides a path to investigate model-free PO methods for the mixed H2/H∞ design problem.

Type of Resource

text

Language

eng

Owning Collections