Learning and decentralized control in linear switched systems

Jansch Porto, Joao Paulo

Permalink

https://hdl.handle.net/2142/110628 Copy

Description

Title

Learning and decentralized control in linear switched systems

Author(s)

Jansch Porto, Joao Paulo

Issue Date

2021-01-28

Director of Research (if dissertation) or Advisor (if thesis)

Dullerud, Geir E

Doctoral Committee Chair(s)

Dullerud, Geir E

Committee Member(s)

• Hu, Bin
• Mitra, Sayan
• Salapaka, Srinivasa
• West, Matthew

Department of Study

Mechanical Sci & Engineering

Discipline

Mechanical Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Nested systems
• switched control
• decentralized control
• MJLS
• Switched Systems
• Reinforcement Learning
• Policy Gradient
• Optimal Control
• Optimization
• Robust Control

Abstract

Switched systems are an important and widely studied area of control theory since their applications can be encountered in many different circumstances. Examples of such systems include robotics, power grids, networked systems, and macroeconomic models. In this dissertation, we focus on the controller synthesis problem for interconnected systems and the application to physical systems. While there are many existing control theory tools to optimize the performance of systems with a centralized structure, we are interested in developing tools that take into consideration the topology of interconnected systems to generate decentralized controllers. In addition, most of the existing literature results consider the controller synthesis problem when the user has full knowledge of the system. With this in mind, we also look into machine learning tools as a way to simplify the synthesis problem. In the first part of the thesis, we develop synthesis methods for decentralized control of switched systems with mode-dependent (more generally, path-dependent) performance specifications. This specification flexibility is important when achievable system performance varies greatly between modes, as a mode-independent specification will lead to designs that do not extract all the system performance available in each mode. More specifically, under these specifications, we derive exact conditions for existence of block lower triangular path-dependent controllers with $\ell_2$-induced norm performance. The synthesis conditions are given in the form of a semidefinite program (SDP) for both uniform and path-dependent performance bounds. Since the given synthesis conditions might become computationally intractable for large-scale systems, we also introduce a basis-based approach that allows computational complexity to be more carefully controlled. We also applied the resulting design methods to a group of miniature quadcopters, illustrating different features of the decentralized control approach, along with some practical engineering considerations. In the second part of this dissertation, we turn to policy gradient approaches with the goal of starting from some sub-optimal controller and using measurement data to optimize the control gains. Recently, policy optimization for control purposes has received renewed attention due to the increasing interest in reinforcement learning. More specifically, there have been studies on the convergence properties of policy gradient methods to learn the optimal quadratic control of linear time-invariant systems. With our overall goal of controlling switched systems, we take the first step of investigating the global convergence of gradient-based policy optimization methods for quadratic optimal control of discrete-time Markovian jump linear systems (MJLS), which are a special case of switched systems. Despite the non-convexity of the resultant problem, we are still able to identify several useful properties such as coercivity, gradient dominance, and almost smoothness. Based on these properties, we show global convergence guarantees of three types of policy optimization methods: the gradient descent method; the Gauss-Newton method; and the natural policy gradient method. We prove that all three methods converge to the optimal state feedback controller for MJLS at a linear rate if initialized at a controller which is mean-square stabilizing. Lastly, we study model-free (data-driven) approaches to the MJLS quadratic control problem. Our simulation results suggest that the data-driven versions of the three studied iterative methods can efficiently learn the optimal controller for MJLS with unknown dynamics. This work brings new insights for understanding the performance of policy gradient methods on the Markovian jump linear quadratic control problem.

Graduation Semester

2021-05

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/110628

Copyright and License Information

Copyright 2021 Joao Paulo Jansch Porto

Owning Collections