University of Illinois Urbana-Champaign

Learning and decentralized control in linear switched systems

Jansch Porto, Joao Paulo

Content Files
JANSCHPORTO-DISSERTATION-2021.pdf

Permalink

https://hdl.handle.net/2142/110628

Description

Title
Learning and decentralized control in linear switched systems
Author(s)
Jansch Porto, Joao Paulo
Issue Date
2021-01-28
Director of Research (if dissertation) or Advisor (if thesis)
Dullerud, Geir E
Doctoral Committee Chair(s)
Dullerud, Geir E
Committee Member(s)
  • Hu, Bin
  • Mitra, Sayan
  • Salapaka, Srinivasa
  • West, Matthew
Department of Study
Mechanical Sci & Engineering
Discipline
Mechanical Engineering
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Date of Ingest
2021-09-17T02:34:13Z
Keyword(s)
  • Nested systems
  • switched control
  • decentralized control
  • MJLS
  • Switched Systems
  • Reinforcement Learning
  • Policy Gradient
  • Optimal Control
  • Optimization
  • Robust Control
Abstract
Switched systems are an important and widely studied area of control theory since their applications can be encountered in many different circumstances. Examples of such systems include robotics, power grids, networked systems, and macroeconomic models. In this dissertation, we focus on the controller synthesis problem for interconnected systems and the application to physical systems. While there are many existing control theory tools to optimize the performance of systems with a centralized structure, we are interested in developing tools that take into consideration the topology of interconnected systems to generate decentralized controllers. In addition, most of the existing literature results consider the controller synthesis problem when the user has full knowledge of the system. With this in mind, we also look into machine learning tools as a way to simplify the synthesis problem. In the first part of the thesis, we develop synthesis methods for decentralized control of switched systems with mode-dependent (more generally, path-dependent) performance specifications. This specification flexibility is important when achievable system performance varies greatly between modes, as a mode-independent specification will lead to designs that do not extract all the system performance available in each mode. More specifically, under these specifications, we derive exact conditions for existence of block lower triangular path-dependent controllers with $\ell_2$-induced norm performance. The synthesis conditions are given in the form of a semidefinite program (SDP) for both uniform and path-dependent performance bounds. Since the given synthesis conditions might become computationally intractable for large-scale systems, we also introduce a basis-based approach that allows computational complexity to be more carefully controlled. We also applied the resulting design methods to a group of miniature quadcopters, illustrating different features of the decentralized control approach, along with some practical engineering considerations. In the second part of this dissertation, we turn to policy gradient approaches with the goal of starting from some sub-optimal controller and using measurement data to optimize the control gains. Recently, policy optimization for control purposes has received renewed attention due to the increasing interest in reinforcement learning. More specifically, there have been studies on the convergence properties of policy gradient methods to learn the optimal quadratic control of linear time-invariant systems. With our overall goal of controlling switched systems, we take the first step of investigating the global convergence of gradient-based policy optimization methods for quadratic optimal control of discrete-time Markovian jump linear systems (MJLS), which are a special case of switched systems. Despite the non-convexity of the resultant problem, we are still able to identify several useful properties such as coercivity, gradient dominance, and almost smoothness. Based on these properties, we show global convergence guarantees of three types of policy optimization methods: the gradient descent method; the Gauss-Newton method; and the natural policy gradient method. We prove that all three methods converge to the optimal state feedback controller for MJLS at a linear rate if initialized at a controller which is mean-square stabilizing. Lastly, we study model-free (data-driven) approaches to the MJLS quadratic control problem. Our simulation results suggest that the data-driven versions of the three studied iterative methods can efficiently learn the optimal controller for MJLS with unknown dynamics. This work brings new insights for understanding the performance of policy gradient methods on the Markovian jump linear quadratic control problem.
Graduation Semester
2021-05
Type of Resource
Thesis
Permalink
http://hdl.handle.net/2142/110628
Copyright and License Information
Copyright 2021 Joao Paulo Jansch Porto

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Mechanical Science and Engineering