Dynamic legged locomotion through trajectory optimization and reinforcement learning

Li, Chuanzheng

Permalink

https://hdl.handle.net/2142/117799 Copy

Description

Title

Dynamic legged locomotion through trajectory optimization and reinforcement learning

Author(s)

Li, Chuanzheng

Issue Date

2022-11-30

Director of Research (if dissertation) or Advisor (if thesis)

West, Matthew

Doctoral Committee Chair(s)

West, Matthew

Committee Member(s)

• Park, Hae-Won
• Bretl, Timothy
• Ramos, Joao
• Righetti, Ludovic

Department of Study

Mechanical Sci & Engineering

Discipline

Mechanical Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Robotics
• Trajectory Optimization
• Reinforcement Learning

Abstract

Agile quadrupeds such as cats and squirrels are capable of planning and per- forming highly dynamic maneuvers that fully utilize their physical capabilities and respect their inherent dynamics. The major challenges in endowing legged robots with such abilities arise from the requirement to generate feasible motions for such high-degree-of-motion systems under tight time constraints, and to reliably and reactively handle the constant making and breaking of contacts with their environment. This dissertation presents a three-part effort in addressing these challenges. First, I introduce the mechatronics design of multiple customized legged hardware platforms for validating the theoretical work that follows. Second, I propose a trajectory optimization framework for planning dynamic locomotion based on a robot’s centroidal momentum (CM), which is the sum of all the links’ momenta at its center of mass (CoM). By parameterizing the ground reaction force (GRF) and swing leg trajectories with Bezier polynomials, this framework can utilize the simple CM dynamics to produce feasible GRF and joint trajectories simultaneously under kinematic and dynamic constraints, while suffering less error caused by numerical integration. Third, I present the application of reinforcement learning (RL) in producing a jumping controller that achieves zero-shot sim-to-real transfer. At its core is a high-fidelity simulation environment enabled by identification of critical modeling details including contact compliance and an improved motor saturation model. Combined with a hierarchical control structure and a two-phase learning curriculum, this RL framework can generate controllers that consistently break the previous height record and produce experiment results that closely match those from simulation. Experimental validation of each controller design is performed on their corresponding customized hardware, proving their effectiveness in realizing dynamic motions on legged robots.

Graduation Semester

2022-12

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Chuanzheng Li

Owning Collections