Adaptive intelligence: Integrating recurrence, deep reinforcement learning, and neuro-adaptive techniques for dynamic unified guidance & control of adaptive systems

Bout, Scott

Permalink

https://hdl.handle.net/2142/122073 Copy

Description

Title

Adaptive intelligence: Integrating recurrence, deep reinforcement learning, and neuro-adaptive techniques for dynamic unified guidance & control of adaptive systems

Author(s)

Bout, Scott

Issue Date

2023-12-08

Director of Research (if dissertation) or Advisor (if thesis)

Chowdhary, Girish

Committee Member(s)

• Williams, Kyle
• Schlossman, Rachel

Department of Study

Aerospace Engineering

Discipline

Aerospace Engineering

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Deep Learning
• Deep Reinforcement Learning
• Neuro-Adaptive Control
• Recurrent Neural Networks
• Deep Neural Networks
• Machine Learning
• Guidance
• Control
• Adaptive Control

Abstract

In modern control systems, the search for algorithms that provide adaptive, unified guidance and control (G&C) capabilities remains a significant challenge. Traditional control mechanisms, such as LQR or PID, often exhibit limitations in performance, especially when compared to cutting edge reinforcement learning algorithms in simulations. This thesis delves deep into the potential of Deep Reinforcement Learning, specifically the Proximal Policy Optimization (PPO) algorithm, aiming to provide a unified G&C solution that surpasses conventional methods in both adaptability and performance. Our research embarked on an ambitious goal: to engineer a guidance and control algorithm with inherent adaptive capabilities. Central to this endeavor was the integration of a Gated Recurrent Unit (GRU) system identification network with PPO. This synthesis allowed us to concatenate the output from the GRU system identification with the state, subsequently feeding the enriched information into both actor and critic networks. As applied to the Lunar Lander Continuous environment, this novel setup enabled optimal flight performance even in the face of unexpected thruster failures. Comparative analysis revealed that the PPO integrated with GRU system identification (sysID) notably enhanced the performance, with the critic network loss converging to zero - a feat not achieved with PPO alone. Moreover, PPO with the addition of GRU system identification demonstrated a markedly higher success rate in lunar landings when juxtaposed against its PPO-only counterpart. Further advancements in adaptive control were explored through the Recurrent Deep Model Reference Adaptive Control (R-DMRAC) algorithm. Our results in the simulated hexcopter environment illuminated that R-DMRAC, with its recurrent neural network foundation, consistently outperformed both DMRAC and traditional MRAC, especially in scenarios characterized by highly nonlinear, state, and time dependent disturbances. The broader implications of our study underscore the transformative potential of incorporating recurrent neural networks into adaptive algorithms. Whether directly deployed as actor and critic networks or used in tandem with other architectures, as demonstrated in our PPO with the addition of GRU system identification and R-DMRAC experiments, recurrent elements markedly boost performance in the adaptive domain. This revelation paves the way for future research and real-world applications where adaptiveness and resilience to dynamic disturbances are paramount.

Graduation Semester

2023-12

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Scott Bout

Owning Collections