Autonomous UAV positioning using multi-agent reinforcement learning with decentralized swarms

Chen, Yifan

Permalink

https://hdl.handle.net/2142/120474 Copy

Description

Title

Autonomous UAV positioning using multi-agent reinforcement learning with decentralized swarms

Author(s)

Chen, Yifan

Issue Date

2023-01-19

Director of Research (if dissertation) or Advisor (if thesis)

Caesar, Matthew

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• UAV
• reinforcement learning

Abstract

The rapid development of unmanned aerial vehicles (UAVs) has aroused public attentions, as the versatility of UAVs allows it to be easily adapted for various activities. In recent years, we start seeing more and more UAVs being applied for research and commercial use. In the meantime, the development of Reinforcement Learning has facilitated more intelligent drone behaviors, such as self-flying drone control and autonomous drone racing. With the recent development in multi-agent reinforcement learning and wireless communi- cations, it is more interesting to explore what UAV swarms can do more than a single UAV itself. In this work, we first apply deep reinforcement learning algorithms in multi-agent domains for decentralized UAV swarms, in which each UAV acts as an independent agent having full control of its own actions. We model the environment using Graph Neural Network and attention-based embedding. This proposed method forms a fixed-size encoding for environments with different number of variables including drones and landmarks, which allows it to scale to different number of UAVs, and makes it feasible for efficient transfer learning. We then corroborate the effectiveness of this method in various different tasks through experiments. UAV interactions in a swarm can be categorized into cooperation and competition, and thus environments are ground as fully-cooperative, fully-competitive, and mixed environments. We conduct experiments on two tasks for drone swarms, with the first one being a fully-cooperative environment, and the second a mixed cooperative-competitive environment. The first environment is called DroneConnect, where UAVs are being used as relays to connect mobile devices in remote areas to set up a temporary communication networks among them. We design and implement a drone-relay system that allows drone swarm to cooperatively maximize coverage for these mobile devices. We compare the coverage ratio of centralized reinforcement learning, decentralized multi-agent reinforcement learning method to an alternative optimization method. We then introduce a team of adversaries into the second use case, namly DroneCombat, where two drone swarms defense and oppose each other autonomously. We investigate the performance of the graph-based method and observe the natural emergence of complex behaviors. We also simulate a more real-world scenario where agents are partial-observable of their local neighborhood instead of being omniscient of the entire environment. With extensive experiments, we show that this vision and sensing limitation can be mitigated by message passing through communication.

Graduation Semester

2023-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Yifan Chen

Owning Collections