Improving cache replacement policy using deep reinforcement learning

Benson, Christopher Edward

Permalink

https://hdl.handle.net/2142/102858 Copy

Description

Title

Improving cache replacement policy using deep reinforcement learning

Author(s)

Benson, Christopher Edward

Issue Date

2018-12-13

Director of Research (if dissertation) or Advisor (if thesis)

Peng, Jian

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Reinforcement Learning
• Machine Learning
• Deep Learning

Abstract

This thesis explores the use of reinforcement learning approaches to improve replacement policies of caches. In today's internet, caches play a vital role in improving performance of data transfers and load speeds. From video streaming to information retrieval from databases, caches allow applications to function more quickly and efficiently. A cache's replacement policy plays a major role in determining the cache's effectiveness and performance. The replacement policy is an algorithm that chooses which piece of data in the cache should be evicted when the cache becomes full and new elements are requested. In computer systems today, most caches use simple heuristic-based policies. Currently used policies are effective but are still far from optimal. Using more optimal cache replacement policies could dramatically improve internet performance and reduce database costs for many industry-based companies. This research examines learning more optimal replacement policies using reinforcement learning. In reinforcement learning, an agent learns to take optimal actions given information about an environment and a reward signal. In this work, deep reinforcement learning algorithms are trained to learn optimal cache replacement policies using a simulated cache environment and database access traces. This research presents the idea of using index-based cache access histories as input data for the reinforcement learning algorithms instead of content-based input. Several approaches are explored including value-based algorithms and policy gradient algorithms. The work presented here also explores the idea of using imitation learning algorithms to mimic optimal cache replacement policies. The algorithms are tested on several different cache sizes and data access patterns to show that these learned policies can outperform currently used replacement policies in a variety of settings.

Graduation Semester

2018-12

Type of Resource

text

Permalink

http://hdl.handle.net/2142/102858

Copyright and License Information

Copyright 2018 Christopher Benson

Owning Collections