University of Illinois Urbana-Champaign

Improving cache replacement policy using deep reinforcement learning

Benson, Christopher Edward

Content Files
BENSON-THESIS-2018.pdf

Permalink

https://hdl.handle.net/2142/102858

Description

Title
Improving cache replacement policy using deep reinforcement learning
Author(s)
Benson, Christopher Edward
Issue Date
2018-12-13
Director of Research (if dissertation) or Advisor (if thesis)
Peng, Jian
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Date of Ingest
2019-02-07T20:44:29Z
Keyword(s)
  • Reinforcement Learning
  • Machine Learning
  • Deep Learning
Abstract
This thesis explores the use of reinforcement learning approaches to improve replacement policies of caches. In today's internet, caches play a vital role in improving performance of data transfers and load speeds. From video streaming to information retrieval from databases, caches allow applications to function more quickly and efficiently. A cache's replacement policy plays a major role in determining the cache's effectiveness and performance. The replacement policy is an algorithm that chooses which piece of data in the cache should be evicted when the cache becomes full and new elements are requested. In computer systems today, most caches use simple heuristic-based policies. Currently used policies are effective but are still far from optimal. Using more optimal cache replacement policies could dramatically improve internet performance and reduce database costs for many industry-based companies. This research examines learning more optimal replacement policies using reinforcement learning. In reinforcement learning, an agent learns to take optimal actions given information about an environment and a reward signal. In this work, deep reinforcement learning algorithms are trained to learn optimal cache replacement policies using a simulated cache environment and database access traces. This research presents the idea of using index-based cache access histories as input data for the reinforcement learning algorithms instead of content-based input. Several approaches are explored including value-based algorithms and policy gradient algorithms. The work presented here also explores the idea of using imitation learning algorithms to mimic optimal cache replacement policies. The algorithms are tested on several different cache sizes and data access patterns to show that these learned policies can outperform currently used replacement policies in a variety of settings.
Graduation Semester
2018-12
Type of Resource
text
Permalink
http://hdl.handle.net/2142/102858
Copyright and License Information
Copyright 2018 Christopher Benson

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Computer Science
Dissertations and Theses from the Siebel School of Computer Science