University of Illinois Urbana-Champaign

Dynamic resource provisioning for data center workloads with data constraints

Li, Shen

Content Files
LI-DISSERTATION-2016.pdf

Permalink

https://hdl.handle.net/2142/90503

Description

Title
Dynamic resource provisioning for data center workloads with data constraints
Author(s)
Li, Shen
Issue Date
2016-04-07
Director of Research (if dissertation) or Advisor (if thesis)
Abdelzaher, Tarek F.
Doctoral Committee Chair(s)
Abdelzaher, Tarek F.
Committee Member(s)
  • Gupta, Indranil
  • Liu, Jie
  • Sha, Lui
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Date of Ingest
2016-07-07T19:53:19Z
Keyword(s)
  • Data Center
  • Dynamic Provisioning
  • Big Data
  • Scheduling
  • Distributed Systems
Abstract
Dynamic resource provisioning, as an important data center software building block, helps to achieve high resource usage efficiency, leading to enormous monetary benefits. Most existing work for data center dynamic provisioning target on stateless servers, where any request can be routed to any server. However, the assumption of stateless behaviors no longer holds for subsystems that subject to data constraints, as a request may depend on a certain dataset stored on a small subset of servers. Routing a request to a server without the required dataset violates data locality or data availability properties, which may negatively impact on the response times. To solve this problem, this thesis provides an unified framework consisting of two main steps: 1) determining the proper amount of resources to serve the workload by analyzing the schedulability utilization bound; 2) avoiding transition penalties during cluster resizing operations by deliberately design data distribution policies. We apply this framework to both storage and computing subsystems, where the former includes distributed file systems, databases, memory caches, and the latter refers to systems such as Hadoop, Spark, and Storm. Proposed solutions are implemented into MemCached, HBase/HDFS, and Spark, and evaluated using various datasets, including Wikipedia, NYC taxi trace, Twitter traces, etc.
Graduation Semester
2016-05
Type of Resource
text
Permalink
http://hdl.handle.net/2142/90503
Copyright and License Information
Copyright 2016 Shen Li

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois