Dynamic resource provisioning for data center workloads with data constraints

Li, Shen

Permalink

https://hdl.handle.net/2142/90503 Copy

Description

Title

Dynamic resource provisioning for data center workloads with data constraints

Author(s)

Li, Shen

Issue Date

2016-04-07

Director of Research (if dissertation) or Advisor (if thesis)

Abdelzaher, Tarek F.

Doctoral Committee Chair(s)

Abdelzaher, Tarek F.

Committee Member(s)

• Gupta, Indranil
• Liu, Jie
• Sha, Lui

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Data Center
• Dynamic Provisioning
• Big Data
• Scheduling
• Distributed Systems

Abstract

Dynamic resource provisioning, as an important data center software building block, helps to achieve high resource usage efficiency, leading to enormous monetary benefits. Most existing work for data center dynamic provisioning target on stateless servers, where any request can be routed to any server. However, the assumption of stateless behaviors no longer holds for subsystems that subject to data constraints, as a request may depend on a certain dataset stored on a small subset of servers. Routing a request to a server without the required dataset violates data locality or data availability properties, which may negatively impact on the response times. To solve this problem, this thesis provides an unified framework consisting of two main steps: 1) determining the proper amount of resources to serve the workload by analyzing the schedulability utilization bound; 2) avoiding transition penalties during cluster resizing operations by deliberately design data distribution policies. We apply this framework to both storage and computing subsystems, where the former includes distributed file systems, databases, memory caches, and the latter refers to systems such as Hadoop, Spark, and Storm. Proposed solutions are implemented into MemCached, HBase/HDFS, and Spark, and evaluated using various datasets, including Wikipedia, NYC taxi trace, Twitter traces, etc.

Graduation Semester

2016-05

Type of Resource

text

Permalink

http://hdl.handle.net/2142/90503

Copyright and License Information

Copyright 2016 Shen Li

Owning Collections