Performance modeling framework for SLO-driven MapReduce environments

Verma, Abhishek

Permalink

https://hdl.handle.net/2142/42276 Copy

Description

Title

Performance modeling framework for SLO-driven MapReduce environments

Author(s)

Verma, Abhishek

Issue Date

2013-02-03T19:30:03Z

Director of Research (if dissertation) or Advisor (if thesis)

Campbell, Roy H.

Doctoral Committee Chair(s)

Campbell, Roy H.

Committee Member(s)

• Gupta, Indranil
• Gropp, William D.
• Cherkasova, Ludmila

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• MapReduce
• Performance Modeling
• Service Level Objectives
• Hadoop

Abstract

Several companies are increasingly using MapReduce for efficient large scale data processing such as personalized advertising, spam detection, and data mining tasks. There is a growing need among MapReduce users to achieve different Service Level Objectives (SLOs). Often, applications need to complete data processing within a certain time deadline. Alternatively, users are interested in completing a set of jobs as fast as possible. Designing, prototyping, and evaluating new resource allocation and job scheduling algorithms to support these SLOs in MapReduce environments is challenging, labor-intensive, and time-consuming. Hence, accurate and efficient workload management and performance modeling tools are needed. Our hypothesis is that performance modeling of MapReduce environments through a combination of measurement, simulation, and analytical modeling for enabling different service level objectives is feasible, novel, and useful. To support this hypothesis, we propose an analytical performance model based on key performance characteristics measured from past job executions and build a simulator capable of replaying these job traces. We survey different attempts at performance modeling and its applications, and contrast our work. To demonstrate the usefulness of our techniques, we apply them to achieve service level objectives such as enabling deadline-driven scheduling, optimizing makespan of a set of MapReduce jobs and comparing hardware alternatives.

Graduation Semester

2012-12

Permalink

http://hdl.handle.net/2142/42276

Copyright and License Information

Copyright 2012 Abhishek Verma

Owning Collections