University of Illinois Urbana-Champaign

Performance modeling framework for SLO-driven MapReduce environments

Verma, Abhishek

Content Files
Abhishek_Verma.pdf

Permalink

https://hdl.handle.net/2142/42276

Description

Title
Performance modeling framework for SLO-driven MapReduce environments
Author(s)
Verma, Abhishek
Issue Date
2013-02-03T19:30:03Z
Director of Research (if dissertation) or Advisor (if thesis)
Campbell, Roy H.
Doctoral Committee Chair(s)
Campbell, Roy H.
Committee Member(s)
  • Gupta, Indranil
  • Gropp, William D.
  • Cherkasova, Ludmila
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Date of Ingest
2013-02-03T19:30:03Z
Keyword(s)
  • MapReduce
  • Performance Modeling
  • Service Level Objectives
  • Hadoop
Abstract
Several companies are increasingly using MapReduce for efficient large scale data processing such as personalized advertising, spam detection, and data mining tasks. There is a growing need among MapReduce users to achieve different Service Level Objectives (SLOs). Often, applications need to complete data processing within a certain time deadline. Alternatively, users are interested in completing a set of jobs as fast as possible. Designing, prototyping, and evaluating new resource allocation and job scheduling algorithms to support these SLOs in MapReduce environments is challenging, labor-intensive, and time-consuming. Hence, accurate and efficient workload management and performance modeling tools are needed. Our hypothesis is that performance modeling of MapReduce environments through a combination of measurement, simulation, and analytical modeling for enabling different service level objectives is feasible, novel, and useful. To support this hypothesis, we propose an analytical performance model based on key performance characteristics measured from past job executions and build a simulator capable of replaying these job traces. We survey different attempts at performance modeling and its applications, and contrast our work. To demonstrate the usefulness of our techniques, we apply them to achieve service level objectives such as enabling deadline-driven scheduling, optimizing makespan of a set of MapReduce jobs and comparing hardware alternatives.
Graduation Semester
2012-12
Permalink
http://hdl.handle.net/2142/42276
Copyright and License Information
Copyright 2012 Abhishek Verma

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois
Dissertations and Theses - Computer Science
Dissertations and Theses from the Siebel School of Computer Science