Learning based hierarchical load balancer

Ghosh, Pathikrit

Permalink

https://hdl.handle.net/2142/115706 Copy

Description

Title

Learning based hierarchical load balancer

Author(s)

Ghosh, Pathikrit

Issue Date

2022-04-25

Director of Research (if dissertation) or Advisor (if thesis)

Kale, Laxmikant

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• High Performance Computing
• Load Balancer, Machine Learning

Abstract

Computer architects are finding newer ways to develop faster systems despite the restrictions with respect to the speed of each individual processing units. Often this involves complex parallel and heterogeneous design of these machines. To fully exploit these machines it is becoming necessary for parallel applications running on these machines to improve their software design and also develop better tools to use along with them. One of the major challenges in this design is the problem of load balancing. Load imbalance can lead to wastage of hardware resources and thereby slow the overall running time of the application. The nature of load balancing that we use and the properties of load balancer often is determined by the application and hardware we run the application on. We can use computation-aware, communication-aware or topology-aware load balancers. Amongst them there are a host of available static or dynamic load balancing options that an application can choose from. Often though this needs extensive domain knowledge and multiple runs of the application to tune the load balancing frequency and the list of load balancers that we will use for our final model. This requires both human time and resource utilization just to find the best possible configurations for one particular application. Inspite of doing all these, we still might not always find the best possible configurations. This makes the challenge of choosing a load balancer complex even with the diverse options available to us. Even with finely tuned load balancing configurations, we may have situations where we do not need a global load balancing step. A global load balancing step can have an overhead that affects application performance if this is done when not needed. We can instead solve the problems by having multiple within node load balancing steps, which has a smaller overhead. This creates another challenge with respect to scaling load balancers to large clusters. In the following dissertation we will focus on solving these issues with load balancers while also proposing models that run better than load balancers proposed after multiple iterations of fine-tuning. Here, we propose a load balancing algorithm that uses runtime system collected statistics to represent the state of an application and then uses learning models to predict a good choice of load balancer to use based on the collected statistics. We also propose the load balancer within a hierarchical framework that allows us to scale the load balancer better across multiple nodes. The hierarchical framework also allows us to avoid the overhead of global load balancing, when within node load balancing can solve the problems with an application.

Graduation Semester

2022-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Pathikrit Ghosh

Owning Collections