New consistency orchestrators for emerging distributed systems

Ahsan, Shegufta Bakht

Permalink

https://hdl.handle.net/2142/108142 Copy

Description

Title

New consistency orchestrators for emerging distributed systems

Author(s)

Ahsan, Shegufta Bakht

Issue Date

2020-05-03

Director of Research (if dissertation) or Advisor (if thesis)

Gupta, Indranil

Doctoral Committee Chair(s)

Gupta, Indranil

Committee Member(s)

• Nahrstedt, Klara
• Borisov, Nikita
• Agrawal, Nitin

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

Internal Orchestrators, Consistency, Failure Detector, Smart Home, Internet of Things (IoT), Service Fabric

Abstract

We are gradually becoming more dependent on various distributed systems, e.g., smart home management systems, bank management, traffic monitoring, etc. Some run on data-centers while others run on edge devices, e.g., smart home applications. In all these environments, efficiently maintaining consistency across such a large number of components is a hard challenge. Consistency ensures a coherent view across the disparate components of a distributed system. An inconsistent view might lead to various issues that directly impact user experience. For example, in a database management system, an inconsistent view of a primary replica might make the system slow, or show stale data to users. If a bank account has two primary-replicas, each of them might show their own version of the account balance -- this leads to incorrect banking transactions. Similarly, in a smart home, failing to isolate concurrent routines (sequence of commands) might end the home in a state not consistent with the user's expectation. E.g., the outcome of two concurrent routines R1 = {Turn all light ON} and R2 = {Turn all light OFF} might end up in a state where some of the lights are ON while others are OFF. Addressing this requires the disparate components of a distributed system to have coordination. Such coordination includes maintaining a consistent view of the failed nodes, leader election, consistent primary replica selection, coherence in smart home's current state, etc. Orchestrators are dedicated entities that use specialized protocols (e.g., Chubby, ZooKeeper etc.) to help coordinate the components. Distributed systems can use i) generic external orchestrators such as Zookeeper, Chubby etc., or 2) build their own internal orchestrator. Unlike external orchestrators, internal orchestrators avoid external dependencies and are flexible and modifiable, e.g., making it relatively easier to provide complex consistency guarantees and providing consistent and reliable distributed data-structures. In this thesis we present new internal orchestrators for maintaining consistency in both edge-based and cloud-based distributed systems. This thesis has the following contributions: for edge-based distributed systems, we develop a smart home orchestrator called SafeHome that offers a congruent end state by guaranteeing stronger properties, e.g., Isolation, Atomicity, Safety. This is an improvement over the best-effort philosophy used in today's smart homes which leads to incongruent states. Second, For cloud-based distributed systems, we reveal and analyze Service Fabric's consistent and scalable failure detector, which is the heart of its internal orchestration mechanisms. Third, we present a new way to decentralize SF's arbitration technique and design a consistent failure detector on top of it, which offers identical consistency guaranties as SF's centralized scheme. We also provide formal proof of correctness and time-bound for both central and distributed arbitrator schemes.

Graduation Semester

2020-05

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/108142

Copyright and License Information

Copyright 2020 Shegufta Ahsan

Owning Collections