Efficient Distributed File System (EDFS)

Kassa, Debessay Fesehaye; Malik, Rahul; Nahrstedt, Klara

Permalink

https://hdl.handle.net/2142/16563 Copy

Description

Title

Efficient Distributed File System (EDFS)

Author(s)

• Kassa, Debessay Fesehaye
• Malik, Rahul
• Nahrstedt, Klara

Issue Date

2010-07-11

Keyword(s)

Cloud architecture, resource allocation, distributed namenode

Abstract

Large scale distributed systems such as cloud computing applications are becoming very common. These applications come with increasing challenges on how to transfer and where to store and compute data. The most prevalent distributed file systems to deal with these challenges is the Hadoop File System (HDFS) which is a variant of the Google File System (GFS). However HDFS has two potential problems. The first one is that it depends on a single name node to manage almost all operations of every data block in the file system. As a result it can be a bottleneck resource and a single point of failure. The second potential problem with HDFS is that it depends on TCP to transfer data. As has been cited in many studies TCP takes many rounds before it can send at the full capacity of the links in the cloud. This results in low link utilization and longer download times. To overcome these problems of HDFS we present a new distributed file system. Our scheme uses a light weight front end server to connect all requests with many name nodes. This helps distribute load of a single name node to many name nodes. Our second contribution is to use an efficient protocol to send and route data. Our protocol can achieve full link utilization and hence decreased download times. Based on simulation our protocol can outperform HDFS and hence GFS.

Type of Resource

text

Language

en

Permalink

http://hdl.handle.net/2142/16563

Owning Collections