Confluence: A System for Lossless Multi-Source Single-Sink Data Collection

Patel, Jay A.; Cho, Brian; Gupta, Indranil

Permalink

https://hdl.handle.net/2142/13158 Copy

Description

Title

Confluence: A System for Lossless Multi-Source Single-Sink Data Collection

Author(s)

• Patel, Jay A.
• Cho, Brian
• Gupta, Indranil

Issue Date

2009-07-24

Keyword(s)

• Distributed Systems
• Computer Networks

Date of Ingest

2009-07-24T21:28:16Z

Abstract

Distributed environments often require collection of large amounts of critical and raw data from multiple locations to a central clearinghouse, e.g., task results or large datasets from multiple clouds, logs from multiple PlanetLab nodes, video transcripts in tele-immersive settings, etc. We present the design, implementation and evaluation of Confluence, a system for rapid and lossless transfer of unique files from multiple source nodes to a single sink node. First, we formally model the multi-source single-sink data collection problem for a static network and present an optimal solution in terms of total transfer time. Second, we build in mechanisms to make the system workable in dynamic networks. The resulting Confluence system builds an adaptive source-2-source (s2s) overlay amongst participating nodes, which exploits spatial as well as temporal heterogeneity of available bandwidth.We conduct an evaluation of Confluence on PlanetLab traces in ns-2. Results show that Confluence can improve total transfer time by as much as 40% (with up to 50 sources).

Type of Resource

• text
• image

Genre of Resource

Technical Report

Language

en

Permalink

http://hdl.handle.net/2142/13158

Owning Collections