University of Illinois Urbana-Champaign

Parallel file system with tunable consistency

Wang, Chen

Content Files
WANG-DISSERTATION-2022.pdf

Permalink

https://hdl.handle.net/2142/116191

Description

Title
Parallel file system with tunable consistency
Author(s)
Wang, Chen
Issue Date
2022-07-08
Director of Research (if dissertation) or Advisor (if thesis)
  • Snir, Marc
  • Mohror, Kathryn
Doctoral Committee Chair(s)
Snir, Marc
Committee Member(s)
  • Huang, Jian
  • Winslett, Marianne
  • Gropp, William
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
Ph.D.
Degree Level
Dissertation
Keyword(s)
  • Parallel file system
  • Consistency model
  • Tunable consistency
  • Parallel Storage
  • I/O Tracing
  • MPI Tracing
Abstract
In high performance computing (HPC) systems, the I/O demands of applications are supported by parallel file systems such as Lustre and GPFS. Most general-purpose parallel file systems (PFS) support the POSIX I/O interface and its consistency model. However, the POSIX standard was defined decades ago for use by a single machine with a single storage device. It is not fit for highly concurrent applications typically seen on modern HPC systems. The major impediment to the PFS performance is the strict adherence to POSIX consistency semantics, which requires sequential consistency in general and atomicity for many operations. The strict enforcement of these requirements impedes caching, generates significant additional traffic, and results in congestion in situations of high sharing, especially for small block reads and writes. The use of the POSIX consistency model for I/O has plagued the HPC community for many years, but it is becoming more problematic due to two key reasons: (1) the rapid increase in the scale of HPC systems; (2) the emergence of the new storage techniques such as persistent memory. They make the overhead of maintaining POSIX consistency relatively higher. Many efforts have been made toward PFSs with relaxed consistency semantics. However, different applications have different consistency requirements. A PFS providing a static consistency model will not work ideally for every application. Moreover, the correctness is not guaranteed if the provided consistency model is weaker than the desired one. In this dissertation, we first propose a multi-level I/O and MPI tracing tool. We then collect detailed traces from 17 representative HPC applications and I/O benchmarks. Next, we employ a trace-driven analysis approach to study the consistency requirements of these applications. And finally, based on this study, we propose and design a parallel file system that provides tunable consistency models.
Graduation Semester
2022-08
Type of Resource
Thesis
Handle URL
https://hdl.handle.net/2142/116191
Copyright and License Information
Copyright 2022 Chen Wang

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois