A highly concurrent software-based sector cache for GPU orchestrated storage access

Masood, Amna

Permalink

https://hdl.handle.net/2142/115951 Copy

Description

Title

A highly concurrent software-based sector cache for GPU orchestrated storage access

Author(s)

Masood, Amna

Issue Date

2022-07-18

Director of Research (if dissertation) or Advisor (if thesis)

Hwu, Wen-mei W

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

Graphics Processing Unit, Storage, Cache

Abstract

Heterogeneous systems with programmable accelerators are becoming increasingly popular with each passing day, as increasing demand for both data and computing power exceeds the capabilities of CPU-only systems. Graphics Processing Units (GPUs) are programmable accelerators widely used to meet the requirements of data and compute intensive applications, such as graph and data analytics, and recommender systems. However, these workloads require access to increasingly larger datasets, many of which cannot fit in GPU memory, hence limiting the portability of these applications to GPUs. In order to increase the effective memory capacity of GPUs, prior work Big Accelerator Memory (BaM) proposed to employ modern Solid State Storage devices as backup memory that GPU threads can access on-demand. To exploit the locality of the data and amortize the high storage access latency, BaM proposed a highly concurrent software-managed cache for GPUs. However, the BaM cache is not scalable for large dataset sizes, as it requires large metadata structures in GPU memory. The size of these structures is proportional to the size of the input dataset and exceeds the GPU memory capacity for larger datasets. To address these limitations, this thesis proposes a high-throughput, highly concurrent software-managed sector cache design. The proposed software-managed sector cache design has the same set of features as the BaM concurrent cache design but with significantly lower metadata size. This makes the proposed sector cache design highly scalable and a direct replacement for BaM cache design. Our proposed cache design provides high throughput and employs optimizations to reduce the number of I/O accesses and exploit temporal and spatial locality. We evaluate our cache design using off-the-shelf GPUs and commodity SSDs and observe that it can provide high throughput of up to 350 GB/s. Through micro-benchmarking with various access patterns, we observe that for same I/O access granularity (4KB) and same cacheline size (4KB), the additional atomics needed for sector state management sector incur only a 10-15% overhead in throughput over BaM’s baseline cache. As the sector cache separates the cache-line granularity from the I/O access granularity, it enables fine-grain access even with large cacheline sizes. When we increase the cacheline size for both sector and baseline cache to 32KB, keeping the sector size at 4KB, the sector cache outperforms the baseline cache by up to 87% for random sector access workloads, as the sector cache’s finer I/O access granularity causes less I/O amplification. For graph applications, sector cache’s performance with coarse cacheline sizes (32KB) is on-par with the baseline cache design, keeping the I/O granularity same for both (4KB), while reducing the metadata by up to 6x. This shows that sector cache can enable consistent or better performance while reducing the metadata size. These results show that sector cache is necessary to scale the baseline system to larger dataset sizes while maintaining consistent performance.

Graduation Semester

2022-08

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Amna Masood

Owning Collections