Efficient memory access in modern accelerators

Asgharimoghaddam, Hadi

Permalink

https://hdl.handle.net/2142/115685 Copy

Description

Title

Efficient memory access in modern accelerators

Author(s)

Asgharimoghaddam, Hadi

Issue Date

2022-04-08

Director of Research (if dissertation) or Advisor (if thesis)

Fletcher, Christopher

Doctoral Committee Chair(s)

Fletcher, Christopher

Committee Member(s)

• Emer, Joel
• Kumar, Rakesh
• Patel, Sanjay
• Solomonik, Edgar

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Computer Architecture
• Memory Systems
• 3D Die-Stacking
• Sparse Tensor Algebra
• Sparse Accelerators
• Dynamic Tiling

Abstract

This dissertation examines various methods of narrowing the performance gap between the main memory and processing units in accelerators. We divide approaches into two categories of improving the main memory system performance and reducing reliance on memory. Then, we offer novel techniques to address each group of approaches. To improve the performance of the memory system, first, we investigate near-DRAM acceleration. In this approach, the processing unit dies are stacked on top of dynamic random access memory (DRAM) dies through 3D die-stacking technology resulting in a power-efficient higher aggregate bandwidth memory system. Second, we propose in-buffer processing as an alternative to 3D die-stacking technology. In this solution, we place accelerators in data buffers of load-reduced dual inline memory module (LRDIMM) memory, which is originally developed to support large memory systems for servers, to avoid relying on 3D/2.5D die stacking. To reduce reliance on the memory system, we investigate various optimizations pertaining to sparse data orchestration. First, we propose hierarchical intersection to eliminate ineffectual data fetch and computation in sparse tensor algebra. Second, we investigate data orchestration, specifically dynamic tiling, in sparse matrix multiplication (SpMSpM) to reduce main memory traffic. Finally, we propose a generalized dynamic tiling and data orchestration mechanism for sparse tensor algebra. We present our generalized data orchestration unit as a generic primitive that can be integrated into accelerators with arbitrary dataflow, encompassing all the optimizations we proposed to reduce reliance on the memory system. This dissertation offers various mechanisms to overcome the memory access bottlenecks resulting in a holistic solution for efficient memory access in modern accelerators.

Graduation Semester

2022-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Hadi Asgharimoghaddam

Owning Collections