Program transformation and code generation for developing, modeling, and optimizing GPU programs

Stevens, James D

Permalink

https://hdl.handle.net/2142/113011 Copy

Description

Title

Program transformation and code generation for developing, modeling, and optimizing GPU programs

Author(s)

Stevens, James D

Issue Date

2021-07-13

Director of Research (if dissertation) or Advisor (if thesis)

Klöckner, Andreas

Doctoral Committee Chair(s)

Klöckner, Andreas

Committee Member(s)

• Gropp, William
• Solomonik, Edgar
• Owens, John

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• Code generation
• OpenCL
• Program transformation
• Polyhedral model
• Dependencies
• Performance model
• GPU
• Microbenchmark
• Black box
• Performance optimization
• Visual programming interface
• Human-guided transformation

Abstract

The ability to efficiently optimize or re-optimize an algorithm for high performance on a particular processor architecture is crucial in a wide spectrum of engineering and scientific applications. To help bridge the gap between fully automated optimization, which does not reliably produce optimal code, and fully manual optimization, which is inefficient and error-prone, we present three mechanisms facilitating a human-guided, partially automated process for production, optimization, and analysis of high-performance code: (1) formal mechanisms for describing and reasoning about program dependencies and loop nest structure within a programming system providing transformation-based code generation for GPUs and CPUs; (2) a system for constructing customizable, black-box performance models for GPUs; and (3) a visual user interface for program optimization and analysis. We demonstrate how our human-accessible dependency and loop-nest semantics enable reasoning about program correctness in a transformation and optimization procedure for two example scientific computing computations. We show how models created using our hardware-agnostic performance modeling approach can predict execution times accurately enough to select the best-performing of multiple implementation variants of a given computation, demonstrating results for three computations on five different GPUs. We show how our program transformation interface enables access to and navigation of a comprehensive search space of implementation variants for a given computation, as well as comparisons of multiple optimization paths. In doing so, we demonstrate how these architecture-agnostic tools, some of which are already being utilized by a persistent user base, further the goal of facilitating production, optimization, and analysis of high-performance code.

Graduation Semester

2021-08

Type of Resource

Thesis

Permalink

http://hdl.handle.net/2142/113011

Copyright and License Information

Copyright 2021 James D. Stevens

Owning Collections