A Transformation to Convert Packing Code to Compact Datatypes for Efficient Zero-Copy Data Transfer

Fredrik Kjolstad; Hoefler, Torsten; Snir, Marc

Permalink

https://hdl.handle.net/2142/26452 Copy

Description

Title

A Transformation to Convert Packing Code to Compact Datatypes for Efficient Zero-Copy Data Transfer

Author(s)

• Fredrik Kjolstad
• Hoefler, Torsten
• Snir, Marc

Issue Date

2011-08

Keyword(s)

Program Transformation, Optimization, Datatypes, Refactoring, Compiler

Abstract

Many high performance applications spend considerable time packing data into contiguous communication buffers. Datatypes provide an alternative by describing the layout of the communicated data. This empowers the runtime system to retrieve non-contiguous elements directly from application data structures. However, programmers find complex datatypes hard to use and are reluctant to invest time and effort to rewrite packing code to datatype code. Fortunately, the transformation from packing code to datatypes can be automated, and the programmer can replace packing code with datatypes at the push of a button. The transformation allows easy porting of applications to new machines that benefit from datatypes, thus improving programmer productivity. We present an algorithm for converting packing code to datatype code based on a novel IR and a suite of optimizations. We have implemented the algorithm in a tool that transforms C packing code to an MPI datatype, and rewrites the packing code consumer to instead use the datatype. Our evaluation shows that our algorithm is applicable to real-world packing code, that it is fast enough to be used interactively, and that the datatypes it produces are compact and well optimized. Finally, we evaluate the performance of the code produced by our tool, showing that it outperforms the original packing code on a state-of-the art system. However, the real benefit of datatypes is in the future, when hardware support for non-contiguous gather-scatter transfers becomes ubiquitous.

Type of Resource

text

Language

en

Permalink

http://hdl.handle.net/2142/26452

Owning Collections