Tutorial on

The Rodinia Benchmark Suite for Heterogeneous Computing

Sunday morning, Mar.14, 2010
in conjunction with ASPLOS 2010

Kevin Skadron and Shuai Che
University of Virginia Dept. of Computer Science, Computer Engineering Program

Slides are now available in PDF

Background and objectives

The growing importance of heterogeneous systems requires benchmarks to facilitate research in architecture, programming languages, and operating systems.  Benchmarks also provide design patterns for users of such systems.  This objectives of this half-day tutorial are to provide an overview of the unique issues in benchmarking for heterogeneous systems; prepare users for convenient use of the Rodinia benchmark suite; and foster discussion and collaboration regarding future benchmark development.  Rodinia currently provides single-threaded C, OpenMP, and CUDA implementations of a diverse set of applications to use as benchmarks in architecture, compiler, and programming-language research; we also have a couple of FPGA implementations and are working to expand these.  Rodinia provides several versions of some codes, showing different levels of optimization, providing reference implementations for research on parallelizing compilers as well as design patterns for users to develop their own parallel codes.  Rodinia is available for download and further information can be found in these publications:

Topics covered, with approximate durations

1.  [30 mins] Why a benchmark suite for heterogeneous computing is needed, related work (review of other suites, why they aren't suitable, how they could
be extended and link up with or replace Rodinia)
...pause for open floor discussion (Q&A as well as free-form discussion among all participants about benchmarking needs)

2.  [45 mins] Brief overview of heterogeneous architecture issues, key properties and optimization strategies for CUDA and FPGA programming model, brief discussion of OpenCL, all using a demo with a very simple toy application
...pause for open floor discussion

3.  [90 mins] Overview of the Rodinia suite (will take questions throughout):
    a.  Which apps and why
    b.  Diversity analysis
        ...pause for open floor discussion re general benchmark-suite construction issues
    c.  Brief presentation of how each app was parallelized and optimized on each platform and what versions and extra materials come with each app
        ...will include some demos
    d.  Very brief presentation of current results that illustrate key properties of some of the benchmarks
    e.  Deeper dive and code walkthrough on 1-2 key segments on each of 1-2 apps
    ... Q&A
    f.  How to use suite: examples of how to use for direct measurement, simulation, compiler studies.  This includes invocation, inputs, etc.
    ... Q&A
    g.  More general discussion of some ways that this benchmark suite can be used for research (we have many suggestions where Rodinia serves as a hand-optimized reference against which to test automated solutions, e.g. parallelization, scheduling, performance modeling)
    ... open floor discussion to brainstorm more ideas and needs, which segues to the next main topic...

4a.  [remaining time] General discussion of benchmarking strategies: how to make benchmark suites more useful, open research questions in benchmarking
4b.  Design exercise!  Working in small groups, attendess will develop with their top-3 wish list for what is missing in current multicore/heterogeneous benchmarking.  The goal is to find consensus on top research needs and perhaps foster some new collaborations
         ==> presentation by each group, open floor discussion to prioritize the items and suggest some possible solutions

5.  [last 10 min] Wrap up and action items

Last updated 15 Mar. 2010