Combining phase identification and statistic modeling for automated parallel benchmark generation

Ye Jin, Xiaosong Ma, Mingliang Liu, Qing Liu, Jeremy Logan, Norbert Podhorszki, Jong Youl Choi, Scott Klasky

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time- and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPrime, a novel tool for tracebased automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPrime benchmarks. They retain the original applications' performance characteristics, in particular their relative performance across platforms. Also, the result benchmarks, already released online, are much more compact and easy-toport compared to the original applications.

Original languageEnglish
Title of host publicationPerformance Evaluation Review
PublisherAssociation for Computing Machinery
Pages309-320
Number of pages12
Volume43
Edition1
DOIs
Publication statusPublished - 24 Jun 2015
EventACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2015 - Portland, United States
Duration: 15 Jun 201519 Jun 2015

Other

OtherACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2015
CountryUnited States
CityPortland
Period15/6/1519/6/15

    Fingerprint

Keywords

  • Asynchronous I/O
  • Benchmark generation
  • HPC applications
  • Markov chain model
  • Phase identification
  • Traces

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Software

Cite this

Jin, Y., Ma, X., Liu, M., Liu, Q., Logan, J., Podhorszki, N., Choi, J. Y., & Klasky, S. (2015). Combining phase identification and statistic modeling for automated parallel benchmark generation. In Performance Evaluation Review (1 ed., Vol. 43, pp. 309-320). Association for Computing Machinery. https://doi.org/10.1145/2745844.2745876