Combining phase identification and statistic modeling for automated parallel benchmark generation

Ye Jin, Xiaosong Ma, Mingliang Liu, Qing Liu, Jeremy Logan, Norbert Podhorszki, Jong Youl Choi, Scott Klasky

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time- and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPrime, a novel tool for tracebased automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPrime benchmarks. They retain the original applications' performance characteristics, in particular their relative performance across platforms. Also, the result benchmarks, already released online, are much more compact and easy-toport compared to the original applications.

Original languageEnglish
Title of host publicationPerformance Evaluation Review
PublisherAssociation for Computing Machinery
Pages309-320
Number of pages12
Volume43
Edition1
DOIs
Publication statusPublished - 24 Jun 2015
EventACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2015 - Portland, United States
Duration: 15 Jun 201519 Jun 2015

Other

OtherACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2015
CountryUnited States
CityPortland
Period15/6/1519/6/15

Fingerprint

Statistics
Personnel
Hardware
Communication
Experiments

Keywords

  • Asynchronous I/O
  • Benchmark generation
  • HPC applications
  • Markov chain model
  • Phase identification
  • Traces

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Hardware and Architecture
  • Software

Cite this

Jin, Y., Ma, X., Liu, M., Liu, Q., Logan, J., Podhorszki, N., ... Klasky, S. (2015). Combining phase identification and statistic modeling for automated parallel benchmark generation. In Performance Evaluation Review (1 ed., Vol. 43, pp. 309-320). Association for Computing Machinery. https://doi.org/10.1145/2745844.2745876

Combining phase identification and statistic modeling for automated parallel benchmark generation. / Jin, Ye; Ma, Xiaosong; Liu, Mingliang; Liu, Qing; Logan, Jeremy; Podhorszki, Norbert; Choi, Jong Youl; Klasky, Scott.

Performance Evaluation Review. Vol. 43 1. ed. Association for Computing Machinery, 2015. p. 309-320.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Jin, Y, Ma, X, Liu, M, Liu, Q, Logan, J, Podhorszki, N, Choi, JY & Klasky, S 2015, Combining phase identification and statistic modeling for automated parallel benchmark generation. in Performance Evaluation Review. 1 edn, vol. 43, Association for Computing Machinery, pp. 309-320, ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2015, Portland, United States, 15/6/15. https://doi.org/10.1145/2745844.2745876
Jin Y, Ma X, Liu M, Liu Q, Logan J, Podhorszki N et al. Combining phase identification and statistic modeling for automated parallel benchmark generation. In Performance Evaluation Review. 1 ed. Vol. 43. Association for Computing Machinery. 2015. p. 309-320 https://doi.org/10.1145/2745844.2745876
Jin, Ye ; Ma, Xiaosong ; Liu, Mingliang ; Liu, Qing ; Logan, Jeremy ; Podhorszki, Norbert ; Choi, Jong Youl ; Klasky, Scott. / Combining phase identification and statistic modeling for automated parallel benchmark generation. Performance Evaluation Review. Vol. 43 1. ed. Association for Computing Machinery, 2015. pp. 309-320
@inproceedings{57eb3039865b44cf9b156d0d690f652d,
title = "Combining phase identification and statistic modeling for automated parallel benchmark generation",
abstract = "Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time- and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPrime, a novel tool for tracebased automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPrime benchmarks. They retain the original applications' performance characteristics, in particular their relative performance across platforms. Also, the result benchmarks, already released online, are much more compact and easy-toport compared to the original applications.",
keywords = "Asynchronous I/O, Benchmark generation, HPC applications, Markov chain model, Phase identification, Traces",
author = "Ye Jin and Xiaosong Ma and Mingliang Liu and Qing Liu and Jeremy Logan and Norbert Podhorszki and Choi, {Jong Youl} and Scott Klasky",
year = "2015",
month = "6",
day = "24",
doi = "10.1145/2745844.2745876",
language = "English",
volume = "43",
pages = "309--320",
booktitle = "Performance Evaluation Review",
publisher = "Association for Computing Machinery",
edition = "1",

}

TY - GEN

T1 - Combining phase identification and statistic modeling for automated parallel benchmark generation

AU - Jin, Ye

AU - Ma, Xiaosong

AU - Liu, Mingliang

AU - Liu, Qing

AU - Logan, Jeremy

AU - Podhorszki, Norbert

AU - Choi, Jong Youl

AU - Klasky, Scott

PY - 2015/6/24

Y1 - 2015/6/24

N2 - Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time- and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPrime, a novel tool for tracebased automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPrime benchmarks. They retain the original applications' performance characteristics, in particular their relative performance across platforms. Also, the result benchmarks, already released online, are much more compact and easy-toport compared to the original applications.

AB - Parallel application benchmarks are indispensable for evaluating/optimizing HPC software and hardware. However, it is very challenging and costly to obtain high-fidelity benchmarks reecting the scale and complexity of state-of-the-art parallel applications. Hand-extracted synthetic benchmarks are time- and labor-intensive to create. Real applications themselves, while offering most accurate performance evaluation, are expensive to compile, port, reconfigure, and often plainly inaccessible due to security or ownership concerns. This work contributes APPrime, a novel tool for tracebased automatic parallel benchmark generation. Taking as input standard communication-I/O traces of an application's execution, it couples accurate automatic phase identification with statistical regeneration of event parameters to create compact, portable, and to some degree reconfigurable parallel application benchmarks. Experiments with four NAS Parallel Benchmarks (NPB) and three real scientific simulation codes confirm the fidelity of APPrime benchmarks. They retain the original applications' performance characteristics, in particular their relative performance across platforms. Also, the result benchmarks, already released online, are much more compact and easy-toport compared to the original applications.

KW - Asynchronous I/O

KW - Benchmark generation

KW - HPC applications

KW - Markov chain model

KW - Phase identification

KW - Traces

UR - http://www.scopus.com/inward/record.url?scp=84955562122&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84955562122&partnerID=8YFLogxK

U2 - 10.1145/2745844.2745876

DO - 10.1145/2745844.2745876

M3 - Conference contribution

AN - SCOPUS:84955562122

VL - 43

SP - 309

EP - 320

BT - Performance Evaluation Review

PB - Association for Computing Machinery

ER -