Automatic identification of application I/O signatures from noisy server-side traces

Yang Liu, Raghul Gunasekaran, Xiaosong Ma, Sudharshan S. Vazhkudai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

53 Citations (Scopus)

Abstract

Competing workloads on a shared storage system cause I/O resource contention and application performance vagaries. This problem is already evident in today’s HPC storage systems and is likely to become acute at ex-ascale. We need more interaction between application I/O requirements and system software tools to help alleviate the I/O bottleneck, moving towards I/O-aware job scheduling. However, this requires rich techniques to capture application I/O characteristics, which remain evasive in production systems. Traditionally, I/O characteristics have been obtained using client-side tracing tools, with drawbacks such as non-trivial instrumentation/development costs, large trace traffic, and inconsistent adoption. We present a novel approach, I/O Signature Identifier (IOSI), to characterize the I/O behavior of data-intensive applications. IOSI extracts signatures from noisy, zero-overhead server-side I/O throughput logs that are already collected on today’s supercomputers, without interfering with the compiling/execution of applications. We evaluated IOSI using the Spider storage system at Oak Ridge National Laboratory, the S3D turbulence application (running on 18,000 Titan nodes), and benchmark-based pseudo-applications. Through our experiments we confirmed that IOSI effectively extracts an application’s I/O signature despite significant server-side noise. Compared to client-side tracing tools, IOSI is transparent, interface-agnostic, and incurs no overhead. Compared to alternative data alignment techniques (e.g., dynamic time warping), it offers higher signature accuracy and shorter processing time.

Original languageEnglish
Title of host publicationProceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014
PublisherUSENIX Association
Pages213-228
Number of pages16
ISBN (Electronic)9781931971089
Publication statusPublished - 1 Jan 2014
Event12th USENIX Conference on File and Storage Technologies, FAST 2014 - Santa Clara, United States
Duration: 17 Feb 201420 Feb 2014

Publication series

NameProceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014

Conference

Conference12th USENIX Conference on File and Storage Technologies, FAST 2014
CountryUnited States
CitySanta Clara
Period17/2/1420/2/14

    Fingerprint

ASJC Scopus subject areas

  • Hardware and Architecture
  • Computer Networks and Communications
  • Software

Cite this

Liu, Y., Gunasekaran, R., Ma, X., & Vazhkudai, S. S. (2014). Automatic identification of application I/O signatures from noisy server-side traces. In Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014 (pp. 213-228). (Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014). USENIX Association.