Automatic identification of application I/O signatures from noisy server-side traces

Yang Liu, Raghul Gunasekaran, Xiaosong Ma, Sudharshan S. Vazhkudai

Research output: Chapter in Book/Report/Conference proceedingConference contribution

50 Citations (Scopus)

Abstract

Competing workloads on a shared storage system cause I/O resource contention and application performance vagaries. This problem is already evident in today’s HPC storage systems and is likely to become acute at ex-ascale. We need more interaction between application I/O requirements and system software tools to help alleviate the I/O bottleneck, moving towards I/O-aware job scheduling. However, this requires rich techniques to capture application I/O characteristics, which remain evasive in production systems. Traditionally, I/O characteristics have been obtained using client-side tracing tools, with drawbacks such as non-trivial instrumentation/development costs, large trace traffic, and inconsistent adoption. We present a novel approach, I/O Signature Identifier (IOSI), to characterize the I/O behavior of data-intensive applications. IOSI extracts signatures from noisy, zero-overhead server-side I/O throughput logs that are already collected on today’s supercomputers, without interfering with the compiling/execution of applications. We evaluated IOSI using the Spider storage system at Oak Ridge National Laboratory, the S3D turbulence application (running on 18,000 Titan nodes), and benchmark-based pseudo-applications. Through our experiments we confirmed that IOSI effectively extracts an application’s I/O signature despite significant server-side noise. Compared to client-side tracing tools, IOSI is transparent, interface-agnostic, and incurs no overhead. Compared to alternative data alignment techniques (e.g., dynamic time warping), it offers higher signature accuracy and shorter processing time.

Original languageEnglish
Title of host publicationProceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014
PublisherUSENIX Association
Pages213-228
Number of pages16
ISBN (Electronic)9781931971089
Publication statusPublished - 1 Jan 2014
Event12th USENIX Conference on File and Storage Technologies, FAST 2014 - Santa Clara, United States
Duration: 17 Feb 201420 Feb 2014

Publication series

NameProceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014

Conference

Conference12th USENIX Conference on File and Storage Technologies, FAST 2014
CountryUnited States
CitySanta Clara
Period17/2/1420/2/14

Fingerprint

Servers
Supercomputers
Turbulence
Scheduling
Throughput
Processing
Costs
Experiments

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software
  • Hardware and Architecture

Cite this

Liu, Y., Gunasekaran, R., Ma, X., & Vazhkudai, S. S. (2014). Automatic identification of application I/O signatures from noisy server-side traces. In Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014 (pp. 213-228). (Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014). USENIX Association.

Automatic identification of application I/O signatures from noisy server-side traces. / Liu, Yang; Gunasekaran, Raghul; Ma, Xiaosong; Vazhkudai, Sudharshan S.

Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014. USENIX Association, 2014. p. 213-228 (Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Liu, Y, Gunasekaran, R, Ma, X & Vazhkudai, SS 2014, Automatic identification of application I/O signatures from noisy server-side traces. in Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014. Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014, USENIX Association, pp. 213-228, 12th USENIX Conference on File and Storage Technologies, FAST 2014, Santa Clara, United States, 17/2/14.
Liu Y, Gunasekaran R, Ma X, Vazhkudai SS. Automatic identification of application I/O signatures from noisy server-side traces. In Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014. USENIX Association. 2014. p. 213-228. (Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014).
Liu, Yang ; Gunasekaran, Raghul ; Ma, Xiaosong ; Vazhkudai, Sudharshan S. / Automatic identification of application I/O signatures from noisy server-side traces. Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014. USENIX Association, 2014. pp. 213-228 (Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014).
@inproceedings{a5c48becdf1845d98bbf91b69463de95,
title = "Automatic identification of application I/O signatures from noisy server-side traces",
abstract = "Competing workloads on a shared storage system cause I/O resource contention and application performance vagaries. This problem is already evident in today’s HPC storage systems and is likely to become acute at ex-ascale. We need more interaction between application I/O requirements and system software tools to help alleviate the I/O bottleneck, moving towards I/O-aware job scheduling. However, this requires rich techniques to capture application I/O characteristics, which remain evasive in production systems. Traditionally, I/O characteristics have been obtained using client-side tracing tools, with drawbacks such as non-trivial instrumentation/development costs, large trace traffic, and inconsistent adoption. We present a novel approach, I/O Signature Identifier (IOSI), to characterize the I/O behavior of data-intensive applications. IOSI extracts signatures from noisy, zero-overhead server-side I/O throughput logs that are already collected on today’s supercomputers, without interfering with the compiling/execution of applications. We evaluated IOSI using the Spider storage system at Oak Ridge National Laboratory, the S3D turbulence application (running on 18,000 Titan nodes), and benchmark-based pseudo-applications. Through our experiments we confirmed that IOSI effectively extracts an application’s I/O signature despite significant server-side noise. Compared to client-side tracing tools, IOSI is transparent, interface-agnostic, and incurs no overhead. Compared to alternative data alignment techniques (e.g., dynamic time warping), it offers higher signature accuracy and shorter processing time.",
author = "Yang Liu and Raghul Gunasekaran and Xiaosong Ma and Vazhkudai, {Sudharshan S.}",
year = "2014",
month = "1",
day = "1",
language = "English",
series = "Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014",
publisher = "USENIX Association",
pages = "213--228",
booktitle = "Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014",

}

TY - GEN

T1 - Automatic identification of application I/O signatures from noisy server-side traces

AU - Liu, Yang

AU - Gunasekaran, Raghul

AU - Ma, Xiaosong

AU - Vazhkudai, Sudharshan S.

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Competing workloads on a shared storage system cause I/O resource contention and application performance vagaries. This problem is already evident in today’s HPC storage systems and is likely to become acute at ex-ascale. We need more interaction between application I/O requirements and system software tools to help alleviate the I/O bottleneck, moving towards I/O-aware job scheduling. However, this requires rich techniques to capture application I/O characteristics, which remain evasive in production systems. Traditionally, I/O characteristics have been obtained using client-side tracing tools, with drawbacks such as non-trivial instrumentation/development costs, large trace traffic, and inconsistent adoption. We present a novel approach, I/O Signature Identifier (IOSI), to characterize the I/O behavior of data-intensive applications. IOSI extracts signatures from noisy, zero-overhead server-side I/O throughput logs that are already collected on today’s supercomputers, without interfering with the compiling/execution of applications. We evaluated IOSI using the Spider storage system at Oak Ridge National Laboratory, the S3D turbulence application (running on 18,000 Titan nodes), and benchmark-based pseudo-applications. Through our experiments we confirmed that IOSI effectively extracts an application’s I/O signature despite significant server-side noise. Compared to client-side tracing tools, IOSI is transparent, interface-agnostic, and incurs no overhead. Compared to alternative data alignment techniques (e.g., dynamic time warping), it offers higher signature accuracy and shorter processing time.

AB - Competing workloads on a shared storage system cause I/O resource contention and application performance vagaries. This problem is already evident in today’s HPC storage systems and is likely to become acute at ex-ascale. We need more interaction between application I/O requirements and system software tools to help alleviate the I/O bottleneck, moving towards I/O-aware job scheduling. However, this requires rich techniques to capture application I/O characteristics, which remain evasive in production systems. Traditionally, I/O characteristics have been obtained using client-side tracing tools, with drawbacks such as non-trivial instrumentation/development costs, large trace traffic, and inconsistent adoption. We present a novel approach, I/O Signature Identifier (IOSI), to characterize the I/O behavior of data-intensive applications. IOSI extracts signatures from noisy, zero-overhead server-side I/O throughput logs that are already collected on today’s supercomputers, without interfering with the compiling/execution of applications. We evaluated IOSI using the Spider storage system at Oak Ridge National Laboratory, the S3D turbulence application (running on 18,000 Titan nodes), and benchmark-based pseudo-applications. Through our experiments we confirmed that IOSI effectively extracts an application’s I/O signature despite significant server-side noise. Compared to client-side tracing tools, IOSI is transparent, interface-agnostic, and incurs no overhead. Compared to alternative data alignment techniques (e.g., dynamic time warping), it offers higher signature accuracy and shorter processing time.

UR - http://www.scopus.com/inward/record.url?scp=85073543975&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85073543975&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85073543975

T3 - Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014

SP - 213

EP - 228

BT - Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST 2014

PB - USENIX Association

ER -