On-the-fly recovery of job input data in supercomputers

Chao Wang, Zhe Zhang, Sudharshan S. Vazhkudai, Xiaosong Ma, Frank Mueller

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Storage system failure is a serious concern as we approach Petascale computing. Even at today's sub-Petascale levels, I/O failure is the leading cause of downtimes and job failures. We contribute a novel, on-the-fly recovery frame-work for job input data into supercomputer parallel file systems. The framework exploits key traits of the HPC I/O workload to reconstruct lost input data during job execution from remote, immutable copies. Each reconstructed data stripe is made immediately accessible in the client request order due to the delayed metadata update and fine-granular locking while unrelated access to the same file remains unaffected. We have implemented the recovery component within the Lustre parallel file system, thus building a novel application-transparent online recovery solution. Our solution is integrated into Lustre 's two-level locking scheme using a two-phase blocking protocol. Combining parametric and simulation studies, our experiments demonstrate a significant improvement in HPC center serviceability and user job turnaround time.

Original languageEnglish
Title of host publicationProceedings of the International Conference on Parallel Processing
Pages620-627
Number of pages8
DOIs
Publication statusPublished - 17 Nov 2008
Externally publishedYes
Event37th International Conference on Parallel Processing, ICPP 2008 - Portland, OR, United States
Duration: 9 Sep 200812 Sep 2008

Other

Other37th International Conference on Parallel Processing, ICPP 2008
CountryUnited States
CityPortland, OR
Period9/9/0812/9/08

    Fingerprint

ASJC Scopus subject areas

  • Software
  • Mathematics(all)
  • Hardware and Architecture

Cite this

Wang, C., Zhang, Z., Vazhkudai, S. S., Ma, X., & Mueller, F. (2008). On-the-fly recovery of job input data in supercomputers. In Proceedings of the International Conference on Parallel Processing (pp. 620-627). [4625901] https://doi.org/10.1109/ICPP.2008.28