Flexible and efficient parallel I/O for large-scale multi-component simulations

Xiaosong Ma, Xiangmin Jiao, Michael Campbell, Marianne Winslett

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

In this paper, we discuss our experience of providing high performance parallel I/O for a large-scale, on-going, multi-disciplinary simulation project for solid propellant rockets. We describe the performance and data management issues observed in this project and present our solutions, including (1) support for relatively fine-grained distribution of irregular datasets in parallel I/O, (2) a flexible data management facility for inter-module communication, and (3) two schemes to overlap computation with I/O. Performance results obtained from both development and production platforms show that our I/O optimizations can dramatically reduce the simulation's visible I/O cost, as well as the number of disk files, and significantly improve the overall performance. Meanwhile, our data management facility helps to provide simulation developers with simple user interfaces for parallel I/O.

Original languageEnglish
Title of host publicationProceedings - International Parallel and Distributed Processing Symposium, IPDPS 2003
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Print)0769519261, 9780769519265
DOIs
Publication statusPublished - 2003
Externally publishedYes
EventInternational Parallel and Distributed Processing Symposium, IPDPS 2003 - Nice, France
Duration: 22 Apr 200326 Apr 2003

Other

OtherInternational Parallel and Distributed Processing Symposium, IPDPS 2003
CountryFrance
CityNice
Period22/4/0326/4/03

    Fingerprint

ASJC Scopus subject areas

  • Computational Theory and Mathematics
  • Theoretical Computer Science
  • Software

Cite this

Ma, X., Jiao, X., Campbell, M., & Winslett, M. (2003). Flexible and efficient parallel I/O for large-scale multi-component simulations. In Proceedings - International Parallel and Distributed Processing Symposium, IPDPS 2003 [1213462] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IPDPS.2003.1213462