Abstract
In this paper, we discuss our experience of providing high performance parallel I/O for a large-scale, on-going, multi-disciplinary simulation project for solid propellant rockets. We describe the performance and data management issues observed in this project and present our solutions, including (1) support for relatively fine-grained distribution of irregular datasets in parallel I/O, (2) a flexible data management facility for inter-module communication, and (3) two schemes to overlap computation with I/O. Performance results obtained from both development and production platforms show that our I/O optimizations can dramatically reduce the simulation's visible I/O cost, as well as the number of disk files, and significantly improve the overall performance. Meanwhile, our data management facility helps to provide simulation developers with simple user interfaces for parallel I/O.
| Original language | English |
|---|---|
| Title of host publication | Proceedings - International Parallel and Distributed Processing Symposium, IPDPS 2003 |
| Publisher | Institute of Electrical and Electronics Engineers Inc. |
| ISBN (Print) | 0769519261, 9780769519265 |
| DOIs | |
| Publication status | Published - 2003 |
| Externally published | Yes |
| Event | International Parallel and Distributed Processing Symposium, IPDPS 2003 - Nice, France Duration: 22 Apr 2003 → 26 Apr 2003 |
Other
| Other | International Parallel and Distributed Processing Symposium, IPDPS 2003 |
|---|---|
| Country | France |
| City | Nice |
| Period | 22/4/03 → 26/4/03 |
Fingerprint
ASJC Scopus subject areas
- Computational Theory and Mathematics
- Theoretical Computer Science
- Software
Cite this
Flexible and efficient parallel I/O for large-scale multi-component simulations. / Ma, Xiaosong; Jiao, Xiangmin; Campbell, Michael; Winslett, Marianne.
Proceedings - International Parallel and Distributed Processing Symposium, IPDPS 2003. Institute of Electrical and Electronics Engineers Inc., 2003. 1213462.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Flexible and efficient parallel I/O for large-scale multi-component simulations
AU - Ma, Xiaosong
AU - Jiao, Xiangmin
AU - Campbell, Michael
AU - Winslett, Marianne
PY - 2003
Y1 - 2003
N2 - In this paper, we discuss our experience of providing high performance parallel I/O for a large-scale, on-going, multi-disciplinary simulation project for solid propellant rockets. We describe the performance and data management issues observed in this project and present our solutions, including (1) support for relatively fine-grained distribution of irregular datasets in parallel I/O, (2) a flexible data management facility for inter-module communication, and (3) two schemes to overlap computation with I/O. Performance results obtained from both development and production platforms show that our I/O optimizations can dramatically reduce the simulation's visible I/O cost, as well as the number of disk files, and significantly improve the overall performance. Meanwhile, our data management facility helps to provide simulation developers with simple user interfaces for parallel I/O.
AB - In this paper, we discuss our experience of providing high performance parallel I/O for a large-scale, on-going, multi-disciplinary simulation project for solid propellant rockets. We describe the performance and data management issues observed in this project and present our solutions, including (1) support for relatively fine-grained distribution of irregular datasets in parallel I/O, (2) a flexible data management facility for inter-module communication, and (3) two schemes to overlap computation with I/O. Performance results obtained from both development and production platforms show that our I/O optimizations can dramatically reduce the simulation's visible I/O cost, as well as the number of disk files, and significantly improve the overall performance. Meanwhile, our data management facility helps to provide simulation developers with simple user interfaces for parallel I/O.
UR - http://www.scopus.com/inward/record.url?scp=84947257456&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84947257456&partnerID=8YFLogxK
U2 - 10.1109/IPDPS.2003.1213462
DO - 10.1109/IPDPS.2003.1213462
M3 - Conference contribution
SN - 0769519261
SN - 9780769519265
BT - Proceedings - International Parallel and Distributed Processing Symposium, IPDPS 2003
PB - Institute of Electrical and Electronics Engineers Inc.
ER -