One optimized I/O configuration per HPC application: Leveraging the configurability of cloud

Mingliang Liu, Jidong Zhai, Yan Zhai, Xiaosong Ma, Wenguang Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

7 Citations (Scopus)

Abstract

There is a trend to migrate HPC (High Performance Computing) applications to cloud platforms, such as the Amazon EC2 Cluster Compute Instances (CCIs). While existing research has mainly focused on the performance impact of virtualized environments and interconnect technologies on parallel programs, we suggest that the configurability enabled by clouds is another important dimension to explore. Unlike on traditional HPC platforms, on a cloud-resident virtual cluster it is easy to change the I/O configurations, such as the choice of file systems, the number of I/O nodes, and the types of virtual disks, to fit the I/O requirements of different applications. In this paper, we discuss how cloud platforms can be employed to form customized and balanced I/O subsystems for individual I/O-intensive MPI applications. Through our preliminary evaluation, we demonstrate that different applications will benefit from individually tailored I/O system configurations. For a given I/O-intensive application, different I/O settings may lead to significant overall application performance or cost difference (up to 2.5-fold). Our exploration indicates that customized system configuration for HPC applications in the cloud is important and non-trivial.

Original languageEnglish
Title of host publicationProceedings of the 2nd Asia-Pacific Workshop on Systems, APSys'11
DOIs
Publication statusPublished - 1 Dec 2011
Event2nd Asia-Pacific Workshop on Systems, APSys'11 - Shanghai, China
Duration: 11 Jul 201112 Jul 2011

Publication series

NameProceedings of the 2nd Asia-Pacific Workshop on Systems, APSys'11

Other

Other2nd Asia-Pacific Workshop on Systems, APSys'11
CountryChina
CityShanghai
Period11/7/1112/7/11

    Fingerprint

ASJC Scopus subject areas

  • Control and Systems Engineering

Cite this

Liu, M., Zhai, J., Zhai, Y., Ma, X., & Chen, W. (2011). One optimized I/O configuration per HPC application: Leveraging the configurability of cloud. In Proceedings of the 2nd Asia-Pacific Workshop on Systems, APSys'11 (Proceedings of the 2nd Asia-Pacific Workshop on Systems, APSys'11). https://doi.org/10.1145/2103799.2103818