Cost-effective cloud HPC resource provisioning by building semi-elastic virtual clusters

Shuangcheng Niu, Jidong Zhai, Xiaosong Ma, Xiongchao Tang, Wenguang Chen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

25 Citations (Scopus)

Abstract

Recent studies have found cloud environments increasingly appealing for executing HPC applications, including tight-ly coupled parallel simulations. While public clouds offer elastic, on-demand resource provisioning and pay-as-you-go pricing, individual users setting up their on-demand virtual clusters may not be able to take full advantage of common cost-saving opportunities, such as reserved instances. In this paper, we propose a Semi-Elastic Cluster (SEC) computing model for organizations to reserve and dynam-ically resize a virtual cloud-based cluster. We present a set of integrated batch scheduling plus resource scaling strategies uniquely enabled by SEC, as well as an online reserved instance provisioning algorithm based on job history. Our trace-driven simulation results show that such a model has a 61.0% cost saving than individual users acquiring and managing cloud resources without causing longer average job wait time. Meanwhile, the overhead of acquiring/maintaining shared cloud instances is shown to take only a few seconds.

Original languageEnglish
Title of host publicationInternational Conference for High Performance Computing, Networking, Storage and Analysis, SC
PublisherIEEE Computer Society
ISBN (Print)9781450323789
DOIs
Publication statusPublished - 1 Jan 2013
Externally publishedYes
Event2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013 - Denver, CO, United States
Duration: 17 Nov 201322 Nov 2013

Other

Other2013 International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2013
CountryUnited States
CityDenver, CO
Period17/11/1322/11/13

    Fingerprint

Keywords

  • Cloud computing
  • Job scheduling
  • Resource provisioning

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications
  • Hardware and Architecture
  • Software

Cite this

Niu, S., Zhai, J., Ma, X., Tang, X., & Chen, W. (2013). Cost-effective cloud HPC resource provisioning by building semi-elastic virtual clusters. In International Conference for High Performance Computing, Networking, Storage and Analysis, SC [56] IEEE Computer Society. https://doi.org/10.1145/2503210.2503236