Elastic scale-out for partition-based database systems

Umar Farooq Minhas, Rui Liu, Ashraf Aboulnaga, Kenneth Salem, Jonathan Ng, Sean Robertson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

An important goal for database systems today is to provide elastic scale-out, i.e., the ability to grow and shrink processing capacity on demand, with varying load. Database systems are difficult to scale since they are stateful - they manage a large database, and it is important when scaling to multiple server machines to provide mechanisms so that these machines can collaboratively manage the database and maintain its consistency. Database partitioning is often used to solve this problem, with each server machine being responsible for one partition. In this paper, we propose that the flexibility provided by a partitioned, shared nothing parallel database system can be exploited to provide elastic scale-out. The idea is to start with a small number of server machines that manage all partitions, and to elastically scale out by dynamically adding new server machines and redistributing database partitions among these servers. We present an implementation of this approach for elastic scale-out using VoltDB - an in-memory, partitioned, shared nothing parallel database system. Our main goal in this paper is to identify several manageability problems that arise when using this approach for elastic scale-out. The paper presents some of these problems and outlines a research agenda for this area.

Original languageEnglish
Title of host publicationProceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012
Pages281-288
Number of pages8
DOIs
Publication statusPublished - 19 Nov 2012
Event2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012 - Arlington, VA, United States
Duration: 1 Apr 20125 Apr 2012

Publication series

NameProceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012

Other

Other2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012
CountryUnited States
CityArlington, VA
Period1/4/125/4/12

    Fingerprint

ASJC Scopus subject areas

  • Software

Cite this

Minhas, U. F., Liu, R., Aboulnaga, A., Salem, K., Ng, J., & Robertson, S. (2012). Elastic scale-out for partition-based database systems. In Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012 (pp. 281-288). [6313694] (Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012). https://doi.org/10.1109/ICDEW.2012.52