Elastic scale-out for partition-based database systems

Umar Farooq Minhas, Rui Liu, Ashraf Aboulnaga, Kenneth Salem, Jonathan Ng, Sean Robertson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

An important goal for database systems today is to provide elastic scale-out, i.e., the ability to grow and shrink processing capacity on demand, with varying load. Database systems are difficult to scale since they are stateful - they manage a large database, and it is important when scaling to multiple server machines to provide mechanisms so that these machines can collaboratively manage the database and maintain its consistency. Database partitioning is often used to solve this problem, with each server machine being responsible for one partition. In this paper, we propose that the flexibility provided by a partitioned, shared nothing parallel database system can be exploited to provide elastic scale-out. The idea is to start with a small number of server machines that manage all partitions, and to elastically scale out by dynamically adding new server machines and redistributing database partitions among these servers. We present an implementation of this approach for elastic scale-out using VoltDB - an in-memory, partitioned, shared nothing parallel database system. Our main goal in this paper is to identify several manageability problems that arise when using this approach for elastic scale-out. The paper presents some of these problems and outlines a research agenda for this area.

Original languageEnglish
Title of host publicationProceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012
Pages281-288
Number of pages8
DOIs
Publication statusPublished - 19 Nov 2012
Externally publishedYes
Event2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012 - Arlington, VA, United States
Duration: 1 Apr 20125 Apr 2012

Other

Other2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012
CountryUnited States
CityArlington, VA
Period1/4/125/4/12

Fingerprint

Servers
Data storage equipment
Processing

ASJC Scopus subject areas

  • Software

Cite this

Minhas, U. F., Liu, R., Aboulnaga, A., Salem, K., Ng, J., & Robertson, S. (2012). Elastic scale-out for partition-based database systems. In Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012 (pp. 281-288). [6313694] https://doi.org/10.1109/ICDEW.2012.52

Elastic scale-out for partition-based database systems. / Minhas, Umar Farooq; Liu, Rui; Aboulnaga, Ashraf; Salem, Kenneth; Ng, Jonathan; Robertson, Sean.

Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012. 2012. p. 281-288 6313694.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Minhas, UF, Liu, R, Aboulnaga, A, Salem, K, Ng, J & Robertson, S 2012, Elastic scale-out for partition-based database systems. in Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012., 6313694, pp. 281-288, 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012, Arlington, VA, United States, 1/4/12. https://doi.org/10.1109/ICDEW.2012.52
Minhas UF, Liu R, Aboulnaga A, Salem K, Ng J, Robertson S. Elastic scale-out for partition-based database systems. In Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012. 2012. p. 281-288. 6313694 https://doi.org/10.1109/ICDEW.2012.52
Minhas, Umar Farooq ; Liu, Rui ; Aboulnaga, Ashraf ; Salem, Kenneth ; Ng, Jonathan ; Robertson, Sean. / Elastic scale-out for partition-based database systems. Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012. 2012. pp. 281-288
@inproceedings{25e609862e0a40a5a85404dd7cce71fd,
title = "Elastic scale-out for partition-based database systems",
abstract = "An important goal for database systems today is to provide elastic scale-out, i.e., the ability to grow and shrink processing capacity on demand, with varying load. Database systems are difficult to scale since they are stateful - they manage a large database, and it is important when scaling to multiple server machines to provide mechanisms so that these machines can collaboratively manage the database and maintain its consistency. Database partitioning is often used to solve this problem, with each server machine being responsible for one partition. In this paper, we propose that the flexibility provided by a partitioned, shared nothing parallel database system can be exploited to provide elastic scale-out. The idea is to start with a small number of server machines that manage all partitions, and to elastically scale out by dynamically adding new server machines and redistributing database partitions among these servers. We present an implementation of this approach for elastic scale-out using VoltDB - an in-memory, partitioned, shared nothing parallel database system. Our main goal in this paper is to identify several manageability problems that arise when using this approach for elastic scale-out. The paper presents some of these problems and outlines a research agenda for this area.",
author = "Minhas, {Umar Farooq} and Rui Liu and Ashraf Aboulnaga and Kenneth Salem and Jonathan Ng and Sean Robertson",
year = "2012",
month = "11",
day = "19",
doi = "10.1109/ICDEW.2012.52",
language = "English",
isbn = "9780769547480",
pages = "281--288",
booktitle = "Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012",

}

TY - GEN

T1 - Elastic scale-out for partition-based database systems

AU - Minhas, Umar Farooq

AU - Liu, Rui

AU - Aboulnaga, Ashraf

AU - Salem, Kenneth

AU - Ng, Jonathan

AU - Robertson, Sean

PY - 2012/11/19

Y1 - 2012/11/19

N2 - An important goal for database systems today is to provide elastic scale-out, i.e., the ability to grow and shrink processing capacity on demand, with varying load. Database systems are difficult to scale since they are stateful - they manage a large database, and it is important when scaling to multiple server machines to provide mechanisms so that these machines can collaboratively manage the database and maintain its consistency. Database partitioning is often used to solve this problem, with each server machine being responsible for one partition. In this paper, we propose that the flexibility provided by a partitioned, shared nothing parallel database system can be exploited to provide elastic scale-out. The idea is to start with a small number of server machines that manage all partitions, and to elastically scale out by dynamically adding new server machines and redistributing database partitions among these servers. We present an implementation of this approach for elastic scale-out using VoltDB - an in-memory, partitioned, shared nothing parallel database system. Our main goal in this paper is to identify several manageability problems that arise when using this approach for elastic scale-out. The paper presents some of these problems and outlines a research agenda for this area.

AB - An important goal for database systems today is to provide elastic scale-out, i.e., the ability to grow and shrink processing capacity on demand, with varying load. Database systems are difficult to scale since they are stateful - they manage a large database, and it is important when scaling to multiple server machines to provide mechanisms so that these machines can collaboratively manage the database and maintain its consistency. Database partitioning is often used to solve this problem, with each server machine being responsible for one partition. In this paper, we propose that the flexibility provided by a partitioned, shared nothing parallel database system can be exploited to provide elastic scale-out. The idea is to start with a small number of server machines that manage all partitions, and to elastically scale out by dynamically adding new server machines and redistributing database partitions among these servers. We present an implementation of this approach for elastic scale-out using VoltDB - an in-memory, partitioned, shared nothing parallel database system. Our main goal in this paper is to identify several manageability problems that arise when using this approach for elastic scale-out. The paper presents some of these problems and outlines a research agenda for this area.

UR - http://www.scopus.com/inward/record.url?scp=84869064729&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84869064729&partnerID=8YFLogxK

U2 - 10.1109/ICDEW.2012.52

DO - 10.1109/ICDEW.2012.52

M3 - Conference contribution

SN - 9780769547480

SP - 281

EP - 288

BT - Proceedings - 2012 IEEE 28th International Conference on Data Engineering Workshops, ICDEW 2012

ER -