E-Store

Fine-grained elastic partitioning for distributed transaction processing systems

Rebecca Taft, Essam Mansour, Marco Serafini, Jennie Duggan, Aaron J. Elmore, Ashraf Aboulnaga, Andrew Pavlo, Michael Stonebraker

Research output: Chapter in Book/Report/Conference proceedingConference contribution

40 Citations (Scopus)

Abstract

On-line transaction processing (OLTP) database management systems (DBMSs) often serve time-varying workloads due to daily, weekly or seasonal fluctuations in demand, or because of rapid growth in demand due to a company's business success. In addition, many OLTP workloads are heavily skewed to "hot" tuples or ranges of tuples. For example, the majority of NYSE volume involves only 40 stocks. To deal with such fluctuations, an OLTP DBMS needs to be elastic; that is, it must be able to expand and contract resources in response to load fluctuations and dynamically balance load as hot tuples vary over time. This paper presents E-Store, an elastic partitioning framework for distributed OLTP DBMSs. It automatically scales resources in response to demand spikes, periodic events, and gradual changes in an application's workload. E-Store addresses localized bottlenecks through a two-tier data placement strategy: cold data is distributed in large chunks, while smaller ranges of hot tuples are assigned explicitly to individual nodes. This is in contrast to traditional single-tier hash and range partitioning strategies. Our experimental evaluation of E-Store shows the viability of our approach and its efficacy under variations in load across a cluster of machines. Compared to single-tier approaches, E-Store improves throughput by up to 130% while reducing latency by 80%.

Original languageEnglish
Title of host publicationProceedings of the VLDB Endowment
PublisherAssociation for Computing Machinery
Pages245-256
Number of pages12
Volume8
Edition3
Publication statusPublished - 1 Nov 2014
Event3rd Workshop on Spatio-Temporal Database Management, STDBM 2006, Co-located with the 32nd International Conference on Very Large Data Bases, VLDB 2006 - Seoul, Korea, Republic of
Duration: 11 Sep 200611 Sep 2006

Other

Other3rd Workshop on Spatio-Temporal Database Management, STDBM 2006, Co-located with the 32nd International Conference on Very Large Data Bases, VLDB 2006
CountryKorea, Republic of
CitySeoul
Period11/9/0611/9/06

Fingerprint

Processing
Industry
Throughput

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

Taft, R., Mansour, E., Serafini, M., Duggan, J., Elmore, A. J., Aboulnaga, A., ... Stonebraker, M. (2014). E-Store: Fine-grained elastic partitioning for distributed transaction processing systems. In Proceedings of the VLDB Endowment (3 ed., Vol. 8, pp. 245-256). Association for Computing Machinery.

E-Store : Fine-grained elastic partitioning for distributed transaction processing systems. / Taft, Rebecca; Mansour, Essam; Serafini, Marco; Duggan, Jennie; Elmore, Aaron J.; Aboulnaga, Ashraf; Pavlo, Andrew; Stonebraker, Michael.

Proceedings of the VLDB Endowment. Vol. 8 3. ed. Association for Computing Machinery, 2014. p. 245-256.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Taft, R, Mansour, E, Serafini, M, Duggan, J, Elmore, AJ, Aboulnaga, A, Pavlo, A & Stonebraker, M 2014, E-Store: Fine-grained elastic partitioning for distributed transaction processing systems. in Proceedings of the VLDB Endowment. 3 edn, vol. 8, Association for Computing Machinery, pp. 245-256, 3rd Workshop on Spatio-Temporal Database Management, STDBM 2006, Co-located with the 32nd International Conference on Very Large Data Bases, VLDB 2006, Seoul, Korea, Republic of, 11/9/06.
Taft R, Mansour E, Serafini M, Duggan J, Elmore AJ, Aboulnaga A et al. E-Store: Fine-grained elastic partitioning for distributed transaction processing systems. In Proceedings of the VLDB Endowment. 3 ed. Vol. 8. Association for Computing Machinery. 2014. p. 245-256
Taft, Rebecca ; Mansour, Essam ; Serafini, Marco ; Duggan, Jennie ; Elmore, Aaron J. ; Aboulnaga, Ashraf ; Pavlo, Andrew ; Stonebraker, Michael. / E-Store : Fine-grained elastic partitioning for distributed transaction processing systems. Proceedings of the VLDB Endowment. Vol. 8 3. ed. Association for Computing Machinery, 2014. pp. 245-256
@inproceedings{9699cc666bc14920937178a17ead9e57,
title = "E-Store: Fine-grained elastic partitioning for distributed transaction processing systems",
abstract = "On-line transaction processing (OLTP) database management systems (DBMSs) often serve time-varying workloads due to daily, weekly or seasonal fluctuations in demand, or because of rapid growth in demand due to a company's business success. In addition, many OLTP workloads are heavily skewed to {"}hot{"} tuples or ranges of tuples. For example, the majority of NYSE volume involves only 40 stocks. To deal with such fluctuations, an OLTP DBMS needs to be elastic; that is, it must be able to expand and contract resources in response to load fluctuations and dynamically balance load as hot tuples vary over time. This paper presents E-Store, an elastic partitioning framework for distributed OLTP DBMSs. It automatically scales resources in response to demand spikes, periodic events, and gradual changes in an application's workload. E-Store addresses localized bottlenecks through a two-tier data placement strategy: cold data is distributed in large chunks, while smaller ranges of hot tuples are assigned explicitly to individual nodes. This is in contrast to traditional single-tier hash and range partitioning strategies. Our experimental evaluation of E-Store shows the viability of our approach and its efficacy under variations in load across a cluster of machines. Compared to single-tier approaches, E-Store improves throughput by up to 130{\%} while reducing latency by 80{\%}.",
author = "Rebecca Taft and Essam Mansour and Marco Serafini and Jennie Duggan and Elmore, {Aaron J.} and Ashraf Aboulnaga and Andrew Pavlo and Michael Stonebraker",
year = "2014",
month = "11",
day = "1",
language = "English",
volume = "8",
pages = "245--256",
booktitle = "Proceedings of the VLDB Endowment",
publisher = "Association for Computing Machinery",
edition = "3",

}

TY - GEN

T1 - E-Store

T2 - Fine-grained elastic partitioning for distributed transaction processing systems

AU - Taft, Rebecca

AU - Mansour, Essam

AU - Serafini, Marco

AU - Duggan, Jennie

AU - Elmore, Aaron J.

AU - Aboulnaga, Ashraf

AU - Pavlo, Andrew

AU - Stonebraker, Michael

PY - 2014/11/1

Y1 - 2014/11/1

N2 - On-line transaction processing (OLTP) database management systems (DBMSs) often serve time-varying workloads due to daily, weekly or seasonal fluctuations in demand, or because of rapid growth in demand due to a company's business success. In addition, many OLTP workloads are heavily skewed to "hot" tuples or ranges of tuples. For example, the majority of NYSE volume involves only 40 stocks. To deal with such fluctuations, an OLTP DBMS needs to be elastic; that is, it must be able to expand and contract resources in response to load fluctuations and dynamically balance load as hot tuples vary over time. This paper presents E-Store, an elastic partitioning framework for distributed OLTP DBMSs. It automatically scales resources in response to demand spikes, periodic events, and gradual changes in an application's workload. E-Store addresses localized bottlenecks through a two-tier data placement strategy: cold data is distributed in large chunks, while smaller ranges of hot tuples are assigned explicitly to individual nodes. This is in contrast to traditional single-tier hash and range partitioning strategies. Our experimental evaluation of E-Store shows the viability of our approach and its efficacy under variations in load across a cluster of machines. Compared to single-tier approaches, E-Store improves throughput by up to 130% while reducing latency by 80%.

AB - On-line transaction processing (OLTP) database management systems (DBMSs) often serve time-varying workloads due to daily, weekly or seasonal fluctuations in demand, or because of rapid growth in demand due to a company's business success. In addition, many OLTP workloads are heavily skewed to "hot" tuples or ranges of tuples. For example, the majority of NYSE volume involves only 40 stocks. To deal with such fluctuations, an OLTP DBMS needs to be elastic; that is, it must be able to expand and contract resources in response to load fluctuations and dynamically balance load as hot tuples vary over time. This paper presents E-Store, an elastic partitioning framework for distributed OLTP DBMSs. It automatically scales resources in response to demand spikes, periodic events, and gradual changes in an application's workload. E-Store addresses localized bottlenecks through a two-tier data placement strategy: cold data is distributed in large chunks, while smaller ranges of hot tuples are assigned explicitly to individual nodes. This is in contrast to traditional single-tier hash and range partitioning strategies. Our experimental evaluation of E-Store shows the viability of our approach and its efficacy under variations in load across a cluster of machines. Compared to single-tier approaches, E-Store improves throughput by up to 130% while reducing latency by 80%.

UR - http://www.scopus.com/inward/record.url?scp=84938093996&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84938093996&partnerID=8YFLogxK

M3 - Conference contribution

VL - 8

SP - 245

EP - 256

BT - Proceedings of the VLDB Endowment

PB - Association for Computing Machinery

ER -