ElasTraS

An elastic, scalable, and self-managing transactional database for the cloud

Sudipto Das, Divyakant Agrawal, Amr El Abbadi

Research output: Contribution to journalArticle

50 Citations (Scopus)

Abstract

A database management system (DBMS) serving a cloud platform must handle large numbers of application databases (or tenants) that are characterized by diverse schemas, varying footprints, and unpredictable load patterns. Scaling out using clusters of commodity servers and sharing resources among tenants (i.e., multi-tenancy) are important features of such systems. Moreover, when deployed on a pay-per-use infrastructure, minimizing the system's operating cost while ensuring good performance is also an important goal. Traditional DBMSs were not designed for such scenarios and hence do not possess the mentioned features critical for DBMSs in the cloud. We present ElasTraS, which combines three design principles to build an elastically-scalable multitenant DBMS for transaction processing workloads. These design principles are gleaned from a careful analysis of the years of research in building scalable key-value stores and decades of research in high performance transaction processing systems. ElasTraS scales to thousands of tenants, effectively consolidates tenants with small footprints while scaling-out large tenants across multiple servers in a cluster. ElasTraS also supports low-latency multistep ACID transactions, is fault-tolerant, self-managing, and highly available to support mission critical applications. ElasTraS leverages Albatross, a low overhead on-demand live database migration technique, for elastic load balancing by adding more servers during high load and consolidating to fewer servers during usage troughs. This elastic scaling minimizes the operating cost and ensures good performance even in the presence of unpredictable changes to the workload. We elucidate the design principles, explain the architecture, describe a prototype implementation, present the detailed design and implementation of Albatross, and experimentally evaluate the implementation using a variety of transaction processing workloads. On a cluster of 20 commodity servers, our prototype serves thousands of tenants and serves more than 1 billion transactions per day while migrating tenant databases with minimal overhead to allow lightweight elastic scaling. Using a cluster of 30 commodity servers, ElasTraS can scale-out a terabyte TPC-C database serving an aggregate throughput of approximately one quarter of a million TPC-C transactions per minute.

Original languageEnglish
Article number5
JournalACM Transactions on Database Systems
Volume38
Issue number1
DOIs
Publication statusPublished - 1 Apr 2013
Externally publishedYes

Fingerprint

Servers
Operating costs
Processing
Resource allocation
Throughput

Keywords

  • ACID
  • Cloud computing
  • Elastic data management
  • Fault-tolerance
  • Scalability
  • Transactions

ASJC Scopus subject areas

  • Information Systems

Cite this

ElasTraS : An elastic, scalable, and self-managing transactional database for the cloud. / Das, Sudipto; Agrawal, Divyakant; El Abbadi, Amr.

In: ACM Transactions on Database Systems, Vol. 38, No. 1, 5, 01.04.2013.

Research output: Contribution to journalArticle

Das, Sudipto ; Agrawal, Divyakant ; El Abbadi, Amr. / ElasTraS : An elastic, scalable, and self-managing transactional database for the cloud. In: ACM Transactions on Database Systems. 2013 ; Vol. 38, No. 1.
@article{35abe7f064034e9bad153c290cc035e3,
title = "ElasTraS: An elastic, scalable, and self-managing transactional database for the cloud",
abstract = "A database management system (DBMS) serving a cloud platform must handle large numbers of application databases (or tenants) that are characterized by diverse schemas, varying footprints, and unpredictable load patterns. Scaling out using clusters of commodity servers and sharing resources among tenants (i.e., multi-tenancy) are important features of such systems. Moreover, when deployed on a pay-per-use infrastructure, minimizing the system's operating cost while ensuring good performance is also an important goal. Traditional DBMSs were not designed for such scenarios and hence do not possess the mentioned features critical for DBMSs in the cloud. We present ElasTraS, which combines three design principles to build an elastically-scalable multitenant DBMS for transaction processing workloads. These design principles are gleaned from a careful analysis of the years of research in building scalable key-value stores and decades of research in high performance transaction processing systems. ElasTraS scales to thousands of tenants, effectively consolidates tenants with small footprints while scaling-out large tenants across multiple servers in a cluster. ElasTraS also supports low-latency multistep ACID transactions, is fault-tolerant, self-managing, and highly available to support mission critical applications. ElasTraS leverages Albatross, a low overhead on-demand live database migration technique, for elastic load balancing by adding more servers during high load and consolidating to fewer servers during usage troughs. This elastic scaling minimizes the operating cost and ensures good performance even in the presence of unpredictable changes to the workload. We elucidate the design principles, explain the architecture, describe a prototype implementation, present the detailed design and implementation of Albatross, and experimentally evaluate the implementation using a variety of transaction processing workloads. On a cluster of 20 commodity servers, our prototype serves thousands of tenants and serves more than 1 billion transactions per day while migrating tenant databases with minimal overhead to allow lightweight elastic scaling. Using a cluster of 30 commodity servers, ElasTraS can scale-out a terabyte TPC-C database serving an aggregate throughput of approximately one quarter of a million TPC-C transactions per minute.",
keywords = "ACID, Cloud computing, Elastic data management, Fault-tolerance, Scalability, Transactions",
author = "Sudipto Das and Divyakant Agrawal and {El Abbadi}, Amr",
year = "2013",
month = "4",
day = "1",
doi = "10.1145/2445583.2445588",
language = "English",
volume = "38",
journal = "ACM Transactions on Database Systems",
issn = "0362-5915",
publisher = "Association for Computing Machinery (ACM)",
number = "1",

}

TY - JOUR

T1 - ElasTraS

T2 - An elastic, scalable, and self-managing transactional database for the cloud

AU - Das, Sudipto

AU - Agrawal, Divyakant

AU - El Abbadi, Amr

PY - 2013/4/1

Y1 - 2013/4/1

N2 - A database management system (DBMS) serving a cloud platform must handle large numbers of application databases (or tenants) that are characterized by diverse schemas, varying footprints, and unpredictable load patterns. Scaling out using clusters of commodity servers and sharing resources among tenants (i.e., multi-tenancy) are important features of such systems. Moreover, when deployed on a pay-per-use infrastructure, minimizing the system's operating cost while ensuring good performance is also an important goal. Traditional DBMSs were not designed for such scenarios and hence do not possess the mentioned features critical for DBMSs in the cloud. We present ElasTraS, which combines three design principles to build an elastically-scalable multitenant DBMS for transaction processing workloads. These design principles are gleaned from a careful analysis of the years of research in building scalable key-value stores and decades of research in high performance transaction processing systems. ElasTraS scales to thousands of tenants, effectively consolidates tenants with small footprints while scaling-out large tenants across multiple servers in a cluster. ElasTraS also supports low-latency multistep ACID transactions, is fault-tolerant, self-managing, and highly available to support mission critical applications. ElasTraS leverages Albatross, a low overhead on-demand live database migration technique, for elastic load balancing by adding more servers during high load and consolidating to fewer servers during usage troughs. This elastic scaling minimizes the operating cost and ensures good performance even in the presence of unpredictable changes to the workload. We elucidate the design principles, explain the architecture, describe a prototype implementation, present the detailed design and implementation of Albatross, and experimentally evaluate the implementation using a variety of transaction processing workloads. On a cluster of 20 commodity servers, our prototype serves thousands of tenants and serves more than 1 billion transactions per day while migrating tenant databases with minimal overhead to allow lightweight elastic scaling. Using a cluster of 30 commodity servers, ElasTraS can scale-out a terabyte TPC-C database serving an aggregate throughput of approximately one quarter of a million TPC-C transactions per minute.

AB - A database management system (DBMS) serving a cloud platform must handle large numbers of application databases (or tenants) that are characterized by diverse schemas, varying footprints, and unpredictable load patterns. Scaling out using clusters of commodity servers and sharing resources among tenants (i.e., multi-tenancy) are important features of such systems. Moreover, when deployed on a pay-per-use infrastructure, minimizing the system's operating cost while ensuring good performance is also an important goal. Traditional DBMSs were not designed for such scenarios and hence do not possess the mentioned features critical for DBMSs in the cloud. We present ElasTraS, which combines three design principles to build an elastically-scalable multitenant DBMS for transaction processing workloads. These design principles are gleaned from a careful analysis of the years of research in building scalable key-value stores and decades of research in high performance transaction processing systems. ElasTraS scales to thousands of tenants, effectively consolidates tenants with small footprints while scaling-out large tenants across multiple servers in a cluster. ElasTraS also supports low-latency multistep ACID transactions, is fault-tolerant, self-managing, and highly available to support mission critical applications. ElasTraS leverages Albatross, a low overhead on-demand live database migration technique, for elastic load balancing by adding more servers during high load and consolidating to fewer servers during usage troughs. This elastic scaling minimizes the operating cost and ensures good performance even in the presence of unpredictable changes to the workload. We elucidate the design principles, explain the architecture, describe a prototype implementation, present the detailed design and implementation of Albatross, and experimentally evaluate the implementation using a variety of transaction processing workloads. On a cluster of 20 commodity servers, our prototype serves thousands of tenants and serves more than 1 billion transactions per day while migrating tenant databases with minimal overhead to allow lightweight elastic scaling. Using a cluster of 30 commodity servers, ElasTraS can scale-out a terabyte TPC-C database serving an aggregate throughput of approximately one quarter of a million TPC-C transactions per minute.

KW - ACID

KW - Cloud computing

KW - Elastic data management

KW - Fault-tolerance

KW - Scalability

KW - Transactions

UR - http://www.scopus.com/inward/record.url?scp=84878271760&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84878271760&partnerID=8YFLogxK

U2 - 10.1145/2445583.2445588

DO - 10.1145/2445583.2445588

M3 - Article

VL - 38

JO - ACM Transactions on Database Systems

JF - ACM Transactions on Database Systems

SN - 0362-5915

IS - 1

M1 - 5

ER -