Zephyr

Live migration in shared nothing databases for elastic cloud platforms

Aaron J. Elmore, Sudipto Das, Divyakant Agrawal, Amr El Abbadi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

111 Citations (Scopus)

Abstract

Multitenant data infrastructures for large cloud platforms hosting hundreds of thousands of applications face the challenge of serving applications characterized by small data footprint and unpredictable load patterns. When such a platform is built on an elastic pay-per-use infrastructure, an added challenge is to minimize the system's operating cost while guaranteeing the tenants' service level agreements (SLA). Elastic load balancing is therefore an important feature to enable scale-up during high load while scaling down when the load is low. Live migration, a technique to migrate tenants with minimal service interruption and no downtime, is critical to allow lightweight elastic scaling. We focus on the problem of live migration in the database layer. We propose Zephyr, a technique to efficiently migrate a live database in a shared nothing transactional database architecture. Zephyr uses phases of on-demand pull and asynchronous push of data, requires minimal synchronization, results no service unavailability and few or no aborted transactions, minimizes the data transfer overhead, provides ACID guarantees during migration, and ensures correctness in the presence of failures. We outline a prototype implementation using an open source relational database engine and an present a thorough evaluation using various transactional workloads. Zephyr's efficiency is evident from the few tens of failed operations, 10-20% change in average transaction latency, minimal messaging, and no overhead during normal operation when migrating a live database.

Original languageEnglish
Title of host publicationProceedings of the ACM SIGMOD International Conference on Management of Data
Pages301-312
Number of pages12
DOIs
Publication statusPublished - 11 Jul 2011
Externally publishedYes
Event2011 ACM SIGMOD and 30th PODS 2011 Conference - Athens, Greece
Duration: 12 Jun 201116 Jun 2011

Other

Other2011 ACM SIGMOD and 30th PODS 2011 Conference
CountryGreece
CityAthens
Period12/6/1116/6/11

Fingerprint

Data transfer
Operating costs
Resource allocation
Synchronization
Engines

Keywords

  • cloud computing
  • database migration
  • elastic data management
  • multitenancy
  • shared nothing architectures

ASJC Scopus subject areas

  • Software
  • Information Systems

Cite this

Elmore, A. J., Das, S., Agrawal, D., & El Abbadi, A. (2011). Zephyr: Live migration in shared nothing databases for elastic cloud platforms. In Proceedings of the ACM SIGMOD International Conference on Management of Data (pp. 301-312) https://doi.org/10.1145/1989323.1989356

Zephyr : Live migration in shared nothing databases for elastic cloud platforms. / Elmore, Aaron J.; Das, Sudipto; Agrawal, Divyakant; El Abbadi, Amr.

Proceedings of the ACM SIGMOD International Conference on Management of Data. 2011. p. 301-312.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Elmore, AJ, Das, S, Agrawal, D & El Abbadi, A 2011, Zephyr: Live migration in shared nothing databases for elastic cloud platforms. in Proceedings of the ACM SIGMOD International Conference on Management of Data. pp. 301-312, 2011 ACM SIGMOD and 30th PODS 2011 Conference, Athens, Greece, 12/6/11. https://doi.org/10.1145/1989323.1989356
Elmore AJ, Das S, Agrawal D, El Abbadi A. Zephyr: Live migration in shared nothing databases for elastic cloud platforms. In Proceedings of the ACM SIGMOD International Conference on Management of Data. 2011. p. 301-312 https://doi.org/10.1145/1989323.1989356
Elmore, Aaron J. ; Das, Sudipto ; Agrawal, Divyakant ; El Abbadi, Amr. / Zephyr : Live migration in shared nothing databases for elastic cloud platforms. Proceedings of the ACM SIGMOD International Conference on Management of Data. 2011. pp. 301-312
@inproceedings{fa63639e464f4d1596cbbdd39fad9789,
title = "Zephyr: Live migration in shared nothing databases for elastic cloud platforms",
abstract = "Multitenant data infrastructures for large cloud platforms hosting hundreds of thousands of applications face the challenge of serving applications characterized by small data footprint and unpredictable load patterns. When such a platform is built on an elastic pay-per-use infrastructure, an added challenge is to minimize the system's operating cost while guaranteeing the tenants' service level agreements (SLA). Elastic load balancing is therefore an important feature to enable scale-up during high load while scaling down when the load is low. Live migration, a technique to migrate tenants with minimal service interruption and no downtime, is critical to allow lightweight elastic scaling. We focus on the problem of live migration in the database layer. We propose Zephyr, a technique to efficiently migrate a live database in a shared nothing transactional database architecture. Zephyr uses phases of on-demand pull and asynchronous push of data, requires minimal synchronization, results no service unavailability and few or no aborted transactions, minimizes the data transfer overhead, provides ACID guarantees during migration, and ensures correctness in the presence of failures. We outline a prototype implementation using an open source relational database engine and an present a thorough evaluation using various transactional workloads. Zephyr's efficiency is evident from the few tens of failed operations, 10-20{\%} change in average transaction latency, minimal messaging, and no overhead during normal operation when migrating a live database.",
keywords = "cloud computing, database migration, elastic data management, multitenancy, shared nothing architectures",
author = "Elmore, {Aaron J.} and Sudipto Das and Divyakant Agrawal and {El Abbadi}, Amr",
year = "2011",
month = "7",
day = "11",
doi = "10.1145/1989323.1989356",
language = "English",
isbn = "9781450306614",
pages = "301--312",
booktitle = "Proceedings of the ACM SIGMOD International Conference on Management of Data",

}

TY - GEN

T1 - Zephyr

T2 - Live migration in shared nothing databases for elastic cloud platforms

AU - Elmore, Aaron J.

AU - Das, Sudipto

AU - Agrawal, Divyakant

AU - El Abbadi, Amr

PY - 2011/7/11

Y1 - 2011/7/11

N2 - Multitenant data infrastructures for large cloud platforms hosting hundreds of thousands of applications face the challenge of serving applications characterized by small data footprint and unpredictable load patterns. When such a platform is built on an elastic pay-per-use infrastructure, an added challenge is to minimize the system's operating cost while guaranteeing the tenants' service level agreements (SLA). Elastic load balancing is therefore an important feature to enable scale-up during high load while scaling down when the load is low. Live migration, a technique to migrate tenants with minimal service interruption and no downtime, is critical to allow lightweight elastic scaling. We focus on the problem of live migration in the database layer. We propose Zephyr, a technique to efficiently migrate a live database in a shared nothing transactional database architecture. Zephyr uses phases of on-demand pull and asynchronous push of data, requires minimal synchronization, results no service unavailability and few or no aborted transactions, minimizes the data transfer overhead, provides ACID guarantees during migration, and ensures correctness in the presence of failures. We outline a prototype implementation using an open source relational database engine and an present a thorough evaluation using various transactional workloads. Zephyr's efficiency is evident from the few tens of failed operations, 10-20% change in average transaction latency, minimal messaging, and no overhead during normal operation when migrating a live database.

AB - Multitenant data infrastructures for large cloud platforms hosting hundreds of thousands of applications face the challenge of serving applications characterized by small data footprint and unpredictable load patterns. When such a platform is built on an elastic pay-per-use infrastructure, an added challenge is to minimize the system's operating cost while guaranteeing the tenants' service level agreements (SLA). Elastic load balancing is therefore an important feature to enable scale-up during high load while scaling down when the load is low. Live migration, a technique to migrate tenants with minimal service interruption and no downtime, is critical to allow lightweight elastic scaling. We focus on the problem of live migration in the database layer. We propose Zephyr, a technique to efficiently migrate a live database in a shared nothing transactional database architecture. Zephyr uses phases of on-demand pull and asynchronous push of data, requires minimal synchronization, results no service unavailability and few or no aborted transactions, minimizes the data transfer overhead, provides ACID guarantees during migration, and ensures correctness in the presence of failures. We outline a prototype implementation using an open source relational database engine and an present a thorough evaluation using various transactional workloads. Zephyr's efficiency is evident from the few tens of failed operations, 10-20% change in average transaction latency, minimal messaging, and no overhead during normal operation when migrating a live database.

KW - cloud computing

KW - database migration

KW - elastic data management

KW - multitenancy

KW - shared nothing architectures

UR - http://www.scopus.com/inward/record.url?scp=79959957556&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79959957556&partnerID=8YFLogxK

U2 - 10.1145/1989323.1989356

DO - 10.1145/1989323.1989356

M3 - Conference contribution

SN - 9781450306614

SP - 301

EP - 312

BT - Proceedings of the ACM SIGMOD International Conference on Management of Data

ER -