Exploiting single-threaded model in multi-core in-memory systems

Chang Yao, Divyakant Agrawal, Gang Chen, Qian Lin, Beng Chin Ooi, Weng Fai Wong, Meihui Zhang

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

The widely adopted single-threaded OLTP model assigns a single thread to each static partition of the database for processing transactions in a partition. This simplifies concurrency control while retaining parallelism. However, it suffers performance loss arising from skewed workloads as well as transactions that span multiple partitions. In this paper, we present a dynamic single-threaded in-memory OLTP system, called LADS, that extends the simplicity of the single-threaded model. The key innovation in LADS is the separation of dependency resolution and execution into two non-overlapping phases for batches of transactions. After the first phase of dependency resolution, the record actions of the transactions are partitioned and ordered. Each independent partition is then executed sequentially by a single thread, avoiding the need for locking. By careful mapping of the tasks to be performed to threads, LADS is able to achieve a high degree of balanced parallelism. We evaluate LADS against H-Store, a partition-based database; DORA, a data-oriented transaction processing system; and SILO, a multi-core in-memory OLTP engine. The experimental study shows that LADS achieves up to 20x higher throughput than existing systems and exhibits better robustness with various workloads.

Original languageEnglish
Article number7486988
Pages (from-to)2635-2650
Number of pages16
JournalIEEE Transactions on Knowledge and Data Engineering
Volume28
Issue number10
DOIs
Publication statusPublished - 1 Oct 2016
Externally publishedYes

Fingerprint

Data storage equipment
Concurrency control
Processing
Innovation
Throughput
Engines

Keywords

  • concurrency control
  • in-memory OLTP system
  • multi-core
  • single-threaded model
  • Transaction management

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Cite this

Yao, C., Agrawal, D., Chen, G., Lin, Q., Ooi, B. C., Wong, W. F., & Zhang, M. (2016). Exploiting single-threaded model in multi-core in-memory systems. IEEE Transactions on Knowledge and Data Engineering, 28(10), 2635-2650. [7486988]. https://doi.org/10.1109/TKDE.2016.2578319

Exploiting single-threaded model in multi-core in-memory systems. / Yao, Chang; Agrawal, Divyakant; Chen, Gang; Lin, Qian; Ooi, Beng Chin; Wong, Weng Fai; Zhang, Meihui.

In: IEEE Transactions on Knowledge and Data Engineering, Vol. 28, No. 10, 7486988, 01.10.2016, p. 2635-2650.

Research output: Contribution to journalArticle

Yao, C, Agrawal, D, Chen, G, Lin, Q, Ooi, BC, Wong, WF & Zhang, M 2016, 'Exploiting single-threaded model in multi-core in-memory systems', IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 10, 7486988, pp. 2635-2650. https://doi.org/10.1109/TKDE.2016.2578319
Yao, Chang ; Agrawal, Divyakant ; Chen, Gang ; Lin, Qian ; Ooi, Beng Chin ; Wong, Weng Fai ; Zhang, Meihui. / Exploiting single-threaded model in multi-core in-memory systems. In: IEEE Transactions on Knowledge and Data Engineering. 2016 ; Vol. 28, No. 10. pp. 2635-2650.
@article{e5ef3f0af7414cb7a7fad87c5508db6e,
title = "Exploiting single-threaded model in multi-core in-memory systems",
abstract = "The widely adopted single-threaded OLTP model assigns a single thread to each static partition of the database for processing transactions in a partition. This simplifies concurrency control while retaining parallelism. However, it suffers performance loss arising from skewed workloads as well as transactions that span multiple partitions. In this paper, we present a dynamic single-threaded in-memory OLTP system, called LADS, that extends the simplicity of the single-threaded model. The key innovation in LADS is the separation of dependency resolution and execution into two non-overlapping phases for batches of transactions. After the first phase of dependency resolution, the record actions of the transactions are partitioned and ordered. Each independent partition is then executed sequentially by a single thread, avoiding the need for locking. By careful mapping of the tasks to be performed to threads, LADS is able to achieve a high degree of balanced parallelism. We evaluate LADS against H-Store, a partition-based database; DORA, a data-oriented transaction processing system; and SILO, a multi-core in-memory OLTP engine. The experimental study shows that LADS achieves up to 20x higher throughput than existing systems and exhibits better robustness with various workloads.",
keywords = "concurrency control, in-memory OLTP system, multi-core, single-threaded model, Transaction management",
author = "Chang Yao and Divyakant Agrawal and Gang Chen and Qian Lin and Ooi, {Beng Chin} and Wong, {Weng Fai} and Meihui Zhang",
year = "2016",
month = "10",
day = "1",
doi = "10.1109/TKDE.2016.2578319",
language = "English",
volume = "28",
pages = "2635--2650",
journal = "IEEE Transactions on Knowledge and Data Engineering",
issn = "1041-4347",
publisher = "IEEE Computer Society",
number = "10",

}

TY - JOUR

T1 - Exploiting single-threaded model in multi-core in-memory systems

AU - Yao, Chang

AU - Agrawal, Divyakant

AU - Chen, Gang

AU - Lin, Qian

AU - Ooi, Beng Chin

AU - Wong, Weng Fai

AU - Zhang, Meihui

PY - 2016/10/1

Y1 - 2016/10/1

N2 - The widely adopted single-threaded OLTP model assigns a single thread to each static partition of the database for processing transactions in a partition. This simplifies concurrency control while retaining parallelism. However, it suffers performance loss arising from skewed workloads as well as transactions that span multiple partitions. In this paper, we present a dynamic single-threaded in-memory OLTP system, called LADS, that extends the simplicity of the single-threaded model. The key innovation in LADS is the separation of dependency resolution and execution into two non-overlapping phases for batches of transactions. After the first phase of dependency resolution, the record actions of the transactions are partitioned and ordered. Each independent partition is then executed sequentially by a single thread, avoiding the need for locking. By careful mapping of the tasks to be performed to threads, LADS is able to achieve a high degree of balanced parallelism. We evaluate LADS against H-Store, a partition-based database; DORA, a data-oriented transaction processing system; and SILO, a multi-core in-memory OLTP engine. The experimental study shows that LADS achieves up to 20x higher throughput than existing systems and exhibits better robustness with various workloads.

AB - The widely adopted single-threaded OLTP model assigns a single thread to each static partition of the database for processing transactions in a partition. This simplifies concurrency control while retaining parallelism. However, it suffers performance loss arising from skewed workloads as well as transactions that span multiple partitions. In this paper, we present a dynamic single-threaded in-memory OLTP system, called LADS, that extends the simplicity of the single-threaded model. The key innovation in LADS is the separation of dependency resolution and execution into two non-overlapping phases for batches of transactions. After the first phase of dependency resolution, the record actions of the transactions are partitioned and ordered. Each independent partition is then executed sequentially by a single thread, avoiding the need for locking. By careful mapping of the tasks to be performed to threads, LADS is able to achieve a high degree of balanced parallelism. We evaluate LADS against H-Store, a partition-based database; DORA, a data-oriented transaction processing system; and SILO, a multi-core in-memory OLTP engine. The experimental study shows that LADS achieves up to 20x higher throughput than existing systems and exhibits better robustness with various workloads.

KW - concurrency control

KW - in-memory OLTP system

KW - multi-core

KW - single-threaded model

KW - Transaction management

UR - http://www.scopus.com/inward/record.url?scp=84990909338&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84990909338&partnerID=8YFLogxK

U2 - 10.1109/TKDE.2016.2578319

DO - 10.1109/TKDE.2016.2578319

M3 - Article

VL - 28

SP - 2635

EP - 2650

JO - IEEE Transactions on Knowledge and Data Engineering

JF - IEEE Transactions on Knowledge and Data Engineering

SN - 1041-4347

IS - 10

M1 - 7486988

ER -