SIMD-Scan

Ultra fast in-memory table scan using on-chip vector processing units

Thomas Willhalm, Nicolae Popovici, Yazan Boshmaf, Hasso Plattner, Alexander Zeier, Jan Schaffner

Research output: Contribution to journalArticle

113 Citations (Scopus)

Abstract

The availability of huge system memory, even on standard servers, generated a lot of interest in main memory database engines. In data warehouse systems, highly compressed columnoriented data structures are quite prominent. In order to scale with the data volume and the system load, many of these systems are highly distributed with a shared-nothing approach. The fundamental principle of all systems is a full table scan over one or multiple compressed columns. Recent research proposed different techniques to speedup table scans like intelligent compression or using an additional hardware such as graphic cards or FPGAs. In this paper, we show that utilizing the embedded Vector Processing Units (VPUs) found in standard superscalar processors can speed up the performance of mainmemory full table scan by factors. This is achieved without changing the hardware architecture and thereby without additional power consumption. Moreover, as on-chip VPUs directly access the system's RAM, no additional costly copy operations are needed for using the new SIMD-scan approach in standard main memory database engines. Therefore, we propose this scan approach to be used as the standard scan operator for compressed column-oriented main memory storage. We then discuss how well our solution scales with the number of processor cores; consequently, to what degree it can be applied in multi-threaded environments. To verify the feasibility of our approach, we implemented the proposed techniques on a modern Intel multicore processor using Intel® Streaming SIMD Extensions 1 (Intel®SSE). In addition, we integrated the new SIMD-scan approach into SAP® Netweaver® Business Warehouse Accelerator2. We conclude with describing the performance benefits of using our approach for processing and scanning compressed data using VPUs in column-oriented main memory database systems.

Original languageEnglish
Pages (from-to)385-394
Number of pages10
JournalProceedings of the VLDB Endowment
Volume2
Issue number1
Publication statusPublished - 2009
Externally publishedYes

Fingerprint

Data storage equipment
Processing
Engines
Hardware
Data warehouses
Warehouses
Random access storage
Data structures
Field programmable gate arrays (FPGA)
Computer systems
Electric power utilization
Servers
Availability
Scanning
Industry

ASJC Scopus subject areas

  • Computer Science (miscellaneous)
  • Computer Science(all)

Cite this

Willhalm, T., Popovici, N., Boshmaf, Y., Plattner, H., Zeier, A., & Schaffner, J. (2009). SIMD-Scan: Ultra fast in-memory table scan using on-chip vector processing units. Proceedings of the VLDB Endowment, 2(1), 385-394.

SIMD-Scan : Ultra fast in-memory table scan using on-chip vector processing units. / Willhalm, Thomas; Popovici, Nicolae; Boshmaf, Yazan; Plattner, Hasso; Zeier, Alexander; Schaffner, Jan.

In: Proceedings of the VLDB Endowment, Vol. 2, No. 1, 2009, p. 385-394.

Research output: Contribution to journalArticle

Willhalm, T, Popovici, N, Boshmaf, Y, Plattner, H, Zeier, A & Schaffner, J 2009, 'SIMD-Scan: Ultra fast in-memory table scan using on-chip vector processing units', Proceedings of the VLDB Endowment, vol. 2, no. 1, pp. 385-394.
Willhalm, Thomas ; Popovici, Nicolae ; Boshmaf, Yazan ; Plattner, Hasso ; Zeier, Alexander ; Schaffner, Jan. / SIMD-Scan : Ultra fast in-memory table scan using on-chip vector processing units. In: Proceedings of the VLDB Endowment. 2009 ; Vol. 2, No. 1. pp. 385-394.
@article{08999df61e3041e49464df23f7558ac4,
title = "SIMD-Scan: Ultra fast in-memory table scan using on-chip vector processing units",
abstract = "The availability of huge system memory, even on standard servers, generated a lot of interest in main memory database engines. In data warehouse systems, highly compressed columnoriented data structures are quite prominent. In order to scale with the data volume and the system load, many of these systems are highly distributed with a shared-nothing approach. The fundamental principle of all systems is a full table scan over one or multiple compressed columns. Recent research proposed different techniques to speedup table scans like intelligent compression or using an additional hardware such as graphic cards or FPGAs. In this paper, we show that utilizing the embedded Vector Processing Units (VPUs) found in standard superscalar processors can speed up the performance of mainmemory full table scan by factors. This is achieved without changing the hardware architecture and thereby without additional power consumption. Moreover, as on-chip VPUs directly access the system's RAM, no additional costly copy operations are needed for using the new SIMD-scan approach in standard main memory database engines. Therefore, we propose this scan approach to be used as the standard scan operator for compressed column-oriented main memory storage. We then discuss how well our solution scales with the number of processor cores; consequently, to what degree it can be applied in multi-threaded environments. To verify the feasibility of our approach, we implemented the proposed techniques on a modern Intel multicore processor using Intel{\circledR} Streaming SIMD Extensions 1 (Intel{\circledR}SSE). In addition, we integrated the new SIMD-scan approach into SAP{\circledR} Netweaver{\circledR} Business Warehouse Accelerator2. We conclude with describing the performance benefits of using our approach for processing and scanning compressed data using VPUs in column-oriented main memory database systems.",
author = "Thomas Willhalm and Nicolae Popovici and Yazan Boshmaf and Hasso Plattner and Alexander Zeier and Jan Schaffner",
year = "2009",
language = "English",
volume = "2",
pages = "385--394",
journal = "Proceedings of the VLDB Endowment",
issn = "2150-8097",
publisher = "Very Large Data Base Endowment Inc.",
number = "1",

}

TY - JOUR

T1 - SIMD-Scan

T2 - Ultra fast in-memory table scan using on-chip vector processing units

AU - Willhalm, Thomas

AU - Popovici, Nicolae

AU - Boshmaf, Yazan

AU - Plattner, Hasso

AU - Zeier, Alexander

AU - Schaffner, Jan

PY - 2009

Y1 - 2009

N2 - The availability of huge system memory, even on standard servers, generated a lot of interest in main memory database engines. In data warehouse systems, highly compressed columnoriented data structures are quite prominent. In order to scale with the data volume and the system load, many of these systems are highly distributed with a shared-nothing approach. The fundamental principle of all systems is a full table scan over one or multiple compressed columns. Recent research proposed different techniques to speedup table scans like intelligent compression or using an additional hardware such as graphic cards or FPGAs. In this paper, we show that utilizing the embedded Vector Processing Units (VPUs) found in standard superscalar processors can speed up the performance of mainmemory full table scan by factors. This is achieved without changing the hardware architecture and thereby without additional power consumption. Moreover, as on-chip VPUs directly access the system's RAM, no additional costly copy operations are needed for using the new SIMD-scan approach in standard main memory database engines. Therefore, we propose this scan approach to be used as the standard scan operator for compressed column-oriented main memory storage. We then discuss how well our solution scales with the number of processor cores; consequently, to what degree it can be applied in multi-threaded environments. To verify the feasibility of our approach, we implemented the proposed techniques on a modern Intel multicore processor using Intel® Streaming SIMD Extensions 1 (Intel®SSE). In addition, we integrated the new SIMD-scan approach into SAP® Netweaver® Business Warehouse Accelerator2. We conclude with describing the performance benefits of using our approach for processing and scanning compressed data using VPUs in column-oriented main memory database systems.

AB - The availability of huge system memory, even on standard servers, generated a lot of interest in main memory database engines. In data warehouse systems, highly compressed columnoriented data structures are quite prominent. In order to scale with the data volume and the system load, many of these systems are highly distributed with a shared-nothing approach. The fundamental principle of all systems is a full table scan over one or multiple compressed columns. Recent research proposed different techniques to speedup table scans like intelligent compression or using an additional hardware such as graphic cards or FPGAs. In this paper, we show that utilizing the embedded Vector Processing Units (VPUs) found in standard superscalar processors can speed up the performance of mainmemory full table scan by factors. This is achieved without changing the hardware architecture and thereby without additional power consumption. Moreover, as on-chip VPUs directly access the system's RAM, no additional costly copy operations are needed for using the new SIMD-scan approach in standard main memory database engines. Therefore, we propose this scan approach to be used as the standard scan operator for compressed column-oriented main memory storage. We then discuss how well our solution scales with the number of processor cores; consequently, to what degree it can be applied in multi-threaded environments. To verify the feasibility of our approach, we implemented the proposed techniques on a modern Intel multicore processor using Intel® Streaming SIMD Extensions 1 (Intel®SSE). In addition, we integrated the new SIMD-scan approach into SAP® Netweaver® Business Warehouse Accelerator2. We conclude with describing the performance benefits of using our approach for processing and scanning compressed data using VPUs in column-oriented main memory database systems.

UR - http://www.scopus.com/inward/record.url?scp=84855233023&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84855233023&partnerID=8YFLogxK

M3 - Article

VL - 2

SP - 385

EP - 394

JO - Proceedings of the VLDB Endowment

JF - Proceedings of the VLDB Endowment

SN - 2150-8097

IS - 1

ER -