Efficient Parallel Skyline Query Processing for High-Dimensional Data

Tang Mingjie, Yongyang Yu, Walid G. Aref, Qutaibah Malluhi, Mourad Ouzzani

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Given a set of multidimensional data points, skyline queries retrieve those points that are not dominated by any other points in the set. Due to the ubiquitous use of skyline queries, there are several outstanding challenges that have not been well addressed. More specifically, in this paper, we are tackling the data straggler and data skew challenges introduced by distributed skyline query processing as well as the ensuing high computing cost of merging skyline candidates. We thus introduce a new efficient three-phase approach for large scale processing of skyline queries. In the first preprocessing phase, the data is partitioned along the Z-order curve. We utilize a novel data partitioning approach that formulates data partitioning as an optimization problem to minimize the size of intermediate data. In the second phase, each computation node partitions the input data points into separate sets, and then performs the skyline computation on each set to produce skyline candidates in parallel. In the final phase, we build an index and employ an efficient algorithm to merge the generated skyline candidates. Extensive experiments demonstrate that the proposed skyline algorithm achieves more than one order of magnitude enhancement in performance compared to existing state-of-the-art approaches.

Original languageEnglish
JournalIEEE Transactions on Knowledge and Data Engineering
DOIs
Publication statusAccepted/In press - 23 Feb 2018

    Fingerprint

Keywords

  • Computer science
  • Distributed databases
  • high-dimensional data
  • Indexes
  • Merging
  • parallel computing
  • Partitioning algorithms
  • query processing
  • Query processing
  • skyline query
  • Task analysis

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Cite this