Parallelizing skyline queries for scalable distribution

Ping Wu, Caijie Zhang, Ying Feng, Ben Y. Zhao, Divyakant Agrawal, Amr El Abbadi

Research output: Chapter in Book/Report/Conference proceedingConference contribution

135 Citations (Scopus)

Abstract

Skyline queries help users make intelligent decisions over complex data, where different and often conflicting criteria are considered. Current skyline computation methods are restricted to centralized query processors, limiting seal-ability and imposing a single point of failure. In this paper, we address the problem of parallelizing skyline query execution over a large number of machines by leveraging content-based data partitioning. We present a novel distributed skyline query processing algorithm (DSL) that discovers skyline points progressively. We propose two mechanisms, recursive region partitioning and dynamic region encoding, to enforce a partial order on query propagation in order to pipeline query execution. Our analysis shows that DSL is optimal in terms of the total number of local query invocations across all machines. In addition, simulations and measurements of a deployed system show that our system load balances communication and processing costs across cluster machines, providing incremental scalability and significant performance improvement over alternative distribution mechanisms.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages112-130
Number of pages19
Volume3896 LNCS
Publication statusPublished - 10 Jul 2006
Externally publishedYes
Event10th International Conference on Extending Database Technology, EDBT 2006 - Munich, Germany
Duration: 26 Mar 200631 Mar 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3896 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other10th International Conference on Extending Database Technology, EDBT 2006
CountryGermany
CityMunich
Period26/3/0631/3/06

Fingerprint

DSL
Skyline
Query
Costs and Cost Analysis
Query processing
Seals
Scalability
Pipelines
Communication
Processing
Data Partitioning
Costs
Load Balance
Query Processing
Partial Order
Partitioning
Encoding
Limiting
Propagation
Alternatives

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Cite this

Wu, P., Zhang, C., Feng, Y., Zhao, B. Y., Agrawal, D., & El Abbadi, A. (2006). Parallelizing skyline queries for scalable distribution. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 3896 LNCS, pp. 112-130). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3896 LNCS).

Parallelizing skyline queries for scalable distribution. / Wu, Ping; Zhang, Caijie; Feng, Ying; Zhao, Ben Y.; Agrawal, Divyakant; El Abbadi, Amr.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 3896 LNCS 2006. p. 112-130 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3896 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wu, P, Zhang, C, Feng, Y, Zhao, BY, Agrawal, D & El Abbadi, A 2006, Parallelizing skyline queries for scalable distribution. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 3896 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3896 LNCS, pp. 112-130, 10th International Conference on Extending Database Technology, EDBT 2006, Munich, Germany, 26/3/06.
Wu P, Zhang C, Feng Y, Zhao BY, Agrawal D, El Abbadi A. Parallelizing skyline queries for scalable distribution. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 3896 LNCS. 2006. p. 112-130. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Wu, Ping ; Zhang, Caijie ; Feng, Ying ; Zhao, Ben Y. ; Agrawal, Divyakant ; El Abbadi, Amr. / Parallelizing skyline queries for scalable distribution. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 3896 LNCS 2006. pp. 112-130 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{b7b3bfd34b654f19a8da83afa9e494cf,
title = "Parallelizing skyline queries for scalable distribution",
abstract = "Skyline queries help users make intelligent decisions over complex data, where different and often conflicting criteria are considered. Current skyline computation methods are restricted to centralized query processors, limiting seal-ability and imposing a single point of failure. In this paper, we address the problem of parallelizing skyline query execution over a large number of machines by leveraging content-based data partitioning. We present a novel distributed skyline query processing algorithm (DSL) that discovers skyline points progressively. We propose two mechanisms, recursive region partitioning and dynamic region encoding, to enforce a partial order on query propagation in order to pipeline query execution. Our analysis shows that DSL is optimal in terms of the total number of local query invocations across all machines. In addition, simulations and measurements of a deployed system show that our system load balances communication and processing costs across cluster machines, providing incremental scalability and significant performance improvement over alternative distribution mechanisms.",
author = "Ping Wu and Caijie Zhang and Ying Feng and Zhao, {Ben Y.} and Divyakant Agrawal and {El Abbadi}, Amr",
year = "2006",
month = "7",
day = "10",
language = "English",
isbn = "3540329609",
volume = "3896 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "112--130",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Parallelizing skyline queries for scalable distribution

AU - Wu, Ping

AU - Zhang, Caijie

AU - Feng, Ying

AU - Zhao, Ben Y.

AU - Agrawal, Divyakant

AU - El Abbadi, Amr

PY - 2006/7/10

Y1 - 2006/7/10

N2 - Skyline queries help users make intelligent decisions over complex data, where different and often conflicting criteria are considered. Current skyline computation methods are restricted to centralized query processors, limiting seal-ability and imposing a single point of failure. In this paper, we address the problem of parallelizing skyline query execution over a large number of machines by leveraging content-based data partitioning. We present a novel distributed skyline query processing algorithm (DSL) that discovers skyline points progressively. We propose two mechanisms, recursive region partitioning and dynamic region encoding, to enforce a partial order on query propagation in order to pipeline query execution. Our analysis shows that DSL is optimal in terms of the total number of local query invocations across all machines. In addition, simulations and measurements of a deployed system show that our system load balances communication and processing costs across cluster machines, providing incremental scalability and significant performance improvement over alternative distribution mechanisms.

AB - Skyline queries help users make intelligent decisions over complex data, where different and often conflicting criteria are considered. Current skyline computation methods are restricted to centralized query processors, limiting seal-ability and imposing a single point of failure. In this paper, we address the problem of parallelizing skyline query execution over a large number of machines by leveraging content-based data partitioning. We present a novel distributed skyline query processing algorithm (DSL) that discovers skyline points progressively. We propose two mechanisms, recursive region partitioning and dynamic region encoding, to enforce a partial order on query propagation in order to pipeline query execution. Our analysis shows that DSL is optimal in terms of the total number of local query invocations across all machines. In addition, simulations and measurements of a deployed system show that our system load balances communication and processing costs across cluster machines, providing incremental scalability and significant performance improvement over alternative distribution mechanisms.

UR - http://www.scopus.com/inward/record.url?scp=33745616079&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745616079&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:33745616079

SN - 3540329609

SN - 9783540329602

VL - 3896 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 112

EP - 130

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -