Rank-aware query optimization

Ihab F. Ilyas, Rahul Shah, Walid G. Aref, Jeffrey Scott Vitter, Ahmed Elmagarmid

Research output: Chapter in Book/Report/Conference proceedingConference contribution

79 Citations (Scopus)

Abstract

Ranking is an important property that needs to be fully supported by current relational query engines. Recently, several rank-join query operators have been proposed based on rank aggregation algorithms. Rank-join operators progressively rank the join results while performing the join operation. The new operators have a direct impact on traditional query processing and optimization. We introduce a rank-aware query optimization framework that fully integrates rank-join operators into relational query engines. The framework is based on extending the System R dynamic programming algorithm in both enumeration and pruning. We define ranking as an interesting property that triggers the generation of rank-aware query plans. Unlike traditional join operators, optimizing for rank-join operators depends on estimating the input cardinality of these operators. We introduce a probabilistic model for estimating the input cardinality, and hence the cost of a rank-join operator. To our knowledge, this paper is the first effort in estimating the needed input size for optimal rank aggregation algorithms. Costing ranking plans, although challenging, is key to the full integration of rank-join operators in real-world query processing engines. We experimentally evaluate our framework by modifying the query optimizer of an open-source database management system. The experiments show the validity of our framework and the accuracy of the proposed estimation model.

Original languageEnglish
Title of host publicationProceedings of the ACM SIGMOD International Conference on Management of Data
EditorsG. Weikum, A.C. Konig, S. Dessloch
Pages203-214
Number of pages12
Publication statusPublished - 2004
Externally publishedYes
EventProceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2004 - Paris, France
Duration: 13 Jun 200418 Jun 2004

Other

OtherProceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2004
CountryFrance
CityParis
Period13/6/0418/6/04

Fingerprint

Mathematical operators
Query processing
Engines
Agglomeration
Dynamic programming
Costs
Experiments
Statistical Models

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Ilyas, I. F., Shah, R., Aref, W. G., Vitter, J. S., & Elmagarmid, A. (2004). Rank-aware query optimization. In G. Weikum, A. C. Konig, & S. Dessloch (Eds.), Proceedings of the ACM SIGMOD International Conference on Management of Data (pp. 203-214)

Rank-aware query optimization. / Ilyas, Ihab F.; Shah, Rahul; Aref, Walid G.; Vitter, Jeffrey Scott; Elmagarmid, Ahmed.

Proceedings of the ACM SIGMOD International Conference on Management of Data. ed. / G. Weikum; A.C. Konig; S. Dessloch. 2004. p. 203-214.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ilyas, IF, Shah, R, Aref, WG, Vitter, JS & Elmagarmid, A 2004, Rank-aware query optimization. in G Weikum, AC Konig & S Dessloch (eds), Proceedings of the ACM SIGMOD International Conference on Management of Data. pp. 203-214, Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD 2004, Paris, France, 13/6/04.
Ilyas IF, Shah R, Aref WG, Vitter JS, Elmagarmid A. Rank-aware query optimization. In Weikum G, Konig AC, Dessloch S, editors, Proceedings of the ACM SIGMOD International Conference on Management of Data. 2004. p. 203-214
Ilyas, Ihab F. ; Shah, Rahul ; Aref, Walid G. ; Vitter, Jeffrey Scott ; Elmagarmid, Ahmed. / Rank-aware query optimization. Proceedings of the ACM SIGMOD International Conference on Management of Data. editor / G. Weikum ; A.C. Konig ; S. Dessloch. 2004. pp. 203-214
@inproceedings{92642bdf2c234690a1db7c459fcc1bde,
title = "Rank-aware query optimization",
abstract = "Ranking is an important property that needs to be fully supported by current relational query engines. Recently, several rank-join query operators have been proposed based on rank aggregation algorithms. Rank-join operators progressively rank the join results while performing the join operation. The new operators have a direct impact on traditional query processing and optimization. We introduce a rank-aware query optimization framework that fully integrates rank-join operators into relational query engines. The framework is based on extending the System R dynamic programming algorithm in both enumeration and pruning. We define ranking as an interesting property that triggers the generation of rank-aware query plans. Unlike traditional join operators, optimizing for rank-join operators depends on estimating the input cardinality of these operators. We introduce a probabilistic model for estimating the input cardinality, and hence the cost of a rank-join operator. To our knowledge, this paper is the first effort in estimating the needed input size for optimal rank aggregation algorithms. Costing ranking plans, although challenging, is key to the full integration of rank-join operators in real-world query processing engines. We experimentally evaluate our framework by modifying the query optimizer of an open-source database management system. The experiments show the validity of our framework and the accuracy of the proposed estimation model.",
author = "Ilyas, {Ihab F.} and Rahul Shah and Aref, {Walid G.} and Vitter, {Jeffrey Scott} and Ahmed Elmagarmid",
year = "2004",
language = "English",
pages = "203--214",
editor = "G. Weikum and A.C. Konig and S. Dessloch",
booktitle = "Proceedings of the ACM SIGMOD International Conference on Management of Data",

}

TY - GEN

T1 - Rank-aware query optimization

AU - Ilyas, Ihab F.

AU - Shah, Rahul

AU - Aref, Walid G.

AU - Vitter, Jeffrey Scott

AU - Elmagarmid, Ahmed

PY - 2004

Y1 - 2004

N2 - Ranking is an important property that needs to be fully supported by current relational query engines. Recently, several rank-join query operators have been proposed based on rank aggregation algorithms. Rank-join operators progressively rank the join results while performing the join operation. The new operators have a direct impact on traditional query processing and optimization. We introduce a rank-aware query optimization framework that fully integrates rank-join operators into relational query engines. The framework is based on extending the System R dynamic programming algorithm in both enumeration and pruning. We define ranking as an interesting property that triggers the generation of rank-aware query plans. Unlike traditional join operators, optimizing for rank-join operators depends on estimating the input cardinality of these operators. We introduce a probabilistic model for estimating the input cardinality, and hence the cost of a rank-join operator. To our knowledge, this paper is the first effort in estimating the needed input size for optimal rank aggregation algorithms. Costing ranking plans, although challenging, is key to the full integration of rank-join operators in real-world query processing engines. We experimentally evaluate our framework by modifying the query optimizer of an open-source database management system. The experiments show the validity of our framework and the accuracy of the proposed estimation model.

AB - Ranking is an important property that needs to be fully supported by current relational query engines. Recently, several rank-join query operators have been proposed based on rank aggregation algorithms. Rank-join operators progressively rank the join results while performing the join operation. The new operators have a direct impact on traditional query processing and optimization. We introduce a rank-aware query optimization framework that fully integrates rank-join operators into relational query engines. The framework is based on extending the System R dynamic programming algorithm in both enumeration and pruning. We define ranking as an interesting property that triggers the generation of rank-aware query plans. Unlike traditional join operators, optimizing for rank-join operators depends on estimating the input cardinality of these operators. We introduce a probabilistic model for estimating the input cardinality, and hence the cost of a rank-join operator. To our knowledge, this paper is the first effort in estimating the needed input size for optimal rank aggregation algorithms. Costing ranking plans, although challenging, is key to the full integration of rank-join operators in real-world query processing engines. We experimentally evaluate our framework by modifying the query optimizer of an open-source database management system. The experiments show the validity of our framework and the accuracy of the proposed estimation model.

UR - http://www.scopus.com/inward/record.url?scp=3142724907&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=3142724907&partnerID=8YFLogxK

M3 - Conference contribution

SP - 203

EP - 214

BT - Proceedings of the ACM SIGMOD International Conference on Management of Data

A2 - Weikum, G.

A2 - Konig, A.C.

A2 - Dessloch, S.

ER -