Interaction-aware scheduling of report-generation workloads

Mumtaz Ahmad, Ashraf Aboulnaga, Shivnath Babu, Kamesh Munagala

Research output: Contribution to journalArticle

29 Citations (Scopus)

Abstract

The typical workload in a database system consists of a mix of multiple queries of different types that run concurrently. Interactions among the different queries in a query mix can have a significant impact on database performance. Hence, optimizing database performance requires reasoning about query mixes rather than considering queries individually. Current database systems lack the ability to do such reasoning. We propose a new approach based on planning experiments and statistical modeling to capture the impact of query interactions. Our approach requires no prior assumptions about the internal workings of the database system or the nature and cause of query interactions, making it portable across systems. To demonstrate the potential of modeling and exploiting query interactions, we have developed a novel interaction-aware query scheduler for report-generation workloads. Our scheduler, called QShuffler, uses two query scheduling algorithms that leverage models of query interactions. The first algorithm is optimized for workloads where queries are submitted in large batches. The second algorithm targets workloads where queries arrive continuously, and scheduling decisions have to be made online. We report an experimental evaluation of QShuffler using TPC-H workloads running on IBM DB2. The evaluation shows that QShuffler, by modeling and exploiting query interactions, can consistently outperform (up to 4x) query schedulers in current database systems.

Original languageEnglish
Pages (from-to)589-615
Number of pages27
JournalVLDB Journal
Volume20
Issue number4
DOIs
Publication statusPublished - 1 Aug 2011
Externally publishedYes

Fingerprint

Scheduling
Scheduling algorithms
Planning
Experiments

Keywords

  • Business intelligence
  • Experiment-driven performance modeling
  • Query interactions
  • Report generation
  • Scheduling
  • Workload management

ASJC Scopus subject areas

  • Hardware and Architecture
  • Information Systems

Cite this

Interaction-aware scheduling of report-generation workloads. / Ahmad, Mumtaz; Aboulnaga, Ashraf; Babu, Shivnath; Munagala, Kamesh.

In: VLDB Journal, Vol. 20, No. 4, 01.08.2011, p. 589-615.

Research output: Contribution to journalArticle

Ahmad, Mumtaz ; Aboulnaga, Ashraf ; Babu, Shivnath ; Munagala, Kamesh. / Interaction-aware scheduling of report-generation workloads. In: VLDB Journal. 2011 ; Vol. 20, No. 4. pp. 589-615.
@article{09e5d0609961434d98e671e5f55d93d1,
title = "Interaction-aware scheduling of report-generation workloads",
abstract = "The typical workload in a database system consists of a mix of multiple queries of different types that run concurrently. Interactions among the different queries in a query mix can have a significant impact on database performance. Hence, optimizing database performance requires reasoning about query mixes rather than considering queries individually. Current database systems lack the ability to do such reasoning. We propose a new approach based on planning experiments and statistical modeling to capture the impact of query interactions. Our approach requires no prior assumptions about the internal workings of the database system or the nature and cause of query interactions, making it portable across systems. To demonstrate the potential of modeling and exploiting query interactions, we have developed a novel interaction-aware query scheduler for report-generation workloads. Our scheduler, called QShuffler, uses two query scheduling algorithms that leverage models of query interactions. The first algorithm is optimized for workloads where queries are submitted in large batches. The second algorithm targets workloads where queries arrive continuously, and scheduling decisions have to be made online. We report an experimental evaluation of QShuffler using TPC-H workloads running on IBM DB2. The evaluation shows that QShuffler, by modeling and exploiting query interactions, can consistently outperform (up to 4x) query schedulers in current database systems.",
keywords = "Business intelligence, Experiment-driven performance modeling, Query interactions, Report generation, Scheduling, Workload management",
author = "Mumtaz Ahmad and Ashraf Aboulnaga and Shivnath Babu and Kamesh Munagala",
year = "2011",
month = "8",
day = "1",
doi = "10.1007/s00778-011-0217-y",
language = "English",
volume = "20",
pages = "589--615",
journal = "VLDB Journal",
issn = "1066-8888",
publisher = "Springer New York",
number = "4",

}

TY - JOUR

T1 - Interaction-aware scheduling of report-generation workloads

AU - Ahmad, Mumtaz

AU - Aboulnaga, Ashraf

AU - Babu, Shivnath

AU - Munagala, Kamesh

PY - 2011/8/1

Y1 - 2011/8/1

N2 - The typical workload in a database system consists of a mix of multiple queries of different types that run concurrently. Interactions among the different queries in a query mix can have a significant impact on database performance. Hence, optimizing database performance requires reasoning about query mixes rather than considering queries individually. Current database systems lack the ability to do such reasoning. We propose a new approach based on planning experiments and statistical modeling to capture the impact of query interactions. Our approach requires no prior assumptions about the internal workings of the database system or the nature and cause of query interactions, making it portable across systems. To demonstrate the potential of modeling and exploiting query interactions, we have developed a novel interaction-aware query scheduler for report-generation workloads. Our scheduler, called QShuffler, uses two query scheduling algorithms that leverage models of query interactions. The first algorithm is optimized for workloads where queries are submitted in large batches. The second algorithm targets workloads where queries arrive continuously, and scheduling decisions have to be made online. We report an experimental evaluation of QShuffler using TPC-H workloads running on IBM DB2. The evaluation shows that QShuffler, by modeling and exploiting query interactions, can consistently outperform (up to 4x) query schedulers in current database systems.

AB - The typical workload in a database system consists of a mix of multiple queries of different types that run concurrently. Interactions among the different queries in a query mix can have a significant impact on database performance. Hence, optimizing database performance requires reasoning about query mixes rather than considering queries individually. Current database systems lack the ability to do such reasoning. We propose a new approach based on planning experiments and statistical modeling to capture the impact of query interactions. Our approach requires no prior assumptions about the internal workings of the database system or the nature and cause of query interactions, making it portable across systems. To demonstrate the potential of modeling and exploiting query interactions, we have developed a novel interaction-aware query scheduler for report-generation workloads. Our scheduler, called QShuffler, uses two query scheduling algorithms that leverage models of query interactions. The first algorithm is optimized for workloads where queries are submitted in large batches. The second algorithm targets workloads where queries arrive continuously, and scheduling decisions have to be made online. We report an experimental evaluation of QShuffler using TPC-H workloads running on IBM DB2. The evaluation shows that QShuffler, by modeling and exploiting query interactions, can consistently outperform (up to 4x) query schedulers in current database systems.

KW - Business intelligence

KW - Experiment-driven performance modeling

KW - Query interactions

KW - Report generation

KW - Scheduling

KW - Workload management

UR - http://www.scopus.com/inward/record.url?scp=79960451994&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79960451994&partnerID=8YFLogxK

U2 - 10.1007/s00778-011-0217-y

DO - 10.1007/s00778-011-0217-y

M3 - Article

AN - SCOPUS:79960451994

VL - 20

SP - 589

EP - 615

JO - VLDB Journal

JF - VLDB Journal

SN - 1066-8888

IS - 4

ER -