Just-in-time processing of continuous queries

Yin Yang, Dimitas Papadias

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

In a data stream management system, a continuous query is processed by an execution plan consisting of multiple operators connected via the "consumer-producer" relationship, i.e., the output of an operator (the "producer") feeds to another downstream operator (the "consumer") as input. Existing techniques execute each operator separately and push all results to its consumers, without considering whether the consumers need them. Consequently, considerable CPU and memory resources are wasted on producing and storing useless intermediate results. Motivated by this, we propose just-in-time (JIT) processing, a novel methodology that enables a consumer to return feedback expressing its current demand to the producer, The latter selectively generates results based on this information. We show, through extensive experiments, that JIT achieves significant sayings in terms of both CPU time and memory consumption.

Original languageEnglish
Title of host publicationProceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08
Pages1150-1159
Number of pages10
DOIs
Publication statusPublished - 2008
Externally publishedYes
Event2008 IEEE 24th International Conference on Data Engineering, ICDE'08 - Cancun, Mexico
Duration: 7 Apr 200812 Apr 2008

Other

Other2008 IEEE 24th International Conference on Data Engineering, ICDE'08
CountryMexico
CityCancun
Period7/4/0812/4/08

Fingerprint

Program processors
Data storage equipment
Processing
Feedback
Experiments

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Information Systems

Cite this

Yang, Y., & Papadias, D. (2008). Just-in-time processing of continuous queries. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08 (pp. 1150-1159). [4497524] https://doi.org/10.1109/ICDE.2008.4497524

Just-in-time processing of continuous queries. / Yang, Yin; Papadias, Dimitas.

Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. p. 1150-1159 4497524.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Yang, Y & Papadias, D 2008, Just-in-time processing of continuous queries. in Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08., 4497524, pp. 1150-1159, 2008 IEEE 24th International Conference on Data Engineering, ICDE'08, Cancun, Mexico, 7/4/08. https://doi.org/10.1109/ICDE.2008.4497524
Yang Y, Papadias D. Just-in-time processing of continuous queries. In Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. p. 1150-1159. 4497524 https://doi.org/10.1109/ICDE.2008.4497524
Yang, Yin ; Papadias, Dimitas. / Just-in-time processing of continuous queries. Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08. 2008. pp. 1150-1159
@inproceedings{d2562cdf2ce9439d8e1aa63d47358de7,
title = "Just-in-time processing of continuous queries",
abstract = "In a data stream management system, a continuous query is processed by an execution plan consisting of multiple operators connected via the {"}consumer-producer{"} relationship, i.e., the output of an operator (the {"}producer{"}) feeds to another downstream operator (the {"}consumer{"}) as input. Existing techniques execute each operator separately and push all results to its consumers, without considering whether the consumers need them. Consequently, considerable CPU and memory resources are wasted on producing and storing useless intermediate results. Motivated by this, we propose just-in-time (JIT) processing, a novel methodology that enables a consumer to return feedback expressing its current demand to the producer, The latter selectively generates results based on this information. We show, through extensive experiments, that JIT achieves significant sayings in terms of both CPU time and memory consumption.",
author = "Yin Yang and Dimitas Papadias",
year = "2008",
doi = "10.1109/ICDE.2008.4497524",
language = "English",
isbn = "9781424418374",
pages = "1150--1159",
booktitle = "Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08",

}

TY - GEN

T1 - Just-in-time processing of continuous queries

AU - Yang, Yin

AU - Papadias, Dimitas

PY - 2008

Y1 - 2008

N2 - In a data stream management system, a continuous query is processed by an execution plan consisting of multiple operators connected via the "consumer-producer" relationship, i.e., the output of an operator (the "producer") feeds to another downstream operator (the "consumer") as input. Existing techniques execute each operator separately and push all results to its consumers, without considering whether the consumers need them. Consequently, considerable CPU and memory resources are wasted on producing and storing useless intermediate results. Motivated by this, we propose just-in-time (JIT) processing, a novel methodology that enables a consumer to return feedback expressing its current demand to the producer, The latter selectively generates results based on this information. We show, through extensive experiments, that JIT achieves significant sayings in terms of both CPU time and memory consumption.

AB - In a data stream management system, a continuous query is processed by an execution plan consisting of multiple operators connected via the "consumer-producer" relationship, i.e., the output of an operator (the "producer") feeds to another downstream operator (the "consumer") as input. Existing techniques execute each operator separately and push all results to its consumers, without considering whether the consumers need them. Consequently, considerable CPU and memory resources are wasted on producing and storing useless intermediate results. Motivated by this, we propose just-in-time (JIT) processing, a novel methodology that enables a consumer to return feedback expressing its current demand to the producer, The latter selectively generates results based on this information. We show, through extensive experiments, that JIT achieves significant sayings in terms of both CPU time and memory consumption.

UR - http://www.scopus.com/inward/record.url?scp=52649103421&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=52649103421&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2008.4497524

DO - 10.1109/ICDE.2008.4497524

M3 - Conference contribution

SN - 9781424418374

SP - 1150

EP - 1159

BT - Proceedings of the 2008 IEEE 24th International Conference on Data Engineering, ICDE'08

ER -