Towards scalable architectures for clickstream data warehousing

Peter Alvaro, Dmitriy V. Ryaboy, Divyakant Agrawal

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. One of the main problems is that click-stream data is inherently collected in a distributed manner, but in general these distributed click-stream logs are collated and pushed upstream in a centralized database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralized control and is therefore highly scalable. The elimination of centralized control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Pages154-177
Number of pages24
Volume4777 LNCS
Publication statusPublished - 1 Dec 2007
Externally publishedYes
Event5th International Workshop on Databases in Networked Information Systems, DNIS 2007 - Aizu-Wakamatsu, Japan
Duration: 17 Oct 200719 Oct 2007

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4777 LNCS
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other5th International Workshop on Databases in Networked Information Systems, DNIS 2007
CountryJapan
CityAizu-Wakamatsu
Period17/10/0719/10/07

Fingerprint

Data Warehousing
Computer Communication Networks
Information Management
Data warehouses
Automatic Data Processing
Databases
Technology
Query
Information management
Scalability
Data Warehouse
Data Streams
Information Processing
Viability
Repository
Latency
Elimination
Data analysis
Retrieval
Sufficient

ASJC Scopus subject areas

  • Computer Science(all)
  • Biochemistry, Genetics and Molecular Biology(all)
  • Theoretical Computer Science

Cite this

Alvaro, P., Ryaboy, D. V., & Agrawal, D. (2007). Towards scalable architectures for clickstream data warehousing. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 4777 LNCS, pp. 154-177). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4777 LNCS).

Towards scalable architectures for clickstream data warehousing. / Alvaro, Peter; Ryaboy, Dmitriy V.; Agrawal, Divyakant.

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4777 LNCS 2007. p. 154-177 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4777 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Alvaro, P, Ryaboy, DV & Agrawal, D 2007, Towards scalable architectures for clickstream data warehousing. in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). vol. 4777 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4777 LNCS, pp. 154-177, 5th International Workshop on Databases in Networked Information Systems, DNIS 2007, Aizu-Wakamatsu, Japan, 17/10/07.
Alvaro P, Ryaboy DV, Agrawal D. Towards scalable architectures for clickstream data warehousing. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4777 LNCS. 2007. p. 154-177. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
Alvaro, Peter ; Ryaboy, Dmitriy V. ; Agrawal, Divyakant. / Towards scalable architectures for clickstream data warehousing. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 4777 LNCS 2007. pp. 154-177 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{ef548750607f487a82e9b915bcd30a37,
title = "Towards scalable architectures for clickstream data warehousing",
abstract = "Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. One of the main problems is that click-stream data is inherently collected in a distributed manner, but in general these distributed click-stream logs are collated and pushed upstream in a centralized database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralized control and is therefore highly scalable. The elimination of centralized control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.",
author = "Peter Alvaro and Ryaboy, {Dmitriy V.} and Divyakant Agrawal",
year = "2007",
month = "12",
day = "1",
language = "English",
isbn = "9783540755111",
volume = "4777 LNCS",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "154--177",
booktitle = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

}

TY - GEN

T1 - Towards scalable architectures for clickstream data warehousing

AU - Alvaro, Peter

AU - Ryaboy, Dmitriy V.

AU - Agrawal, Divyakant

PY - 2007/12/1

Y1 - 2007/12/1

N2 - Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. One of the main problems is that click-stream data is inherently collected in a distributed manner, but in general these distributed click-stream logs are collated and pushed upstream in a centralized database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralized control and is therefore highly scalable. The elimination of centralized control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.

AB - Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. One of the main problems is that click-stream data is inherently collected in a distributed manner, but in general these distributed click-stream logs are collated and pushed upstream in a centralized database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralized control and is therefore highly scalable. The elimination of centralized control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.

UR - http://www.scopus.com/inward/record.url?scp=38149005477&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=38149005477&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:38149005477

SN - 9783540755111

VL - 4777 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 154

EP - 177

BT - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

ER -