Knuckles: Bringing the database to the data

Peter Alvaro, Dmitriy V. Ryaboy, Divyakant Agrawal

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. Although click-stream data tends to be collected in a distributed manner to support scaling the servers that host the websites, in general these partitioned click-stream logs are collated and pushed upstream to a centralised database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralised control and is therefore highly scalable. The elimination of centralised control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.

Original languageEnglish
Pages (from-to)214-225
Number of pages12
JournalInternational Journal of Computational Science and Engineering
Volume5
Issue number3-4
DOIs
Publication statusPublished - 1 Dec 2010
Externally publishedYes

Fingerprint

Data warehouses
Query
Data Warehousing
Information Management
Data Warehouse
Data Streams
Information Processing
Viability
Repository
Information management
Latency
Elimination
Scalability
Websites
Data analysis
Retrieval
Servers
Server
Scaling
Tend

Keywords

  • data partitioning.
  • distributed databases
  • parallel query processing
  • peer-to-peer computing

ASJC Scopus subject areas

  • Computational Mathematics
  • Modelling and Simulation
  • Computational Theory and Mathematics
  • Hardware and Architecture
  • Software

Cite this

Knuckles : Bringing the database to the data. / Alvaro, Peter; Ryaboy, Dmitriy V.; Agrawal, Divyakant.

In: International Journal of Computational Science and Engineering, Vol. 5, No. 3-4, 01.12.2010, p. 214-225.

Research output: Contribution to journalArticle

Alvaro, Peter ; Ryaboy, Dmitriy V. ; Agrawal, Divyakant. / Knuckles : Bringing the database to the data. In: International Journal of Computational Science and Engineering. 2010 ; Vol. 5, No. 3-4. pp. 214-225.
@article{eb2be37fff834a4cad39a231b8c5e5f4,
title = "Knuckles: Bringing the database to the data",
abstract = "Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. Although click-stream data tends to be collected in a distributed manner to support scaling the servers that host the websites, in general these partitioned click-stream logs are collated and pushed upstream to a centralised database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralised control and is therefore highly scalable. The elimination of centralised control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.",
keywords = "data partitioning., distributed databases, parallel query processing, peer-to-peer computing",
author = "Peter Alvaro and Ryaboy, {Dmitriy V.} and Divyakant Agrawal",
year = "2010",
month = "12",
day = "1",
doi = "10.1504/IJCSE.2010.037677",
language = "English",
volume = "5",
pages = "214--225",
journal = "International Journal of Computational Science and Engineering",
issn = "1742-7185",
publisher = "Inderscience Enterprises Ltd",
number = "3-4",

}

TY - JOUR

T1 - Knuckles

T2 - Bringing the database to the data

AU - Alvaro, Peter

AU - Ryaboy, Dmitriy V.

AU - Agrawal, Divyakant

PY - 2010/12/1

Y1 - 2010/12/1

N2 - Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. Although click-stream data tends to be collected in a distributed manner to support scaling the servers that host the websites, in general these partitioned click-stream logs are collated and pushed upstream to a centralised database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralised control and is therefore highly scalable. The elimination of centralised control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.

AB - Click-stream data warehousing has emerged as a monumental information management and processing challenge for commercial enterprises. Traditional solutions based on commercial DBMS technology often suffer from poor scalability and large processing latencies. Although click-stream data tends to be collected in a distributed manner to support scaling the servers that host the websites, in general these partitioned click-stream logs are collated and pushed upstream to a centralised database storage repository, creating storage bottlenecks. In this paper, we propose a design of an ad-hoc retrieval system suitable for click-stream data warehouses, in which the data remains distributed and database queries are rewritten to be executed against the distributed data. The query rewrite does not involve any centralised control and is therefore highly scalable. The elimination of centralised control is achieved by supporting a restricted subset of SQL, which is sufficient for most click-stream data analysis. Evaluations conducted using both synthetic and real data establish the viability of this approach.

KW - data partitioning.

KW - distributed databases

KW - parallel query processing

KW - peer-to-peer computing

UR - http://www.scopus.com/inward/record.url?scp=78650661144&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78650661144&partnerID=8YFLogxK

U2 - 10.1504/IJCSE.2010.037677

DO - 10.1504/IJCSE.2010.037677

M3 - Article

AN - SCOPUS:78650661144

VL - 5

SP - 214

EP - 225

JO - International Journal of Computational Science and Engineering

JF - International Journal of Computational Science and Engineering

SN - 1742-7185

IS - 3-4

ER -