Efficient aggregation algorithms for compressed data warehouses

Jianzhong Li, Jaideep Srivastava

Research output: Contribution to journalArticle

32 Citations (Scopus)

Abstract

Aggregation and cube are important operations for online analytical processing (OLAP). Many efficient algorithms to compute aggregation and cube for relational OLAP have been developed. Some work has been done on efficiently computing cube for multidimensional data warehouses that store data sets in multidimensional arrays rather than in tables. However, to our knowledge, there is nothing to date in the literature describing aggregation algorithms on compressed data warehouses for multidimensional OLAP. This paper presents a set of aggregation algorithms on compressed data warehouses for multidimensional OLAP. These algorithms operate directly on compressed data sets, which are compressed by the mapping-complete compression methods, without the need to first decompress them. The algorithms have different performance behaviors as a function of the data set parameters, sizes of outputs and main memory availability. The algorithms are described and the I/O and CPU cost functions are presented in this paper. A decision procedure to select the most efficient algorithm for a given aggregation request is also proposed. The analysis and experimental results show that the algorithms have better performance on sparse data than the previous aggregation algorithms.

Original languageEnglish
Pages (from-to)515-529
Number of pages15
JournalIEEE Transactions on Knowledge and Data Engineering
Volume14
Issue number3
DOIs
Publication statusPublished - May 2002
Externally publishedYes

Fingerprint

Data warehouses
Agglomeration
Processing
Cost functions
Program processors
Availability
Data storage equipment

Keywords

  • Aggregation
  • Aggregation on compressed data warehouses
  • Data warehouse
  • Multidimensional array
  • OLAP

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Electrical and Electronic Engineering
  • Artificial Intelligence
  • Information Systems

Cite this

Efficient aggregation algorithms for compressed data warehouses. / Li, Jianzhong; Srivastava, Jaideep.

In: IEEE Transactions on Knowledge and Data Engineering, Vol. 14, No. 3, 05.2002, p. 515-529.

Research output: Contribution to journalArticle

@article{b9281622070e4088a5e4899c89339e15,
title = "Efficient aggregation algorithms for compressed data warehouses",
abstract = "Aggregation and cube are important operations for online analytical processing (OLAP). Many efficient algorithms to compute aggregation and cube for relational OLAP have been developed. Some work has been done on efficiently computing cube for multidimensional data warehouses that store data sets in multidimensional arrays rather than in tables. However, to our knowledge, there is nothing to date in the literature describing aggregation algorithms on compressed data warehouses for multidimensional OLAP. This paper presents a set of aggregation algorithms on compressed data warehouses for multidimensional OLAP. These algorithms operate directly on compressed data sets, which are compressed by the mapping-complete compression methods, without the need to first decompress them. The algorithms have different performance behaviors as a function of the data set parameters, sizes of outputs and main memory availability. The algorithms are described and the I/O and CPU cost functions are presented in this paper. A decision procedure to select the most efficient algorithm for a given aggregation request is also proposed. The analysis and experimental results show that the algorithms have better performance on sparse data than the previous aggregation algorithms.",
keywords = "Aggregation, Aggregation on compressed data warehouses, Data warehouse, Multidimensional array, OLAP",
author = "Jianzhong Li and Jaideep Srivastava",
year = "2002",
month = "5",
doi = "10.1109/TKDE.2002.1000340",
language = "English",
volume = "14",
pages = "515--529",
journal = "IEEE Transactions on Knowledge and Data Engineering",
issn = "1041-4347",
publisher = "IEEE Computer Society",
number = "3",

}

TY - JOUR

T1 - Efficient aggregation algorithms for compressed data warehouses

AU - Li, Jianzhong

AU - Srivastava, Jaideep

PY - 2002/5

Y1 - 2002/5

N2 - Aggregation and cube are important operations for online analytical processing (OLAP). Many efficient algorithms to compute aggregation and cube for relational OLAP have been developed. Some work has been done on efficiently computing cube for multidimensional data warehouses that store data sets in multidimensional arrays rather than in tables. However, to our knowledge, there is nothing to date in the literature describing aggregation algorithms on compressed data warehouses for multidimensional OLAP. This paper presents a set of aggregation algorithms on compressed data warehouses for multidimensional OLAP. These algorithms operate directly on compressed data sets, which are compressed by the mapping-complete compression methods, without the need to first decompress them. The algorithms have different performance behaviors as a function of the data set parameters, sizes of outputs and main memory availability. The algorithms are described and the I/O and CPU cost functions are presented in this paper. A decision procedure to select the most efficient algorithm for a given aggregation request is also proposed. The analysis and experimental results show that the algorithms have better performance on sparse data than the previous aggregation algorithms.

AB - Aggregation and cube are important operations for online analytical processing (OLAP). Many efficient algorithms to compute aggregation and cube for relational OLAP have been developed. Some work has been done on efficiently computing cube for multidimensional data warehouses that store data sets in multidimensional arrays rather than in tables. However, to our knowledge, there is nothing to date in the literature describing aggregation algorithms on compressed data warehouses for multidimensional OLAP. This paper presents a set of aggregation algorithms on compressed data warehouses for multidimensional OLAP. These algorithms operate directly on compressed data sets, which are compressed by the mapping-complete compression methods, without the need to first decompress them. The algorithms have different performance behaviors as a function of the data set parameters, sizes of outputs and main memory availability. The algorithms are described and the I/O and CPU cost functions are presented in this paper. A decision procedure to select the most efficient algorithm for a given aggregation request is also proposed. The analysis and experimental results show that the algorithms have better performance on sparse data than the previous aggregation algorithms.

KW - Aggregation

KW - Aggregation on compressed data warehouses

KW - Data warehouse

KW - Multidimensional array

KW - OLAP

UR - http://www.scopus.com/inward/record.url?scp=0036565571&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0036565571&partnerID=8YFLogxK

U2 - 10.1109/TKDE.2002.1000340

DO - 10.1109/TKDE.2002.1000340

M3 - Article

VL - 14

SP - 515

EP - 529

JO - IEEE Transactions on Knowledge and Data Engineering

JF - IEEE Transactions on Knowledge and Data Engineering

SN - 1041-4347

IS - 3

ER -