Dimensionality reduction for similarity searching in dynamic databases

K. V. Ravi Kanth, Divyakant Agrawal, Ambuj Singh

Research output: Contribution to journalArticle

186 Citations (Scopus)

Abstract

Databases are increasingly being used to store multi-media objects such as maps, images, audio and video. Storage and retrieval of these objects is accomplished using multidimensional index structures such as R*-trees and SS-trees. As dimensionality increases, query performance in these index structures degrades. This phenomenon, generally referred to as the dimensionality curse, can be circumvented by reducing the dimensionality of the data. Such a reduction is however accompanied by a loss of precision of query results. Current techniques such as QBIC use SVD transform-based dimensionality reduction to ensure high query precision. The drawback of this approach is that SVD is expensive to compute, and therefore not readily applicable to dynamic databases. In this paper, we propose novel techniques for performing SVD-based dimensionality reduction in dynamic databases. When the data distribution changes considerably so as to degrade query precision, we recompute the SVD transform and incorporate it in the existing index structure. For recomputing the SVD-transform, we propose a novel technique that uses aggregate data from the existing index rather than the entire data. This technique reduces the SVD-computation time without compromising query precision. We then explore efficient ways to incorporate the recomputed SVD-transform in the existing index structure without degrading subsequent query response times. These techniques reduce the computation time by a factor of 20 in experiments on color and texture image vectors. The error due to approximate computation of SVD is less than 10%.

Original languageEnglish
Pages (from-to)166-176
Number of pages11
JournalSIGMOD Record
Volume27
Issue number2
Publication statusPublished - 1 Jun 1998
Externally publishedYes

Fingerprint

Singular value decomposition
Mathematical transformations
Textures
Color

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Information Systems
  • Software

Cite this

Ravi Kanth, K. V., Agrawal, D., & Singh, A. (1998). Dimensionality reduction for similarity searching in dynamic databases. SIGMOD Record, 27(2), 166-176.

Dimensionality reduction for similarity searching in dynamic databases. / Ravi Kanth, K. V.; Agrawal, Divyakant; Singh, Ambuj.

In: SIGMOD Record, Vol. 27, No. 2, 01.06.1998, p. 166-176.

Research output: Contribution to journalArticle

Ravi Kanth, KV, Agrawal, D & Singh, A 1998, 'Dimensionality reduction for similarity searching in dynamic databases', SIGMOD Record, vol. 27, no. 2, pp. 166-176.
Ravi Kanth KV, Agrawal D, Singh A. Dimensionality reduction for similarity searching in dynamic databases. SIGMOD Record. 1998 Jun 1;27(2):166-176.
Ravi Kanth, K. V. ; Agrawal, Divyakant ; Singh, Ambuj. / Dimensionality reduction for similarity searching in dynamic databases. In: SIGMOD Record. 1998 ; Vol. 27, No. 2. pp. 166-176.
@article{91d64114321244e0ace3f62d4d8cce78,
title = "Dimensionality reduction for similarity searching in dynamic databases",
abstract = "Databases are increasingly being used to store multi-media objects such as maps, images, audio and video. Storage and retrieval of these objects is accomplished using multidimensional index structures such as R*-trees and SS-trees. As dimensionality increases, query performance in these index structures degrades. This phenomenon, generally referred to as the dimensionality curse, can be circumvented by reducing the dimensionality of the data. Such a reduction is however accompanied by a loss of precision of query results. Current techniques such as QBIC use SVD transform-based dimensionality reduction to ensure high query precision. The drawback of this approach is that SVD is expensive to compute, and therefore not readily applicable to dynamic databases. In this paper, we propose novel techniques for performing SVD-based dimensionality reduction in dynamic databases. When the data distribution changes considerably so as to degrade query precision, we recompute the SVD transform and incorporate it in the existing index structure. For recomputing the SVD-transform, we propose a novel technique that uses aggregate data from the existing index rather than the entire data. This technique reduces the SVD-computation time without compromising query precision. We then explore efficient ways to incorporate the recomputed SVD-transform in the existing index structure without degrading subsequent query response times. These techniques reduce the computation time by a factor of 20 in experiments on color and texture image vectors. The error due to approximate computation of SVD is less than 10{\%}.",
author = "{Ravi Kanth}, {K. V.} and Divyakant Agrawal and Ambuj Singh",
year = "1998",
month = "6",
day = "1",
language = "English",
volume = "27",
pages = "166--176",
journal = "SIGMOD Record",
issn = "0163-5808",
publisher = "Association for Computing Machinery (ACM)",
number = "2",

}

TY - JOUR

T1 - Dimensionality reduction for similarity searching in dynamic databases

AU - Ravi Kanth, K. V.

AU - Agrawal, Divyakant

AU - Singh, Ambuj

PY - 1998/6/1

Y1 - 1998/6/1

N2 - Databases are increasingly being used to store multi-media objects such as maps, images, audio and video. Storage and retrieval of these objects is accomplished using multidimensional index structures such as R*-trees and SS-trees. As dimensionality increases, query performance in these index structures degrades. This phenomenon, generally referred to as the dimensionality curse, can be circumvented by reducing the dimensionality of the data. Such a reduction is however accompanied by a loss of precision of query results. Current techniques such as QBIC use SVD transform-based dimensionality reduction to ensure high query precision. The drawback of this approach is that SVD is expensive to compute, and therefore not readily applicable to dynamic databases. In this paper, we propose novel techniques for performing SVD-based dimensionality reduction in dynamic databases. When the data distribution changes considerably so as to degrade query precision, we recompute the SVD transform and incorporate it in the existing index structure. For recomputing the SVD-transform, we propose a novel technique that uses aggregate data from the existing index rather than the entire data. This technique reduces the SVD-computation time without compromising query precision. We then explore efficient ways to incorporate the recomputed SVD-transform in the existing index structure without degrading subsequent query response times. These techniques reduce the computation time by a factor of 20 in experiments on color and texture image vectors. The error due to approximate computation of SVD is less than 10%.

AB - Databases are increasingly being used to store multi-media objects such as maps, images, audio and video. Storage and retrieval of these objects is accomplished using multidimensional index structures such as R*-trees and SS-trees. As dimensionality increases, query performance in these index structures degrades. This phenomenon, generally referred to as the dimensionality curse, can be circumvented by reducing the dimensionality of the data. Such a reduction is however accompanied by a loss of precision of query results. Current techniques such as QBIC use SVD transform-based dimensionality reduction to ensure high query precision. The drawback of this approach is that SVD is expensive to compute, and therefore not readily applicable to dynamic databases. In this paper, we propose novel techniques for performing SVD-based dimensionality reduction in dynamic databases. When the data distribution changes considerably so as to degrade query precision, we recompute the SVD transform and incorporate it in the existing index structure. For recomputing the SVD-transform, we propose a novel technique that uses aggregate data from the existing index rather than the entire data. This technique reduces the SVD-computation time without compromising query precision. We then explore efficient ways to incorporate the recomputed SVD-transform in the existing index structure without degrading subsequent query response times. These techniques reduce the computation time by a factor of 20 in experiments on color and texture image vectors. The error due to approximate computation of SVD is less than 10%.

UR - http://www.scopus.com/inward/record.url?scp=17944378423&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=17944378423&partnerID=8YFLogxK

M3 - Article

VL - 27

SP - 166

EP - 176

JO - SIGMOD Record

JF - SIGMOD Record

SN - 0163-5808

IS - 2

ER -