Combining hashing and abstraction in sparse high dimensional feature spaces

Cornelia Caragea, Adrian Silvescu, Prasenjit Mitra

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

With the exponential increase in the number of documents available online, e.g., news articles, weblogs, scientific documents, the development of effective and efficient classification methods is needed. The performance of document classifiers critically depends, among other things, on the choice of the feature representation. The commonly used "bag of words" and n-gram representations can result in prohibitively high dimensional input spaces. Data mining algorithms applied to these input spaces may be intractable due to the large number of dimensions. Thus, dimensionality reduction algorithms that can process data into features fast at runtime, ideally in constant time per feature, are greatly needed in high throughput applications, where the number of features and data points can be in the order of millions. One promising line of research to dimensionality reduction is feature clustering. We propose to combine two types of feature clustering, namely hashing and abstraction based on hierarchical agglomerative clustering, in order to take advantage of the strengths of both techniques. Experimental results on two text data sets show that the combined approach uses significantly smaller number of features and gives similar performance when compared with the "bag of words" and n-gram approaches.

Original languageEnglish
Title of host publicationProceedings of the National Conference on Artificial Intelligence
Pages3-9
Number of pages7
Volume1
Publication statusPublished - 2012
Externally publishedYes
Event26th AAAI Conference on Artificial Intelligence and the 24th Innovative Applications of Artificial Intelligence Conference, AAAI-12 / IAAI-12 - Toronto, ON
Duration: 22 Jul 201226 Jul 2012

Other

Other26th AAAI Conference on Artificial Intelligence and the 24th Innovative Applications of Artificial Intelligence Conference, AAAI-12 / IAAI-12
CityToronto, ON
Period22/7/1226/7/12

Fingerprint

Data mining
Classifiers
Throughput

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence

Cite this

Caragea, C., Silvescu, A., & Mitra, P. (2012). Combining hashing and abstraction in sparse high dimensional feature spaces. In Proceedings of the National Conference on Artificial Intelligence (Vol. 1, pp. 3-9)

Combining hashing and abstraction in sparse high dimensional feature spaces. / Caragea, Cornelia; Silvescu, Adrian; Mitra, Prasenjit.

Proceedings of the National Conference on Artificial Intelligence. Vol. 1 2012. p. 3-9.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Caragea, C, Silvescu, A & Mitra, P 2012, Combining hashing and abstraction in sparse high dimensional feature spaces. in Proceedings of the National Conference on Artificial Intelligence. vol. 1, pp. 3-9, 26th AAAI Conference on Artificial Intelligence and the 24th Innovative Applications of Artificial Intelligence Conference, AAAI-12 / IAAI-12, Toronto, ON, 22/7/12.
Caragea C, Silvescu A, Mitra P. Combining hashing and abstraction in sparse high dimensional feature spaces. In Proceedings of the National Conference on Artificial Intelligence. Vol. 1. 2012. p. 3-9
Caragea, Cornelia ; Silvescu, Adrian ; Mitra, Prasenjit. / Combining hashing and abstraction in sparse high dimensional feature spaces. Proceedings of the National Conference on Artificial Intelligence. Vol. 1 2012. pp. 3-9
@inproceedings{2a5abc00839642088a8f5ffafb7b7041,
title = "Combining hashing and abstraction in sparse high dimensional feature spaces",
abstract = "With the exponential increase in the number of documents available online, e.g., news articles, weblogs, scientific documents, the development of effective and efficient classification methods is needed. The performance of document classifiers critically depends, among other things, on the choice of the feature representation. The commonly used {"}bag of words{"} and n-gram representations can result in prohibitively high dimensional input spaces. Data mining algorithms applied to these input spaces may be intractable due to the large number of dimensions. Thus, dimensionality reduction algorithms that can process data into features fast at runtime, ideally in constant time per feature, are greatly needed in high throughput applications, where the number of features and data points can be in the order of millions. One promising line of research to dimensionality reduction is feature clustering. We propose to combine two types of feature clustering, namely hashing and abstraction based on hierarchical agglomerative clustering, in order to take advantage of the strengths of both techniques. Experimental results on two text data sets show that the combined approach uses significantly smaller number of features and gives similar performance when compared with the {"}bag of words{"} and n-gram approaches.",
author = "Cornelia Caragea and Adrian Silvescu and Prasenjit Mitra",
year = "2012",
language = "English",
isbn = "9781577355687",
volume = "1",
pages = "3--9",
booktitle = "Proceedings of the National Conference on Artificial Intelligence",

}

TY - GEN

T1 - Combining hashing and abstraction in sparse high dimensional feature spaces

AU - Caragea, Cornelia

AU - Silvescu, Adrian

AU - Mitra, Prasenjit

PY - 2012

Y1 - 2012

N2 - With the exponential increase in the number of documents available online, e.g., news articles, weblogs, scientific documents, the development of effective and efficient classification methods is needed. The performance of document classifiers critically depends, among other things, on the choice of the feature representation. The commonly used "bag of words" and n-gram representations can result in prohibitively high dimensional input spaces. Data mining algorithms applied to these input spaces may be intractable due to the large number of dimensions. Thus, dimensionality reduction algorithms that can process data into features fast at runtime, ideally in constant time per feature, are greatly needed in high throughput applications, where the number of features and data points can be in the order of millions. One promising line of research to dimensionality reduction is feature clustering. We propose to combine two types of feature clustering, namely hashing and abstraction based on hierarchical agglomerative clustering, in order to take advantage of the strengths of both techniques. Experimental results on two text data sets show that the combined approach uses significantly smaller number of features and gives similar performance when compared with the "bag of words" and n-gram approaches.

AB - With the exponential increase in the number of documents available online, e.g., news articles, weblogs, scientific documents, the development of effective and efficient classification methods is needed. The performance of document classifiers critically depends, among other things, on the choice of the feature representation. The commonly used "bag of words" and n-gram representations can result in prohibitively high dimensional input spaces. Data mining algorithms applied to these input spaces may be intractable due to the large number of dimensions. Thus, dimensionality reduction algorithms that can process data into features fast at runtime, ideally in constant time per feature, are greatly needed in high throughput applications, where the number of features and data points can be in the order of millions. One promising line of research to dimensionality reduction is feature clustering. We propose to combine two types of feature clustering, namely hashing and abstraction based on hierarchical agglomerative clustering, in order to take advantage of the strengths of both techniques. Experimental results on two text data sets show that the combined approach uses significantly smaller number of features and gives similar performance when compared with the "bag of words" and n-gram approaches.

UR - http://www.scopus.com/inward/record.url?scp=84868279238&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84868279238&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84868279238

SN - 9781577355687

VL - 1

SP - 3

EP - 9

BT - Proceedings of the National Conference on Artificial Intelligence

ER -