A comprehensive study of features and algorithms for URL-based topic classification

Eda Baykan, Monika Henzinger, Ludmila Marian, Ingmar Weber

Research output: Contribution to journalArticle

40 Citations (Scopus)

Abstract

Given only the URL of a Web page, can we identify its topic? We study this problem in detail by exploring a large number of different feature sets and algorithms on several datasets. We also show that the inherent overlap between topics and the sparsity of the information in URLs makes this a very challenging problem. Web page classification without a page's content is desirable when the content is not available at all, when a classification is needed before obtaining the content, or when classification speed is of utmost importance. For our experiments we used five different corpora comprising a total of about 3 million (URL, classification) pairs. We evaluated several techniques for feature generation and classification algorithms. The individual binary classifiers were then combined via boosting intometabinary classifiers. We achieve typical F-measure values between 80 and 85, and a typical precision of around 86. The precision can be pushed further over 90 while maintaining a typical level of recall between 30 and 40.

Original languageEnglish
Article number15
JournalACM Transactions on the Web
Volume5
Issue number3
DOIs
Publication statusPublished - 1 Jul 2011

    Fingerprint

Keywords

  • ODP
  • Topic classification
  • URL

ASJC Scopus subject areas

  • Computer Networks and Communications

Cite this