Large-scale support vector learning with structural kernels

Aliaksei Severyn, Alessandro Moschitti

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

In this paper, we present an extensive study of the cutting-plane algorithm (CPA) applied to structural kernels for advanced text classification on large datasets. In particular, we carry out a comprehensive experimentation on two interesting natural language tasks, e.g. predicate argument extraction and question answering. Our results show that (i) CPA applied to train a non-linear model with different tree kernels fully matches the accuracy of the conventional SVM algorithm while being ten times faster; (ii) by using smaller sampling sizes to approximate subgradients in CPA we can trade off accuracy for speed, yet the optimal parameters and kernels found remain optimal for the exact SVM. These results open numerous research perspectives, e.g. in natural language processing, as they show that complex structural kernels can be efficiently used in real-world applications. For example, for the first time, we could carry out extensive tests of several tree kernels on millions of training instances. As a direct benefit, we could experiment with a variant of the partial tree kernel, which we also propose in this paper.

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2010, Proceedings
Pages229-244
Number of pages16
EditionPART 3
DOIs
Publication statusPublished - 25 Oct 2010
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2010 - Barcelona, Spain
Duration: 20 Sep 201024 Sep 2010

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
NumberPART 3
Volume6323 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2010
CountrySpain
CityBarcelona
Period20/9/1024/9/10

    Fingerprint

Keywords

  • Natural Language Processing
  • Structural Kernels
  • Support Vector Machines

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Severyn, A., & Moschitti, A. (2010). Large-scale support vector learning with structural kernels. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2010, Proceedings (PART 3 ed., pp. 229-244). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 6323 LNAI, No. PART 3). https://doi.org/10.1007/978-3-642-15939-8_15