Fast linearization of tree kernels over large-scale data

Aliaksei Severyn, Alessandro Moschitti

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Convolution tree kernels have been successfully applied to many language processing tasks for achieving state-of-the-art accuracy. Unfortunately, higher computational complexity of learning with kernels w.r.t. using explicit feature vectors makes them less attractive for large-scale data. In this paper, we study the latest approaches to solve such problems ranging from feature hashing to reverse kernel engineering and approximate cutting plane training with model compression. We derive a novel method that relies on reverse-kernel engineering together with an efficient kernel learning method. The approach gives the advantage of using tree kernels to automatically generate rich structured feature spaces and working in the linear space where learning and testing is fast. We experimented with training sets up to 4 million examples from Semantic Role Labeling. The results show that (i) the choice of correct structural features is essential and (ii) we can speed-up training from weeks to less than 20 minutes.

Original languageEnglish
Title of host publicationIJCAI International Joint Conference on Artificial Intelligence
Pages2162-2168
Number of pages7
Publication statusPublished - 1 Dec 2013
Event23rd International Joint Conference on Artificial Intelligence, IJCAI 2013 - Beijing, China
Duration: 3 Aug 20139 Aug 2013

Other

Other23rd International Joint Conference on Artificial Intelligence, IJCAI 2013
CountryChina
CityBeijing
Period3/8/139/8/13

Fingerprint

Reverse engineering
Trees (mathematics)
Linearization
Convolution
Labeling
Computational complexity
Semantics
Testing
Processing

ASJC Scopus subject areas

  • Artificial Intelligence

Cite this

Severyn, A., & Moschitti, A. (2013). Fast linearization of tree kernels over large-scale data. In IJCAI International Joint Conference on Artificial Intelligence (pp. 2162-2168)

Fast linearization of tree kernels over large-scale data. / Severyn, Aliaksei; Moschitti, Alessandro.

IJCAI International Joint Conference on Artificial Intelligence. 2013. p. 2162-2168.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Severyn, A & Moschitti, A 2013, Fast linearization of tree kernels over large-scale data. in IJCAI International Joint Conference on Artificial Intelligence. pp. 2162-2168, 23rd International Joint Conference on Artificial Intelligence, IJCAI 2013, Beijing, China, 3/8/13.
Severyn A, Moschitti A. Fast linearization of tree kernels over large-scale data. In IJCAI International Joint Conference on Artificial Intelligence. 2013. p. 2162-2168
Severyn, Aliaksei ; Moschitti, Alessandro. / Fast linearization of tree kernels over large-scale data. IJCAI International Joint Conference on Artificial Intelligence. 2013. pp. 2162-2168
@inproceedings{a9d879b2cb7d486eafc4d1877446bdcc,
title = "Fast linearization of tree kernels over large-scale data",
abstract = "Convolution tree kernels have been successfully applied to many language processing tasks for achieving state-of-the-art accuracy. Unfortunately, higher computational complexity of learning with kernels w.r.t. using explicit feature vectors makes them less attractive for large-scale data. In this paper, we study the latest approaches to solve such problems ranging from feature hashing to reverse kernel engineering and approximate cutting plane training with model compression. We derive a novel method that relies on reverse-kernel engineering together with an efficient kernel learning method. The approach gives the advantage of using tree kernels to automatically generate rich structured feature spaces and working in the linear space where learning and testing is fast. We experimented with training sets up to 4 million examples from Semantic Role Labeling. The results show that (i) the choice of correct structural features is essential and (ii) we can speed-up training from weeks to less than 20 minutes.",
author = "Aliaksei Severyn and Alessandro Moschitti",
year = "2013",
month = "12",
day = "1",
language = "English",
isbn = "9781577356332",
pages = "2162--2168",
booktitle = "IJCAI International Joint Conference on Artificial Intelligence",

}

TY - GEN

T1 - Fast linearization of tree kernels over large-scale data

AU - Severyn, Aliaksei

AU - Moschitti, Alessandro

PY - 2013/12/1

Y1 - 2013/12/1

N2 - Convolution tree kernels have been successfully applied to many language processing tasks for achieving state-of-the-art accuracy. Unfortunately, higher computational complexity of learning with kernels w.r.t. using explicit feature vectors makes them less attractive for large-scale data. In this paper, we study the latest approaches to solve such problems ranging from feature hashing to reverse kernel engineering and approximate cutting plane training with model compression. We derive a novel method that relies on reverse-kernel engineering together with an efficient kernel learning method. The approach gives the advantage of using tree kernels to automatically generate rich structured feature spaces and working in the linear space where learning and testing is fast. We experimented with training sets up to 4 million examples from Semantic Role Labeling. The results show that (i) the choice of correct structural features is essential and (ii) we can speed-up training from weeks to less than 20 minutes.

AB - Convolution tree kernels have been successfully applied to many language processing tasks for achieving state-of-the-art accuracy. Unfortunately, higher computational complexity of learning with kernels w.r.t. using explicit feature vectors makes them less attractive for large-scale data. In this paper, we study the latest approaches to solve such problems ranging from feature hashing to reverse kernel engineering and approximate cutting plane training with model compression. We derive a novel method that relies on reverse-kernel engineering together with an efficient kernel learning method. The approach gives the advantage of using tree kernels to automatically generate rich structured feature spaces and working in the linear space where learning and testing is fast. We experimented with training sets up to 4 million examples from Semantic Role Labeling. The results show that (i) the choice of correct structural features is essential and (ii) we can speed-up training from weeks to less than 20 minutes.

UR - http://www.scopus.com/inward/record.url?scp=84896062611&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84896062611&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:84896062611

SN - 9781577356332

SP - 2162

EP - 2168

BT - IJCAI International Joint Conference on Artificial Intelligence

ER -