Injecting relational structural representation in neural networks for question similarity

Antonio Uva, Daniele Bonadiman, Alessandro Moschitti

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Effectively using full syntactic parsing information in Neural Networks (NNs) to solve relational tasks, e.g., question similarity, is still an open problem. In this paper, we propose to inject structural representations in NNs by (i) learning an SVM model using Tree Kernels (TKs) on relatively few pairs of questions (few thousands) as gold standard (GS) training data is typically scarce, (ii) predicting labels on a very large corpus of question pairs, and (iii) pre-training NNs on such large corpus. The results on Quora and SemEval question similarity datasets show that NNs trained with our approach can learn more accurate models, especially after fine tuning on GS.

Original languageEnglish
Title of host publicationACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Short Papers)
PublisherAssociation for Computational Linguistics (ACL)
Pages285-291
Number of pages7
ISBN (Electronic)9781948087346
Publication statusPublished - 1 Jan 2018
Event56th Annual Meeting of the Association for Computational Linguistics, ACL 2018 - Melbourne, Australia
Duration: 15 Jul 201820 Jul 2018

Publication series

NameACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)
Volume2

Conference

Conference56th Annual Meeting of the Association for Computational Linguistics, ACL 2018
CountryAustralia
CityMelbourne
Period15/7/1820/7/18

Fingerprint

Neural networks
Syntactics
Labels
Tuning

ASJC Scopus subject areas

  • Software
  • Computational Theory and Mathematics

Cite this

Uva, A., Bonadiman, D., & Moschitti, A. (2018). Injecting relational structural representation in neural networks for question similarity. In ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Short Papers) (pp. 285-291). (ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers); Vol. 2). Association for Computational Linguistics (ACL).

Injecting relational structural representation in neural networks for question similarity. / Uva, Antonio; Bonadiman, Daniele; Moschitti, Alessandro.

ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Short Papers). Association for Computational Linguistics (ACL), 2018. p. 285-291 (ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers); Vol. 2).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Uva, A, Bonadiman, D & Moschitti, A 2018, Injecting relational structural representation in neural networks for question similarity. in ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Short Papers). ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), vol. 2, Association for Computational Linguistics (ACL), pp. 285-291, 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, 15/7/18.
Uva A, Bonadiman D, Moschitti A. Injecting relational structural representation in neural networks for question similarity. In ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Short Papers). Association for Computational Linguistics (ACL). 2018. p. 285-291. (ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)).
Uva, Antonio ; Bonadiman, Daniele ; Moschitti, Alessandro. / Injecting relational structural representation in neural networks for question similarity. ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Short Papers). Association for Computational Linguistics (ACL), 2018. pp. 285-291 (ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)).
@inproceedings{30a1baaba9094cdead8fea95fa9b168a,
title = "Injecting relational structural representation in neural networks for question similarity",
abstract = "Effectively using full syntactic parsing information in Neural Networks (NNs) to solve relational tasks, e.g., question similarity, is still an open problem. In this paper, we propose to inject structural representations in NNs by (i) learning an SVM model using Tree Kernels (TKs) on relatively few pairs of questions (few thousands) as gold standard (GS) training data is typically scarce, (ii) predicting labels on a very large corpus of question pairs, and (iii) pre-training NNs on such large corpus. The results on Quora and SemEval question similarity datasets show that NNs trained with our approach can learn more accurate models, especially after fine tuning on GS.",
author = "Antonio Uva and Daniele Bonadiman and Alessandro Moschitti",
year = "2018",
month = "1",
day = "1",
language = "English",
series = "ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)",
publisher = "Association for Computational Linguistics (ACL)",
pages = "285--291",
booktitle = "ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Short Papers)",

}

TY - GEN

T1 - Injecting relational structural representation in neural networks for question similarity

AU - Uva, Antonio

AU - Bonadiman, Daniele

AU - Moschitti, Alessandro

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Effectively using full syntactic parsing information in Neural Networks (NNs) to solve relational tasks, e.g., question similarity, is still an open problem. In this paper, we propose to inject structural representations in NNs by (i) learning an SVM model using Tree Kernels (TKs) on relatively few pairs of questions (few thousands) as gold standard (GS) training data is typically scarce, (ii) predicting labels on a very large corpus of question pairs, and (iii) pre-training NNs on such large corpus. The results on Quora and SemEval question similarity datasets show that NNs trained with our approach can learn more accurate models, especially after fine tuning on GS.

AB - Effectively using full syntactic parsing information in Neural Networks (NNs) to solve relational tasks, e.g., question similarity, is still an open problem. In this paper, we propose to inject structural representations in NNs by (i) learning an SVM model using Tree Kernels (TKs) on relatively few pairs of questions (few thousands) as gold standard (GS) training data is typically scarce, (ii) predicting labels on a very large corpus of question pairs, and (iii) pre-training NNs on such large corpus. The results on Quora and SemEval question similarity datasets show that NNs trained with our approach can learn more accurate models, especially after fine tuning on GS.

UR - http://www.scopus.com/inward/record.url?scp=85063133751&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063133751&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85063133751

T3 - ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers)

SP - 285

EP - 291

BT - ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Short Papers)

PB - Association for Computational Linguistics (ACL)

ER -