Accurate sentence matching with hybrid siamese networks

Massimo Nicosia, Alessandro Moschitti

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Recent neural network approaches to sentence matching compute the probability of two sentences being similar by minimizing a logistic loss. In this paper, we learn sentence representations by means of a siamese network, which: (i) uses encoders that share parameters; and (ii) enables the comparison between two sentences in terms of their euclidean distance, by minimizing a contrastive loss. Moreover, we add a multilayer perceptron in the architecture to simultaneously optimize the contrastive and the logistic losses. This way, our network can exploit a more informative feedback, given by the logistic loss, which is also quantified by the distance that the two sentences have according to their representation in the euclidean space. We show that jointly minimizing the two losses yields higher accuracy than minimizing them independently. We verify this finding by evaluating several baseline architectures in two sentence matching tasks: question paraphrasing and textual entailment recognition. Our network approaches the state of the art, while being much simpler and faster to train, and with less parameters than its competitors.

Original languageEnglish
Title of host publicationCIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management
PublisherAssociation for Computing Machinery
Pages2235-2238
Number of pages4
VolumePart F131841
ISBN (Electronic)9781450349185
DOIs
Publication statusPublished - 6 Nov 2017
Event26th ACM International Conference on Information and Knowledge Management, CIKM 2017 - Singapore, Singapore
Duration: 6 Nov 201710 Nov 2017

Other

Other26th ACM International Conference on Information and Knowledge Management, CIKM 2017
CountrySingapore
CitySingapore
Period6/11/1710/11/17

Fingerprint

Logistics
Competitors
Neural networks
Euclidean distance
Train

ASJC Scopus subject areas

  • Business, Management and Accounting(all)
  • Decision Sciences(all)

Cite this

Nicosia, M., & Moschitti, A. (2017). Accurate sentence matching with hybrid siamese networks. In CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management (Vol. Part F131841, pp. 2235-2238). Association for Computing Machinery. https://doi.org/10.1145/3132847.3133156

Accurate sentence matching with hybrid siamese networks. / Nicosia, Massimo; Moschitti, Alessandro.

CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management. Vol. Part F131841 Association for Computing Machinery, 2017. p. 2235-2238.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Nicosia, M & Moschitti, A 2017, Accurate sentence matching with hybrid siamese networks. in CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management. vol. Part F131841, Association for Computing Machinery, pp. 2235-2238, 26th ACM International Conference on Information and Knowledge Management, CIKM 2017, Singapore, Singapore, 6/11/17. https://doi.org/10.1145/3132847.3133156
Nicosia M, Moschitti A. Accurate sentence matching with hybrid siamese networks. In CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management. Vol. Part F131841. Association for Computing Machinery. 2017. p. 2235-2238 https://doi.org/10.1145/3132847.3133156
Nicosia, Massimo ; Moschitti, Alessandro. / Accurate sentence matching with hybrid siamese networks. CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management. Vol. Part F131841 Association for Computing Machinery, 2017. pp. 2235-2238
@inproceedings{ee5b6d1d0c5c4a1c9f4c9dde8258c4e0,
title = "Accurate sentence matching with hybrid siamese networks",
abstract = "Recent neural network approaches to sentence matching compute the probability of two sentences being similar by minimizing a logistic loss. In this paper, we learn sentence representations by means of a siamese network, which: (i) uses encoders that share parameters; and (ii) enables the comparison between two sentences in terms of their euclidean distance, by minimizing a contrastive loss. Moreover, we add a multilayer perceptron in the architecture to simultaneously optimize the contrastive and the logistic losses. This way, our network can exploit a more informative feedback, given by the logistic loss, which is also quantified by the distance that the two sentences have according to their representation in the euclidean space. We show that jointly minimizing the two losses yields higher accuracy than minimizing them independently. We verify this finding by evaluating several baseline architectures in two sentence matching tasks: question paraphrasing and textual entailment recognition. Our network approaches the state of the art, while being much simpler and faster to train, and with less parameters than its competitors.",
author = "Massimo Nicosia and Alessandro Moschitti",
year = "2017",
month = "11",
day = "6",
doi = "10.1145/3132847.3133156",
language = "English",
volume = "Part F131841",
pages = "2235--2238",
booktitle = "CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - Accurate sentence matching with hybrid siamese networks

AU - Nicosia, Massimo

AU - Moschitti, Alessandro

PY - 2017/11/6

Y1 - 2017/11/6

N2 - Recent neural network approaches to sentence matching compute the probability of two sentences being similar by minimizing a logistic loss. In this paper, we learn sentence representations by means of a siamese network, which: (i) uses encoders that share parameters; and (ii) enables the comparison between two sentences in terms of their euclidean distance, by minimizing a contrastive loss. Moreover, we add a multilayer perceptron in the architecture to simultaneously optimize the contrastive and the logistic losses. This way, our network can exploit a more informative feedback, given by the logistic loss, which is also quantified by the distance that the two sentences have according to their representation in the euclidean space. We show that jointly minimizing the two losses yields higher accuracy than minimizing them independently. We verify this finding by evaluating several baseline architectures in two sentence matching tasks: question paraphrasing and textual entailment recognition. Our network approaches the state of the art, while being much simpler and faster to train, and with less parameters than its competitors.

AB - Recent neural network approaches to sentence matching compute the probability of two sentences being similar by minimizing a logistic loss. In this paper, we learn sentence representations by means of a siamese network, which: (i) uses encoders that share parameters; and (ii) enables the comparison between two sentences in terms of their euclidean distance, by minimizing a contrastive loss. Moreover, we add a multilayer perceptron in the architecture to simultaneously optimize the contrastive and the logistic losses. This way, our network can exploit a more informative feedback, given by the logistic loss, which is also quantified by the distance that the two sentences have according to their representation in the euclidean space. We show that jointly minimizing the two losses yields higher accuracy than minimizing them independently. We verify this finding by evaluating several baseline architectures in two sentence matching tasks: question paraphrasing and textual entailment recognition. Our network approaches the state of the art, while being much simpler and faster to train, and with less parameters than its competitors.

UR - http://www.scopus.com/inward/record.url?scp=85037327971&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85037327971&partnerID=8YFLogxK

U2 - 10.1145/3132847.3133156

DO - 10.1145/3132847.3133156

M3 - Conference contribution

VL - Part F131841

SP - 2235

EP - 2238

BT - CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management

PB - Association for Computing Machinery

ER -