Regularized and retrofitted models for learning sentence representation with context

Tanay Kumar Saha, Shafiq Rayhan Joty, Naeemul Hassan, Mohammad Al Hasan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Vector representation of sentences is important for many text processing tasks that involve classifying, clustering, or ranking sentences. For solving these tasks, bag-of-word based representation has been used for a long time. In recent years, distributed representation of sentences learned by neural models from unlabeled data has been shown to outperform traditional bag-of-words representations. However, most existing methods belonging to the neural models consider only the content of a sentence, and disregard its relations with other sentences in the context. In this paper, we first characterize two types of contexts depending on their scope and utility. We then propose two approaches to incorporate contextual information into content-based models. We evaluate our sentence representation models in a setup, where context is available to infer sentence vectors. Experimental results demonstrate that our proposed models outshine existing models on three fundamental tasks, such as, classifying, clustering, and ranking sentences.

Original languageEnglish
Title of host publicationCIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management
PublisherAssociation for Computing Machinery
Pages547-556
Number of pages10
VolumePart F131841
ISBN (Electronic)9781450349185
DOIs
Publication statusPublished - 6 Nov 2017
Event26th ACM International Conference on Information and Knowledge Management, CIKM 2017 - Singapore, Singapore
Duration: 6 Nov 201710 Nov 2017

Other

Other26th ACM International Conference on Information and Knowledge Management, CIKM 2017
CountrySingapore
CitySingapore
Period6/11/1710/11/17

Fingerprint

Ranking
Clustering
Information content

Keywords

  • Classification
  • Clustering
  • Discourse
  • Distributed representation of sentences
  • Feature learning
  • Ranking
  • Retrofitting
  • Sen2Vec

ASJC Scopus subject areas

  • Business, Management and Accounting(all)
  • Decision Sciences(all)

Cite this

Saha, T. K., Rayhan Joty, S., Hassan, N., & Al Hasan, M. (2017). Regularized and retrofitted models for learning sentence representation with context. In CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management (Vol. Part F131841, pp. 547-556). Association for Computing Machinery. https://doi.org/10.1145/3132847.3133011

Regularized and retrofitted models for learning sentence representation with context. / Saha, Tanay Kumar; Rayhan Joty, Shafiq; Hassan, Naeemul; Al Hasan, Mohammad.

CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management. Vol. Part F131841 Association for Computing Machinery, 2017. p. 547-556.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Saha, TK, Rayhan Joty, S, Hassan, N & Al Hasan, M 2017, Regularized and retrofitted models for learning sentence representation with context. in CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management. vol. Part F131841, Association for Computing Machinery, pp. 547-556, 26th ACM International Conference on Information and Knowledge Management, CIKM 2017, Singapore, Singapore, 6/11/17. https://doi.org/10.1145/3132847.3133011
Saha TK, Rayhan Joty S, Hassan N, Al Hasan M. Regularized and retrofitted models for learning sentence representation with context. In CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management. Vol. Part F131841. Association for Computing Machinery. 2017. p. 547-556 https://doi.org/10.1145/3132847.3133011
Saha, Tanay Kumar ; Rayhan Joty, Shafiq ; Hassan, Naeemul ; Al Hasan, Mohammad. / Regularized and retrofitted models for learning sentence representation with context. CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management. Vol. Part F131841 Association for Computing Machinery, 2017. pp. 547-556
@inproceedings{713b8a50e28a4fcf9749982014bcf2a0,
title = "Regularized and retrofitted models for learning sentence representation with context",
abstract = "Vector representation of sentences is important for many text processing tasks that involve classifying, clustering, or ranking sentences. For solving these tasks, bag-of-word based representation has been used for a long time. In recent years, distributed representation of sentences learned by neural models from unlabeled data has been shown to outperform traditional bag-of-words representations. However, most existing methods belonging to the neural models consider only the content of a sentence, and disregard its relations with other sentences in the context. In this paper, we first characterize two types of contexts depending on their scope and utility. We then propose two approaches to incorporate contextual information into content-based models. We evaluate our sentence representation models in a setup, where context is available to infer sentence vectors. Experimental results demonstrate that our proposed models outshine existing models on three fundamental tasks, such as, classifying, clustering, and ranking sentences.",
keywords = "Classification, Clustering, Discourse, Distributed representation of sentences, Feature learning, Ranking, Retrofitting, Sen2Vec",
author = "Saha, {Tanay Kumar} and {Rayhan Joty}, Shafiq and Naeemul Hassan and {Al Hasan}, Mohammad",
year = "2017",
month = "11",
day = "6",
doi = "10.1145/3132847.3133011",
language = "English",
volume = "Part F131841",
pages = "547--556",
booktitle = "CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management",
publisher = "Association for Computing Machinery",

}

TY - GEN

T1 - Regularized and retrofitted models for learning sentence representation with context

AU - Saha, Tanay Kumar

AU - Rayhan Joty, Shafiq

AU - Hassan, Naeemul

AU - Al Hasan, Mohammad

PY - 2017/11/6

Y1 - 2017/11/6

N2 - Vector representation of sentences is important for many text processing tasks that involve classifying, clustering, or ranking sentences. For solving these tasks, bag-of-word based representation has been used for a long time. In recent years, distributed representation of sentences learned by neural models from unlabeled data has been shown to outperform traditional bag-of-words representations. However, most existing methods belonging to the neural models consider only the content of a sentence, and disregard its relations with other sentences in the context. In this paper, we first characterize two types of contexts depending on their scope and utility. We then propose two approaches to incorporate contextual information into content-based models. We evaluate our sentence representation models in a setup, where context is available to infer sentence vectors. Experimental results demonstrate that our proposed models outshine existing models on three fundamental tasks, such as, classifying, clustering, and ranking sentences.

AB - Vector representation of sentences is important for many text processing tasks that involve classifying, clustering, or ranking sentences. For solving these tasks, bag-of-word based representation has been used for a long time. In recent years, distributed representation of sentences learned by neural models from unlabeled data has been shown to outperform traditional bag-of-words representations. However, most existing methods belonging to the neural models consider only the content of a sentence, and disregard its relations with other sentences in the context. In this paper, we first characterize two types of contexts depending on their scope and utility. We then propose two approaches to incorporate contextual information into content-based models. We evaluate our sentence representation models in a setup, where context is available to infer sentence vectors. Experimental results demonstrate that our proposed models outshine existing models on three fundamental tasks, such as, classifying, clustering, and ranking sentences.

KW - Classification

KW - Clustering

KW - Discourse

KW - Distributed representation of sentences

KW - Feature learning

KW - Ranking

KW - Retrofitting

KW - Sen2Vec

UR - http://www.scopus.com/inward/record.url?scp=85037375000&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85037375000&partnerID=8YFLogxK

U2 - 10.1145/3132847.3133011

DO - 10.1145/3132847.3133011

M3 - Conference contribution

AN - SCOPUS:85037375000

VL - Part F131841

SP - 547

EP - 556

BT - CIKM 2017 - Proceedings of the 2017 ACM Conference on Information and Knowledge Management

PB - Association for Computing Machinery

ER -