Con-S2V: A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec

Tanay Kumar Saha, Shafiq Rayhan Joty, Mohammad Al Hasan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a novel approach to learn distributed representation of sentences from unlabeled data by modeling both content and context of a sentence. The content model learns sentence representation by predicting its words. On the other hand, the context model comprises a neighbor prediction component and a regularizer to model distributional and proximity hypotheses, respectively. We propose an online algorithm to train the model components jointly. We evaluate the models in a setup, where contextual information is available. The experimental results on tasks involving classification, clustering, and ranking of sentences show that our model outperforms the best existing models by a wide margin across multiple datasets. Code related to this chapter is available at: https://github.com/tksaha/con-s2v/tree/jointlearning Data related to this chapter are available at: https://www.dropbox.com/sh/ruhsi3c0unn0nko/AAAgVnZpojvXx9loQ21WP_MYa?dl=0

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings
PublisherSpringer Verlag
Pages753-769
Number of pages17
ISBN (Print)9783319712482
DOIs
Publication statusPublished - 1 Jan 2017
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2017 - Skopje, Macedonia, The Former Yugoslav Republic of
Duration: 18 Sep 201722 Sep 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10534 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2017
CountryMacedonia, The Former Yugoslav Republic of
CitySkopje
Period18/9/1722/9/17

    Fingerprint

Keywords

  • Embedding of sentences
  • Extra-sentential context
  • Sen2Vec

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Saha, T. K., Rayhan Joty, S., & Al Hasan, M. (2017). Con-S2V: A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings (pp. 753-769). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10534 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-71249-9_45