Con-S2V: A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec

Tanay Kumar Saha, Shafiq Rayhan Joty, Mohammad Al Hasan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present a novel approach to learn distributed representation of sentences from unlabeled data by modeling both content and context of a sentence. The content model learns sentence representation by predicting its words. On the other hand, the context model comprises a neighbor prediction component and a regularizer to model distributional and proximity hypotheses, respectively. We propose an online algorithm to train the model components jointly. We evaluate the models in a setup, where contextual information is available. The experimental results on tasks involving classification, clustering, and ranking of sentences show that our model outperforms the best existing models by a wide margin across multiple datasets. Code related to this chapter is available at: https://github.com/tksaha/con-s2v/tree/jointlearning Data related to this chapter are available at: https://www.dropbox.com/sh/ruhsi3c0unn0nko/AAAgVnZpojvXx9loQ21WP_MYa?dl=0

Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings
PublisherSpringer Verlag
Pages753-769
Number of pages17
ISBN (Print)9783319712482
DOIs
Publication statusPublished - 1 Jan 2017
EventEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2017 - Skopje, Macedonia, The Former Yugoslav Republic of
Duration: 18 Sep 201722 Sep 2017

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10534 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

OtherEuropean Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2017
CountryMacedonia, The Former Yugoslav Republic of
CitySkopje
Period18/9/1722/9/17

Fingerprint

Model
Online Algorithms
Component Model
Margin
Proximity
Context
Framework
Ranking
Clustering
Evaluate
Prediction
Experimental Results
Modeling

Keywords

  • Embedding of sentences
  • Extra-sentential context
  • Sen2Vec

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Saha, T. K., Rayhan Joty, S., & Al Hasan, M. (2017). Con-S2V: A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings (pp. 753-769). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10534 LNAI). Springer Verlag. https://doi.org/10.1007/978-3-319-71249-9_45

Con-S2V : A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec. / Saha, Tanay Kumar; Rayhan Joty, Shafiq; Al Hasan, Mohammad.

Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings. Springer Verlag, 2017. p. 753-769 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10534 LNAI).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Saha, TK, Rayhan Joty, S & Al Hasan, M 2017, Con-S2V: A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec. in Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10534 LNAI, Springer Verlag, pp. 753-769, European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML PKDD 2017, Skopje, Macedonia, The Former Yugoslav Republic of, 18/9/17. https://doi.org/10.1007/978-3-319-71249-9_45
Saha TK, Rayhan Joty S, Al Hasan M. Con-S2V: A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings. Springer Verlag. 2017. p. 753-769. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). https://doi.org/10.1007/978-3-319-71249-9_45
Saha, Tanay Kumar ; Rayhan Joty, Shafiq ; Al Hasan, Mohammad. / Con-S2V : A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec. Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings. Springer Verlag, 2017. pp. 753-769 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{3e0e3de66b614df5b53337a2f8499e38,
title = "Con-S2V: A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec",
abstract = "We present a novel approach to learn distributed representation of sentences from unlabeled data by modeling both content and context of a sentence. The content model learns sentence representation by predicting its words. On the other hand, the context model comprises a neighbor prediction component and a regularizer to model distributional and proximity hypotheses, respectively. We propose an online algorithm to train the model components jointly. We evaluate the models in a setup, where contextual information is available. The experimental results on tasks involving classification, clustering, and ranking of sentences show that our model outperforms the best existing models by a wide margin across multiple datasets. Code related to this chapter is available at: https://github.com/tksaha/con-s2v/tree/jointlearning Data related to this chapter are available at: https://www.dropbox.com/sh/ruhsi3c0unn0nko/AAAgVnZpojvXx9loQ21WP_MYa?dl=0",
keywords = "Embedding of sentences, Extra-sentential context, Sen2Vec",
author = "Saha, {Tanay Kumar} and {Rayhan Joty}, Shafiq and {Al Hasan}, Mohammad",
year = "2017",
month = "1",
day = "1",
doi = "10.1007/978-3-319-71249-9_45",
language = "English",
isbn = "9783319712482",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
publisher = "Springer Verlag",
pages = "753--769",
booktitle = "Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings",

}

TY - GEN

T1 - Con-S2V

T2 - A Generic Framework for Incorporating Extra-Sentential Context into Sen2Vec

AU - Saha, Tanay Kumar

AU - Rayhan Joty, Shafiq

AU - Al Hasan, Mohammad

PY - 2017/1/1

Y1 - 2017/1/1

N2 - We present a novel approach to learn distributed representation of sentences from unlabeled data by modeling both content and context of a sentence. The content model learns sentence representation by predicting its words. On the other hand, the context model comprises a neighbor prediction component and a regularizer to model distributional and proximity hypotheses, respectively. We propose an online algorithm to train the model components jointly. We evaluate the models in a setup, where contextual information is available. The experimental results on tasks involving classification, clustering, and ranking of sentences show that our model outperforms the best existing models by a wide margin across multiple datasets. Code related to this chapter is available at: https://github.com/tksaha/con-s2v/tree/jointlearning Data related to this chapter are available at: https://www.dropbox.com/sh/ruhsi3c0unn0nko/AAAgVnZpojvXx9loQ21WP_MYa?dl=0

AB - We present a novel approach to learn distributed representation of sentences from unlabeled data by modeling both content and context of a sentence. The content model learns sentence representation by predicting its words. On the other hand, the context model comprises a neighbor prediction component and a regularizer to model distributional and proximity hypotheses, respectively. We propose an online algorithm to train the model components jointly. We evaluate the models in a setup, where contextual information is available. The experimental results on tasks involving classification, clustering, and ranking of sentences show that our model outperforms the best existing models by a wide margin across multiple datasets. Code related to this chapter is available at: https://github.com/tksaha/con-s2v/tree/jointlearning Data related to this chapter are available at: https://www.dropbox.com/sh/ruhsi3c0unn0nko/AAAgVnZpojvXx9loQ21WP_MYa?dl=0

KW - Embedding of sentences

KW - Extra-sentential context

KW - Sen2Vec

UR - http://www.scopus.com/inward/record.url?scp=85040228633&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85040228633&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-71249-9_45

DO - 10.1007/978-3-319-71249-9_45

M3 - Conference contribution

AN - SCOPUS:85040228633

SN - 9783319712482

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 753

EP - 769

BT - Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2017, Proceedings

PB - Springer Verlag

ER -