Transfer learning for industrial applications of named entity recognition

Lingzhen Chen, Alessandro Moschitti, Giuseppe Castellucci, Andrea Favalli, Raniero Romagnoli

Research output: Contribution to journalConference article

Abstract

In this paper, we propose a Transfer Learning technique for Named Entity Recognition that is able to flexibly deal with domain changes. The proposed technique is able to manage both the case when the set of named entities does not change and the case when the set of named entities changes in the target domain. In particular, we focus on the case when the target data contains only the annotation of a target named entity, and the source data is no longer available for the target task. Our solution consists in transferring the parameters from a source model, which are then fine-tuned with the target data. The model architecture is modified when recognizing a new category by adding properly new neurons to the model. Our experiments show that it is possible to effectively transfer learned parameters in both the scenarios, resulting in strong performances over the target categories without degrading the performances on the other named entities.

Original languageEnglish
Pages (from-to)129-140
Number of pages12
JournalCEUR Workshop Proceedings
Volume2244
Publication statusPublished - 1 Jan 2018
Event2nd Workshop on Natural Language for Artificial Intelligence, NL4AI 2018 - Trento, Italy
Duration: 22 Nov 201823 Nov 2018

Fingerprint

Industrial applications
Neurons
Experiments

Keywords

  • Named entity recognition
  • Recurrent neural network
  • Sequence labeling
  • Transfer learning

ASJC Scopus subject areas

  • Computer Science(all)

Cite this

Chen, L., Moschitti, A., Castellucci, G., Favalli, A., & Romagnoli, R. (2018). Transfer learning for industrial applications of named entity recognition. CEUR Workshop Proceedings, 2244, 129-140.

Transfer learning for industrial applications of named entity recognition. / Chen, Lingzhen; Moschitti, Alessandro; Castellucci, Giuseppe; Favalli, Andrea; Romagnoli, Raniero.

In: CEUR Workshop Proceedings, Vol. 2244, 01.01.2018, p. 129-140.

Research output: Contribution to journalConference article

Chen, L, Moschitti, A, Castellucci, G, Favalli, A & Romagnoli, R 2018, 'Transfer learning for industrial applications of named entity recognition', CEUR Workshop Proceedings, vol. 2244, pp. 129-140.
Chen L, Moschitti A, Castellucci G, Favalli A, Romagnoli R. Transfer learning for industrial applications of named entity recognition. CEUR Workshop Proceedings. 2018 Jan 1;2244:129-140.
Chen, Lingzhen ; Moschitti, Alessandro ; Castellucci, Giuseppe ; Favalli, Andrea ; Romagnoli, Raniero. / Transfer learning for industrial applications of named entity recognition. In: CEUR Workshop Proceedings. 2018 ; Vol. 2244. pp. 129-140.
@article{9a88430e7e3548d785391e704ec87c7b,
title = "Transfer learning for industrial applications of named entity recognition",
abstract = "In this paper, we propose a Transfer Learning technique for Named Entity Recognition that is able to flexibly deal with domain changes. The proposed technique is able to manage both the case when the set of named entities does not change and the case when the set of named entities changes in the target domain. In particular, we focus on the case when the target data contains only the annotation of a target named entity, and the source data is no longer available for the target task. Our solution consists in transferring the parameters from a source model, which are then fine-tuned with the target data. The model architecture is modified when recognizing a new category by adding properly new neurons to the model. Our experiments show that it is possible to effectively transfer learned parameters in both the scenarios, resulting in strong performances over the target categories without degrading the performances on the other named entities.",
keywords = "Named entity recognition, Recurrent neural network, Sequence labeling, Transfer learning",
author = "Lingzhen Chen and Alessandro Moschitti and Giuseppe Castellucci and Andrea Favalli and Raniero Romagnoli",
year = "2018",
month = "1",
day = "1",
language = "English",
volume = "2244",
pages = "129--140",
journal = "CEUR Workshop Proceedings",
issn = "1613-0073",
publisher = "CEUR-WS",

}

TY - JOUR

T1 - Transfer learning for industrial applications of named entity recognition

AU - Chen, Lingzhen

AU - Moschitti, Alessandro

AU - Castellucci, Giuseppe

AU - Favalli, Andrea

AU - Romagnoli, Raniero

PY - 2018/1/1

Y1 - 2018/1/1

N2 - In this paper, we propose a Transfer Learning technique for Named Entity Recognition that is able to flexibly deal with domain changes. The proposed technique is able to manage both the case when the set of named entities does not change and the case when the set of named entities changes in the target domain. In particular, we focus on the case when the target data contains only the annotation of a target named entity, and the source data is no longer available for the target task. Our solution consists in transferring the parameters from a source model, which are then fine-tuned with the target data. The model architecture is modified when recognizing a new category by adding properly new neurons to the model. Our experiments show that it is possible to effectively transfer learned parameters in both the scenarios, resulting in strong performances over the target categories without degrading the performances on the other named entities.

AB - In this paper, we propose a Transfer Learning technique for Named Entity Recognition that is able to flexibly deal with domain changes. The proposed technique is able to manage both the case when the set of named entities does not change and the case when the set of named entities changes in the target domain. In particular, we focus on the case when the target data contains only the annotation of a target named entity, and the source data is no longer available for the target task. Our solution consists in transferring the parameters from a source model, which are then fine-tuned with the target data. The model architecture is modified when recognizing a new category by adding properly new neurons to the model. Our experiments show that it is possible to effectively transfer learned parameters in both the scenarios, resulting in strong performances over the target categories without degrading the performances on the other named entities.

KW - Named entity recognition

KW - Recurrent neural network

KW - Sequence labeling

KW - Transfer learning

UR - http://www.scopus.com/inward/record.url?scp=85057850605&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057850605&partnerID=8YFLogxK

M3 - Conference article

VL - 2244

SP - 129

EP - 140

JO - CEUR Workshop Proceedings

JF - CEUR Workshop Proceedings

SN - 1613-0073

ER -