A generalized topic modeling approach for automatic document annotation

Suppawong Tuarob, Line C. Pouchard, Prasenjit Mitra, C. Lee Giles

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Ecological and environmental sciences have become more advanced and complex, requiring observational and experimental data from multiple places, times, and thematic scales to verify their hypotheses. Over time, such data have not only increased in amount, but also in diversity and heterogeneity of the data sources that spread throughout the world. This heterogeneity poses a huge challenge for scientists who have to manually search for desired data. ONEMercury has recently been implemented as part of the DataONE project to alleviate such problems and to serve as a portal for accessing environmental and observational data across the globe. ONEMercury harvests metadata records from multiple archives and repositories, and makes them searchable. However, harvested metadata records sometimes are poorly annotated or lacking meaningful keywords, which could impede effective retrieval. We propose a methodology that learns the annotation from well-annotated collections of metadata records to automatically annotate poorly annotated ones. The problem is first transformed into the tag recommendation problem with a controlled tag library. Then, two variants of an algorithm for automatic tag recommendation are presented. The experiments on four datasets of environmental science metadata records show that our methods perform well and also shed light on the natures of different datasets. We also discuss relevant topics such as using topical coherence to fine-tune parameters and experiments on cross-archive annotation.

Original languageEnglish
Pages (from-to)111-128
Number of pages18
JournalInternational Journal on Digital Libraries
Volume16
Issue number2
DOIs
Publication statusPublished - 7 Mar 2015

    Fingerprint

Keywords

  • Metadata annotation
  • Tag recommendation
  • Topic model

ASJC Scopus subject areas

  • Library and Information Sciences

Cite this