Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification

Jingbin Wang, Yihua Zhou, Kanghong Duan, Jim Jing Yan Wang, Halima Bensmail

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

In this paper we study the problem of learning from multiple modal data for purpose of document classification. In this problem, each document is composed two different modals of data, i.e., An image and a text. Cross-modal factor analysis (CFA) has been proposed to project the two different modals of data to a shared data space, so that the classification of a image or a text can be performed directly in this space. A disadvantage of CFA is that it has ignored the supervision information. In this paper, we improve CFA by incorporating the supervision information to represent and classify both image and text modals of documents. We project both image and text data to a shared data space by factor analysis, and then train a class label predictor in the shared space to use the class label information. The factor analysis parameter and the predictor parameter are learned jointly by solving one single objective function. With this objective function, we minimize the distance between the projections of image and text of the same document, and the classification error of the projection measured by hinge loss function. The objective function is optimized by an alternate optimization strategy in an iterative algorithm. Experiments in two different multiple modal document data sets show the advantage of the proposed algorithm over other CFA methods.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1882-1888
Number of pages7
ISBN (Electronic)9781479986965
DOIs
Publication statusPublished - 12 Jan 2016
EventIEEE International Conference on Systems, Man, and Cybernetics, SMC 2015 - Kowloon Tong, Hong Kong
Duration: 9 Oct 201512 Oct 2015

Publication series

NameProceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015

Other

OtherIEEE International Conference on Systems, Man, and Cybernetics, SMC 2015
CountryHong Kong
CityKowloon Tong
Period9/10/1512/10/15

Keywords

  • Cross-modal factor analysis
  • Multiple modal learning
  • Supervised learning

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Energy Engineering and Power Technology
  • Information Systems and Management
  • Control and Systems Engineering

Fingerprint Dive into the research topics of 'Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification'. Together they form a unique fingerprint.

  • Cite this

    Wang, J., Zhou, Y., Duan, K., Wang, J. J. Y., & Bensmail, H. (2016). Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification. In Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015 (pp. 1882-1888). [7379461] (Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC.2015.329