Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification

Jingbin Wang, Yihua Zhou, Kanghong Duan, Jim Jing Yan Wang, Halima Bensmail

Research output: Chapter in Book/Report/Conference proceedingConference contribution

14 Citations (Scopus)

Abstract

In this paper we study the problem of learning from multiple modal data for purpose of document classification. In this problem, each document is composed two different modals of data, i.e., An image and a text. Cross-modal factor analysis (CFA) has been proposed to project the two different modals of data to a shared data space, so that the classification of a image or a text can be performed directly in this space. A disadvantage of CFA is that it has ignored the supervision information. In this paper, we improve CFA by incorporating the supervision information to represent and classify both image and text modals of documents. We project both image and text data to a shared data space by factor analysis, and then train a class label predictor in the shared space to use the class label information. The factor analysis parameter and the predictor parameter are learned jointly by solving one single objective function. With this objective function, we minimize the distance between the projections of image and text of the same document, and the classification error of the projection measured by hinge loss function. The objective function is optimized by an alternate optimization strategy in an iterative algorithm. Experiments in two different multiple modal document data sets show the advantage of the proposed algorithm over other CFA methods.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1882-1888
Number of pages7
ISBN (Electronic)9781479986965
DOIs
Publication statusPublished - 12 Jan 2016
EventIEEE International Conference on Systems, Man, and Cybernetics, SMC 2015 - Kowloon Tong, Hong Kong
Duration: 9 Oct 201512 Oct 2015

Other

OtherIEEE International Conference on Systems, Man, and Cybernetics, SMC 2015
CountryHong Kong
CityKowloon Tong
Period9/10/1512/10/15

Fingerprint

Factor analysis
Labels
Hinges
Objective function
Experiments

Keywords

  • Cross-modal factor analysis
  • Multiple modal learning
  • Supervised learning

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Energy Engineering and Power Technology
  • Information Systems and Management
  • Control and Systems Engineering

Cite this

Wang, J., Zhou, Y., Duan, K., Wang, J. J. Y., & Bensmail, H. (2016). Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification. In Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015 (pp. 1882-1888). [7379461] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SMC.2015.329

Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification. / Wang, Jingbin; Zhou, Yihua; Duan, Kanghong; Wang, Jim Jing Yan; Bensmail, Halima.

Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015. Institute of Electrical and Electronics Engineers Inc., 2016. p. 1882-1888 7379461.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Wang, J, Zhou, Y, Duan, K, Wang, JJY & Bensmail, H 2016, Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification. in Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015., 7379461, Institute of Electrical and Electronics Engineers Inc., pp. 1882-1888, IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015, Kowloon Tong, Hong Kong, 9/10/15. https://doi.org/10.1109/SMC.2015.329
Wang J, Zhou Y, Duan K, Wang JJY, Bensmail H. Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification. In Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015. Institute of Electrical and Electronics Engineers Inc. 2016. p. 1882-1888. 7379461 https://doi.org/10.1109/SMC.2015.329
Wang, Jingbin ; Zhou, Yihua ; Duan, Kanghong ; Wang, Jim Jing Yan ; Bensmail, Halima. / Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification. Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 1882-1888
@inproceedings{5e923dc22c5244a796d41dd70e259622,
title = "Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification",
abstract = "In this paper we study the problem of learning from multiple modal data for purpose of document classification. In this problem, each document is composed two different modals of data, i.e., An image and a text. Cross-modal factor analysis (CFA) has been proposed to project the two different modals of data to a shared data space, so that the classification of a image or a text can be performed directly in this space. A disadvantage of CFA is that it has ignored the supervision information. In this paper, we improve CFA by incorporating the supervision information to represent and classify both image and text modals of documents. We project both image and text data to a shared data space by factor analysis, and then train a class label predictor in the shared space to use the class label information. The factor analysis parameter and the predictor parameter are learned jointly by solving one single objective function. With this objective function, we minimize the distance between the projections of image and text of the same document, and the classification error of the projection measured by hinge loss function. The objective function is optimized by an alternate optimization strategy in an iterative algorithm. Experiments in two different multiple modal document data sets show the advantage of the proposed algorithm over other CFA methods.",
keywords = "Cross-modal factor analysis, Multiple modal learning, Supervised learning",
author = "Jingbin Wang and Yihua Zhou and Kanghong Duan and Wang, {Jim Jing Yan} and Halima Bensmail",
year = "2016",
month = "1",
day = "12",
doi = "10.1109/SMC.2015.329",
language = "English",
pages = "1882--1888",
booktitle = "Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - Supervised Cross-Modal Factor Analysis for Multiple Modal Data Classification

AU - Wang, Jingbin

AU - Zhou, Yihua

AU - Duan, Kanghong

AU - Wang, Jim Jing Yan

AU - Bensmail, Halima

PY - 2016/1/12

Y1 - 2016/1/12

N2 - In this paper we study the problem of learning from multiple modal data for purpose of document classification. In this problem, each document is composed two different modals of data, i.e., An image and a text. Cross-modal factor analysis (CFA) has been proposed to project the two different modals of data to a shared data space, so that the classification of a image or a text can be performed directly in this space. A disadvantage of CFA is that it has ignored the supervision information. In this paper, we improve CFA by incorporating the supervision information to represent and classify both image and text modals of documents. We project both image and text data to a shared data space by factor analysis, and then train a class label predictor in the shared space to use the class label information. The factor analysis parameter and the predictor parameter are learned jointly by solving one single objective function. With this objective function, we minimize the distance between the projections of image and text of the same document, and the classification error of the projection measured by hinge loss function. The objective function is optimized by an alternate optimization strategy in an iterative algorithm. Experiments in two different multiple modal document data sets show the advantage of the proposed algorithm over other CFA methods.

AB - In this paper we study the problem of learning from multiple modal data for purpose of document classification. In this problem, each document is composed two different modals of data, i.e., An image and a text. Cross-modal factor analysis (CFA) has been proposed to project the two different modals of data to a shared data space, so that the classification of a image or a text can be performed directly in this space. A disadvantage of CFA is that it has ignored the supervision information. In this paper, we improve CFA by incorporating the supervision information to represent and classify both image and text modals of documents. We project both image and text data to a shared data space by factor analysis, and then train a class label predictor in the shared space to use the class label information. The factor analysis parameter and the predictor parameter are learned jointly by solving one single objective function. With this objective function, we minimize the distance between the projections of image and text of the same document, and the classification error of the projection measured by hinge loss function. The objective function is optimized by an alternate optimization strategy in an iterative algorithm. Experiments in two different multiple modal document data sets show the advantage of the proposed algorithm over other CFA methods.

KW - Cross-modal factor analysis

KW - Multiple modal learning

KW - Supervised learning

UR - http://www.scopus.com/inward/record.url?scp=84964444011&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964444011&partnerID=8YFLogxK

U2 - 10.1109/SMC.2015.329

DO - 10.1109/SMC.2015.329

M3 - Conference contribution

AN - SCOPUS:84964444011

SP - 1882

EP - 1888

BT - Proceedings - 2015 IEEE International Conference on Systems, Man, and Cybernetics, SMC 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -