Sequence of the most informative joints (SMIJ)

A new representation for human skeletal action recognition

Ferda Ofli, Rizwan Chaudhry, Gregorij Kurillo, René Vidal, Ruzena Bajcsy

Research output: Contribution to journalArticle

164 Citations (Scopus)

Abstract

Much of the existing work on action recognition combines simple features with complex classifiers or models to represent an action. Parameters of such models usually do not have any physical meaning nor do they provide any qualitative insight relating the action to the actual motion of the body or its parts. In this paper, we propose a new representation of human actions called sequence of the most informative joints (SMIJ), which is extremely easy to interpret. At each time instant, we automatically select a few skeletal joints that are deemed to be the most informative for performing the current action based on highly interpretable measures such as the mean or variance of joint angle trajectories. We then represent the action as a sequence of these most informative joints. Experiments on multiple databases show that the SMIJ representation is discriminative for human action recognition and performs better than several state-of-the-art algorithms.

Original languageEnglish
Pages (from-to)24-38
Number of pages15
JournalJournal of Visual Communication and Image Representation
Volume25
Issue number1
DOIs
Publication statusPublished - 1 Jan 2014
Externally publishedYes

Fingerprint

Classifiers
Trajectories
Experiments

Keywords

  • Bag-of-words
  • Berkeley MHAD
  • Cross-database generalization
  • HDM05
  • Human action recognition
  • Human action representation
  • Informative joints
  • Linear dynamical systems
  • MSR Action3D
  • Normalized edit distance

ASJC Scopus subject areas

  • Electrical and Electronic Engineering
  • Media Technology
  • Computer Vision and Pattern Recognition
  • Signal Processing

Cite this

Sequence of the most informative joints (SMIJ) : A new representation for human skeletal action recognition. / Ofli, Ferda; Chaudhry, Rizwan; Kurillo, Gregorij; Vidal, René; Bajcsy, Ruzena.

In: Journal of Visual Communication and Image Representation, Vol. 25, No. 1, 01.01.2014, p. 24-38.

Research output: Contribution to journalArticle

Ofli, Ferda ; Chaudhry, Rizwan ; Kurillo, Gregorij ; Vidal, René ; Bajcsy, Ruzena. / Sequence of the most informative joints (SMIJ) : A new representation for human skeletal action recognition. In: Journal of Visual Communication and Image Representation. 2014 ; Vol. 25, No. 1. pp. 24-38.
@article{29abc03b317d418fa74aa3869e44432a,
title = "Sequence of the most informative joints (SMIJ): A new representation for human skeletal action recognition",
abstract = "Much of the existing work on action recognition combines simple features with complex classifiers or models to represent an action. Parameters of such models usually do not have any physical meaning nor do they provide any qualitative insight relating the action to the actual motion of the body or its parts. In this paper, we propose a new representation of human actions called sequence of the most informative joints (SMIJ), which is extremely easy to interpret. At each time instant, we automatically select a few skeletal joints that are deemed to be the most informative for performing the current action based on highly interpretable measures such as the mean or variance of joint angle trajectories. We then represent the action as a sequence of these most informative joints. Experiments on multiple databases show that the SMIJ representation is discriminative for human action recognition and performs better than several state-of-the-art algorithms.",
keywords = "Bag-of-words, Berkeley MHAD, Cross-database generalization, HDM05, Human action recognition, Human action representation, Informative joints, Linear dynamical systems, MSR Action3D, Normalized edit distance",
author = "Ferda Ofli and Rizwan Chaudhry and Gregorij Kurillo and Ren{\'e} Vidal and Ruzena Bajcsy",
year = "2014",
month = "1",
day = "1",
doi = "10.1016/j.jvcir.2013.04.007",
language = "English",
volume = "25",
pages = "24--38",
journal = "Journal of Visual Communication and Image Representation",
issn = "1047-3203",
publisher = "Academic Press Inc.",
number = "1",

}

TY - JOUR

T1 - Sequence of the most informative joints (SMIJ)

T2 - A new representation for human skeletal action recognition

AU - Ofli, Ferda

AU - Chaudhry, Rizwan

AU - Kurillo, Gregorij

AU - Vidal, René

AU - Bajcsy, Ruzena

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Much of the existing work on action recognition combines simple features with complex classifiers or models to represent an action. Parameters of such models usually do not have any physical meaning nor do they provide any qualitative insight relating the action to the actual motion of the body or its parts. In this paper, we propose a new representation of human actions called sequence of the most informative joints (SMIJ), which is extremely easy to interpret. At each time instant, we automatically select a few skeletal joints that are deemed to be the most informative for performing the current action based on highly interpretable measures such as the mean or variance of joint angle trajectories. We then represent the action as a sequence of these most informative joints. Experiments on multiple databases show that the SMIJ representation is discriminative for human action recognition and performs better than several state-of-the-art algorithms.

AB - Much of the existing work on action recognition combines simple features with complex classifiers or models to represent an action. Parameters of such models usually do not have any physical meaning nor do they provide any qualitative insight relating the action to the actual motion of the body or its parts. In this paper, we propose a new representation of human actions called sequence of the most informative joints (SMIJ), which is extremely easy to interpret. At each time instant, we automatically select a few skeletal joints that are deemed to be the most informative for performing the current action based on highly interpretable measures such as the mean or variance of joint angle trajectories. We then represent the action as a sequence of these most informative joints. Experiments on multiple databases show that the SMIJ representation is discriminative for human action recognition and performs better than several state-of-the-art algorithms.

KW - Bag-of-words

KW - Berkeley MHAD

KW - Cross-database generalization

KW - HDM05

KW - Human action recognition

KW - Human action representation

KW - Informative joints

KW - Linear dynamical systems

KW - MSR Action3D

KW - Normalized edit distance

UR - http://www.scopus.com/inward/record.url?scp=84891625198&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84891625198&partnerID=8YFLogxK

U2 - 10.1016/j.jvcir.2013.04.007

DO - 10.1016/j.jvcir.2013.04.007

M3 - Article

VL - 25

SP - 24

EP - 38

JO - Journal of Visual Communication and Image Representation

JF - Journal of Visual Communication and Image Representation

SN - 1047-3203

IS - 1

ER -