Sequence of the Most Informative Joints (SMIJ): A new representation for human skeletal action recognition

Ferda Ofli, Rizwan Chaudhry, Gregorij Kurillo, René Vidal, Ruzena Bajcsy

Research output: Chapter in Book/Report/Conference proceedingConference contribution

59 Citations (Scopus)

Abstract

Much of the existing work on action recognition combines simple features (e.g., joint angle trajectories, optical flow, spatio-temporal video features) with somewhat complex classifiers or dynamical models (e.g., kernel SVMs, HMMs, LDSs, deep belief networks). Although successful, these approaches represent an action with a set of parameters that usually do not have any physical meaning. As a consequence, such approaches do not provide any qualitative insight that relates an action to the actual motion of the body or its parts. For example, it is not necessarily the case that clapping can be correlated to hand motion or that walking can be correlated to a specific combination of motions from the feet, arms and body. In this paper, we propose a new representation of human actions called Sequence of the Most Informative Joints (SMIJ), which is extremely easy to interpret. At each time instant, we automatically select a few skeletal joints that are deemed to be the most informative for performing the current action. The selection of joints is based on highly interpretable measures such as the mean or variance of joint angles, maximum angular velocity of joints, etc. We then represent an action as a sequence of these most informative joints. Our experiments on multiple databases show that the proposed representation is very discriminative for the task of human action recognition and performs better than several state-of-the-art algorithms.

Original languageEnglish
Title of host publication2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2012
Pages8-13
Number of pages6
DOIs
Publication statusPublished - 20 Aug 2012
Event2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2012 - Providence, RI, United States
Duration: 16 Jun 201221 Jun 2012

Publication series

NameIEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops
ISSN (Print)2160-7508
ISSN (Electronic)2160-7516

Other

Other2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2012
CountryUnited States
CityProvidence, RI
Period16/6/1221/6/12

ASJC Scopus subject areas

  • Computer Vision and Pattern Recognition
  • Electrical and Electronic Engineering

Fingerprint Dive into the research topics of 'Sequence of the Most Informative Joints (SMIJ): A new representation for human skeletal action recognition'. Together they form a unique fingerprint.

  • Cite this

    Ofli, F., Chaudhry, R., Kurillo, G., Vidal, R., & Bajcsy, R. (2012). Sequence of the Most Informative Joints (SMIJ): A new representation for human skeletal action recognition. In 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPRW 2012 (pp. 8-13). [6239231] (IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops). https://doi.org/10.1109/CVPRW.2012.6239231