Multicamera audio-visual analysis of dance figures using segmented body model

F. Ofli, Y. Demir, E. Erzin, Y. Yemez, A. M. Tekalp

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We present a multi-camera system for audio-visual analysis of dance figures. The multi-view video of a dancing actor is acquired using 8 synchronized cameras. The motion capture technique of the proposed system is based on 3D tracking of the markers attached to the person's body in the scene. The resulting set of 3D points is then used to extract the body motion features as 3D displacement vectors whereas MFC coefficients serve as the audio features. In the multi-modal analysis phase, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features such as legs and arms, separately, to determine the recurrent elementary audio and body motion patterns in the first stage. Then in the second stage, we investigate the correlation of body motion patterns with audio patterns that can be used towards estimation and synthesis of realistic audio-driven body animation.

Original languageEnglish
Title of host publication15th European Signal Processing Conference, EUSIPCO 2007 - Proceedings
Pages2115-2119
Number of pages5
Publication statusPublished - 1 Dec 2007
Event15th European Signal Processing Conference, EUSIPCO 2007 - Poznan, Poland
Duration: 3 Sep 20077 Sep 2007

Publication series

NameEuropean Signal Processing Conference
ISSN (Print)2219-5491

Other

Other15th European Signal Processing Conference, EUSIPCO 2007
CountryPoland
CityPoznan
Period3/9/077/9/07

    Fingerprint

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Ofli, F., Demir, Y., Erzin, E., Yemez, Y., & Tekalp, A. M. (2007). Multicamera audio-visual analysis of dance figures using segmented body model. In 15th European Signal Processing Conference, EUSIPCO 2007 - Proceedings (pp. 2115-2119). (European Signal Processing Conference).