Multicamera audio-visual analysis of dance figures using segmented body model

Ferda Ofli, Y. Demir, E. Erzin, Y. Yemez, A. M. Tekalp

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

We present a multi-camera system for audio-visual analysis of dance figures. The multi-view video of a dancing actor is acquired using 8 synchronized cameras. The motion capture technique of the proposed system is based on 3D tracking of the markers attached to the person's body in the scene. The resulting set of 3D points is then used to extract the body motion features as 3D displacement vectors whereas MFC coefficients serve as the audio features. In the multi-modal analysis phase, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features such as legs and arms, separately, to determine the recurrent elementary audio and body motion patterns in the first stage. Then in the second stage, we investigate the correlation of body motion patterns with audio patterns that can be used towards estimation and synthesis of realistic audio-driven body animation.

Original languageEnglish
Title of host publicationEuropean Signal Processing Conference
Pages2115-2119
Number of pages5
Publication statusPublished - 1 Dec 2007
Externally publishedYes
Event15th European Signal Processing Conference, EUSIPCO 2007 - Poznan, Poland
Duration: 3 Sep 20077 Sep 2007

Other

Other15th European Signal Processing Conference, EUSIPCO 2007
CountryPoland
CityPoznan
Period3/9/077/9/07

Fingerprint

Cameras
Modal analysis
Hidden Markov models
Animation

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Ofli, F., Demir, Y., Erzin, E., Yemez, Y., & Tekalp, A. M. (2007). Multicamera audio-visual analysis of dance figures using segmented body model. In European Signal Processing Conference (pp. 2115-2119)

Multicamera audio-visual analysis of dance figures using segmented body model. / Ofli, Ferda; Demir, Y.; Erzin, E.; Yemez, Y.; Tekalp, A. M.

European Signal Processing Conference. 2007. p. 2115-2119.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ofli, F, Demir, Y, Erzin, E, Yemez, Y & Tekalp, AM 2007, Multicamera audio-visual analysis of dance figures using segmented body model. in European Signal Processing Conference. pp. 2115-2119, 15th European Signal Processing Conference, EUSIPCO 2007, Poznan, Poland, 3/9/07.
Ofli F, Demir Y, Erzin E, Yemez Y, Tekalp AM. Multicamera audio-visual analysis of dance figures using segmented body model. In European Signal Processing Conference. 2007. p. 2115-2119
Ofli, Ferda ; Demir, Y. ; Erzin, E. ; Yemez, Y. ; Tekalp, A. M. / Multicamera audio-visual analysis of dance figures using segmented body model. European Signal Processing Conference. 2007. pp. 2115-2119
@inproceedings{4c6514a40ffe4fbf823c1daac8e3621e,
title = "Multicamera audio-visual analysis of dance figures using segmented body model",
abstract = "We present a multi-camera system for audio-visual analysis of dance figures. The multi-view video of a dancing actor is acquired using 8 synchronized cameras. The motion capture technique of the proposed system is based on 3D tracking of the markers attached to the person's body in the scene. The resulting set of 3D points is then used to extract the body motion features as 3D displacement vectors whereas MFC coefficients serve as the audio features. In the multi-modal analysis phase, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features such as legs and arms, separately, to determine the recurrent elementary audio and body motion patterns in the first stage. Then in the second stage, we investigate the correlation of body motion patterns with audio patterns that can be used towards estimation and synthesis of realistic audio-driven body animation.",
author = "Ferda Ofli and Y. Demir and E. Erzin and Y. Yemez and Tekalp, {A. M.}",
year = "2007",
month = "12",
day = "1",
language = "English",
isbn = "9788392134022",
pages = "2115--2119",
booktitle = "European Signal Processing Conference",

}

TY - GEN

T1 - Multicamera audio-visual analysis of dance figures using segmented body model

AU - Ofli, Ferda

AU - Demir, Y.

AU - Erzin, E.

AU - Yemez, Y.

AU - Tekalp, A. M.

PY - 2007/12/1

Y1 - 2007/12/1

N2 - We present a multi-camera system for audio-visual analysis of dance figures. The multi-view video of a dancing actor is acquired using 8 synchronized cameras. The motion capture technique of the proposed system is based on 3D tracking of the markers attached to the person's body in the scene. The resulting set of 3D points is then used to extract the body motion features as 3D displacement vectors whereas MFC coefficients serve as the audio features. In the multi-modal analysis phase, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features such as legs and arms, separately, to determine the recurrent elementary audio and body motion patterns in the first stage. Then in the second stage, we investigate the correlation of body motion patterns with audio patterns that can be used towards estimation and synthesis of realistic audio-driven body animation.

AB - We present a multi-camera system for audio-visual analysis of dance figures. The multi-view video of a dancing actor is acquired using 8 synchronized cameras. The motion capture technique of the proposed system is based on 3D tracking of the markers attached to the person's body in the scene. The resulting set of 3D points is then used to extract the body motion features as 3D displacement vectors whereas MFC coefficients serve as the audio features. In the multi-modal analysis phase, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features such as legs and arms, separately, to determine the recurrent elementary audio and body motion patterns in the first stage. Then in the second stage, we investigate the correlation of body motion patterns with audio patterns that can be used towards estimation and synthesis of realistic audio-driven body animation.

UR - http://www.scopus.com/inward/record.url?scp=84863754138&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84863754138&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9788392134022

SP - 2115

EP - 2119

BT - European Signal Processing Conference

ER -