Çok bakişli i̇şitsel-görsel dans verilerinin analizi ve sentezi

Translated title of the contribution: Analysis and synthesis of multiview audio-visual dance figures

Ferda Ofli, Y. Demir, C. Canton-Ferrer, J. Tilmanne, K. Balci, E. Bozkurt, I. Kizoǧlu, Y. Yemez, E. Erzin, A. M. Tekalp, L. Akarun, A. T. Erdem

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper presents a framework for audio-driven human body motion analysis and synthesis. The video is analyzed to capture the time-varying posture of the dancer's body whereas the musical audio signal is processed to extract the beat information. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Body movements of the dancer are characterized by a set of recurring semantic motion patterns, i.e., dance figures. Each dance figure is modeled in a supervised manner with a set of HMM (Hidden Markov Model) structures and the associated beat frequency. In synthesis, given an audio signal of a learned musical type, the motion parameters of the corresponding dance figures are synthesized via the trained HMM structures in synchrony with the input audio signal based on the estimated tempo information. Finally, the generated motion parameters are animated along with the musical audio using a graphical animation tool. Experimental results demonstrate the effectiveness of the proposed framework.

Original languageUndefined/Unknown
Title of host publication2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU
DOIs
Publication statusPublished - 26 Nov 2008
Externally publishedYes
Event2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU - Aydin, Turkey
Duration: 20 Apr 200822 Apr 2008

Other

Other
CountryTurkey
CityAydin
Period20/4/0822/4/08

Fingerprint

Hidden Markov models
Model structures
dance
Animation
video
Semantics
Annealing
semantics
Motion analysis

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering
  • Communication

Cite this

Ofli, F., Demir, Y., Canton-Ferrer, C., Tilmanne, J., Balci, K., Bozkurt, E., ... Erdem, A. T. (2008). Çok bakişli i̇şitsel-görsel dans verilerinin analizi ve sentezi. In 2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU [4632725] https://doi.org/10.1109/SIU.2008.4632725

Çok bakişli i̇şitsel-görsel dans verilerinin analizi ve sentezi. / Ofli, Ferda; Demir, Y.; Canton-Ferrer, C.; Tilmanne, J.; Balci, K.; Bozkurt, E.; Kizoǧlu, I.; Yemez, Y.; Erzin, E.; Tekalp, A. M.; Akarun, L.; Erdem, A. T.

2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU. 2008. 4632725.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ofli, F, Demir, Y, Canton-Ferrer, C, Tilmanne, J, Balci, K, Bozkurt, E, Kizoǧlu, I, Yemez, Y, Erzin, E, Tekalp, AM, Akarun, L & Erdem, AT 2008, Çok bakişli i̇şitsel-görsel dans verilerinin analizi ve sentezi. in 2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU., 4632725, Aydin, Turkey, 20/4/08. https://doi.org/10.1109/SIU.2008.4632725
Ofli F, Demir Y, Canton-Ferrer C, Tilmanne J, Balci K, Bozkurt E et al. Çok bakişli i̇şitsel-görsel dans verilerinin analizi ve sentezi. In 2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU. 2008. 4632725 https://doi.org/10.1109/SIU.2008.4632725
Ofli, Ferda ; Demir, Y. ; Canton-Ferrer, C. ; Tilmanne, J. ; Balci, K. ; Bozkurt, E. ; Kizoǧlu, I. ; Yemez, Y. ; Erzin, E. ; Tekalp, A. M. ; Akarun, L. ; Erdem, A. T. / Çok bakişli i̇şitsel-görsel dans verilerinin analizi ve sentezi. 2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU. 2008.
@inproceedings{6db9f1536c614ebd80827d29a2742c62,
title = "{\cC}ok bakişli i̇şitsel-g{\"o}rsel dans verilerinin analizi ve sentezi",
abstract = "This paper presents a framework for audio-driven human body motion analysis and synthesis. The video is analyzed to capture the time-varying posture of the dancer's body whereas the musical audio signal is processed to extract the beat information. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Body movements of the dancer are characterized by a set of recurring semantic motion patterns, i.e., dance figures. Each dance figure is modeled in a supervised manner with a set of HMM (Hidden Markov Model) structures and the associated beat frequency. In synthesis, given an audio signal of a learned musical type, the motion parameters of the corresponding dance figures are synthesized via the trained HMM structures in synchrony with the input audio signal based on the estimated tempo information. Finally, the generated motion parameters are animated along with the musical audio using a graphical animation tool. Experimental results demonstrate the effectiveness of the proposed framework.",
author = "Ferda Ofli and Y. Demir and C. Canton-Ferrer and J. Tilmanne and K. Balci and E. Bozkurt and I. Kizoǧlu and Y. Yemez and E. Erzin and Tekalp, {A. M.} and L. Akarun and Erdem, {A. T.}",
year = "2008",
month = "11",
day = "26",
doi = "10.1109/SIU.2008.4632725",
language = "Undefined/Unknown",
isbn = "9781424419999",
booktitle = "2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU",

}

TY - GEN

T1 - Çok bakişli i̇şitsel-görsel dans verilerinin analizi ve sentezi

AU - Ofli, Ferda

AU - Demir, Y.

AU - Canton-Ferrer, C.

AU - Tilmanne, J.

AU - Balci, K.

AU - Bozkurt, E.

AU - Kizoǧlu, I.

AU - Yemez, Y.

AU - Erzin, E.

AU - Tekalp, A. M.

AU - Akarun, L.

AU - Erdem, A. T.

PY - 2008/11/26

Y1 - 2008/11/26

N2 - This paper presents a framework for audio-driven human body motion analysis and synthesis. The video is analyzed to capture the time-varying posture of the dancer's body whereas the musical audio signal is processed to extract the beat information. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Body movements of the dancer are characterized by a set of recurring semantic motion patterns, i.e., dance figures. Each dance figure is modeled in a supervised manner with a set of HMM (Hidden Markov Model) structures and the associated beat frequency. In synthesis, given an audio signal of a learned musical type, the motion parameters of the corresponding dance figures are synthesized via the trained HMM structures in synchrony with the input audio signal based on the estimated tempo information. Finally, the generated motion parameters are animated along with the musical audio using a graphical animation tool. Experimental results demonstrate the effectiveness of the proposed framework.

AB - This paper presents a framework for audio-driven human body motion analysis and synthesis. The video is analyzed to capture the time-varying posture of the dancer's body whereas the musical audio signal is processed to extract the beat information. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Body movements of the dancer are characterized by a set of recurring semantic motion patterns, i.e., dance figures. Each dance figure is modeled in a supervised manner with a set of HMM (Hidden Markov Model) structures and the associated beat frequency. In synthesis, given an audio signal of a learned musical type, the motion parameters of the corresponding dance figures are synthesized via the trained HMM structures in synchrony with the input audio signal based on the estimated tempo information. Finally, the generated motion parameters are animated along with the musical audio using a graphical animation tool. Experimental results demonstrate the effectiveness of the proposed framework.

UR - http://www.scopus.com/inward/record.url?scp=56449084971&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=56449084971&partnerID=8YFLogxK

U2 - 10.1109/SIU.2008.4632725

DO - 10.1109/SIU.2008.4632725

M3 - Conference contribution

SN - 9781424419999

BT - 2008 IEEE 16th Signal Processing, Communication and Applications Conference, SIU

ER -