İşitsel-görsel dans verilerinin birleşik i̇linti analizi

Translated title of the contribution: Joint correlation analysis of audio-visual dance figures

Ferda Ofli, Y. Demir, E. Erzin, Y. Yemez, A. M. Tekalp

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper we present a framework for analysis of dance figures from audio-visual data. Our audio-visual data is the multiview video of a dancing actor which is acquired using 8 synchronized cameras. The multi-camera motion capture technique of this framework is based on 3D tracking of the markers attached to the dancer's body, using stereo color information. The extracted 3D points are used to calculate the body motion features as 3D displacement vectors. On the other hand, MFC coefficients serve as the audio features. In the first stage of the two stage analysis task, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features, separately, to extract the recurrent elementary audio and body motion patterns. In the second stage, the correlation of body motion patterns with audio patterns is investigated to create a correlation model that can be used during the synthesis of an audio-driven body animation.

Original languageUndefined/Unknown
Title of host publication2007 IEEE 15th Signal Processing and Communications Applications, SIU
DOIs
Publication statusPublished - 1 Dec 2007
Externally publishedYes
Event2007 IEEE 15th Signal Processing and Communications Applications, SIU - Eskisehir, Turkey
Duration: 11 Jun 200713 Jun 2007

Other

Other
CountryTurkey
CityEskisehir
Period11/6/0713/6/07

Fingerprint

dance
Cameras
Hidden Markov models
Animation
Color
video

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Communication
  • Signal Processing

Cite this

Ofli, F., Demir, Y., Erzin, E., Yemez, Y., & Tekalp, A. M. (2007). İşitsel-görsel dans verilerinin birleşik i̇linti analizi. In 2007 IEEE 15th Signal Processing and Communications Applications, SIU [4298617] https://doi.org/10.1109/SIU.2007.4298617

İşitsel-görsel dans verilerinin birleşik i̇linti analizi. / Ofli, Ferda; Demir, Y.; Erzin, E.; Yemez, Y.; Tekalp, A. M.

2007 IEEE 15th Signal Processing and Communications Applications, SIU. 2007. 4298617.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Ofli, F, Demir, Y, Erzin, E, Yemez, Y & Tekalp, AM 2007, İşitsel-görsel dans verilerinin birleşik i̇linti analizi. in 2007 IEEE 15th Signal Processing and Communications Applications, SIU., 4298617, Eskisehir, Turkey, 11/6/07. https://doi.org/10.1109/SIU.2007.4298617
Ofli F, Demir Y, Erzin E, Yemez Y, Tekalp AM. İşitsel-görsel dans verilerinin birleşik i̇linti analizi. In 2007 IEEE 15th Signal Processing and Communications Applications, SIU. 2007. 4298617 https://doi.org/10.1109/SIU.2007.4298617
Ofli, Ferda ; Demir, Y. ; Erzin, E. ; Yemez, Y. ; Tekalp, A. M. / İşitsel-görsel dans verilerinin birleşik i̇linti analizi. 2007 IEEE 15th Signal Processing and Communications Applications, SIU. 2007.
@inproceedings{472d80b4008040fe950978463497cd8a,
title = "İşitsel-g{\"o}rsel dans verilerinin birleşik i̇linti analizi",
abstract = "In this paper we present a framework for analysis of dance figures from audio-visual data. Our audio-visual data is the multiview video of a dancing actor which is acquired using 8 synchronized cameras. The multi-camera motion capture technique of this framework is based on 3D tracking of the markers attached to the dancer's body, using stereo color information. The extracted 3D points are used to calculate the body motion features as 3D displacement vectors. On the other hand, MFC coefficients serve as the audio features. In the first stage of the two stage analysis task, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features, separately, to extract the recurrent elementary audio and body motion patterns. In the second stage, the correlation of body motion patterns with audio patterns is investigated to create a correlation model that can be used during the synthesis of an audio-driven body animation.",
author = "Ferda Ofli and Y. Demir and E. Erzin and Y. Yemez and Tekalp, {A. M.}",
year = "2007",
month = "12",
day = "1",
doi = "10.1109/SIU.2007.4298617",
language = "Undefined/Unknown",
isbn = "1424407192",
booktitle = "2007 IEEE 15th Signal Processing and Communications Applications, SIU",

}

TY - GEN

T1 - İşitsel-görsel dans verilerinin birleşik i̇linti analizi

AU - Ofli, Ferda

AU - Demir, Y.

AU - Erzin, E.

AU - Yemez, Y.

AU - Tekalp, A. M.

PY - 2007/12/1

Y1 - 2007/12/1

N2 - In this paper we present a framework for analysis of dance figures from audio-visual data. Our audio-visual data is the multiview video of a dancing actor which is acquired using 8 synchronized cameras. The multi-camera motion capture technique of this framework is based on 3D tracking of the markers attached to the dancer's body, using stereo color information. The extracted 3D points are used to calculate the body motion features as 3D displacement vectors. On the other hand, MFC coefficients serve as the audio features. In the first stage of the two stage analysis task, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features, separately, to extract the recurrent elementary audio and body motion patterns. In the second stage, the correlation of body motion patterns with audio patterns is investigated to create a correlation model that can be used during the synthesis of an audio-driven body animation.

AB - In this paper we present a framework for analysis of dance figures from audio-visual data. Our audio-visual data is the multiview video of a dancing actor which is acquired using 8 synchronized cameras. The multi-camera motion capture technique of this framework is based on 3D tracking of the markers attached to the dancer's body, using stereo color information. The extracted 3D points are used to calculate the body motion features as 3D displacement vectors. On the other hand, MFC coefficients serve as the audio features. In the first stage of the two stage analysis task, we perform Hidden Markov Model (HMM) based unsupervised temporal segmentation of the audio and body motion features, separately, to extract the recurrent elementary audio and body motion patterns. In the second stage, the correlation of body motion patterns with audio patterns is investigated to create a correlation model that can be used during the synthesis of an audio-driven body animation.

UR - http://www.scopus.com/inward/record.url?scp=50249167786&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=50249167786&partnerID=8YFLogxK

U2 - 10.1109/SIU.2007.4298617

DO - 10.1109/SIU.2007.4298617

M3 - Conference contribution

SN - 1424407192

SN - 9781424407194

BT - 2007 IEEE 15th Signal Processing and Communications Applications, SIU

ER -