Gradient-based 2D-To-3D Conversion for Soccer Videos

Kiana Calagari, Mohamed Elgharib, Piotr Didyk, Alexandre Kaspar, Wojciech Matusik, Mohamed Hefeeda

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Citations (Scopus)

Abstract

A wide spread adoption of 3D videos and technologies is hindered by the lack of high-quality 3D content. One promis-ing solution to address this problem is to use automated 2D-To-3D conversion. However, current conversion meth-ods, while general, produce low-quality results with artifacts that are not acceptable to many viewers. We address this problem by showing how to construct a high-quality, domain-specific conversion method for soccer videos. We propose a novel, data-driven method that generates stereoscopic frames by transferring depth information from similar frames in a database of 3D stereoscopic videos. Creating a database of 3D stereoscopic videos with accurate depth is, however, very dificult. One of the key findings in this paper is showing that computer generated content in current sports computer games can be used to generate high-quality 3D video ref-erence database for 2D-To-3D conversion methods. Once we retrieve similar 3D video frames, our technique transfers depth gradients to the target frame while respecting object boundaries. It then computes depth maps from the gradients, and generates the output stereoscopic video. We implement our method and validate it by conducting user-studies that evaluate depth perception and visual comfort of the converted 3D videos. We show that our method produces high-quality 3D videos that are almost indistinguishable from videos shot by stereo cameras. In addition, our method signifocantly out-performs the current state-of-The-Art method. For example, up to 20% improvement in the perceived depth is achieved by our method, which translates to improving the mean opinion score from Good to Excellent.

Original languageEnglish
Title of host publicationMM 2015 - Proceedings of the 2015 ACM Multimedia Conference
PublisherAssociation for Computing Machinery, Inc
Pages331-340
Number of pages10
ISBN (Print)9781450334594
DOIs
Publication statusPublished - 13 Oct 2015
Event23rd ACM International Conference on Multimedia, MM 2015 - Brisbane, Australia
Duration: 26 Oct 201530 Oct 2015

Other

Other23rd ACM International Conference on Multimedia, MM 2015
CountryAustralia
CityBrisbane
Period26/10/1530/10/15

Fingerprint

Depth perception
Computer games
Sports
Cameras

Keywords

  • 2D-To-3D conversion
  • 3D video
  • Depth estimation

ASJC Scopus subject areas

  • Media Technology
  • Computer Graphics and Computer-Aided Design
  • Computer Vision and Pattern Recognition
  • Software

Cite this

Calagari, K., Elgharib, M., Didyk, P., Kaspar, A., Matusik, W., & Hefeeda, M. (2015). Gradient-based 2D-To-3D Conversion for Soccer Videos. In MM 2015 - Proceedings of the 2015 ACM Multimedia Conference (pp. 331-340). Association for Computing Machinery, Inc. https://doi.org/10.1145/2733373.2806262

Gradient-based 2D-To-3D Conversion for Soccer Videos. / Calagari, Kiana; Elgharib, Mohamed; Didyk, Piotr; Kaspar, Alexandre; Matusik, Wojciech; Hefeeda, Mohamed.

MM 2015 - Proceedings of the 2015 ACM Multimedia Conference. Association for Computing Machinery, Inc, 2015. p. 331-340.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Calagari, K, Elgharib, M, Didyk, P, Kaspar, A, Matusik, W & Hefeeda, M 2015, Gradient-based 2D-To-3D Conversion for Soccer Videos. in MM 2015 - Proceedings of the 2015 ACM Multimedia Conference. Association for Computing Machinery, Inc, pp. 331-340, 23rd ACM International Conference on Multimedia, MM 2015, Brisbane, Australia, 26/10/15. https://doi.org/10.1145/2733373.2806262
Calagari K, Elgharib M, Didyk P, Kaspar A, Matusik W, Hefeeda M. Gradient-based 2D-To-3D Conversion for Soccer Videos. In MM 2015 - Proceedings of the 2015 ACM Multimedia Conference. Association for Computing Machinery, Inc. 2015. p. 331-340 https://doi.org/10.1145/2733373.2806262
Calagari, Kiana ; Elgharib, Mohamed ; Didyk, Piotr ; Kaspar, Alexandre ; Matusik, Wojciech ; Hefeeda, Mohamed. / Gradient-based 2D-To-3D Conversion for Soccer Videos. MM 2015 - Proceedings of the 2015 ACM Multimedia Conference. Association for Computing Machinery, Inc, 2015. pp. 331-340
@inproceedings{f11707e9dad1486a86d8811ae29bbaca,
title = "Gradient-based 2D-To-3D Conversion for Soccer Videos",
abstract = "A wide spread adoption of 3D videos and technologies is hindered by the lack of high-quality 3D content. One promis-ing solution to address this problem is to use automated 2D-To-3D conversion. However, current conversion meth-ods, while general, produce low-quality results with artifacts that are not acceptable to many viewers. We address this problem by showing how to construct a high-quality, domain-specific conversion method for soccer videos. We propose a novel, data-driven method that generates stereoscopic frames by transferring depth information from similar frames in a database of 3D stereoscopic videos. Creating a database of 3D stereoscopic videos with accurate depth is, however, very dificult. One of the key findings in this paper is showing that computer generated content in current sports computer games can be used to generate high-quality 3D video ref-erence database for 2D-To-3D conversion methods. Once we retrieve similar 3D video frames, our technique transfers depth gradients to the target frame while respecting object boundaries. It then computes depth maps from the gradients, and generates the output stereoscopic video. We implement our method and validate it by conducting user-studies that evaluate depth perception and visual comfort of the converted 3D videos. We show that our method produces high-quality 3D videos that are almost indistinguishable from videos shot by stereo cameras. In addition, our method signifocantly out-performs the current state-of-The-Art method. For example, up to 20{\%} improvement in the perceived depth is achieved by our method, which translates to improving the mean opinion score from Good to Excellent.",
keywords = "2D-To-3D conversion, 3D video, Depth estimation",
author = "Kiana Calagari and Mohamed Elgharib and Piotr Didyk and Alexandre Kaspar and Wojciech Matusik and Mohamed Hefeeda",
year = "2015",
month = "10",
day = "13",
doi = "10.1145/2733373.2806262",
language = "English",
isbn = "9781450334594",
pages = "331--340",
booktitle = "MM 2015 - Proceedings of the 2015 ACM Multimedia Conference",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - Gradient-based 2D-To-3D Conversion for Soccer Videos

AU - Calagari, Kiana

AU - Elgharib, Mohamed

AU - Didyk, Piotr

AU - Kaspar, Alexandre

AU - Matusik, Wojciech

AU - Hefeeda, Mohamed

PY - 2015/10/13

Y1 - 2015/10/13

N2 - A wide spread adoption of 3D videos and technologies is hindered by the lack of high-quality 3D content. One promis-ing solution to address this problem is to use automated 2D-To-3D conversion. However, current conversion meth-ods, while general, produce low-quality results with artifacts that are not acceptable to many viewers. We address this problem by showing how to construct a high-quality, domain-specific conversion method for soccer videos. We propose a novel, data-driven method that generates stereoscopic frames by transferring depth information from similar frames in a database of 3D stereoscopic videos. Creating a database of 3D stereoscopic videos with accurate depth is, however, very dificult. One of the key findings in this paper is showing that computer generated content in current sports computer games can be used to generate high-quality 3D video ref-erence database for 2D-To-3D conversion methods. Once we retrieve similar 3D video frames, our technique transfers depth gradients to the target frame while respecting object boundaries. It then computes depth maps from the gradients, and generates the output stereoscopic video. We implement our method and validate it by conducting user-studies that evaluate depth perception and visual comfort of the converted 3D videos. We show that our method produces high-quality 3D videos that are almost indistinguishable from videos shot by stereo cameras. In addition, our method signifocantly out-performs the current state-of-The-Art method. For example, up to 20% improvement in the perceived depth is achieved by our method, which translates to improving the mean opinion score from Good to Excellent.

AB - A wide spread adoption of 3D videos and technologies is hindered by the lack of high-quality 3D content. One promis-ing solution to address this problem is to use automated 2D-To-3D conversion. However, current conversion meth-ods, while general, produce low-quality results with artifacts that are not acceptable to many viewers. We address this problem by showing how to construct a high-quality, domain-specific conversion method for soccer videos. We propose a novel, data-driven method that generates stereoscopic frames by transferring depth information from similar frames in a database of 3D stereoscopic videos. Creating a database of 3D stereoscopic videos with accurate depth is, however, very dificult. One of the key findings in this paper is showing that computer generated content in current sports computer games can be used to generate high-quality 3D video ref-erence database for 2D-To-3D conversion methods. Once we retrieve similar 3D video frames, our technique transfers depth gradients to the target frame while respecting object boundaries. It then computes depth maps from the gradients, and generates the output stereoscopic video. We implement our method and validate it by conducting user-studies that evaluate depth perception and visual comfort of the converted 3D videos. We show that our method produces high-quality 3D videos that are almost indistinguishable from videos shot by stereo cameras. In addition, our method signifocantly out-performs the current state-of-The-Art method. For example, up to 20% improvement in the perceived depth is achieved by our method, which translates to improving the mean opinion score from Good to Excellent.

KW - 2D-To-3D conversion

KW - 3D video

KW - Depth estimation

UR - http://www.scopus.com/inward/record.url?scp=84962878065&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84962878065&partnerID=8YFLogxK

U2 - 10.1145/2733373.2806262

DO - 10.1145/2733373.2806262

M3 - Conference contribution

SN - 9781450334594

SP - 331

EP - 340

BT - MM 2015 - Proceedings of the 2015 ACM Multimedia Conference

PB - Association for Computing Machinery, Inc

ER -