GazeStereo3D

Seamless disparity manipulations

Petr Kellnhofer, Piotr Didyk, Karol Myszkowski, Mohamed M. Hefeeda, Hans Peter Seidel, Wojciech Matusik

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Producing a high quality stereoscopic impression on current displays is a challenging task. The content has to be carefully prepared in order to maintain visual comfort, which typically affects the quality of depth reproduction. In this work, we show that this problem can be significantly alleviated when the eye fixation regions can be roughly estimated. We propose a new method for stereoscopic depth adjustment that utilizes eye tracking or other gaze prediction information. The key idea that distinguishes our approach from the previous work is to apply gradual depth adjustments at the eye fixation stage, so that they remain unnoticeable. To this end, we measure the limits imposed on the speed of disparity changes in various depth adjustment scenarios, and formulate a new model that can guide such seamless stereoscopic content processing. Based on this model, we propose a real-time controller that applies local manipulations to stereoscopic content to find the optimum between depth reproduction and visual comfort. We show that the controller is mostly immune to the limitations of low-cost eye tracking solutions. We also demonstrate benefits of our model in off-line applications, such as stereoscopic movie production, where skillful directors can reliably guide and predict viewers' attention or where attended image regions are identified during eye tracking sessions. We validate both our model and the controller in a series of user experiments. They show significant improvements in depth perception without sacrificing the visual quality when our techniques are applied.

Original languageEnglish
Article numbera68
JournalACM Transactions on Graphics
Volume35
Issue number4
DOIs
Publication statusPublished - 11 Jul 2016

Fingerprint

Controllers
Depth perception
Display devices
Processing
Costs
Experiments

Keywords

  • Eye tracking
  • Gaze tracking
  • Gaze-contingent display
  • Remapping
  • Retargeting
  • Stereoscopic 3D

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design

Cite this

Kellnhofer, P., Didyk, P., Myszkowski, K., Hefeeda, M. M., Seidel, H. P., & Matusik, W. (2016). GazeStereo3D: Seamless disparity manipulations. ACM Transactions on Graphics, 35(4), [a68]. https://doi.org/10.1145/2897824.2925866

GazeStereo3D : Seamless disparity manipulations. / Kellnhofer, Petr; Didyk, Piotr; Myszkowski, Karol; Hefeeda, Mohamed M.; Seidel, Hans Peter; Matusik, Wojciech.

In: ACM Transactions on Graphics, Vol. 35, No. 4, a68, 11.07.2016.

Research output: Contribution to journalArticle

Kellnhofer, P, Didyk, P, Myszkowski, K, Hefeeda, MM, Seidel, HP & Matusik, W 2016, 'GazeStereo3D: Seamless disparity manipulations', ACM Transactions on Graphics, vol. 35, no. 4, a68. https://doi.org/10.1145/2897824.2925866
Kellnhofer P, Didyk P, Myszkowski K, Hefeeda MM, Seidel HP, Matusik W. GazeStereo3D: Seamless disparity manipulations. ACM Transactions on Graphics. 2016 Jul 11;35(4). a68. https://doi.org/10.1145/2897824.2925866
Kellnhofer, Petr ; Didyk, Piotr ; Myszkowski, Karol ; Hefeeda, Mohamed M. ; Seidel, Hans Peter ; Matusik, Wojciech. / GazeStereo3D : Seamless disparity manipulations. In: ACM Transactions on Graphics. 2016 ; Vol. 35, No. 4.
@article{3652e89427524f07ba90d5b47cf91180,
title = "GazeStereo3D: Seamless disparity manipulations",
abstract = "Producing a high quality stereoscopic impression on current displays is a challenging task. The content has to be carefully prepared in order to maintain visual comfort, which typically affects the quality of depth reproduction. In this work, we show that this problem can be significantly alleviated when the eye fixation regions can be roughly estimated. We propose a new method for stereoscopic depth adjustment that utilizes eye tracking or other gaze prediction information. The key idea that distinguishes our approach from the previous work is to apply gradual depth adjustments at the eye fixation stage, so that they remain unnoticeable. To this end, we measure the limits imposed on the speed of disparity changes in various depth adjustment scenarios, and formulate a new model that can guide such seamless stereoscopic content processing. Based on this model, we propose a real-time controller that applies local manipulations to stereoscopic content to find the optimum between depth reproduction and visual comfort. We show that the controller is mostly immune to the limitations of low-cost eye tracking solutions. We also demonstrate benefits of our model in off-line applications, such as stereoscopic movie production, where skillful directors can reliably guide and predict viewers' attention or where attended image regions are identified during eye tracking sessions. We validate both our model and the controller in a series of user experiments. They show significant improvements in depth perception without sacrificing the visual quality when our techniques are applied.",
keywords = "Eye tracking, Gaze tracking, Gaze-contingent display, Remapping, Retargeting, Stereoscopic 3D",
author = "Petr Kellnhofer and Piotr Didyk and Karol Myszkowski and Hefeeda, {Mohamed M.} and Seidel, {Hans Peter} and Wojciech Matusik",
year = "2016",
month = "7",
day = "11",
doi = "10.1145/2897824.2925866",
language = "English",
volume = "35",
journal = "ACM Transactions on Graphics",
issn = "0730-0301",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

TY - JOUR

T1 - GazeStereo3D

T2 - Seamless disparity manipulations

AU - Kellnhofer, Petr

AU - Didyk, Piotr

AU - Myszkowski, Karol

AU - Hefeeda, Mohamed M.

AU - Seidel, Hans Peter

AU - Matusik, Wojciech

PY - 2016/7/11

Y1 - 2016/7/11

N2 - Producing a high quality stereoscopic impression on current displays is a challenging task. The content has to be carefully prepared in order to maintain visual comfort, which typically affects the quality of depth reproduction. In this work, we show that this problem can be significantly alleviated when the eye fixation regions can be roughly estimated. We propose a new method for stereoscopic depth adjustment that utilizes eye tracking or other gaze prediction information. The key idea that distinguishes our approach from the previous work is to apply gradual depth adjustments at the eye fixation stage, so that they remain unnoticeable. To this end, we measure the limits imposed on the speed of disparity changes in various depth adjustment scenarios, and formulate a new model that can guide such seamless stereoscopic content processing. Based on this model, we propose a real-time controller that applies local manipulations to stereoscopic content to find the optimum between depth reproduction and visual comfort. We show that the controller is mostly immune to the limitations of low-cost eye tracking solutions. We also demonstrate benefits of our model in off-line applications, such as stereoscopic movie production, where skillful directors can reliably guide and predict viewers' attention or where attended image regions are identified during eye tracking sessions. We validate both our model and the controller in a series of user experiments. They show significant improvements in depth perception without sacrificing the visual quality when our techniques are applied.

AB - Producing a high quality stereoscopic impression on current displays is a challenging task. The content has to be carefully prepared in order to maintain visual comfort, which typically affects the quality of depth reproduction. In this work, we show that this problem can be significantly alleviated when the eye fixation regions can be roughly estimated. We propose a new method for stereoscopic depth adjustment that utilizes eye tracking or other gaze prediction information. The key idea that distinguishes our approach from the previous work is to apply gradual depth adjustments at the eye fixation stage, so that they remain unnoticeable. To this end, we measure the limits imposed on the speed of disparity changes in various depth adjustment scenarios, and formulate a new model that can guide such seamless stereoscopic content processing. Based on this model, we propose a real-time controller that applies local manipulations to stereoscopic content to find the optimum between depth reproduction and visual comfort. We show that the controller is mostly immune to the limitations of low-cost eye tracking solutions. We also demonstrate benefits of our model in off-line applications, such as stereoscopic movie production, where skillful directors can reliably guide and predict viewers' attention or where attended image regions are identified during eye tracking sessions. We validate both our model and the controller in a series of user experiments. They show significant improvements in depth perception without sacrificing the visual quality when our techniques are applied.

KW - Eye tracking

KW - Gaze tracking

KW - Gaze-contingent display

KW - Remapping

KW - Retargeting

KW - Stereoscopic 3D

UR - http://www.scopus.com/inward/record.url?scp=84979998484&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84979998484&partnerID=8YFLogxK

U2 - 10.1145/2897824.2925866

DO - 10.1145/2897824.2925866

M3 - Article

VL - 35

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

SN - 0730-0301

IS - 4

M1 - a68

ER -