Adaptive streaming of interactive free viewpoint videos to heterogeneous clients

Ahmed Hamza, Mohamed Hefeeda

Research output: Chapter in Book/Report/Conference proceedingConference contribution

22 Citations (Scopus)

Abstract

Recent advances in video capturing and rendering technologies have paved the way for new video streaming applications. Freeviewpoint video (FVV) streaming is one such application where users are able to interact with the scene by navigating to different viewpoints. Free-viewpoint videos are composed of multiple streams representing the captured scene and its geometry from different vantage points. Rendering non-captured views at the client requires transmitting multiple views with associated depth map streams, thereby increasing the network traffic requirements for such systems. Adding to the complexity of these systems is the fact that different component streams contribute differently to the quality of the final rendered view. In this paper, we present a free-viewpoint video streaming system based on HTTP adaptive streaming and the multi-view-plus-depth (MVD) representation. We propose a novel quality-aware rate adaptation method for FVV streaming based on a virtual view distortion model. This view distortion model represents the relation between the distortion of the texture and depth components of reference views and a target virtual view and enables the streaming client to find the best set of representations to request from the server. We have implemented the proposed rate adaptation method in a prototype FVV DASH-based streaming system and performed objective and subjective evaluation experiments. Our experimental results show that the proposed FVV streaming rate adaptation method improves the user's quality-of-experience and increases the visual quality of rendered virtual views by up to 4 dB for some video sequences. Moreover, users have rated the quality of videos streamed using our proposed method higher than videos streamed using other rate adaptation methods in the literature.

Original languageEnglish
Title of host publicationProceedings of the 7th International Conference on Multimedia Systems, MMSys 2016
PublisherAssociation for Computing Machinery, Inc
Pages99-110
Number of pages12
ISBN (Electronic)9781450342971
DOIs
Publication statusPublished - 10 May 2016
Event7th ACM International Conference on Multimedia Systems, MMSys 2016 - Klagenfurt, Austria
Duration: 10 May 201613 May 2016

Other

Other7th ACM International Conference on Multimedia Systems, MMSys 2016
CountryAustria
CityKlagenfurt
Period10/5/1613/5/16

Fingerprint

Video streaming
HTTP
Servers
Textures
Geometry
Experiments

Keywords

  • 3D video
  • DASH
  • Free-viewpoint video
  • Rate adaptation
  • Video streaming

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Human-Computer Interaction
  • Software

Cite this

Hamza, A., & Hefeeda, M. (2016). Adaptive streaming of interactive free viewpoint videos to heterogeneous clients. In Proceedings of the 7th International Conference on Multimedia Systems, MMSys 2016 (pp. 99-110). [2910610] Association for Computing Machinery, Inc. https://doi.org/10.1145/2910017.2910610

Adaptive streaming of interactive free viewpoint videos to heterogeneous clients. / Hamza, Ahmed; Hefeeda, Mohamed.

Proceedings of the 7th International Conference on Multimedia Systems, MMSys 2016. Association for Computing Machinery, Inc, 2016. p. 99-110 2910610.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Hamza, A & Hefeeda, M 2016, Adaptive streaming of interactive free viewpoint videos to heterogeneous clients. in Proceedings of the 7th International Conference on Multimedia Systems, MMSys 2016., 2910610, Association for Computing Machinery, Inc, pp. 99-110, 7th ACM International Conference on Multimedia Systems, MMSys 2016, Klagenfurt, Austria, 10/5/16. https://doi.org/10.1145/2910017.2910610
Hamza A, Hefeeda M. Adaptive streaming of interactive free viewpoint videos to heterogeneous clients. In Proceedings of the 7th International Conference on Multimedia Systems, MMSys 2016. Association for Computing Machinery, Inc. 2016. p. 99-110. 2910610 https://doi.org/10.1145/2910017.2910610
Hamza, Ahmed ; Hefeeda, Mohamed. / Adaptive streaming of interactive free viewpoint videos to heterogeneous clients. Proceedings of the 7th International Conference on Multimedia Systems, MMSys 2016. Association for Computing Machinery, Inc, 2016. pp. 99-110
@inproceedings{0e7eb0a65019490394b241d480d3fe6e,
title = "Adaptive streaming of interactive free viewpoint videos to heterogeneous clients",
abstract = "Recent advances in video capturing and rendering technologies have paved the way for new video streaming applications. Freeviewpoint video (FVV) streaming is one such application where users are able to interact with the scene by navigating to different viewpoints. Free-viewpoint videos are composed of multiple streams representing the captured scene and its geometry from different vantage points. Rendering non-captured views at the client requires transmitting multiple views with associated depth map streams, thereby increasing the network traffic requirements for such systems. Adding to the complexity of these systems is the fact that different component streams contribute differently to the quality of the final rendered view. In this paper, we present a free-viewpoint video streaming system based on HTTP adaptive streaming and the multi-view-plus-depth (MVD) representation. We propose a novel quality-aware rate adaptation method for FVV streaming based on a virtual view distortion model. This view distortion model represents the relation between the distortion of the texture and depth components of reference views and a target virtual view and enables the streaming client to find the best set of representations to request from the server. We have implemented the proposed rate adaptation method in a prototype FVV DASH-based streaming system and performed objective and subjective evaluation experiments. Our experimental results show that the proposed FVV streaming rate adaptation method improves the user's quality-of-experience and increases the visual quality of rendered virtual views by up to 4 dB for some video sequences. Moreover, users have rated the quality of videos streamed using our proposed method higher than videos streamed using other rate adaptation methods in the literature.",
keywords = "3D video, DASH, Free-viewpoint video, Rate adaptation, Video streaming",
author = "Ahmed Hamza and Mohamed Hefeeda",
year = "2016",
month = "5",
day = "10",
doi = "10.1145/2910017.2910610",
language = "English",
pages = "99--110",
booktitle = "Proceedings of the 7th International Conference on Multimedia Systems, MMSys 2016",
publisher = "Association for Computing Machinery, Inc",

}

TY - GEN

T1 - Adaptive streaming of interactive free viewpoint videos to heterogeneous clients

AU - Hamza, Ahmed

AU - Hefeeda, Mohamed

PY - 2016/5/10

Y1 - 2016/5/10

N2 - Recent advances in video capturing and rendering technologies have paved the way for new video streaming applications. Freeviewpoint video (FVV) streaming is one such application where users are able to interact with the scene by navigating to different viewpoints. Free-viewpoint videos are composed of multiple streams representing the captured scene and its geometry from different vantage points. Rendering non-captured views at the client requires transmitting multiple views with associated depth map streams, thereby increasing the network traffic requirements for such systems. Adding to the complexity of these systems is the fact that different component streams contribute differently to the quality of the final rendered view. In this paper, we present a free-viewpoint video streaming system based on HTTP adaptive streaming and the multi-view-plus-depth (MVD) representation. We propose a novel quality-aware rate adaptation method for FVV streaming based on a virtual view distortion model. This view distortion model represents the relation between the distortion of the texture and depth components of reference views and a target virtual view and enables the streaming client to find the best set of representations to request from the server. We have implemented the proposed rate adaptation method in a prototype FVV DASH-based streaming system and performed objective and subjective evaluation experiments. Our experimental results show that the proposed FVV streaming rate adaptation method improves the user's quality-of-experience and increases the visual quality of rendered virtual views by up to 4 dB for some video sequences. Moreover, users have rated the quality of videos streamed using our proposed method higher than videos streamed using other rate adaptation methods in the literature.

AB - Recent advances in video capturing and rendering technologies have paved the way for new video streaming applications. Freeviewpoint video (FVV) streaming is one such application where users are able to interact with the scene by navigating to different viewpoints. Free-viewpoint videos are composed of multiple streams representing the captured scene and its geometry from different vantage points. Rendering non-captured views at the client requires transmitting multiple views with associated depth map streams, thereby increasing the network traffic requirements for such systems. Adding to the complexity of these systems is the fact that different component streams contribute differently to the quality of the final rendered view. In this paper, we present a free-viewpoint video streaming system based on HTTP adaptive streaming and the multi-view-plus-depth (MVD) representation. We propose a novel quality-aware rate adaptation method for FVV streaming based on a virtual view distortion model. This view distortion model represents the relation between the distortion of the texture and depth components of reference views and a target virtual view and enables the streaming client to find the best set of representations to request from the server. We have implemented the proposed rate adaptation method in a prototype FVV DASH-based streaming system and performed objective and subjective evaluation experiments. Our experimental results show that the proposed FVV streaming rate adaptation method improves the user's quality-of-experience and increases the visual quality of rendered virtual views by up to 4 dB for some video sequences. Moreover, users have rated the quality of videos streamed using our proposed method higher than videos streamed using other rate adaptation methods in the literature.

KW - 3D video

KW - DASH

KW - Free-viewpoint video

KW - Rate adaptation

KW - Video streaming

UR - http://www.scopus.com/inward/record.url?scp=84973875504&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84973875504&partnerID=8YFLogxK

U2 - 10.1145/2910017.2910610

DO - 10.1145/2910017.2910610

M3 - Conference contribution

SP - 99

EP - 110

BT - Proceedings of the 7th International Conference on Multimedia Systems, MMSys 2016

PB - Association for Computing Machinery, Inc

ER -