Tutorial

 

360° Live-Action Content Production – Challenges, Limitations and Current State of the Art

Presenter: Dr. Sebastian Knorr, Senior Research Scientist, Technical University of Berlin, Germany

Abstract

With the increasing commercialization of virtual reality (VR) as a medium, one of the main factors that drives the uptake of devices is the content. One of the most popular formats to deliver this content is 360° video, which is often called cinematic VR. The tutorial will give an overview about advantages and disadvantages of different 360-capture systems currently used for shooting 360° live-action content. Then, challenges and limitations will be explained in detail. As end-users only see a fraction of the entire 360 panorama, the so-called viewport, the understanding of visual attention in omni-directional content is highly important in cinematic VR content creation, in particular for storytelling, quality control and coding and streaming. Thus, part of the tutorial will be dedicated to saliency prediction, storytelling, new approaches in quality control of stereoscopic omni-directional content and new methods in streaming of cinematic VR.

Outline

  • Fundamentals in 360° content capturing
    • Comparison of different camera setups and current capature systems
    • Overview of different formats (equirectangular, cube maps, etc.)
    • Challenges and limitations in 360° content capture
  • Quality control in 360° content production
    • Overview about common artefacts and misalignments in 2D and stereo 3D omnidirectional content
    • Introduction of a new framework for quality control in post-production workflows
  • Visual attention in 360° content
    • Storytelling in 360 – how to obtain visual attention of the final end-users and how to guide a viewer in 360 content
    • Saliency prediction using deep convolutional neural networks
    • Viewport-aware adaptive steaming

 

Dr. Sebastian Knorr is Senior Research Scientist and Lecturer within the Communication Systems Group at TU Berlin. His research interests are in computer vision, 3D image processing and immersive imaging technologies, in particular virtual reality applications. Between 2017 and 2018 he was Senior Research Scientist within the V-SENSE project at Trinity College Dublin. Sebastian was Managing Director of imcube labs GmbH, Germany and General Manager of Beijing imcube Technologies Co., Ltd. between 2009 and 2016. Besides the management of both companies, he also worked on 3D TV, cinema and giant screen projects from BBC, BSkyB, Warner Bros., etc. as 3D post-conversion stereographer. Between 2002 and 2009, he acted as project manager and senior researcher in the Communication Systems Group at TU Berlin, Germany and received the Dr.-Ing. degree (Ph.D.) in computer vision with highest honors in 2008. Sebastian has been actively involved in several European projects including VISNET, 3DTV, 3FLEX, AutoPost and 3DQC. He is a member of the Institute of Electrical and Electronics Engineers (IEEE), the Association for Computing Machinery (ACM) and the German Society of Television- and Cinema Technology (FKTG e.V.). Sebastian received the German Multimedia Business Award of the Federal Ministry of Economics and Technology in 2008, and was awarded by the initiative “Germany-Land of Ideas” which is sponsored by the German government, commerce and industry in 2009, respectively. In 2012, he received the Scott Helt Memorial Award for the best paper published in the IEEE Transactions on Broadcasting in 2011.

 

Contact Details

Dr. Sebastian Knorr
Senior Research Fellow & Lecturer
Communication Systems Group
Sekr. EN-1, TU Berlin

+49.30.314-23828
knorr@tu-berlin.de