1) Development of Enhanced Light Field Representation Solutions – To enable real-time streaming of Light Field (LF) content, flexible LF coded representations will be investigated, aiming to manage the massive amount of data involved and to predict the user’s movement in a fully immersive experience. For this purpose, scalable LF coding solutions will be developed aiming at supporting random access and region-of-interest (ROI) coding with high coding efficiency.2) Development of Light Field Processing Tools – The different LF capturing approaches have different spatio-angular tradeoffs and may suffer from low spatial resolution, limited depth-of-field, or high computational complexity. To overcome such limitations, advanced algorithms that can estimate accurate geometry information, create 3D models from LFs, and synthesize spatial/angular super-resolved images with high quality and efficiency are needed. To this aim, efficient LF geometry estimation and virtual view synthesis algorithms beyond conventional multi-view approaches will be investigated. Tools like segmentation and inpainting, that may especially useful for interactive LF editing, will also be considered.3) Development of Efficient Packaging Solutions for Light Field Streaming – Ultra-realistic scene rendering from LFs is a very appealing functionality for future interactive and immersive streaming services. One reason for this is the decoupling of computational cost of scene rendering from the rendered scene complexity, contrary to what happens in computer-generated 3D scenes. However, LF imaging requires a huge amount of data for proper scene rendering. To enable interactive LF rendering without requiring the whole LF to be available at the receiver, efficient packaging of the encoded LF content is needed. This would allow restricting network delivery to only the subset of the LF image that is needed to reconstruct the required view. For this to be done in an efficient way, adequate prediction mechani...

LIFESYS focus on three main scientific and technological developments: (i) novel light field content processing tools; (ii) improved scalable light field image and video coding techniques; and (iii) light field creative content generation. In this context, three main research objectives have been defined:Development of a Light Field Processing Toolbox – The light field representation format should allow a scalable representation of the content, in a manner where the various layers are defined by the author during the creative process, instead of making it a pure coding decision. For this, efficient depth-estimation and LF depth-based rendering tools will be developed, as well as LF processing tools, such as image correction, filtering and editing tools.Development of Efficient Codecs for Light Field Content – In order to enable 3D LF content to be presented on various types of displays, such as legacy displays and also newer 3D LF displays, with different characteristics in terms of spatial and view resolutions, an efficient scalable codec will be developed. New coding approaches, beyond state-of-the-art and standard-based approaches will be investigated aiming to support new scalability features opened by LF content.Development of a Light Field Authoring Application – In order to demonstrate the capabilities for the end-user and content creators of the LF technology, a showcase application will be developed that includes creative authoring tools to be used with LF content

This project aims to advance the state-of-the-art in terms of 3D holoscopic content representation, processing and coding. To enable adequate compatibility (to some extent) with legacy displays (e.g., 2D, stereoscopic, or multi-view) efficient scalable representations will be investigated. Whenever appropriate, the emerging scalable extensions of the High Efficiency Video Coding (SHVC) standard will be explored by extending these solutions to 3D holoscopic content. Additionally, since some envisaged delivery channels are critical in terms of channel errors/data losses, this project will also investigate new error control techniques that are specific for 3D holoscopic content.