1) Development of Enhanced Light Field Representation Solutions – To enable real-time streaming of Light Field (LF) content, flexible LF coded representations will be investigated, aiming to manage the massive amount of data involved and to predict the user’s movement in a fully immersive experience. For this purpose, scalable LF coding solutions will be developed aiming at supporting random access and region-of-interest (ROI) coding with high coding efficiency.2) Development of Light Field Processing Tools – The different LF capturing approaches have different spatio-angular tradeoffs and may suffer from low spatial resolution, limited depth-of-field, or high computational complexity. To overcome such limitations, advanced algorithms that can estimate accurate geometry information, create 3D models from LFs, and synthesize spatial/angular super-resolved images with high quality and efficiency are needed. To this aim, efficient LF geometry estimation and virtual view synthesis algorithms beyond conventional multi-view approaches will be investigated. Tools like segmentation and inpainting, that may especially useful for interactive LF editing, will also be considered.3) Development of Efficient Packaging Solutions for Light Field Streaming – Ultra-realistic scene rendering from LFs is a very appealing functionality for future interactive and immersive streaming services. One reason for this is the decoupling of computational cost of scene rendering from the rendered scene complexity, contrary to what happens in computer-generated 3D scenes. However, LF imaging requires a huge amount of data for proper scene rendering. To enable interactive LF rendering without requiring the whole LF to be available at the receiver, efficient packaging of the encoded LF content is needed. This would allow restricting network delivery to only the subset of the LF image that is needed to reconstruct the required view. For this to be done in an efficient way, adequate prediction mechani...

LIFESYS focus on three main scientific and technological developments: (i) novel light field content processing tools; (ii) improved scalable light field image and video coding techniques; and (iii) light field creative content generation. In this context, three main research objectives have been defined:Development of a Light Field Processing Toolbox – The light field representation format should allow a scalable representation of the content, in a manner where the various layers are defined by the author during the creative process, instead of making it a pure coding decision. For this, efficient depth-estimation and LF depth-based rendering tools will be developed, as well as LF processing tools, such as image correction, filtering and editing tools.Development of Efficient Codecs for Light Field Content – In order to enable 3D LF content to be presented on various types of displays, such as legacy displays and also newer 3D LF displays, with different characteristics in terms of spatial and view resolutions, an efficient scalable codec will be developed. New coding approaches, beyond state-of-the-art and standard-based approaches will be investigated aiming to support new scalability features opened by LF content.Development of a Light Field Authoring Application – In order to demonstrate the capabilities for the end-user and content creators of the LF technology, a showcase application will be developed that includes creative authoring tools to be used with LF content

This project aims to advance the state-of-the-art in terms of 3D holoscopic content representation, processing and coding. To enable adequate compatibility (to some extent) with legacy displays (e.g., 2D, stereoscopic, or multi-view) efficient scalable representations will be investigated. Whenever appropriate, the emerging scalable extensions of the High Efficiency Video Coding (SHVC) standard will be explored by extending these solutions to 3D holoscopic content. Additionally, since some envisaged delivery channels are critical in terms of channel errors/data losses, this project will also investigate new error control techniques that are specific for 3D holoscopic content.

The main objective of the project ?Quality Control on the Go? (QC OTG) is the development of a tool, integrated in the process of capture and ingest, which enables the analysis, in real time, of the quality of audio-visual content in three dimensions - video, audio and file container - in an automatic way and compliant with acquisition systems based on files and SDI . This project is promoted by the consortium formed by MOG Technologies (leader), IST, ISCTE-IUL and IPN. The new product to be developed under the scope of this project is targeted at the audio-visual and broadcast sectors (especially producers of audio-visual content), presenting a set of innovative features, of which we highlight: analysis, detection of artifacts of audio and video contents in various formats (already available or near completion, as for example Sony XAVC) and with different targets (UHDTB, HDTV, Internet, Mobile); Fully integration of the features of capture and ingest with the system of quality control of the video, audio, and file container, ensuring the operation proceeds at real time; Integration of a library with a wide range of modular functions that enable extraction and analysis of information of video and audio (semantic and syntactic); Higher efficiency of the ingest process by introducing a single step of quality control therefore reducing the necessary computational resources.Aiming at fulfilling the goals and innovative characteristics that the project proposes, the consortium has established a methodology that comprises research and industrial development activities, counting with the involvement of a total of 26 human resources of the consortium and with a total investment of 1.192.093,72 Euros (including the acquisition of equipment and expenses with activities of promotion and dissemination of the results).

This COST Action undertakes coordinated research collaboration, at European level, in 3D multimedia creation, encoding, delivery and reception of services and applications over future networking technologies. A scientific framework is devised to integrate the main elements of the delivery chain, such as 3D content creation and encoding evolution, transmission across heterogeneous networks and user consumption, taking perceived quality as an overall key performance factor. Several individual R&D efforts are currently running across Europe, targeted at 3D technologies. This Action aims at beyond this trend by tightening closer together scattered efforts and integrating all technological elements with user 3D quality perception. To reach this goal, this Action fuels cooperation between researchers and industry experts, envisaging production of technical and scientific deliverables for researchers, scientists, engineers and managers, new business model recommendations for content and service providers through joint meetings across academia and industry, plus documentation and multimedia presentations to promote 3D technology in the society. The benefits include increased relevancy of European research, contribution to the development and sustainability of new and better 3D multimedia communications technology and provision of a platform for faster launching and adoption of related new products and services across Europe for end users’ beneficiary

The main objective of SAAS is to develop an UAV remote control system that is mostly independent of a specific vehicle model or design and that makes use of the available terrestrial wireless networks. The individual challenges and objectives to be addressed are:• Real-Time Video Transmission and Control using Terrestrial Wireless Networks: Investigate the possibility of transmitting real time video and control commands with adequate quality for a pilot to control and make decisions on the vehicle course using the available radio networks (GSM, UMTS, HSDPA, Wi-Fi, LTE).• Semi-autonomous Flight Control: Develop autonomous control capabilities for the aircraft to perform its self-rescue in the case of loss of radio connection and that can also allow a reduced piloting workload.• Remote Visualization and Control Application: Implement an application for tablet PCs that enables the visualization of the video captured by the aircraft cameras together with other telemetry information and allows the pilot to command the aircraft (either using the touchscreen, the accelerometers or an additional attached joystick).• Prototype Implementation and Flight Tests: Implement a prototype using a small rotary-wing aircraft (possibly a tricopter/quadcopter) and evaluate the performance in terms of video reception, control, and autonomy capability.

The aim of the project 3D VIVANT is to capture events automatically in 3D and deliver them for realistic, interactive and immersive play back to home users-viewers. This will require the project to create, develop and integrate a comprehensive range of components from the generic technology of 3D imaging. Real and virtual 3D content will be homogeneously combined/mixed producing a novel form of rich and interactive content, defined as “3D Holoscopic content”. 3D Holoscopic imaging is a technique for creating full colour 3D optical models that exist in space independently of the viewer. The images exhibit continuous parallax throughout the viewing zone. The continuous nature of the images produced with this method eliminates the effect of 'cardboarding' (flattening of objects into discrete depth planes) and flipping (a visible effect created by moving between image fields) present in multiview stereoscopic systems. Unlike multiview stereoscopy, information about a point is contained in many different parts of the image plane. In this respect 3D Holoscopic imaging is akin to holography. However 3D Holoscopic imaging is more advantageous, since it can operate under incoherent illumination, which is in contrast with holography, and hence it allows more conventional live capture and display procedures to be adopted. Furthermore, in viewing the 3D optical model accommodation and convergence work in unison (i.e. the viewer’s eyes focus and converge to the same point) to prevent eye-strain from occurring. The 3D VIVANT project will investigate the possibility of using different technologies for capture and display of 3D content. For the capture, 3D VIVANT will take full advantage of 3D Holoscopic imaging technology, where a single camera is required. Hence in this project a world first single aperture ultra-high definition 3D Holoscopic imaging camera will be constructed which will permit live capture of 3D content. For the display, the project will take advantage of Hologr...

Specification of audiovisual representation standards (MPEG-4, MPEG-7 and MPEG-21).

The main goal of VISNET II is the provision of a working environment with appropriate integration activities and research infrastructure that will ease and stimulate the interaction and cooperation among European researchers and institutions with recognized excellence in various fields embraced by audiovisual networking technologies. The fulfilment of this objective will promote and contribute to the advances of research in this area and ultimately improve the quality and quantity of research results. Due to the high commitment of this NoE in conducting extensive dissemination activities, results will easily span across its borders, thus benefiting the whole scientific and technological communities

European platform for the development of the MPEG-4 standard

Participation in the development of the MPEG-4 standard.

Increase the collaboration between national universities, research institutes and companies in the area of audiovisual signals processing and coding

Support of the participation in ISO/MPEG meetings for the development of the MPEG-4 standard.

Development of low bitrate solutions for audiovisual mobile communications.Contribution to the development of the MPEG-4 standard.