XReco is an innovative ecosystem that provides media creators with new possibilities in XR content production and media data management. To achieve this, we have utilized a range of advanced technologies, which you can learn more about on this page.
Page Content Navigation
Click for larger image
Level 1 – Neural Media Repository (NMR)
The NMR is a modern, AI powered, content sharing platform. The involved neural technology will enable search & retrieval of media content based on the analysis of visual appearance , geometrical characteristics and correspondences between heterogeneous representations of (three-dimensional) XR content.
The NMR will provide search and retrieval over media and XR content generated both within the XReco platform and held in external repositories. In particular, the novel content, generated from the retrieved media and thanks to the XReco services, will be made available again in the NMR and offered via a marketplace.
Level 1 – Rights and Monetization Management
Generating novel content from multiple sources, especially in an AI enabled automatic (or semiautomatic) manner, will imply crucial challenges in terms of the definition of ownership and monetization rights. The XReco services will be able to select partially, or totally, previously generated media and use, merge and update them according to the decision taken by the AI implementations involved in each XR technology.
The Right and Monetization Management system implemented within XReco and based on Blockchain technology will take into account a specifically designed ownership modelling, assigning the correct, and proportional IP rights to the content providers who own the sources used in each generation process.
Level 2 – Neural Radiance Fields (NeRF)
Neural Radiance Fields (NeRF) provide implicit 3D-based scene representations, created from sparse 2D views of a scene, and enable the synthesis of novel viewpoints. They give the ability to implicitly parameterize scenes in the 3D domain from a collection of 2D images and will facilitate the creation of XR content. More specifically, the XReco users will be able to create novel, three-dimensional content from a collection of photos retrieved from heterogeneous sources within the NMR.
Level 2 – Relighting / Delighting
Creating content form multiple external sources has to face the challenge of merging images taken at different times of the day, with different weather and lighting conditions. The NeRF services implemented in XReco will be able to analyse such differences and compensate them when different media content is selected to be used in the generation of new XR media. The light equalizing will be then exploited to provide coherent assets to the other XR services for the generation of 3D assets.
Level 2 – 3D Reconstruction
3D modelling, reconstruction and rendering are crucial tasks for the XReco services. The 3D reconstruction methods implemented will be capable of creating multi-textured 3D triangle meshes from unorganised sets of images retrieved from the XReco NMR by analysing the correspondences between feature points detected in those images to create newly reconstructed 3D assets.
Level 2.5 – Holoportation (live – non linked to the NMR)
Going beyond the concept of 2D based videoconferencing, holoportation aims at bringing a real feeling of presence in remote immersive experiences. Users are captured as volumes and remotely projected directly into an XR scenario in real time. Within XReco, the holoportation system will be used to provide a fully volumetric representation of humans to be instantly immersed in an XR environment such as a broadcasting TV studio with the possibility of giving an unlimited number of viewpoints. The broadcasting director can move a virtual camera around a holoported journalist or reporter with a cheap and easy to reproduce production system.
Level 2.5 – Free Viewpoint Video (live – non linked to the NMR)
The FVV system allows users to navigate a natural scene in an immersive fashion. It is capable of rendering in real-time the view from a virtual camera placed at any point within the captured scenario. Similarly to the holoportation, the FVV system can virtually place a journalist in a virtual 3D scene providing the required viewpoints in real-time and as a simple 2D video stream.
Level 3 – Media Broadcasting Authoring Tool
Facilitating the production workflow in the media broadcasting industry is one of the main goals of XReco. The Media Broadcasting Authoring Tool will provide an easy-to-use tool to the media industry professionals for being able to use the XReco technologies, both for search-and-retrieval and for the XR content generation, and an intuitive interface for the merging, manipulation and generation of timelines for XR based broadcasting media.
Level 3 – Tourism Authoring Tool
Similarly, to the media industry, XReco will provide a tool for the generation of XR based experience for the tourism industry. The authoring tool, in this case, will serve as interface for the search and retrieval of content related to historical buildings, relevant historical figures, the generation of novel content thanks to the XR services and the placement of the media within XR touristic applications.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.