The way we consume media is changing dramatically. Gone are the days of strictly linear viewing; today’s audiences demand immersive, interactive, and personalized experiences. At the forefront of this transformation is XReco, a groundbreaking Horizon Europe Innovation Project designed to build the data-driven ecosystem the media industry needs to thrive in this new era.

Streamlined Media & XR Content with XReco

XReco isn’t just about new gadgets; it’s about reinventing the pipeline. The project aims to streamline data sharing, revolutionize content discovery, and support the creation of innovative media —especially Extended Reality (XR) experiences and virtual productions — all while ensuring fair monetization and rights management.

But how does this work in practice? To demonstrate its powerful capabilities, the XReco team developed a fascinating use case: a TV documentary exploring the life and work of Guglielmo Marconi, the pioneer of wireless telegraphy.

Marconi’s story, rich with revolutionary inventions that laid the foundation for modern communication, is the perfect canvas for XReco. The project’s ambition to seamlessly integrate XR content into traditional production allows historical artifacts — vintage electronics and complex machinery — to be brought to life in an engaging, educational, and entertaining way.

XReco: Turning Historical Archives into 3D Media

The first challenge in any historical production is gathering the source material. Historical archives are often fragmented, with assets scattered across online news outlets, broadcaster archives, and open web repositories, each using different metadata standards.

This is where XReco Connectors come in. They allow media providers to upload diverse assets — images, 2D videos, texts, and 3D models — into a unified platform. XReco standardizes the metadata, ensuring that available contents, such as  assets related to Marconi, are efficiently organized, interlinked, and easily accessible. This process effectively creates a unified digital collection of Marconi’s legacy.

Once the content is ingested, production teams need to find exactly what they need. The XReco Orchestrator serves as a unified, user-friendly dashboard for managing and transforming these assets.

The platform uses advanced neural cross-modal descriptors for indexing. What does this mean for the users? It means they can search across different media types using more than just text. For the Marconi documentary, the editorial team could search for “telegraph” to find initial images, and then use those images to perform a reverse image search, instantly uncovering related visual assets and descriptions of Marconi’s early innovations.

Immersive XR & 3D Production with XReco

One of the XReco’s most exciting features is its cutting-edge content production capabilities. The platform allows creators to transform traditional 2D footage into realistic 3D models using advanced services like NeRF (Neural Radiance Fields), Gaussian Splatting (GS), and Structure from Motion (SfM).

Accessible via the XReco Orchestrator interface, these tools give users fine-tuned control over the reconstruction quality. This ability to turn flat images into immersive 3D experiences is a game-changer. Instead of just showing a picture of Marconi’s inventions, the team built detailed 3D models, allowing viewers to explore these technological marvels from every angle, fostering a deeper understanding of their complexity.

The XReco platform is designed for flexibility, integrating seamlessly with third-party tools to meet high-end production needs.

For the demonstrator, the team pushed the boundaries of virtual production:

  • Virtual Environments: A virtual scene was created using a 3D Gaussian Splatting reconstruction of the Rai Radio Museum, generated from existing 2D footage.
  • Digital Humans: A photorealistic digital avatar of Marconi, created using Unreal MetaHuman and animated with a motion capture suit, was placed into virtual environments populated by the 3D models built with Xreco technologies.
  • Holoportation: A human expert was “holoported” into the virtual environment to interact with a real-life presenter.
  • Real-Time Interaction: Within the virtual Rai Radio Museum, different types of avatars—including one rendered in real-time using Free Viewpoint Video (FVV) technology—were able to interact.

Advanced XR & 3D Storytelling for Media Makers

The platform’s compatibility with industry-standard engines like Unity and Unreal further empowers developers to create visually rich and engaging XR experiences.

For any media organization, creating content is only half the battle; protecting and exploiting those assets is crucial. XReco provides robust tools for managing content licensing and monetization. The platform allows providers to define specific usage conditions—for example, offering basic content for free while charging for access to high-fidelity 3D models.

Through the Marconi documentary demonstrator (Video 1), XReco validates its capacity to unify diverse media workflows, enhance content discovery, enable advanced 3D reconstructions, and integrate immersive technologies, paving the way for innovative storytelling in the evolving media ecosystem.

Video 1: Marconi documentary demonstrator and making-of.

.

Several partners worked together on this demonstrator. The broadcaster RAI created the script, the Metahuman, the virtual environments (the Marconi’s ship and his lab) and organized the shooting in one of its TV studios (Video 2), with the support of Maria Baratta, the artistic director, and colleagues from the production centre of Turin; UPM’s FVV technology was used to build the Marconi expert; the Marconi fan was built with i2CAT 4DGS; the holoported journalist with i2CAT holoportation. The 3D models themselves were created and optimised with the technologies and expertise of UPM, CERTH, i2CAT, and Rai.

Video 2: Rai demonstrator backstage.

The Guglielmo Marconi documentary showcases just a fraction of XReco’s potential. By offering a comprehensive and integrated toolkit — spanning efficient content ingestion, semantic search, advanced 3D reconstruction, and robust licensing — XReco is addressing the key challenges of modern virtual production.

XReco is paving the way for a new era of media, where data is accessible, experiences are immersive, and the lines between the physical and virtual worlds are seamlessly blurred.

About The Author

This article was written by the Rai Centre for Research, Technological Innovation and Experimentation (CRITS) team. They conduct applied research in artificial intelligence (AI) and Extended Reality (XR) technologies.

The team works with partners on projects in Europe and beyond, exploring new technologies and turning their research into real world applications.

Watch all tutorials of the XReco platform tools.

Never miss the latest XReco developments.

Share this article!

Notable sources from the video:

Nebra Ark:

Hotspot #2, Text (Timecode: 1:46-01:50):

Hotspot #3, Text (Timecode: 00:17-00:19):

Einstein Tower:
Hotspot #1, Image (Timecode: 00:30-00:32):

Hotspot #4, Image (Timecode: 01:18-01:20):

Hotspot #4, Text (Timecode: 01:18-01:20):

Phonograph:

Hotspot #2, Image (Timecode: 00:32-00:35):

Hotspot #2, Text (Timecode: 00:32-00:35):

Hotspot #3, Audio and Text (Timecode: 00:51-01:03):

Palace of the Republic:

Hotspot #1, Text (Timecode: 00:36-00:39):

Jewish Museum, Berlin:

Hotspot #1, Image (Timecode: 01:40-01:44): By Sean Gallup/Getty Images.

Hotspot #1, Text (Timecode: 01:40-01:44): 

Hotspot #3, Image (Timecode: 01:25-01:27): By Britta Pedersen/picture alliance/dpa-Zentralbild.

Hotspot #4, Image (Timecode: 01:28-01:30):

Veste Wachsenburg

Hotspot #1, Image (Timecode: 01:32-01:36): By hwo/ picture alliance / imageBROKER.

Hotspot #1, Text (Timecode: 01:32-01:36):