Volumetric Video Compression and Distribution
In recent years, volumetric video has gained a lot of attention and relevance, especially in the realms of virtual reality (VR), augmented reality (AR), virtual productions and media distribution.
To achieve and enhance immersion in these kinds of applications, volumetric video is becoming essential to reach the best quality and deliver the most meaningful experiences. For example, contents that can be seen from any position and angle in 6 degrees of freedom (6DoF) can be used to accurately represent users, or even some parts of the real world.
Figure 1: An example of a holo-conference application that uses volumetric video to represent and stream participants in real-time.
Efficient Point Cloud Compression for Real-Time Volumetric Video
Point clouds are one of the most common volumetric representation formats, it is widely used because it provides detailed information about volumetric scenes. In other words, point clouds can store high-definition geometrical information such as positions, normals, transformations and colors. And when captured with good quality devices they can represent volumetric content with high accuracy, making them a very good choice for VR applications.
However, there are several shortcomings that need to be addressed. For instance, point clouds in raw format require a huge amount of memory and resources to accurately describe the geometry, colors, normals and all the different attributes of the volume. On top of that, they require a lot of bandwidth for transmission. Furthermore, to achieve high quality, better resolution is required, which implies the need for larger point clouds and therefore higher memory and bandwidth demands. This problem becomes even more complex, when real-time constraints are considered. For this reason, it is mandatory to have efficient mechanisms to compress, deliver and render point clouds, such as the ones developed in the scope of XRECO, whose goal is to reduce these major constraints allowing users and developers to have high quality volumetric video content in real time. The Moving Picture Expert Group (MPEG) has been developing compression standards, such as Video based PCC (V-PCC), obtaining good efficiency in the bandwidth reduction of volumetric data, these standards often fall short in terms of performance. The only real time application available is the V-PCC standard-compliant decoder developed by Nokia, which performs well with offline compressed point clouds representing a single person.
To achieve the goal of having a real time volumetric video encoder and decoder, i2cat has been working on developing an efficient volumetric compression system for interactive immersive applications. Said system takes both the geometrical data and the color details of a given point cloud and stores them as images which can be used as input for traditional video codecs. In other words, the volumetric data is transformed into images that can be understood by a video codec such as H.264 or H.265 and the […]
