Peking University, February 16, 2022:  Beijing Winter Olympics Exhibition hut, located in the center of the Beijing Winter Olympics Village, is a hut filled with scientific and technical charm. The hut has three exhibit areas with sixteen new technical breakthroughs relating to the Winter Olympics on show. The Beijing Cabin has become a popular venue for athletes, coaches, and team officials to immerse themselves in the Winter Olympics’ technological innovations.

A popular display of high-tech project  displays is the "Interactive Multi-Dimensional Watching Experience Technology System for Ice and Snow" project booth, which is essentially a free-viewpoint video system that provides users with a new immersive viewing experience. When users place their hands in front of the booth, they are able to control the viewing angle on the screen depending on how they swing their hands. The interactive technology allows audiences to experience the game as if they are there, watching it and experiencing it.

This technology, led by Peking University and created in collaboration with national laboratories, enterprises, and other universities, is part of the National Key Research and Development Program for the Winter Olympics in Science and Technology,



The leader of this project is Professor Chen Baoquan, Executive Director of the PKU Frontier Computing Research Center. His objective was to create an interactive multidimensional viewing experience established under free-viewpoint technology that would allow audiences to experience a brand-new way of viewing.

As a result of this technological development, audiences can choose things like which angle they watch the games from or which athletes they focus on based on their own preferences. They are even able to experience this at home via their own devices.
 
Concept behind this technology
 
This technology was made possible thanks to the end-to-end free-point video system built by Chen Baoquan's team. However, achieving real-time 360 degrees viewing for audiences at any place, any time was not an easy task to achieve. At the previous test competition, 42 U-shaped cameras were positioned at the stadium. Due to certain limitations in the venue, 42 cameras were sparsely distributed on the track of 210m, and the distance between each camera was 5m, which required more intelligent algorithms to supplement the middle view picture and realize seamless free view in the whole field. To compensate for the space between cameras, they configured a system capable of synthesizing new virtual viewpoints through 3D reconstruction. Then, relying on adaptive coding and transmission technology to predict the user's actions while changing viewpoint, targeted calculation, and data transmission are carried out.
 

Chen Baoquan describes, “We realized the image generation of new viewpoints through 3D reconstruction in order to avoid frames jumping from one to another caused by the limited number of cameras on the spot. The insertion of virtual viewpoints enabled enables us to finally realize the experience of watching the games smoothly from multiple viewpoints”

However, following the creation of this technology, new problems emerged. There was a concern that most viewers may not be able to find the best viewing angle of the games independently, which might result in them missing out on the highlights and exciting moments of the games.
 

With this in mind, the team also designed "Twin Deduction of VR Winter Olympic Games and Creative Design Software of Audio-visual Language". Through the analysis of previous events, and in cooperation with professionals, a video camera trajectory was designed to direct the audiences to the best viewing angle of the games where most of the action and exciting moments are most likely to take place. This offers the audience enough "telepresence" but, at the same time, ensures that they do not miss out on the crucial and exciting moments of the games.
 

From "seeing clearly" to "seeing reality"

The audience's need for video viewing is shifting from "seeing clearly" to "seeing reality" in the 5G era, thus understanding the requirements of ultra-high resolution and immersive viewing is critical.

The high resolution of 8K of the free-viewpoint video system can fulfill the audience's need to enlarge the image and observe finer details of the game.

What’s more, with 5G mobile phones and 5G networks, users can also watch the games through the free-viewpoint video system without requiring special equipment or dedicated bandwidth. This allows interactive VR technology to reach ordinary people directly.

In addition to adding a touch of bright color from Peking University to this grand event, Chen hopes to establish China's independent VR video standard and encourage the development of VR technology through the promotion of the Beijing Winter Olympics’ science and technology breakthroughs. In Chen Baoquan's view, the significance of "Winter Olympics with Science and Technology" is to allow the Olympic Games to benefit people's lives and illustrate to them what their lives will look like in the future with the advancements and improvements brought by technology.

Written by: June Tan Rui Min
Edited by: Hu Shaocong
Source: PKU News (Chinese)