A new technology for realistic video calls
Last year, Google announced a new innovative technology to chat with other people: an evolution of a video call called Project Starline. It’s a sort of window through which you can see the life-sized and three-dimensional person you’re communicating with. You can talk naturally, make gestures, and make eye contact with the perception of having that person in your room.
The difficulty of convincing your brain that there is a real human being seated just a few feet away is highlighted in Google’s study paper. Naturally, the image must be high-resolution and free of distracting artifacts, but it must also appear accurate from your position in the booth. Another problem is the audio, which requires the system to sound like a person’s words are coming from their own mouth. Then there’s the issue of maintaining eye contact.
However, Project Starline could someday provide a similar sense of presence to virtual or augmented reality without the need for users to wear heavy headsets or trackers.
The paper specifies how much hardware is required to begin solving these issues. The system is centered on a big 65-inch 8K panel with a 60 Hz refresh rate. Google’s engineers have set up three “capture pods” around it, each capable of taking color imagery as well as depth data. Four additional tracking cameras, four microphones, two loudspeakers, and infrared projectors are also included in the system. A total of seven video streams are collected, including color images from four angles and three depth maps. The audio is recorded at 44.1 kHz and encoded at 256 bits per second.
Naturally, all of this hardware generates a lot of data, and Google claims that transmission bandwidth varies between 30 and 100 megabits per second, depending on “the texture detail in the user’s clothes and the magnitude of their gestures”. So it’s a lot more than a usual video call, but nothing that a typical office in a big city couldn’t manage. To encode and decode all of this data, Project Starline has four high-end Nvidia graphics cards (two Quadro RTX 6000 cards and two Titan RTX cards) with an average end-to-end latency of 105.8 milliseconds.
Employees who have used Starline across the three sites where it has been installed, according to Google’s research paper, believe it outperforms traditional video conferencing in terms of creating a sense of presence, and personal connection, as well as assisting with attentiveness and reaction-gauging. According to the company, 117 people held 308 meetings in its telepresence booths over the course of nine months, with an average meeting time of just over 35 minutes.
Anyway, we still don’t know when or even if the system will be commercialized. There’s also scant information on how much Starline’s hardware will cost in the real world. For the time being, Google claims Project Starline will be available “at more Google offices around the United States”.
A technology that may bring back to life telephone booths where we could pay to call in a more realistic way. And when they are affordable, we will install them inside our homes as a replacement for the phone.
Source theverge.com
