If you’ve seen someone wear a Virtual Reality (VR) headset, you know how ridiculous they look. No matter how good the technology is, the way it is currently presented to others is still laughable. No matter how much convincing you make the experience sound, unless a consumer tries on the headsets themselves, they can’t experience VR the way it is meant to be.
In order to find a solution to this issue, the people over at Google Research and Daydream Labs teamed up to present a possible solution.
According to a blogpost by Google Research, they have come up with a way to let spectators see the full face of the person wearing the VR headset. That way they can fully see the expressions of the headset wearer while they are immersed in the experience. By removing the barrier between the viewers and the demonstrators (barrier in this case being the VR headset), Google is hoping that this process will help immerse the spectators more fully in the experience, allowing them to have the “complete picture”.
In order to achieve this, Google uses a combination of “3D vision, machine learning and graphics techniques” to remove the headset. The technique also involves making a 3D model of the person’s face, tracking their eye movements and then compiling the visuals in the virtual space. To create a model face, Google uses “gaze-dependent dynamic appearance” where users sit in front of the monitor and track a marker with their eyes. The face is then calibrated and synchronized with a database of textures.
When that’s done, a camera is used to capture the user wearing the VR headset in front of a green screen while streaming a video in the headset. Following that, the 3D face model is aligned to give the perception of the headset being removed. And finally, the composition is rendered and checked to make sure the modeled face is consistent with the content in the camera stream.
The result? It is actually impressive! A realistic representation of what the wearer of the headset may be expressing and where they may be looking. However, as apparent from the GIF, it is not perfect. You can still tell that there is some sort digital help that was required to make the top half of the face visible. But conclusively, it is still better than nothing and definitely an improvement over watching someone experiencing VR without any kind of expression.
Source: Google Research Blog, The Next Web
Google VR can beat Samsung Gear VR !