Connect with us

Latest Posts

Meta Demonstrates Quest 2 Body Tracking Without Trackers

Published

on

ADVERTISEMENT

Advertisements

Meta researchers have demonstrated body tracking of the Quest 2 without additional trackers.

Current VR systems ship with a headset and grip controllers, so they just keep track of the position of your head and hands. The position of the elbows, torso, and legs can be estimated using a class of algorithms called inverse kinematics (IK), but that’s only Sometimes accurate for elbows and rarely correct for legs. There are a very large number of potential solutions for each combination of head and hand positions.

Due to IK restrictions, some Today’s virtual reality apps only show your hands, and many of them only give you the upper body. Computer headsets that use SteamVR tracking support additional trackers like HTC’s Vive Tracker, but the three needed for body tracking cost north of $350 and so this is not supported in most games.

But in a new research paper titled QuestSim, Meta researchers demonstrate a neural network-driven system that can estimate reasonable full body posture with only tracking data from Quest 2 and its controllers. No additional trackers or external sensors are required.

The resulting avatar’s movement closely matches the user’s real movement. The researchers claim the accuracy and resulting jitter are better than worn-out IMU trackers — devices that only have an accelerometer and gyroscope like the Pico Fitness Band advertised from Pico 4 (Pico claims to run on its own machine learning algorithm).

but, There is a problem here. As shown in the video, this system is designed to prioritize reasonable production full body Stand making sure the avatar’s hands match the exact position of the user’s hands. The system latency is also 160ms – over 11 frames at 72Hz. Given these issues, QuestSim will only be suitable for seeing other people’s bodies, not your own when looking down.

Current legless meta avatars

But still, seeing the full body movement of other avatars would be much better than the legless upper bodies that are often criticized for current Meta avatars. So is this system, or something like that, coming to Quest 2?

Metatech director Andrew Bosworth certainly alluded to this last week. When Asked about leg tracking on Instagram, “Ask me anything,” Bosworth replied:

“Yeah we’ve made fun of legless avatars a lot, and I think that’s pretty fair and I think it’s pretty funny.

Having legs on your avatar that don’t match your real legs is very frustrating for people. But of course we can put legs on other people, and you can see that, and it doesn’t bother you at all.

So we’re working on legs that look natural to a spectator – because they don’t know how your real legs are positioned – but it’s possible that when you look at your legs you’ll still see nothing. This is our current strategy.”

The short-term solution may not be as good as this algorithm. Machine learning research papers tend to run on powerful PC GPUs with relatively low frame rates. The paper did not mention the performance of the described system, but the RTX 3080 is listed as being used for training.

Meta Connect’s annual AR/VR event takes place in just over two weeks, so any body-tracking announcement is likely to happen during that.

ADVERTISEMENT

Click to comment

Leave a Reply

Your email address will not be published.

Trending

Advertisements

Copyright © 2022 strongbat.com. Theme by The Nitesh Arya.