[ad_1]
Last night, the Roblox test venue was called “Avatar Gallery – (Work)It became available briefly, revealing a host of new Avatar features, including a dynamic face tracking system that allows users to animate their facial expressions using their realistic movements. This experimental feature, though now disabled and set to private, could indicate Roblox will soon release the technology to the public.
Dynamic face tracking works by connecting a user’s camera, whether on a phone or PC, to their Roblox account. Once activated, an avatar preview window can be enabled from the menu in the experience, allowing users to see their avatar’s eyes and head movements in real time, as they operate the microphone and camera.
If you want to see it in action, check out the following video:
Roblox first announced this update in March 2022 as a critical step toward making the metaverse a part of people’s everyday lives through natural and believable avatar interactions. This technology is based on the Face Motion Coding System (FACS), which defines a set of controls to deform the 3D facial mesh according to the position of the user’s facial muscles. The deep learning-based method uses a two-stage architecture: face detection and FACS regression.
To ensure optimum performance, Roblox has implemented a fast variant of the face detection algorithm known as MTCNN. The modified algorithm provides 10x acceleration, allowing for real-time face detection on different devices. In addition, the facial features predicted by MTCNN are used to align the square of the facial circumference before the FACS regression stage, which reduces computation.
The synthetic data pipeline Roblox uses allowed iterative optimization of expression and model strength. The developers added synthetic sequences to enhance the response to missing expressions and to train balanced across different facial identities. Subnetwork temporal filtering of FACS weights, improved backbones, and error-free ground-truth data from synthetic data contribute to high-quality animations with minimal computing.
Since some Roblox experiences already have access to the experimental dynamic face tracking feature, the brief availability of the Avatar Showcase is likely a bug. However, this may also indicate that the technology is nearing its official release.
[ad_2]
Source link