Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction

We present dynamic neural radiance fields for modeling the appearance and\ndynamics of a human face. Digitally modeling and reconstructing a talking human\nis a key building-block for a variety of applications. Especially, for\ntelepresence applications in AR or VR, a faithful reproduction of the\nappearance including novel viewpoints or head-poses is required. In contrast to\nstate-of-the-art approaches that model the geometry and material properties\nexplicitly, or are purely image-based, we introduce an implicit representation\nof the head based on scene representation networks. To handle the dynamics of\nthe face, we combine our scene representation network with a low-dimensional\nmorphable model which provides explicit control over pose and expressions. We\nuse volumetric rendering to generate images from this hybrid representation and\ndemonstrate that such a dynamic neural scene representation can be learned from\nmonocular input data only, without the need of a specialized capture setup. In\nour experiments, we show that this learned volumetric representation allows for\nphoto-realistic image generation that surpasses the quality of state-of-the-art\nvideo-based reenactment methods.\n

Paper

Similar papers

© 2026 NYSGPT2525 LLC