SyncDreamer MAGIC-Talk PortraitTalk

SyncDreamer: Controllable and Expressive Avatar Generation Beyond the Talking Head

CVPR 2026 Highlight
Fatemeh Nazarieh1*, Zhenhua Feng2, Diptesh Kanojia1, Josef Kittler3,4, Muhammad Awais3,4
1School of Computer Science and Electronic Engineering, University of Surrey, UK
2School of Artificial Intelligence and Computer Science, Jiangnan University, China
3Centre for Vision, Speech and Signal Processing, University of Surrey, UK
4Institute for People-Centered AI, University of Surrey, UK
* Corresponding author
Paper Code

Abstract

Generating realistic and expressive talking avatars from minimal input remains a central challenge in digital human synthesis. We introduce SyncDreamer, a diffusion-based framework that learns synchronized, semantically grounded audio–visual generation of expressive avatars. Our model integrates an audio dynamics encoder, a visual adapter, and a text-conditioned diffusion process to produce temporally coherent, controllable facial motion while preserving identity under diverse poses and expressions.

Experiments demonstrate state-of-the-art performance in expressivity and identity consistency, enabling new possibilities in virtual communication, entertainment, and personalized digital avatars.

Results and Demonstrations

Expressive Avatar Generation

SyncDreamer produces expressive talking avatars from a single reference image and speech audio, capturing emotion and natural head motion while maintaining identity.

Singing Avatar Generation

SyncDreamer enables singing avatar generation by capturing fine-grained audio dynamics, producing temporally synchronized motion that reflects musical rhythm, pitch variation, and expressive performance.

Robustness to Audio Variability

SyncDreamer demonstrates strong robustness to variations in audio characteristics such as rhythm, energy, and speaker identity. Using the same reference image, our model produces synchronized and expressive motion that adapts naturally to each unique input voice.

Reference ID

Reference ID