Self-supervised Pose Adaptation for Cross-Domain Image Animation

Abstract

Image animation is to animate a still image of the object of interest using poses extracted from another video sequence. Through training on a large-scale video dataset, most existing approaches aim to explore disentangled appearance and pose representations of training frames. Then, the desired output with a specific appearance and pose can be synthesized via recombining learned representations. However, in some real applications, test images may lack corresponding video ground- truth or follow a different distribution than training video frames (i.e., different domains), which largely limit the performance of existing works. In this work, we propose domain-independent pose representations that are compatible with and accessible by still images from a different domain. Specifically, we devise a two-stage self-supervised pose adaptation framework for general image animation tasks. A domain-independent pose adaptation generative adversarial network (DIPA-GAN) and a shuffle-patch generative adversarial network (Shuffle-patch GAN) are pro- posed to penalize the rationality of the synthesized frame’s pose and appearance, respectively. Finally, experiments evaluated on various image animation tasks, which include same/cross- domain moving objects, facial expression transfer and human pose retargeting, demonstrate the superiority of the proposed framework over prior works.