Facial appearance transfer (FAT) is a critical component of various facial editing tasks. It aims to transfer the facial appearance of a reference into a target with good visual consistency. When there are considerable visual differences between a reference and a target, however, it may introduce visual artifacts into the results. To tackle this problem, we propose a facial appearance map with illumination-aware and region-aware properties that allows seamless FAT. We formulate the appearance-map generation as label propagation (LP) on a similarity graph, and propose a new regularization structure to facilitate the adaptive appearance-map diffusion. Solving the original LP model of appearance map in general requires on the order <inline-formula><tex-math notation="LaTeX">$O(kn^2)$</tex-math></inline-formula> time for an <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula>-nodes graph where each node has <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> neighbors. It may be computationally prohibitive for an image with a large spatial resolution. To tackle this problem, we mathematically analyze the graph-based LP model and propose a fast algorithm with smart subset sampling. It selects a subset with <inline-formula><tex-math notation="LaTeX">$m$</tex-math></inline-formula> nodes of the graph with <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> nodes (<inline-formula><tex-math notation="LaTeX">$m\ll n$</tex-math></inline-formula>) to approximate the solution to the original system, which significantly reduces its computational requirements from <inline-formula><tex-math notation="LaTeX">$O(kn^2)$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$O(m^2n)$</tex-math></inline-formula>. Based on the adaptive LP-based appearance map, we construct a framework to achieve various editing effects with FAT, including face replacement, face dubbing, face swapping, and transfiguring. Comparisons with related methods show the effectiveness of the adaptive LP model for FAT. Qualitative and quantitative evaluations verify the computational improvements of the approximation algorithm.
Paper
Full text
Adaptive Label Propagation for Facial Appearance Transfer
Semantic Scholar · Computer Science · 2019
Abstract
Facial appearance transfer (FAT) is a critical component of various facial editing tasks. It aims to transfer the facial appearance of a reference into a target with good visual consistency. When there are considerable visual differences between a reference and a target, however, it may introduce visual artifacts into the results. To tackle this problem, we propose a facial appearance map with illumination-aware and region-aware properties that allows seamless FAT. We formulate the appearance-map generation as label propagation (LP) on a similarity graph, and propose a new regularization structure to facilitate the adaptive appearance-map diffusion. Solving the original LP model of appearance map in general requires on the order <inline-formula><tex-math notation="LaTeX">$O(kn^2)$</tex-math></inline-formula> time for an <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula>-nodes graph where each node has <inline-formula><tex-math notation="LaTeX">$k$</tex-math></inline-formula> neighbors. It may be computationally prohibitive for an image with a large spatial resolution. To tackle this problem, we mathematically analyze the graph-based LP model and propose a fast algorithm with smart subset sampling. It selects a subset with <inline-formula><tex-math notation="LaTeX">$m$</tex-math></inline-formula> nodes of the graph with <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> nodes (<inline-formula><tex-math notation="LaTeX">$m\ll n$</tex-math></inline-formula>) to approximate the solution to the original system, which significantly reduces its computational requirements from <inline-formula><tex-math notation="LaTeX">$O(kn^2)$</tex-math></inline-formula> to <inline-formula><tex-math notation="LaTeX">$O(m^2n)$</tex-math></inline-formula>. Based on the adaptive LP-based appearance map, we construct a framework to achieve various editing effects with FAT, including face replacement, face dubbing, face swapping, and transfiguring. Comparisons with related methods show the effectiveness of the adaptive LP model for FAT. Qualitative and quantitative evaluations verify the computational improvements of the approximation algorithm.