In this paper, we propose FrankenMask, a novel framework that allows swapping and rearranging face parts in semantic masks for automatic editing of shape-related facial attributes. This is a novel yet challenging task as substituting face parts in a semantic mask requires to account for possible spatial misalignment and the adaptation of surrounding regions. We obtain such a feature by combining a Transformer encoder to learn the spatial relationships of facial parts, with an encoder–decoder architecture, which reconstructs a complete mask from the composition of local parts. Reconstruction and attribute classification results demonstrate the effective synthesis of facial images, while showing the generation of accurate and plausible facial attributes. Code is available at https://github.com/TFonta/FrankenMask_semantic
FrankenMask: Manipulating semantic masks with transformers for face parts editing / Fontanini, Tomaso; Ferrari, Claudio; Lisanti, Giuseppe; Galteri, Leonardo; Berretti, Stefano; Bertozzi, Massimo; Prati, Andrea. - In: PATTERN RECOGNITION LETTERS. - ISSN 0167-8655. - STAMPA. - 176:(2023), pp. 14-20. [10.1016/j.patrec.2023.10.010]
FrankenMask: Manipulating semantic masks with transformers for face parts editing
Ferrari, Claudio;Lisanti, Giuseppe;Galteri, Leonardo;Berretti, Stefano;
2023
Abstract
In this paper, we propose FrankenMask, a novel framework that allows swapping and rearranging face parts in semantic masks for automatic editing of shape-related facial attributes. This is a novel yet challenging task as substituting face parts in a semantic mask requires to account for possible spatial misalignment and the adaptation of surrounding regions. We obtain such a feature by combining a Transformer encoder to learn the spatial relationships of facial parts, with an encoder–decoder architecture, which reconstructs a complete mask from the composition of local parts. Reconstruction and attribute classification results demonstrate the effective synthesis of facial images, while showing the generation of accurate and plausible facial attributes. Code is available at https://github.com/TFonta/FrankenMask_semanticFile | Dimensione | Formato | |
---|---|---|---|
prl2023_faceswap.pdf
Accesso chiuso
Tipologia:
Versione finale referata (Postprint, Accepted manuscript)
Licenza:
Tutti i diritti riservati
Dimensione
2.37 MB
Formato
Adobe PDF
|
2.37 MB | Adobe PDF | Richiedi una copia |
I documenti in FLORE sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.