Neural Implicit Morphing of Face Images

Guilherme Schardong1
Tiago Novello2
Hallison Paz2
Iurii Medvedev1
Vinícius da Silva3
Luiz Velho2
Nuno Gonçalves1,4
1ISR, University of Coimbra
2IMPA
3PUC-Rio
4INCM
description Paper code Code image Poster

Results of morphings using our approach. On the left, a morphing between an image of Yann Lecun and an image of Geoffrey Hinton, also as a tribute for all the researchers that helped establish the field. On the right, a morphing between two images in the FRLL-Morphs dataset. Our approach takes landmarks into consideration, resulting in an aligment of the most important face features during the entire morphing process.

News

description [Feb 26th 2024] Paper accepted to #CVPR24.
description [Mar 2nd 2024] Yann Lecun mentioned this work on X!
description [Mar 06th 2024] Page online.

Capabilities

The approach supports subjects with different facial expressions. In the example below the target is smiling, but the source is not.

The faces may also be in different poses. In this example, the target is in a 45º profile and we morph until the middle between source and target. Notice that our approach is the only one where the eyes are not looking at the camera, which would be the expected behavior.

It also deals with occlusion and faces in the wild. In this example the eyes are occluded by the glasses.

It is possible to transfer features between faces.

We use the Poisson blending to define a boundary value problem to selectively blend parts of the target and source images. The example shows different cloning strategies of the half-space region of I_1 into I_0.

Our approach may also be used to align images for generative morphing using Diffusion Autoencoders (diffAE). Line 1 presents a morphing between two faces using our neural warping + diffAE. Line 2 shows the results of diffAE using no alignment.

Paper

Neural Implicit Morphing of Face Images

Guilherme Schardong, Tiago Novello, Hallison Paz, Iurii Medvedev, Vinícius da Silva, Luiz Velho, and Nuno Gonçalves.

description Paper preprint (PDF)
description Arxiv
insert_comment BibTeX

Citation

@inproceedings{schardong2024neural,
  title = {Neural Implicit Morphing of Face Images},
  author = {Schardong, Guilherme and Novello, Tiago and Paz, Hallison and Medvedev, Iurii and Silva, Vin{\'\i}icius da and Velho, Luiz and Gon\c{c}alves, Nuno},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2024},
  month = {June},
  pages = {7321-7330}
}

Poster

Acknowledgements

We would like to thank Daniel Perazzo for help with the landmark interface. We would also like to thank colleagues from ISR/UC and Visgraf/IMPA for discussions and suggestions that helped us improve our work.
The authors would like to thank Fundação de Ciência e Tecnologia (FCT project UIDB/00048/2020) Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq grant 150991/2023-1) and Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro (FAPERJ) for funding this work.