StyleGene: Crossover and Mutation of Region-level Facial Genes for Kinship Face Synthesis

Computer Vision and Pattern Recognition (CVPR)

Hao Li    Xianxu Hou     Zepeng Huang     Linlin Shen^*

Shenzhen University

Figure 1: Our StyleGene method synthesizes kinship faces with resemblance to parents, exhibiting diversity and reasonable variations. The first row is the input grandparents, and the second and third rows are their descendants generated by our method.

Abstract

High-fidelity kinship face synthesis has many potential applications, such as kinship verification, missing child identification, and social media analysis. However, it is challenging to synthesize high-quality descendant faces with genetic relations due to the lack of large-scale, high-quality annotated kinship data. This paper proposes RFG (Region-level Facial Gene) extraction framework to address this issue. We propose to use IGE (Image-based Gene Encoder), LGE (Latent-based Gene Encoder) and Gene Decoder to learn the RFGs of a given face image, and the relationships between RFGs and the latent space of StyleGAN2. As cycle-like losses are designed to measure the distances between the output of Gene Decoder and image encoder, and that between the output of LGE and IGE, only face images are required to train our framework, i.e. no paired kinship face data is required. Based upon the proposed RFGs, a crossover and mutation module is further designed to inherit the facial parts of parents. A Gene Pool has also been used to introduce the variations into the mutation of RFGs. The diversity of the faces of descendants can thus be significantly increased. Qualitative, quantitative, and subjective experiments on FIW, TSKinFace, and FF-Databases clearly show that the quality and diversity of kinship faces generated by our approach are much better than the existing state-of-the-art methods.

Figure 2: The overall framework of our method. The image-based gene encoder  learns to encode an independent RFG representation for each facial region in the training stage. The gene decoder  maps RFGs to the space of StyleGAN2. For ease of use, our latent-based gene encoder maps latent codes (obtained by an image inversion encoder to RFGs. In the inference stage, the RFGs of both parents are first extracted using latent-based gene encoder, and then the RFGs of the descendants are assembled by crossover and mutation module. The RFGs will be mapped back to the space using gene decoder, and finally the high-fidelity face is generated by a pre-trained StyleGAN2 generator G.

Figure 3: Example of different weights  used in crossover of RFGs. While the redder regions look more similar to the father, and the bluer regions look more similar to the mother.

Figure 4: Disentangled editing of facial regions.

Figure 5: Comparison of children faces synthesized by StyleGene and baselines. Left most three columns of (a) FIW and (b) FF-Database depict the father, mother, and real children, and right five or three columns depict the faces synthesized by ours, StyleDNA, ChildPredictor, ChildGAN and DNA-Net.

Acknowledgement

This work was jointly supported by National Natural Science Foundation of China (62206180 and 82261138629), Guangdong Basic and Applied Basic Research Foundation (2023A1515010688 and 2022A1515011018), and Shenzhen Municipal Science and Technology Innovation Council (JCYJ20220531101412030).

Bibtex

@inproceedings{

anonymous2023stylegene,

title={StyleGene: Crossover and Mutation of Region-level Facial Genes for Kinship Face Synthesis},

author={Anonymous},

booktitle={Conference on Computer Vision and Pattern Recognition 2023},

year={2023},

url={https://openreview.net/forum?id=2S1WWyS_N2}

}

Downloads