Face Super-resolution Reconstruction based on Adaptive Global Residual Network

Jinzhao Li; Yali Zhang

doi:10.54691/vnjv1m10

Authors

Jinzhao Li
Yali Zhang

DOI:

https://doi.org/10.54691/vnjv1m10

Keywords:

Face Super-Resolution; Attention Mechanism; Activation Function; Residual Network.

Abstract

At present, some face super-resolution works do not consider the degradation factors of face images in reality, and the resolution of reconstructed face images is not high enough, and there are problems such as blur, smoothness, artifacts, and unclear details. This paper uses adaptive activation function and global attention mechanism to propose an adaptive global residual network AGRNet for face super-resolution reconstruction; then uses a formula with random parameters to simulate the process of face degradation in reality, and uses multi-scale discriminators and composite loss functions to expand it to AGRNet-HR, which can reconstruct degraded faces in reality with higher resolution. The SSIM value and PSNR value of AGRNet on the Helen test set are 0.8352 and 27.54 respectively, the LPIPS value of AGRNet-HR on the CelebAHQ test set is 0.2633, and the FID value on the real test set composed of low-resolution faces and old photos in CelebA is 26.27. Comparing the index values and image visual effects of the experimental results with mainstream methods, it shows that AGRNet and AGRNet-HR are more competitive, and the effectiveness of the key modules of the model is verified through ablation experiments.

Downloads

Download data is not yet available.

References

[1] G Ramponi. Warped distance for space variant linear image interpolation[J]. IEEE Transactions on Image Processing, 2004, 13(5): 629-639.

[2] J W Hwang, H S Lee. Adaptive image interpolation based on local gradient features[J]. IEEE Signal Processing Letters, 2004, 11(3): 359-362.

[3] L Zhang, X Wu. An edge-guided image interpolation algorithm via directional filtering and data fusion[J]. IEEE Transactions on Image Processing, 2006, 15(8): 2226-2238.

[4] A Tekalp, M Ozkan, M Sezan. High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration[C]. IEEE International Conference on Acoustics, 1992: 169-172.

[5] M Aguena, N Mascarenhas. Multispectral image data fusion using POCS and super-resolution[J]. Computer Vision and Image Understanding, 2006, 102(2): 178-187.

[6] H Chang, D Y Yeung, Y Xiong. Super-resolution through neighbor embedding[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008:1-8.

[7] J Yang, J Wright, T Huang, et al. Image super-resolution as sparse representation of raw image patces[C]. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2008: 1-8.

[8] C Ma, Z Y Jiang, Y M Rao, et al. Deep face super-resolution with iterative collaboration between attentive recovery and landmark estimation[C]. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR’20). IEEE, 5569–5578.

[9] C F Chen, D H Gong, H Wang, et al. Learning Spatial Attention for Face Super-Resolution[J].IEEE Transactions on Image Processing, 2021, 30:1219-1231.

[10] C Y Wang, J J Jiang, Z W Zhong, et al. Spatial-Frequency Mutual Learning for Face Super-Resolution[C]. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada, 2023, pp. 22356-22366.

[11] J Y Liang, J Z Cao, G L Sun, et al. SwinIR: Image Restoration Using Swin Transformer[C]. In Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada, 2021, pp. 1833-1844.

[12] Z Liu, Y T Lin, Y Cao, et al. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows[C]. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada, 2021, pp. 9992-10002.

[13] H Guo, J M Li, T Dai, et al. MambaIR: A Simple Baseline forImage Restoration withState-Space Model[C]. European Conference on Computer Vision.Springer, Cham, 2025.DOI:10.1007/978-3-031-72649-1_13.

[14] A Gu, T Dao. Mamba: Linear-Time Sequence Modeling with Selective State Spaces[J]. ArXiv:2312.00752, 2024.

[15] K M He, X Y Zhang, S Q Ren, et al. Deep residual learning for image recognition[C]. In Proceedings of IEEE Coference on Computer Vision and Pattern Recognition. Washington D. C. , USA: IEEE Press, 2016:770-778.

[16] K M He, X Y Zhang, S Q Ren, et al. Identity Mappings in Deep Residual Networks[J]. Springer, Cham, 2016.DOI:10.1007/978-3-319-46493-0_38.

[17] N N Ma, X Y Zhang, M Liu, et al. Activate or not: Learning customized activation[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021: 8032-8042.

[18] R H R Hahnloser, R Sarpeshkar, M A Mahowald, et al. Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit[J]. Nature, 2000, 405(6789): 947-951.

[19] P Ramachandran, B Zoph, Q V Le. Searching for activation functions[P]. ArXiv:1710.05941, 2017.

[20] K M He, X Y Zhang, S Q Ren, et al. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[C]. In Proceedings of the IEEE International Conference on Computer Vision, pages 1026–1034, 2015. 2, 4

[21] Y C Liu, Z Shao, N Hoffmann. Global Attention Mechanism: Retain Information to Enhance Channel-SpatialInteractions[J].2021.DOI:10.48550/arXiv.2112.05561.

[22] S Woo, J Park, J Y Lee,et al. CBAM: Convolutional Block Attention Module[J]. Springer, Cham, 2018. DOI:10.1007/978-3-030-01234-2_1.

[23] K Simonyan, A Zisserman.Very Deep Convolutional Networks for Large-Scale Image Recognition[J]. Computer Science, 2014.DOI:10.48550/arXiv.1409.1556.

[24] D Kingma, J Ba. Adam: A Method for Stochastic Optimization[J]. Computer Science, 2014.DOI:10.48550/ arXiv.1412.6980.

[25] Z Liu, P Luo, X Wang, et al. Deep Learning Face Attributes in the Wild[J]. IEEE, 2016.DOI:10.1109/ICCV. 2015.425.

[26] C Sagonas, G Tzimiropoulos, S Zafeiriou, et al. A Semi-automatic Methodology for Facial Landmark Annotation[J].IEEE, 2013.DOI:10.1109/CVPRW.2013.13 2.

[27] T Karras, S Laine, T Aila, A style-based generator architecture for generative adversarial networks[J]. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 4401–4410, 2019.

[28] T Karras, T Aila, S Laine,et al. Progressive Growing of GANs for Improved Quality, Stability, and Variation[J]. 2017.DOI:10.48550/arXiv.1710.10196.