Illya Bakurov
ABSTRACT
The growing production of digital content and its dissemination across the worldwide web require eficient and precise management. In this context, image quality assessment measures (IQAMs) play a pivotal role in guiding the development of numerous image processing systems for compression, enhancement, and restoration. The structural similarity index (SSIM) is one of the most common IQAMs for estimating the similarity between a pristine reference image and its corrupted variant. The multi-scale SSIM is one of its most popular variants that allows assessing image quality at multiple spatial scales. This paper proposes a two-stage genetic programming (GP) approach to evolve novel multi-scale IQAMs, that are simultaneously more effective and efficient. We use GP to perform feature selection in the first stage, while the second stage generates the final solutions. The experimental results show that the proposed approach outperforms the existing MS-SSIM. A comprehensive analysis of the feature selection indicates that, for extracting multi-scale similarities, spatially-varying convolutions are more effective than dilated convolutions. Moreover, we provide evidence that the IQAMs learned for one database can be successfully transferred to previously unseen databases. We conclude the paper by presenting a set of evolved multi-scale IQAMs and providing their interpretation.
- 2022. 24 Noteworthy Video Consumption Statistics [2021 Edition]. https://techjury.net/blog/video-consumption-statistics/#gref. Accessed: 19.01.2022.Google Scholar
- 2022. 31 Mind-Boggling Instagram Stats & Facts for 2022. https://www.wordstream.com/blog/ws/2017/04/20/instagram-statistics. Accessed: 19.01.2022.Google Scholar
- 2022. Global social media statistics research summary 2022. https://www.smartinsights.com/social-media-marketing/social-media-strategy/new-global-social-media-research/. Accessed: 19.01.2022.Google Scholar
- 2022. The State Of Online Video For 2020. https://www.forbes.com/sites/tjmccue/2020/02/05/looking-deep-into-the-state-of-online-video-for-2020/?sh=73ed21902eac. Accessed: 19.01.2022.Google Scholar
- Yuval Bahat and Tomer Michaeli. 2021. What's in the Image? Explorable Decoding of Compressed Images. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 2907--2916.Google Scholar
- Illya Bakurov, Marco Buzzelli, Mauro Castelli, Leonardo Vanneschi, and Raimondo Schettini. 2020. Parameters optimization of the Structural Similarity Index. London Imaging Meeting 2020: Future Colour Imaging 2020, 19--23. Google ScholarCross Ref
- Illya Bakurov, Marco Buzzelli, Mauro Castelli, Leonardo Vanneschi, and Raimondo Schettini. 2021. General Purpose Optimization Library (GPOL): A Flexible and Efficient Multi-Purpose Optimization Library in Python. Applied Sciences 11, 11 (2021). Google ScholarCross Ref
- Illya Bakurov, Marco Buzzelli, Raimondo Schettini, Mauro Castelli, and Leonardo Vanneschi. 2022. Structural similarity index (SSIM) revisited: A data-driven approach. Expert Systems with Applications 189 (2022), 116087. Google ScholarDigital Library
- Illya Bakurov, Leonardo Vanneschi, Mauro Castelli, and Francesco Fontanella. 2018. EDDA-V2--An Improvement of the Evolutionary Demes Despeciation Algorithm. In International Conference on Parallel Problem Solving from Nature. Springer, 185--196.Google ScholarCross Ref
- Simone Bianco, Luigi Celona, and Paolo Napoletano. 2021. Disentangling Image distortions in deep feature space. Pattern Recognition Letters 148 (2021), 128--135. Google ScholarCross Ref
- Simone Bianco, Claudio Cusano, Flavio Piccoli, and Raimondo Schettini. 2020. Personalized Image Enhancement Using Neural Spline Color Transforms. IEEE Transactions on Image Processing 29 (2020), 6223--6236. Google ScholarCross Ref
- S. Bosse, D. Maniry, K. Müller, T. Wiegand, and W. Samek. 2018. Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment. IEEE Transactions on Image Processing 27, 1 (2018), 206--219.Google ScholarCross Ref
- Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin P. Murphy, and Alan Loddon Yuille. 2018. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40 (2018), 834--848.Google ScholarCross Ref
- Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. 2017. Rethinking Atrous Convolution for Semantic Image Segmentation. ArXiv abs/1706.05587 (2017).Google Scholar
- Ionut Cosmin Duta, Li Liu, Fan Zhu, and Ling Shao. 2020. Pyramidal Convolution: Rethinking Convolutional Neural Networks for Visual Recognition. ArXiv.Google Scholar
- Max Ehrlich, Ser-Nam Lim, Larry S. Davis, and Abhinav Shrivastava. 2020. Quantization Guided JPEG Artifact Correction. In ECCV.Google Scholar
- Fei Gao, Yi Wang, Panpeng Li, Min Tan, Jun Yu, and Yani Zhu. 2017. DeepSim: Deep similarity for image quality assessment. Neurocomputing 257 (2017), 104--114.Google ScholarCross Ref
- K. Gu, M. Liu, G. Zhai, X. Yang, and W. Zhang. 2015. Quality Assessment Considering Viewing Distance and Image Resolution. IEEE Transactions on Broadcasting 61, 3 (2015), 520--531.Google ScholarCross Ref
- Justin Johnson, Alexandre Alahi, and Fei Fei Li. 2016. Perceptual Losses for Real-Time Style Transfer and Super-Resolution, Vol. 9906. 694--711. Google ScholarCross Ref
- Ke Gu, Guangtao Zhai, Xiaokang Yang, Wenjun Zhang, and Min Liu. 2013. Structural similarity weighting for image quality assessment. In 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW). 1--6.Google Scholar
- J.R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press.Google ScholarDigital Library
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. [n. d.]. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems. 2012.Google Scholar
- Wenjie Luo, Yujia Li, Raquel Urtasun, and Richard Zemel. 2016. Understanding the Effective Receptive Field in Deep Convolutional Neural Networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems (Barcelona, Spain) (NIPS'16). Curran Associates Inc., Red Hook, NY, USA, 4905--4913.Google ScholarDigital Library
- Davide Mazzini. 2018. Guided Upsampling Network for Real-Time Semantic Segmentation. ArXiv abs/1807.07466 (2018).Google Scholar
- Anush Krishna Moorthy and Alan Conrad Bovik. 2010. A Two-Step Framework for Constructing Blind Image Quality Indices. IEEE Signal Processing Letters 17, 5 (2010), 513--516. Google ScholarCross Ref
- Nikolay Ponomarenko, Lina Jin, O. Ieremeiev, Vladimir Lukin, Karen Egiazarian, J. Astola, Benoit Vozel, Kacem Chehdi, Marco Carli, Federica Battisti, and C.-C. Jay Kuo. 2015. Image database TID2013: Peculiarities, results and perspectives. Signal Processing: Image Communication 30 (01 2015), 57--77. Google ScholarDigital Library
- H.R. Sheikh, A.C. Bovik, and G. de Veciana. 2005. An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Transactions on Image Processing 14, 12 (2005), 2117--2128. Google ScholarDigital Library
- Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1409.1556Google Scholar
- Robert Streijl, Stefan Winkler, and David Hands. 2016. Mean opinion score (MOS) revisited: methods and applications, limitations and alternatives. Multimedia Systems 22 (03 2016), 213--227. Google ScholarDigital Library
- Leonardo Trujillo, Luis Muñoz, Edgar Galván-López, and Sara Silva. 2016. neat Genetic Programming: Controlling bloat naturally. Information Sciences 333 (2016), 21--43. Google ScholarDigital Library
- Leonardo Vanneschi, Illya Bakurov, and Mauro Castelli. 2017. An initialization technique for geometric semantic GP based on demes evolution and despeciation. In Evolutionary Computation (CEC), 2017 IEEE Congress on. IEEE, 113--120.Google ScholarDigital Library
- Leonardo Vanneschi, Mauro Castelli, and Sara Silva. 2010. Measuring Bloat, Overfitting and Functional Complexity in Genetic Programming. In Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation (Portland, Oregon, USA) (GECCO '10). Association for Computing Machinery, New York, NY, USA, 877--884. Google ScholarDigital Library
- Abhinau K. Venkataramanan, Chengyang Wu, Alan Conrad Bovik, Ioannis Katsavounidis, and Zafar Shahid. 2021. A Hitchhiker's Guide to Structural Similarity. IEEE Access 9 (2021), 28872--28896.Google ScholarCross Ref
- Cong Wang, Wanshu Fan, Yutong Wu, and Zhixun Su. 2020. Weakly supervised single image dehazing. Journal of Visual Communication and Image Representation 72 (2020), 102897. Google ScholarCross Ref
- Zhou Wang and A. C. Bovik. 2002. A universal image quality index. IEEE Signal Processing Letters 9 (2002), 81--84.Google ScholarCross Ref
- Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. 2004. Image Quality Assessment: From Error Visibility to Structural Similarity. IEEE TRANSACTIONS ON IMAGE PROCESSING 13, 4 (2004), 600--612.Google ScholarDigital Library
- Zhou Wang and Qiang Li. 2010. Li, Q.: Information content weighting for perceptual image quality assessment. IEEE Image Proc. 20(5), 1185--1198. IEEE transactions on image processing : a publication of the IEEE Signal Processing Society 20 (11 2010), 1185--98. Google ScholarDigital Library
- Zhou Wang, Eero P. Simoncelli, and Alan C. Bovik. 2003. Multi-Scale Structural Similarity for Image Quality Assessment.Google Scholar
- Fisher Yu and Vladlen Koltun. 2016. Multi-Scale Context Aggregation by Dilated Convolutions. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2--4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1511.07122Google Scholar
- Fisher Yu, Vladlen Koltun, and Thomas Funkhouser. 2017. Dilated Residual Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Hu Zhang, Keke Zu1, Jian Lu, Yuru Zou, and Deyu Meng. 2021. EPSANet: An Efficient Pyramid Squeeze Attention Block on Convolutional Neural Network. ArXiv.Google Scholar
- L. Zhang, L. Zhang, X. Mou, and D. Zhang. 2011. FSIM: A Feature Similarity Index for Image Quality Assessment. IEEE Transactions on Image Processing 20, 8 (2011), 2378--2386.Google ScholarDigital Library
- Xiaoshuai Zhang, Wenhan Yang, Yueyu Hu, and Jiaying Liu. 2018. Dmcnn: Dual-Domain Multi-Scale Convolutional Neural Network for Compression Artifacts Removal. In 2018 25th IEEE International Conference on Image Processing (ICIP). 390--394. Google ScholarCross Ref
- Jing Zhao, Ruiqin Xiong, Jizheng Xu, and Tiejun Huang. 2019. Learning a Deep Convolutional Network for Subband Image Denoising. 1420--1425. Google ScholarCross Ref
- Simone Zini, Simone Bianco, and Raimondo Schettini. 2020. Deep residual autoencoder for blind universal jpeg restoration. IEEE Access 8 (2020), 63283--63294.Google ScholarCross Ref
Index Terms
- Genetic programming for structural similarity design at multiple spatial scales
Recommendations
Structural similarity index (SSIM) revisited: A data-driven approach
AbstractSeveral contemporaneous image processing and computer vision systems rely upon the full-reference image quality assessment (IQA) measures. The single-scale structural similarity index (SS-SSIM) is one of the most popular measures, and ...
Highlights- Optimization of the parameters of quality assessment measures.
- Definition of a ...
Joint structural similarity and entropy estimation for coded-exposure image restoration
We address the image deblurring using coded exposure which can keep image content that may be lost by a traditional shutter. In the restoration of a coded exposure image, the automatic estimation of smear length is the key problem. Because the coded ...
Image quality assessment based on improved structural SIMilarity
PCM'12: Proceedings of the 13th Pacific-Rim conference on Advances in Multimedia Information ProcessingIn this paper, we propose a novel image quality assessment (IQA) based on an Improved Structural SIMilarity (ISSIM) which considers the spatial distributions of image structures. The existing structural similarity (SSIM) metric, which measures structure ...
Comments