Skip to main content
SpringerLink
Log in

Experiments in evolutionary image enhancement with ELAINE

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

Image enhancement is an image processing procedure in which the image’s original information is refined, for example by highlighting specific features to ease post-processing analyses by a human or machine. This procedure remains challenging since each set of images is often taken under diverse conditions which makes it hard to find an image enhancement solution that fits all conditions. State-of-the-art image enhancement pipelines apply filters that solve specific issues; therefore, it is still hard to generalise these pipelines to all types of problems encountered. We have recently introduced a Genetic Programming approach named ELAINE (EvoLutionAry Image eNhancEment) for evolving image enhancement pipelines based on pre-defined image filters. In this paper, we showcase its potential to create solutions under a real-estate marketing scenario by comparing it with a manual approach and an existing tool for automatic image enhancement. The ELAINE obtained results far exceed those obtained by manual combinations of filters and by the one-click method, in all the metrics explored. We further explore the potential of creating non-photorealistic effects by applying the evolved pipelines to different types of images. The results highlight ELAINE’s potential to transform input images into either suitable real-estate images or non-photorealistic renderings, thus transforming contents and possibly enhancing its aesthetic appeal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. http://elaine.dei.uc.pt

  2. https://www.dpchallenge.com/

References

  1. W. Banzhaf, F.D. Francone, R.E. Keller, P. Nordin, Genetic programming: an introduction: on the automatic evolution of computer programs and its applications (Morgan Kaufmann Publishers Inc., San Francisco, 1998)

    MATH  Google Scholar 

  2. S. Bazeille, I. Quidu, L. Jaulin, J.P. Malkasse, Automatic underwater image pre-processing. Proceedings of CMM’06 (2006)

  3. Y. Bi, B. Xue, M. Zhang, Genetic programming with image-related operators and a flexible program structure for feature learning in image classification. IEEE Trans. Evol. Comput. 25(1), 87–101 (2021). https://doi.org/10.1109/TEVC.2020.3002229

    Article  Google Scholar 

  4. Y. Bi, B. Xue, M. Zhang, Genetic programming with image-related operators and a flexible program structure for feature learning in image classification. IEEE Trans. Evol. Comput. 25(1), 87–101 (2021). https://doi.org/10.1109/TEVC.2020.3002229

    Article  Google Scholar 

  5. A. Buades, B. Coll, J.M. Morel, Non-local means denoising. Image Process. Line 1, 208–212 (2011). https://doi.org/10.5201/ipol.2011.bcm_nlm

    Article  MATH  Google Scholar 

  6. S. Colton, P. Torres, Evolving approximate image filters. In: M. Giacobini, A. Brabazon, S. Cagnoni, G.A.D. Caro, A. Ekárt, A. Esparcia-Alcázar, M. Farooq, A. Fink, P. Machado, J. McCormack, M. O’Neill, F. Neri, M. Preuss, F. Rothlauf, E. Tarantino, S. Yang (eds.) Applications of Evolutionary Computing, EvoWorkshops 2009: EvoCOMNET, EvoENVIRONMENT, EvoFIN, EvoGAMES, EvoHOT, EvoIASP, EvoINTERACTION, EvoMUSART, EvoNUM, EvoSTOC, EvoTRANSLOG, Tübingen, Germany, April 15-17, 2009. Proceedings, Lecture Notes in Computer Science, vol. 5484, pp. 467–477. Springer (2009). https://doi.org/10.1007/978-3-642-01129-0_53

  7. J. Correia, T. Martins, P. Machado, Evolutionary Data Augmentation in Deep Face Detection. In: GECCO 2019—Proceedings of the 2019 Genetic and Evolutionary Computation Conference. Prague, Czech Republic (2019)

  8. J. Correia, L. Vieira, N. Rodriguez-Fernandez, J. Romero, P. Machado, Evolving image enhancement pipelines. In: J. Romero, T. Martins, N. Rodríguez-Fernández (eds.) Artificial Intelligence in Music, Sound, Art and Design—10th International Conference, EvoMUSART 2021, Held as Part of EvoStar 2021, Virtual Event, April 7-9, 2021, Proceedings, Lecture Notes in Computer Science, vol. 12693, pp. 82–97. Springer (2021). https://doi.org/10.1007/978-3-030-72914-1_6

  9. H.T. Esfandarani, P. Milanfar, NIMA: neural image assessment. CoRR http://arxiv.org/abs/1709.05424 (2017)

  10. F.A. Fortin, F.M. De Rainville, M.A. Gardner, M. Parizeau, C. Gagné, DEAP: Evolutionary algorithms made easy. J. Mach. Learn. Res. 13, 2171–2175 (2012)

    MathSciNet  Google Scholar 

  11. E.V. Geert, J. Wagemans, Order, complexity, and aesthetic appreciation. Psych. Aesthet., Creat. Arts 14, 135–154 (2020)

    Article  Google Scholar 

  12. D. Ghadiyaram, T. Goodall, L.K. Choi, A.C. Bovik, Perceptual image enhancement, in Encyclopaedia Image Processing. ed. by P.A. Laplante (CRC Press, Boca Raton, 2018)

    Google Scholar 

  13. L. He, F. Gao, W. Hou, L. Hao, Objective image quality assessment: a survey. Int. J. Computer Math. 91(11), 2374–2388 (2014). https://doi.org/10.1080/00207160.2013.816415

    Article  MathSciNet  MATH  Google Scholar 

  14. J. Immerkær, Fast noise variance estimation. Comput. Vis. Image Underst. 64(2), 300–302 (1996). https://doi.org/10.1006/cviu.1996.0060

    Article  Google Scholar 

  15. C. Johnson, J. McCormack, I. Santos, J. Romero, Understanding aesthetics and fitness measures in evolutionary art systems. Complexity 2019, 1–14 (2019). https://doi.org/10.1155/2019/3495962

    Article  Google Scholar 

  16. J. Lim, M. Heo, C. Lee, C.S. Kim, Contrast enhancement of noisy low-light images based on structure-texture-noise decomposition. J. Visual Commun. Image Represent. 45, 107–121 (2017). https://doi.org/10.1016/j.jvcir.2017.02.016. http://www.sciencedirect.com/science/article/pii/S1047320317300603

  17. N. Limare, J.L. Lisani, J.M. Morel, A.B. Petro, C. Sbert, Simplest color balance. Image Process. On Line (2011). https://doi.org/10.5201/ipol.2011.llmps-scb

    Article  Google Scholar 

  18. P. Machado, A. Cardoso, All the truth about nevar. Appl. Intell. 16(2), 101–118 (2002). https://doi.org/10.1023/A:1013662402341

    Article  MATH  Google Scholar 

  19. P. Machado, J. Romero, M. Nadal, A. Santos, J. Correia, A. Carballal, Computerized measures of visual complexity. Acta Psychologica 160, 43–57 (2015). https://doi.org/10.1016/j.actpsy.2015.06.005. https://www.sciencedirect.com/science/article/pii/S0001691815300160

  20. A. Mittal, A.K. Moorthy, A.C. Bovik, No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21(12), 4695–4708 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  21. C. Munteanu, A. Rosa, Evolutionary image enhancement with user behaviour modeling. ACM SIGAPP Appl. Comput. Rev. 9, 87 (2000). https://doi.org/10.1145/372202.372352

    Article  Google Scholar 

  22. A. Pease, S. Colton, R. Ramezani, J. Charnley, K. Reed, A discussion on serendipity in creative systems. In: M. Maher, T. Veale, R. Saunders, O. Bown (eds.), Proceedings of the 4th International Conference on Computational Creativity, ICCC 2013, pp. 64–71. University of Sydney, Faculty of Architecture, Design and Planning (2013). http://www.computationalcreativity.net/iccc2013/. Fourth International Conference on Computational Creativity, ICCC 2013 ; Conference date: 12-06-2013 Through 14-06-2013

  23. J.L. Pech-Pacheco, G. Cristobal, J. Chamorro-Martinez, J. Fernandez-Valdivia, Diatom autofocusing in brightfield microscopy: a comparative study. In: Proceedings 15th International Conference on Pattern Recognition. ICPR-2000, vol. 3, pp. 314–317 (2000)

  24. E. Peli, Contrast in complex images. J. Opt. Soc. Am. A 7(10), 2032–2040 (1990). https://doi.org/10.1364/JOSAA.7.002032

    Article  Google Scholar 

  25. D. Rex Finley, Hsp color model—alternative to hsv (hsb) and hsl (2006). http://alienryderflex.com/hsp.html

  26. N. Rodriguez-Fernandez, S. Alvarez-Gonzalez, I. Santos, A. Torrente-Patiño, A. Carballal, J. Romero, Validation of an aesthetic assessment system for commercial tasks. Entropy 24(1), (2022). https://doi.org/10.3390/e24010103. https://www.mdpi.com/1099-4300/24/1/103

  27. L. Rundo, A. Tangherloni, M. Nobile, C. Militello, D. Besozzi, G. Mauri, P. Cazzaniga, Medga: a novel evolutionary method for image enhancement in medical imaging systems. Expert Syst. Appl. 119, 87 (2018). https://doi.org/10.1016/j.eswa.2018.11.013

    Article  Google Scholar 

  28. J.C. Russ, Image processing handbook, 4th edn. (CRC Press Inc., USA, 2002)

    Book  MATH  Google Scholar 

  29. T. Shan, S. Wang, X. Zhang, L. Jiao, Automatic image enhancement driven by evolution based on ridgelet frame in the presence of noise, in Applications of evolutionary computing. ed. by F. Rothlauf, J. Branke, S. Cagnoni, D.W. Corne, R. Drechsler, Y. Jin, P. Machado, E. Marchiori, J. Romero, G.D. Smith, G. Squillero (Springer Berlin Heidelberg, Berlin, 2005), pp.304–313

    Chapter  Google Scholar 

  30. H. Talebi, P. Milanfar, Fast multi-layer laplacian enhancement. IEEE Trans. Comput. Imag. (2016). https://doi.org/10.1109/TCI.2016.2607142

    Article  Google Scholar 

  31. G. Wang, L. Li, Q. Li, K. Gu, Z. Lu, J. Qian, Perceptual evaluation of single-image super-resolution reconstruction. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 3145–3149 (2017)

  32. W. Wang, Z. Chen, X. Yuan, X. Wu, Adaptive image enhancement method for correcting low-illumination images. Inf. Sci. 496, 25–41 (2019). https://doi.org/10.1016/j.ins.2019.05.015

    Article  MathSciNet  Google Scholar 

  33. C.Y. Wong, G. Jiang, M.A. Rahman, S. Liu, S.C.F. Lin, N. Kwok, H. Shi, Y.H. Yu, T. Wu, Histogram equalization and optimal profile compression based approach for colour image enhancement. J. Visual Commun. Image Represent. 38, 802–813 (2016). https://doi.org/10.1016/j.jvcir.2016.04.019

    Article  Google Scholar 

  34. S. Zhuo, X. Zhang, X. Miao, T. Sim, Enhancing low light images using near infrared flash images. Proceedings—International Conference on Image Processing, ICIP pp. 2537–2540 (2010). https://doi.org/10.1109/ICIP.2010.5652900

Download references

Acknowledgements

This work is funded by the Foundation for Science and Technology (FCT), I.P./MCTES through national funds (PIDDAC), within the scope of CISUC R &D Unit—UIDB /00326/2020 or project code UIDP/00326/2020 and under the grant SFRH/BD/ 143553/2019. This work is also funded by the INDITEX-UDC Program for predoctoral research stays through the Collaboration Agreement between the UDC and INDITEX for the internationalization of doctoral studies. Juan Romero received funding from Spanish Ministry of Universities for mobility stays of professors and researchers in foreign centres of higher education and research. Juan Romero and Adrian Carballal received funding with reference PID2020-118362RB-I00, from the State Program of R+D+i Oriented to the Challenges of the Society of the Spanish Ministry of Science, Innovation and Universities.

Author information

Authors and Affiliations

  1. Department of Informatics Engineering, Centre for Informatics and Systems of the University of Coimbra, University of Coimbra, Coimbra, Portugal

    João Correia, Daniel Lopes, Leonardo Vieira & Penousal Machado

  2. CITIC—Research Center of Information and Communication Technologies, University of A Coruña, Coruña, Spain

    Nereida Rodriguez-Fernandez, Adrian Carballal & Juan Romero

Authors
  1. João Correia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel Lopes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leonardo Vieira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nereida Rodriguez-Fernandez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Adrian Carballal
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Juan Romero
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Penousal Machado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João Correia.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Correia, J., Lopes, D., Vieira, L. et al. Experiments in evolutionary image enhancement with ELAINE. Genet Program Evolvable Mach 23, 557–579 (2022). https://doi.org/10.1007/s10710-022-09445-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-022-09445-9

Keywords

Search

Navigation