Skip to main content

Advertisement

Log in

A genetic programming framework in the automatic design of combination models for salient object detection

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

In computer vision, the salient object detection problem consists of finding the most attention-grabbing objects in images. In the last years, many researchers have proposed salient object detection algorithms to address this problem. However, most of the algorithms perform well only on images with specific conditions and they do not solve the general problem. To cope with a more significant number of image types than those where each standalone saliency detection method performs well, novel methods search to generate a combination model that improves the overall performance of detecting salient objects in images. The contribution of this work is oriented towards the automatic design of combination models by using genetic programming. The proposed approach automatically selects the algorithms to be combined and the combination operators that result in an improvement in the overall performance. The evolutionary approach uses as input a set of candidate saliency detection methods and a set of combination operators. The set of input saliency detection methods includes algorithms from the state-of-the-art. The set of combination operators includes fuzzy logic combination rules, morphological operations, and image processing filters. The outcome of each run of the evolutionary process is a combination model that describes how the input models have to be combined. An advantage of the proposed approach is that these models explain and give insight about which standalone methods are important to improve the response in the solution of the saliency detection problem. The improvement of the final combination models is demonstrated by comparing their performance against that of several state-of-the-art saliency detection methods, that of several classic combination models, and that of other evolutionary computation-based approaches, on four benchmark datasets. The results were analyzed using two statistical tests, the Wilcoxon rank-sum test, and the t-test. Both tests confirmed that the proposed approach outperforms all of the other algorithms under test and that its performance advantage is statistically significant.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. The FT and HC maps were computed using the code from https://github.com/MingMingCheng/CmCode.

  2. The source code of the MDC method is available at https://github.com/huangxm14-thu/SaliencyMDC.

  3. The MBS method was taken from https://github.com/jimmie33/MBS.

References

  1. R. Achanta, F. Estrada, P. Wils, S. Süsstrunk, Salient region detection and segmentation, in Proceedings of the International Conference on Computer Vision Systems (Springer, 2008), pp. 66–75

  2. R. Achanta, S. Hemami, F. Estrada, S. Susstrunk, Frequency-tuned salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604

  3. S. Afzali, B. Xue, H. Al-Sahaf, M. Zhang, A supervised feature weighting method for salient object detection using particle swarm optimization, in Computational Intelligence (SSCI), 2017 IEEE Symposium Series on (IEEE, 2017), pp. 1–8

  4. H. Al-Sahaf, A. Al-Sahaf, B. Xue, M. Johnston, M. Zhang, Automatically evolving rotation-invariant texture image descriptors by genetic programming. IEEE Trans. Evolut. Comput. 21(1), 83–101 (2017)

    Google Scholar 

  5. H. Al-Sahaf, A. Song, K. Neshatian, M. Zhang, Two-tier genetic programming: towards raw pixel-based image classification. Expert Syst. Appl. 39(16), 12291–12301 (2012)

    Article  Google Scholar 

  6. H. Al-Sahaf, M. Zhang, A. Al-Sahaf, M. Johnston, Keypoints detection and feature extraction: a dynamic genetic programming approach for evolving rotation-invariant texture image descriptors. IEEE Trans. Evolut. Comput. 21(6), 825–844 (2017)

    Article  Google Scholar 

  7. H. Al-Sahaf, M. Zhang, M. Johnston, Binary image classification: a genetic programming approach to the problem of limited training instances. Evolut. comput. 24(1), 143–182 (2016)

    Article  Google Scholar 

  8. B. Alexe, T. Deselaers, V. Ferrari, What is an object?, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 73–80

  9. S. Alpert, M. Galun, R. Basri, A. Brandt, image segmentation by probabilistic bottom-up aggregation and cue integration, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2007)

  10. D. Batra, A. Kowdle, D. Parikh, J. Luo, T. Chen, icoseg: Interactive co-segmentation with intelligent scribble guidance, in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on (IEEE, 2010), pp. 3169–3176

  11. S. Bianco, G. Ciocca, R. Schettini, Combination of video change detection algorithms by genetic programming. IEEE Trans. Evolut. Comput. 21, 914–928 (2017)

    Article  Google Scholar 

  12. A. Borgi, D.N. Sihite, L. Itti, Salient object detection: a benchmark, in Proceedings of the European Conference on Computer Vision (Springer, Berlin, Heidelberg, 2012), pp. 414–429

  13. A. Borji, M. Cheng, H. Jiang, J. Li, Salient object detection: a survey. CoRR (2014). arXiv:1411.5878

  14. A. Borji, M.M. Cheng, H. Jiang, J. Li, Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  15. A. Borji, L. Itti, State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 185–207 (2013)

    Article  Google Scholar 

  16. N. Bruce, J. Tsotsos, Saliency based on information maximization, in Advances in Neural Information Processing Systems (NIPS, 2006), pp. 155–162

  17. K.Y. Chang, T.L. Liu, H.T. Chen, S.H. Lai, Fusing generic objectness and visual saliency for salient object detection, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2011), pp. 914–921

  18. S. Chen, L. Zheng, X. Hu, P. Zhou, A comparative study of saliency aggregation for salient object detection, in Proceedings of the International Conference on Image and Graphics (Springer, 2015), pp. 30–42

  19. M.M. Cheng, N.J. Mitra, X. Huang, P.H. Torr, S.M. Hu, Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)

    Article  Google Scholar 

  20. M.M. Cheng, G.X. Zhang, N.J. Mitra, X. Huang, S.M. Hu, Global contrast based salient region detection, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416

  21. L. Dozal, G. Olague, E. Clemente, D.E. Hernández, Brain programming for the evolution of an artificial dorsal stream. Cognit. Comput. 6(3), 528–557 (2014)

    Article  Google Scholar 

  22. M. Du, X. Wu, W. Chen, Z. Li, Supervised training and contextually guided salient object detection. Digit. Signal Process. 63, 44–55 (2017)

    Article  Google Scholar 

  23. A.E. Eiben, J. Smith, From evolutionary computation to the evolution of things. Nature 521(7553), 476–482 (2015)

    Article  Google Scholar 

  24. Y. Fang, Z. Chen, W. Lin, C.W. Lin, Saliency-based image retargeting in the compressed domain, in Proceedings of the ACM International Conference on Multimedia (ACM, 2011), pp. 1049–1052

  25. Y. Fang, W. Lin, B.S. Lee, C.T. Lau, Z. Chen, C.W. Lin, Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum. IEEE Trans. Multimed. 14(1), 187–198 (2012)

    Article  Google Scholar 

  26. H. Fu, X. Cao, Z. Tu, Cluster-based co-saliency detection. IEEE Trans. Image Process. 22(10), 3766–3778 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  27. W. Fu, M. Johnston, M. Zhang, Genetic programming for edge detection: a gaussian-based approach. Soft Comput. 20(3), 1231–1248 (2016)

    Article  Google Scholar 

  28. A. Garcia-Diaz, X.R. Fdez-Vidal, X.M. Pardo, R. Dosil, Decorrelation and distinctiveness provide with human-like saliency, in International Conference on Advanced Concepts for Intelligent Vision Systems (Springer, 2009), pp. 343–354

  29. S. Goferman, L. Zelnik-Manor, A. Tal, Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)

    Article  Google Scholar 

  30. J. Harel, C. Koch, P. Perona, Graph-based visual saliency, in Advances in Neural Information Processing Systems (NIPS, 2007), pp. 545–552

  31. X. Hou, L. Zhang, Saliency detection: A spectral residual approach, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–8

  32. X. Hou, L. Zhang, Dynamic visual attention: searching for coding length increments, in Advances in Neural Information Processing Systems (NIPS, 2009), pp. 681–688

  33. X. Huang, Y. Zhan, 300 fps salient object detection via minimum directional contrast. IEEE Trans. Image Process. 26, 4243–4254 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  34. M. Iqbal, S.S. Naqvi, W.N. Browne, C. Hollitt, M. Zhang, Learning feature fusion strategies for various image types to detect salient objects. Pattern Recognit. 60, 106–120 (2016)

    Article  Google Scholar 

  35. M. Iqbal, B. Xue, H. Al-Sahaf, M. Zhang, Cross-domain reuse of extracted knowledge in genetic programming for image classification. IEEE Trans. Evolut. Comput. 21(4), 569–587 (2017)

    Article  Google Scholar 

  36. L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)

    Article  Google Scholar 

  37. B. Jiang, L. Zhang, H. Lu, C. Yang, M.H. Yang, Saliency detection via absorbing markov chain, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2013), pp. 1665–1672

  38. H. Jiang, J. Wang, Z. Yuan, T. Liu, N. Zheng, S. Li, Automatic salient object segmentation based on context and shape prior, in Proceedings of the British Machine Vision Conference, vol. 6 (2011), p. 9

  39. H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, S. Li, Salient object detection: a discriminative regional feature integration approach, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 2083–2090

  40. Z. Jiang, L.S. Davis, Submodular salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 2043–2050

  41. T. Judd, K. Ehinger, F. Durand, A. Torralba, Learning to predict where humans look, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2009), pp. 2106–2113

  42. J. Kim, D. Han, Y.W. Tai, J. Kim, Salient region detection via high-dimensional color transform, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014), pp. 883–890

  43. J. Kim, V. Pavlovic, A shape preserving approach for salient object detection using convolutional neural networks, in Proceedings of the IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 609–614

  44. C. Koch, S. Ullman, Shifts in selective visual attention: towards the underlying neural circuitry, in Matters of Intelligence, ed. by L.M. Vaina (Springer, 1987), pp. 115–141

  45. J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, vol. 1 (MIT press, Cambridge, 1992)

    MATH  Google Scholar 

  46. J.R. Koza, Human-competitive results produced by genetic programming. Genet. Program. Evol. Mach. 11(3–4), 251–284 (2010)

    Article  Google Scholar 

  47. K. Krawiec, B. Bhanu, Visual learning by evolutionary and coevolutionary feature synthesis. IEEE Trans. Evolut. Comput. 11(5), 635–650 (2007)

    Article  Google Scholar 

  48. O. Le Meur, Z. Liu, Saliency aggregation: Does unity make strength? in Asian Conference on Computer Vision (Springer, 2014), pp. 18–32

  49. G. Li, Y. Yu, Visual saliency detection based on multiscale deep cnn features. IEEE Trans. Image Process. 25(11), 5012–5024 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  50. H. Li, J. Chen, H. Lu, Z. Chi, Cnn for saliency detection with low-level feature integration. Neurocomputing 226, 212–220 (2017)

    Article  Google Scholar 

  51. J. Li, Y. Tian, L. Duan, T. Huang, Estimating visual saliency through single image optimization. IEEE Signal Process. Lett. 20(9), 845–848 (2013)

    Article  Google Scholar 

  52. X. Li, H. Lu, L. Zhang, X. Ruan, M.H. Yang, Saliency detection via dense and sparse reconstruction, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2013), pp. 2976–2983

  53. Y. Liang, M. Zhang, W.N. Browne, Genetic programming for evolving figure-ground segmentors from multiple features. Appl. Soft Comput. 51, 83–95 (2017)

    Article  Google Scholar 

  54. Y. Liang, M. Zhang, W.N. Browne, Image feature selection using genetic programming for figure-ground segmentation. Eng. Appl. Artif. Intell. 62, 96–108 (2017)

    Article  Google Scholar 

  55. N. Liu, J. Han, DHSNet: Deep hierarchical saliency network for salient object detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 678–686

  56. T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, H.Y. Shum, Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 353–367 (2011)

    Article  Google Scholar 

  57. Z. Liu, W. Zou, O. Le Meur, Saliency tree: a novel saliency detection framework. IEEE Trans. Image Process. 23(5), 1937–1952 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  58. J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 3431–3440

  59. Y.F. Ma, H.J. Zhang, Contrast-based image attention analysis by using fuzzy growing, in Proceedings of the ACM International Conference on Multimedia (ACM, 2003), pp. 374–381

  60. L. Mai, Y. Niu, F. Liu, Saliency aggregation: a data-driven approach, in Proceedings of the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 1131–1138

  61. N. Murray, M. Vanrell, X. Otazu, C.A. Parraga, Saliency estimation using a non-parametric low-level vision model, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 433–440

  62. S.S. Naqvi, W.N. Browne, C. Hollitt, Combining object-based local and global feature statistics for salient object search, in Proceedings of the IEEE International Conference on Image and Vision Computing New Zealand (IEEE, 2013), pp. 394–399

  63. S.S. Naqvi, W.N. Browne, C. Hollitt, Genetic algorithms based feature combination for salient object detection, for autonomously identified image domain types, in Proceedings of the IEEE Congress on Evolutionary Computation (IEEE, 2014), pp. 109–116

  64. S.S. Naqvi, W.N. Browne, C. Hollitt, Feature quality-based dynamic feature selection for improving salient object detection. IEEE Trans. Image Process. 25(9), 4298–4313 (2016)

    MathSciNet  MATH  Google Scholar 

  65. T.V. Nguyen, M. Kankanhalli, As-similar-as-possible saliency fusion. Multimed. Tools Appl. 76(8), 10501–10519 (2017)

    Article  Google Scholar 

  66. Y. Niu, L. Lin, Y. Chen, L. Ke, Machine learning-based framework for saliency detection in distorted images. Multimed. Tools Appl. 76(24), 26329–26353 (2017)

    Article  Google Scholar 

  67. G. Olague, L. Trujillo, Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming. Image Vis. Comput. 29(7), 484–498 (2011)

    Article  Google Scholar 

  68. F. Perazzi, P. Krähenbühl, Y. Pritch, A. Hornung, Saliency filters: contrast based filtering for salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 733–740

  69. R. Poli, W.B. Langdon, N.F. McPhee, Field Guide to Genetic Programming. Online website (2008)

  70. J. Shi, Q. Yan, L. Xu, J. Jia, Hierarchical image saliency detection on extended cssd. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 717–729 (2016)

    Article  Google Scholar 

  71. N. Singh, R. Arya, R. Agrawal, A novel approach to combine features for salient object detection using constrained particle swarm optimization. Pattern Recognit. 47(4), 1731–1739 (2014)

    Article  Google Scholar 

  72. W. Tan, B. Yan, Salient object detection via multiple saliency weights. Multimed. Tools Appl. 76(23), 25091–25107 (2017)

    Article  Google Scholar 

  73. N. Tong, H. Lu, X. Ruan, M.H. Yang, Salient object detection via bootstrap learning, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 1884–1892

  74. A.M. Treisman, G. Gelade, A feature-integration theory of attention. Cognit. Psychol. 12(1), 97–136 (1980)

    Article  Google Scholar 

  75. L. Vanneschi, S. Silva, M. Castelli, L. Manzoni, Geometric semantic genetic programming for real life applications, in Genetic Programming Theory and Practice XI, ed. by R. Riolo, J. Moore, M. Kotanchek (Springer, 2014), pp. 191–209

  76. H. Vojodi, A. Fakhari, A.M.E. Moghadam, A new evaluation measure for color image segmentation based on genetic programming approach. Image Vis. Comput. 31(11), 877–886 (2013)

    Article  Google Scholar 

  77. D. Walther, C. Koch, Modeling attention to salient proto-objects. Neural Netw. 19(9), 1395–1407 (2006)

    Article  MATH  Google Scholar 

  78. J. Wang, A. Borji, C.C.J. Kuo, L. Itti, Learning a combined model of visual saliency for fixation prediction. IEEE Trans. Image Process. 25(4), 1566–1579 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  79. L. Wang, H. Lu, X. Ruan, M.H. Yang, Deep networks for saliency detection via local estimation and global search, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 3183–3192

  80. X. Wei, Z. Tao, C. Zhang, X. Cao, Structured saliency fusion based on dempster-shafer theory. IEEE Signal Process. Lett. 22(9), 1345–1349 (2015)

    Article  Google Scholar 

  81. C. Xie, L. Shang, Anomaly detection in crowded scenes using genetic programming, in Proceedings of the IEEE Congress on Evolutionary Computation (IEEE, 2014), pp. 1832–1839

  82. Q. Yan, L. Xu, J. Shi, J. Jia, Hierarchical saliency detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 1155–1162

  83. C. Yang, L. Zhang, H. Lu, X. Ruan, M.H. Yang, Saliency detection via graph-based manifold ranking, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (IEEE, 2013), pp. 3166–3173

  84. Y. Yuan, C. Li, J. Kim, W. Cai, D.D. Feng, Dense and sparse labeling with multi-dimensional features for saliency detection. IEEE Trans. Circuits Syst. Video Technol. 28, 1130–1143 (2018)

    Article  Google Scholar 

  85. Y. Zhai, M. Shah, Visual attention detection in video sequences using spatiotemporal cues, in Proceedings of the ACM International Conference on Multimedia (ACM, 2006), pp. 815–824

  86. J. Zhang, S. Sclaroff, Z. Lin, X. Shen, B. Price, R. Mech, Minimum barrier salient object detection at 80 fps, in Proceedings of the IEEE International Conference on Computer Vision (2015), pp. 1404–1412

  87. L. Zhang, J. Li, H. Lu, Saliency detection via extreme learning machine. Neurocomputing 218, 103–112 (2016)

    Article  Google Scholar 

  88. L. Zhang, M.H. Tong, T.K. Marks, H. Shan, G.W. Cottrell, Sun: A bayesian framework for saliency using natural statistics. J. Vis. 8(7), 32–32 (2008)

    Article  Google Scholar 

  89. W. Zhu, S. Liang, Y. Wei, J. Sun, Saliency optimization from robust background detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014), pp. 2814–2821

Download references

Acknowledgements

Marco A. Contreras-Cruz and Diana E. Martinez-Rodriguez thank to National Council of Science and Technology (CONACYT) for the financial support through the scholarships with Identification Numbers 568675/302121 and 736576/291047, respectively.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Victor Ayala-Ramirez.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Contreras-Cruz, M.A., Martinez-Rodriguez, D.E., Hernandez-Belmonte, U.H. et al. A genetic programming framework in the automatic design of combination models for salient object detection. Genet Program Evolvable Mach 20, 285–325 (2019). https://doi.org/10.1007/s10710-019-09345-5

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-019-09345-5

Keywords

Navigation