Abstract
In computer vision, the salient object detection problem consists of finding the most attention-grabbing objects in images. In the last years, many researchers have proposed salient object detection algorithms to address this problem. However, most of the algorithms perform well only on images with specific conditions and they do not solve the general problem. To cope with a more significant number of image types than those where each standalone saliency detection method performs well, novel methods search to generate a combination model that improves the overall performance of detecting salient objects in images. The contribution of this work is oriented towards the automatic design of combination models by using genetic programming. The proposed approach automatically selects the algorithms to be combined and the combination operators that result in an improvement in the overall performance. The evolutionary approach uses as input a set of candidate saliency detection methods and a set of combination operators. The set of input saliency detection methods includes algorithms from the state-of-the-art. The set of combination operators includes fuzzy logic combination rules, morphological operations, and image processing filters. The outcome of each run of the evolutionary process is a combination model that describes how the input models have to be combined. An advantage of the proposed approach is that these models explain and give insight about which standalone methods are important to improve the response in the solution of the saliency detection problem. The improvement of the final combination models is demonstrated by comparing their performance against that of several state-of-the-art saliency detection methods, that of several classic combination models, and that of other evolutionary computation-based approaches, on four benchmark datasets. The results were analyzed using two statistical tests, the Wilcoxon rank-sum test, and the t-test. Both tests confirmed that the proposed approach outperforms all of the other algorithms under test and that its performance advantage is statistically significant.
Similar content being viewed by others
Notes
The FT and HC maps were computed using the code from https://github.com/MingMingCheng/CmCode.
The source code of the MDC method is available at https://github.com/huangxm14-thu/SaliencyMDC.
The MBS method was taken from https://github.com/jimmie33/MBS.
References
R. Achanta, F. Estrada, P. Wils, S. Süsstrunk, Salient region detection and segmentation, in Proceedings of the International Conference on Computer Vision Systems (Springer, 2008), pp. 66–75
R. Achanta, S. Hemami, F. Estrada, S. Susstrunk, Frequency-tuned salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2009), pp. 1597–1604
S. Afzali, B. Xue, H. Al-Sahaf, M. Zhang, A supervised feature weighting method for salient object detection using particle swarm optimization, in Computational Intelligence (SSCI), 2017 IEEE Symposium Series on (IEEE, 2017), pp. 1–8
H. Al-Sahaf, A. Al-Sahaf, B. Xue, M. Johnston, M. Zhang, Automatically evolving rotation-invariant texture image descriptors by genetic programming. IEEE Trans. Evolut. Comput. 21(1), 83–101 (2017)
H. Al-Sahaf, A. Song, K. Neshatian, M. Zhang, Two-tier genetic programming: towards raw pixel-based image classification. Expert Syst. Appl. 39(16), 12291–12301 (2012)
H. Al-Sahaf, M. Zhang, A. Al-Sahaf, M. Johnston, Keypoints detection and feature extraction: a dynamic genetic programming approach for evolving rotation-invariant texture image descriptors. IEEE Trans. Evolut. Comput. 21(6), 825–844 (2017)
H. Al-Sahaf, M. Zhang, M. Johnston, Binary image classification: a genetic programming approach to the problem of limited training instances. Evolut. comput. 24(1), 143–182 (2016)
B. Alexe, T. Deselaers, V. Ferrari, What is an object?, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2010), pp. 73–80
S. Alpert, M. Galun, R. Basri, A. Brandt, image segmentation by probabilistic bottom-up aggregation and cue integration, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2007)
D. Batra, A. Kowdle, D. Parikh, J. Luo, T. Chen, icoseg: Interactive co-segmentation with intelligent scribble guidance, in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on (IEEE, 2010), pp. 3169–3176
S. Bianco, G. Ciocca, R. Schettini, Combination of video change detection algorithms by genetic programming. IEEE Trans. Evolut. Comput. 21, 914–928 (2017)
A. Borgi, D.N. Sihite, L. Itti, Salient object detection: a benchmark, in Proceedings of the European Conference on Computer Vision (Springer, Berlin, Heidelberg, 2012), pp. 414–429
A. Borji, M. Cheng, H. Jiang, J. Li, Salient object detection: a survey. CoRR (2014). arXiv:1411.5878
A. Borji, M.M. Cheng, H. Jiang, J. Li, Salient object detection: a benchmark. IEEE Trans. Image Process. 24(12), 5706–5722 (2015)
A. Borji, L. Itti, State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 185–207 (2013)
N. Bruce, J. Tsotsos, Saliency based on information maximization, in Advances in Neural Information Processing Systems (NIPS, 2006), pp. 155–162
K.Y. Chang, T.L. Liu, H.T. Chen, S.H. Lai, Fusing generic objectness and visual saliency for salient object detection, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2011), pp. 914–921
S. Chen, L. Zheng, X. Hu, P. Zhou, A comparative study of saliency aggregation for salient object detection, in Proceedings of the International Conference on Image and Graphics (Springer, 2015), pp. 30–42
M.M. Cheng, N.J. Mitra, X. Huang, P.H. Torr, S.M. Hu, Global contrast based salient region detection. IEEE Trans. Pattern Anal. Mach. Intell. 37(3), 569–582 (2015)
M.M. Cheng, G.X. Zhang, N.J. Mitra, X. Huang, S.M. Hu, Global contrast based salient region detection, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 409–416
L. Dozal, G. Olague, E. Clemente, D.E. Hernández, Brain programming for the evolution of an artificial dorsal stream. Cognit. Comput. 6(3), 528–557 (2014)
M. Du, X. Wu, W. Chen, Z. Li, Supervised training and contextually guided salient object detection. Digit. Signal Process. 63, 44–55 (2017)
A.E. Eiben, J. Smith, From evolutionary computation to the evolution of things. Nature 521(7553), 476–482 (2015)
Y. Fang, Z. Chen, W. Lin, C.W. Lin, Saliency-based image retargeting in the compressed domain, in Proceedings of the ACM International Conference on Multimedia (ACM, 2011), pp. 1049–1052
Y. Fang, W. Lin, B.S. Lee, C.T. Lau, Z. Chen, C.W. Lin, Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum. IEEE Trans. Multimed. 14(1), 187–198 (2012)
H. Fu, X. Cao, Z. Tu, Cluster-based co-saliency detection. IEEE Trans. Image Process. 22(10), 3766–3778 (2013)
W. Fu, M. Johnston, M. Zhang, Genetic programming for edge detection: a gaussian-based approach. Soft Comput. 20(3), 1231–1248 (2016)
A. Garcia-Diaz, X.R. Fdez-Vidal, X.M. Pardo, R. Dosil, Decorrelation and distinctiveness provide with human-like saliency, in International Conference on Advanced Concepts for Intelligent Vision Systems (Springer, 2009), pp. 343–354
S. Goferman, L. Zelnik-Manor, A. Tal, Context-aware saliency detection. IEEE Trans. Pattern Anal. Mach. Intell. 34(10), 1915–1926 (2012)
J. Harel, C. Koch, P. Perona, Graph-based visual saliency, in Advances in Neural Information Processing Systems (NIPS, 2007), pp. 545–552
X. Hou, L. Zhang, Saliency detection: A spectral residual approach, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2007), pp. 1–8
X. Hou, L. Zhang, Dynamic visual attention: searching for coding length increments, in Advances in Neural Information Processing Systems (NIPS, 2009), pp. 681–688
X. Huang, Y. Zhan, 300 fps salient object detection via minimum directional contrast. IEEE Trans. Image Process. 26, 4243–4254 (2017)
M. Iqbal, S.S. Naqvi, W.N. Browne, C. Hollitt, M. Zhang, Learning feature fusion strategies for various image types to detect salient objects. Pattern Recognit. 60, 106–120 (2016)
M. Iqbal, B. Xue, H. Al-Sahaf, M. Zhang, Cross-domain reuse of extracted knowledge in genetic programming for image classification. IEEE Trans. Evolut. Comput. 21(4), 569–587 (2017)
L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal. Mach. Intell. 20(11), 1254–1259 (1998)
B. Jiang, L. Zhang, H. Lu, C. Yang, M.H. Yang, Saliency detection via absorbing markov chain, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2013), pp. 1665–1672
H. Jiang, J. Wang, Z. Yuan, T. Liu, N. Zheng, S. Li, Automatic salient object segmentation based on context and shape prior, in Proceedings of the British Machine Vision Conference, vol. 6 (2011), p. 9
H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, S. Li, Salient object detection: a discriminative regional feature integration approach, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 2083–2090
Z. Jiang, L.S. Davis, Submodular salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 2043–2050
T. Judd, K. Ehinger, F. Durand, A. Torralba, Learning to predict where humans look, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2009), pp. 2106–2113
J. Kim, D. Han, Y.W. Tai, J. Kim, Salient region detection via high-dimensional color transform, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014), pp. 883–890
J. Kim, V. Pavlovic, A shape preserving approach for salient object detection using convolutional neural networks, in Proceedings of the IEEE International Conference on Pattern Recognition (IEEE, 2016), pp. 609–614
C. Koch, S. Ullman, Shifts in selective visual attention: towards the underlying neural circuitry, in Matters of Intelligence, ed. by L.M. Vaina (Springer, 1987), pp. 115–141
J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, vol. 1 (MIT press, Cambridge, 1992)
J.R. Koza, Human-competitive results produced by genetic programming. Genet. Program. Evol. Mach. 11(3–4), 251–284 (2010)
K. Krawiec, B. Bhanu, Visual learning by evolutionary and coevolutionary feature synthesis. IEEE Trans. Evolut. Comput. 11(5), 635–650 (2007)
O. Le Meur, Z. Liu, Saliency aggregation: Does unity make strength? in Asian Conference on Computer Vision (Springer, 2014), pp. 18–32
G. Li, Y. Yu, Visual saliency detection based on multiscale deep cnn features. IEEE Trans. Image Process. 25(11), 5012–5024 (2016)
H. Li, J. Chen, H. Lu, Z. Chi, Cnn for saliency detection with low-level feature integration. Neurocomputing 226, 212–220 (2017)
J. Li, Y. Tian, L. Duan, T. Huang, Estimating visual saliency through single image optimization. IEEE Signal Process. Lett. 20(9), 845–848 (2013)
X. Li, H. Lu, L. Zhang, X. Ruan, M.H. Yang, Saliency detection via dense and sparse reconstruction, in Proceedings of the IEEE International Conference on Computer Vision (IEEE, 2013), pp. 2976–2983
Y. Liang, M. Zhang, W.N. Browne, Genetic programming for evolving figure-ground segmentors from multiple features. Appl. Soft Comput. 51, 83–95 (2017)
Y. Liang, M. Zhang, W.N. Browne, Image feature selection using genetic programming for figure-ground segmentation. Eng. Appl. Artif. Intell. 62, 96–108 (2017)
N. Liu, J. Han, DHSNet: Deep hierarchical saliency network for salient object detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 678–686
T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, H.Y. Shum, Learning to detect a salient object. IEEE Trans. Pattern Anal. Mach. Intell. 33(2), 353–367 (2011)
Z. Liu, W. Zou, O. Le Meur, Saliency tree: a novel saliency detection framework. IEEE Trans. Image Process. 23(5), 1937–1952 (2014)
J. Long, E. Shelhamer, T. Darrell, Fully convolutional networks for semantic segmentation, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 3431–3440
Y.F. Ma, H.J. Zhang, Contrast-based image attention analysis by using fuzzy growing, in Proceedings of the ACM International Conference on Multimedia (ACM, 2003), pp. 374–381
L. Mai, Y. Niu, F. Liu, Saliency aggregation: a data-driven approach, in Proceedings of the Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 1131–1138
N. Murray, M. Vanrell, X. Otazu, C.A. Parraga, Saliency estimation using a non-parametric low-level vision model, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2011), pp. 433–440
S.S. Naqvi, W.N. Browne, C. Hollitt, Combining object-based local and global feature statistics for salient object search, in Proceedings of the IEEE International Conference on Image and Vision Computing New Zealand (IEEE, 2013), pp. 394–399
S.S. Naqvi, W.N. Browne, C. Hollitt, Genetic algorithms based feature combination for salient object detection, for autonomously identified image domain types, in Proceedings of the IEEE Congress on Evolutionary Computation (IEEE, 2014), pp. 109–116
S.S. Naqvi, W.N. Browne, C. Hollitt, Feature quality-based dynamic feature selection for improving salient object detection. IEEE Trans. Image Process. 25(9), 4298–4313 (2016)
T.V. Nguyen, M. Kankanhalli, As-similar-as-possible saliency fusion. Multimed. Tools Appl. 76(8), 10501–10519 (2017)
Y. Niu, L. Lin, Y. Chen, L. Ke, Machine learning-based framework for saliency detection in distorted images. Multimed. Tools Appl. 76(24), 26329–26353 (2017)
G. Olague, L. Trujillo, Evolutionary-computer-assisted design of image operators that detect interest points using genetic programming. Image Vis. Comput. 29(7), 484–498 (2011)
F. Perazzi, P. Krähenbühl, Y. Pritch, A. Hornung, Saliency filters: contrast based filtering for salient region detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (IEEE, 2012), pp. 733–740
R. Poli, W.B. Langdon, N.F. McPhee, Field Guide to Genetic Programming. Online website (2008)
J. Shi, Q. Yan, L. Xu, J. Jia, Hierarchical image saliency detection on extended cssd. IEEE Trans. Pattern Anal. Mach. Intell. 38(4), 717–729 (2016)
N. Singh, R. Arya, R. Agrawal, A novel approach to combine features for salient object detection using constrained particle swarm optimization. Pattern Recognit. 47(4), 1731–1739 (2014)
W. Tan, B. Yan, Salient object detection via multiple saliency weights. Multimed. Tools Appl. 76(23), 25091–25107 (2017)
N. Tong, H. Lu, X. Ruan, M.H. Yang, Salient object detection via bootstrap learning, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 1884–1892
A.M. Treisman, G. Gelade, A feature-integration theory of attention. Cognit. Psychol. 12(1), 97–136 (1980)
L. Vanneschi, S. Silva, M. Castelli, L. Manzoni, Geometric semantic genetic programming for real life applications, in Genetic Programming Theory and Practice XI, ed. by R. Riolo, J. Moore, M. Kotanchek (Springer, 2014), pp. 191–209
H. Vojodi, A. Fakhari, A.M.E. Moghadam, A new evaluation measure for color image segmentation based on genetic programming approach. Image Vis. Comput. 31(11), 877–886 (2013)
D. Walther, C. Koch, Modeling attention to salient proto-objects. Neural Netw. 19(9), 1395–1407 (2006)
J. Wang, A. Borji, C.C.J. Kuo, L. Itti, Learning a combined model of visual saliency for fixation prediction. IEEE Trans. Image Process. 25(4), 1566–1579 (2016)
L. Wang, H. Lu, X. Ruan, M.H. Yang, Deep networks for saliency detection via local estimation and global search, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 3183–3192
X. Wei, Z. Tao, C. Zhang, X. Cao, Structured saliency fusion based on dempster-shafer theory. IEEE Signal Process. Lett. 22(9), 1345–1349 (2015)
C. Xie, L. Shang, Anomaly detection in crowded scenes using genetic programming, in Proceedings of the IEEE Congress on Evolutionary Computation (IEEE, 2014), pp. 1832–1839
Q. Yan, L. Xu, J. Shi, J. Jia, Hierarchical saliency detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2013), pp. 1155–1162
C. Yang, L. Zhang, H. Lu, X. Ruan, M.H. Yang, Saliency detection via graph-based manifold ranking, in Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition (IEEE, 2013), pp. 3166–3173
Y. Yuan, C. Li, J. Kim, W. Cai, D.D. Feng, Dense and sparse labeling with multi-dimensional features for saliency detection. IEEE Trans. Circuits Syst. Video Technol. 28, 1130–1143 (2018)
Y. Zhai, M. Shah, Visual attention detection in video sequences using spatiotemporal cues, in Proceedings of the ACM International Conference on Multimedia (ACM, 2006), pp. 815–824
J. Zhang, S. Sclaroff, Z. Lin, X. Shen, B. Price, R. Mech, Minimum barrier salient object detection at 80 fps, in Proceedings of the IEEE International Conference on Computer Vision (2015), pp. 1404–1412
L. Zhang, J. Li, H. Lu, Saliency detection via extreme learning machine. Neurocomputing 218, 103–112 (2016)
L. Zhang, M.H. Tong, T.K. Marks, H. Shan, G.W. Cottrell, Sun: A bayesian framework for saliency using natural statistics. J. Vis. 8(7), 32–32 (2008)
W. Zhu, S. Liang, Y. Wei, J. Sun, Saliency optimization from robust background detection, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2014), pp. 2814–2821
Acknowledgements
Marco A. Contreras-Cruz and Diana E. Martinez-Rodriguez thank to National Council of Science and Technology (CONACYT) for the financial support through the scholarships with Identification Numbers 568675/302121 and 736576/291047, respectively.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Contreras-Cruz, M.A., Martinez-Rodriguez, D.E., Hernandez-Belmonte, U.H. et al. A genetic programming framework in the automatic design of combination models for salient object detection. Genet Program Evolvable Mach 20, 285–325 (2019). https://doi.org/10.1007/s10710-019-09345-5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10710-019-09345-5