Fabio F. Faria
Eduardo Valle
ABSTRACT
In Content-based Image Retrieval (CBIR), accurately ranking the returned images is of paramount importance, since users consider mostly the topmost results. The typical ranking strategy used by many CBIR systems is to employ image content descriptors, so that returned images that are most similar to the query image are placed higher in the rank. While this strategy is well accepted and widely used, improved results may be obtained by combining multiple image descriptors. In this paper we explore this idea, and introduce algorithms that learn to combine information coming from different descriptors. The proposed learning to rank algorithms are based on three diverse learning techniques: Support Vector Machines (CBIR-SVM), Genetic Programming (CBIR-GP), and Association Rules (CBIR-AR). Eighteen image content descriptors(color, texture, and shape information) are used as input and provided as training to the learning algorithms. We performed a systematic evaluation involving two complex and heterogeneous image databases (Corel e Caltech) and two evaluation measures (Precision and MAP). The empirical results show that all learning algorithms provide significant gains when compared to the typical ranking strategy in which descriptors are used in isolation. We concluded that, in general, CBIR-AR and CBIR-GP outperforms CBIR-SVM. A fine-grained analysis revealed the lack of correlation between the results provided by CBIR-AR and the results provided by the other two algorithms, which indicates the opportunity of an advantageous hybrid approach.
- R. Agrawal, T. Imielinski, and A. Swami. Mining association rules between sets of items in large databases. In SIGMOD, pages 207--216, 1993. Google Scholar
Digital Library
- H. M. Almeida, M. Gonçalves, M. Cristo, and P. Calado. A combined component approach for finding collection-adapted ranking functions based on genetic programming. In SIGIR '07, pages 399--406, 2007. Google ScholarDigital Library
- B. Boser, I. Guyon, and V. Vapnik. A training algorithm for optimal margin classifiers. In COLT, pages 144--152. Springer, 1992. Google ScholarDigital Library
- Y. Cao, X. Jun, T. Liu, H. Li, Y. Huang, and H. Hon. Adapting ranking svm to document retrieval. In SIGIR '06, pages 186--193, 2006. Google ScholarDigital Library
- A. warkacioglu and F. Yarman-Vural. Sasi: a generic texture descriptor for image retrieval. Pattern Recognition, 36(11):2615--2633, 2003.Google ScholarCross Ref
- R. da S. Torres and A. X. Falcão. Content-based image retrieval: Theory and applications. Revista de Informática Teórica e Aplicada, 13(2):161--185, 2006.Google Scholar
- R. da S. Torres, A. X. Falcão, M. A. Goncalves, J. P. Papa, B. Zhang, W. Fan, and E. A. Fox. A genetic programming framework for content-based image retrieval. Pattern Recognition, 42(2):283--292, 2009. Google ScholarDigital Library
- W. Fan, M. D. Gordon, and P. Pathak. Genetic programming-based discovery of ranking functions for effective web search. J. Manage. Inf. Syst., 21(4):37--56, 2005. Google ScholarDigital Library
- U. Fayyad and K. Irani. Multi interval discretization of continuous-valued attributes for classification learning. In IJCAI., pages 1022--1027, 1993.Google Scholar
- C. D. Ferreira, R. da S. Torres, M. A. Goncalves, and W. Fan. Image Retrieval with Relevance Feedback based on Genetic Programming. In SBBD, pages 120--134, 2008. Google ScholarDigital Library
- A. Frome, Y. Singer, and J. Malik. Image retrieval and classification using local distance functions. In B. Schölkopf, J. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 417--424, Cambridge, MA, 2007. MIT Press.Google ScholarDigital Library
- R. Gonzalez and R. Woods. Digital Image Processing. Addison-Wesley, 1992. Google ScholarDigital Library
- P. Gosselin and M. Cord. Active learning methods for interactive image retrieval. IEEE Transactions on Image Processing, 17(7):1200--1211, 2008. Google ScholarDigital Library
- J. Han, S. J. McKenna, and R. Wang. Learning query-dependent distance metrics for interactive image retrieval. In M. Fritz, B. Schiele, and J. H. Piater, editors, ICVS, volume 5815 of Lecture Notes in Computer Science, pages 374--383. Springer, 2009. Google ScholarDigital Library
- R. Herbrich, T. Graepel, and K. Obermayer. Large margin rank boundaries for ordinal regression. In A. Smola, P. Bartlett, B. Schölkopf, and D. Schuurmans, editors, Advances in Large Margin Classifiers, pages 115--132, Cambridge, MA, 2000. MIT Press.Google Scholar
- P. Hong, Q. Tian, and T. S. Huang. Incorporate support vector machines to content-based image retrieval with relevant feedback. In In Proc. IEEE International Conference on Image Processing (ICIP), pages 750--753, 2000.Google Scholar
- Y. Hu, M. Li, and N. Yu. Multiple-instance ranking: Learning to rank images for image retrieval. Computer Vision and Pattern Recognition, IEEE Computer Society Conference on, pages 1--8, 2008.Google Scholar
- C. Huang and Q. Liu. An orientation independent texture descriptor for image retireval. In ICCCS, pages 772--776, 2007.Google Scholar
- J. Huang, R. Kumar, M. Mitra, W. Zhu, and R. Zabih. Image indexing using color correlograms. In CVPR, pages 762--768, 1997. Google ScholarDigital Library
- T. Joachims. Optimizing search engines using clickthrough data. In SIGKDD, pages 133--142, 2002. Google ScholarDigital Library
- T. Joachims. Training linear SVMs in linear time. In SIGKDD, pages 217--226, 2006. Google ScholarDigital Library
- V. Kovalev and S. Volmer. Color co-occurence descriptors for querying-by-example. In MMM, pages 32--38, 1998. Google ScholarDigital Library
- J. Koza. Genetic Programming: On the programming of computers by natural selection. MIT Press, 1992. Google ScholarDigital Library
- D. Lee and H. Kim. A fast content-based indexing and retrieval technique by the shape information in large image database. Journal of Systems and Software, 56(2):165--182, 2001. Google ScholarDigital Library
- E. Levina and P. Bickel. The earth movers distance is the mallows distance: Some insights from statistics. In Eighth IEEE International Conference on In Computer Vision, volume 2, pages 251--256, 2001.Google Scholar
- F. Li, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Image Understanding, 106(1):59--70, 2007. Google ScholarDigital Library
- Y. Liu, J. Xu, T. Qin, W. Xiong, and H. Li. LETOR: Benchmark dataset for research on learning to rank for information retrieval. In Learning to Rank Workshop in conjuntion with SIGIR, 2007.Google Scholar
- T. Lu and C. Chang. Color image retrieval technique based on color features and image bitmap. Inf. Processing and Management, 43(2):461--472, 2007. Google ScholarDigital Library
- S. D. MacArthur, C. E. Brodley, A. C. Kak, and L. S. Broderick. Interactive content-based image retrieval using relevance feedback. Comput. Vis. Image Underst., 88(2):55--75, 2002. Google ScholarDigital Library
- F. Mahmoudi, J. Shanbehzadeh, A. Eftekhari-Moghadam, and H. Soltanian-Zadeh. Image retrieval based on shape similarity by edge orientation autocorrelogram. Pattern Recognition, 36(8):1725--1736, 2003.Google ScholarCross Ref
- B. Manjunath, J. Ohm, V. Vasudevan, and A. Yamada. Color and texture descriptors. IEEE Trans. Circuits Syst. Video Techn., 11(6):703--715, 2001. Google ScholarDigital Library
- T. Ojala, M. Pietikäinen, and T. Mäenpää. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell., 24(7):971--987, 2002. Google ScholarDigital Library
- G. Pass, R. Zabih, and J. Miller. Comparing images using color coherence vectors. In ACM Multimedia, pages 65--73, 1996. Google ScholarDigital Library
- D. Ritendra, J. Dhiraj, L. Jia, and Z. W. James. Image retrieval: Ideas, influences, and trends of the new age. ACM Comput. Surv., 40(2):1--60, April 2008. Google ScholarDigital Library
- H. Shao, J. W. Zhang, W. C. Cui, and H. Zhao. Automatic Feature Weight Assignment based on Genetic Algorithm for Image Retrieval. In IEEE International Conference on Robotics, Intelligent Systems and Signal Processing, pages 731--735, 2003.Google Scholar
- R. Stehling, M. Nascimento, and A. Falcão. A compact and efficient image retrieval approach based on border/interior pixel classification. In CIKM, pages 102--109, 2002. Google ScholarDigital Library
- M. Stricker and M. Orengo. Similarity of color images. In Storage and Retrieval for Image and Video Databases (SPIE), pages 381--392, 1995.Google ScholarCross Ref
- M. Swain and D. Ballard. Color indexing. International Journal of Computer Vision, 7(1):11--32, 1991. Google ScholarDigital Library
- B. Tao and B. Dickinson. Texture recognition and image retrieval using gradient indexing. Journal of Visual Communication and Image Representation, 11(3):327--342, 2000.Google ScholarDigital Library
- M. Unser. Sum and difference histograms for texture classification. IEEE Trans. on Pattern Analysis and Machine Intelligence, 8(1):118--125, 1986. Google ScholarDigital Library
- A. Veloso, H. M. Almeida, M. Gon¸calves, and W. Meira. Learning to rank at query-time using association rules. In SIGIR, pages 267--274, 2008. Google ScholarDigital Library
- A. Veloso, W. M. Jr., and M. J. Zaki. Lazy associative classification. In ICDM, pages 645--654, 2006. Google ScholarDigital Library
- A. Williams and P. Yoon. Content-based image retrieval using joint correlograms. Multimedia Tools Appl., 34(2):239--248, 2007. Google ScholarDigital Library
- Y. Yue, T. Finley, F. Radlinski, and T. Joachims. A support vector method for optimizing average precision. In SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, pages 271--278, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- J. A. M. Zegarra, N. J. Leite, and R. da S. Torres. Wavelet-based Feature Extraction for Fingerprint Image Retrieval. Journal of Computational and Applied Mathematics, 227(2):294--307, May 2009. Google ScholarDigital Library
- L. Zhang, F. Lin, and B. Zhang. Support vector machine learning for image retrieval. In ICIP (2), pages 721--724, 2001.Google Scholar
- J. Zobel and A. Moffat. Exploring the similarity space. SIGIR Forum, 32(1):18--34, 1998. Google ScholarDigital Library
Index Terms
- Learning to rank for content-based image retrieval
Recommendations
A comprehensive study on learning to rank for content-based image retrieval
Recently, various learning to rank approaches have been proposed in the information retrieval realm, with their promising performance in general document and web page retrieval applications. Based on these achievements, in this paper, we investigate and ...
Multi-Scale Local Spatial Binary Patterns for Content-Based Image Retrieval
AMT 2013: Proceedings of the 9th International Conference on Active Media Technology - Volume 8210Content-based image retrieval (CBIR) has been widely studied in recent years. CBIR usually employs feature descriptors to describe the concerned characters of images, such as geometric descriptor and texture descriptor. Many texture descriptors in ...
Histogram refinement for texture descriptor based image retrieval
Texture descriptors such as local binary patterns (LBP) have been successfully employed for feature extraction in image retrieval algorithms because of their high discriminating ability and computational efficiency. In this paper, we propose histogram ...
Comments