Automatic image annotation is among the fundamental problems in computer vision and pattern recognition, and it is becoming increasingly important in order to develop algorithms that are able to search and browse large-scale image collections. In this paper, we propose a label propagation framework based on Kernel Canonical Correlation Analysis (KCCA), which builds a latent semantic space where correlation of visual and textual features are well preserved into a semantic embedding. The proposed approach is robust and can work either when the training set is well annotated by experts, as well as when it is noisy such as in the case of user-generated tags in social media. We report extensive results on four popular datasets. Our results show that our KCCA-based framework can be applied to several state-of-the-art label transfer methods to obtain significant improvements. Our approach works even with the noisy tags of social users, provided that appropriate denoising is performed. Experiments on a large scale setting show that our method can provide some benefits even when the semantic space is estimated on a subset of training images.

Automatic image annotation via label transfer in the semantic space / Uricchio, Tiberio; Ballan, Lamberto; Seidenari, Lorenzo; Del Bimbo, Alberto. - In: PATTERN RECOGNITION. - ISSN 0031-3203. - ELETTRONICO. - 71:(2017), pp. 144-157. [10.1016/j.patcog.2017.05.019]

Automatic image annotation via label transfer in the semantic space

Uricchio, Tiberio;Ballan, Lamberto;Seidenari, Lorenzo;Del Bimbo, Alberto
2017

Abstract

Automatic image annotation is among the fundamental problems in computer vision and pattern recognition, and it is becoming increasingly important in order to develop algorithms that are able to search and browse large-scale image collections. In this paper, we propose a label propagation framework based on Kernel Canonical Correlation Analysis (KCCA), which builds a latent semantic space where correlation of visual and textual features are well preserved into a semantic embedding. The proposed approach is robust and can work either when the training set is well annotated by experts, as well as when it is noisy such as in the case of user-generated tags in social media. We report extensive results on four popular datasets. Our results show that our KCCA-based framework can be applied to several state-of-the-art label transfer methods to obtain significant improvements. Our approach works even with the noisy tags of social users, provided that appropriate denoising is performed. Experiments on a large scale setting show that our method can provide some benefits even when the semantic space is estimated on a subset of training images.
2017
71
144
157
Uricchio, Tiberio; Ballan, Lamberto; Seidenari, Lorenzo; Del Bimbo, Alberto
File in questo prodotto:
File Dimensione Formato  
1-s2.0-S0031320317302066-main.pdf

Accesso chiuso

Tipologia: Pdf editoriale (Version of record)
Licenza: Tutti i diritti riservati
Dimensione 2.24 MB
Formato Adobe PDF
2.24 MB Adobe PDF   Richiedi una copia
1605.04770.pdf

accesso aperto

Tipologia: Preprint (Submitted version)
Licenza: Open Access
Dimensione 4.6 MB
Formato Adobe PDF
4.6 MB Adobe PDF

I documenti in FLORE sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificatore per citare o creare un link a questa risorsa: https://hdl.handle.net/2158/1089323
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 74
  • ???jsp.display-item.citation.isi??? 62
social impact