NoisyArt: Exploiting the NoisyWeb for Zero-shot Classification and Artwork Instance Recognition

NoisyArt is a dataset design to support research on webly-supervised recognition of artworks, consisting of more than 90,000 images and in more than 3000 webly-supervised classes, with a subset of 200 verified test images. Candidate artworks are identified using publicly available metadata repositories, and images are automatically acquired using search engines. Textual description and other information are provided for each artwork and artist, enabling experimentation on multi-modal techniques like zero-shot classification of artworks and captioning. Several techniques were used to mitigate label noise and domain shift in webly-supervised training settings, with the end of improve instance recognition performance. Further zero-shot learning experiments demonstrate the benefits and limitations of this kind of approaches in the challenging setting of data scarcity and noisy labels for the set of seen classes. This chapter combines and extends our ongoing work on NoisyArt dataset.