Webly-supervised Zero-shot Learning for Artwork Instance Recognition

This paper describes experiments on supervised approaches to webly-labeled artwork instance recog-nition and zero-shot learning for unseen artwork instance recognition. We build on our earlier workon webly-supervised learning using theNoisyArtdataset. The dataset consists of more than 90,000images and in more than 3,000 webly-supervised classes, and a subset of 200 classes with verifiedtest images. Document embeddings are provided for short descriptions of all artworks.NoisyArtisdesigned to support research on webly-supervised artwork instance recognition, zero-shot learning,and other approaches to visual recognition of cultural heritage objects. We report results of experi-ments on artwork instance recognition using theNoisyArtdataset of webly-labeled images as well ason the CMU-Oxford Sculptures dataset. In addition, we perform extensive experiments on zero-shotlearning using webly-labeled training images for unseen artwork recognition. Our results demonstratethe benefits and limitations of zero-shot learning for instance recognition over webly-supervised data