Towards a Crosslinguistic Identification of Action Concepts. Automatic Clustering of Video Scenes Based on the IMAGACT Multilingual Ontology

This work focuses on the automatic identification of action concepts, performed through machine learning algorithms and applied to a linguistic dataset derived from the IMAGACT ontology of actions. IMAGACT contains a set of 1,010 actions, represented by video scenes and enriched with linguistic data in several languages. In particular, each scene is linked to the complete set of verbs that can refer to the depicted action in every considered language. Starting from these data, automatic clustering of scenes has been performed using the linked lexical items as a feature set, following the idea that a similar group of verbs can refer to similar actions. Hierarchical agglomerative clustering has been performed to set up an evaluation campaign and create a gold standard of validated clusters. Then, a semi-supervised method based on Affinity Propagation was trained on these data. An evaluation of cluster coherence has been performed, reporting promising results. An interactive web version of the action map has also been created, to allow users to browse the clusters of videos.