In this paper we propose a novel data augmentation approach for visual content domains that have scarce training datasets, compositing synthetic 3D objects within real scenes. We show the performance of the proposed system in the context of object detection in thermal videos, a domain where i) training datasets are very limited compared to visible spectrum datasets and ii) creating full realistic synthetic scenes is extremely cumbersome and expensive due to the difficulty in modeling the thermal properties of the materials of the scene. We compare different augmentation strategies, including state of the art approaches obtained through RL techniques, the injection of simulated data and the employment of a generative model, and study how to best combine our proposed augmentation with these other techniques. Experimental results demonstrate the effectiveness of our approach, and our single-modality detector achieves state-of-the-art results on the FLIR ADAS dataset.

Partially Fake it Till you Make It: Mixing Real and Fake Thermal Images for Improved Object Detection / Bongini F.; Berlincioni L.; Bertini M.; Del Bimbo A.. - ELETTRONICO. - (2021), pp. 5482-5490. ((Intervento presentato al convegno ACM International Conference on Multimedia , ACMMM 21 tenutosi a Virtual Event China nel October 20 - 24, 2021 [10.1145/3474085.3475679].

Partially Fake it Till you Make It: Mixing Real and Fake Thermal Images for Improved Object Detection

Bongini F.;Berlincioni L.;Bertini M.;Del Bimbo A.
2021

Abstract

In this paper we propose a novel data augmentation approach for visual content domains that have scarce training datasets, compositing synthetic 3D objects within real scenes. We show the performance of the proposed system in the context of object detection in thermal videos, a domain where i) training datasets are very limited compared to visible spectrum datasets and ii) creating full realistic synthetic scenes is extremely cumbersome and expensive due to the difficulty in modeling the thermal properties of the materials of the scene. We compare different augmentation strategies, including state of the art approaches obtained through RL techniques, the injection of simulated data and the employment of a generative model, and study how to best combine our proposed augmentation with these other techniques. Experimental results demonstrate the effectiveness of our approach, and our single-modality detector achieves state-of-the-art results on the FLIR ADAS dataset.
MM '21: Proceedings of the 29th ACM International Conference on Multimedia
ACM International Conference on Multimedia , ACMMM 21
Virtual Event China
October 20 - 24, 2021
Bongini F.; Berlincioni L.; Bertini M.; Del Bimbo A.
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in FLORE sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/2158/1283044
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact