Toward Open-World Multimedia Forensics Through Media Signature Encoding

Countering image and video manipulations is getting more and more relevant in several fields such as investigation, intelligence and forensics. Multimedia forensics researchers keep developing new tools and updating available detectors to discriminate the processing the media has been subjected to. While these tools can be utilized efficiently in controlled settings, they are generally unreliable in open-world scenarios where the investigated material may have been subjected to several unknown manipulations. In this paper, we present a novel framework to discriminate different toolchains of media manipulation and processing. We introduce the concept of media signature encoding to map image and video contents to latent spaces where media produced by similar processing toolchains cluster together. We demonstrate that this property still holds for toolchain that are not known when building the encoder, expanding the range of applications for our framework to open-world contexts where forensic analysts may face both familiar and unfamiliar manipulation techniques. A significant advantage of this approach lies in its ability to create, in principle, media signatures from any kind of forensic features. We evaluated the effectiveness of the proposed framework in two different experimental setups involving digital images and videos. Results show that encoded signatures are capable of determining whether: (i) a media under analysis belongs to a known life cycle or an entirely novel processing toolchain; (ii) a subset of media items share the same history. This framework can be considered a first step towards the use of forensic features to characterize media life cycles in open-world settings.