We have seen a rise in video based user communication in the last year, unfortunately fueled by the spread of COVID-19 disease. Efficient low-latency delay of transmission of video is a challenging problem which must also deal with the segmented nature of network infrastructure not always allowing a high throughput. Lossy video compression is a basic requirement to enable such technology widely. While this may compromise the quality of the streamed video there are recent deep learning based solutions to restore quality of a lossy compressed video. Considering the very nature of video conferencing, bitrate allocation in video streaming could be driven semantically, differentiating quality between the talking subjects and the background. Currently there have not been any work studying the restoration of semantically coded video using deep learning. In this work we show how such videos can be efficiently generated by shifting bitrate with masks derived via computer vision and how a deep generative adversarial network can be trained to restore video quality. Our study shows that the combination of semantic coding and learning based video restoration can provide superior results.

Increasing Video Perceptual Quality with GANs and Semantic Coding / Galteri L.; Bertini M.; Seidenari L.; Uricchio T.; Del Bimbo A.. - ELETTRONICO. - (2020), pp. 862-870. (Intervento presentato al convegno 28th ACM International Conference on Multimedia, MM 2020 tenutosi a usa nel 2020) [10.1145/3394171.3413508].

Increasing Video Perceptual Quality with GANs and Semantic Coding

Galteri L.;Bertini M.;Seidenari L.;Uricchio T.;Del Bimbo A.
2020

Abstract

We have seen a rise in video based user communication in the last year, unfortunately fueled by the spread of COVID-19 disease. Efficient low-latency delay of transmission of video is a challenging problem which must also deal with the segmented nature of network infrastructure not always allowing a high throughput. Lossy video compression is a basic requirement to enable such technology widely. While this may compromise the quality of the streamed video there are recent deep learning based solutions to restore quality of a lossy compressed video. Considering the very nature of video conferencing, bitrate allocation in video streaming could be driven semantically, differentiating quality between the talking subjects and the background. Currently there have not been any work studying the restoration of semantically coded video using deep learning. In this work we show how such videos can be efficiently generated by shifting bitrate with masks derived via computer vision and how a deep generative adversarial network can be trained to restore video quality. Our study shows that the combination of semantic coding and learning based video restoration can provide superior results.
2020
MM 2020 - Proceedings of the 28th ACM International Conference on Multimedia
28th ACM International Conference on Multimedia, MM 2020
usa
2020
Galteri L.; Bertini M.; Seidenari L.; Uricchio T.; Del Bimbo A.
File in questo prodotto:
File Dimensione Formato  
output.pdf

accesso aperto

Tipologia: Pdf editoriale (Version of record)
Licenza: Tutti i diritti riservati
Dimensione 5.26 MB
Formato Adobe PDF
5.26 MB Adobe PDF

I documenti in FLORE sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificatore per citare o creare un link a questa risorsa: https://hdl.handle.net/2158/1222582
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 7
  • ???jsp.display-item.citation.isi??? 2
social impact