Multi-sensor Model-based Data Fusion for Remote Sensing Applications

The thesis addresses a widespread topic of remote sensing, namely pansharpening, representing a specific instance of image fusion, where a panchromatic image, characterized by high spatial resolution and no spectral information, is pixelwise merged with a set of multispectral images, featuring complementary characteristics, i.e., lower spatial resolution and spectral diversity. Thus, the aim of pansharpening is to generate a final image product featuring the spatial information of the panchromatic and the spectral content of the multispectral data. The first contribution of the thesis is to provide a twofold representation of the state of the art of pansharpening: one from a fusion methodology perspective and one from a quality assessment standpoint. Initially, we present a review of the most widespread fusion techniques and algorithms, with particular attention to the following major categories: Component Substitution, Multi-Resolution Analysis, Variational Optimization, and Machine Learning. Furthermore, several state-of-the-art hybrid approaches, involving any combinations of the former categories, are also described. Thereafter, we introduce a second review of the most popular quality evaluation protocols, both at full and reduced resolutions, proposed over the years in the corresponding literature. The second contribution of the thesis is to present an investigation on the data-format reproducibility of pansharpening, both in terms of fusion and quality assessment. The aim of this study is to demonstrate whether the pansharpening process is influenced by the particular data-format over which the input imagery is represented, such as digital number, spectral radiance and spectral reflectance. It will be theoretically proven and experimentally demonstrated that Multi-Resolution Analysis methods are unaffected by the format of the data, which is not always true for Component Substitution methods; for the latter, only the employment of regression-based solutions allows to reach data-format reproducibility. On the quality assessment, it will be demonstrated that purely spectral indexes, such as the Spectral Angle Mapper, feature a significant data-format dependence, whereas for indexes balancing the spectral and radiometric similarity, like those based on hypercomplex numbers, i.e., Q2n, such a dependence weakens and completely vanishes for purely radiometric indexes, such as those based on error summation, e.g., Relative Dimensionless Global Error in Synthesis. The third and final contribution of the thesis is to provide a critical comparison of the most widespread full-resolution quality assessment protocols, such as the quality-with-no-reference, QNR, and its more recent variations, a.k.a QNR-like. Specifically, we present a thorough discussion of the pros and cons of each protocol, aimed at identifying strengths and weaknesses in order to support future research developments. In addition, the problem of the combination of the two spatio-spectral distortion indexes forming the general QNR-like index, is also addressed, by studying and testing solutions based on coefficient estimation instead of exploiting coefficients that are fixed to a constant value. Experiments both at reduced and full resolutions, comprising a wide qualitative analysis, are considered to support the statements on the QNR-like protocols. The study highlights the interesting features of the Filter-based QNR protocol and the spatial distortion index of the Regression-based QNR, thus suggesting the use of these complementary quality assessment measures to provide a comprehensive and consistent assessment at full resolution.