On trustworthy measurements when testing dependable systems: a discussion and experiences

The scientific literature and the industrial practice agree since many years on the fundamental role of experimental evaluation (testing) of critical systems for the assessment of the dependability attributes, and consequently on the relevance of achieving trustworthy measurements. This paper discusses and motivates with the support of three case studies, the possible role of the body of knowledge offered by measurement theory (metrology) to quantitatively assess the quality of measuring instruments, i.e., the tools, and the results collected. The paper first introduces notions of metrology and contextualises them for the dependability evaluation of systems. Successively it presents three case studies developed by the authors where attention to principles from measurement theory and the metrological assessment of tools and results are carried out. The paper ultimately reviews the main guidelines identified discussing their application in the case studies.