Towards Effective Anomaly Detection in Complex Dynamic Systems

Anomaly detection can be used to infer the presence of errors or intrusions without observing the target service or application, but detecting variations in the observable parts of the system on which the service or the application resides. This is a promising technique in complex software-intensive systems, where either instrumenting the services’ internals is exceedingly time-consuming, or encapsulation makes them not accessible. Unfortunately, in such systems anomaly detection is often made ineffective due to their dynamicity, which implies changes in the services or their expected workload. The main target of this Thesis is to present our approach to enhance the efficacy of anomaly detection in complex dynamic systems. Evolving and Dynamic systems may often change their behavior, adapting it to the current context, making the characterization of the expected behavior, and consequently the identification of anomalies, a hard challenge. As a result, there are no clear stateof-the-art answers on applying error or anomaly detection in highly dynamic and complex systems, while some frameworks for performing anomaly detection in complex - not highly dynamic - systems have been described in the literature. To contribute filling this gap, we put a promising state-of-the-art solution to work on data flows related to the Secure! system, a Crisis Management System which is structured as a Service Oriented Architecture (SOA). At first, we observed that applying such strategy as it was described for non-dynamic systems does not provide comparable detection scores, therefore we tried to adapt it by i) expanding the data collecting strategy, ii) considering additional information on the system, and iii) performing dedicated tuning of parameters of such strategy. This process led us to a customized version of the basic solution which has comparable scores with respect to other works targeting non-dynamic complex systems. At this point, we conducted an extensive experimental campaign targeting both the Secure! and the jSeduite SOAs based on the injection of specific types of anomalies to substantiate and confirm the progresses we obtained during our process. However, the main result we obtained through these experiments was a precise definition of design guidelines that are mainly related to the necessity of frequently reconfiguring both the monitoring strategy and the detection algorithms to suit an adaptive notion of expected and anomalous behavior, avoiding interferences and minimizing detection overheads. After reporting and presenting these guidelines according to specific viewpoints, we present MADneSs, a framework which implements our approach to anomaly detection that is tailored for such systems. The framework includes an adaptive multi-layer monitoring module. Monitored data is then processed by the anomaly detector, which adapts its parameters depending on the current behavior of the system, providing an anomaly alert. Lastly, we explore possible future implications explicitly targeting Systems-ofSystems, an architectural paradigm which in the recent years has started being adopted when building dynamic complex systems.