Extending Markov Modulated Poisson processes for learning and prediction

The focus of this thesis is on extensions of Markov Modulated Poisson processes (MMPPs) with the aim of showing the benefits of continuous time models in approximating event-driven processes over standard techniques like machine learning. Generative models like MMPPs enable methods of quantitative evaluation supporting diagnostic and predictive analytics; their ability of generating new synthetic sequences of events permits to calculate properties both through analytic formulas and Monte Carlo simulation of sequences. The first chapter of the thesis expands on the goal of the thesis and presents preliminaries on markovian processes, respectively discrete and continuous time hidden Markov models (HMMs and CT-HMMs), Markov Arrival processes (MAPs) and MMPPs. Finally, the chapter introduces the contributions offered in this thesis. The subsequent part of the thesis details several extensions of MMPPs directed to addressing different types of events produced in real world systems. In the second chapter is described the first extension to MMPP for the efficient learning of processes that produce events with a discrete type: Marked Markov Modulated Poisson Processes are MMPPs associated with a mark at each observation; while this mark can be any type of information, focus is given to simplifications for specifically discrete types of events. In the third chapter is described the second extension, Marked Markov Modulated Compound Process (M3CPP), optimized for processes where events are marked by a combination of any number of discrete dimensions and possibly a single continuous one. An open problem of MMPPs is the restriction to exponential distributed times in the sojourn in states: to lower the impact of this restriction an extension is presented where normal states are grouped into collections with specific inter-collection transitions, effectively creating macrostates with acyclic phase-type distributed sojourn times. For all extensions the likelihood is calculated and the algorithm for parameter estimation is presented via Expectation Maximization (EM). The extensions are studied on both simulated and real world datasets. For M3CPP the comparison is made to the machine learning approach of Long Short-term Memory (LSTM) showing that the proposed extension is well performing with respect to current techniques both on simulated and real world datasets. The fourth chapter presents the conclusions and future work.