Application of Expectation Maximisation Algorithm on Mixed Distributions

Abstract

Mixed distributions are a statistical tool used for modelling a range of phenomena in fields as diverse as marketing, genetics, medicine, artificial intelligence, and finance. A mixture model is capable of describing a quite complex distribution of data, often in situations where a single parametric distribution is unable to provide a satisfactory result. The Expectation Maximisation (EM) algorithm is an iterative maximum likelihood method typically used to estimate parameters in incomplete data problems, such as mixtures. This thesis presents a thorough analysis of mixture modelling and estimation via the EM algorithm for normal, Weibull, exponential, gamma, loglogistic, and uniform component distributions. Full derivations of relevant EM equations are provided, including censored EM equations for exponential and Weibull component distributions. Goodness-of-fit tests assess how well an hypothesised statistical model fits a set of observations. This thesis considers two goodness-of-fit testing frameworks, the first being formal hypothesis based testing, the second being model selection via information criteria. It has been empirically justified that critical values for Kolmogorov-Smirnov, Kuiper, Cramér-von Mises, and Anderson-Darling goodness-of-fit tests don’t exhibit the same parameter independent properties as single distributions. Critical values are in fact parameter dependent, as well as being dependent on sample size, significance level, and truncation level. A comprehensive analysis is also provided of model selection via information criteria, for the Akaike information criterion, and Bayesian information criterion. Goodness-of-fit testing in this manner was found to be more appropriate for mixture modelling. The work culminates with the application of previously discussed statistical methodology to an analysis of limit-order inter-arrival times, and mid-price waiting times on the London Stock Exchange. It is reasoned that censored mixtures which include a Weibull component most appropriately model this data.