Discriminant Analysis Methods for Large Scale and Complex Datasets

Abstract

Discriminant analysis (DA) methods are effective and intuitive classifiers for correlated, Gaussian data. However, they cannot be used in high dimensional problems without modification (that is, when the number of features outnumbers that of the observations) and/or non-Gaussian data, due to the strict parametric assumptions underlying such classifiers. In this thesis we aim to extend DA to modern settings. First, we introduce a new class of priors for Bayesian hypothesis testing, which we name ``cake priors". These priors allow the use of diffuse priors while achieving theoretically sound inferences. In this thesis we develop a foundation for hypothesis testing as a means for feature selection for DA classifiers, with the resultant Bayesian test statistic taking the form of a penalised likelihood ratio test statistic, allowing for natural comparison between nested models. Further, we show that these resultant tests using cake priors are Chernoff consistent. We next introduce a new method of performing high dimensional discriminant analysis, named multiDA. We achieve this by constructing a hybrid model which integrates both a diagonal discriminant analysis model and feature selection components based on likelihood ratio statistics, and provide heuristic arguments suggesting sound asymptotic performance for feature selection. Finally, we develop a method for generalised DA, named genDA. This method extends DA beyond the usual Gaussian response, and utilises generalised linear latent variable models in the place of Gaussian distributions, allowing for flexible modelling of multi-distributional response data, whilst capturing and exploiting between feature correlation structure. R packages implementing the classification methodology using efficient computational routines are available as multiDA and genDA and have been released on GitHub. Using these packages, we demonstrate competitive performance on simulated and benchmark datasets.