Show simple item record

dc.contributor.advisorStorkey, Amos
dc.contributor.advisorAgakov, Felix
dc.contributor.authorOrchard, Peter Raymond
dc.date.accessioned2015-02-23T14:47:30Z
dc.date.available2015-02-23T14:47:30Z
dc.date.issued2014-11-27
dc.identifier.urihttp://hdl.handle.net/1842/9955
dc.description.abstractOne of the fundamental tasks in science is to find explainable relationships between observed phenomena. Recent work has addressed this problem by attempting to learn the structure of graphical models - especially Gaussian models - by the imposition of sparsity constraints. The graphical lasso is a popular method for learning the structure of a Gaussian model. It uses regularisation to impose sparsity. In real-world problems, there may be latent variables that confound the relationships between the observed variables. Ignoring these latents, and imposing sparsity in the space of the visibles, may lead to the pruning of important structural relationships. We address this problem by introducing an expectation maximisation (EM) method for learning a Gaussian model that is sparse in the joint space of visible and latent variables. By extending this to a conditional mixture, we introduce multiple structures, and allow side information to be used to predict which structure is most appropriate for each data point. Finally, we handle non-Gaussian data by extending each sparse latent Gaussian to a Gaussian copula. We train these models on a financial data set; we find the structures to be interpretable, and the new models to perform better than their existing competitors. A potential problem with the mixture model is that it does not require the structure to persist in time, whereas this may be expected in practice. So we construct an input-output HMM with sparse Gaussian emissions. But the main result is that, provided the side information is rich enough, the temporal component of the model provides little benefit, and reduces efficiency considerably. The GWishart distribution may be used as the basis for a Bayesian approach to learning a sparse Gaussian. However, sampling from this distribution often limits the efficiency of inference in these models. We make a small change to the state-of-the-art block Gibbs sampler to improve its efficiency. We then introduce a Hamiltonian Monte Carlo sampler that is much more efficient than block Gibbs, especially in high dimensions. We use these samplers to compare a Bayesian approach to learning a sparse Gaussian with the (non-Bayesian) graphical lasso. We find that, even when limited to the same time budget, the Bayesian method can perform better. In summary, this thesis introduces practically useful advances in structure learning for Gaussian graphical models and their extensions. The contributions include the addition of latent variables, a non-Gaussian extension, (temporal) conditional mixtures, and methods for efficient inference in a Bayesian formulation.en
dc.contributor.sponsorEngineering and Physical Sciences Research Council (EPSRC)en
dc.language.isoenen
dc.publisherThe University of Edinburghen
dc.relation.hasversionAgakov, F. V., Orchard, P. R., and Storkey, A. (2012). Discriminative Mixtures of Sparse Latent Fields for Risk Management. In AISTATS.en
dc.relation.hasversionOrchard, P. R., Agakov, F. V., and Storkey, A. (2013). Bayesian Inference in Sparse Gaussian Graphical Models. ArXiv e-prints, 1309.7311.en
dc.subjectsparsityen
dc.subjectGaussianen
dc.subjectlatent variablesen
dc.subjectcopulaen
dc.subjectGWisharten
dc.subjectHMCen
dc.titleSparse inverse covariance estimation in Gaussian graphical modelsen
dc.typeThesis or Dissertationen
dc.type.qualificationlevelDoctoralen
dc.type.qualificationnamePhD Doctor of Philosophyen


Files in this item

This item appears in the following Collection(s)

Show simple item record