Algorithms for Gaussian Processes on Some Geometric Structures

1 Introduction 2
1.1 Gaussian Processes Regression 2
1.2 Stationary Kernels on Euclidean Spaces 3
1.2.1 Matern and Heat Kernels 4
1.3 Computational Algorithms 5
1.3.1 Sampling and Conditioning via Finite-dimensional Feature Maps . . 5
1.3.2 Getting Finite-dimensional Feature Maps 5
1.3.3 Efficient Conditioning via Variational Inference 6
1.4 Goals and Structure of the work 6
2 Symmetric Spaces 7
2.1 Definition and Classification 7
2.2 Lie Structure of Symmetric Spaces 9
2.3 Harmonic Analysis 10
3 Gaussian Processes on Symmetric Spaces 12
3.1 Stationary Gaussian Processes 12
3.2 Stationary Gaussian Processes on Symmetric Spaces 13
3.3 Computational Algorithms 13
3.3.1 Pointwise Kernel Evaluation 14
3.3.2 Finite-dimensional Feature Maps 14
3.3.3 Variational Inference 16
3.4 Matern and Heat Kernels 16
4 Application to the Space of Symmetric Positive Definite Matrices 18
4.1 Efficient Evaluation of Heat and Matern Kernels 19
Купить

Gaussian process regression based on stationary Gaussian processes provides a powerful frame­work for data efficient learning in a relatively low dimension. One of the key features of the framework is the ability to quantify uncertainty associated to the predictions. This is often used in applications involving automatic decision making, including optimization [24], reinforcement learning [8] and more [21].
In some applications, inputs of the unknown function lie in a non-Euclidean space like a manifold or a graph. Although one can often model a function like this by embedding these inputs into a Euclidean space, the inner structure of the input space, which is an important modeling assumption, is lost. In practice this hinders data efficiency and impairs overall modeling quality. It is thus important to study Gaussian process regression with inputs on such spaces directly.
Recent developments on that account include [4] where the general case of compact Rieman- nian manifolds is studied and [3] where the object of consideration are Gaussian processes on graph-structured finite sets.
There exists, however, a number of examples of noncompact manifolds of great significance for applications for which the theory and the corresponding computational techniques are yet to be developed. Arguably, the most important ones are the manifold of postive definite matrices and the hyperbolic space — examples of the class of noncompact symmetric spaces.
In this work we study stationary Gaussian processes (most notably, based on heat and Matern kernels) on such spaces and computational approaches for Gaussian process regression on them.
These techniques include: efficient (approximate) algorithms for point-wise kernel evaluation and differentiation with respect to parameters; efficient algorithms for sampling, conditioning and sampling from the conditional Gaussian process.
In the next three parts of the introduction we give an overview of Gaussian processes regres­sion on Euclidean spaces. In the forth and final part of the introduction we give a brief but more specific account of goals of the thesis and of the structure of the further text.
Notation We use lowercase bold to denote vectors (e.g. x) and uppercase upright bold to denote matrices (e.g. A).
For a function f(•) on X and x G Xn we put f(x) = [f(x1),...,f(xn)]. Similarly, for a function f (•, •) on X x Y and x G Xn, y G Ym we put f (x, y) = [f (xi,yj)]iКупить