Adaptive Gradient Langevin Algorithms for Stochastic Optimization and Bayesian Inference

Abstract

This thesis focuses on adaptive Stochastic Gradient Langevin Dynamics (SGLD) algorithms to solve optimization and Bayesian inference problems. SGLD algorithms consist in a stochastic gradient descent with exogenous noise added in order to escape local minima and saddle points. Contrary to the classic Langevin Stochastic Differential Equation, we study the case where the exogenous noise is adaptive i.e. not constant but depends on the position of the procedure. In a first part we prove the convergence of SGLD algorithms for the L1-Wasserstein distance and for the Total Variation distance. In a second part we apply SGLD algorithms to optimization and inference problems arising in Machine Learning and in Numerical Probability and we introduce the Layer Langevin algorithm. A last part is devoted to the numerical simulation of stochastic processes.

Pierre Bras

PhD Student in Applied Mathematics

I am a PhD student under the direction of Gilles Pagès, interested in Machine Learning, Stochastic Optimization and Numerical Probability.