Approximate Bayesian inference through the lens of large deviations

Abstract:

I will discuss an approach to approximate Bayesian inference rooted in stochastic analysis, specifically, using large deviations theory. I will first summarise what a large deviations principle is and how they appear to be fundamental in probability theory and statistical physics. I will then use path-wise large deviations to rigorously generalise the main construction in stochastic thermodynamics, the effective action, to two interesting cases: SDEs with state-dependent diffusion coefficients, and SDEs driven by more general processes than Wiener noise. Lastly I will examine things in the case of coupled stochastic processes. In particular, the following will be proven, making use of some technical innovations to existing large deviations principles: when two generic processes are synchronised in a certain way, the minimisation of large deviations in their trajectories implies an approximate Bayesian inference rule is satisfied by the processes in the joint path space. This theorem is the main result of the talk. Along the way we will make some contact with related principles in non-equilibrium statistical mechanics like Jaynes’ maximum calibre principle and Seifert’s stochastic thermodynamics, and celebrate an interesting observation on the mathematical structure of the free energy principle—namely, that it appears to intertwine two different large deviations principles in an essential way.

This talk will be based on ongoing work related to arXiv:2206.12996 and arXiv:2210.12761. It will be primarily a technical account of that work. I will review large deviations theory and some key equations of Bayesian mechanics, but I will assume some background in statistical physics and stochastic processes.

Slides: TNB June 2023.pdf