Advancements in Variational Bayesian Computation: Theory and Applications in Hybrid and Particle-based Methods

Abstract

This thesis advances Variational Bayesian inference methods to address challenges in statistical models with complex structures, intractable likelihoods, and high-dimensional settings. Traditional Variational Bayes (VB) techniques often fall short when dealing with models involving numerous nuisance parameters or requiring restrictive parametric assumptions.

To overcome these limitations, we introduce the Hybrid Variational Bayes (HVB) framework, which employs a hybrid variational structure between parameters of interest and nuisance parameters. HVB enhances inference accuracy in models like the Bayesian Lasso and state-space models by better capturing parameter relationships.

We also develop Particle Mean-Field Variational Bayes (PMFVB), a novel particle-based methodology that updates particle positions using Langevin diffusion processes. PMFVB converges toward the optimal variational density without relying on mutual independence among latent variables or restrictive parametric assumptions. This approach broadens the applicability of VB and enhances accuracy, validated through theoretical analysis and applications in Bayesian deep learning models.

Additionally, we explore Particle Flow Variational Inference (PFVI), inspired by the Wasserstein gradient flow. While traditional particle flow methods face practical issues due to intractable optimal maps in the Jordan–Kinderlehrer–Otto (JKO) scheme, we provide a state-of-the-art convergence analysis of PFVI algorithms under the Logarithmic Sobolev inequality. This work addresses existing challenges and offers practical solutions for Bayesian computation.

Overall, this thesis contributes to the field by developing advanced variational inference frameworks that enhance computational efficiency and accuracy in complex Bayesian models, expanding the tools available for statistical analysis in high-dimensional and intricate settings.