PAC-Bayes Bounds on Variational Tempered Posteriors for Markov Models

Datasets displaying temporal dependencies abound in science and engineering applications, with Markov models representing a simplified and popular view of the temporal dependence structure. In this paper, we consider Bayesian settings that place prior distributions over the parameters of the transition kernel of a Markov model, and seek to characterize the resulting, typically intractable, posterior distributions. We present a Probably Approximately Correct (PAC)-Bayesian analysis of variational Bayes (VB) approximations to tempered Bayesian posterior distributions, bounding the model risk of the VB approximations. Tempered posteriors are known to be robust to model misspecification, and their variational approximations do not suffer the usual problems of over confident approximations. Our results tie the risk bounds to the mixing and ergodic properties of the Markov data generating model. We illustrate the PAC-Bayes bounds through a number of example Markov models, and also consider the situation where the Markov model is misspecified.

Objective Bayesian Inference in Probit Models with Intrinsic Priors Using Variational Approximations

There is not much literature on objective Bayesian analysis for binary classification problems, especially for intrinsic prior related methods. On the other hand, variational inference methods have been employed to solve classification problems using probit regression and logistic regression with normal priors. In this article, we propose to apply the variational approximation on probit regression models with intrinsic prior. We review the mean-field variational method and the procedure of developing intrinsic prior for the probit regression model. We then present our work on implementing the variational Bayesian probit regression model using intrinsic prior. Publicly available data from the world’s largest peer-to-peer lending platform, LendingClub, will be used to illustrate how model output uncertainties are addressed through the framework we proposed. With LendingClub data, the target variable is the final status of a loan, either charged-off or fully paid. Investors may very well be interested in how predictive features like FICO, amount financed, income, etc. may affect the final loan status.

Variational Approximations for Steady Unidirectional Slip Flows in Microchannels

Consider steady flows in a channel, cross section Ω, with the Navier slip boundary condition, and let the volume flowrate be denoted by Q. We present a new simple approximation, a rigorous lower bound on Q, requiring, along with the channel's area and perimeter, the calculation of just the torsional rigidity and two other domain functionals. This avoids the need for solving the partial differential equation repeatedly for differing values of the slip parameter. It also provides the opportunity to give tables for different shapes, requiring, for each shape, just its area and perimeter and the three domain functionals previously mentioned. We expect that for shapes used in practice, the approximation will be good for the entire range of slip parameter. This is illustrated with the case of Ω being rectangular.