Bayesian Aspects of Lasso Regression

It is well known that the Lasso estimate for linear regression corresponds to a Bayesian posterior mode under a particular prior distribution on the regression coefficients; however, the strength of this connection has yet to be greatly exploited. I describes a fully Bayesian approach to the Lasso regression problem that simultaneously addresses parameter estimation, prediction of future observations, and the question of variable selection/model uncertainty. The methodology introduced in this talk differs from the usual approach to Lasso regression and previous Bayesian treatments in several key respects. Emphasis is placed throughout on the use of the posterior mean rather than the usual Lasso approach of estimation via the posterior mode. Use of the posterior mean for estimation is shown to facilitate the prediction of future observations via the posterior predictive distribution, and discrepancies between Bayesian prediction and typical Lasso point prediction are highlighted. The questions of model uncertainty and variable selection are handled in a fully Bayesian manner by direct modeling of variable inclusion/exclusion with prior distributions. It is shown that the Lasso solution to variable selection can disagree with proper Bayesian model selection via Bayes factors. Three new Gibbs samplers for posterior inference are introduced. An R library with C++ source code is provided that implements all of the Markov chain Monte Carlo methodology discussed.