Scaling priors for regression models

Introduction
Setup
Results

Introduction

Park and Casella 2008 propose the Bayesian lasso, and argue that a “scaled prior”

\begin{align} \vy \mid \mx, \vb, \sigma^2 &\sim \N(\mx \vb, \sigma^2 \mi)\\ \vb \mid \lambda, \sigma^2 &\sim \Lapl(0, \lambda / \sqrt{\sigma^2}) \end{align}

leads to a unimodal posterior \(p(\vb, \sigma^2 \mid \mx, \vy)\), whereas an unconditional prior

\begin{equation} \vb \mid \lambda \sim \Lapl(0, \lambda) \end{equation}

does not. We previously noted that for the spike and slab prior

\begin{equation} b_j \mid \pi, \sigma_b^2 \sim \pi \N(0, \sigma_b^2) + (1 - \pi) \delta_0(\cdot), \end{equation}

the (approximate) posterior \(p(\vb \mid \mx, \vy, \sigma_b^2)\) sometimes converges to a point mass on zero. Here, we investigate how scaling the slab variance changes this behavior.

Setup

%matplotlib inline
%config InlineBackend.figure_formats = set(['retina'])

import matplotlib.pyplot as plt
plt.rcParams['figure.facecolor'] = 'w'
plt.rcParams['font.family'] = 'Nimbus Sans'

Scaling priors for regression models

Table of Contents

Introduction

Setup

Results