Spike-and-Slab Regression
Let's supplement our model with a prior distribution called peak-plateau regression.
The gist of it is this: we'll assume that the β coefficients are either exactly 0 with a probability of 1/2, or "something unknown" with a probability of 1/2.
Accordingly, this distribution can be formally written mathematically as follows: β_j, the j-th coefficient of the model, is normally distributed with mean 0 and some variance—the variance γ_j * τ_j squared, where γ_j is either 1 with a probability of 1/2 or 0 with a probability of 1/2.
Accordingly, if this factor γ_j is equal to 0, then β_j is normally distributed with mean 0 and variance 0. And what is a random variable with variance 0? It's a constant. That is, if γ_j takes the value 0, it turns out that β_j is exactly equal to 0 with probability 1/2.
This results in a sharp peak: with probability 1/2, β_j is equal to 0, and with probability 1/2, γ_j is equal to 1, and the variance of β_j is equal to 1. Our a priori assumption is that the variance of β_j is equal to τ_j squared.
And τ_j squared is assumed to be a random variable with an inverse gamma distribution.
For those who don't know what an inverse gamma distribution is, it's simply a distribution that guarantees that τ_j squared is non-negative.
Well, since the variance can't be negative, the inverse gamma distribution is required to be non-negative, and, accordingly, a_1 and a_2 are parameters that determine the shape of the inverse gamma distribution.
Accordingly, we'll choose the shape of the gamma distribution so that τ_j squared takes on fairly large values.
Then, with probability 1/2, the variance of β_j equals 0, meaning we're absolutely certain that β_j equals 0, and with probability 1/2, the variance of β_j, τ_j squared, takes on a huge value, meaning we're absolutely uncertain about the value of β_j.
So, we end up with a mixture of a peak (we're absolutely certain that the coefficient is 0) and a plateau (we don't know where the coefficient lies).
Therefore, this regression is called peak-plateau regression.
===========================
Subscribe to the channel - / @Основыанализаданных
R Programming Course - • Основы программирования на R
Econometrics Basics Course in R - • Основы эконометрики в R