7.3. Analytical linear Bayesian estimation
This method consists mainly of the analytical formulation of the posterior distribution under assumptions: the problem can be considered linear and the prior distributions are normally distributed (or non-informative/flat, as noted at the end of this section).
In the specific case of a linear model, one can then write \(f_\theta(\mathbf{x})=h^T(\mathbf{x})\theta\) where \(h(\mathbf{x})\) is the regressor vector. This way of writing the model can include an “hidden virtual” \(\theta_0 = 1\) whose purpose is to integrate a constant term into the regression (to describe a pedestal). Using the statistical approach introduced in Brief reminder of theoretical aspects, one can also define the covariance matrix of the residuals which will be written hereafter as \(\Sigma={\rm diag}(\sigma_{\varepsilon_1},\ldots,\sigma_{\varepsilon_n})\)
From there, one can construct the design matrix \( H = [h(\mathbf{x}_1), \ldots, h( \mathbf{x}_n)]^T \in M_{n,p}(\mathbb{R})\) whose columns define the subspace onto which the model is projected. With a normal prior, which follows the form \(\theta \sim \mathcal{N}(m_\theta, \Sigma_\theta)\) the posterior is also expected to be normal, so it can be written \(\pi(\theta|\mathbf{y}) \sim \mathcal{N}(m^{post}_\theta,\Sigma^{post}_\theta)\) where its parameters are expressed as
and
It is also possible, as introduced in Introduction to Bayesian approach, to use a non-informative prior such as Jeffrey’s prior: it is an improper flat prior (\(\pi(\mathbf{y})\propto 1\)) [Bio15], whose posterior distribution (in the linear case) is also Gaussian. For this prior, the posterior parameters are equivalent to those obtained with a Gaussian prior, given in Equation 7.7 and Equation 7.8 with all references to \(\Sigma_\theta\) removed:
This final form corresponds to the expected results obtained when only considering linear regression within the weighted least squares approach [BF10].
