# Markov chain principle

A usual approach to explain Markov chain theory on a continuous space is to start with a 
transition kernel $P(x,A)$ where $x \in \mathbb{R}^{p}$ and $A \in \mathcal{B}$, where $\mathcal{B}$ 
is the Borel $\sigma$-algebra on $\mathbb{R}^{p}$ {cite}`wakefield2013bayesian`. This transition 
kernel is a conditional distribution function that represents the probability of moving from $x$ to 
a point in the set $A$. It is interesting to notice two properties: $P(x,\mathbb{R}^{p}) = 1$ and 
$P(x,\{x\})$ is not necessarily zero, meaning that a transition might be possible from $x$ to $x$. 
For a single estimation, from a given starting point $x_0$, this can be summarised as 
$\mathbb{P}(x_{1} \in A | x_0) = P(x_0,A)$. The Markov chain is defined as a sequence by repeatedly 
applying this transition kernel a certain number of times, leading to the k-th estimate ($k \ge 1$) $ 
\mathbb{P}(x_{k} \in A | x_0) = P^{k}(x_0,A)$ where $P^{k}$ denotes the k-th iteration of the kernel 
$P$ {cite}`tierney1994markov`.

The important property of a Markov chain is the invariant distribution, $\pi^{*}$, which is the only 
distribution satisfying the following relation

```{math}
:label: eq_invDistrMCMC

\pi^{*}(dy)=\int_{\mathbb{R}^{p}} P(x,dy)\pi(x)dx
```

where $\pi$ is the density with respect to the Lebesgue measure of $\pi^{*}$ (meaning $\pi^{*} (dy) 
= \pi(y)dy$). This invariant distribution is an equilibrium distribution for the chain that is the 
target of the sequence of transitions, as 

```{math}
\lim_{k \rightarrow \infty} P^{k}(x,A) = \pi^{*}(A)
```

The Markov chain Monte Carlo approach (MCMC) approach works as follows: the invariant 
distribution is assumed known, since it is the distribution from which we wish to sample, while the 
transition kernel is unknown and to be determined. This might seem to be "the proverbial needle in a 
haystack" but the idea is to be able to express the target density using a transition probability 
kernel $p(x,y)$ (describing the move from $x$ to $y$) as 

```{math}
:label: eq_trankernGeneForm

P(x,dy) = p(x,y)dy + r(x)\delta_{x}(dy)
```

where $\delta_{x}(dy)=1$ and 0 otherwise, while $r(x)=1-\int_{\mathbb{R}^{p}} p(x,y)dy$ is the 
probability that the chain remains at its current location. If the transition part of this function, 
$p(.,.)$, satisfies the *reversibility condition* (also called *time reversibility*, 
*detailed balance*, *microscopic reversibility*...) 

```{math}
:label: eq_reversconditionMCMC

\pi(x)p(x,y)=\pi(y)p(y,x)
```

then $\pi(.)$ is the invariant density of $P(x,.)$ {cite}`tierney1994markov`.