In certain cases, it is useful to transform a random variable to another random variable
via a transformation
. For instance, this situation arises typically if one wants to sample from the probability density function
of a positively-valued random variable
. Markov chain Monte Carlo (MCMC) algorithms are conventionally designed to draw correlated samples from a desired target (density)
of a random variable
taking values over the real line. So, if the target of interest
has support over the positive real line and the MCMC algorithm samples from a density
with support over the real line, then a random variable transformation, such as
, can help resolve the matter. In particular, the transformation allows to sample from
via the MCMC method of choice and then the inverse transformation
converts the simulated Markov chain to a set of sample points from
. Obviously, it is needed to find the form of the target density
on which MCMC will be applied.
Although such random variable transformations are common practice, one may need to look up the formula for the transformation to pass from the original density to the transformed density
. The main source of confusion is whether one needs the Jacobian associated with the transformation
or with the inverse transformation
.
There is a way to retrieve the formula intuitively via a geometric argument, rather than trying to uncover it mnemonically. The main argument is that of area preservation in the case of univariate random variables. It suffices to realize that for a small displacement, the area below the curves of the two densities is the same, which means that
.
This realization suffices to recover the remaining steps. It follows that
.
Notice that
,
which gives the transformed density
.
The Jacobian in the univariate case is the derivative , associated with the inverse transformation
. The absolute value of the derivative ensures that the density
is non-negative. Having understood the univariate case, the multivariate scenario follows straightforwardly as
,
where denotes the determinant of the Jacobian of
.
To follow through with the example
,
notice that ,
. So, the derivative
becomes
,
whence
.
The target log-density for MCMC is thus
.