Probability integral transform.

Theorem
Let X be a random variable with distribution function F(x) then Y = F(x)∼ U(0,1).

Proof

Let distribution function of Y be equal to G(y).

G(y) = P(Y ≤ y) = P(F(x) ≤ y)

Since F(x) is a distribution function it is monotonically increasing and its inverse function is well defined

G(y) = P(x ≤ F⁻¹(y))
G(y) = F(F⁻¹(y))

G(y) = y which is the distribution function of uniform distribution.

On differentiating the above equation.

g(y) = 1 where 0 ≤ y ≤ 1

Sampling using Probability Integral Transform.

Also called inverse transform sampling

Method

The inverse transform sampling method works as follows:

1. Generate a random number u from the standard uniform distribution in the interval [0,1].
2. Find the inverse of the desired CDF, e.g. F⁻¹(x).
3. Compute X = F⁻¹(u). The computed random variable X has distribution F(x).

Example

Lets try to obtain random samples from exponential distribution with θ =1 through inverse transform sampling and compare it with randomly samples values from the exponential distribution.

uniform_random_variates = np.random.uniform(size = 500)

uniform_random_variates holds 500 random samples from uniform distribution i.e, U(0,1)

def inverse_cdf_exponential(variate):
    return -(np.log(1-variate))

The above function runs the given value through the inverse CDF of exponential function

calculated_exponential_variate = inverse_cdf_exponential(uniform_random_variates)
exponential_variate = np.random.exponential(size = 500)

calculated_exponential_variate and exponential_variate holds values from exponential distribution obtained through inverse transform sampling and simple random sampling respectively.

fig, (ax1, ax2) = plt.subplots(1, 2,figsize=(20, 10),sharey=True)
ax1.hist(calculated_exponential_variate,color='turquoise')
ax1.set_title('Inverse Transform Sampling')
ax1.set_ylabel('Frequency',fontsize = 'medium')
ax1.set_xlabel('simulated exponential variates')
ax2.hist(exponential_variate, color ='steelblue')
ax2.set_title('Sampling from exponential distribution')
ax2.set_xlabel('sampled exponential variates')

Therefore, we can observe that this method follows a very neat procedure to obtain random samples from distribution with a closed from inverse CDF.