The Inverse CDF Method

Hello. This post is a math and probability post. I will talk about generating non-uniform random variables/numbers with the Inverse CDF Method. The inverse CDF method involves computing quantiles from probabilities and using standard uniform random variables to generate non-uniform random variables.

This topic relates to Probability Theory, and Monte Carlo Simulations.

Introduction

Generating random numbers allows us to simulate natural random events without the actual events occurring. For example, instead of flipping a coin 1000 times, we can use software to simulate or emulate this for us.

There is a lot of utility and usefulness in generating random numbers (and/or variables). Applications include finding averages, estimating the mean or frequency/probability distribution, extracting data for decision making and so on.

The Uniform Random Variable

The uniform random variable is a continuous random variable which takes on values from parameters to . If is a uniform random variable the we denote the random variable as . The uniform random variable takes on any value between a and b, inclusive.

The continuous probability distribution of a uniform random variable is:

If the parameters and are 0 and 1 respectively then is a standard uniform random variable . It would be denoted as . The continuous probability distribution of a standard uniform random variable is just .

The Cumulative Distribution Function (CDF)

The Cumulative Distribution Function or the CDF is the probability that a real-valued random variable with a given probability distribution is less than or equal to a quantity . It is often denoted by .

Some CDF Properties

1) The CDF is a non-decreasing function.

2) (An upper bound and horizontal asymptote at if approaches .)

3)  (A lower bound and horizontal asymptote at if approaches -infinity.)

The following image of a sample CDF summarizes the above three properties. This image is for a standard normal distribution. (Source: https://upload.wikimedia.org/wikipedia/commons/2/2b/Cumulative_distribution_function_for_normal_distribution%2C_mean_0_and_sd_1.png)

For a uniform random variable, the CDF is is .

Proof:

For the case of a standard uniform random variable, substituting and yields the CDF as .

The Inverse CDF Method For Generating Non-Uniform Random Numbers

We have discovered that the standard uniform random variable takes on values between 0 and 1 inclusive. The CDF of a (continuous) distribution also takes on values between 0 and 1 inclusive. In addition, the inverse CDF is also an increasing function (of ). These facts are used when using the Inverse CDF Method for generating non-uniform random numbers.

We now have the necessary background to understand the Inverse CDF Method. We use uniform random variables to generate non-uniform random variables. The algorithm is as follows:

1) Obtain or generate a draw (realization) from the standard uniform distribution

2) The draw from the CDF is given by .

Examples of the Inverse CDF Method

Example One

Suppose we only know how to generate or sample Unif(0,1) random variables. We want to generate Unif(a,b) random variables. The Inverse CDF Method allows us to do this as follows.

The CDF of Unif(a,b) is for any in the open interval . To obtain the inverse CDF, we solve for in .

The resulting inverse CDF is .

To generate a Unif(a, b) random variable from a random variable , we would set the random variable X as:

A check can be made for a = 0 and b = 1 to see that .

Example Two

In this example, we generate an exponential random variable with and the rate lambda as . The continuous probability distribution of the exponential random variable is:

Integrating with bounds from 0 to x gives:

.

Solving for in gives us .

The formula for X is . Alternatively, we can use since from .

Example Three

This third example deals with the Pareto distribution. Given the shape parameter and the scale parameter , the Pareto distribution has a probability density function (pdf) of:

The cumulative distribution function (CDF) of the Pareto distribution involved integrating with bounds from to .

To find the generating formula, set and solve for . This u could be replaced by the standard uniform random variable .

The generating formula would be with . You could replace with just since both and are standard uniform random variables.

References

• Financial Mathematics: A Comprehensive Treatment by Giuseppe Campolieti and Roman N. Makarov
• http://www.math.wm.edu/~leemis/chart/UDR/PDFs/Pareto.pdf
• http://isites.harvard.edu/fs/docs/icb.topic1475777.files/stat110hw5solutions.pdf