Hi there. In this post, I will talk about the famous Normal Distribution. This topic is covered in numerous introductory probability and statistics courses and is well known. The normal distribution is sometimes referred to as a Gaussian or a bell-shaped curve. This post serves as an overview of the normal distribution and its related topics.
This post is suitable for a student who is first learning about the normal distribution assuming a good calculus background, and knows basic summary statistics and integration. If there are any topics that seem to crazy for you, it may be best for you to skip it and search for material that suits you.
- What Is The Normal Distribution?
- The Standard Normal Distribution: A Special Case
- The Cumulative Distribution Function (CDF)
- The Z-Score Table
- 68, 95, 99.7% “Rule”
- Using the Z-Score Table
- More Properties (Moment Generating Functions, Skewness, Kurtosis)
- Distributions Related to Normal (Chi-Squared, t-Dist, Lognormal, Multivariate Normal)
The featured red normal distribution image is taken from http://www.statlect.com/ucdnrm1.htm.
1) What Is The Normal Distribution?
The normal distribution or Gaussian is a continuous probability distribution where it is symmetric at its mean (pronounced like mew). It is the only distribution where its mean is equal to its median and mode.
The density function of the normal distribution is:
We denote a normal random variable as .
The mean of the normal distribution is given by .
Its variance is given by
The area under the curve for the normal distribution is one or:
The Normal distribution is a symmetric (zero skewness) distribution where the mean, median and mode are all equal to each other.
2) The Standard Normal Distribution: A Special Case
When and in the normal distribution, we have the standard normal distribution as a special case. A random variable that follows a standard normal distribution is often denoted by .
Alternatively, one can say that if then the random variable has a N(0, 1) distribution. This N(0,1) distribution is known as the standard normal distribution (Casella & Berger).
We can show that the standard normal random variable has a mean of 0 and a variance of 1 using expected value and variance properties when .
The mean, median and mode (peak) of the standard normal distribution is at .
3) The Cumulative Distribution Function (CDF)
The CDF is the probability that a real-valued random variable with a given probability distribution is less than or equal to a quantity . It is often denoted by .
For the standard normal distribution we have the CDF denoted by . Other notations include , or .
We can also have the equal or greater than case with or or or or
For any random variable X:
for values and .
If then is the median ( percentile). For a standard normal distribution, the median is at .
is the same as and is the same as
With the standard normal case: or
4) The Z-Score Table
We now look at the z-score table. The Z-Score Table applies to the standard normal distribution only and it helps us computes probabilities.
In and in the quantity is the z-score.
From the formula , we can have (where and ).
The z-score is the number of standard deviations above or below the mean . Remember that can be positive or negative while is the square root of and cannot be negative. If the z-score is positive then is greater than the mean . A negative z-score represents less than the mean .
The z-score has no units (such as km, inches, etc.).
One has to be careful when using the z-score table. There are different versions of the z-score table. One corresponds to , one goes with , and a few have
If one is asked to find the probability of a random variable below a certain value then you should use and refer to the z-score table and the diagrams carefully!
It may be helpful to know that such that .
5) 68, 95, 99.7% “Rule”
There is a memory aid called the 68, 95, 99.7 rule. Three numbers represents (approximate) probabilities associated with a range of 1, 2 and 3 standard deviations (respectively) in the standard normal distribution.
1) Case 1 (Between -1 and +1 standard deviations from the mean).
Meaning: 68% of the values are between -1 and +1 standard deviations (z-scores of -1 and + 1) from the mean.
2)Case 2 (Between -2 and +2 standard deviations from the mean).
Meaning: 95% of the values are between -2 and +2 standard deviations from the mean.
3) Case 3 (Between -3 and +3 standard deviations from the mean).
Meaning: 99.7% of the values are between -2 and +2 standard deviations from the mean.
If you wanted the probability for 0 to 1 standard deviation from the mean in the standard normal distribution, just simply divide by 2 from 0.68 to get 0.34. Likewise with the 0.95 and 0.997 values.
The image below summarizes the 68, 95 and 99.7 rule:
This image is taken from http://www.oswego.edu/~srp/stats/6895997.htm
6) Using the Z-Score Table
It is not expected of students (and people in general) to compute probabilities given z-scores by hand. A z-score table takes care of the computations. Here is a sample z-score table.
This z-score table deals with the case of or the probability of the random variable is less than (or equal to) a z-score quantity. If you want the case, you would use as stated in the bottom box of the above image.
Notice how there are two separate tables in the image where each side refers to a negative z-score (below the mean) and a positive z-score (above the mean). Remember that regardless of the sign of the z-score we deal with according to the image.
The left side of the table refers to the ones digit and the tenths of the z-score and the top part refers to the hundreths of the z-score. As an example, if I wanted to find the probability of less than 2.11, I look for 2.1 in the right table and then 0.01 from the top while still being in the same row as 2.1.
When you want to use the z-score table but you are only given a normal random variable (not a standard normal) such as and not , you need to standardize the random variable into as follows:
Suppose the average test score was an 72 out of 100 and the standard deviation is 5. Assuming normality, if we wanted the probability (or percentage) of those who scored between a score of 65 to 85 (inclusive) we would have:
So about 92.7% of students had a score between 65 and a 85 which is between -1.4 standard deviation and 2.6 standard deviations from the mean score of 72 (assuming normality).
7) More Properties (Extra Material)
Moment Generating Function, Skewness and Kurtosis Of A Normal
These next topics are more mathematical/probabilistic in nature. It is more suited for math majors, probability and statistics majors, econometrics people, etc.
a) Moment Generating Function
An alternate way to characterize a random variable besides its probability density function (pdf) is through its moment generating function (mgf). For a normal random variable with a mean and variance , its moment generating function is
For a standard normal random variable where and the moment generating function above condenses to .
b) Skewness of Normal
Skewness refers to the symmetry of a distribution. The skewness formula is given by The normal distribution has zero skewness and it is a symmetric probability distribution.
Here is an image which summarizes skewness:
This image is taken from http://www.managedfuturesinvesting.com/managed-futures/news/aisource-news/2015/10/13/what-is-skewness.
c) Kurtosis of Normal
Kurtosis refers to the peak and shape of a distribution. The kurtosis of a distribution is . For the normal distribution, its kurtosis is 3 and its excess kurtosis is the kurtosis minus 3 which is just 0.
The normal distribution is known as a mesokurtic distribution.
This image provides a good visual of kurtosis:
The image is from http://www.pqsystems.com/eline/2001/02/b.htm.
8) Probability Distributions Related to the Normal
There are many other probability distribution that are related to the normal distribution. Some of them are derived from the normal distribution.
a) Chi-Squared Distribution
Assuming that is a N(0,1) random variable, then . This means that the square of a standard normal random variable is a chi squared random variable with one degree of freedom.
If are independent and each for then
The probability distribution of a chi squared random variable with p degrees of freedom is:
b) Student’s t-Distribution
Suppose we have a random sample from a distribution. The ratio has a Student’s t distribution with (n – 1) degrees of freedom. The ratio T is also a random variable and its probability density function is:
c) Lognormal Distribution
If X has a random variable whose logarithm is normally distributed ( ), then the random variable X has a lognormal distribution. The lognormal probability density function is:
The expected value of the lognormal pdf is and the variance of a lognormal random variable is
d) Multivariate Normal Distribution
When we extend the univariate normal from before into the multivariate normal distribution, we now deal with linear algebra topics such as vectors and matrices.
The multivariate normal distribution in -dimensions is denoted by is given by:
where is the determinant of the variance-covariance matrix , is the mean vector and for i = 1, 2, …, p, and .
Notes & Summary
- You can use or in the continuous case as the probability/integral/measure at a point is zero.
- A picture/visual of the normal distribution can be helpful.
- Remember that the normal distribution is symmetric. That property can be useful in certain computations.
- Probabilities are synonomous with areas for a continuous probability distribution such as the normal distribution.
- The standard normal distribution is a special case of the normal distribution.
Casella, G. and Berger R.L. (2002), Statistical Inference, 2nd Edition, Duxbury.
Johnson, R.A. and Wichern, D.W. (2007). Applied Multivariate Statistical Analysis, 6th Edition. Pearson Prentice Hall, New Jersey.