In probability theory, the Central Limit Theorem (CLT) establishes that, in many situations, when independent random variables are added, their properly normalized sum tends toward a normal distribution (informally a bell curve) even if the original variables themselves are not normally distributed. The theorem is a key concept in probability theory because it implies that probabilistic and statistical methods that work for normal distributions can be applicable to many problems involving other types of distributions. This theorem has seen many changes during the formal development of probability theory. Previous versions of the theorem date back to 1811, but in its modern general form, this fundamental result in probability theory was precisely stated as late as 1920, thereby serving as a bridge between classical and modern probability theory.
For example, suppose that a sample is obtained containing many observations, each observation being randomly generated in a way that does not depend on the values of the other observations, and that the arithmetic mean of the observed values is computed. If this procedure is performed many times, the central limit theorem says that the probability distribution of the average will closely approximate a normal distribution. A simple example of this is that if one flips a coin many times, the probability of getting a given number of heads will approach a normal distribution, with the mean equal to half the total number of flips. At the limit of an infinite number of flips, it will equal a normal distribution.
A mathematically provable principle about obtaining means of samples that has two major ramifications:
– The standard deviation of averages of samples from the population will be approximately equal to the standard deviation of the population divided by the square root of the sample size.
– Regardless of the shape of the original distribution (even for very non-normal distributions such as exponential distributions), the distributions of averages of samples from the population approach the shape of a normal distribution.
References
Wikipedia. Central Limit Theorem. https://en.wikipedia.org/wiki/Central_limit_theorem
