Two Funds Both Returned 8 Percent. Standard Deviation Shows They Were Not Even Close.

How Standard Deviation Was Developed

The earliest mathematical foundation came from Abraham de Moivre, who in 1738 approximated the binomial distribution using a curve that incorporated a measure of spread equivalent to what we now call the standard deviation. De Moivre was working on probability problems related to gambling, and his bell-shaped approximation was a practical tool for calculating probabilities over many trials.

Carl Friedrich Gauss advanced the concept in the context of astronomical observation. Astronomers making repeated measurements of a star's position encountered variation in their measurements, and they needed a principled way to analyze that variation. Gauss, around 1795 and formalized in 1809, assumed that measurement errors followed a symmetric, bell-shaped distribution and developed the method of least squares to find the best-fitting estimate for the true position. The curve he described is now called the Gaussian distribution or normal distribution, and its width is parameterized by what we now call the standard deviation.

Karl Pearson coined the term "standard deviation" in 1893. Pearson was a mathematician at University College London who had become interested in applying rigorous statistical methods to biological data, particularly the study of heredity and natural variation. Working with Francis Galton, whose book Natural Inheritance had introduced the concept of regression toward the mean in 1889, Pearson formalized the standard deviation as the primary measure of dispersion in a dataset. Between 1893 and 1912, Pearson published 18 papers under the general title "Mathematical Contributions to the Theory of Evolution," which introduced standard deviation, correlation, regression analysis, and the chi-squared goodness-of-fit test into the statistical toolkit. The modern discipline of statistics as practiced in scientific research descends directly from this body of work.

How the Calculation Works

The calculation involves several steps. First, find the mean of the dataset by summing all values and dividing by the count. Second, calculate how far each data point lies from the mean and square each of those distances. Squaring makes all values positive and amplifies larger deviations more than smaller ones. Third, average the squared distances and take the square root. The result is the standard deviation, expressed in the same units as the original data.

There is one important variation: whether to divide by n or by n minus 1 when averaging the squared distances. Dividing by n gives the population standard deviation, appropriate when the dataset contains every member of the group being measured. Dividing by n minus 1 gives the sample standard deviation, appropriate when the dataset is a sample drawn from a larger population.

The n minus 1 correction is called Bessel's correction, named for Friedrich Bessel, the German mathematician and astronomer who identified the problem in the early nineteenth century. A sample drawn from a population tends to cluster closer to the sample mean than to the population mean, which causes the sample's dispersion to underestimate the population's true dispersion. The n minus 1 correction compensates for this bias. For large datasets, the difference between n and n minus 1 is negligible. For datasets with fewer than 30 observations, the correction can be meaningful.

The 68-95-99.7 Rule

Standard deviation has specific interpretations when data follows a normal distribution. In a normal distribution, approximately 68 percent of data points fall within one standard deviation of the mean, approximately 95 percent fall within two standard deviations, and approximately 99.7 percent fall within three standard deviations. This relationship is called the empirical rule or the 68-95-99.7 rule.

This rule is why manufacturers set quality tolerances in multiples of sigma. A process with three-sigma control has a defect rate of approximately 0.27 percent, meaning fewer than 3 in 1,000 units fall outside the tolerance range. Six Sigma, the quality management methodology developed at Motorola in the 1980s and popularized by General Electric under Jack Welch in the 1990s, targets defect rates of 3.4 per million opportunities, which corresponds to six standard deviations of separation between the process mean and the specification limits. The methodology's name is taken directly from this statistical target.

Standard Deviation in Finance

Finance adopted standard deviation as the primary measure of investment risk through the work of Harry Markowitz, whose 1952 paper "Portfolio Selection" in the Journal of Finance introduced what became known as Modern Portfolio Theory. Markowitz argued that investors should evaluate portfolios not just by expected return but by the combination of return and variance (the square of standard deviation). A portfolio with the same expected return but lower variance was strictly preferable.

Markowitz's framework showed mathematically that diversification reduces portfolio variance without necessarily reducing expected return, because assets that are imperfectly correlated have volatility that partially cancels. This insight, for which Markowitz received the Nobel Prize in Economics in 1990, made standard deviation the central quantity in portfolio construction.

In finance, standard deviation of returns is commonly called volatility. Historical volatility, calculated from past return data, is a backward-looking measure. Implied volatility, derived from options prices, is a forward-looking market estimate of expected future standard deviation. The VIX index, often called the "fear gauge" of the stock market, measures the implied volatility of the S&P 500 index and represents the market's consensus estimate of the standard deviation of S&P 500 returns over the next 30 days, annualized.

When Standard Deviation Misleads

Standard deviation treats positive and negative deviations symmetrically. A fund that is sometimes 10 percent above its mean and sometimes 10 percent below its mean has the same standard deviation as a fund that is always close to its mean in either direction. But investors typically care more about downside deviations than upside deviations. A fund that occasionally surges 20 percent above its mean and rarely falls is different in practical terms from one that occasionally falls 20 percent.

Practical Uses Beyond Finance

Manufacturing quality control uses standard deviation to monitor process consistency. A process producing parts with a small standard deviation of dimensions is more consistent than one with a large standard deviation. Control charts, developed by Walter Shewhart at Bell Telephone Laboratories in the 1920s, plot process measurements against control limits set at multiples of the standard deviation to detect when a process has shifted.

Clinical research uses standard deviation in reporting experimental results. Effect sizes in psychology and medicine are often expressed in standard deviation units, allowing comparisons across studies that use different measurement scales. A drug that improves a symptom score by one standard deviation has a larger effect than one that improves it by 0.2 standard deviations, regardless of what units the score uses.

Conclusion

From de Moivre's 1738 probability curves to Gauss's astronomy error analysis to Karl Pearson's 1893 coining of the term to Markowitz's 1952 portfolio theory, standard deviation developed as a tool for making the variation in data visible and quantifiable. The calculation reveals what the average cannot: not what the typical value is, but how typical the typical value actually is.

For calculating standard deviation on any set of numbers, ToolHQ's standard deviation calculator handles both population and sample versions, shows the step-by-step calculation, and outputs the variance and mean alongside the result.

Frequently Asked Questions

Try These Free Tools