To say a statistic is unbiased is to say that, on average, its value is equal to the parameter it is estimating. Sampling error, the inherent randomness which determines which observations appear in the sample, ensures that each individual sample will produce a different estimate; any given sample’s statistic won’t be exactly equal to the parameter. But if many samples are taken, the average of all those statistics will converge on the true population parameter’s value.
The arithmetic average is an unbiased statistic. Each sample’s mean will be different, some above the true population mean and some below. Because the discrepancy is random, however, with many samples the overestimates and underestimates balance out. The average of all the differences between each sample average and the true population average will be close to zero. That’s what makes the mean unbiased.
Standard deviation is more complicated. There are two different equations for standard deviation. In both cases, you first calculate the mean, then the sum of squared deviations from the mean. If the data represents a complete population, you then divide the sum of squared deviations by N, the total number of observations in the entire population. With data from a sample, however, you divide by n - 1, the number of scores in the sample minus 1. This is because if you use the first approach with data from a sample, dividing by N, the statistic will be biased.
More specifically, using the population standard deviation equation with sample data has the effect of systematically underestimating the population parameter. As with the mean, each sample produces a slightly different estimate, but unlike the mean, the difference between estimate and parameter is not completely random. More samples will underestimate the population variability than overestimate it. As a result, the average of all sample estimates of standard deviation would be lower than the true population value.
Dividing by n - 1 when calculating a sample’s standard deviation corrects for that bias, producing an unbiased estimate. Why? It has to do with the fact that the standard deviation is a measure of variability, a measure of how spread out the data is. When you estimate the population standard deviation from a sample, you’re trying to use a small subset of the population to estimate the variability in the entire population.
But, there’s an inherent problem here. When we calculate the standard deviation of a sample, we’re using the sample mean, not the population mean, because the population mean is usually unknown. The sample mean, being calculated from the data in the sample, is more likely to be closer to the individual data points in the sample than the population mean would be. This means the squared deviations from the sample mean are likely to be smaller, on average, than the squared deviations from the population mean. This is what leads to the underestimate of the variance, and hence the standard deviation, when we divide by N’ instead of n - 1.
Now, why n - 1 instead of, say, n - 2 or n - 3? This has to do with something in statistics known as ‘degrees of freedom’. Degrees of freedom are, in a way, the number of values that are free to vary given the constraints in the problem.
In the case of variance and standard deviation, the constraint is the sample mean. Once we’ve calculated the sample mean, the sum of the deviations from the mean must equal zero. This means that once we know the deviations of n - 1 values, the deviation of the nth value is completely determined. So, we say we have n - 1 degrees of freedom.
When you divide by n - 1 instead of ‘n’, you are effectively adjusting the sample variance and sample standard deviation upwards to account for the fact that you’ve underestimated the squared deviations because you used the sample mean instead of the population mean. This adjustment gives a larger value for the sample variance and sample standard deviation, which on average gives a better (i.e., unbiased) estimate of the population variance and population standard deviation.
This visualization can demonstrate this effect: if you take many samples and calculate the sample standard deviation using n and n - 1 in the denominator, you’ll see that the average of the sample standard deviations calculated with n - 1 in the denominator will be closer to the true population standard deviation, while those calculated with ‘n’ in the denominator will tend to underestimate it. This shows how the n - 1 adjustment corrects for the bias in estimating the population standard deviation from a sample.