The Five Assumptions That Guarantee Ordinary Least Squares is the Best Linear Unbiased Estimator

For a linear regression model to be mathematically optimal, the errors it makes must be entirely random and independent of the variables being measured. If that single binding constraint fails, the entire apparatus of Ordinary Least Squares (OLS) begins to drift from the truth. The Gauss-Markov theorem establishes exactly what conditions must hold for OLS to be crowned the Best Linear Unbiased Estimator (BLUE). In practice, these conditions almost never hold perfectly, forcing analysts to measure how far their data deviates from the theoretical ideal.[2][6]

The acronym BLUE unpacks the specific guarantees of the theorem. "Linear" means the model's parameters are estimated as a linear combination of the sample data. "Unbiased" ensures that, over infinite repeated samples, the model's average estimate will perfectly match the true population parameter. "Best" is the most mathematically demanding claim: it means the estimator has the lowest possible variance among all linear unbiased estimators. As statistician Jim Frost notes, "Best means it has the tightest sampling distribution" [4].[4]

The first two of the five assumptions establish the basic geometry of the data. Assumption one requires linearity in parameters, meaning the relationship between the independent variables and the dependent variable can be drawn as a straight line, even if the variables themselves are squared or transformed [5]. Assumption two demands random sampling. The data points must be drawn randomly from the population, ensuring that the sample accurately represents the broader reality without systematic selection bias [2].[2][5]

The third assumption prohibits perfect collinearity. In a multiple regression model, no independent variable can be a perfect linear combination of another. If a dataset includes both a temperature in Celsius and a temperature in Fahrenheit, the model's matrix algebra collapses. The design matrix must have full column rank, meaning each variable brings mathematically unique information to the equation [3]. Without this rank of at least 1 for every parameter, the OLS estimator cannot isolate the individual effect of any single variable.[3]

The fourth assumption—zero conditional mean of the error term—is the most difficult to satisfy in the real world. It requires that the unobserved errors in the model are completely uncorrelated with the independent variables [1]. If an analyst is modeling wages based on education, but omits a crucial variable like innate ability, that omitted variable gets absorbed into the error term. If ability is correlated with education, the error term is no longer independent, and the OLS estimate becomes fundamentally biased.[1]

The final assumption is homoskedasticity, which dictates that the variance of the error term must remain constant across all levels of the independent variables [2]. If a model predicts household spending based on income, the errors typically fan out as income rises, because wealthier households have more discretionary variance. This expanding cone of data violates the fifth assumption, altering the mathematical properties of the resulting regression line.[2]

The mathematical elegance of the Gauss-Markov theorem lies in how these assumptions split the BLUE properties. The proof of unbiasedness relies exclusively on the first four assumptions [2]. If a model suffers from heteroskedasticity (violating the fifth assumption), the OLS estimates remain perfectly unbiased. They will still center on the true population parameter, meaning the model is not systematically overestimating or underestimating the effect.[2][6]

However, violating that fifth assumption strips OLS of its "Best" title. Without homoskedasticity, the estimator no longer possesses the minimum possible variance [4]. The estimates become wider and less precise, and the standard errors calculated by the model become mathematically invalid. This destroys the reliability of p-values and confidence intervals, making it impossible to accurately test hypotheses even if the core coefficient is unbiased.[4]

The theorem's origins trace back to Carl Friedrich Gauss in 1821 and Andrey Markov in 1900. Gauss initially proved the least squares method's optimality under the assumption of normally distributed errors. Markov later generalized the proof, demonstrating that normality was not required—only the specific moments of the error distribution outlined in the five assumptions [3]. According to the Wolfram MathWorld entry, the theorem states that "in a linear regression model in which the errors have expectation zero and are uncorrelated and have equal variances, the best linear unbiased estimator of the coefficients is given by the ordinary least squares estimator" [1].[1][3]

In modern data science, the Gauss-Markov theorem serves less as a guarantee and more as a diagnostic baseline. When the assumptions fail, analysts do not abandon OLS; they apply corrections. Robust standard errors are deployed to fix the variance inflation caused by heteroskedasticity, while instrumental variables are used to force the zero conditional mean assumption back into alignment [5]. The theorem defines the perfect vacuum, allowing statisticians to measure the friction of the real world.[5][6]