LASSO Regression (Least Absolute Shrinkage and Selection Operator)

LASSO regression — the Least Absolute Shrinkage and Selection Operator — is a form of linear regression that adds a penalty for large coefficients, and by doing so it both stabilizes the model and selects which variables to keep. Ordinary regression fits coefficients that minimize prediction error alone, which can produce a bloated model that overfits noise when there are many predictors. LASSO adds an L1 penalty: the sum of the absolute values of the coefficients, multiplied by a tuning knob usually written as lambda. Minimizing error plus that penalty pulls every coefficient toward zero, and, crucially, the geometry of the absolute-value penalty pushes the weakest coefficients all the way to exactly zero. Those variables drop out of the model entirely. So LASSO does regression and feature selection in a single step, producing a sparse model that keeps only the predictors earning their place.

That built-in selection is why LASSO is so useful when you have many candidate variables and suspect only a few really matter. In modern datasets — marketing mix models with dozens of channels, customer datasets with hundreds of features, genomics with thousands — most predictors add little but noise, and a model that uses them all is fragile and hard to read. By zeroing out the irrelevant ones, LASSO yields a shorter, more interpretable equation and often predicts better on new data because it has resisted overfitting. The lambda knob controls how aggressive it is: a larger lambda means a heavier penalty, more coefficients driven to zero, and a sparser model; a smaller lambda keeps more variables. You choose lambda by cross-validation, testing which value predicts best on held-out data, so the model's complexity is tuned to the evidence rather than guessed.

LASSO's closest cousin is ridge regression, and the difference between them comes down to the shape of the penalty. Ridge uses an L2 penalty — the sum of the squared coefficients — while LASSO uses an L1 penalty — the sum of the absolute values. That sounds like a technicality, but it changes the behavior completely. Ridge shrinks all coefficients smoothly toward zero but almost never sets any exactly to zero, so every variable stays in the model, just with a smaller weight. LASSO, because of the sharp corners of the absolute-value penalty, drives some coefficients precisely to zero and eliminates those variables outright. So ridge is pure shrinkage and LASSO is shrinkage plus selection. If you want a sparse model that names a subset of important features, you reach for LASSO; if you want to keep everything but tame it, you reach for ridge.

Neither is universally better; they suit different situations, which is the practical point. Ridge tends to win when many predictors each contribute a little and are correlated with one another, because it shares weight among them rather than arbitrarily picking one. LASSO tends to win when you believe the truth is sparse — only a handful of variables genuinely matter — and you want the model to say which. LASSO has a known quirk with groups of correlated predictors: it often keeps just one from the group and zeros the rest, which can look unstable. The elastic net was invented precisely to bridge the two, blending the L1 and L2 penalties to get some selection and some grouping at once. Choosing among LASSO, ridge, and elastic net is a judgment about whether you want selection, shrinkage, or a mix, tuned by cross-validation.

Use LASSO when you have more candidate features than you trust, want a model you can read, and believe only some predictors truly matter. Standardize your variables first, because the L1 penalty is scale-sensitive and will unfairly punish features measured in large units. Tune lambda by cross-validation rather than by eye, and look at how the selected set changes across the regularization path — the sequence of models as lambda varies — to judge which variables are robustly chosen and which flicker in and out. Treat the surviving coefficients as a selection of useful predictors, not as unbiased effect sizes, because the penalty deliberately biases them toward zero. When predictors are heavily correlated, consider the elastic net so LASSO's tendency to arbitrarily drop members of a correlated group does not mislead you.

The traps are forgetting to standardize features, so the penalty falls unevenly and selection is distorted; reading LASSO coefficients as clean, unbiased effect estimates when they are shrunk on purpose; and trusting the exact set of selected variables as if it were stable, when correlated predictors make LASSO's picks jumpy. People also pick lambda arbitrarily instead of by cross-validation, landing on a model that is too sparse to predict or too dense to interpret. And they use LASSO for causal claims it cannot support — it selects predictors that help forecast, not variables proven to cause the outcome. The discipline is to scale inputs, tune lambda with data, treat the selection as informative but not gospel, reach for elastic net under correlation, and keep LASSO in its lane as a prediction-and-selection tool rather than a causal verdict.

The term, coined by statistician Robert Tibshirani in 1996, is an acronym for Least Absolute Shrinkage and Selection Operator.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where lasso regression (least absolute shrinkage and selection operator) is a core concern: