Validity

Validity is the degree to which a metric actually measures the thing it claims to measure. A valid measure of brand awareness genuinely captures brand awareness — not something next to it, not noise, and not an artifact of how a survey question happened to be worded. Validity is the first question to ask of any number, because a measure that does not represent its concept will mislead you no matter how carefully you collect or chart it.

The reason validity sits above every other measurement quality is simple: marketers act on metrics. When you act on an invalid measure, you act on a number that does not reflect reality, so even flawless execution points you in the wrong direction. If a score you call 'customer satisfaction' really tracks how recently someone bought, every decision you base on it will quietly drift away from the thing you meant to improve.

Validity is constantly confused with reliability, yet the two are distinct, and a metric can have one without the other. Reliability is consistency: the same conditions produce the same reading. Validity is correctness: the reading reflects the real thing. Picture a bathroom scale that always reads exactly five pounds light. It is perfectly reliable — it never wavers — and completely invalid, because it never tells you your true weight. Chasing a number that is reliable but invalid is one of the most common and expensive measurement mistakes a team can make.

You generally want both, and in that order. A valid-but-noisy measure at least points at the right target and can be steadied with a larger sample; a reliable-but-invalid measure points confidently at the wrong target, which is far harder to notice and far more damaging. See the companion entries on a reliable metric and a relevant metric for how these qualities interact.

Validity also differs from the other qualities a good metric should have, and it helps to name them precisely. Relevance asks whether a valid measure actually bears on the decision in front of you. Sensitivity asks whether the measure detects real change when it happens. Objectivity asks whether the number rests on fact rather than opinion. Predictive validity — a specific kind of validity — asks whether the measure forecasts a future outcome it ought to predict. A number can be valid in the broad sense yet still fall short on one of these dimensions, which is why measurement reviews test them separately.

Establishing validity means demonstrating that a metric corresponds to its concept, not merely asserting that it does. The standard approaches mirror the recognised types of validity. Content validity asks whether the measure covers the full concept rather than one convenient slice of it. Construct validity asks whether the measure behaves as the concept should — rising and falling with things it ought to relate to, and staying flat against things it should not. Criterion validity asks whether the measure lines up with an external outcome, either at the same time (concurrent) or later (predictive).

The recurring failures are easy to name and easy to commit: assuming validity instead of testing it, mistaking reliability for validity by trusting a steady but wrong number, and letting a proxy that is simply easy to measure stand in for the concept that actually matters without ever checking that the two move together. Each failure feels like rigour in the moment because there is a real, repeatable number on the dashboard — which is exactly why invalid measures survive so long.

This page covers validity the measurement concept. There is also a company called Validity — an email data-quality and deliverability business behind Sender Score, Everest, and BriteVerify — which has its own entry at Validity (email platform). If you arrived here looking for the deliverability tools rather than the measurement idea, follow that link.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →