Survey Weighting

Survey weighting is the practice of assigning each respondent a weight — a multiplier — so that the sample, once weighted, matches the known makeup of the population it is meant to describe. Raw survey data almost never mirror the population exactly. Some people are far more likely to answer than others, certain groups get sampled at higher rates by design, and nonresponse skews who ends up in the data. If young men are scarce in your responses but plentiful in the population, their answers, left alone, count too little. Weighting fixes this by giving each response a value that reflects how many people in the population it stands for. A respondent from an under-represented group counts for more; one from an over-represented group counts for less. The weighted totals then reproduce the population's structure rather than the sample's accident.

The weights come from comparing the sample to trusted external figures — usually census benchmarks or other reliable population counts — on characteristics such as age, sex, region, education, or race. Where a group is thinner in the sample than in the population, its weight rises; where it is thicker, its weight falls. There are several sources of weight. Design weights correct for known unequal selection probabilities baked into the sampling plan. Nonresponse and post-stratification weights correct for who actually answered. Once applied, every estimate — a mean, a percentage, a cross-tab — uses the weights, so a weighted approval rating or a weighted purchase intent reflects the population, not the raw respondents. Weighting does not add information that was never collected; it rebalances the information you have so the loudest-answering groups do not distort the picture.

Survey weighting is not one technique but a family, and knowing which is which keeps the adjustment honest. Design weights come first and are the most clear-cut: if your sampling plan deliberately drew some people with a higher chance than others — oversampling a small region to study it closely, say — each respondent's design weight is the inverse of their selection probability, so the oversampled group is scaled back to its true share. Post-stratification comes after the data are in. It sorts respondents into cells defined by traits like age-by-region, then scales each cell so its weighted size equals the population's known size for that cell. Both aim at the same target — a sample that matches the population — but design weights address how you sampled, while post-stratification addresses who responded.

Raking, also called iterative proportional fitting, handles the common case where you know the population totals for several traits separately but not for every combination of them. You might have census figures for age and, separately, for region, but not a full age-by-region table. Raking adjusts the weights to match the age margins, then the region margins, then age again, cycling until all the margins line up at once. It is the workhorse behind many published polls precisely because full joint benchmarks are rarely available. The distinction to hold onto is this: design weights fix unequal sampling, post-stratification fixes response composition against a full cross-classification, and raking fixes it against separate margins. All three reweight responses toward the population; they differ in what information about the population they require and lean on.

Using survey weighting well means correcting real imbalances without manufacturing false precision. Weight on variables that genuinely relate to what you are measuring and where the sample truly departs from the population — age, region, education for many opinion studies — using trustworthy benchmarks such as recent census data. Keep the weights from swinging too wildly: a handful of respondents carrying enormous weights can dominate an estimate and make it jumpy, so analysts often trim or cap extreme weights to steady the result. Always report that estimates are weighted and describe how, so readers can judge the adjustment. And remember the effective sample size: heavy weighting reduces how much independent information the sample carries, which widens the true margin of error even though the headcount is unchanged.

The failures are as instructive as the method. Weighting cannot rescue a badly biased sample — if a whole slice of the population is essentially absent, no multiplier conjures their views, and heavy weighting of the few who did respond just amplifies their idiosyncrasies. Weighting to the wrong benchmarks, or to variables unrelated to the outcome, adds noise rather than accuracy. Extreme, untrimmed weights can hand a poll to a tiny, unrepresentative handful. And reporting weighted results without saying so, or without acknowledging the shrunken effective sample size, oversells their certainty. Used with judgment — sensible variables, sound benchmarks, controlled weights, and honest reporting — survey weighting turns an imperfect sample into a fair reflection of the population. Overused or misused, it dresses a broken sample in a costume of representativeness.

Weighting in survey sampling grew out of twentieth-century probability-sampling theory, where each respondent's weight reflects how many population members that respondent represents.

Etymology: source.

Search interest for this term over the last five years:

View interest-over-time on Google Trends →

Areas of marketing where survey weighting is a core concern: