Types of Biased Samples in Everyday Life

Types of biased samples: start with who is missing

Suppose a neighborhood café asks customers whether its opening hours work for them. The answers may be perfectly recorded, yet the result could still be misleading. People who cannot visit during the current hours are unlikely to be standing in the café when the question is asked. The problem lies in who had a chance to answer.

A biased sample systematically favors some outcomes because of the way people or items enter the sample. It is different from ordinary sampling variation: even a carefully selected random sample can differ from its population by chance. Bias is more concerning because collecting more responses through the same flawed process may not remove it. OpenStax emphasizes defining the population and examining how the sample is selected when evaluating a study.

The labels for types of biased samples describe points where selection can go wrong. They can overlap. A street-corner survey might be both a convenience sample and an example of undercoverage. Rather than trying to assign one perfect label, ask which people could be reached, which were invited, and which ultimately responded.

Convenience samples favor the easy to reach

A convenience sample consists of people who are readily available to the researcher. Asking nearby coworkers about commuting habits is quick, but those coworkers may share an office location, schedule, or transport options that other commuters do not. The sample can describe the coworkers who answered; it cannot automatically stand in for everyone who commutes to work.

Timing matters as much as location. A librarian who surveys visitors on a weekday afternoon misses people who only come after work or on weekends. Even if every visitor present agrees to answer, access to the survey depends on when the librarian is there. A long list of responses does not correct that mismatch.

Convenience sampling can be useful for testing whether a questionnaire is understandable or gathering initial ideas. Trouble starts when a result from an easy-to-reach group is presented as an estimate for a broader population. Name both groups plainly: who was actually approached, and who the claim is meant to cover?

Voluntary response attracts motivated voices

A voluntary-response sample forms when people decide for themselves whether to participate, such as by answering an open online poll. Those with strong opinions may be more inclined to respond than those who feel indifferent. The visible result can therefore overstate how widespread a strongly held view is.

This issue is not solved merely by putting the poll where many people can see it. Visibility does not give everyone the same probability of participation. Followers of a particular account may differ from the intended population, and the followers who choose to click may differ again from those who scroll past.

There is a useful distinction between being invited and opting in. A researcher might draw a random list of invitees, which improves the invitation stage, but participation can still depend on personal interest. When reading a poll, check whether respondents were selected for contact, recruited because they were nearby, or simply invited to take part.

Undercoverage leaves groups out of reach

Undercoverage occurs when the method used to build or reach a sample excludes part of the target population, or gives that part little opportunity to be included. A survey conducted only through an app will miss people in the target population who do not use it. A list of current customers will not capture people who stopped buying because of a problem the survey hopes to investigate.

The target population determines whether a gap is an error. If the question is specifically about active app users, surveying app users may be appropriate. If the question is about all residents, the same approach has a coverage problem. There is no way to judge coverage without first stating whom the result is supposed to describe.

Consider also people who are absent from a record because of the outcome being studied. Looking only at successful businesses to learn what business decisions lead to success excludes those that closed. This is often called survivorship bias. The practical check is similar: identify which cases the source could never show you.

Nonresponse changes the invited sample

Nonresponse happens after selection: someone was invited or selected but did not provide usable answers. If the people who do not respond differ on the question being studied, the respondents may no longer represent the intended population. A randomly selected invitation list is valuable, but it is not a guarantee that the final set of answers remains representative.

Imagine a workplace survey about meeting times. Employees with crowded schedules might be less able to complete it, and their preferences could be precisely what the survey needs to capture. Sending reminders or offering several ways to respond may help participation, but a high response count alone does not establish that the remaining missing answers are harmless.

Keep selection bias separate from response bias. The former concerns who ends up in the sample; the latter concerns the answers people give, perhaps because a question is leading or sensitive. Both can distort a survey, and both deserve scrutiny, but changing the wording of a question will not bring excluded people into the sample.

A synthetic example and a practical checklist

All counts, proportions, and calculations in this example are synthetic and illustrative, not findings from a real café. Suppose a café wants to describe all its registered visitors. In this synthetic example, its records classify 80% as weekday visitors and 20% as weekend visitors. Staff collect a convenience sample: 20 weekday visitors and 10 weekend visitors answer a question about whether the hours suit them. In the synthetic responses, 16 weekday visitors and 4 weekend visitors say yes.

The synthetic sample’s overall yes proportion is 20 out of 30, or about 66.7%. Yet weekend visitors make up 10 out of 30 responses, or about 33.3%, rather than the synthetic population share of 20%. Applying the stated group shares to the synthetic within-group results gives 80% × (16 ÷ 20) + 20% × (4 ÷ 10) = 72%. That calculation shows how the group’s mix affects the reported result. It is not a verified population estimate: convenience recruitment could also make respondents unrepresentative within each group, and the calculation cannot fix that.

Before trusting a sampling claim, use this checklist:

• Define the population the claim is about.
• Identify who could be contacted and who could not.
• Ask how invitees were chosen and whether participation was optional.
• Check who did not respond and whether their views might differ.
• Treat adjustments cautiously unless their assumptions are supported.