The Theory of Large Numbers, Explained

The Theory of Large Numbers: Averages Settle Down

The theory of large numbers is usually called the law of large numbers. It describes a basic pattern in probability: when we repeat a random process under stable conditions and average the results, that average tends to move toward the process’s expected value as the number of observations increases.

The idea is easy to recognize in everyday life. A single commute can be unusually short or long, but an average travel time calculated from many comparable commutes is often more stable. A single sale may be unusually large, but an average transaction value across many comparable sales can provide a more dependable summary. The law concerns the behavior of the average, not the predictability of each individual outcome.

This is a long-run statement about a sequence of observations. It does not say that the average must equal the expected value after a particular number of trials. It says that, under suitable assumptions, the chance of the average being noticeably far from the expected value becomes smaller as the sample grows.

Expected Value Is the Reference Point

The expected value is a probability-weighted average of possible outcomes. It is not necessarily an outcome that will occur, and it is not necessarily the ordinary average of the values listed without considering their probabilities. For a fair six-sided die, for example, the expected roll is three and one-half, even though no individual roll has that value. This die example is a standard illustration; any particular sequence of rolls discussed here is synthetic.

If many independent rolls of that synthetic fair die are averaged, the sample average tends to get closer to three and one-half. Individual rolls still vary, and a later roll does not become more likely to be high or low because earlier rolls were unusual. The law applies to the accumulated average, not to a claim that randomness is correcting itself on the next trial.

The same reasoning applies to measurements, service times, costs, or other quantities when their probability model is meaningful. Before applying the law, we need a defensible definition of what counts as an observation and what population or process those observations represent.

A Worked Synthetic Example

Consider a deliberately synthetic game in which each round awards either zero points or ten points. Suppose the model assigns probability one-half to each result. The expected score is therefore five points per round. All values and probabilities in this example are illustrative and synthetic; they are not measurements from a real game.

Imagine one synthetic sequence of four rounds producing scores of zero, ten, ten, and zero. Its average is five points. Imagine a different synthetic sequence of four rounds producing ten, ten, ten, and ten. Its average is ten points. Both sequences are compatible with the same model, and neither sequence changes the expected score of a future round.

Now imagine extending the first synthetic sequence to a much larger collection of independent rounds. The cumulative average may move above or below five at different stages, but large positive and negative deviations tend to offset each other over time. The law of large numbers predicts increasing stability around five, not a smooth path toward five and not a guarantee that the final observed average will be exactly five.

This example also shows why a streak is not evidence that the probability model has changed. Four synthetic ten-point results in a row may look surprising, but they do not force the next round to produce zero points. The long-run average and the next individual outcome answer different questions.

What the Law Does Not Say

The law of large numbers does not say that a large sample contains every kind of person, location, time, or condition in the population. A survey can have many responses and still be biased if its participants were selected in a way that excludes important groups. Repeating the same biased procedure produces a more precise estimate of the wrong target.

It also does not remove dependence. Weather observations recorded on adjacent days may be related, and customers arriving at one time of day may differ from customers arriving at another. When observations are strongly dependent or when the underlying process changes, the simple independent-and-identically-distributed model may not apply. More advanced versions of the law cover some dependent settings, but those conditions must be justified rather than assumed.

Finally, the law does not promise a fixed error reduction for every dataset. The required sample size depends on the variability of the observations, the desired accuracy, and the probability model. A heavy-tailed distribution can produce occasional extreme values that make averages settle slowly, even when an expected value exists.

Law of Large Numbers and Central Limit Theorem

The law of large numbers and the central limit theorem are related but distinct. The law explains why a sample average tends to approach an expected value. The central limit theorem describes how standardized sample averages often become approximately normally distributed under suitable conditions, which helps with probability calculations and uncertainty intervals.

A useful distinction is convergence versus measurement of uncertainty. The law gives a long-run direction: larger samples generally make the average more reliable under the model. The central limit theorem helps quantify the remaining random variation around that average. OpenStax discusses this connection in its treatment of sample means and the central limit theorem.

Neither theorem repairs a poor measurement process. If observations are systematically selected, recorded with an important error, or drawn from a changing population, mathematical results based on an idealized model may not describe the data. Good statistical reasoning checks the data-generating process before applying a theorem.

Using the Idea in Everyday Decisions

When comparing averages, ask what process generated the observations. A restaurant’s average wait time may become more stable after many visits, but that stability may describe only the hours, locations, and customers included in the records. A personal spending average may be useful for planning, but an unusual annual expense can matter greatly even if it has little effect on a long-run average.

It is also important to distinguish more data from better information. More observations help when they are relevant, measured consistently, and sufficiently independent for the intended analysis. A large administrative dataset can still miss people who never enter the system. A large set of online responses can still overrepresent people who choose to respond.

The law of large numbers is therefore both reassuring and limited. Repetition can make random variation less influential in an average, but only within a suitable probability model. The model’s target, assumptions, and data collection process determine what the stabilized average actually means.

Practical checklist

• Define the population, process, or question represented by each observation.
• Check whether the observations are comparable and whether the process is reasonably stable.
• Look for selection bias, missing groups, measurement changes, and dependence.
• Treat the expected value as a long-run reference, not a promise about the next result.
• Report uncertainty and inspect unusual observations instead of assuming a large sample solves every problem.