When a 99% Accurate Alert Is Only Half Right
Even a strong detector can produce a surprising share of false alerts when the event it seeks is rare. The missing ingredient is the base rate.
Read the guideTwelve field guides
Each guide starts with a common misconception, works through a synthetic example, and ends with the boundary conditions that make the result useful.
Definitions are checked against primary teaching or standards sources. Examples, calculations, figures, and prose are created for this publication.
12 items shown
Even a strong detector can produce a surprising share of false alerts when the event it seeks is rare. The missing ingredient is the base rate.
Read the guideA combined rate can reverse the comparison inside every subgroup when the groups appear in different proportions.
Read the guideA skewed population can produce nearly normal averages. The change belongs to repeated sampling, not to the original observations.
Read the guideMore responses can tighten an estimate around the wrong answer when the people who can respond differ from the people you want to describe.
Read the guideFrom 4% to 6% is an increase of two percentage points and a relative increase of 50%. Both are correct, but they answer different questions.
Read the guidePositive, negative, near-zero, and nonlinear patterns show what Pearson's coefficient summarizes and what only a scatterplot reveals.
Read the guideA single delayed delivery can pull the mean away from every ordinary experience. Learn what mean, median, and trimmed mean actually summarize.
Read the guideA narrow vertical scale can reveal small changes or visually magnify them. The right choice depends on the graphical encoding and honest context.
Read the guideAt a 5% threshold, testing twenty independent null effects creates about a 64% chance of at least one false alarm. A small p-value needs its full search context.
Read the guideA confidence interval is a procedure with a long-run success rate. Watch many intervals succeed and a few fail even when every calculation is correct.
Read the guideSelecting the highest first scores also selects unusually favorable noise. A second measurement often looks less extreme without any intervention.
Read the guideFair coin sequences naturally produce clusters and long runs. A pattern can look designed without changing the probability of the next independent outcome.
Read the guideNo guides match this topic.