When a 99% Accurate Alert Is Only Half Right

The misconception

Suppose a detector catches 99% of defective components and correctly clears 99% of nondefective components. When an alert appears, a quick answer may be that the component has a 99% chance of being defective. That reverses the condition.

The 99% sensitivity is the probability of an alert given that a component is defective. The question after an alert is the probability of a defect given that an alert occurred. Those two conditional probabilities are not generally equal. To reverse the condition, we also need to know how common defects are before the detector is used.

This error is called base-rate neglect. It appears whenever evidence is interpreted without accounting for the starting frequency of the event under consideration.

A synthetic experiment

Imagine 10,000 programmed components. Exactly 1%, or 100 components, are marked defective by the generator. The remaining 9,900 are marked nondefective. No physical components were inspected, and the example makes no claim about any real production process.

Apply a detector with 99% sensitivity. It alerts on 99 of the 100 defective components and misses one. Give it 99% specificity as well. It correctly clears 9,801 of the 9,900 nondefective components, but alerts on the remaining 99.

The alert column now contains 198 components: 99 true alerts and 99 false alerts. Select one component uniformly from that alert column. Its probability of being defective is 99 / 198, or 50%. Strong performance in both classes has not disappeared. The rare base rate means the large nondefective group supplies as many false alerts as the small defective group supplies true alerts.

The linked experiment lets the reader change prevalence, sensitivity, and specificity. Holding detector performance fixed while moving prevalence from 1% to 10% dramatically increases the share of alerts that correspond to defects. Lowering the false-positive rate can have an equally important effect when the target event is rare.

The interface should show counts before percentages. A grid of 10,000 items or a four-cell frequency table makes each denominator visible and reduces the chance of swapping conditional probabilities.

Derivation and reasoning

Let D mean defective and + mean alert. Bayes’ rule gives:

P(D | +) = P(+ | D) * P(D) / P(+)

The total probability of an alert comes from both possible classes:

P(+) = P(+ | D) * P(D) + P(+ | not D) * P(not D)

Combining these expressions gives:

P(D | +) = sensitivity * prevalence / (sensitivity * prevalence + falsePositiveRate * (1 - prevalence))

For the synthetic example, the numerator is 0.99 * 0.01 = 0.0099. The false-alert contribution is 0.01 * 0.99 = 0.0099. The numerator is therefore half of the denominator.

Specificity and false-positive rate describe complementary outcomes among nondefective components:

falsePositiveRate = 1 - specificity

The quantity calculated after an alert is often called positive predictive value. It is not an intrinsic property of a detector alone. It changes when the population prevalence changes, even if sensitivity and specificity stay fixed.

Natural frequencies reach the same result without symbolic manipulation. Start with an explicit population size, split it by base rate, then apply performance rates inside each branch. This is also a useful audit: the four resulting cells must add back to the total population.

Assumptions

The calculation treats prevalence, sensitivity, and specificity as known constants. Real estimates of these quantities have uncertainty. If they come from a sample, a complete analysis should propagate that uncertainty rather than display the posterior probability as exact.

Detector performance is assumed to transfer to the population where it is used. A detector evaluated under one operating condition may behave differently under another. Changes in equipment, thresholds, materials, or measurement quality can change both sensitivity and specificity.

The example also defines one binary condition and one binary alert. Multi-stage systems, repeated alerts, dependent tests, and multiple defect types require a more detailed probability model. Applying the same detector twice does not necessarily provide independent evidence.

Failure cases

Do not call the detector “99% accurate” without defining the metric. Overall accuracy can be dominated by the common class and may hide poor sensitivity. Reporting the confusion matrix, sensitivity, specificity, prevalence, and predictive value prevents a single flattering number from carrying several meanings.

Do not import a base rate from an unrelated population. The correct prior frequency is tied to a defined target population and time. A convenient published number may not apply to the current process.

Do not interpret the synthetic 50% result as a benchmark for any real alert system. It follows exactly from the invented 1%, 99%, and 99% settings. Change those inputs and the answer changes. The experiment teaches conditional reasoning; it does not validate equipment or recommend an operating threshold.

Source notes

The UC Berkeley statistics text derives conditional probability and Bayes’ rule and discusses base-rate errors. The OpenStax section demonstrates contingency tables as a way to calculate conditional probabilities. This guide uses an original component-count example and brief paraphrase rather than reproducing either source’s scenarios or wording.

Sources checked

These sources support the statistical definitions and boundaries. The scenario, prose, calculations, and figure on this page are original to Count & Chance.

• Probability: Axioms and Fundaments, University of California, Berkeley Department of Statistics. Checked 2026-08-19.
• 3.4 Contingency Tables, OpenStax, Rice University. Checked 2026-08-19.