How Bayes's Theorem Updates Probabilities With New Evidence

This article provides a straightforward guide to how Bayes's theorem mathematically updates the probability of a hypothesis as new evidence emerges. It breaks down the core components of the Bayesian formula, explains the step-by-step mechanics of shifting from initial beliefs to updated conclusions, and demonstrates this continuous learning process through a practical calculation.

The Core Formula

In probability theory, Bayes's theorem provides a mathematical rule for inverting conditional probabilities. It allows you to find the probability of a cause given its effect. The formula is written as:

P(H|E) = [P(E|H) × P(H)] / P(E)

To understand how the theorem updates information, you must understand its four distinct components:

  • P(H) is the Prior Probability: This is your initial belief about the probability of the hypothesis (H) being true, before you account for the new evidence.
  • P(E|H) is the Likelihood: This is the probability of observing the new evidence (E) assuming that your initial hypothesis (H) is true.
  • P(E) is the Marginal Likelihood: This is the total probability of observing the evidence (E) under all possible scenarios, whether the hypothesis is true or false.
  • P(H|E) is the Posterior Probability: This is the updated probability of the hypothesis (H) being true, calculated after the new evidence (E) is taken into account.

The Mechanism of Updating

Bayes's theorem functions as a mathematical engine that transforms a Prior Probability into a Posterior Probability.

You start with a baseline assumption, P(H). When new data arrives, you do not throw away your baseline assumption; instead, you weight it against the new data. The theorem does this by multiplying the prior probability by a scaling factor, which is P(E|H) / P(E).

This scaling factor measures the strength of the new evidence:

  • If the evidence is much more likely to occur when the hypothesis is true than under any other circumstance, the scaling factor will be greater than 1. Multiplying your prior by this number increases the probability, resulting in a higher posterior probability.
  • If the evidence is unlikely to occur when the hypothesis is true, the scaling factor will be less than 1. This reduces the initial probability, resulting in a lower posterior probability.
  • If the evidence is equally likely to occur regardless of whether the hypothesis is true, the scaling factor equals 1. The evidence is uninformative, and the posterior probability remains identical to the prior probability.

The Iterative Process

The true power of Bayes's theorem lies in its continuous, iterative nature. Probability updating is not a one-time event.

Once you calculate the Posterior Probability, P(H|E), that number becomes your new Prior Probability, P(H), for the next calculation. When a second piece of evidence arrives, you plug your new prior into the exact same formula to generate a second posterior. With each new piece of valid evidence, the probability shifts closer to the objective truth, gradually minimizing the influence of your original, potentially flawed starting assumption.

A Step-by-Step Mathematical Example

Imagine a factory that uses two machines to produce widgets. You want to know the probability that a defective widget came from Machine A (your hypothesis, H).

1. Establish the Prior, P(H) Machine A produces 60% of the factory's widgets, and Machine B produces 40%. Without looking at the widget, the initial probability that any given widget came from Machine A is 60%.

  • P(H) = 0.60

2. Introduce New Evidence and Determine the Likelihood, P(E|H) You pick up a widget and see that it is defective. This defect is your new evidence (E). Historical data shows that Machine A has a 2% defect rate, while Machine B has a 5% defect rate. Therefore, the probability of a widget being defective, given it came from Machine A, is 2%.

  • P(E|H) = 0.02

3. Calculate the Marginal Likelihood, P(E) You must calculate the total probability of finding a defective widget from the factory floor as a whole. You do this by adding the defects from Machine A and Machine B together.

  • Defects from Machine A: 0.60 (production share) × 0.02 (defect rate) = 0.012
  • Defects from Machine B: 0.40 (production share) × 0.05 (defect rate) = 0.020
  • P(E) = 0.012 + 0.020 = 0.032

4. Calculate the Updated Posterior Probability, P(H|E) Now, plug the numbers into Bayes's theorem to update your initial 60% assumption based on the evidence that the widget is defective:

  • P(H|E) = (0.02 × 0.60) / 0.032
  • P(H|E) = 0.012 / 0.032
  • P(H|E) = 0.375

By incorporating the new evidence (the defect), Bayes's theorem mathematically updates the probability that the widget came from Machine A from 60% down to 37.5%. Because Machine B has a higher defect rate, the presence of a defect shifts the probability away from Machine A. If you discover a second flaw on the same widget that is specific to one of the machines, 37.5% becomes your new prior, and you repeat the calculation.