1.9 Bayes’ Theorem
Now that we understand the various probabilities, we can use this knowledge to understand and apply Bayes' theorem (named after the Reverend Thomas Bayes). This theorem has a central place in the probabilistic theory and gives us a completely new perspective to think about probability.
Bayes’ theorem provides a structured way to convert a conditional probability from one direction, say (|), to the other direction, (|). When dealing with decision-making or scientific inference, it allows us to update our beliefs in the light of new evidence. Mathematically, Bayes’ theorem can be denoted as:
, provided that () > 0
|
Bayes’ Theorem Proof
From conditional probability, we know:
, provided that () > 0 (1.19)
, provided that () > 0 (1.20)
From the symmetry of intersection:
()
From 1.19 and 1.20, we get:
(1.21)
Substituting 1.21 into 1.19 yields:
|
While the mathematical equation of Bayes' theorem applies to any probability distribution involving events A and B, it takes on a particularly insightful interpretation when A represents a hypothesis H, and B represents some observed evidence E. In this scenario, the formula can be rewritten as follows:
, provided that () > 0
Where:
(|) is the probability of the hypothesis being true given the evidence .
(|) is the probability of the evidence occurring given that the hypothesis is true.
() is the prior probability of the hypothesis being true (before considering the evidence).
() is the total probability of evidence occurring.
This formula gives us a way of relating the probability of the hypothesis before getting the evidence, denoted as (), to the probability of the hypothesis after incorporating the evidence, denoted as (|). Thus, () is referred to as the prior probability, while (|) is termed the posterior probability. The connecting factor between these two probabilities, expressed as (|)/(), is called the likelihood ratio. Using this terminology, Bayes' theorem can be rephrased as "the posterior probability equals the prior probability multiplied by the likelihood ratio."
Now that we know the definition and mathematical formula of Bayes’ theorem, let's spend some time to understand the intuition behind it.