1.8 Dependent Events and Conditional Probability
When the outcome of the previous event affects the outcome of the subsequent event, they are called Dependent events. Say, drawing a queen card from a deck of playing cards after removing one card (and not putting it back) is a dependent event. Dependent events can lead to Conditional Probability.
Conditional probability is one of the most fundamental concepts in probability and machine learning because it allows us to understand the relationship between two or more events. If events are correlated, the occurrence of one event can influence the probability of another event. It addresses the question “What is the probability of an event occurring, given that another event has already occurred?”.
When we condition an event A on another event B, we enter the universe where B has already occurred. It implies that the only outcomes now possible are the ones that are part of B. In other words, the sample space gets reduced to B (Fig 1.18).
Mathematically, the probability of A given that (aka conditioned on) B already happened is defined as:
Provided that () > 0
The terms of the formula can be inferred as follows:
- ( | ) is the probability of event , given event has already occurred
- ( ) is the probability that both events and will occur (i.e., Intersection of and )
- () is the probability that event will occur
Similarly, the probability of B given that (aka conditioned on) A already happened is defined as:
Provided that () > 0
Example 1.30. Consider a standard deck of 52 playing cards, from which we have drawn a black card. What is the probability that the drawn card is a king?
Let's begin by identifying which event has already taken place and which event we need to determine. In this context, the event that has already occurred is the drawing of a black card, which can be denoted as (). Our objective is to calculate the probability of the drawn card being a king, given the condition that it's black, denoted as ( | ).
As it is evident that there is a condition involved in this example, we can use conditional probability to find the required probability. This can be denoted as:
(1.16)
Where:
- ( | ) is the probability of drawing a king card, given a black card has already been drawn
- ( ) is the probability of drawing a card that is both a king and black
- () is the probability of drawing a black card
For intuitive understanding, we can represent 1.16 visually as:
The probability of drawing a black card can be calculated as:
(1.17)
The probability of drawing a card that is both a king and black:
(1.18)
Substituting 1.17 and 1.18 into 1.16 yields:
Thus, the probability that the drawn black card is a king is 7.69%.
It’s important to note that ( | ) is not equal to ( | ). Say, the probability of a pet being cute given that it’s a kitten i.e., ( | ) is pretty high, assuming kittens are generally cute. However, this doesn't imply that any cute pet has a high probability of being a kitten only i.e., ( | ). Puppies and Hamsters are also cute :).
But what if we have ( | ) and we want to determine ( | )? This is where
Bayes' theorem comes into play. In the next section, we discuss this fundamental theorem.