Bayes' Law Primer

I'm currently writing a blog post that uses Bayes' Law but don't want to muddy the post with a review in layman's terms. So I have something to link, here is a short description and a chance to flex my teaching muscles before the school year starts.

Bayes' Law

For those who aren't sure, Bayes' Law tells us that the probability event \(X\) occurs given we know that event \(Y\) has occurred can easily be computed. It is written as \(\text{Pr}(X \mid Y)\) and the vertical bar is meant like the word "given", in other words, the event \(X\) is distinct from the event \(X \mid Y\) (\(X\) given \(Y\)). Bayes' law, states that
\[\text{Pr}(X \mid Y) = \frac{\text{Pr}(X \text{ and } Y \text{ both occur})}{\text{Pr}(Y)}.\]
This effectively is a re-scaling of the events by the total probability of the given event: \(\text{Pr}(Y)\).

For example, if \(X\) is the event that a \(3\) is rolled on a fair die and \(Y\) is the event that the roll is odd. We know of course that \(\text{Pr}(Y) = \frac{1}{2}\) since half of the rolls are odd. The event \(X \text{ and } Y \text{ both occur}\) in this case is the same as \(X\) since the roll can only be \(3\) is the roll is odd. Thus
\[\text{Pr}(X \text{ and } Y \text{ both occur}) = \frac{1}{6}\]
and we can compute the conditional probability
\[\text{Pr}(X \mid Y) = \frac{\frac{1}{6}}{\frac{1}{2}} = \frac{1}{3}.\]
As we expect, one out of every three odd rolls is a \(3\).

Bayes' Law Extended Form

Instead of considering a single event \(Y\), we can consider a range of \(n\) possible events \(Y_1, Y_2, \ldots, Y_n\) that may occur. We require that one of these \(Y\)-events must occur and that they cover all possible events that could occur. For example \(Y_1\) is the event that H2O is vapor, \(Y_2\) is the event that H2O is water and\(Y_3\) is the event that H2O is ice.

In such a case we know that since the \(Y\)-events are distinct
\[\text{Pr}(X) = \text{Pr}(X \text{ and } Y_1 \text{ both occur}) + \text{Pr}(X \text{ and } Y_2 \text{ both occur}) + \text{Pr}(X \text{ and } Y_3 \text{ both occur}).\]
Using Bayes' law, we can reinterpret as
\[\text{Pr}(X \text{ and } Y_j \text{ both occur}) =  \text{Pr}(X \mid Y_j) \cdot \text{Pr}(Y_j)\]
and the above becomes
\[\text{Pr}(X) = \text{Pr}(X \mid Y_1) \cdot \text{Pr}(Y_1) + \text{Pr}(X \mid Y_2) \cdot \text{Pr}(Y_2) + \text{Pr}(X \mid Y_3) \cdot \text{Pr}(Y_3).\]
The same is true if we replace \(3\) with an arbitrary number of events \(n\).
About Bossy Lobster