Bayes' Law Primer

I'm currently writing a blog post that uses Bayes' Law but don't want to muddy the post with a review in layman's terms. So I have something to link, here is a short description and a chance to flex my teaching muscles before the school year starts.

Bayes' Law

For those who aren't sure, Bayes' Law tells us that the probability event $X$ occurs given we know that event $Y$ has occurred can easily be computed. It is written as $\text{Pr}(X \mid Y)$ and the vertical bar is meant like the word "given", in other words, the event $X$ is distinct from the event $X \mid Y$ ($X$ given $Y$). Bayes' law, states that
$$\text{Pr}(X \mid Y) = \frac{\text{Pr}(X \text{ and } Y \text{ both occur})}{\text{Pr}(Y)}.$$
This effectively is a re-scaling of the events by the total probability of the given event: $\text{Pr}(Y)$.

For example, if $X$ is the event that a $3$ is rolled on a fair die and $Y$ is the event that the roll is odd. We know of course that $\text{Pr}(Y) = \frac{1}{2}$ since half of the rolls are odd. The event $X \text{ and } Y \text{ both occur}$ in this case is the same as $X$ since the roll can only be $3$ is the roll is odd. Thus
$$\text{Pr}(X \text{ and } Y \text{ both occur}) = \frac{1}{6}$$
and we can compute the conditional probability
$$\text{Pr}(X \mid Y) = \frac{\frac{1}{6}}{\frac{1}{2}} = \frac{1}{3}.$$
As we expect, one out of every three odd rolls is a $3$.

Bayes' Law Extended Form

Instead of considering a single event $Y$, we can consider a range of $n$ possible events $Y_1, Y_2, \ldots, Y_n$ that may occur. We require that one of these $Y$-events must occur and that they cover all possible events that could occur. For example $Y_1$ is the event that H2O is vapor, $Y_2$ is the event that H2O is water and$Y_3$ is the event that H2O is ice.

In such a case we know that since the $Y$-events are distinct
$$\text{Pr}(X) = \text{Pr}(X \text{ and } Y_1 \text{ both occur}) + \text{Pr}(X \text{ and } Y_2 \text{ both occur}) + \text{Pr}(X \text{ and } Y_3 \text{ both occur}).$$
Using Bayes' law, we can reinterpret as
$$\text{Pr}(X \text{ and } Y_j \text{ both occur}) =  \text{Pr}(X \mid Y_j) \cdot \text{Pr}(Y_j)$$
and the above becomes
$$\text{Pr}(X) = \text{Pr}(X \mid Y_1) \cdot \text{Pr}(Y_1) + \text{Pr}(X \mid Y_2) \cdot \text{Pr}(Y_2) + \text{Pr}(X \mid Y_3) \cdot \text{Pr}(Y_3).$$
The same is true if we replace $3$ with an arbitrary number of events $n$.
About Bossy Lobster