Learn Bayes' Theorem

Swipe to show menu

Bayes' theorem is a fundamental result in probability theory that allows you to update your beliefs about the likelihood of an event based on new information. It is especially useful when you want to revise the probability of a hypothesis in light of observed evidence.

The theorem is derived from the definition of conditional probability. If you have two events, $A$ and $B$ , the probability of $A$ given $B$ is:

P(A | B) = \frac{P(A \text{ and } B)}{P(B)}

Similarly, the probability of $B$ given $A$ is:

P(B | A) = \frac{P(A \text{ and } B)}{P(A)}

Since $P(A \text{ and } B) = P(B \text{ and } A)$ , you can rearrange these equations to get:

P(A | B) = \frac{P(B | A) * P(A)}{P(B)}

This is Bayes' theorem.

You can use Bayes' theorem in many practical situations. Suppose a disease affects 1% of a population. There is a test for the disease that is 99% accurate:

If you have the disease, it gives a positive result 99% of the time (true positive rate);
If you do not have the disease, it gives a negative result 99% of the time (true negative rate).

If a person tests positive, what is the probability that they actually have the disease?

Let $D$ be the event "person has the disease" and $T$ be the event "test is positive." You know:

$P(D) = 0.01$ (prior probability of disease);
$P(\text{not } D) = 0.99$ ;
$P(T | D) = 0.99$ (probability of a positive test given disease);
$P(T | \text{not } D) = 0.01$ (probability of a positive test given no disease).

You want to find $P(D | T)$ , the probability the person has the disease given a positive test result. By Bayes' theorem:

P(D | T) = \frac{[P(T | D) * P(D)]}{P(T)}

To find $P(T)$ , use the law of total probability:

P(T) = P(T | D) * P(D) + P(T | \text{not } D) * P(\text{not } D)=\\ = 0.99 * 0.01 + 0.01 * 0.99=\\ = 0.0099 + 0.0099 =\\= 0.0198

So,

P(D | T) = \frac{0.99 * 0.01}{0.0198} \approx 0.5

Even with a positive test, the chance the person actually has the disease is only about 50%, due to the low prevalence of the disease in the population. This illustrates how Bayes' theorem helps you interpret evidence in the context of prior probabilities.


              123456789101112
            
# Bayes' Theorem: Medical Testing
p_d = 0.01        # Prior: P(Disease)
p_pos_d = 0.99    # Sensitivity: P(Positive | Disease)
p_pos_nd = 0.01   # False Positive: P(Positive | No Disease)

# Law of Total Probability: P(Positive)
p_pos = (p_pos_d * p_d) + (p_pos_nd * (1 - p_d))

# Posterior: P(Disease | Positive)
p_d_pos = (p_pos_d * p_d) / p_pos

print(f"P(Disease | Positive test): {p_d_pos:.4f}")

Everything was clear?

Thanks for your feedback!

Section 1. Chapter 7

Ask AI

Ask anything or try one of the suggested questions to begin our chat

Section 1. Chapter 7