Understanding Conditional Probability & Bayes' Theorem

Conditional Probability

Conditional probability measures the chance of an event happening given that another event has already occurred.

Formula:

where:

• $P(A \mid B)$ means "the probability of A given B";
• $P(A \cap B)$ is the probability that both A and B happen;
• $P(B)$ is the probability that B happens (must be > 0).

Example 1: Conditional Probability — Weather and Traffic

Suppose:

• Event A: "I am late to work";
• Event B: "It is raining".

Given:

• $P(A \cap B) = 0.10$ (10% chance it rains AND I am late);
• $P(B) = 0.20$ (20% chance it rains on any day).

Then:

Interpretation:
If it is raining, there's a 50% chance I will be late to work.

Bayes' Theorem

Bayes' Theorem helps us find $P(A \mid B)$ when it's hard to measure directly, by relating it to $P(B \mid A)$.

Formula:

Step-by-Step Breakdown

Step 1: Understanding $P(A \mid B)$
This reads as "the probability of A given B".

Example: If A = "having a disease" and B = "testing positive", then $P(A \mid B)$ asks:
Given a positive test, what are the chances the person actually has the disease?

Step 2: Numerator = $P(B \mid A) \cdot P(A)$

• $P(B \mid A)$ = probability of testing positive if you have the disease (test sensitivity);
• $P(A)$ = prior probability of A (disease prevalence).

Step 3: Denominator = $P(B)$
This is the total probability that B happens (testing positive), from both true positives and false positives.

Expanded:

Where:

• $P(B \mid \neg A)$ = false positive rate;
• $P(\neg A)$ = probability of not having the disease.

Bayes' Theorem — Medical Test

Suppose:

• Event A: "Having a disease";
• Event B: "Testing positive".

Given:

• Disease prevalence: $P(A) = 0.01$;
• Sensitivity: $P(B \mid A) = 0.99$;
• False positive rate: $P(B \mid \neg A) = 0.05$.

Step 1: Calculate total probability of testing positive

Step 2: Apply Bayes' Theorem

Interpretation:
Even if you test positive, there's only about a 16.7% chance you actually have the disease — because the disease is rare and there are false positives.

Key Takeaways

• Conditional probability finds the chance of A happening when we know B has occurred;
• Bayes' Theorem flips conditional probabilities, letting us update beliefs when direct measurement is hard;
• Both concepts are essential in data science, machine learning, medical testing, and decision-making.