Probability Theory Basics

1. Basic Concepts of Probability Theory

We will start our way of learning probability theory by considering some basic definitions and rules: what is a stochastic experiment and random event, what is independence and incompatibility of events in the context of probability theory, what is the probability and how can we calculate probabilities of different elementary events.

Stochastic Experiment and Random Event

Probability and It's Properties

Geometrical Probability

Challenge: Solving the Task Using Geometric Probability

Independence and Incompatibility of Random Events

Conditional Probability

2. Probability of Complex Events

In real-life tasks, we often have to deal with complex relationships and, as a result, calculate probabilities of several events or events that depend on each other. Let's consider how we can do this using probability theory.

Inclusion-Exclusion Principle

Challenge: Solving the Task Using Inclusion-Exclusion Principle

The Multiplication Rule of Probability

Law of Total Probability

Bayes' Theorem

Challenge: Solving the Task Using Bayes' Theorem

3. Commonly Used Discrete Distributions

To solve many real problems in probability theory, special models have been created that describe a particular situation. Let's consider some of the most used models that can be used to describe some discrete results of stochastic experiments.

Binomial Distribution

Challenge: Solving Task Using Binomial Distribution

Multinomial Distribution

Geometric Distribution

Poisson Distribution

Challenge: Solving Task Using Poisson Distribution

4. Commonly Used Continuous Distributions

What if the result of a stochastic experiment cannot be described by a discrete value? For this, models that work with continuous values are used. Consider the most popular of these models.

Continuous Uniform Distribution

Exponential Distribution

Challenge: Solving Task Using Exponential Distribution

Gaussian Distribution

Challenge: Solving Task Using Gaussian Distribution

5. Covariance and Correlation

Often we are faced with the task of checking the dependence of the results of different stochastic experiments on each other. Moreover, it is necessary not only to assess the presence of dependencies but also to somehow quantify the degree of dependencies. To solve these problems, we can use covariance and correlation.

What is Covariance?

What is Correlation?

Challenge: Solving the Task Using Correlation

Conditional Probability

Conditional probability is the probability of an event occurring, given that another event has already occurred. It represents the updated probability based on the knowledge or information about the occurrence of another event.
The conditional probability of A given B is defined as follows:

If events A and B are independent, then
P(A intersection B) = P(A)*P(B),
and as a result, conditional probability P(A|B)=P(A).

Note
It makes sense to introduce a conditional probability only if P(B) is not equal to zero.

Let's look at the example.

Assume that there are 5 children in the family, and at least one is a girl. Calculate the probability that the eldest child is a boy assuming that each child can be a girl or a boy with equal probability.

Due to this assumption, we can use the classic definition of probability and calculate the corresponding probability using conditional probability.
Let event A is that the eldest child is a boy. Event B is that there is at least one girl. We can solve this task as follows: