**A Bernoulli trial** is a statistical experiment with only two possible outcomes, usually success and failure, with fixed probabilities of occurrence on each trial. It was considered in more detail in <a href='https://codefinity.com/courses/v2/d0f462b1-cab1-4c91-be20-5af852b3ae2d/47c8ed72-9c63-49a9-9972-63a439ed63df/14117fc2-7e30-4838-83a9-6edf8dd84500' target='_blank'>the Probability Theory Basics course</a>.

In a Bernoulli process, each trial is independent, meaning the outcome of one trial does not affect the outcome of any other trial. The probability of success, denoted by p, is the same for every trial. The probability of failure is indicated by q = 1 - p. 
Let's try to apply the law of large numbers to this scheme. Assume that we provide n experiments and want to calculate the total number of successful results. According to the law of large numbers law, we can do it as follows:

Each variable in the numerator represents the **result of one experiment**: if the experiment is successful, this variable equals 1 ( with the corresponding probability of success p), and if the experiment is not successful, this variable equals 0 with probability 1-p. It is obvious that **the conditions of the law of large numbers are satisfied in this case**: all these variables are independent (since the experiments are carried out independently), equally distributed, and have a finite distribution (this can be seen from the distribution series).   
Thus, using the law of large numbers, we can estimate **the probabilities of an event's occurrence** by analyzing **the occurrence frequency of an event**.   
Let's look at an example: suppose we flip a deformed coin with a displaced center of gravity (head and tail probabilities are different). Our task is to estimate the probability of the head falling out. Look at the code below: 

import numpy as np
import matplotlib.pyplot as plt
# Set the probability of heads to 0.3
p = 0.3

# Generate 2000 flips of the coin with probability of heads equal to p
coin_flips = np.random.choice([1, 0], size=2000, p=[p, 1-p])
# Function that will calculate mean value of subsamples
def mean_value(data, subsample_size):
  return data[:subsample_size].mean()

# Visualizing the results
x = np.arange(2000)
y = np.zeros(2000)
for i in range(1, 2000):
  y[i] = mean_value(coin_flips, x[i])

plt.plot(x, y, label='Estimated probability')
plt.xlabel('Number of elements to calculate probability')
plt.ylabel('Probability of success')
plt.axhline(y=p, color='k', label='Real probability of success')
plt.legend()
plt.show()

Similarly, the law of large numbers can be generalized for a **polynomial scheme**: for 1, we consider the occurrence of the event/events of interest to us, and for 0, all other results. Let's look at an example:

import numpy as np
import matplotlib.pyplot as plt

# Our distribution with 4 possible values
outcomes = ['Red', 'Blue', 'Black', 'Green']
# Probabilities of corresponding values
probs = [0.3, 0.2, 0.4, 0.1]

# Generate samples
samples = np.random.choice(outcomes, size=2000, p=probs)

# Suppose we want to determine the probability of occurrence of red or black colors. 
# Let's transform the data in such a way that 1 stands in place of 'Red' and 'Black' colors, 
# and 0 in place of other colors
encoded_samples = np.where(np.logical_or(samples == 'Red', samples == 'Black'), 1, 0)

# Function that will calculate mean value of subsamples
def mean_value(data, subsample_size):
  return data[:subsample_size].mean()

# Visualizing the results
x = np.arange(2000)
y = np.zeros(2000)
for i in range(1, 2000):
  y[i] = mean_value(encoded_samples, x[i])

plt.plot(x, y, label='Estimated probability')
plt.xlabel('Number of elements to calculate probability')
plt.ylabel('Probability of success')
plt.axhline(y=probs[0]+probs[2], color='k', label='Real probability of success')
plt.legend()
plt.show()

Can we say that the frequency of occurrence of a certain event can be interpreted as the probability of this event if the frequency is calculated from a fairly large number of experiments?

Now we will understand some fundamental theoretical concepts which are used in solving real live tasks: absolutely continuous and discrete random variables, probability density function, cumulative distribution function, the characteristics of a random variable, etc.

The limit theorems of probability theory are fundamental laws of probability theory that are often used in practice in a wide variety of areas, such as: building confidence intervals, estimating distribution parameters, providing A/B testings, creating ensembles of ML models, etc. Now we will consider two of the most commonly used: the Law of Large Numbers and the Central Limit Theorem.

When we work with real data we usually do not know from which distribution this data was obtained. In order to determine this, we must be able to correctly estimate the parameters of this distribution and the type of distribution, which we will learn to do in this section.

We have already learned how to estimate the parameters of the population. But to estimate the parameter, we make an assumption about the population distribution. Can we say that our assumption is correct? How do we prove that the estimated parameters are the real parameters of the population? Can we show that two sets of samples are independent?  To answer these questions, it is necessary to consider the concept of hypothesis testing.

Probability Theory Mastering

Law of Large Numbers for Bernoulli Process

Can we say that the frequency of occurrence of a certain event can be interpreted as the probability of this event if the frequency is calculated from a fairly large number of experiments?