Summary  
This chapter covers computing variance and covariance and implementing code to generate a covariance matrix by centering data and performing matrix multiplication.

General domain of usage  
Statistical data analysis

**Variance** measures how much a variable deviates from its mean.

Definition

The formula for **variance** of a variable $$x$$ is:

$$
\mathrm{Var}(x) = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2
$$

**Covariance** measures how two variables change together. 

The formula for **Covariance** of variables $$x$$ and $$y$$ is:

$$
\mathrm{Cov}(x, y) = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})
$$

The **covariance matrix** generalizes covariance to multiple variables. For a dataset $$X$$ with $$d$$ features and $$n$$ samples, the covariance matrix $$\Sigma$$ is a $$d \times d$$ matrix where each entry $$\Sigma_{ij}$$ is the covariance between feature $$i$$ and feature $$j$$, computed with denominator $$n-1$$ to be an unbiased estimator.

import numpy as np

# Example data: 3 samples, 2 features
X = np.array([[2.5, 2.4],
              [0.5, 0.7],
              [2.2, 2.9]])

# Center the data (subtract mean)
X_centered = X - np.mean(X, axis=0)

# Compute covariance matrix manually
cov_matrix = (X_centered.T @ X_centered) / X_centered.shape[0]
print("Covariance matrix:\n", cov_matrix)

In the code above, you manually center the data and compute the covariance matrix using matrix multiplication. This matrix captures how each pair of features varies together.

Which statement accurately describes the relationship between variance, covariance, and the covariance matrix

A comprehensive intermediate course guiding learners through the motivation, mathematical foundations, and practical implementation of Principal Component Analysis (PCA) for dimensionality reduction in data science and machine learning.

Variance, Covariance, and the Covariance Matrix