Summary  
This chapter explains how to calculate variance and covariance for one or more variables and how to construct a covariance matrix by centering data and using matrix operations.  

General domain of usage  
Statistical data analysis

**Variance** measures how much a variable deviates from its mean.

Definition

The formula for **variance** of a variable $$x$$ is:

$$
\mathrm{Var}(x) = \frac{1}{n} \sum_{i=1}^n (x_i - \bar{x})^2
$$

**Covariance** measures how two variables change together. 

The formula for **Covariance** of variables $$x$$ and $$y$$ is:

$$
\mathrm{Cov}(x, y) = \frac{1}{n-1} \sum_{i=1}^n (x_i - \bar{x})(y_i - \bar{y})
$$

The **covariance matrix** generalizes covariance to multiple variables. For a dataset $$X$$ with $$d$$ features and $$n$$ samples, the covariance matrix $$\Sigma$$ is a $$d \times d$$ matrix where each entry $$\Sigma_{ij}$$ is the covariance between feature $$i$$ and feature $$j$$, computed with denominator $$n-1$$ to be an unbiased estimator.

import numpy as np

# Example data: 3 samples, 2 features
X = np.array([[2.5, 2.4],
              [0.5, 0.7],
              [2.2, 2.9]])

# Center the data (subtract mean)
X_centered = X - np.mean(X, axis=0)

# Compute covariance matrix manually
cov_matrix = (X_centered.T @ X_centered) / X_centered.shape[0]
print("Covariance matrix:\n", cov_matrix)

In the code above, you manually center the data and compute the covariance matrix using matrix multiplication. This matrix captures how each pair of features varies together.

Which statement accurately describes the relationship between variance, covariance, and the covariance matrix

A comprehensive intermediate course guiding learners through the motivation, mathematical foundations, and practical implementation of Principal Component Analysis (PCA) for dimensionality reduction in data science and machine learning.

Explore the motivation, challenges, and benefits of reducing data dimensions in machine learning and data science.

Delve into the mathematical concepts that underpin PCA, including variance, covariance, and eigenvectors.

Apply PCA to real datasets using Python, interpret the results, visualize explained variance and component loadings, and compare model performance before and after PCA.

Variance, Covariance, and the Covariance Matrix