Summary  
The chapter explains how to derive principal components by formulating a variance-maximization problem under a unit-norm constraint and solving it as an eigenvalue problem on the covariance matrix. It then shows how to implement code that computes the covariance matrix, performs eigen decomposition, and selects the eigenvector with the largest eigenvalue as the first principal component.

General domain of usage  
Dimensionality reduction in machine learning

PCA seeks a new set of axes, called **principal components**, such that the projected data has **maximum variance**. The first principal component, denoted as $$w_{\raisebox{-0.5pt}{$1$}}$$, is chosen to maximize the variance of the projected data:

$$
\mathrm{Var}(X w_1)
$$

Subject to the constraint that $$\|w_{\raisebox{-0.5pt}{$1$}}\| = 1$$. The solution to this maximization problem is the **eigenvector** of the covariance matrix corresponding to the largest eigenvalue.

The optimization problem is:

$$
\max_{w} \ w^T \Sigma w \quad \text{subject to} \quad \|w\| = 1
$$

The solution is any vector $$w$$ that satisfies $$\Sigma w = \lambda w$$, where $$\lambda$$ is the corresponding eigenvalue. In other words, $$w$$ is an **eigenvector** of the covariance matrix $$\Sigma$$ associated with eigenvalue $$\lambda$$.

import numpy as np

# Assume cov_matrix from earlier
X = np.array([[2.5, 2.4],
              [0.5, 0.7],
              [2.2, 2.9]])
X_centered = X - np.mean(X, axis=0)
cov_matrix = (X_centered.T @ X_centered) / X_centered.shape[0]

# Find the principal component (eigenvector with largest eigenvalue)
values, vectors = np.linalg.eig(cov_matrix)
principal_component = vectors[:, np.argmax(values)]
print("First principal component:", principal_component)

**This principal component** is the direction along which the data has the highest variance. Projecting data onto this direction gives the most informative **one-dimensional representation** of the original dataset.

Which statement best describes the role of the covariance matrix in the derivation of PCA using linear algebra

A comprehensive intermediate course guiding learners through the motivation, mathematical foundations, and practical implementation of Principal Component Analysis (PCA) for dimensionality reduction in data science and machine learning.

Derivation of PCA Using Linear Algebra