Summary  
This chapter covers computing principal components by formulating maximum-variance projection as an eigenvalue problem on the covariance matrix and extracting the leading eigenvector.  

General domain of usage  
Dimensionality reduction for data analysis.

PCA seeks a new set of axes, called **principal components**, such that the projected data has **maximum variance**. The first principal component, denoted as $$w_{\raisebox{-0.5pt}{$1$}}$$, is chosen to maximize the variance of the projected data:

$$
\mathrm{Var}(X w_1)
$$

Subject to the constraint that $$\|w_{\raisebox{-0.5pt}{$1$}}\| = 1$$. The solution to this maximization problem is the **eigenvector** of the covariance matrix corresponding to the largest eigenvalue.

The optimization problem is:

$$
\max_{w} \ w^T \Sigma w \quad \text{subject to} \quad \|w\| = 1
$$

The solution is any vector $$w$$ that satisfies $$\Sigma w = \lambda w$$, where $$\lambda$$ is the corresponding eigenvalue. In other words, $$w$$ is an **eigenvector** of the covariance matrix $$\Sigma$$ associated with eigenvalue $$\lambda$$.

import numpy as np

# Assume cov_matrix from earlier
X = np.array([[2.5, 2.4],
              [0.5, 0.7],
              [2.2, 2.9]])
X_centered = X - np.mean(X, axis=0)
cov_matrix = (X_centered.T @ X_centered) / X_centered.shape[0]

# Find the principal component (eigenvector with largest eigenvalue)
values, vectors = np.linalg.eig(cov_matrix)
principal_component = vectors[:, np.argmax(values)]
print("First principal component:", principal_component)

**This principal component** is the direction along which the data has the highest variance. Projecting data onto this direction gives the most informative **one-dimensional representation** of the original dataset.

Which statement best describes the role of the covariance matrix in the derivation of PCA using linear algebra

A comprehensive intermediate course guiding learners through the motivation, mathematical foundations, and practical implementation of Principal Component Analysis (PCA) for dimensionality reduction in data science and machine learning.

Explore the motivation, challenges, and benefits of reducing data dimensions in machine learning and data science.

Delve into the mathematical concepts that underpin PCA, including variance, covariance, and eigenvectors.

Apply PCA to real datasets using Python, interpret the results, visualize explained variance and component loadings, and compare model performance before and after PCA.

Derivation of PCA Using Linear Algebra