Leer Comparing Model Performance Before and After PCA

Veeg om het menu te tonen

PCA can be used as a preprocessing step before training machine learning models. In this chapter, you will compare the performance of a LogisticRegression classifier on the original standardized data and on data reduced to two principal components. This practical approach highlights how dimensionality reduction can impact both the effectiveness and efficiency of your models.


              1234567891011121314151617181920212223242526272829
            
from sklearn.datasets import load_iris
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score

data = load_iris()
X = data.data
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

X_train, X_test, y_train, y_test = train_test_split(X_scaled, data.target, test_size=0.3, random_state=42)

clf_orig = LogisticRegression(max_iter=200)
clf_orig.fit(X_train, y_train)
y_pred_orig = clf_orig.predict(X_test)
acc_orig = accuracy_score(y_test, y_pred_orig)

pca = PCA(n_components=2)
X_train_pca = pca.fit_transform(X_train)
X_test_pca = pca.transform(X_test)
clf_pca = LogisticRegression(max_iter=200)
clf_pca.fit(X_train_pca, y_train)
y_pred_pca = clf_pca.predict(X_test_pca)
acc_pca = accuracy_score(y_test, y_pred_pca)

print(f"Accuracy on original data: {acc_orig:.2f}")
print(f"Accuracy after PCA (2 components): {acc_pca:.2f}")

The code above splits the data, trains a logistic regression model on both the original and PCA-reduced data, and compares their accuracies. Notice that a perfect accuracy of 1.0 on the original data may indicate overfitting, where the model fits the training data too closely and may not generalize well. Applying PCA reduces dimensionality, which can help mitigate overfitting. After PCA, accuracy drops slightly to 0.91, showing a better balance between performance and generalization, with increased speed and interpretability.

Was alles duidelijk?

Bedankt voor je feedback!

Sectie 1. Hoofdstuk 12

Vraag AI

Vraag wat u wilt of probeer een van de voorgestelde vragen om onze chat te starten.

Sectie 1. Hoofdstuk 12