Overfitting, Underfitting, and Generalization
When you train a machine learning model, you want it to learn the underlying patterns in your data so it can make accurate predictions on new, unseen data. However, the model's ability to generalize depends on how complex it is and how its hyperparameters are set. If your model is too complex, it might memorize the training data—including noise and outliers—rather than learning general trends. This is known as overfitting. On the other hand, if your model is too simple, it may fail to capture important relationships in the data, resulting in poor performance even on the training set. This is called underfitting. Hyperparameters such as tree depth in decision trees, the number of neighbors in k-nearest neighbors, or the regularization strength in linear models directly influence whether your model is likely to underfit, overfit, or strike the right balance for good generalization.
Overfitting occurs when a model performs well on training data but poorly on test data. Underfitting occurs when a model performs poorly on both.
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758import numpy as np import matplotlib.pyplot as plt # Generate sample data np.random.seed(0) X = np.linspace(0, 10, 100) y_true = np.sin(X) y = y_true + np.random.normal(scale=0.3, size=X.shape) # Underfitting: Linear model from sklearn.linear_model import LinearRegression lin_reg = LinearRegression() lin_reg.fit(X.reshape(-1, 1), y) y_pred_linear = lin_reg.predict(X.reshape(-1, 1)) # Good fit: Polynomial degree 3 from sklearn.preprocessing import PolynomialFeatures from sklearn.linear_model import Ridge poly3 = PolynomialFeatures(degree=3) X_poly3 = poly3.fit_transform(X.reshape(-1, 1)) ridge3 = Ridge(alpha=0.1) ridge3.fit(X_poly3, y) y_pred_poly3 = ridge3.predict(X_poly3) # Overfitting: Polynomial degree 15 poly15 = PolynomialFeatures(degree=15) X_poly15 = poly15.fit_transform(X.reshape(-1, 1)) ridge15 = Ridge(alpha=1e-10) ridge15.fit(X_poly15, y) y_pred_poly15 = ridge15.predict(X_poly15) # Plotting plt.figure(figsize=(15, 4)) plt.subplot(1, 3, 1) plt.scatter(X, y, color='gray', label='Noisy data') plt.plot(X, y_true, 'g--', label='True function') plt.plot(X, y_pred_linear, 'r', label='Underfit (Linear)') plt.title('Underfitting') plt.legend() plt.subplot(1, 3, 2) plt.scatter(X, y, color='gray', label='Noisy data') plt.plot(X, y_true, 'g--', label='True function') plt.plot(X, y_pred_poly3, 'b', label='Good fit (Degree 3)') plt.title('Good Fit') plt.legend() plt.subplot(1, 3, 3) plt.scatter(X, y, color='gray', label='Noisy data') plt.plot(X, y_true, 'g--', label='True function') plt.plot(X, y_pred_poly15, 'm', label='Overfit (Degree 15)') plt.title('Overfitting') plt.legend() plt.tight_layout() plt.show()
Obrigado pelo seu feedback!
Pergunte à IA
Pergunte à IA
Pergunte o que quiser ou experimente uma das perguntas sugeridas para iniciar nosso bate-papo
Can you explain how to choose the right model complexity to avoid underfitting and overfitting?
What are some common techniques to prevent overfitting in machine learning?
Could you describe the role of regularization in these examples?
Awesome!
Completion rate improved to 9.09Overfitting, Underfitting, and Generalization
Deslize para mostrar o menu
When you train a machine learning model, you want it to learn the underlying patterns in your data so it can make accurate predictions on new, unseen data. However, the model's ability to generalize depends on how complex it is and how its hyperparameters are set. If your model is too complex, it might memorize the training data—including noise and outliers—rather than learning general trends. This is known as overfitting. On the other hand, if your model is too simple, it may fail to capture important relationships in the data, resulting in poor performance even on the training set. This is called underfitting. Hyperparameters such as tree depth in decision trees, the number of neighbors in k-nearest neighbors, or the regularization strength in linear models directly influence whether your model is likely to underfit, overfit, or strike the right balance for good generalization.
Overfitting occurs when a model performs well on training data but poorly on test data. Underfitting occurs when a model performs poorly on both.
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758import numpy as np import matplotlib.pyplot as plt # Generate sample data np.random.seed(0) X = np.linspace(0, 10, 100) y_true = np.sin(X) y = y_true + np.random.normal(scale=0.3, size=X.shape) # Underfitting: Linear model from sklearn.linear_model import LinearRegression lin_reg = LinearRegression() lin_reg.fit(X.reshape(-1, 1), y) y_pred_linear = lin_reg.predict(X.reshape(-1, 1)) # Good fit: Polynomial degree 3 from sklearn.preprocessing import PolynomialFeatures from sklearn.linear_model import Ridge poly3 = PolynomialFeatures(degree=3) X_poly3 = poly3.fit_transform(X.reshape(-1, 1)) ridge3 = Ridge(alpha=0.1) ridge3.fit(X_poly3, y) y_pred_poly3 = ridge3.predict(X_poly3) # Overfitting: Polynomial degree 15 poly15 = PolynomialFeatures(degree=15) X_poly15 = poly15.fit_transform(X.reshape(-1, 1)) ridge15 = Ridge(alpha=1e-10) ridge15.fit(X_poly15, y) y_pred_poly15 = ridge15.predict(X_poly15) # Plotting plt.figure(figsize=(15, 4)) plt.subplot(1, 3, 1) plt.scatter(X, y, color='gray', label='Noisy data') plt.plot(X, y_true, 'g--', label='True function') plt.plot(X, y_pred_linear, 'r', label='Underfit (Linear)') plt.title('Underfitting') plt.legend() plt.subplot(1, 3, 2) plt.scatter(X, y, color='gray', label='Noisy data') plt.plot(X, y_true, 'g--', label='True function') plt.plot(X, y_pred_poly3, 'b', label='Good fit (Degree 3)') plt.title('Good Fit') plt.legend() plt.subplot(1, 3, 3) plt.scatter(X, y, color='gray', label='Noisy data') plt.plot(X, y_true, 'g--', label='True function') plt.plot(X, y_pred_poly15, 'm', label='Overfit (Degree 15)') plt.title('Overfitting') plt.legend() plt.tight_layout() plt.show()
Obrigado pelo seu feedback!
