Overfitting and Regularization
As demonstrated in the previous chapter, using PolynomialFeatures, you can create a complex decision boundary. Second-degree polynomial features can even produce the boundaries shown in the image below:
And it is only a degree of two. A higher degree may yield even more complex shapes. But there is a problem with it. The decision boundary built by Logistic Regression may become too complicated, causing the model to overfit.
Overfitting is when the model, instead of learning general patterns in data, builds a very complex decision boundary to handle every training instance. Still, it does not perform as well on data it has never seen, while performing well on unseen data is a primary task of the machine learning model.
The regularization tackles the problem of overfitting. In fact, l2 regularization is used in the LogisticRegression class by default. But you need to configure how strongly the model should be regularized. It is controlled by a C parameter:
- greater
C- lower regularization, more overfitting; - lower
C- stronger regularization, less overfitting (but possibly underfitting).
What values of C will result in a good model depends on the dataset, thus better to choose it using the GridSearchCV.
When using Logistic Regression with regularization, it's essential to scale your data. Regularization penalizes large coefficients, and without scaling, features with larger values can distort the results. In fact, scaling is almost always necessary - even when regularization is not used.
The LogisticRegression class includes regularization by default, so you should either remove regularization(by setting penalty=None) or scale the data(e.g., using StandardScaler).
If you're using both PolynomialFeatures and StandardScaler, make sure to apply StandardScaler after generating the polynomial features. Scaling the data before polynomial expansion can distort the resulting features, since operations like squaring or multiplying already standardized values may lead to unnatural distributions.
1. Choose the INCORRECT statement.
2. What is the correct order to preprocess data
Kiitos palautteestasi!
Kysy tekoälyä
Kysy tekoälyä
Kysy mitä tahansa tai kokeile jotakin ehdotetuista kysymyksistä aloittaaksesi keskustelumme
Mahtavaa!
Completion arvosana parantunut arvoon 5.88Overfitting and Regularization
Pyyhkäise näyttääksesi valikon
As demonstrated in the previous chapter, using PolynomialFeatures, you can create a complex decision boundary. Second-degree polynomial features can even produce the boundaries shown in the image below:
And it is only a degree of two. A higher degree may yield even more complex shapes. But there is a problem with it. The decision boundary built by Logistic Regression may become too complicated, causing the model to overfit.
Overfitting is when the model, instead of learning general patterns in data, builds a very complex decision boundary to handle every training instance. Still, it does not perform as well on data it has never seen, while performing well on unseen data is a primary task of the machine learning model.
The regularization tackles the problem of overfitting. In fact, l2 regularization is used in the LogisticRegression class by default. But you need to configure how strongly the model should be regularized. It is controlled by a C parameter:
- greater
C- lower regularization, more overfitting; - lower
C- stronger regularization, less overfitting (but possibly underfitting).
What values of C will result in a good model depends on the dataset, thus better to choose it using the GridSearchCV.
When using Logistic Regression with regularization, it's essential to scale your data. Regularization penalizes large coefficients, and without scaling, features with larger values can distort the results. In fact, scaling is almost always necessary - even when regularization is not used.
The LogisticRegression class includes regularization by default, so you should either remove regularization(by setting penalty=None) or scale the data(e.g., using StandardScaler).
If you're using both PolynomialFeatures and StandardScaler, make sure to apply StandardScaler after generating the polynomial features. Scaling the data before polynomial expansion can distort the resulting features, since operations like squaring or multiplying already standardized values may lead to unnatural distributions.
1. Choose the INCORRECT statement.
2. What is the correct order to preprocess data
Kiitos palautteestasi!
