Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.
Overfitting. Regularization | Logistic Regression
Classification with Python

Overfitting. Regularization

As shown in the previous chapter, using PolynomialFeatures, you can get a pretty complex decision boundary. Second-degree polynomial features can even get you the boundaries in the picture below.

And it is only a degree of two. A higher degree may yield even more complex shapes. But there is a problem with it. The decision boundary built by Logistic Regression may become too complicated, causing the model to overfit.
Overfitting is when the model, instead of learning general patterns in data, builds a very complex decision boundary to handle every training instance. Still, it does not perform as well on data it has never seen, while performing well on unseen data is a primary task of the Machine Learning model.

The regularization tackles the problem of overfitting. In fact, ℓ2 regularization is used in the LogisticRegression class by default. But you need to configure how strongly the model should be regularized. It is controlled by a C parameter.

  • greater C – lower regularization, more overfitting;
  • lower C – stronger regularization, less overfitting (but possibly underfitting).
carousel-imgcarousel-imgcarousel-img

What values of C will result in a good model depends on the dataset, thus better to choose it using the GridSearchCV.

Note

If you build a Logistic Regression with regularization, you must scale the data.

The LogisticRegression class includes regularization by default, so you should either remove regularization(by setting penalty=None) or scale the data(e.g., using StandardScaler).

Note

If you use both PolynomialFeatures and StandardScaler for preprocessing, the StandardScaler should be applied after the PolynomialFeatures. Generally, you want to apply the StandardScaler when all other changes to features are done.

1. Choose the INCORRECT statement.
2. What is the correct order to preprocess data.

Choose the INCORRECT statement.

Select the correct answer

What is the correct order to preprocess data.

Select the correct answer

Everything was clear?

Section 2. Chapter 5
course content

Course Content

Classification with Python

Overfitting. Regularization

As shown in the previous chapter, using PolynomialFeatures, you can get a pretty complex decision boundary. Second-degree polynomial features can even get you the boundaries in the picture below.

And it is only a degree of two. A higher degree may yield even more complex shapes. But there is a problem with it. The decision boundary built by Logistic Regression may become too complicated, causing the model to overfit.
Overfitting is when the model, instead of learning general patterns in data, builds a very complex decision boundary to handle every training instance. Still, it does not perform as well on data it has never seen, while performing well on unseen data is a primary task of the Machine Learning model.

The regularization tackles the problem of overfitting. In fact, ℓ2 regularization is used in the LogisticRegression class by default. But you need to configure how strongly the model should be regularized. It is controlled by a C parameter.

  • greater C – lower regularization, more overfitting;
  • lower C – stronger regularization, less overfitting (but possibly underfitting).
carousel-imgcarousel-imgcarousel-img

What values of C will result in a good model depends on the dataset, thus better to choose it using the GridSearchCV.

Note

If you build a Logistic Regression with regularization, you must scale the data.

The LogisticRegression class includes regularization by default, so you should either remove regularization(by setting penalty=None) or scale the data(e.g., using StandardScaler).

Note

If you use both PolynomialFeatures and StandardScaler for preprocessing, the StandardScaler should be applied after the PolynomialFeatures. Generally, you want to apply the StandardScaler when all other changes to features are done.

1. Choose the INCORRECT statement.
2. What is the correct order to preprocess data.

Choose the INCORRECT statement.

Select the correct answer

What is the correct order to preprocess data.

Select the correct answer

Everything was clear?

Section 2. Chapter 5
some-alt