Challenge: Classifying Unseparateble Data

In this Challenge, you are given the following dataset:

import pandas as pd

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/b71ff7ac-3932-41d2-a4d8-060e24b00129/circles.csv')

print(df.head())


              1234
            
import pandas as pd

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/b71ff7ac-3932-41d2-a4d8-060e24b00129/circles.csv')
print(df.head())

Here is its plot.

import pandas as pd

import matplotlib.pyplot as plt

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/b71ff7ac-3932-41d2-a4d8-060e24b00129/circles.csv')

plt.scatter(df['X1'], df['X2'], c=df['y'])


              12345
            
import pandas as pd
import matplotlib.pyplot as plt

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/b71ff7ac-3932-41d2-a4d8-060e24b00129/circles.csv')
plt.scatter(df['X1'], df['X2'], c=df['y'])

The dataset is for sure not linearly separable. Let's look at the Logistic Regression performance:

import pandas as pd

from sklearn.preprocessing import StandardScaler

from sklearn.linear_model import LogisticRegression

from sklearn.model_selection import cross_val_score

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/b71ff7ac-3932-41d2-a4d8-060e24b00129/circles.csv')

X = df[['X1', 'X2']]

y = df['y']

X = StandardScaler().fit_transform(X)

lr = LogisticRegression().fit(X, y)

print(cross_val_score(lr, X, y).mean())


              123456789101112
            
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/b71ff7ac-3932-41d2-a4d8-060e24b00129/circles.csv')
X = df[['X1', 'X2']]
y = df['y']

X = StandardScaler().fit_transform(X)
lr = LogisticRegression().fit(X, y)
print(cross_val_score(lr, X, y).mean())

The result is awful. Regular Logistic Regression is not suited for this task. Your task is to check whether the PolynomialFeatures will help. To find the best C parameter, you will use the GridSearchCV class.

In this challenge, the Pipeline is used. You can think of it as a list of preprocessing steps. Its .fit_transform() method sequentially applies .fit_transform() to each item.

Task

Swipe to start coding

Build a Logistic Regression model with polynomial features and find the best C parameter using GridSearchCV

Create a pipeline to make an X_poly variable that will hold the polynomial features of degree 2 of X and be scaled.
Create a param_grid dictionary to tell the GridSearchCV you want to try values [0.01, 0.1, 1, 10, 100] of a C parameter.
Initialize and train a GridSearchCV object.

Solution

import pandas as pd

from sklearn.linear_model import LogisticRegression

from sklearn.preprocessing import StandardScaler, PolynomialFeatures

from sklearn.model_selection import cross_val_score, GridSearchCV

from sklearn.pipeline import Pipeline

# Read the data and assign `X`, `y` variables

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/b71ff7ac-3932-41d2-a4d8-060e24b00129/circles.csv')

X = df[['X1', 'X2']]

y = df['y']

# Preprocess the data using `Pipeline`

pipe = Pipeline([('poly', PolynomialFeatures(2, include_bias=False)), # Polynomial Features

('scaler', StandardScaler())]) # Scaling

X_poly = pipe.fit_transform(X)

# Find the best model using `GridSearchCV`

lr = LogisticRegression()

param_grid = {'C': [0.01, 0.1, 1, 10, 100]}

grid_cv = GridSearchCV(lr, param_grid).fit(X_poly, y)

# Print the best score and the best estimator

print(grid_cv.best_score_)

print(grid_cv.best_estimator_)

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 6

import pandas as pd

from sklearn.linear_model import LogisticRegression

from sklearn.preprocessing import StandardScaler, PolynomialFeatures

from sklearn.model_selection import cross_val_score, GridSearchCV

from sklearn.pipeline import Pipeline

# Read the data and assign `X`, `y` variables

df = pd.read_csv('https://codefinity-content-media.s3.eu-west-1.amazonaws.com/b71ff7ac-3932-41d2-a4d8-060e24b00129/circles.csv')

X = df[['X1', 'X2']]

y = df['y']

# Preprocess the data using `Pipeline`

pipe = Pipeline([('poly', PolynomialFeatures(___, include_bias=False)), # Polynomial Features

('scaler', ___())]) # Scaling

X_poly = pipe.fit_transform(X)

# Find the best model using `GridSearchCV`

lr = LogisticRegression()

param_grid = {'C': [___]}

grid_cv = GridSearchCV(lr, param_grid).___(X_poly, y)

# Print the best score and the best estimator

print(grid_cv.best_score_)

print(grid_cv.best_estimator_)