KNeighborsClassifier

Implementing k-Nearest Neighbors is pretty straightforward. We only need to import and use the KNeighborsClassifier class.

Constructor:

• KNeighborsClassifier(n_neighbors = 5)
  • n_neighbors – number of neighbors (k). Default value is 5;

Methods:

• fit(X, y) – Fit the training set;
• predict(X) – Predict the class for X;
• score(X, y) – Returns the accuracy for the X, y set.

Once you imported the class and created a class object like this:

# Importing the class
from sklearn.neighbors import KNeighborsClassifier
knn = KNeighborsClassifier(n_neighbors=3)

You need to feed it the training data using the .fit() method:

knn.fit(X_scaled, y)

And that's it! You can predict new values now.

y_pred = knn.predict(X_new_scaled)

Scaling the data

However, remember that the data must be scaled. StandardScaler is commonly used for this purpose:

Constructor:

• StandardScaler().

Methods:

• fit(X) – calculates $\bar{x}$ and $s$ for X;
• transform(X) – returns $X_{\text{scaled}}$ using $\bar{x}, s$ from .fit();
• fit_transform(X) – .fit(X) then .transform(X).

You must compute $\bar x$ and $s$ only on the training set using .fit() or .fit_transform(). Then use .transform() on the test set so both sets are scaled identically:

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

Using different scaling values for train and test harms predictions.

Example

We predict whether a person enjoys Star Wars VI using their ratings for Episodes IV and V (from The Movies Dataset). After training, we test two users: one rated IV/V as 5 and 5, the other as 4.5 and 4.