Implementing k-NN

KNeighborsClassifier

Implementing k-Nearest Neighbors is pretty straightforward. We only need to import and use the KNeighborsClassifier class.

Once you imported the class and created a class object like this:

python
9
1
2
from sklearn.neighbors import KNeighborsClassifier # Import the class
knn = KNeighborsClassifier(n_neighbors=3)

You need to feed it the training data using the .fit() method:

python
knn.fit(X_scaled, y)

And that's it! You can predict new values now.

python
y_pred = knn.predict(X_new_scaled)

Scaling the data

However, remember that the data must be scaled. Let's take a closer look at the StandardScaler commonly used for scaling the data.
StandardScaler just subtracts the sample's mean and then divides the result by the sample's standard deviation.

Note

If the terms sample mean and sample standard deviation sound unfamiliar, you can check out our Learning Statistics with Python. . But an understanding of what those numbers are for is not mandatory, so you can just ignore the meanings of x̄ and s and move on :)

You should calculate x̄ and s on the training set(using either .fit() or .fit_transform()) and use the same x̄ and s to preprocess the data you are predicting(using .transform()). Here is an example:

python
9
1
2
3
4
5
6
from sklearn.preprocessing import StandardScaler # import the class
# `X` is the training set's features
# `X_new` is the new instances' features
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X) # calculates x̄ and s and scales the `X`
X_new_scaled = scaler.transform(X_new) # scales the `X_new` with x̄ and s calculated in the previous line

If you use different x̄ and s for training set and new instances your predictions will likely be worse.

Example

So to perform a k-NN classification in Python, you need to use KNeighborsClassifier and StandardScaler. Your code will look like this:

python
99
1
2
3
4
5
6
7
8
9
10
11
from sklearn.neighbors import KNeighborsClassifier
from sklearn.preprocessing import StandardScaler

# Load and preprocess `X`, `y`, and `X_new` here

scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
X_new_scaled = scaler.transform(X_new)

knn = KNeighborsClassifier(n_neighbors=13).fit(X_scaled, y)
y_pred = knn.predict(X_new_scaled)

Here is a simple example where we try to predict whether the person will like Star Wars VI based on his ratings for Star Wars IV and V. We will read the data from the URL.

The data is taken from The Movies Dataset with extra preprocessing. A person considered liking Star Wars VI if he/her rated it more than 4/5.