Summary  
This chapter introduces the k-Nearest Neighbors algorithm, which predicts an instance’s class by finding its k closest neighbors in feature space and selecting the most frequent class among them.

General domain of usage  
Supervised classification tasks in machine learning.

Let's start our classification adventure with the simplest task - **binary classification**. Suppose we want to classify sweets as cookies/not cookies based on a **single feature**: their weight.

A simple way to predict the class of a new instance is to look at its **closest neighbor**. In our example, we must find a sweet that weighs most similarly to the new instance.

That is the idea behind **k-Nearest Neighbors**(**k-NN**) - we just look at the neighbors. 
The k-NN algorithm assumes that similar things exist in **close proximity**. In other words, similar things are near each other. **k** in the k-NN stands for the number of neighbors we consider when making a prediction.

In the example above, we considered only **1** neighbor, so it was 1-Nearest Neighbor. But usually, k is set to a bigger number, as looking only at one neighbor can be **unreliable**:

If **k** (number of neighbors) is greater than one, we choose the **most frequent class** in the neighborhood as a prediction. Here is an example of predicting two new instances with **k=3**:

As you can see, changing the **k** may cause different predictions.

Occasionally, k-NN produces a **tie** when multiple classes appear equally among the nearest neighbors. Most libraries, including scikit-learn, break ties by **choosing the first class** in their internal order - something to keep in mind, since it can subtly affect reproducibility and interpretation.

Note

In the k-Nearest Neighbors algorithm, how is the class of a new instance predicted when k > 1?

Master the core classification algorithms that power modern machine learning. Explore how models like k-NN, logistic regression, decision trees, and random forests make predictions, evaluate their accuracy, and understand when to use each. Build the skills to compare models and choose the best one for your data.

Discover how the k-nearest neighbors algorithm makes predictions based on similarity. Learn to handle multiple features, tune parameters, and apply cross-validation to improve accuracy.

Understand how logistic regression models probabilities and classifies outcomes. Practice implementing it, interpreting decision boundaries, and applying regularization to prevent overfitting.

Learn how decision trees split data into meaningful groups based on feature values. Explore how parameters like tree depth and minimum samples per leaf affect model performance and generalization.

Explore how random forests combine multiple decision trees to improve accuracy and robustness. Understand the role of randomness and apply this ensemble method to real-world data.

Evaluate models using metrics such as accuracy, precision, recall, and F1-score. Learn to interpret confusion matrices and compare multiple classifiers to identify the best-performing model.

What is k-NN