Summary
This chapter explains how to use a pipeline to chain preprocessing steps with a final estimator and why applying .transform (rather than .fit_transform) on new data ensures consistent feature encoding. It also shows how to include a model as the last pipeline step so that fitting and predicting can be done in a single call.

General domain of usage
Machine learning model development

`Pipeline` was previously used for **preprocessing**, but its real purpose is to chain preprocessing with a **final predictor**. The last step in a pipeline can be any estimator (typically a model) that produces predictions.

When calling `.fit()`, each transformer runs `.fit_transform()`.
When calling `.predict()`, the pipeline uses `.transform()` before sending data to the final estimator.
This is required because **new data must be transformed exactly like the training data**.

Note

## Why `.transform()`?

Using `.fit_transform()` on new data could change encodings (e.g., in `OneHotEncoder`), creating mismatched columns and unreliable predictions.
`.transform()` guarantees **consistent preprocessing**, ignoring unseen categories and keeping the same column order.

Here is how one-hot encoded training data looks like:

If `.fit_transform()` were applied to **new instances**, the `OneHotEncoder` could generate columns in a different order or even introduce new ones. This would cause the new data to be transformed **inconsistently with the training set**, making predictions **unreliable**.


However, using `.transform()` ensures that the new data is encoded **exactly as the training data**, ignoring categories not seen during training:

## Adding the Final Estimator

Simply add the model as the **last step** of the pipeline:

```python
pipe = make_pipeline(
    ct,
    SimpleImputer(strategy='most_frequent'),
    StandardScaler(),
    KNeighborsClassifier()
)
pipe.fit(X, y)
pipe.predict(X_new)
```

This allows the whole workflow—preprocessing + prediction—to run with one call.

Which statements about the final estimator and preprocessing in a pipeline are correct?

Master the fundamentals of Machine Learning and the Scikit-learn library. Explore the complete ML workflow, from handling missing values and encoding categorical data to scaling features. Build efficient, leak-proof data preprocessing pipelines using ColumnTransformer. Transform raw datasets into model-ready structures and implement robust predictive pipelines.

Final Estimator

Why .transform()?

Adding the Final Estimator

Why `.transform()`?