Ensemble Learning

1. Basic Principles of Building Ensemble Models

What is an ensemble? How are ensembles different from standard machine-learning models? What are the types of ensembles? Let's consider the answers to these questions.

What is Ensemble of Models?

Bagging Models

Boosting Models

Stacking Models

2. Commonly Used Bagging Models

Let's consider some commonly used bagging ensemble models, the features of their use, and also apply some of them to solve real-life tasks.

Bagging Classifier

Challenge: Solving Task Using Bagging Classifier

Bagging Regressor

Challenge: Solving Task Using Bagging Regressor

Random Forest

Challenge: Determining Feature Importances Using Random Forest

ExtraTrees

3. Commonly Used Boosting Models

The mechanism of work of boosting models differs from bagging models. Now we will explore these distinctions, gain insights into utilizing model boosting for problem-solving, and illustrate its functionality through practical demonstrations.

AdaBoost Classifier

Challenge: Solving Task Using AdaBoost Classifier

Challenge: Solving Task Using AdaBoost Regressor

Gradient Boosting

XGBoost

Challenge: Solving Task Using XGBoost

4. Commonly Used Stacking Models

Let's consider some commonly used stacking ensemble models, the features of their use, and also apply some of them to solve real-life tasks.

Stacking Classifier

Challenge: Solving Task Using Stacking Classifier

Challenge: Solving Task Using Stacking Regressor

Using Ensembles As Base Models

Course Summary

Challenge: Solving Task Using AdaBoost Regressor

AdaBoost Regressor is an ensemble learning algorithm used for regression tasks.

The principle of work of such a regressor coincides with the principle of work of the AdaBoost Classifier. The only difference is that we use some regression algorithms (linear regression, decision tree regressor, polynomial regression, etc.) as a base model.

The AdaBoostRegressor class in Python provides tools to train the model and make predictions.