Notice: This page requires JavaScript to function properly.
Please enable JavaScript in your browser settings or update your browser.
ExtraTrees | Commonly Used Bagging Models
Ensemble Learning

ExtraTrees

Extra Trees, short for Extremely Randomized Trees, is a bagging ensemble learning technique that builds upon the concept of decision trees to create a more robust and diverse model.

How does ExtraTrees algorithm work?

It is a variation of the Random Forest algorithm but introduces even more randomness into the tree-building process:

  1. The extra trees algorithm, like the random forests algorithm, creates many decision trees, but the sampling for each tree is random, without replacement;
  2. A specific number of features from the total set of features is also selected randomly for each tree;
  3. Extra trees' most important and unique characteristic is the random selection of a splitting value for a feature. Instead of calculating a locally optimal value using Gini or entropy to split the data, the algorithm randomly selects a split value. This makes the trees diversified and uncorrelated.

    Note

    We can also use .feature_importances_ attribute to measure the features' impact on the model's result.

Example

We can use ExtraTrees in Python just like Random Forest using the ExtraTreesClassifier or ExtraTreesRegressor classes:

You can find the official documentation with all the necessary information about implementing this model in Python on the official website. Go here if needed.

In which of the ensembles are the base models more diverse and uncorrelated?

Select the correct answer

Everything was clear?

Section 2. Chapter 7
course content

Course Content

Ensemble Learning

ExtraTrees

Extra Trees, short for Extremely Randomized Trees, is a bagging ensemble learning technique that builds upon the concept of decision trees to create a more robust and diverse model.

How does ExtraTrees algorithm work?

It is a variation of the Random Forest algorithm but introduces even more randomness into the tree-building process:

  1. The extra trees algorithm, like the random forests algorithm, creates many decision trees, but the sampling for each tree is random, without replacement;
  2. A specific number of features from the total set of features is also selected randomly for each tree;
  3. Extra trees' most important and unique characteristic is the random selection of a splitting value for a feature. Instead of calculating a locally optimal value using Gini or entropy to split the data, the algorithm randomly selects a split value. This makes the trees diversified and uncorrelated.

    Note

    We can also use .feature_importances_ attribute to measure the features' impact on the model's result.

Example

We can use ExtraTrees in Python just like Random Forest using the ExtraTreesClassifier or ExtraTreesRegressor classes:

You can find the official documentation with all the necessary information about implementing this model in Python on the official website. Go here if needed.

In which of the ensembles are the base models more diverse and uncorrelated?

Select the correct answer

Everything was clear?

Section 2. Chapter 7
some-alt