Leer Iterative Search with HalvingGridSearchCV | Advanced and Bayesian Tuning Techniques

Successive Halving is a resource-efficient hyperparameter tuning method that works in rounds:

Start by evaluating many hyperparameter combinations with minimal resources;
After each round, keep only the best-performing candidates;
Allocate more resources to these candidates in the next round;
Repeat the process until you identify the top performers.

This approach quickly narrows down promising settings while saving computation by discarding poor options early.

Successive Halving: Reducing Computation

Successive halving streamlines hyperparameter search by focusing resources on the best candidates through an iterative process:

Begin with many hyperparameter combinations, each given minimal resources;
Evaluate all candidates and rank their performance;
Eliminate a fixed proportion of the lowest-performing candidates after each round;
Increase resources for the remaining, better-performing candidates;
Repeat these steps until only the top candidates remain.

This approach quickly narrows down the search, saving computation and time compared to exhaustive grid search.


              12345678910111213141516171819202122232425262728293031
            
from sklearn.experimental import enable_halving_search_cv  # noqa
from sklearn.model_selection import HalvingGridSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

# Load example data
data = load_iris()
X, y = data.data, data.target

# Define a simple parameter grid
param_grid = {
    'n_estimators': [10, 50, 100],
    'max_depth': [2, 4, 6]
}

# Initialize the classifier
clf = RandomForestClassifier(random_state=42)

# Set up HalvingGridSearchCV
halving_search = HalvingGridSearchCV(
    estimator=clf,
    param_grid=param_grid,
    factor=2,
    random_state=42
)

# Fit the model
halving_search.fit(X, y)

# Show the best parameters
print(halving_search.best_params_)

Note

HalvingGridSearchCV often delivers similar accuracy to regular GridSearchCV but is much more runtime efficient. It quickly eliminates underperforming hyperparameter combinations using fewer resources, so you spend less time training models that are unlikely to perform well. Regular GridSearchCV tests every possible combination, which can be slow for large search spaces but may guarantee finding the absolute best set if resources are unlimited.

HalvingGridSearchCV is still experimental in scikit-learn. To use it, you must enable experimental features by importing:

from sklearn.experimental import enable_halving_search_cv  # noqa
from sklearn.model_selection import HalvingGridSearchCV

Was alles duidelijk?

Bedankt voor je feedback!

Sectie 3. Hoofdstuk 3

Vraag AI

Vraag wat u wilt of probeer een van de voorgestelde vragen om onze chat te starten.

Suggested prompts:

Can you explain how the factor parameter affects the successive halving process?

What are the advantages of using HalvingGridSearchCV over traditional GridSearchCV?

Are there any limitations or caveats when using HalvingGridSearchCV?

Awesome!

Completion rate improved to 9.09

Veeg om het menu te tonen