Lära CatBoost | Framework Deep Dive

CatBoost is a gradient boosting framework known for three core innovations:

Native categorical feature support: unlike most frameworks that require you to manually encode categorical variables, CatBoost lets you pass raw string or integer categories directly. The model handles encoding internally, preserving information and reducing the risk of information leakage.
Ordered boosting to reduce overfitting: traditional boosting can overfit when calculating statistics for categorical features, as it may use information from the entire dataset. CatBoost solves this by processing data in random orders and, for each data point, using only prior data in the sequence to compute statistics. This approach mimics real-world scenarios and improves generalization.
Efficient categorical encoding: instead of expanding the feature space with one-hot encoding or using arbitrary label encodings, CatBoost uses target statistics and hash-based techniques. This approach efficiently captures complex interactions between categories and the target, reduces memory usage, and often leads to better predictive performance.


              12345678910111213141516171819202122232425
            
import pandas as pd
from catboost import CatBoostClassifier, Pool

# Create a synthetic dataset with categorical features
data = pd.DataFrame({
    "color": ["red", "blue", "green", "blue", "red", "green", "red", "blue"],
    "size": ["S", "M", "L", "S", "M", "L", "M", "S"],
    "price": [10, 15, 7, 12, 14, 8, 13, 11],
    "label": [0, 1, 0, 1, 1, 0, 1, 0]
})

# Specify categorical feature indices
cat_features = ["color", "size"]

# Split features and target
X = data[["color", "size", "price"]]
y = data["label"]

# Initialize CatBoostClassifier with minimal preprocessing
model = CatBoostClassifier(iterations=50, learning_rate=0.1, verbose=0)
model.fit(X, y, cat_features=cat_features)

# Predict on the training data
preds = model.predict(X)
print("Predictions:", preds)

Note

CatBoost's native handling of categorical features provides several advantages over traditional one-hot encoding. One-hot encoding can dramatically increase the dimensionality of your dataset, especially when categorical features have many unique values, leading to slower training and higher memory usage. It also fails to capture relationships between categories and the target variable. In contrast, CatBoost's approach leverages target statistics and sophisticated encoding schemes that reduce memory overhead and can uncover subtle patterns in categorical data, often resulting in better predictive performance with less feature engineering.

Uppgift

Swipe to start coding

You are given a synthetic binary classification dataset. Your goal is to train a CatBoostClassifier using the scikit-learn–style API and evaluate its performance.

Follow the steps:

Generate a classification dataset and split it into train/test sets.
Initialize a CatBoostClassifier with the following parameters:
- iterations=150;
- learning_rate=0.1;
- depth=6;
- random_state=42;
- verbose=False.
Train the model on the training data.
Predict labels for the test data.
Compute accuracy and store it in accuracy_value.
Print the dataset shapes, model depth, and accuracy score.

Lösning

Var allt tydligt?

Tack för dina kommentarer!

Avsnitt 2. Kapitel 3

single

Fråga AI

Fråga vad du vill eller prova någon av de föreslagna frågorna för att starta vårt samtal

Awesome!

Completion rate improved to 11.11

Svep för att visa menyn