Impara Feature Engineering Pipeline for Time Series

Scorri per mostrare il menu

Building an effective feature engineering pipeline for time series data ensures that you can systematically apply transformations and create new features without manual repetition or risk of data leakage. A well-structured pipeline allows you to preprocess new data consistently and supports reproducible experiments for model development and validation.

To create a robust pipeline, you should follow a clear sequence of steps:

Sort your data chronologically to maintain the time order;
Create lag features to capture previous values of your target or predictors;
Generate rolling window statistics, such as rolling means or standard deviations, to summarize recent trends;
Add expanding window features, capturing cumulative statistics from the beginning of the series up to each time point;
Add calendar-based features, such as extracting the month, day of week, or hour, to incorporate cyclical or seasonal patterns.

This order is important. Lags and rolling features must be computed before calendar features to avoid introducing look-ahead bias or data leakage. Handling missing values, especially those created by lag and rolling operations, should be done after all new features are added but before model training.


              12345678910111213141516171819202122232425262728293031
            
import pandas as pd

# Example time series DataFrame
df = pd.DataFrame({
    'timestamp': pd.date_range('2023-01-01', periods=10, freq='D'),
    'value': [12, 15, 13, 14, 16, 18, 17, 19, 20, 22]
})
df = df.sort_values('timestamp').reset_index(drop=True)

# 1. Create lag features
df['lag_1'] = df['value'].shift(1)
df['lag_2'] = df['value'].shift(2)

# 2. Rolling window statistics (window=3)
df['rolling_mean_3'] = df['value'].rolling(window=3, min_periods=1).mean()
df['rolling_std_3'] = df['value'].rolling(window=3, min_periods=1).std()

# 3. Expanding window features
df['expanding_mean'] = df['value'].expanding(min_periods=1).mean()
df['expanding_sum'] = df['value'].expanding(min_periods=1).sum()

# 4. Calendar features
df['day_of_week'] = df['timestamp'].dt.dayofweek
df['month'] = df['timestamp'].dt.month

# 5. Handle missing values (fill lags and rolling std with 0)
df['lag_1'] = df['lag_1'].fillna(0)
df['lag_2'] = df['lag_2'].fillna(0)
df['rolling_std_3'] = df['rolling_std_3'].fillna(0)

print(df)

Tutto è chiaro?

Grazie per i tuoi commenti!

Sezione 1. Capitolo 10

Chieda ad AI

Chieda pure quello che desidera o provi una delle domande suggerite per iniziare la nostra conversazione

Sezione 1. Capitolo 10