Aprenda Feature Engineering Pipeline for Time Series

Deslize para mostrar o menu

Building an effective feature engineering pipeline for time series data ensures that you can systematically apply transformations and create new features without manual repetition or risk of data leakage. A well-structured pipeline allows you to preprocess new data consistently and supports reproducible experiments for model development and validation.

To create a robust pipeline, you should follow a clear sequence of steps:

Sort your data chronologically to maintain the time order;
Create lag features to capture previous values of your target or predictors;
Generate rolling window statistics, such as rolling means or standard deviations, to summarize recent trends;
Add expanding window features, capturing cumulative statistics from the beginning of the series up to each time point;
Add calendar-based features, such as extracting the month, day of week, or hour, to incorporate cyclical or seasonal patterns.

This order is important. Lags and rolling features must be computed before calendar features to avoid introducing look-ahead bias or data leakage. Handling missing values, especially those created by lag and rolling operations, should be done after all new features are added but before model training.


              12345678910111213141516171819202122232425262728293031
            
import pandas as pd

# Example time series DataFrame
df = pd.DataFrame({
    'timestamp': pd.date_range('2023-01-01', periods=10, freq='D'),
    'value': [12, 15, 13, 14, 16, 18, 17, 19, 20, 22]
})
df = df.sort_values('timestamp').reset_index(drop=True)

# 1. Create lag features
df['lag_1'] = df['value'].shift(1)
df['lag_2'] = df['value'].shift(2)

# 2. Rolling window statistics (window=3)
df['rolling_mean_3'] = df['value'].rolling(window=3, min_periods=1).mean()
df['rolling_std_3'] = df['value'].rolling(window=3, min_periods=1).std()

# 3. Expanding window features
df['expanding_mean'] = df['value'].expanding(min_periods=1).mean()
df['expanding_sum'] = df['value'].expanding(min_periods=1).sum()

# 4. Calendar features
df['day_of_week'] = df['timestamp'].dt.dayofweek
df['month'] = df['timestamp'].dt.month

# 5. Handle missing values (fill lags and rolling std with 0)
df['lag_1'] = df['lag_1'].fillna(0)
df['lag_2'] = df['lag_2'].fillna(0)
df['rolling_std_3'] = df['rolling_std_3'].fillna(0)

print(df)

Tudo estava claro?

Obrigado pelo seu feedback!

Seção 1. Capítulo 10

Pergunte à IA

Pergunte o que quiser ou experimente uma das perguntas sugeridas para iniciar nosso bate-papo

Seção 1. Capítulo 10