Learn Filtering Meaningless Rules and Identifying High-Value Product Bundles | High Performance Rule Mining and Scale Optimization

Swipe to show menu

To maximize the impact of market basket analysis, you must separate truly valuable association rules from those that are either trivial or irrelevant for your business objectives. This process ensures your recommendations and promotions are based on actionable insights rather than on noise or obvious patterns.

Filtering Criteria

The first step in filtering association rules is to set minimum thresholds for support, confidence, and lift. These metrics help you eliminate rules that are either too rare to be useful or not statistically significant.

Support threshold: ignore rules that occur in too few transactions, as they may be outliers or lack business impact;
Confidence threshold: discard rules that do not show strong enough association between the antecedent and consequent;
Lift threshold: remove rules with lift values close to or below 1, as they do not indicate a meaningful association beyond random chance.

Redundancy

Many rules in your mining results may be redundant, meaning they provide no new information compared to other rules. For example, if both "milk → bread" and "milk, butter → bread" have similar support and confidence, the more specific rule may not add value. Identifying and removing redundant rules helps you focus on the most concise and informative patterns.

Business Relevance

Not all statistically strong rules are useful for your business. To determine if a rule is actionable, ask:

Does the rule suggest a product pairing that you can promote together?
Is the combination logistically feasible (e.g., not pairing products from unrelated departments)?
Can the rule inform store layout or cross-selling strategies?
Does the rule align with your business goals, such as increasing basket size or introducing new products?

Example: Filtering Rules to Isolate High-Value Bundles

Suppose you have mined a set of association rules from your transaction data. By applying thresholds and removing redundancy, you can isolate a small set of high-value product bundles that are both statistically significant and align with your business objectives.


              12345678910111213141516171819202122232425262728293031
            
import pandas as pd

# Example association rules DataFrame
rules = pd.DataFrame({
    'antecedents': [['milk'], ['bread'], ['milk', 'bread'], ['chips'], ['chips', 'salsa']],
    'consequents': [['bread'], ['milk'], ['butter'], ['salsa'], ['soft drinks']],
    'support': [0.30, 0.28, 0.15, 0.10, 0.08],
    'confidence': [0.7, 0.6, 0.5, 0.4, 0.75],
    'lift': [1.2, 1.1, 0.95, 1.0, 1.5]
})

# Setting filtering thresholds
min_support = 0.10
min_confidence = 0.6
min_lift = 1.1

# Filtering rules by thresholds and explicitly creating a copy
filtered_rules = rules[
    (rules['support'] >= min_support) &
    (rules['confidence'] >= min_confidence) &
    (rules['lift'] >= min_lift)
].copy()

# Removing redundant rules: keep only the most general (shortest antecedents)
filtered_rules['antecedent_length'] = filtered_rules['antecedents'].apply(len)
non_redundant_rules = filtered_rules.sort_values('antecedent_length').drop_duplicates('consequents')

# Identifying top product bundles (by lift)
top_bundles = non_redundant_rules.sort_values('lift', ascending=False)

print(top_bundles[['antecedents', 'consequents', 'support', 'confidence', 'lift']])

1. What is the main reason for setting a minimum lift threshold when filtering association rules?

2. Which of the following best describes a redundant association rule?

Everything was clear?

Thanks for your feedback!

Section 2. Chapter 4

Ask AI

Ask anything or try one of the suggested questions to begin our chat

Section 2. Chapter 4