Skip to content
Tutorials Center
My learning

Topics

Lesson 19 of 39 ยท Association Rules

Association Rule Types and Measures

Explore multilevel, multidimensional and quantitative association rules, closed and maximal itemsets, interestingness measures like lift and conviction, and associative classification.

With support and confidence and an algorithm such as Apriori or FP-Growth, you can produce thousands of rules from a real data set. Most are obvious, redundant or misleading. This lesson shows the kinds of rules mining can produce, how to shrink the output with closed and maximal itemsets, and how to rank rules with better measures than confidence. It ends with associative classification, which turns rules into a classifier.

Types of association rules

Rules can be classified along several lines.

Kinds of association rules

Boolean Item present or absent.
Quantitative Ranges like age 20-29.
Multilevel Milk and also dairy.
Multidimensional Age, city and item together.

Also: single-level, sequential, negative.

1 / 2
  • Boolean (binary) rules use presence only: -> {butter}.
  • Quantitative rules involve numeric attributes, usually after discretizing them (Data Transformation): age in [20, 29] and income = high -> buys = laptop.
  • Single-dimensional rules use one predicate, such as buys: buys(bread) -> buys(butter). Multidimensional rules combine several, such as age(20-29) and occupation(student) -> buys(laptop).
  • Single-level versus multilevel rules: items live in a hierarchy (2% milk, milk, dairy, food). A rule at a high level has more support but is less specific; at a low level it is specific but rare. Multilevel mining uses reduced support at lower levels.
  • Negative rules: -> not {tea}, for items that substitute for each other.
  • Sequential rules add order: then {case} within 7 days.

Multidimensional rules in pandas

Turn each attribute-value pair into an "item" and reuse the same algorithms. This one-hot trick covers boolean, categorical and discretized numeric attributes.

Python Attribute-value pairs as items

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Too many rules: closed and maximal itemsets

If {A, B, C} is frequent then so are , , , {A, B}, {A, C} and {B, C}. Reporting all of them is redundant. Two compact representations exist:

  • A closed frequent itemset is frequent and has no superset with the same support. Closed itemsets keep all support information: from them you can recover the support of every frequent itemset.
  • A maximal frequent itemset is frequent and has no frequent superset at all. Maximal sets are the most compact, but lose the exact supports of subsets.

Maximal โІ Closed โІ Frequent.

Python Closed and maximal itemsets

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

For example {a, c} is frequent but not closed: its superset {a, b, c} has the same support (2), so nothing is lost by keeping only the superset. Also {b, c} has support 3, so it is closed.

Beyond confidence: interestingness measures

Confidence ignores how common the consequent is. Several measures fix this. With P(A), P(B) and P(A, B):

Measure Formula Reading
Lift P(A,B) / (P(A) P(B)) > 1 positive, 1 independent, < 1 negative
Leverage P(A,B) - P(A) P(B) Extra co-occurrence over chance, in absolute terms
Conviction (1 - P(B)) / (1 - confidence) How much more often the rule would be wrong if A and B were independent; infinity for an always-true rule
Cosine P(A,B) / sqrt(P(A) P(B)) Null-invariant: ignores transactions with neither
Kulczynski `(P(B A) + P(A

Null-invariant measures are not affected by the number of transactions that contain neither A nor B. In big sparse data (most baskets contain neither), lift can be inflated simply by adding empty baskets, while cosine and Kulczynski stay stable.

Python Compute several measures

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Look at the second row: with many empty baskets lift jumps from 4 to 400, while cosine and Kulczynski do not move. Choose a null-invariant measure when your data is mostly "neither".

A misleading high-confidence rule

Python High confidence, lift below 1

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The rule "tea -> coffee" has 40 percent confidence, but coffee is bought by 75 percent of everyone: tea buyers are less likely to buy coffee. A rule is only interesting if lift (or another measure) is clearly away from independence.

Simpson's paradox

An association that appears in aggregate data can reverse inside every subgroup. Always check important rules within segments such as region or season, because a hidden variable (a confounder) can fake or hide the relation. Subgroup checks are the practical defense.

Associative classification

Associative classification (CBA, CMAR and similar) uses rules for prediction. Mine class association rules whose consequent is a class label, {age<30, student} -> class=buys_laptop, keep those with high support and confidence, rank them (by confidence, then support, then shorter rule), and classify a new record with the first matching rule, or a vote among the best rules. It is interpretable, handles many attributes and often competes with decision trees. See Data Mining Tasks and Applications for the wider picture.

Python A tiny associative classifier

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Practical guidance

  1. Start with a clear business question; choose items accordingly (product category rather than SKU).
  2. Use hierarchies to reach adequate support.
  3. Filter rules with lift (and a null-invariant measure) as well as confidence.
  4. Prefer closed or maximal itemsets to cut redundancy.
  5. Validate on a later time period; check subgroups for Simpson's paradox.
  6. Turn rules into actions and test them (placement, bundles, recommendations).

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. A maximal frequent itemset is one that:

    Maximal sets are the borders of the frequent region; no larger set is frequent.
  2. 2. A rule has confidence 0.4, but the consequent occurs in 75 percent of all baskets. Lift is:

    Lift = 0.4 / 0.75 = about 0.53, a negative association.
  3. 3. Which measure is NOT affected by the number of baskets containing neither A nor B?

    Cosine and Kulczynski are null-invariant; lift and support change with the total count.
  4. 4. A rule 'age 20-29 and student -> buys laptop' is:

    It combines several attributes (age, job, purchase) and uses a numeric range.
  5. 5. Associative classification uses:

    Rules whose consequent is a class label are ranked and used to classify new records.

Summary

  • Rules can be boolean or quantitative, single or multidimensional, single or multilevel, negative or sequential.
  • Closed itemsets keep all support information; maximal itemsets are the most compact.
  • Rank rules with lift, leverage, conviction, and null-invariant cosine or Kulczynski, not confidence alone.
  • Check subgroups to avoid Simpson's paradox and validate rules on new data.
  • Associative classification turns class association rules into an interpretable classifier.

Next: Classification Basics