Learning Classifier Systems
Learning Classifier Systems (LCS)
• The system has three layers:
– A performance system that interacts with
environment,
– An apportionment of credit algorithm that rates
rules as to usefulness,
– A rule discovery algorithm that generates
plausible new rules to replace less useful rules.
Performance System Cycles
• Message is posted in the message list from the
input interface.
• Each rule is matched against the message list
• All matching rules compete to post in the next
message list via bidding process; winning rule
posts in the new message list
• The output interface checks the new message
and produces an effector action.
• The new message list replaces the previous one.
• Repeat.
Overview of LCS
Rule format
• Rule
– Condition = {0,1,#}k
– Action = message to be posted in the message list
– Strength = rule’s usefulness to the system
Example (Wolf or Grandmother?)
Encoding
teeth
kind
ears
num. of legs
smart
scream
runaway
kiss
Wolf
1
0
1
1
0
1
1
#
1
1
0
1
#
0
0
1
GrandMa
0
0
Matching
Message List
0
[N]
[M]
1
0
0
Condition
Action
Strength
#
1
#
#
#010
100
1
#
0
1
1###
50
0
#
0
#
0011
100
1
#
#
#
1010
1000
1
0
1
#
0111
1000
Action
Strength
Condition
#
1
#
#
#010
100
0
#
0
#
0011
100
Bidding Process
Bid(R,t) = β × specificity(R) × Strength(R,t)
Specificity(R)= number of non # / k
[M]
Rule
id
Condition
r1
#
1
#
r3
0
#
0
Action
Strength
#
#010
100
#
0011
100
β = 0.2
Bid(r1) = 0.2 × ¼ × 100 = 5
Bid(r3) = 0.2 × ½ × 100 = 10
r3 posts its message in the new message list.
Credit assignment: Bucket Brigade
r3
coupled
r5
executed
Environment
Bucket
10
Bucket
150
Reward
200
Credit assignment: Bucket Brigade
r3
Bucket
10
r5
Bucket
150
Environment
Reward
200
Genetic Algorithms
• Fitness = rule strength
• Parents: Strong classifiers (best, roulette
wheel, etc.)
• Mutation: alter parts of parent’s string
• Crossover: exchange parts of parents’ strings
• Offspring replaces a weak rule.
Genetic Algorithms (cont.)
Crossover
Parent 1
Parent 2
Crossover point
0
0
1
0
1
1
#
#
0
0
1
0
1
1
0
0
1
0
1
0
0
1
0
0
1
0
1
0
0
1
#
#
Mutation
Parent 1
Parent 2
0
0
1
0
1
1
#
#
0
0
1
0
1
0
#
#
1
0
1
0
0
1
0
0
1
0
1
0
0
1
0
0
Maze Environment
(Signal smell-ahead bump
heading score location)
40 5 f N 5 (1,2)
A
Environment
Message List
GF
Condition
Action Strengt
h
# >0 # # # #
GF
1000
# <0 # # # # ∧ TL
TL
1000
# <0 # # # # ∧ TR
TR
1000
References
• A Mathematical framework for Studying
Learning in Classifier Systems, John H.
Holland, Phsyca D, Vol 2, No 1-3, 1986, pp.
307-317
• A First Order Logic Classifier System, Drew
Mellor Gecco ’05