EVOLUTIONARY ALGORITHMS
VS.
POKER GAMES
Yikan Chen (yc2r@virginia.edu)
Weikeng Qin (wq7yt@virginia.edu)
1
OUTLINE
Evolutionary
Algorithm
Poker!
2
EVOLUTIONARY ALGORITHM
3
EVOLUTIONARY ALGORITHM

Evolution Process
Crossover
Mutation
Natural Selection

Evolutionary Algorithm
4
EVOLUTIONARY ALGORITHM

Encoding and Crossover
1
1
1
0
0
1
1
0
0
1
0
0
1
0
1
1
0
1
0
0
0
1
1
0
1
1
1
0
1
0
1
1
5
EVOLUTIONARY ALGORITHM

Mutation
1
1
1
0
0
1
1
0
1
1
0
0
0
1
1
1
6
EVOLUTIONARY ALGORITHM

Natural Selection
Run the roulette-wheel selection based on the
fitness value of candidates
7
EVOLUTIONARY ALGORITHM

Important Parameters
 Crossover
rate
 Mutation rate
 Elite rate
 Fitness function

Demo
http://userweb.elec.gla.ac.uk/y/yunli/ga_demo/
8
EVOLUTIONARY ALGORITHM & POKER

AKQ 2-player game
 $1
blinds for each player
 Player1 bet or fold
 Player2 call or fold
9
EVOLUTIONARY ALGORITHM & POKER
Derive the optimal strategy using EA
 Chromosomal representations

 Fij:

fold threshold when Pi got Cardj
Card1
Card2
Card3
P1
2/3
0
0
P2
1
2/3
0
Fitness functions
10
EVOLUTIONARY ALGORITHM & POKER

Fitness functions
 Fi:
fitness function
 Wij: money won by candidate I against candidate j
11
12
13
EVOLUTIONARY ALGORITHM & POKER


Decreased fluctuation
Further decreased
fluctuation
400-500
Var(f11) ;
generations Var(f22)
Mean(f11);
Mean(f22)
Count only
wins
.065;
.067
.67;
.60
Penalize
failure
.037;
.035
.67;
.70
Penalize
Failure
heavier
.028;
.024
.67;
.74
14
EVOLUTIONARY ALGORITHM & POKER
Real Texas Hold’em
 Encoding Strategy (Turn and River)

 Hand
strength (player confidence)
 Fraction of opponent raise (opponent confidence)
 Total raise (profit)
15
EVOLUTIONARY ALGORITHM & POKER

Fitness Criterion
16
EVOLUTIONARY ALGORITHM & POKER

Performance
17
ARTIFICIAL NEURAL NETWORK: REVIEW
18
ARTIFICIAL NEURAL NETWORK: REVIEW
a1
a2
w1
w2
……
∑
f
output
wn
an
b
1
19
ARTIFICIAL NEURAL NETWORK: REVIEW
Input
output
Hidden Layer
20
E-ANN (EVOLUTIONARY ANN)

a
Simplest Encoding Method
b
c
d
d
c
b
a
21
NEAT E-ANN
http://www.cs.utexas.edu/users/nn/
 Neuro Evolution of Augmenting Topologies
 Encoding Strategy: Node-based

 Neuron
gene table
 Link gene table

Innovation number
 Global
database of innovations
 Each innovation has unique ID number
22
NEAT E-ANN
23
NEAT E-ANN

Mutation
 Perturb
weights
 Add a link gene
 Add a neuron gene

Crossover
 By
innovation number
24
NEAT E-ANN

Crossover
4
4
6
5
1
1
1->4
5
2
2
2->4
1
1->4
3
3->4
2
2->4
3
4
2->5
3
3->4
1
5
5->4
4
2->5
2
3
8
1->5
5
5->4
6
5->6
7
6->4
9
3>5
10
1->6
25
NEAT E-ANN

Crossover
4
6
5
1
1
1->4
2
2->4
3
3->4
2
4
2->5
5
5->4
3
6
5->6
7
6->4
8
1->5
9
3>5
10
1->6
26
E-ANN & POKER

Simplified Poker Model
 1-10
 Initial
credit: 10 chips
 One chip ante at the beginning
 Call, raise (1 chip each time), fold
 Tournament
27
E-ANN & POKER
Two player game
28
E-ANN & POKER
29
E-ANN & POKER

Four different types of opponents
Tight Aggressive (TA)
Loose Aggressive (LP)
Tight Passive (TP)
Loose Passive (LP)
30
E-ANN & POKER
α: min win probability to call
 β: min win probability to raise

31
E-ANN & POKER
A: player type
B: player action
32
E-ANN & POKER
33
E-ANN & POKER

Bluffing……
34
Thanks!
35