Assignment 2 Observations
Mausam
© Mausam
1
The SAT Encoding
• Induced Subgraph Isomorphism
– NP complete problem
– Applications: similarities between chemical compounds;
protein-interaction networks, social networks, circuit design..
• Variables
– X(i,j) denotes that ith node (in smaller graph) mapped to jth
node (in larger graph)
– Self edge: If node i has a self-edge and node j doesn’t
• remove X(i,j)
– Self-edge: If node j has a self-edge and node i doesn’t
• remove X(i,j)
© Mausam
2
The SAT encoding
• Each node maps to exactly one other node
– X(i,1)  X(i,2)  X(i,3)…  X(i,n)
– Pairs of  X(i,1)   X(i,2);  X(i,1)   X(i,3) …
• Each node in larger graph maps to at most one node
– Pairs of X(1,j)X(2,j); X(1,j)X(3,j); …
• Disallowed pairs of mappings
– If edge(i1,i2) and no edge(j1,j2): X(i1,j1)X(i2,j2);
– If no edge(i1,i2) and edge(j1,j2): X(i1,j1)X(i2,j2);
– If edge(i1,i2) and edge(j2,j1): X(i1,j1)X(i2,j2);
• Overall – O(nm) vars, O(n2m2) clauses
© Mausam
3
Fun optimizations
U
A
C
B
S
P
Yes. A mapping is: M(A) = S,
M(B) = Q, M(C) = R
R
T
The edges from P to other nodes
don‘t matter since no node in G
got mapped to P.
Q
• Degree trick: Can B ever map to P/S/R?
– No: Indegree of B is greater than indegree of P/S/R
– Remove X(B,P), X(B,S), X(B,R)
• Common neighbor constraint: Can A-U and C-T?
–
–
–
–
• …
© Mausam
They both don’t have an edge
They both have outdegrees 1
No: because AC have common neighbor 1; UT 0
X(A,U)X(C,T)
4
Observations
• People solved ~100+80 node problems…
– For an NP hard problem that’s not too bad, is it?
• Paresh’s observations
– adding redundant clauses helps miniSAT for tough
problems.
© Mausam
5
Best Competitors
• Kim Wallmark
• Alan Ludwig
• Haroon Barri
© Mausam
6
Assignment 3 Discussion
Mausam
© Mausam
7
BlackJack: State Space
•
•
•
•
•
MinSum: the minimum sum of the player's hand so far.
NumAces: the number of Aces in the player's hand so far.
dMinSum: the minimum sum of the dealer's hand so far.
dNumAces: the number of Aces in the dealer's hand so far.
isTwoCards: a Boolean that represents that the player's hand has
just two cards so far.
• isBlackJack: a Boolean that represents that the player got a
BlackJack.
• pair: has value between 0 and 10. Zero indicates that the player
doesn't have a pair. Any other value i indicates that the player has
two cards of value i.
• turn: a Boolean to indicate whether it is player's turn or dealer's.
© Mausam
8
BlackJack: Action Space
• Stand: Turn = 0
• Hit: get a new card with prob… update minSum,
numAces, isTwoCard, pair
• Double: get a new card, update minSum,
numAces, etc.;
– Turn = 1.
– Allowed when isTwoCard = 1
• Default: run default dealer policy.
– Allowed with turn = 1
• Split: tricky!
– Allowed with pair > 0; isTwoCard = 1
© Mausam
9
Equations
Q( s, HIT )   P( s' | s, HIT )V ( s' )
s'
Q( s, STAND)   P( s' | s, STAND)V ( s' )
s'
Q( s, DEF )   P( s' | s, DEF )V ( s' )
s'
Q( s, DOUB )  2 P( s' | s, DOUB )V ( s' )
s'
© Mausam
10
BlackJack Split (say only three cards)
• (c,c)
• Q((c,c), SPLIT)
– [(c,1) & (c,1)] p1p1
– [(c,1) & (c,2)] p1p2
– [(c,1) & (c,3)] p1p3
– [(c,2) & (c,1)] p2p1
– [(c,2) & (c,2)] p2p2
– [(c,2) & (c,3)] p2p3
– [(c,3) & (c,1)] p3p1
– [(c,3) & (c,2)] p3p2
– [(c,3) & (c,3)] p3p3
– 2p1p1V(c,1)
– p1p2V(c,1)
–
–
–
–
–
–
–
+ p1p2V(c,2)
p1p3V(c,1) +
p1p3V(c,3)
p2p1V(c,1) + p2p1V(c,2)
2p2p2V(c,2)
p2p3V(c,3) + p2p3V(c,3)
p1p3V(c,1) +
p1p3V(c,3)
p2p3V(c,3) + p2p3V(c,3)
2p3p3V(c,3)
2p1(p1+p2+p3) = 2p1
© Mausam
11
Equations
Q( s, HIT )   P( s' | s, HIT )V ( s' )
s'
Q( s, STAND)   P( s' | s, STAND)V ( s' )
s'
Q( s, DEF )   P( s' | s, DEF )V ( s' )
s'
Q( s, DOUB )  2 P( s' | s, DOUB )V ( s' )
s'
Q((c, c,...), SPLIT )  2 P(c' )V (c, c' ,...)
c'
© Mausam
12
Value of Terminal States
• If isTwoCards = 1, minSum=11, numAces = 1
– Return 1.5
• If minSum>21
– Return -1
• If minSum < 21 and dMinSum > 21
– Return 1
• …
• …
© Mausam
13
Can you solve it with Expectimax?
• No
– Because there is possibility of infinite loop
• Values are well-formed
– Loop of size n becomes less probable as n increases
– (see beginning example from lecture on MDPs)
– EXCEPTION: Q(1010, split) for p> 0.5
• Solution: use Value Iteration/Policy Iteration
© Mausam
14
Optimizations
• Dealer’s policy is fixed. So, run the Markov chain ahead
of time to generate a table
–
–
–
–
–
–
•
•
•
•
Pr(dealer gets 17|first card=c)
Pr(dealer gets 18|first card=c)
Pr(dealer gets 19|first card=c)
Pr(dealer gets 20|first card=c)
Pr(dealer gets 21|first card=c)
Pr(dealer busts|first card=c)
Remove dMinSum, dNumAces, turn.
Add dFirstCard
Remove Default action
Change the equations for double/stand based on
computed probabilities
© Mausam
15
Comments
• numAces – not required. Convert to hasAce
– Only first ace is relevant. After that each ace = 1
• Handle infinite splits by allowing say, 10 splits
– Should work fine in practice
• Modification: have the bet amount in state space
– Needn’t multiply by 2 in equations for doubling
– Need to use this info when computing terminal rewards
© Mausam
16
Difference in solutions?
• In real BJ: Dealer and player BJ is a push
– In our BJ: player wins
• ???
• In real BJ: there is a 4/6/8 card deck
– In our BJ: infinite cards
• You can easily modify code to handle 1st case
– 2nd case: Monte carlo sampling!
© Mausam
17
Should you play our BlackJack?
• Expected reward before game starts
– SUM[Pr(game)*expected reward of the game]
• Deal three cards… and use V(s) to compute
exp reward
• My expected reward: $0.058
– Yes! We should play the game
© Mausam
18
Discussion
Mausam
© Mausam
19
Followup Courses
• 546 Machine Learning/Data Mining
• 515 Probabilistic Graphical Models
• 517 Natural Language Processing
• ??? AI/Statistics for Big Data
• 571 Robotics
• 579 Intelligent control
• 574 Special Topics in AI
• 528 Computational Neuroscience
•© Mausam
510 Human computer Interaction
20
Final Exam
• 50% objective
–
–
–
–
True/false
Multiple choice
Fill in the blanks
Match the following…
• 25% short answer questions
– 1 word – 5 line responses…
– opinions/strengths/weaknesses/etc…
• 25% Problem solving
– Problem modeling; short calculations…
© Mausam
21
Why do an exam?
• Goal: revise and analyze the whole course
• Expected preparation time
– ~1 day
– Go over lecture slides
– Read parts of the notes that are unclear…
© Mausam
22
What worked
• Programming Assignments
– You hated them
– You loved them
– You learned through them
• Written Assignments(?)
– Forced us to be regular
• Classes
– I absolutely enjoyed the interaction!
• Emails/bulletin boards
• Videos(?)
© Mausam
23
What didn’t work
• Programming environment
– People didn’t know unix
– People didn’t know Java!
• Packed
– Once someone got behind, it got tough to recover
• Workload
© Mausam
24
Thanks
• To all of you
– who contributed to helping each other through
newsgroup and other questions
– who asked terrific questions in class
– who gave regular feedback on class, assignments,
grading, organization
© Mausam
25