Today’s Topics
• Representing Knowledge in a Logic
One way to get domain knowledge into a computer, which is
still a largely unsolved, major AI challenge
• Propositional Logic
Natural deduction, Resolution theorem proving
Satisfiability
Efficiency Techniques: Pure Symbols and Unit Clauses
• Predicate Logic (aka First-Order Predicate Calculus,
FOPC; also First-Order Logic, FOL)
Next lecture
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
1
Why Logic?
• Historically important in AI
• However, less practical ‘success stories’
than several other parts of AI
– Good, Old-Fashioned AI? (GOFAI)
– Importance 5, 10, 25 years from now?
• Probabilistic logic growing area (later in term)
• Useful in other parts of CS
– Formal analysis of programs and protocols,
computer security
– Relates to SQL of databases
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
2
Two Goals
1. Learn how to represent (simple) English
sentences in a formal representation
•
Need a way to enter ‘world facts’ into computers
in a manner that algo’s can use effectively
•
ML weak at making use of ‘domain knowledge,’
but some research on this (including mine)
2. Learn how to mechanically make sound
(ie, ‘correct’) inferences/deductions
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
3
Physical Symbol
Hypothesis
Allen
Newell
1927
-1992
(Newell & Simon, Turing Award Lecture, 1976)
Intelligence (human or machine) achieved
through the use of
1. Symbols that represent the significant aspects
of the given problem domain
2. Operations on these basic and compound
symbols that generate potential solutions
3. Search to find a solution among
the possibilities
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
4
FOPC
A well-defined representation
of knowledge (though maybe
too black-and-white)
Sound:
inferred knowledge correct
(‘logically follows’)
Complete:
all possible inferences
can be mechanically produced
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
5
Propositional Logic:
Subset of FOPC
Components of Prop Logic
Atomic sentences
P, Q, ItIsRaining, etc
like random variables, but are ‘truth valued’
Compound sentences
if P and Q are sentences, so are
(P)
¬P
Key ideas of logical
11/17/15
P ˄ Q
P ˅ Q
P→Q
P↔Q
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
reasoning easier to learn
in the simpler case of
propositional logic
6
Well-Formed Formula (wff’s)
Well formed?
P˄˅Q
¬¬¬P
P˄Q˅R
- should use parens, though: P ˄ (Q ˅ R)
We’ll later use
P ˄ Q ˄ ¬R → Z as ‘shorthand’ for (P ˄ Q ˄ (¬R)) → Z
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
7
Propositional Logic:
Connectives
Symbols Used
AND: ˄ & comma
OR:
˅
NOT: ¬
~
IMPLIES:
→
!
EQUIVALANCE: ↔
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
8
P → Q as a Venn Diagram
- Which One is Most Accurate?
P
Q
P
Q
P
Q
Q
11/17/15
P
Q
P
CS 540 - Fall 2015 (Shavlik©), Lecture 25, Week 11
9
P → Q as a Venn Diagram
- Which One is Most Accurate?
If we know P is true we can also infer Q is true
Knowing Q is false also tells us that P is false
But if P is false, we can’t infer anything about Q
Q
11/17/15
P
CS 540 - Fall 2015 (Shavlik©), Lecture 25, Week 11
10
Some Propositional
Logic Practice
Let
R = is it raining
H = it is hot outside
U = I’ll take my umbrella
“It is raining but not hot.”
R˄¬H
“It is neither raining nor hot.”
¬ (R ˅ H)
Remember, in logic
everything ‘evaluates’
to TRUE or FALSE
“It is not raining and it is not hot.”
¬ R ˄ ¬ H // Same as above via ‘DeMorgan’s Law’
“When it is raining, I take my umbrella.”
R→U
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
11
Interpretations and Models
https://en.wikipedia.org/wiki/Interpretation_(logic)
Duck or rabbit?
Technical note:
an interpretation assigns a truth-value
to each proposition we have
(like a ‘complete world state’)
models are interpretations that
make all of our wff’s true
Our mechanical-inference algorithm needs to be told
the ‘semantics’ (ie, ‘meaning’) of our atomic symbols.
It can then compute truth value of any compound wff.
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
12
Example: Models
Models of { P ˅ ¬Q , P → Q } ?
1. P = TRUE and Q = TRUE
2. P = FALSE and Q = FALSE
Ie, there are FOUR interpretations of
{ P, Q }, TWO of which are models
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
13
Some Definitions
A logical sentence is called
valid (or tautological)
if TRUE for ALL models
satisfiable
if TRUE for SOME models
unsatisfiable
if TRUE for NO models
Valid?
Satisfiable?
Unsatisfiable?
P
P ˅ ¬P
P ˄ ¬P
(P → Q) ↔ (¬P ˅ Q)
11/17/15
next slide
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
14
Using Truth Tables to
Judge Compound Sentences
P Q
F
F
F
T
T
F
T
T
P→Q
¬P ˅ Q
(P → Q) ↔ (¬P ˅ Q)
All possible models
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
15
Using Truth Tables to
Judge Compound Sentences
Given an interpretation, we can judge
the truth value of any compound WFF
by creating one row of a table like this
P Q
So TAUTOLOGICAL
P→Q
¬P ˅ Q
(P → Q) ↔ (¬P ˅ Q)
F
F
T
T
T
F
T
T
T
T
T
F
F
F
T
T
T
T
T
T
All possible models
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
16
‘Natural Deduction’
in Prop Logic
Inference rules allow us to deduce
NEW wff’s from known wff’s
- an alternative to generating
all or part of a truth table
Notation for inference rules (these will be our
operators/actions in a search process):
given these wffs are know to be true
we can logical deduce these wffs
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
17
Deduction in Prop Logic (2)
AND Elimination (AE)
wff1 ˄ wff2 ˄ … ˄ wffN
If a conjunct is true,
then so is each wff in it
wffi
AND Introduction (AI)
wff1 , wff2 , … , wffN
wff1 ˄ wff2 ˄ … ˄ wffN
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
If we know a bunch of
wff’s are true, their
conjunction is also true
18
Deduction in Prop Logic (3)
OR Elimination (OE) – not an inference rule
wff1 ˅ wff2 ˅ … ˅ wffN
nothing can be deduced
If a disjunct is true,
we cannot deduce
anything
OR Introduction (OI)
wff1
wff1 ˅ wff2
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
If we know a wff’s is
true, we can ‘disjoin’ it
with any other wff and
that new wff is also true
19
Deduction in Prop Logic (4)
Double Negation Elimination (DNE)
¬ ¬ wff
Two negations
cancel out
wff
Double Negation Introduction (DNI)
wff
If we ‘need’ a negation
¬ ¬ wff
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
sign, we can introduce
TWO
20
Deduction in Prop Logic (5)
Equivalence Elimination (EE)
wff1 ↔ wff2
wff1 → wff2 ˄ wff2 → wff1
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
21
Deduction in Prop Logic (6)
Modus Ponens (MP)
wff1 → wff2 , wff1
wff2
If the IF is true in an
implication, then we
can deduce the THEN
Modus Ponens (alternate form)
¬ wff1 ˅ wff2 , wff1
wff2
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
22
Deduction in Prop Logic (7)
Unit Resolution (UR)
wff1 ˅ wff2 , ¬ wff2
wff1
Given we know wff2 is
false, it cannot satisfy
the OR
Unit Resolution (alternate form)
¬ wff1 → wff2 , ¬ wff2
Aside: this is called
‘modus tollens’
wff1
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
23
Deduction in Prop Logic (8)
Resolution (Rez)
wff1 ˅ wff2 , ¬ wff2 ˅ wff3
wff1 ˅ wff3
Can understand via a
‘case analysis’
(ie, wff2 is either true
or false, in one case
we know wff1 and the
other we know wff3)
Resolution is really the ONLY inference rule we
need, IF we pre-process our given knowledge
and create a ‘canonical’ form called conjunctive
normal form (CNF) – more later
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
24
Deduction in Prop Logic (9 )
Another Way to View the Resolution Rule
wff1 ˅ wff2 , ¬ wff2 ˅ wff3
wff1 ˅ wff3
Convert to implications
¬ wff1  wff2 , wff2  wff3
¬ wff1  wff3
Let wff4 = ¬ wff1
wff4  wff2 , wff2  wff3
wff4  wff3
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
25
Some IMPLICIT Rules
(we’ll assume some special-purpose
s/w handles these outside of our search algo;
in paper-and-pencil questions, no need to justify
using them)
wff1 ˄ wff2
wff2 ˄ wff1
wff
(wff)
11/17/15
wff1 ˅ wff2
wff2 ˅ wff1
(wff)
wff
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
26
EXAMPLE: Searching for Proofs
Given: (1) P
Show:
R
(2) Q
(3) P ˄ Q → R
State:
Initial State:
Goal Test:
Actions:
Search
Strategy:
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
27
EXAMPLE: Searching for Proofs
Given: (1) P
Show:
R
State:
(2) Q
(3) P ˄ Q → R
the wff’s we know
(deduction only ADDs to this list)
Initial State: the givens
Goal Test:
the wff ‘to show’ is in the state
Actions:
our inference rules
Search
Strategy:
some sort of best-first search?
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
28
Searching for a Proof
AI on 1 and 2
1: P
2: Q
3: P ˄ Q → R
OI on 1 and 2
1, 2, 3
4: P ˄ Q
...
1, 2, 3
4: P ˅ Q
DNI on 1
1, 2, 3
4: ¬¬P
MP on 3 and 4
1, 2, 3, 4
5: R
11/17/15
In logic, we only ADD
wffs, so each node could
simply show the NEW
wffss that are added to all
the wff’s of all ancestors
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
29
We’ll Search on ‘Scratch Paper’ and
Only Show the SOLUTION PATH
Given: (1) P
Show:
S
(2) ¬ Q
# WFF
1
P
2 ¬Q
3 P → (R ˄ W)
4 R˄¬Q→S
5 R˄W
6 R
7 R˄¬Q
8 S
11/17/15
(3) P → (R ˄ W)
(4) R ˄ ¬ Q → S
Justification
given
given
given
given
MP on 1 and 3
AE on 5
AI on 2 and 6
MP on 4 and 7
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
30
Resolution Theorem Proving
(how computers do logical inferencing)
• First put all our wff’s in clausal form
(aka, conjunctive normal form; CNF)
– Essentially, use many of our inference rules in some
preprocessing code (more details later)
– Results in an AND of ORs with some NOTs inside
(¬A ˅ B ˅ ¬Z) ˄ (A) ˄ … ˄ (D ˅ ¬F ˅ S ˅ ¬W)
– NEGATE the wff we want to prove
– Use the RESOLUTION RULE to derive a contradiction, ie
… ˄ (false) ˄ …
– If successful (might not be), we have proven the wff
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
31
Resolution Theorem Proving
Recap (basis of Prolog)
To show wff1 given some other wff’s in CNF
assume ¬ wff1
demonstrate this leads to a contradiction
via something of this form
wff ˅ false, ¬ wff ˅ false
false ˅ false
or equivalently
wff, ¬ wff
□
11/17/15
□ is
used to
represent
the
EMPTY
clause
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
32
A Convenient Notation for Searching for
Resolution Proofs (same problem as previous example)
P
¬Q
¬R˅Q˅S
¬P˅W
¬S
¬P˅R
¬P˅Q˅S
¬P˅S
S
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
□
We could
also
simply
leave this
box empty
33
Putting WFF’s into
Clausal/CNF Form (pg 253-254 of text)
1. Eliminate all the ↔’s using EE rule
2. Eliminate →’s using P → Q same as ¬ P ˅ Q
3. Push NOT signs next to atomic symbols using
i. ¬ ¬ P
becomes P
ii. ¬(P ˄ Q) becomes ¬P ˅ ¬ Q // DeMorgan’s Law
iii. ¬(P ˅ Q) becomes ¬P ˄ ¬ Q // DeMorgan’s Law
4. Distribute ORs over ANDs using
(P ˅ (Q ˄ R)) becomes ((P ˅ Q) ˄ (P ˅ R))
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
34
EXAMPLE: Putting WFF’s into
Clausal/CNF Form
(P ˄ R → ¬ Q) ˄ (Q ↔ R)
1.
Eliminate all the ↔’s using EE rule
(P ˄ R → ¬ Q) ˄ (Q → R) ˄ (R → Q)
2.
Eliminate →’s using P → Q same as ¬ P ˅ Q
(¬ (P ˄ R) ˅ ¬ Q) ˄ (¬ Q ˅ R) ˄ (¬ R ˅ Q)
3.
Push NOT signs next to atomic symbols using …
(¬ P ˅ ¬ R ˅ ¬ Q) ˄ (¬ Q ˅ R) ˄ (¬ R ˅ Q)
4.
Distribute ORs over ANDs using
No change (ie, not needed in this example)
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
35
More Practice
WFFs
p˄r ↔q
w˅¬z→a
c˄d→e˄¬f
m˄g→¬b˅h
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
36
More Practice
Good to remember that
(a ˄ b ˄ c ˄ d) → e becomes
¬a˅¬b˅¬c˅¬d˅e
and p → (q ˄ r) same as
(p → q) ˄ (p → r)
CNF (recall: combined by AND)
WFFs
11/17/15
(p ˄ r) ↔ q
¬p˅¬r˅q
¬q˅ p
¬q˅ r
(w ˅ ¬ z) → a
¬w˅a
z˅a
(c ˄ d) → (e ˄ ¬ f)
¬c˅¬d˅ e
¬c˅¬d˅¬f
(m ˄ g) → (¬ b ˅ h)
¬m˅¬g˅¬b˅h
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
Each
row is
called a
clause
37
Satisfying a Set of WFFs
Sometimes we want to find a
MODEL of a set of WFFs
– Eg, a solution to a set of logical constraints
– Recall a model is a setting of the truth values of
the atomic symbols that makes all the wff’s true
– Satisfiability is fundamental to
NP completeness in CS theory
(https://en.wikipedia.org/wiki/Boolean_satisfiability_problem)
11/24/15
CS 540 - Fall 2015 (Shavlik©), Lecture 26, Week 12
38
Searching for Models
• If our set of WFF’s has N propositions,
we might need to consider 2N ‘world states’
• Can set up as a search task
Set A=true
Set B=true
Set A=false
Set B=false
• Some often-useful special cases greatly help
– one is that we meet satisfy all wff’s without
needing to set ALL the prop’s (‘short circuiting’)
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
39
Useful Preprocessing Tricks for
Satisfiability (recall ‘constraint propagation’ in Sudoku)
We have a set of clauses (ie, WFFs in CNF)
Pure Symbol: if an atomic symbol always appears with the same ‘sign’
(eg, only P appears or only  P appears), then we can set its truth
value, so no need to try both TRUE and FALSE
Unit Clause Propagation: if a clause has only ONE item in it,
then we have a ‘forced move,’ eg
… { p  q  r  …   z } … { r } … { b  …   r } …
forces us to set r = false and we get
… { p  q  false  …   z } … { true } … { b  …  true } …
which simplifies to
… { p  q  …   z } …
11/24/15
CS 540 - Fall 2015 (Shavlik©), Lecture 26, Week 12
40
Example
(Q  ¬ A  ¬ R)
(P  ¬ R)
(P  ¬ B  ¬ Q)
(¬ P  R)
(A)
A must be true
simplify
(Q  false  ¬ R)
(P  ¬ R)
(P  ¬ B  ¬ Q)
(¬ P  R)
(true)
(Q  ¬ R)
(P  ¬ R)
(P  ¬ B  ¬ Q)
(¬ P  R)
(P  ¬ R)
(true  ¬ R)
(P  ¬ R)
(Q  ¬ R)
(P  ¬ R)
(¬ P  R)
(¬ P  R)
(¬ P  R)
simplify
let B = false
(Q  ¬ R)
(P  ¬ R)
(P  true  ¬ Q)
(¬ P  R)
simplify
let Q = true
Now only 22 combo’s to try instead of 25
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
41
Where We Are
• Have a well-defined, though simple, logic
• Can represent ‘world/domain knowledge’
(or the rules of chess, taxes, calculus, etc)
• We have mechanisms for determining what logically
follows (don’t want to derive EVERYTHING that follows
since unbounded, eg P, ¬¬P, ¬¬¬¬P, etc)
– truth tables (brute-force enumeration)
– ‘natural deduction’
– resolution theorem proving
• Can search for models of sets of WFFs (‘satisfiability’)
11/17/15
CS 540 - Fall 2015 (Shavlik©), Lecture 24, Week 11
42