Chapter 7 METHODS OF INFERENCE 知識推論法 Expert Systems sstseng 1 7.1演繹與歸納( Deductive and Induction ) • 演繹( Deduction): 藉由前提假設而推論出結論 • 歸納(Induction): 一葉知秋,見微知柱; 從小處歸 納出一個大觀. • 直觀(Intuition): 尚未被證實的理論. • 啟發(Heuristics): 從既有經驗所獲得的規則. • Generate and test: 不斷的嘗試, 從錯誤中學習. Expert Systems sstseng 2 • Abduction: 從已成立的結論往回推論, 以 得導致此結論的前提. • Autoepitemic: Self-knowledge • Nonmonotonic: 舊有的知識可能被新的 知識更新,取代 • 類比(Analogy): 藉由比較方式, 類似的前 提會導致相同的結論(ex:醫療診斷) Expert Systems sstseng 3 三段論( Syllogism ) • 三段論( Syllogism )是一種邏輯論證, 容易理解且已被為整證明過. – 前提(Premise): 會寫程式的人都很聰明 – 前提(Premise): John會寫程式 – 結論(Conclusion): 因此, John很聰明. • 一般而言,三段論根據演繹的論證包含了 兩個前提和一個結論. Expert Systems sstseng 4 定言三段論(Categorical Syllogism) 定言命題的型態 形態 概要 意思 A E I O 所有S為P 沒有S為P 某些S為P 某些S不為P 完全肯定 完全否定 部分肯定 部分否定 Expert Systems sstseng 5 •三段論的標準形態(Standard form) 大前提:所有M為P 小前提:所有S為M 結論:所有S為P - P代表結論的「謂詞」(Predicate),又稱為「大詞」(Major term) - S代表結論的「主詞」(Subject),又稱作「小詞」(Minor term)。 - 含有大詞的前提稱為「大前提」(Major premise); - 含有小詞的前提稱為「小前提」(Minor premise)。 - M稱為「中詞」(Middle term) Expert Systems sstseng 6 模式 (Mood) • Patterns of Categorical Statement Figure-1 Figure-2 Figure-3 Figure-4 MP SM PM SM MP MS PM MS AAA-1 AAA-2 AAA-3 AAA-4 大前提 MP PM MP PM 小前提 SM SM MS MS 大前提 小前提 • 4種AAA模式 形態 Expert Systems sstseng 7 • ex: AAA-1 – 所有M為P 所有S為M ∴所有S為P • 我們利用 決策程序(decision procedure)來證明 論證是否有效 • 三段論運用維思圖 (Venn Diagrams)來驗證 Expert Systems sstseng 8 • ex: AAA-1的決策程序 所有M為P 所有S為M ∴所有S為P S P S P S P M M M (a)維思圖 (b)大前提後 (c)小前提後 Expert Systems sstseng 9 • ex: AEE-1的決策程序 所有M為P 沒有S為M ∴沒有S為P S P S P S P M M M (a)維思圖 (b)大前提後 (c)小前提後 Expert Systems sstseng 10 “某些”的表示規則 • 1. 如果該類別是空的,則填上較暗色彩. • 2. 完全肯定或否定的陳述句(Universal statement): A 和 E, 會先畫 • 3. 若該類別至少有一個物件,則以 *標示. • 4. 若兩個相鄰的類別沒有一個物件存在, 則用*標在線上. • 5. 已經塗上暗色的的部分不需要標上*. Expert Systems sstseng 11 ex: IAI-1的決策程序 某些P為M 所有M為S ∴某些P為S S P S P * M (a) 所有M是S Expert Systems sstseng M (b) 一些P是M 12 7.2 狀態與問題空間(State and problem spaces) • 樹狀結構( Tree ): 點(nodes), 線(edges) • 有向性(Directed)/無向性(Indirected) • 雙向圖(Digraph): 圖中所有的線都是 有方向性的 • 晶格(Lattice): 有方向性但沒有循環性 的圖 Expert Systems sstseng 13 • 藉由描述物件在圖中的行為來定義該圖, 此方法我們稱做“問題狀態空間” – – – – – – 初始狀態(Initial state) 可用的運算元(Operator) 狀態空間(State space) 路徑(Path) 目標測試(Goal test) 路徑成本(Path cost) Expert Systems sstseng 14 有限狀態機器 ( Finite State Machine ) • 用以辨識 WHILE和WRITE的有限狀態機器 H I L E W R T I 開始 B N G E not G not E I 成功 not I not N 錯誤 Expert Systems sstseng 15 在問題空間中尋解 (Finding solution in problem space) • 狀態空間( State space )可以視為一個 問題空間( problem space ). • 在問題空間裡求解需要找出一條有效的 路徑(開始->答案). • “猴子與香蕉問題”的問題空間 • 旅行推銷員問題(Traveling salesman problem) • 圖形演算法, AND-OR Trees…等等. Expert Systems sstseng 16 猴子在躺 椅上 Ex: 猴子與香蕉 跳下躺椅 觀察到躺椅 不在香蕉下 問題 • 假設: – 房子裡有一懸掛的香蕉 – 房子裡只有一張躺椅跟一把梯子 – 猴子無法直接拿到香蕉 • 指示: – – – – – 跳下躺椅 移動梯子 把梯子移到香蕉下的位置 爬上梯子 摘下香蕉 • 初始狀態: – 猴子在躺椅上 猴子在地 躺椅位於 板上 觀察到躺椅 香蕉下 在香蕉下 Expert Systems sstseng 移動躺椅 躺椅不位 猴子不在 於香蕉下 觀察到猴子 梯子上 不在梯子上 觀察到猴子 不在梯子上 移動猴子 猴子在梯 梯子不位 子上 觀察到梯子 於香蕉下 不在香蕉下 觀察到梯子 在香蕉下 移動梯子 梯子在香 蕉下 爬上梯子 猴子在梯 子上 摘下香蕉 猴子成功 得到香蕉 17 Ex:旅行推銷員問題 ( Travel Salesman Problem ) D B A C A (a) 旅行推銷員的問題描述 B A C C D A B D B B C A C AB D D B A C D B B C D C AC (b) 搜尋路徑(粗線是解答路徑) Expert Systems sstseng 18 非結構化問題 (Ill-structured problem) • 非結構化問題( Ill-structured problems ) 是指很多不確定因素的結合. – – – – – – 目標不明顯 問題空間範圍尚未被介定 問題狀態空間非離散的 中間狀態的不易實行 可用運算元未知 時間限制 Expert Systems sstseng 19 Ex:旅遊代理人 特徵 客戶的反應 目標不明顯 我在想要到底要去哪裡 問題空間範圍未被介定 我不確定要去哪裡 問題狀態不是離散的 我只是想去旅遊,目的地並不 重要 中間的狀態不易實行 我沒有足夠的錢去 狀態的可用運算元未知 我不知道怎麼可以籌到錢 時間限制 我必須儘快出發 Expert Systems sstseng 20 7.3 規則式推論(Rules of Inference) • 三段論( Syllogism ) 只是邏輯陳述的其中一 種方式. • 命題邏輯(Propositional logic) pq p______ q 這種命題邏輯的推論型態有很多種名子:direct reasoning(直接推論), modus ponens(離斷率), law of detachment(分離律), and assuming the antecedent(假設前提). Expert Systems sstseng 21 離斷率的真值表( Truth table for Modus Ponense ) p T T F F q T F T F p→q T F T T (p→q)p T F F F Expert Systems sstseng (p→q) p→q T T T T 22 Law of Inference Schemata 1.Law of Detachment p→q p ∴q 2.Law of the Contrapositive p→q ∴~q→~p 3. Law of Modus Tollens p→q ~q ∴~p 4.Chain Rules(Law of the Syllogism) p→q q→r ∴p→r 5.Law of Disjunctive Inference pq ~p ∴q ~(~p) ∴p 6.Law of the Double Negation Expert Systems sstseng pq ~q ∴p 23 7.De Morgan’s Law ~(pq) ∴~p ~q ~(pq) ∴~p ~q 8.Law of Simplification pq ∴p ~(pq) ∴~q 9.Law of Conjunction p q ∴pq 10.Law of Disjunctive Addition p ∴pq 11. Law of Conjunctive Argument ~(pq) p ∴~q ~(pq) q ∴~p Table 3.8 Some Rules of Inference for Propositional Logic Expert Systems sstseng 24 命題邏輯分解 ( Resolution in propositional Logic ) 1. 將命題(proposition)轉換成clause form形式. 2. 重複以下動作直到產生矛盾或是無法再繼續 (1) 選擇兩個clauses, 我們稱之為parent clauses (2) 將parent clauses作disjunction,產生出來的 clause稱為resolvent, 再把有L和~L這樣形式的 元素移除 (3) 若最後resolvent是空的(empty),則矛盾產生; 若否則再加入另一個clause作disjunction,同(2) Expert Systems sstseng 25 Given Axioms Converted to Clause Form p p 1. (p q) r ~p ~q (s t) q ~s q 3. ~t q 4. t Figure. t r 2. 5. 命題邏輯的facts (A Few Facts in Propositional Logic) Expert Systems sstseng 26 ~pˇ~qˇr ~r p ~p ˇ~q ~q ~t ˇq ~t Figure. t 命題邏輯之解 (Resolution in Propositional Logic) Expert Systems sstseng 27 加入量詞的推論 (Resolution with quantifiers) EX(form Nilsson): 任何會閱讀(Read)的皆具有讀寫能力(Literate). 海豚(Dolphin)沒有讀寫能力 一些海豚很聰明(Intelligent). 試證明:存在一些很聰明但沒有閱讀能力的海豚. Expert Systems sstseng 28 轉換: x [ R ( x ) -→ L(x)] x [ D ( x ) -→ -L ( x ) ] x[D(x) & To prove: I(x)] x[I(x)&-R(x)] Expert Systems sstseng 29 (1) - (4): x [- R ( x ) OR L ( x ) ] & y [ - D ( y ) OR - L ( y ) ] & D ( A ) & I ( A ) & z [- I ( z ) OR R ( z ) ] (5) - (9): C1=-R(x) OR L(x) C2=-D(y) OR -L(y) C3=D(A) C4=I(A) C5=-I(z) OR R(z) Expert Systems sstseng 30 • 第二敘述邏輯 (The second order logic) 的 量詞 (quantifiers)範圍含括函數符號 (function symbols)與敘述符號( predicate symbols) • 如果P是一文件中的任一敘述(predicate –則 – x =y = ( for every P [P(x) P(y) ] Expert Systems sstseng 31 7.4 推斷鏈 D3 Initial facts A2 A1 D2 B C D1 E Solution inference + inference +… + inference Chain backward Chaining(向後鏈結) forward Chaining(向前鏈結) 從事實來推論結果 假設可能的解答是成立的,再 尋求相關的證據證明之 Expert Systems sstseng 32 • 前向鏈結( Forward Chaining ): rule1: 大象(x) 哺乳類(x) rule2: 哺乳類(x) 動物(x) face:John 是一知大象. 大象(John) 成立 X=John (變數替代-Unification) 大象(x) 哺乳類(x) 哺乳類(John) 成立 X’=X=John 哺乳類(x) 動物(x’) 動物(John) 成立 Expert Systems sstseng 33 • 變數替代( Unification ) 將變數(variable)用事實(fact)取代,反覆的進行變 數替代最後即可得出結論. rule1:A1 and B1 C1 rule2:A2 and C1 D2 rule3: A3 and B2 D3 rule4:C1 and D3 G facts:A1 is true B1 is true A2 is true A3 is true B2 is true Expert Systems sstseng 34 • 前向推論( Forward reasoning ): {A1, A2, A3, B1, B2, B3} fire r1 {A1, A2, A3, B1, B2, B3, C1 } match {r1, r3} match {r1, r2, r3} match fire r2 {A1, A2, A3, B1, B2, B3, C1, D2} fire r3 {A1, A2, A3, B1, B2, B3, C1 D2, {r1, r2, r3} D3 } fire r4 {A1, A2, A3, B1, B2, B3, C1 D2, D3, {r1, r2, r3, r4} G} GOAL Expert Systems sstseng 35 rule1:A1 and B1 C1 rule2:A2 and C1 D2 rule3: A3 and B2 D3 rule4:C1 and D3 D4 rule5: C1 and D4 G’ facts: A1, A2, B1, B2, A3 Expert Systems sstseng 36 •後向推論( Backward reasoning ): 1. 假設 G’ 成立 R5 Verify C1 and D4 訊問使用者D4之值, 若為 False則G’不成立 R1 Verify A1 and B1 OK OK 2. 假設 G 成立 R4 Verify C1 and D3 R3 Verify A1 and B1 OK OK Verify A3 and B2 OK OK Expert Systems sstseng 37 A1 A2 B1 A3 B2 C1 D2 D3 ? G GOAL Expert Systems sstseng 38 •前向鏈結( Good application of forward chaining ) Goal Facts Broad and Not Deep or too many possible goals Expert Systems sstseng 39 •後向鏈結( Good application of backward chaining ) Narrow and Deep GOALS Facts Expert Systems sstseng 40 •前向鏈結( Forward Chaining ) Planning Monitoring Control Data-driven Explanation not facilitated •後向鏈結( Backward chaining ) Diagnosis Goal-driven Explanation facilitated Expert Systems sstseng 41 類比 (Analogy) • 將舊有相關的情境作為推論的參考 • Consider tic-tac-toe with values as a magic square (15 game) »6 »7 »2 1 5 9 8 3 4 • 18 game from set {2,3,4,5,6,7,8,9,10} • 21 game from set {3,4,5,6,7,8,9,10,11} Expert Systems sstseng 42 Nonmonotonic reasoning • 在nonmonotonic system中,隨著元素的增 加,並不一定會增加定理的數量. • 以一個簡單的例子來說,假設某一事實是 和時間相關的,若今時間改變了,則該事實 就不再是有效(令人信服)的了. Expert Systems sstseng 43 7.5 Reasoning Under Uncertainty (不確定性推論) • 不確定性 (Uncertainty)是指在作決策時缺少足夠的資訊. • 處理不確定性的理論有:Classical probability, Bayescian probability, Dempster-Shafer theory, and Zadeh’s fuzzy theory. • 在 MYCIN 和 PROSPECTOR的系統中,即使所有前提確 實證明了結論是未知的,也會產生很多結論. Expert Systems sstseng 44 導致不確定的錯誤 Example Turn the value off Turn value-1 Turn value-1 off Value is stuck Value is not stuck Turn value-1 to 5 Turn value-1 to 5.4 Turn value-1 to 5.4 or 6 or 0 Value-1 setting is 5.4 or 5.5 or 5.1 Value-1 setting is 7.5 Value-1 is not stuck because it’s never been stuck before Output is normal and so value-1 is in good condition Error Ambiguous Incomplete Incorrect False positive False negative Imprecise Inaccurate Unreliable Random error Systematic error Invalid induction Reason What value? Which way? Correct is on Value is not stuck Value is stuck Correct is 5.4 Correct is 9.2 Equipment error Statistical fluctuation Miscalibration Value is stuck Invalid deduction Value is stuck in open position • 假說是需被驗證的. • Type 1 error (false positive) 是指接受了一個不會成立(F)的假設. • Type 2 error (false negative) 是指否決了一個會被成立(T)的假設. Expert Systems sstseng 45 • 度量的誤差(Error of measurement) – 精確度(Precision) • 公釐量尺的精確度比公分量尺高. – 正確性(accuracy) Expert Systems sstseng 46 錯誤與歸納(Error & Induction) “歸納”的過程和“演繹”是相反的 火警警報聲響起 ∴ 有火災發生了. 加入一個更強烈的參數: 火警警報聲響起 & 聞道煙味 ∴有火災發生了. Expert Systems sstseng 47 雖然加入了一個強烈的參數, 但依然無法證明確 實發生火災. 衣服著火了 Expert Systems sstseng 48 演繹錯誤 (Deductive errors) p→q q ∴p If the value is in good condition, than the output is normal The output is normal ∴ The value is in good condition. Expert Systems sstseng 49 Baye’s Theorem (貝氏定理) • Conditional probability(條件機率), P(A | B) , 是指在B 事件成立的情況之下A事件發生的機率. Crash= Brand X(0.6)+ Not X(0.1)=0.7 • P( X|C) = P( C | X) P(X) = P(C) (0.75)(0.8) = 0.7 6 7 • 假設你擁有某不知名廠牌的硬碟壞掉了,那麼此硬碟是 X牌和其他非X牌的機率各是多少? Expert Systems sstseng 50 硬碟損壞的決策樹 Act Choose Brand X P(X)=0.8 Don’t Choose Brand X P(X’)=0.2 No Crash P(C’ | X’)=0.5 P(C’∩X’)=0.1 Crash P(C | X’)=0.5 P(C ∩X’)=0.1 Prior P(Hi) Crash P(C | X)=0.75 No Crash P(C’ | X)=0.25 P(C’ ∩ X)=0.2 P(C ∩X)=0.6 Conditional P(E | Hi ) Joint -P(E ∩ Hi ) =P(E | Hi ) P( Hi ) 0.1 P(X’ | C’) = 0.1+0.2 =1/3 0.1 P(X’ | C) = 0.1+0.6 =1/7 0.2 P(X | C’) = 0.2+0.1 =2/3 Expert Systems sstseng 0.6 P(X | C) = 0.6+0.1 =6/7 Posterior P(H i | E) = P (E ∩ Hi) iP(E∩Hi) 51 P(E∩Hi) = P(E | H i) P(Hi) P(E ∩Hj) P(E | Hj) P(Hj) j j P(E | Hi)P(Hi) = P(E) • Bayes’ Theorem(貝氏定理)利用決策樹作分 析,最常用於商業事務或社會科學. P(Hi | E) = • 探礦專家系統可決定何處是值得探勘的 Expert Systems sstseng 52 假設推論與向後歸納 (Hypothetical Reasoning and Backward Induction.) Probabilities No Oil Oil P(O’)=0.4 Prior P(O)=0.6 Subjective Opinion of Site - P (Hi) -Test +Test -Test +Test P(- | O’) P(+ | O’) P(- | O) P(+ | O) =0.9 =0.1 =0.2 =0.8 P(-∩O’) P(+∩O’) P(-∩O) P(+∩O) =0.36 =0.04 =0.12 =0.48 P(+)=P(+∩O)+P(+∩O’)=0.48+0.04=0.52 Conditional Seismic Test Result Joint P(E∩H) =P(E | Hi) P(Hi) P(-)=P(-∩O)+P(-∩O’)=0.12+0.36=0.48 Expert Systems sstseng 53 -Test +Test Probabilities P(-)=0.48 P(+)=0.52 Unconditional P (E) No Oil Oil No Oil Oil P(O’|-) P(O|-) P(O’|+) P(O|+) = (9)(4) = (2)(6) = (1)(4) = (8)(6) 0.48 0.48 = 3/4 = 1/4 0.52 0.52 = 1/13 Posterior of Site - P(Hi | E) P ( E| Hi) P (Hi) = P(E) = 12/13 P(-∩O) P(+∩O) P(-∩O) P(+∩O) Joint -P(E∩H) =0.36 =0.04 =0.12 =0.48 =P(Hi | E) P(E) Expert Systems sstseng 54 • • • • Oil release , if successful Drilling expense Seismic survey Expected payoff (success) $1250000 -$200000 -$50000 • 846153=1000000 *12/13 – 1000000*1/13 • Fail • -500000= 1000000*1/4- 1000000*3/4 • Expected payoff (total) • 416000= 846153*0.52 – 50000 * 0.48 Expert Systems sstseng 55 時態推論 (Temporal reasoning) 和Markov chain • 時態推論(Temporal reasoning): 推論的事 件和時間相關 • 時態邏輯(Temporal logic) • The system’s progression through a sequence of status is called a Stochastic process(隨機過程) if it is probabilistic. Expert Systems sstseng 56 – P11 P12 – P21 P22 – Pmn 是指從狀態m轉換到狀態n的機率. S = { P1,P2, …, Pn} where P1+P2+…+Pn= 1 S2 = S1 T S2 = [0.8,0.2] = [0.2,0.8] 0.1 0.9 0.6 0.4 Expert Systems sstseng 57 • 假設有10%的人擁有X牌的硬碟會再買該牌的 硬碟.60%的人不使用X牌硬碟但下次會買.問再 一段時間之後會有多少人使用X牌硬碟? S3 = [0.5,0.5], S4 = [0.35,0.65], S5 = [0.425,0.575], S6 = [0.3875,0.6125] S7 = [0.40625,0.59375], S8 = [0.396875,0.602125] Steady state matrix Expert Systems sstseng 58 The odds of belief • “這個病人渾身佈滿紅疹” • 提議A: “這個病人罹患麻疹” • P(A|B) :(在B事件下A成立的可信度)is not necessarily a probability if the events and propositions can not be repeated or has a math basis. Expert Systems sstseng 59 • 在C事件下A,B相斥的機率 =P(A|C)/ P(B|C) • If B = A’ – odds =P(A|C)/ P(A’|C) =P(A|C)/ (1-P(A|C) ) • Likelihood of P = 0.95 – Odds = .95/(1-.95) = 19 to 1 Expert Systems sstseng 60 Sufficiency and necessity Bayes’ Theorem is P(E | H)P(H) P(H|E) = P(E) Negation P(H’|E) = P(E | H’)P(H’) P(E) Expert Systems sstseng 61 P(H | E) P(E | H) P(H) = P(H’ | E) P(E | H’) P(H’) Defining the prior odds on H as P(H) O(H) = P(H’) P(H | E) O(H | E) = P(H’ | E) Likelihood ratio Expert Systems sstseng 62 P(E | H) LS= P(E | H’) O(H | E) = LS O(H) odds-likelihood form of Bayes’ Theorem. The factor LS is also called likelihood of sufficiency because if LS =∞ then the evidence E is logically sufficient for concluding that H is true. Expert Systems sstseng 63 Likelihood of necessity, LN, is defined similarly to LS as LN= O(H | E’) O(H) P(H | E’) P(H’ | E’) P(E’ | H) = P(E’ | H’) = P(H) P(H’) O(H | E’) = LN O(H) LN=0,then P(H | E’) = 0. This means that H must be false when E’ true. Thus if E is not present then H is false, which means that E is necessary for H. Expert Systems sstseng 64 LS 0 Small(0<LS<<1) 1 Large(1<<LS) Effect on Hypothesis H is false when E is true or E’ is necessary for concluding H E is unfavorable for concluding H E has no effect on belief of H E is favorable for concluding H E is logically sufficient for H or Observing E means H must be true Expert Systems sstseng 65 LN 0 small(0<LN<<1) 1 large(1<<LN) Effect on Hypothesis H is false when E is true or E is necessary for H Absence of E is unfavorable for concluding H Absence of E has no effect on H Absence of E is favorable of H Absence of E is logically sufficient for H Expert Systems sstseng 66 推論鏈中不確定性 (Uncertainty in inference chains) • Uncertainty may be present in rules, evidence used by the rules, or both. Expert Systems sstseng 67 Expert Inconsistency If LS > 1 then P(E | H’) < P(E | H) 1 – P(E | H’) > 1 – P(E | H) 1-P(E | H) LN= 1-P(E | H’) <1 Case 1:LS>1 and LN <1 Case 2:LS<1 and LN >1 Case 3:LS= LN = 1 Expert Systems sstseng 68 Exercise 1. 考慮以下的事實與規則,試以前向鏈結和後向鏈結描 述其推論過程。 事實: A1, A2, A3, A4, B1, B2 規則: R1: A1 and A3 --> C2 R2: A1 and B1 --> C1 R3: A2 and C2 --> D2 R4: A3 and B2 --> D3 R5: C1 and D2 --> G1 R6: B1 and B2 --> D4 R7: A1 and A2 and A3 --> D2 R8: C1 and D3 --> G2 R9: C2 and A4 --> G3 目標: G1, G2 and G3 Expert Systems sstseng 69