advertisement

1.3 Expected Loss, Decision Rules, and Risk Motivation: In the previous section, we introduced the loss of making a decision (taking an action). In this section, we consider the “expected” loss of making a decision. Two types of expected loss are considered: Bayesian expected loss Frequentist risk (a) Bayesian Expected Loss: Definition: The Bayesian expected loss of an action a is , a E L , a L , a dF L , a d where distribution of and F are the prior density and cumulative , respectively. Example 4 (continue): Let 1 0.99, 2 0.01 . Then, , a1 E L , a1 1 L1 , a1 2 L 2 , a1 0.99 500 0.011000 485 and , a2 E L , a2 1 L1 , a2 2 L 2 , a2 0.99 300 0.01 300 294 1 (b) Frequentist Risk: Definition: A (nonrandomized) decision rule R . If X is a function from into X x0 is observed, then x0 is the action that will be taken. Two decision rules, 1 and 2 , are considered equivalent if P 1 X 2 X 1, for every . Definition: The risk function of a decision rule X is defined by R , E L , X L , x dF X x | L , x f x | dx Definition: If R , 1 R , 2 , for all , , then the decision rule 1 with strict inequality for some R-better than the decision rule is 2 . A decision rule is admissible if there exists no R-better decision rule. On the other hand, a decision rule is inadmissible if there does exist an R-better decision rule. Note: A rule 1 is R-equivalent to 2 if R , 1 R , 2 , for all . 2 Example 4 (continue): R1 , a1 L1 , a1 500 300 L1 , a2 R1 , a2 and R 2 , a1 L 2 , a1 1000 300 L 2 , a2 R 2 , a2 . Therefore, both a1 and a2 are admissible. Example 5: Let X ~ N ,1, L , a a , 1 X X , 2 X 2 Note that E X and X . 2 Var X 1. Then, R , 1 E L , 1 X E L , X E X 2 Var X 1 and X X R , 2 E L , 2 X E L , E 2 2 X X E E 2 2 2 2 X 2 X 2 E 2 2 2 2 2 4 2 2 2 2 X X E E 2 2 4 2 2 2 Var ( X ) 2 X Var 0 4 4 4 2 2 1 2 4 4 3 2 Definition: The Bayes risk of a decision rule on with respect to a prior distribution is defined as r , E R , R , a d Example 5 (continue): Let ~ N 0, 2 , Then, 1 2 2 2 e 2 r , 1 E R , 1 E 1 1 and 1 2 r , 2 E R , 2 E 4 4 1 E 2 1 Var ( ) 4 4 4 4 1 2 4 4 4 2