advertisement

Feature Selection with Branch and Bound Norman Poh Steps 1. Construct an ordered tree satisfying: where Jk means k variables to be eliminated and the order of Zp is determined by a discrimination criterion 2. Traverse the tree from right to left in depth-first search pattern 3. Evaluate the criterion at each level and sort them 4. Prune the tree The objective is to fine the optimal value: while keeping a constantly updated lower bound: Whenever the criterion evaluated for any node is less than the bound B, all nodes that are successors of that node also have criterion values less than B. So, we prune them. “6 choose 2” How to construct the order tree? • Determine the number of levels of the tree – In the “6 choose 2” examples, we have to eliminate 4 variables and so the number of levels is 4 • Each child node should have a number greater than it’s parent’s • The right-most leave node (the note at the last level) should reach the largest variable index (that is 6 in the “6 choose 2” example) • A number of examples are given next “5 choose 2” “5 choose 3” “6 choose 3” Your tutorial question: 1. Construct an ordered tree according to “4 choose 2” 3. Put the tree 2. Eliminate features x1, x2, x3 and x4 – calculate the criterion and sort them Reference • PATRENAHALLI M. NARENDRA and KEINOSUKE, FUKUNAGA , “A Branch and Bound Algorithm for Feature Subset Selection”, IEEE TRANSACTIONS ON COMPUTERS, VOL. C-26, NO. 9, SEPTEMBER 1977