Novel Approaches to Optimised
Self-configuration in High
Performance Multiple Experts
A. F. R. Rahman M.C. Fairhurst and S. Hoque
BCL Technologies Inc.
University of Kent
USA
UK
Basic Problem Statement
• Given a number of experts working on the
same problem, is group decision superior to
individual decisions?
Is Democracy the answer?
• Infinite Number of Experts
• Each Expert Should be Competent
How Does It Relate to Character
Recognition?
Each Expert has its:
• Strengths and Weaknesses
• Peculiarities
• Fresh Approach to Feature Extraction
• Fresh Approach to Classification
• But NOT 100% Correct!
Practical Resource Constraints
Unfortunately, We Have Limited
• Number of Experts
• Number of Training Samples
• Feature Size
• Classification Time
• Memory Size
Solution
• Clever Algorithms to Exploit Experts
– Complimentary Information
– Redundancy: Check and Balance
– Simultaneous Use of Arbitrary Features and
Classification Routines
How are they Employed?
Expert1
Expert 2
Horizontal Systems
Expert n
How are they Employed?
Expert 1
Expert 2
Expert n
Vertical Systems
How are they Employed?
• Combined System:
– A hybrid of Horizontal and
VBertical
– More Complicated to
Analyse?
– Even more Complicated to
Optimise?
What to Optimise?
• Number of Experts in a
configuration
• Type of Expert in each Position in
the hierarchy
• Optimising Criteria
– Do we want a fast system?
Or
– Do we want an accurate System?
Proposed Methodology
• Genetic Algorithm: A Generalised
Search and Optimisation Method
• Problem Coding:
– Chromosome Structure
– Fitness Function
– Genetic Operators
Methodology
• Chromosome Structure: A
Classifier is a Machine Obeying a
Set of Production Rules. A
Generalised Rule is:
<classifier>::=<condition>:<message>
– <condition> part is a pattern matching
device
– <message> part is a feedback
mechanism
Methodology
• Fitness Function:
Fitness = Correct_Patterns/Total_Patterns
• Correct_Patterns corresponds to the number of
correctly identified patterns in one cycle
• Total_Patterns corresponds to the number of total
patterns being fed to the optimising process
Methodology
• Genetic Operators:
– Reproduction:
• Weighted Roulette Wheel (Goldberg)
• Stochastic Remainder Selection (Booker)
• Tournament Selection (Brindle)
– Crossover: Swapping at [1,l-1]
– Mutation: Random variation
• Single gene only
Selection of a Specific Problem
Expert 1
Expert 2
Expert 3
Expert 4
Decision Compilation
Selection of a Database
• Machine Printed Characters Extracted from
British Envelopes
• Collected Off-line
• Total 34 Classes (0-9, A-Z, no Distinction
between 0/O and I/1)
• Total Samples of Over 10,200 characters
• Size Normalised to 16X24
Performance of the Classifiers
Classifiers
% Error
BWS
1.76
FWS
1.52
MPC
3.90
MLP
1.66
Performance of the Combination
Classifier
Position
BWS
1
FWS
4
% Error
1.03
MPC
3
MLP
2
The Optimised Combination
Classifier
Position
BWS
Unused
FWS
2
% Error
0.92
MPC
1
MLP
3
Generality of the Solution:
Generation of a Vertical System
Input Pattern
Expert 1
Input Pattern
Expert 1
Expert 2
Expert 2
Expert 4
Expert 3
Decision Compilation
Classification Decision
Classification Decision
Optimization for the Vertical System
Optimized Parameters
BWS
Sub-set
size
FWS
Sub-set
size
MPC
Sub-set
size
MLP
Sub-set
size
2
10
4
8
1
5
3
2
Combined % Error: 1.01
Generality of the Solution:
Generation of a Horizontal System
Input pattern
Input Pattern
Decision Combination
Expert 1
Expert 2
Expert 2
Expert 3
Expert 3
Expert 4
Decision Compilation
Classification Decision
Decision Compilation
Classification Decision
Optimization for the Horizontal System
Optimized
Parameter
Weighting
Factor
BWS
FWS
MPC
MLP
Error %
0.14
0.53
0.11
0.22
0.92
Conclusion
• Multiple Expert Solutions can be made more
Robust by optimising these structures
• Optimisation is made with GA approach
• The adopted multiple expert configuration is
generic: it can produce both vertical and
horizontal systems (in addition to the hybrid
system)
• The optimization approach is generic: it man
optimize both vertical and horizontal systems (in
addition to the hybrid system)