Heuristic Programming in
Business Solutions
ISECON’02
Kurt F. Lauckner
Department of Computer Science
Eastern Michigan University
csc_lauckner@online.emich.edu
1
What is “Soft Computing”
Often called Heuristic programming
Includes:
Expert Systems and Fuzzy Logic
Neural Networks
Evolutionary Systems
Genetic Algorithms
Genetic Programming
Other Evolutionary Structures
2
Expert Systems
Expert systems are also called Rule-based
systems.
Expert’s expertise is built into the program
through a collection of rules.
The desired program functions at the same level
as the human expert.
The rules are typically of the form:
If (some condition) then (some action)
Example: If (gas near empty AND going on long trip)
then (stop at gas station AND fill the gas tank AND
check the oil).
EXCON: An expert system used by DEC
3
Expert Systems
Two major parts of an expert system:
The knowledge base: The collection of rules
that make up the expert system.
The inference engine: A program that uses
the rules by making several passes over them.
On each pass, the inference engine looks for all rules
whose condition is satisfied (if part).
It then takes the action (then part) and makes another
pass over all the rules looking for matching condition.
This goes on until no rules’ conditions are matched.
The results are all those action parts left.
4
Expert Systems
Inference engines can pass through the rules
in different directions:
Forward chaining: Going from a rule’s
condition to a rule’s action and using the
action as a new condition.
Backward chaining: Goes in the other
direction.
Example: Medical doctors use both.
Forward chaining: Going to the doctor with symptoms
(stomach pain). The doctor will come up with a diagnosis
(ulcer).
Backward chaining: The doctor asks if patient has been eating5
green apples knowing green apples cause stomach aches.
Expert System -- Case Study
Business Resource Software, Inc. (BRS) has a
series of expert systems products that are a
Knowledge Base of Business Experts. Their
clientele include:
American Express
AT&T
Back-In-A-Flash
Boeing
Century 21
Dun & Bradstreet
E.I.DuPont
Eastman Kodak
Ernst & Young
Federal Express
Harvard University
IBM Corp.
J.C.Penny
McDonalds
Motorola
New York Life
Olympus
Pfizer
Roadway Express
Shell Oil
Texas Instruments
Toyota
Wells Fargo Bank
Xerox
6
BSR -- Expert System Products
 Plan Write Expert Edition -Evaluates business idea and
documents business plan with an
Expert Assistant.
 Plan Write for Marketing-Produces a complete, fully
documented marketing plan with
defined objectives.
 Plan Write for Pricing -- Helps
you to set the optimal price on
any product or service.
 Insight for Sales Strategy -Analyzes complex sales,
identifies roadblocks, rates the
potential for closing the sale,
provides a pipeline of all
forecasted sales.
 Plan Write for Hi-Tech
Marketing -- A special version
 Plan Write for Sales -designed to assist the high
Develops and documents a plan
technology industries to produce
for selling including strategy and
a complete, fully documented
tactics.
marketing plan with defined
objectives and tactical lists for http://www.business-strategy-brs.com/customers.asp
7
implementation.
Fuzzy Logic
Fuzzy logic comprises fuzzy sets and
approximate reasoning
A fuzzy “fact” is any assertion or piece of
information, and can have a “degree of truth”,
usually a value between 0 and 1
Fuzziness: “A type of imprecision which is
associated with ... classes in which there is no
sharp transition from membership to nonmembership” - Zadeh (1970)
8
Linguistic Variables
Linguistic variable: “a variable whose values
are words or sentences in a natural or artificial
language.” - Zadeh
Linguistic variables translate ordinary
language into logical or numerical statements
Imprecision of linguistic variables makes
them useful for reasoning
9
Linguistic Variables
In fuzzy set theory
we can translate the
body temperatures of
patients into linguistic
values with an
appropriate degree of
membership µ.
µ(x)
1
low normal
raised
With the linguistic variable fever,
fuzzy membership functions can
be defined: --low
--normal
--raised
--strong_fever
strong_fever
0
10
96°F
98°F
100°F 102°F 104°F 106°F 108°F 110°F
x
Fuzzy Logic -- Case Study
Economic red flags in risk analysis and audit
planning
Touche Ross [1974], Coopers and Lybrand
[1977], and Price Waterhouse [1985] released
a list of red flags for auditors
Deshmukh and Romine describe a fuzzy
number based spreadsheet approach in
assessing the risk of management fraud using
red flags http://www.swcollege.com/acct/jac/jac12/jac12_article1.html
11
Fuzzy Logic -- Case Study
 Uses FuziCalc® which
provides a spreadsheet
based environment that
incorporates fuzzy numbers
in the calculations
 Define fuzzy sets:
 Strong internal control
 Weak internal control
 Define that the internal
control is effective if it
detects or prevents 90% of
the errors or irregularities
 80% effective control may
have a membership of 0.9
in a fuzzy set of "strong
internal control" and a
membership of 0.1 in the
fuzzy set "weak internal
control”
 These concepts are useful in
measuring and quantifying
red flags
12
Fuzzy Logic -- Case Study
 “The Loebbecke, Eining, and Willingham [1989] model
stipulates that the probability of management fraud is a
function of three factors: condition, attitude, and motivation.
 “An effort must be made to aggregate red flags for each factor
and then combine these three factors to determine the
probability of management fraud.
 “There are no statistical formulas to do this.
 “The fuzzy number based spreadsheet can help accomplish
this by mathematically manipulating fuzzy numbers.
 “If each red flag is considered equally important then the
beliefs for all red flags are equally weighed and the belief for
the Condition factor can be calculated”
13
Neural Networks
Neuron: Basic building-block of the brain.
There are several specialized types, but all have
the same basic structure:
The basic structure of an animal neuron.
14
What is topology of NNs
NNs consist of processing elements (PEs) and
weighted connections
15
What NNs are and why they are
useful
Connection weights store the information
Weight values usually determined by learning
procedure
Each PE acts independently of all others
Each PE relies only on local information
Connection pattern provides redundancy and
facilitates fault tolerance
The most common NN uses backpropagation
16
Processing Element
X1
X2
Wj 2
Wj 1
Yj = f(X,W)
X3 Wj 3
qj
Yj
Wj n
Xn
Propagation Function
(Linear Combiner)
q = f(X,W)
Example:
qj 
n
x w
i
i1
ji
Activation Function
Example (Sigmoid):
1
Yj 
 qj
1 e
17
Neural Network -- Case Study
Real Estate Example--Home price estimator
The 17 inputs include:
Number of bedrooms, bathrooms, & other rooms.
Garage type (attached, detached or none)
Patio (yes or no)
Air conditioning
Specialized windows (high R factor, security)
Area code
One hidden layer with 8 nodes
18
Real Estate Case Study (Con’t)
Using 100 different homes
With 17 inputs and selling price of the home
Used EasyNN a Windows under $40 neural
network program shell
Trained network for several hours
Results of homes not known to the neural
network were estimated to within 2% of
actual pricing
19
LVQ or Kohonen Networks
LVQ (Learning Vector Quantizer) neural
networks are often called Kohonen networks
Probably second only to backpropagation in
number of applications
No rigorous mathematical derivation
Seem to be more biologically oriented than
most paradigms
Reduce dimensionality of inputs
LVQ-I Makes use of unsupervised learning
Network learns from input patterns alone
20
Self-Organizing Feature Maps
 Primary features of input cause
corresponding local responses in the
output PE field.
 Are non-linear mappings of input space
onto the output PE space (field).
The mapping is referred to as a selforganizing feature maps
Extensions of LVQ-I resemble the way
certain areas of the brain are organized
21
General Comments about LVQ-I
Main accomplishments of LVQ-I networks is
they cluster input data patterns into classes,
thus reducing dimensionality of data (e.g.,
automatically parameterizes the data)
Input vectors are usually normalized
Weight vector normalization is done for all
weights connected to a given output PE from
all input PEs
22
Kohonen Neural Network -- Case Study
Marketing: A campaign is being run to increase sales of a
product. It is intended to do a targeted mailshot to individuals
most likely to be interested in the product. There is information
available from a consumer survey which includes information
about current purchasers of the product. An unsupervised net is
trained using information from the consumer survey. The groups
formed are examined and all individuals in the groups rich in
existing purchasers are used as targets for the mailshot.
Data Cleaning: Data being extracted from a database is being
analysed but unfortunately the data is too inconsistent for the
analysis technique. An unsupervised net is used to create groups
and the analysis is based on the individual groups instead of the
entire database.
http://www.neusciences.com/Technologies/nn_unsupervised.htm
23
Evolution--Darwin
“…if variations useful to any organic being
do occur, assuredly individuals thus
characterized will have the best chance of
being preserved in the struggle for life; and
from the strong principle of inheritance they
will tend to produce offspring similarly
characterized. This principle of preservation,
I have called, for the sake of brevity, Natural
Selection.”
--C. Darwin, 1859
24
Evolution--Darwin
Translating Darwin’s statement, the four
essential preconditions for the occurrence
of evolution by natural selection are:
 Reproduction of individuals in the population
 Variation that affects the likelihood of
survival of individuals
Heredity in reproduction (e.g.,like begets like)
 Competition caused by finite resources.
--Banzhaf
25
Evolutionary Algorithms
Basic Ingredients of EAs:
 Populations of solutions
 Innovation operations (e.g., mutation)
 Conservation operations (e.g.,crossover)
 Quality differentials (e.g.,graded fitness
function, richness of differences)
 Selection (e.g., differences in population
are used to select fittest.
26
Genetic Algorithm--Holland
 Uses binary genome of fixed length
1 0 0 1 1 1 0 1 0
 Crossover most common and usually of
the is one-point crossover variety
Parents
1 0 0 1 1 1 0 1 0
1 1 0 1 0 1 1 1 1
1 0 0 1 1 1 1 1 1
1 1 0 1 0 1 0 1 0
Offspring
Mutation used very sparingly
 Classifier systems used to select fittest
consist of three parts:
 Rule-based programming language
 Simulated market economy
 Genetic algorithm
27
Genetic Algorithm -- Case Study
Improve model for tactical asset allocation
Improve model for international equity
strategies
Improvements of up to 82% in cumulative
portfolio value as compared with passive
benchmark model
28
Evolutionary Programming
Essential Common Themes:
Stochastic decision making: GP uses pseudorandom numbers to mimic the randomness of
natural evolution.
Program structures: GP assembles variable
length program structures from basic units called
functions and terminals.
Genetic operators: GP transforms the initial
programs in the population using genetic operators
(e.g. crossover, mutation, reproduction)
Simulated evolution of a population by means
29
of fitness-based selection.
Primitives of Genetic Programs
Terminal set: terminals are the inputs to the
GP program (named by fact they are terminals
of a tree structure)
30
GP Primitives (Con’t)
Function set: these are the statements,
operators, and functions available to the GP
system.
Boolean functions ( AND, NOT, OR, XOR etc.)
Arithmetic functions (plus, minus, multiply etc.)
Variable assignment functions ( x :=3)
Indexed memory functions
Conditional statements (IF, THEN, ELSE, CASE)
Control transfer statements (GOTO, CALL, JMP)
Loop statements (WHILE…DO, REPEAT…)
Subroutines (for example a robot: turn left, stop…) 31
Executable Program Structures
Functions and terminals must be assembled
into some structure that can be executed and
evaluated.
Two methods of initializing tree structure:
Full method of tree initialization
Pick functions as nodes until maximum depth reached
Pick terminals (leaves) of the tree from terminal set
Grow method of tree initialization
Pick randomly from functions and terminal sets
If function selected continue with that branch
If terminal selected the branch ends
32
Execution of Structure
In the tree structure use the pre-order
traversal to test fitness of all individuals of the
population
Apply selection algorithm
Use crossover, mutation and reproduction to
create new population
Continue until termination criterion met or
maximum number of generations reached
33
Genetic Programming -- Case Study
GP paradigm tool STROGANOFF (1990s)
Used in predicting USD/SF currency exchange
rates
Used in predicting movements of the Dow
Jones industrial index
Used in predicting movements of the
Nikkei225 index of the Tokyo Stock Exchange
Other applications: derivative markets,
banking and finance, marketing
34