20 KEYS TO
BETTER
SIMULATING
Master Économie et Affaires Internationales
Paris Dauphine -October 2007
Dr. Ramón Mahía
Professor of Applied Economics Department
www.uam.es/ramon.mahia
20 KEYS TO
BETTER
SIMULATING

STRUCTURE OF DOCUMENT AND EXPOSITION
Not a technical document
 Our experience in credible simulation for
complex systems in the real world
Text focused on know-how to do useful real
simulations rather than technical skills
 Simulation as a way of using econometrics
in a useful way with analytical restrictions
20 KEYS TO
BETTER
SIMULATING
STRUCTURE OF DOCUMENT AND EXPOSITION
 I. What does Simulation mean?
 Concept
 Simulation Vs other topics
 Some Simulation “last-names”
II. Basic elements of a Simulation Model
III. 20 Keys for an Efficient Simulation
20 KEYS TO
BETTER
SIMULATING
“To
I. WHAT DOES SIMULATION MEAN?
represent
something,
feigning
or
imitating what it is not”
 Simplified representation of a real complex
system useful for:
 Understanding the working of a real system
 Experimenting with, for evaluating different
strategies to be developed on it.
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
Real System:
Simulated System:
A lot of elements
inter-related
Few elements and
selected relations
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
Arrival of powerful computers and flexible
programming systems = generalized use of
simulation

Modeler
Use of complex
techniques
User
Use of simple
interfaces
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
Simulation with a model is a wide
spread exercise:

•Macro economic relations: trade, labor,
supply – demand,
•Financial markets: treasury ship, stock
exchange, commodity prices…
•Technology innovation processes
•Business strategy games
•Business organization games…
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
FINANCE: Wall Street Raider
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
MARKETING: MarktStrat
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
BUSSINES ORGANISATION: EIS Game
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
MACRO-POLICY: National Budget Simulation
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
 Borders
between
Simulation,
Forecast
and
Structural Analysis are diffuse.
Structural
Analysis
Forecasting
Simulation
 But the analytical approach, technical resources,
and ways of use permit us to distinguishes it.
20 KEYS TO
BETTER
SIMULATING

I. WHAT DOES SIMULATION MEAN?
Simulations Vs. Optimization

Optimization
systems
concentrates
mainly
on
reaching a well predefined objective given a set of
restrictions.

Simulation is an open strategy that use the links
between inputs and outputs without setting a priori
what must be considered an optimum solution.

That’s why we usually say that simulation models are
not “resolved”, but “runned”.
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
 Simulations Vs. Optimization: Design car routes to
pick up employees to the factory from distant locations
OPTIMISATION
Objective function:
• Minimize time
Restrictions:
• 3 cars
• 1 hour to finish
• 13 passengers
Results:
• Route design
SIMULATION
Inputs:
• Cars
•time to finish
• # of passengers
Ouputs (results):
• Route design
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
 Deterministic (MKT Mix effects evaluation)
Deterministic Inputs (controlled values):
• Advertising effort
• Price policy
• Distribution policy
• Sales Force
 Random (Agricultural Crop Yield)
Random Inputs (not controlled values):
• Climate conditions
• Plagues
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
Regression analysis
 Deterministic + Random (MKT Mix evaluation)
Deterministic Inputs (controlled values):
• Advertising effort
• Price policy
• Distribution policy
• Sales Force
Inputs to be randomly modeled (forecasted)
• MKT Mix of existing or new competitors
• Economic conditions of country
• Market Demand (2nd stage input)
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
 Static Vs. Dynamic: Does “passing of time” result in a key
variable (even an input) for the simulation system?
 Example: Time as a basic input of a simulation system (two
real examples):
 Economic impact of an immigration flow will not only depend on the
amount of immigration, but in the speed of it.
 Effects on prices or trade flows of a tariff removal between two
areas, will drastically change if you consider a gradual removal or
not.
20 KEYS TO
BETTER
SIMULATING
I. WHAT DOES SIMULATION MEAN?
 Time as a basic input…..(example from Femise)
The inputs and outputs
of the model will be
aggregated in a single
year
The inputs and outputs
of the model will be
proportional distributed
in a basis of a 7 years
scenario
20 KEYS TO
BETTER
SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL?
 (i) System to be analyzed.
The collection of elements and its interactions which is
trying to be analysed by means of the simulation.
It is critical to identify the system (or sub-system) of
interest and concentrate the effort on a suitable
dimension, but…..
 ….without missing the links with the rest of the systems
or other collateral sub-systems
20 KEYS TO
BETTER
SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL
 (i) System to be analyzed
TRADE BARRIES
TRADE FLOWS
TRADE SUPPORT
POLITICAL ISSUES
PRODUCTION STRUCTURES
MACRO ECONOMIC ENVIROMENT
20 KEYS TO
BETTER
SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL
 (ii) Analytical – Mathematical Model
(an example for
international trade market equilibrium):
•Equilibrium reached making equal the inverse functions
of imports and exports revenues
J
I
  h (Q , Y )dQ    u (Z ,W )dZ
j 1
Ad-Valorem
Tariff Matrix
I
j
j
j
j
J
1
j 1 1  AD j
MAX 
J
i 1
h
j
i
i
(Q j , Y j )dQ j
I
J
i
j
Import
Inverse
Function
I
   u i ( Z i , Wi )dZ j    Tij X ij    TTij XX ij
i 1
Export
Inverse
Function
j 1 i 1
Ad-Quantum
Tariff Matrix
j 1 i 1
Existing
Quota
Regimes
20 KEYS TO
BETTER
SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL
 (iii) Inputs, Outputs,
(coherent design of inputs
and outputs)
Tariffs to be removed
Changes in trade flows
Span of time
Changes in trade prices
Productivities
Changes in Va by sector
Fiscal pressure
Changes in employment
Exchange rate
Changes in fiscal revenues
Saving rate
……
……
20 KEYS TO
BETTER
SIMULATING
II. BASIC ELEMENTS OF A SIMULATION MODEL
 (v) Interface:
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
Try to understand the problem
Try another approach
Is there a
simulation
problem?
NO
YES
Prepare an offer to the client
If you have convinced…
Information recollection and treatment
Mathematical formulation of the model
Computer interface design
Validation of simulation model
NO
Calibrate
Does it
work?
YES
Design of Simulation scenarios
Results analysis and validation of the simulation process
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
1.- Be sure that you a have a
simulation problem





Have to deal with a multivariate problem
Can clearly identify inputs and outputs
Input variables can vary in a wide range of values
Output variables clearly respond to changes in inputs
There is not a single scenario to be established
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
2.- Offer feasibility of its
prospects and involve the end
users in the whole proposal
 Don’t make the mistake of offer maximum proposals
and bear in mind its realism.
 The final user usually prefers work proposals in
which they feel an active part from the beginning
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
3.- Program enough time to study
deeply the system to be
analysed
 The largest part of the technical decisions regarding
the estimation, calibrating, scenario and interface
design are conditioned by the comprehension of the
elements and interrelations of the system to be
analysed.
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
4.- Integrate in your team
theoretical experts been
familiar with the system
 “Research needs more heads than hands”.
 Save time, which could be used to improve the
technical issues for the simulation model
 Help even for taking technical decisions
 Establish a theoretical rigour to the whole of the
analysis
 Give reality to the simulation mechanism
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
5.- Prioritise the wishes of users
in all the stages of the
construction of the simulation
model and take their advices
 Nobody is interested in a technically refined tool that
does not serve their interests
 The users only want the model to be adjusted to their
demands, nor the other way round.
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
6.- Study “in deep” the work
carried out by others
 Originality must never be an aim in itself
 Explore previous main sources of data - limitations of
the exercise - different techniques available
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
7.- Dedicate time to analyze all of
the available data






"Measure twice, and cut once".
Use homogeneous data
Choose carefully the samples
Assess the data provided by the end user
Be extremely scrupulous in the handling of data
Agree with the user that data to be used responds
faithfully to the reality perceived by him.
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
8.- Explore the analytical mathematical techniques that
best adapt to the system
 Its adaptation towards capturing the specific
phenomena observed in the specific system
 its feasibility in calculating
 its flexibility
 the quantity of theoretical hypothesis required
 its robustness towards eventual changes
 its simplicity
 the available resources
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
9.- Try to adapt the analytical
technique to the problem and
not the other way round
 Guide the technical procedures by the suitability of
the real characteristics observed in the system.
 The technique used is only valid if it works,
independently
of
the
“objective”
scientific
considerations
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
10.- Do not complicate the
technical models if it does not
lead to clear benefits
 "If your intention is to discover the truth, do it with
simplicity and leave the elegance for the tailors."
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
11.- Take care about the forecast
power of the model
 Try to anticipate the needs with regard to the
prediction at the time of choosing the variables
 Evaluate with the focus on “cross validation” its
vulnerability to eventual prediction errors in the key
variables
 Avoid using single results as regards to the
prediction; use always intervals of variation or
alternative values with probabilities of occurrence.
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
12.- If a prediction exercise is
necessary try to involve the
user criteria in the interface
 "If you have to forecast, forecast often."
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
13.- If inferential statistics are
used, check the sensitivity of
the system to changes in the
estimations
 When the analytical procedure means the use of
statistical inference, the system can depend more or
less critically on these estimations
 Check the sensitivity of the results of the system
against variations in the estimated coeficients
 Check the robustness of the estimations with
question such as changes in the data sample
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
14.- Do not underestimate the
political or qualitative aspects
of which experts advise you
 The systems are not capable of being modelled using
purely quantitative structures.
 It is not a question of choosing between a
quantitative or qualitative approximation, rather than
knowing how to combine both.
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
15.- Let simplicity guide the
design of the interface in all of
its score
 "The majority of the fundamental ideas in science are
essentially simple and, as a general rule, they can be
expressed in language understandable to everyone."
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
16.- Call for software
professionals into the design of
the interface
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
17.- Establish different levels of
user for the handling of the
interface
 Directors, politicians, media technicians, technical
experts, etc.
 “There is no inept user, only badly designed
systems”.
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
18.- Instruct the users on the
correct use of the system
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
19.- Limit the use of the model to
real scenarios
 The greatest part of the simulation systems can be
labelled as “rubbish in / rubbish out”.
 Design an interface which stops, or at least warns the
user, of possible errors in the design of scenarios.
20 KEYS TO
BETTER
SIMULATING
III. 20 KEYS
20.- Ensure the perfect display of
the results
 Pretest: Before giving the final ok to designs in this
area, carry out several tests among your work
colleagues to make sure that the results are
understandable and do not oppose the proposals
suggested to improve it.