Certification Requirements for the Level 1 AIE Analyst
Overview
Hubbard Decision Research promotes the use of Applied Information Economics by qualified individuals.
The AIE certification ensures organizations that use AIE that the certified individual is, in fact, proficient
with the AIE method. There are four levels of certification in AIE each proving an increasing proficiency
with the AIE method. This document explains the detailed requirements to pass both the written test
and the PEC so that candidates can prepare themselves. A Level 1 Analyst is defined as follows:
“A level 1 analyst is qualified to assist on risk/return analysis project and AIE implementation
projects. The level-1 analyst can build decision models, run Monte Carlo simulations, compute
information values, and execute empirical studies. They understand various financial analysis
techniques and sampling methodologies. A level-1 analyst works under the supervision of a level 2
analyst. Certification requires completion of level-1 training, a score of 85% on the level-1 Written
Exam covering the AIE process and financial and statistical methods, and completion of the level-1
Practical Experience Checklist. “
In other words, it is the objective of the Level 1 Analyst certification to ensure that an individual is
capable of performing all the tasks described above. The material required to prepare for the written
test are PEC are:
1. Attendance of the AIE training seminar (in rare cases, Level 1 training may be waved subject to
the judgment of the assigned evaluator)
2. Reviewing printed training material provided in their Level 1 training
3. The book “How to Measure Anything: Finding the Value of Intangibles in Business”
4. Practice with the AIE tools, especially the AIE Monte Carlo tool with VIA calculations
5. Optional: The candidate may review any introductory statistics or finance text if they feel they
are too unfamiliar with these basic concepts
Written Test
During the proctored written test, the candidate will be expected to perform each of the following tasks.
The candidate will need a score of 85% or better to pass this portion of certification.
1.
Demonstrate proficiency with basic cost/benefit modeling including:
a. Building one or more decision models based on written case explanations and oral
interviews of the proctor
b. Explaining how to elicit definitions of intangibles from clients by using the clarification
chain, thought experiment or other methods of clarifying measurement definitions
c. Identify common exclusions and error in business cases
d. Explaining and using basic financial calculations like NPV and IRR
2. Demonstrate proficiency with calibration concepts including:
a. Explaining the calibration process for both ranges and binary questions
3.
4.
5.
6.
b. Computing and compare expected vs. actual percent correct
c. Explaining methods for improving calibrated estimates including the equivalent bet and
other methods
d. How to address certain common questions or objections about calibration
e. Using the Excel calibration score template
Demonstrate proficiency with modeling uncertainty by:
a. Calculating simple probabilities of events given other simple probabilities and conditions
b. Explaining differences among the five distribution types used in the AIE tool and
situations where they would each be used or potential issues with their use in certain
situations
c. Conducting and interpret the Monte Carlo simulation of a decision in an example
d. Using the decision model and Monte Carlo results to identify potential risk mitigation
and value enhancement strategies
e. Eliciting an investment boundary from a client (portrayed by test proctor) using the AIE
investment boundary template
f. Set up a simple conditional probability simulation in a Monte Carlo
Demonstrate proficiency with information value calculations including:
a. Explaining opportunity loss and expected opportunity loss
b. Computing and explaining information values for basic binary decision problems
c. Computing information values with the AIE tools and interpret the results including the
explanation of each of the columns in the tool
d. Computing a marginal EVPI
Demonstrate proficiency with measurement methods
a. Given an EVPI and threshold of two or more particular uncertain variables, identify and
explain potential measurement methods including
i. Decomposition
ii. Instinctive Bayesian
iii. Non-Bayesian empirical methods (see list in 5.b below)
iv. Bayesian methods (see list in 5.b below)
b. Given AIE empirical methods templates, explain and use empirical methods including:
i. Computing a 90% CI of the mean of a population with small or large samples
ii. Computing a 90% CI of the proportion of a population having some property
based on a sample of the population
iii. Use of the method for “measuring to a threshold”
iv. Computing the 90% CI of the difference between a test group and control group
in a controlled experiment
v. Use of a simple (not a range) Bayesian inversion for binary probabilities
Demonstrate familiarity the components of an AIE risk/return analysis project including:
a. Explaining the basic AIE philosophy and approach to measurement
b. Explain the roles in a project and a workshop
c. Explain the types of workshops
d. Explain the phases and objectives of the project
7. Show familiarity with (not actually compute) more advanced AIE concepts by
a. Explaining in narrative form one or more potential optimization approaches to the case
study investment
b. Explaining the use of a classification chart
Practical Experience Checklist
The candidate will be assigned the following tasks to be completed with minimal direct supervision or
correction as judged by the evaluator
1. One or more parts of the business model will be assigned to the candidate
2. The candidate will be asked to audit the model with errors (known to the evaluator but not to
the candidate) included
3. The candidate will be asked to perform Monte Carlo simulations and VIA
4. The candidate will be asked to perform some part of an actual empirical measurement
5. The candidate will be asked to facilitate part of a workshop
6. The candidate will become calibrated
7. The candidate will complete one or more parts of the final deliverable
Each of these will be evaluated as follows:
A.
B.
C.
D.
Task was completed at an acceptable level without intervention by evaluator
Task was completed at an acceptable level after some intervention by evaluator
Task was completed only after significant supervision
Task could not be completed by candidate
The requirements for successful completion of the PEC are:
1.
At least 4 of the tasks must be evaluated as an “A” and the remainder must be evaluated as no
lower than “B”
2. “C” and “D” evaluated tasks must be repeated until they can be raised to meet the requirements
in #1 above