Analytic Techniques For
Decision Support
ASMC – PDI
Orlando, FL
June 3, 2010
Ron Maccaroni
rmaccaroni@cox.net
1
Decision Support
Definition
Enabling decision makers to make informed
decisions that wisely and better utilize
resources, and improve program and
operational effectiveness.
Why is this important?
2
Why Should We Care About
Good Decision Support?
 Huge annual deficits contributing to a
$13+ trillion, and growing, federal debt.
 Trillions in unfunded liabilities.
 Medicare, Medicaid, and Social Security
costs will increase dramatically.
 Uncertain world climate can lead to future
unexpected events that increase financial
burden on Federal budget/programs.
 Growing financial burden on the States
and Localities for services and social
needs.
3
What It Means To YOU!
Today there are challenges in all areas
of government.
But!
There are also opportunities for those
who are prepared.
4
Objectives Of Presentation
Emphasize the need for relevant,
data-driven analysis to support
decision making.
Demonstrate the need and benefits
of using a structured approach to
analysis.
Discuss an example of a structured
analytical approach.
5
The “Watchwords”
 Accountability and Transparency
 GAO changed its name from “General
Accounting Office” to “Government
Accountability Office.”
 Accounting and auditing professions are now
referred to as the “accountability professions.”
 Many politicians, academicians, private sector
leaders, and “citizen intermediaries” refer to
both terms.
 Laws passed that call for either or both.
6
What’s Different Today?
The New Playing Field
Focus on achieving and measuring
results!
 Agencies must demonstrate they are getting
results when preparing budgets and reporting
annually.
 Citizens are more interested in seeing results
from public expenditures and meeting their
expectations.
 Greater use of the Internet for public access to
7
information on government operations.
Working For The People!
Budget of the U.S. Government
Fiscal Year 2011
 The American people deserve a Federal
Government that:
 Respects their tax dollars, and uses them
effectively and efficiently.
 Is transparent, fair, and responsive.
 Is constantly looking to streamline what works
and to eliminate what does not.
8
Drivers Of Stronger Analysis To
Support Decision Making
DRIVERS
NEW PLAYING FIELD
RESULTS
Higher Leadership and
Accountability
Expectations
The Management
Process Has Changed
• A Broader Role for the
CFO
Management Reform
Legislation
• Focus on
Performance and
Results--Tied To
Budget Decisions
• More and Better Data
Available
Presidential
Initiatives
• Emphasis on More
Timely and Better
Accounting for
Resources
• More Difficult Decisions
• Need and Opportunity for
Stronger Analysis
Ahead: Large and Growing Federal Budget Deficits
9
What Is The Focus Of
Decision Making
Starts The Mission . .
What should / needs to be done?
 For whom?
 Why is it important – what is the desired /
expected impact?

Which Drives Subsequent Decisions . .
How should it be done?
 By whom?
 When?
 At what cost?

10
. . . Leading to Performance
Management Decisions
Execute
Goal/Objective
Programs
 Outcomes – Results
 Customers
 Outputs – Products/Services
 Activities
 Inputs – Resources
 Budget
 Key questions:
 Is resource use having intended results/impact?
 Is the investment worth the results?
 How good is the management process?
11
Typical Types Of Analysis
Supporting Decisions
Problem/Issue
Budgetary Resources
Financial Condition
Business Processes
Cost / Benefit
Policy / Program Design Alternatives
Program Operations or Impact
12
How Was The Analysis
Done?
A decision-maker should always
know how an analysis was done in
order to:
 Know how solid it is.
 Be comfortable with and be able to
defend any decision made based on the
analysis.
13
So . . .
What’s In YOUR Analysis?
In conducting analysis, how do you
know:
 What question / issue / concern / need
you should focus on?
 How reliable, verifiable, relevant,
sufficient, and consistent the data you
depend on are?
 The best type of analysis to perform?
Are you at risk if you don’t know?
14
Basic Guidelines For
Decision Support
 Ask questions; be willing to wonder.
 Define the problem / concern / question /
need.
 Examine the evidence -- data.
 Analyze assumptions and biases.
 Avoid emotional reasoning.
 Don’t use either / or thinking, or over
generalize.
 Consider other interpretations.
 Tolerate uncertainty / questions.
15
Structured Approach To
Conducting Analysis
 Objective and systematic way of
collecting, examining / analyzing and
reporting information to support
decision-making.
 Conceptual way of thinking.
 Applies to all aspects of managing
programs, finances, and operations.
16
Structured Approach To
Conducting Analysis (cont’d)
 To be effective, analysts must:
 Understand organization’s business.
 Understand how analytic “need” relates to the
business.
 Apply sound, thorough analytical methods
specifically tied to the “need.”
 Provide useful information to support decisionmaking.
 Analysis for decision-making
leads to continual line of questioning.
17
Five Step Structured Analytic
Approach
Step 1 - Define
Questions
Step 2 - Identify Data Sources
Step 3 - Collect Data
Step 4 – Conduct Analysis
Step 5 – Present Results
18
Step 1
Defining Questions
 Critical starting point -- Question
will drive the entire analysis.
 Analysis must zero in on user’s
need.
 Critical considerations:
 The user’s business and role in it.
 What the user needs / wants to
know and why.
 How the results will be used for
decision-making.
 Thoroughness requires asking 6 key
questions = “what,” “where,”
“when,” “who,” “how,” and “why”
19
What Is The Question?
 How effective is
the highway
patrol at
increasing
safety?
20
What Are The Real Questions?
 How effective is the highway patrol . . .
 Do resources used give adequate security?
 Are fatalities per 100,000 miles going down?
 Is ratio of convictions/violations going up?
 . . . at increasing safety?
 Is the number of accidents going down?
 Are there fewer safety-related incidents?
 Is there less crime on the roads?
21
Step 2
Identify Data Required
 Key question = “What data do you need to
answer the question?”
 Data collected from:
Primary sources = you collect
Secondary sources = others collected
 Sources and quantity of data gather
depend on:
The questions to be answered.
The availability of data.
The time allowed for the analysis.
 If data doesn’t exist, revisit questions.
22
Consider Three Data Categories
 Budgetary
 Prospective
 Retrospective
 Program
 Strategic and annual plans and performance
 Program design, operation, and results
 Financial
 Cost of operations and financial position
(assets, liabilities, net worth, revenue,
expenses)
 Cost data by office, program, function, output
(product or service), or outcome (result)
23
The Data Quality Axiom
Assumption = All decision are data
driven.
The quality of the decision
can be no better than
the quality of the data supporting the
decision!
24
Step 3
Collecting The Data
(Sound like the easy step?)
 Key questions in deciding how to collect data:
Where and how are you going to get the data?
Will it enable you to answer the questions?
 Common data sources:
Records = reports, manual/automated files.
People = within and outside the organization.
 Time available key to source and amount.
 Verification important = reliability, completeness,
accuracy, relevancy.
25
Data Collection Hierarchy
TIME REQUIREMENT
Standard Agency Reports
Low
Automated Systems
Manual Files
Personal Interviews
Focus Groups
Surveys
High
26
Step 4
Analyzing The Data
 Analytic method = steps taken to
understand, display, or interpret data.
 Methodology dependent on question(s).
 Methods numerous and varied.
Important to know purpose / function of each
and when to use them.
 Review results of analysis:
Did it address question(s)?
How sensitive is it to internal / external
influences?
27
Analysis Axiom
The rigor of the analysis must
reflect the risk / significance of
the potential impact of the
decision!
28
Conduct Analysis Methods
Screening
There Categories - Which fits question(s)?
 Descriptive = What exists?
 How was the high-tech training program implemented?
 Normative = What exists compared to what was
expected?
 Were job-placement goals of the high-tech training
program met?
 Impact = Did the initiative achieve the intended
results?
 Did the high-tech training program have an effect on
job-placement rates?
29
Determining The Type Of
Analysis
 Primary Data Analysis
 You collect, assemble, and analyze data.
 Secondary Data Analysis
 You use data gathered and in some cases
analyzed prior to your own analysis.
 Evaluation Synthesis
 You use the combined results from two or
more previous analysis.
30
Analytical Methods
And Their Uses
Content
Trend
Statistical
X
X
X
X
X
-
-
Financial Condition and
Operations
X
X
X
X
X
-
-
Budgetary Resources
X
X
X
X
-
-
-
Business Processes
X
X
X
X
X
X
-
Cost-Benefit and CostEffectiveness
X
X
X
X
X
X
-
Policy and Program Design
Alternatives
X
X
X
X
X
X
X
Program Operations
X
X
X
X
X
-
-
Program Impact
X
-
X
X
-
X
X
Category
Problem/Issue
Case
Study
Benchmarking
BeforeAfter
Field
Experiment
31
Putting The Steps
Together - Example
Step 1
 Should $5M be added to inoculation
program advertising for face-to-face
contact to enhance its effectiveness?
Is current approach meeting its goals?
If not, why and what can/needs to be done?
If yes, can more aggressive goals be
set/met?
What additional benefits are expected
from face-to-face approach?
32
Putting The Steps Together –
Example (cont’d)
Step 2
 Program, budgetary, and financial
Step 3
 Reports, systems, files, and interviews
Step 4
 Normative and impact
 Primary, secondary, and evaluation synthesis
 Content, trend, statistical, case study,
benchmarking, before-after, field experiment
33
Step 5
Reporting Analytic Results
 Important phase of process.
Impact decision-making; bring about change.
 Reporting methods (written vs. oral vs. video)
have different advantages.
 Goal = convince audience that work done and
results reported is reasonable, appropriate, and
actionable.
 Answering analytic objective, i.e. question, is most
critical goal.
 Remember what drove the analysis – customer’s
need / interest / role, and planned use of results.
34
Structured Approach Steps
Interactive And Overlapping
35
Revalidating Questions
(Continuous throughout analysis!)
 Regularly re-check and validate questions.
 Questions may need to be modified, dropped, redefined
or added.
Analysis Feedback Loop
Define
Questions
Identify
Data
Sources
Collect
Data
Conduct
Analysis
Report
Results
36
Key Points to Remember
 Accountability expectations drive need for “hard”
decisions, and therefore critical analysis.
 Structured approach = objective/systematic way
of collecting, examining, and reporting
information.
 Five step structured approach:





Identify question(s)
Identify data
Collect data
Analyze data
Report results
 Steps highly interrelated – drives need to revisit
results of each step throughout process.
37
Summary
Analysis
 Growing need = support difficult decisions.
 Growing importance = support
performance and financial accountability.
 Best if:
Follows a structured approach.
Based on user’s/customer’s
questions/need.
Impacts decision-making/results.
38
Remember . . .
African Proverb
Every morning in Africa, a gazelle wakes
up. It knows it must run faster than the
fastest lion or it will be killed. Every
morning a lion wakes up. It knows it must
outrun the slowest gazelle, or it will
starve to death. It doesn’t matter
whether you are a lion or a gazelle.
When the sun comes up, you’d better
start running!
39