Decision Support
Building Analytical
Skills
ASMC – PDI
Orlando, FL
June 3, 2010
Ron Maccaroni
rmaccaroni@cox.net
1
Decision Support
Definition
Enabling decision makers to make informed
decisions that wisely and better utilize
resources, and improve program and
operational effectiveness.
Why is this important?
2
Why Should We Care About
Better Decision Support?
 Huge annual deficits contributing to an
$13+ trillion debt.
 Trillions in unfunded liabilities.
 Medicare, Medicaid, and Social Security
costs will increase dramatically.
 Uncertain world climate can lead to future
unexpected events that increase financial
burden on Federal budget/programs.
 Growing financial burden on the States
and Localities.
3
Objectives of Presentation
Emphasize the need for relevant,
data-driven analysis to support
decision making.
Demonstrate the need and benefits
of using a structured approach to
analysis.
Discuss an example of a structured
analytical approach.
4
The New Watchwords
 Accountability and Transparency
 GAO changed its name from “General
Accounting Office” to “Government
Accountability Office.”
 Accounting and auditing professions are now
referred to as the “accountability professions.”
 Many politicians, academicians, private sector
leaders, and “citizen intermediaries” refer to
both terms.
 Laws passed that call for either or both.
5
The New Playing Field
What’s Different Today?
Focus on achieving and measuring
results!
 Agencies must demonstrate they are getting
results when preparing budgets and reporting
annually.
 Citizens are more interested in seeing results
from public expenditures and meeting their
expectations.
 Greater use of the Internet for public access to
6
information on government operations.
Working for the People!
 The American people deserve a Federal
Government that:
 Respects their tax dollars, and uses them
effectively and efficiently.
 Is transparent, fair, and responsive.
 Is constantly looking to streamline what works
and to eliminate what does not.
Budget of the U.S. Government
Fiscal Year 2011
7
Drivers Of Stronger Analysis To
Support Decision-making
DRIVERS
NEW PLAYING FIELD
RESULTS
Higher Leadership and
Accountability
Expectations
The Management
Process Has Changed
• A Broader Role for the
CFO
Management Reform
Legislation
• Focus on
Performance and
Results--Tied To
Budget Decisions
• More and Better Data
Available
Presidential
Initiatives
• Emphasis on More
Timely and Better
Accounting for
Resources
• More Difficult Decisions
• Need and Opportunity for
Stronger Analysis
Ahead: Large and Growing Federal Budget Deficits
8
Federal Management Process Has
Changed
ASSESSING
RESULTS:
Various
REPORTING:
CFO Act +, GPRA,
FMFIA
PLANNING:
GPRA
BUDGETING:
GPRA
9
The Mission
Should Drive Decision Making
Starting Decisions . . .
What should / do I need to do?
 For whom?
 Why is it important – what impact do I
want to have?

Drive Subsequent Decisions . . .
How will I do it?
 When?
 At what cost?

10
Decision Making and
Performance Management
Execute
Goal/Objective
Programs
Outcomes – Results
 Customers
 Outputs – Products/Services
 Activities
 Inputs – Resources
 Budget

 Key questions:
 Is the resource use having the intended results/impact?
 Is the investment worth the results?
 How good is the management process?
11
Where Analysis is Needed to
Support Decisions
 Problem/Issue
 Budgetary Resources
 Financial Condition
 Business Processes
 Cost/Benefit
 Policy/Program Delivery Alternatives
 Program Operations and Impact
12
How Was the Analysis
Done?
A decision-maker should always
know how an analysis was done in
order to:
 Know how solid it is.
 Be comfortable with and be able to
defend any decision made based on the
analysis.
13
So . . .
What’s in YOUR Analysis?
Questions on which your future may
depend:
 In conducting analysis, how do you
know:
 What question/issue/concern/need you
should focus on?
 How reliable, verifiable, relevant, sufficient,
and consistent the data you depend on are?
 The best type of analysis to perform?
 Are you at risk if you don’t know?
14
Keys to Decision Support
 Critical Thinking
 Creativity
 Broad Perspective
 Analysis
 Qualitative
 Quantitative
 Communications Skills
15
Guidelines
 Ask questions; be willing to wonder.
 Define the problem / concern / question /
need.
 Examine the evidence.
 Analyze assumptions and biases.
 Avoid emotional reasoning.
 Don’t use either/or thinking, or over
generalize.
 Consider other interpretations.
 Tolerate uncertainty/questions.
16
What it Means to YOU!
Today there are challenges in all areas of
government.
But!
There are also opportunities for those who
are prepared.
17
Structured Approach to
Conducting Analysis
 Objective and systematic way of
collecting, examining and reporting
information to support decisionmaking.
 Conceptual way of thinking.
 Applies to all aspects of managing
programs, finances, and operations.
18
Structured Approach to
Conducting Analysis (cont’d)
 To be effective, analysts must:
 Understand organization’s business.
 Understand how analytic “need” relates to the
business.
 Apply sound, thorough analytical methods
specifically tied to the “need.”
 Provide useful information to support decisionmaking.
 Analysis for decision-making
leads to continual line of questioning.
19
Five Step Structured Analytic
Approach
Step 1 - Define
Questions
Step 2 - Identify Data Sources
Step 3 - Collect Data
Step 4 – Conduct Analysis
Step 5 – Present Results
20
Step 1
Defining Questions
 Question will drive the entire
analysis.
 Analysis must zero in on user’s
need.
 Critical considerations:
 The user’s business and role in it.
 What the user needs to know and
why.
 How the results will be used for
decision-making.
 Thoroughness requires asking 6 key
questions = “what,” “where,”
“when,” “who,” “how,” and “why”
21
Getting the Question Right
 How effective is
the highway
patrol at
increasing
safety?
22
Getting at the Real Question
 How effective is the highway patrol . . .
 Do resources used give adequate security?
 Are fatalities per 100,000 miles going down?
 Is ratio of convictions/violations going up?
 . . . at increasing safety?
 Is the number of accidents going down?
 Are there fewer safety-related incidents?
 Is there less crime on the roads?
23
Step 2
Identifying Data Required
 Key question = “What data do you need to
answer the question?”
 Data collected from:
Primary sources = you collect
Secondary sources = others collected
 Sources and quantity of data gather
depend on:
The questions to be answered.
The availability of data.
The time allowed for the analysis.
 If data doesn’t exist, revisit questions.
24
Consider Three Data Categories
 Budgetary
 Prospective
 Retrospective
 Program
 Strategic and annual plans
 Program design, operation, and results
 Financial
 Cost of operation and financial position
(assets, liabilities, net worth, revenue,
expenses)
 Cost data by office, program, function, output,
or outcome
25
Assessing Data Quality
Assumption = All decision are data
driven.
The Quality Axiom
The quality of the decision
can be no better than
the quality of the data supporting the
decision!
26
Step 3
Collecting the Data
(Sounds like the easy step?)
 Key questions in deciding how to collect data:
Where and how are you going to get the data?
How will you use it to answer the questions?
 Common data sources:
Records = reports, manual/automated files.
People = within and outside the organization.
 Time available key to source and amount.
 Verification important = reliability, completeness,
accuracy, relevancy.
27
Data Collection Hierarchy
TIME REQUIREMENT
Standard Agency Reports
Low
Automated Systems
Manual Files
Personal Interviews
Focus Groups
Surveys
High
28
Step 4
Analyzing the Data
 Analytic method = steps taken to
understand, display, or interpret data.
 Methodology dependent on question(s).
 Methods numerous and varied.
Important to know purpose/function of each
and when to use them.
 Review results of analysis:
Did it address question(s)?
How sensitive is it to internal/external
influences?
29
Step 4 Axiom
The rigor of the analysis must
reflect the risk of the potential
impact of the decision!
30
Conduct Analysis Methods
Screening
There Categories - Which fits question(s)?
 Descriptive = What exists?
 How was the high-tech training program implemented?
 Normative = What exists compared to what was
expected?
 Were job-placement goals of the high-tech training
program met?
 Impact = Did the initiative achieve the intended
results?
 Did the high-tech training program have an effect on
job-placement rates?
31
Determining the Type of Analysis
 Primary Data Analysis
 You collect, assemble, and analyze data.
 Secondary Data Analysis
 You use data gathered and in some cases
analyzed prior to your own analysis.
 Evaluation Synthesis
 You use the combined results from two or
more previous analysis.
32
Analytical Methods and Their Uses
Content
Trend
Statistical
Case
Study
Benchmarking
BeforeAfter
Field
Experiment
X
X
X
X
X
-
-
Financial Condition and
Operations
X
X
X
X
X
-
-
Budgetary Resources
X
X
X
X
-
-
-
Business Processes
X
X
X
X
X
X
-
Cost-Benefit and CostEffectiveness
X
X
X
X
X
X
-
Policy and Program Design
Alternatives
X
X
X
X
X
X
X
Program Operations
X
X
X
X
X
-
-
Program Impact
X
-
X
X
-
X
X
Category
Problem/Issue
33
Step 5
Reporting Analytic Results
 Important phase of process.
 Impact decision-making; bring about change.
 Reporting methods (written vs. oral vs. video)
have different advantages.
 Goal = convince audience that work done and
results reported is reasonable, appropriate, and
actionable.
 Answering analytic objective, i.e. question, is most critical
goal.
 Remember what drove the analysis:
 Who is the user/customer and what is their role?
 In what subject/issue is the user interested?
 What does the user want to know?
 What does the user plan to do with the results?
34
Structured Approach Steps
Interactive and Overlapping
 Each step dependent upon the successful
completion of previous step.
 “Recheck” step plans for and results of each
step to ensure synchronization with previous
steps.
 As complexity/comprehensiveness of one step
grows, so grows
complexity/comprehensiveness of succeeding
steps, and time required.
35
Structured Approach Steps
Interactive and Overlapping (cont’d)
36
Revalidating Questions
(Continuous throughout analysis!)
 Regularly re-check and validate questions.
 Questions may need to be modified, dropped, redefined
or added.
Analysis Feedback Loop
Define
Questions
Identify
Data
Sources
Collect
Data
Conduct
Analysis
Report
Results
37
Key Points to Remember
 Accountability expectations drive need for critical
analysis.
 Structured approach = objective/systematic way
of collecting, examining, and reporting
information.
 Five step structured approach:





Identify question(s)
Identify data
Collect data
Analyze data
Report results
 Steps highly interrelated – drives need to revisit
results of each step throughout process.
38
Summary
Analysis
 Growing need = support difficult decisions.
 Growing importance = support
performance and financial accountability.
 Best if:
 Follows a structured approach.
 Based on user’s/customer’s questions/need.
 Impacts decision-making/results.
Ron Maccaroni -- rmaccaroni@cox.net
39
Remember . . .
African Proverb
Every morning in Africa, a gazelle wakes up. It
knows it must run faster than the fastest lion or
it will be killed. Every morning a lion wakes up.
It knows it must outrun the slowest gazelle, or it
will starve to death. It doesn’t matter whether
you are a lion or a gazelle. When the sun comes
up, you’d better start running!
40