Reasonable ROI Tracking – Measuring
Internal and External Impact
Marie Beason
Indiana Grantmakers Alliance
Section I: Introduction
Thesis of the Presentation
Expectations for companies as responsible corporate
citizens are rising, and stakeholders are demanding
results. Clearly defined goals with strong social
impact makes for a more compelling story to tell your
stakeholders, resulting in greater impact for the
business.
Telling a Compelling Story
All of your stakeholders need some level of
information.. the “story.”
Section II: Key Issues
Why Measure?
Increasing Expectations
• Internal and external stakeholders are demanding more “results”
• Companies are expected to help address issues, not just give money
• Employees expect their company to be a leading corporate citizen
• Community Leaders expect a commitment to the community
• Customers/Clients expect a company to meet standards
Competitive Advantage
• Most companies measure only outputs, not outcomes
• Most companies are unable to articulate how CI affects their business
(reputation and/or employee recruitment, retention, morale, etc.)
• Trend is toward defining specific goals for major programs; opportunity still
exists to be a leader in an industry or geographic area
Program Improvement
• Findings can be relevant, timely and useful
Which Programs Need to be Measured?
APC focuses primarily on its headquarters community and where its
employees live and work
Program
Content
Resources
Signature Program:
Teacher Training in Grand
Ville
Targeted support of teachers in 3
public schools in poor
communities
$100,000 annually
50 volunteers
Employee contributions
Major Focus Areas:
Education, Social Services
Support of critical needs in
education and social services in
Grand Ville
$75,000 annually
Business / Client-Driven
Support for events, organizations
based on client requests to sales
reps around the country
$25,000 annually
Support for matching gifts and
dollars for doers
$50,000 annually
Employee Engagement
(Volunteerism, Dollars for
Doers, Matching Gifts, etc.)
Multiple Stakeholders with
Varying Needs for Results
Companies need to identify priority stakeholders, issues of concern to those
stakeholders and then determine level of measurement needed for each
stakeholder
(Internal, Regional, &
Local)
Expectations
Stakeholders
Internal
External
• Employees
• CEO/Board/Foundation Board
• Senior Mgmt/Executive Team
• Other Departments
• Business Units
• Domestic/International Offices
• Shareholders
• Business Partners
• Key Opinion Leaders (Influencers)
• Community Partners
• Grantees
• Consumers
•The “Field”
• General Public
• Business Impact – Reputation;
Employee Involvement/Pride –
Retention, Recruitment, Morale
• Business Alignment/CSR Integration
• Continuous Improvement, Learning
about Best Practices
• Social Impact
• Community Expectations
• Transparency
• Social Impact – addressing issues of
concern, which will vary by geography
and by stakeholder
Barriers to Measurement
• Different programs have different expectations of “results”;
lack of internal consensus on what to measure
• Portfolio of programs – major initiatives require more rigorous
outcomes measurement
• Limited budgets or support for deep measurement and
evaluation
• Most companies have not engaged senior management effectively
to gain support for increased measurement
• Limited internal knowledge/capacity regarding
measurement and evaluation
• Most companies have limited staff and do not have the capacity or
knowledge to measure their programs
Section III: Defining Measurement
How Can Measurement Data Be Used?
Proactive
vs.
Reactive
What Can Be Measured?
Inputs
Monitoring:
What is the quantity
and quality of the
resources we have
invested in this
program? Are they
aligned with our
program goals?
Outputs
Performance
measurement:
What is the
quantity and
quality of the
products or
services we have
delivered?
Outcomes
Impact
Outcome
measurement:
What are the
results for
participants,
grantees,
employees, the
business, and
other
stakeholders?
Impact study:
What are the
social and larger
business impacts
attributable to
the program/
company?
Methods of Measurement
Monitoring
Performance
measurement
Documenting
Tracking investment and progress
Minimum evaluation expertise required
Low cost
Outcome
measurement
Impact study
Research
Determining causality
High level evaluation expertise required
High cost
Strategies for Measurement
Low-Cost
Strategies
•Develop a program
model with clear and
measurable goals
•Ensure that program
vision, goals and
objectives align
•Focus on measuring
inputs and outputs
•Create a tracking
system to regularly
monitor and report
progress (inputs and
outputs)
Medium-Cost
Strategies
• Develop a program model with
clear and measurable goals
• Create a measurement
framework that lays out key
measurement questions,
indicators, data sources, data
collection methods
• Focus on measuring inputs,
outputs and short-term
outcomes
• Utilize data collection tools that
have been developed or could
be developed easily
• Create a tracking system to
regularly monitor and report
progress
Higher Cost
Strategies
• Develop a program model
with clear and
measurable goals
• Hire an external
measurement expert to
design data collection
plans, systems and tools
• Provide PD for grantee’s
internal staff around data
collection and
measurement
• Fund peer-to-peer
exchanges re:
measurement
Section III:
Defining Your Approach
1. What Can Be Measured?
Inputs
All the resources
needed to support a
program:
• People
• Dollars
• Marketing
• Lines of business
Outputs
The specific
activities and/or
programs that
serve a particular
target audience:
• Signature
programs
• Focused giving
• Employee
engagement
activities
Outcomes
Impact
Affects on target
audience:
• Community awareness,
perception of company
• Grantee knowledge,
attitudes, behaviors
• Employee morale,
retention, engagement,
team work, leadership,
productivity
The long-term and
aggregate effect of
the program on
overall target
audience and the
company:
• Social impact
• Business impact
(Reputation
among key
stakeholders)
3. What Level of Information Is Needed?
Inputs
Programs
Signature
Major Issue or
Focus Area
Giving
Business-Driven
Giving
Employee
Programs
Outputs
Short-term Outcomes
Long-term Outcomes
Impact
Levels of Information Needed by Stakeholders
Program
Content
Key Stakeholder(s)
Reached
Level of Information
Needed
Impact; Long-term
outcomes
Signature
Focused
All
Major Issue or
Focus Area Giving
Focused on
several issues
Regional Stakeholders;
Community Partners;
Employees
Outputs; short-term
outcomes
Local Area Giving
Broad: based on
local needs
Local community leaders,
employees
Inputs and outputs, some
short-term outcomes
Client-Driven
Giving
Broad: based on
client requests
Clients, Business
Partners
Inputs
Employee
Programs
Broad: based on
employee
interests and
company
priorities
Employees, Community
Partners
Varies by program;
primarily inputs and
outputs, some short-term
outcomes
3. What Tools Can Be Used to Measure?
In House:
External:
• Financial databases/reporting
• HR surveys and data
(recruitment, retention, morale,
productivity, performance)
• Gifts databases
• Grantee reports
• Line of Business systems and
reports
• Existing data (consider marketing
and sales information)
• Media Hits
• Consumer data
•
•
•
•
•
Document/File Review
Surveys
Interviews
Focus Groups
Core Capacity Assessment Tool
(CCAT)
How to Measure… An Example: APC
Resources
Level of
Measures
Measurement
Tools
Outcomes
• Teacher survey
• Employee survey
• Media hits
Program
Content
Signature
Program: Teacher
Training in Grand
Ville
Targeted support of
teachers in 3 public
schools in poor
communities
Major Focus
Areas: Education,
Social Services
Support of critical
needs in education
& social services in
HQ community
$75,000 annually
Outputs
Client-Driven
Support for events,
organizations based
on client requests
$25,000 annually
Inputs
• Financial
databases
Employee
Engagement
(Volunteerism,
Dollars for Doers,
Matching Gifts,
etc.)
Support for
matching gifts and
dollars for doers
Outputs Outcomes
• Financial
databases
• HR, survey
• Media hits
$100,000 annually
50 volunteers
Employee
contributions
$50,000 annually
• Financial
databases
• Gifts database
• Grantee reports
4. What Resources Do You Currently Have
or Can You Leverage?
Skills and resources
(Also consider other
departments)
Existing data/ tools
(Also consider other
departments)
Company culture
Broader context
•
•
•
•
•
People (time and expertise)
Money
Technology
Partners
Leadership
•
•
•
Marketing reports
Demographic analyses
Community data, reports
•
•
HR data
Department data
5. Who Participates in Measurement?
Reviewer
Reviewer
C-Suite
Executives
Senior
Leaders and
Managers
Recipients
of Findings
Implementer /
Coordinator
Community
Stakeholders,
Grantees,
Volunteers
Community
Involvement
Staff
Measurement
6. How Much Will It Cost?
Monitoring
1 – 2%
Performance
measurement
3 – 5%
Outcome
measurement
5 – 10%
Impact study
10% +
7. Evaluating Your Program
Which resources
were most important
for providing “highquality” service?
Inputs
Which strategies (program
qualities) were most
important for achieving the
desired outcomes (including
business outcomes)?
Outcomes
Strategies
The key “learningbased” question
What was the quantity
and quality of the
resources we used to
implement our
programs?
The key “learningbased” question
How much service did we
provide and what was the
quality of that service?
How were our employees
involved (to what extent, in
what ways)?
Did our business or
employees change?
Did our service
recipients change? If
so, how much and in
what ways?
Key takeaways
Maximize the resources you already have
•
•
•
•
•
•
People (time and expertise)
Money
Technology
Partners
Leadership
Existing data/ tools (Program participation, Program Reach, Marketing
data, Company data, local data about the region and community)
Leverage the resources of other departments
•
•
HR
Marketing
Select your measurement level based on your resources,
the program you want to measure, and the type of
information you need for your targeted audiences
Resources
 CEP website (foundations and strategy reports):
www.effectivephilanthropy.org/index.php?page=publications
 DHFS Evaluation Resource Guide. Wisconsin Department of Health and Family
Services. Office of Strategic Finance Evaluation Section. Updated June 2006.
http://dhs.wisconsin.gov/aboutDHS/OPIB/policyresearch/EvaluationResourceGuide06-06.pdf
 Diversity in Philanthropy website:
www.diversityinphilanthropy.org/news/case_studies/pdfs/DPP-Evaluation-Case-Study2.pdf
 Davidson, Jane. ‘Evaluation Methodology Basics: The Nuts and Bolts of Sound
Evaluation’: www.sagepub.com/authorDetails.nav?contribId=532276
 GEO website (see report on Foundations and Evaluation and the Change Agent Project):
www.geofunders.org/publications.aspx
 Joint Committee on National Standards: www.jcsee.org
 Kellogg Foundation Program Evaluation and Logic Model Manuals:
www.wkkf.org/~/media/475A9C21974D416C90877A268DF38A15.ashx
 Mertens, Donna; and Ginsberg, Pauline. The Handbook of Social Research Ethics:
www.sagepub.com/refbooksProdDesc.nav?prodId=Book230293
 RWJF Evaluation Series: http://www.rwjf.org/files/research/50349.quality.checklist.final.pdf