Generic logic models for research,
technology development,
deployment, and innovation
Collected by
Gretchen B Jordan, 360 Innovation LLC
February 2015
Contact: gretchen.jordan@Comcast.net
Table of Contents
(diagrams and references)
Technology and Market Readiness
Research, Development and Deployment Program -Federal
Research, Development and Deployment Program - State
Deployment and Diffusion
Innovation (R&D through Adoption)
Health Sciences Impact Framework
Early Market and Supply Chain Effects
Basic Science - Program and Project
Experimental Program to Stimulate Competitive Research (EPSCoR)
3
6
9
12
16
20
23
27
30
2
Example 1.
Research, Technology/ Development and
Deployment Programs
U.S. DOE Energy Efficiency and Renewable
Energy (EERE) 2007
Jordan, G. and J. Mortensen. Logic Models for a Diverse Portfolio of Research,
Technology, and Deployment Programs, Presentation at American Evaluation Association
Annual Conference, November 8, 2007
http://comm.eval.org/researchtechnologyanddevelopmenteval/tigresources/new-item/newitem2#AEARTD2007
3
Example 1. U.S. DOE EERE Proposed measurement
framework includes 5 inter-related areas that will provide
useful data
Draft 08/03/07
Program Management
External
Factors
Technology
Readiness
Market
Readiness
External
Factors
Outcomes & Impacts
4
(Feedback loops not shown)
Example 1. Proposed Generic Measures for U.S.
DOE/EERE categorize R&D and market readiness
External
Factors
Market Needs/
Opportunities
DOE business
infrastructure
Congressional
earmarks
RD&D
progress
outside EERE
Characteristics
of competing &
supporting
technologies
Draft 08/03/07
Program Management
Portfolio balance
Stakeholder involvement
Performance-based planning
Human capital
Program infrastructure
Fiscal responsibility
Technology Readiness
RD&D capabilities/infrastructure
R&D Advances (non-stage gate)
RD&D stage
-Preliminary investigation
-Detailed investigation
-Development
-Validation
-Commercial launch
RD&D cycle time
Technology characteristics
Breadth of applications
Knowledge transfer & utilization
Options value of technology
Quality implementation
Efficiency
Continuous improvement
External
Factors
Market Readiness
Knowledge infrastructure
• Access, adequacy of tech info (mkt assess, decision
support tools, websites, general ed.)
Policy/Government infrastructure
• Supportiveness of codes, standards, regulations,
incentives, physical infrastructure
Business infrastructure
• Manufacturing, distribution, installation, and
servicing capacities
• Financial capacities
• Economic attractiveness (NPV, IRR, ROI) to supply
chain; competitive advantage
End user
• Visible demonstrations of technology/practice
• Economic attractiveness (NPV, IRR, ROI, payback)
to end user; relative advantage
State, local,
other federal
policies and
incentives
Economics
(Material &
labor costs,
energy prices,
etc.)
Social/Cultural
norms
(preferences,
time horizon,
etc.)
Technology Attractiveness – Market Acceptance
Ultimate Outcomes
Market size & share
Energy benefits
Environmental benefits
Economic benefits
Characteristics
of competing &
supporting
technologies
Security benefits
Spillovers in market
5
Example 2.
Research, Technology/ Development and
Deployment Programs
U.S. DOE Energy Efficiency and Renewable
Energy (EERE) 2004
Jordan, G., John Mortensen, John Reed, George Teather. Using Logic Models in
Managing Performance of Research and Technology Programs, IAMOT 13th
International Conference on Management of Technology, April 4, 2004
6
(2004)
Example 2: EERE’s draft logic model links
strategies/activities to goals
Inputs
Activities
Federal, state & local government funding
Private funding, Personnel, Facilities, Past R&D results
Program
planning &
assessment
Develop &
maintain
program
infrastructure
Conduct
research
Develop
technology
Demonstrate
technology
Developing
government &
market
infrastructure
Deploy
technology
Benefit
estimates,
Priorities
identified,
Budget
requests,
Program plans
Public &
private labs
and test beds,
Knowledge
bases,
Trained S&T
personnel,
Partnerships
New
knowledge,
proof of
concepts as
represented by
data,
publications
For
Programs,
CFO, OMB,
Congress
Programs,
partners
Outcomes
Program
funding in
appropriate
areas;
Efficiency,
Fiscal
responsibility
Relevant S&T
expertise,
capabilities
and facilities to
deliver
programs
Performance
analysis
Test, improve,
& validate
commercialscale
technology,
Give industry
hands-on
experience
Codes and
standards,
Trained
personnel,
Audits tools,
State
programs
Government
purchases,
Information
disseminated,
Early seeding
of technologies
R&D
community
R&D Community,
Industry
Relevant
industries
Relevant
markets
Potential
purchasers
Concepts &
designs with
possible
applications,
Knowledge
spill-over
Potentially
commercializable
technologies to
replace
existing or fill a
system need
Investment by
industry in
innovative or
advanced
commercial
products
Favorable
policies,
capable
delivery
channels for
EERE
products
Widespread
adoption of
EERE
products; More
productive use
of energy
National R&D capabilities, including
options if circumstances change
Technology
prototypes
-initial
-intermediate
-commercial
Political
environment
Quality of R&D
proposals
Feedback Loops
Outputs
External
Influences
Spin-off products and their associated
benefits
New products & businesses
Unpredictable
nature of R&D
Cost and
performance of
competing
technologies
Industry
willingness to
take risk
Energy prices
Economic,
security, and
environmental
benefits
Technology
leadership
State of the
economy
Government
policies and
regulations
7
(2004)
Example 2: Each box in the logic model is a
potential measurement area
External
Influences
Inputs
Activities
Federal funding (millions of nominal $)
Private funding (millions of nominal $)
Program
planning &
assessment
Develop &
maintain
program
infrastructure
Conduct
research
Develop
technology
Federal personnel (FTEs)
# of RD&D facilities
Demonstrate
technology
Developing
government &
market
infrastructure
Cost and
performance of
competing
technologies
(varies by
technology)
Deploy
technology
Oil prices
($/barrel)
NG prices
($/tcf)
Feedback Loops
% programs
w/benefit
estimates
Outputs
For
Outcomes
% program
w/program
plans
Programs,
CFO, OMB,
Congress
# of
partnerships
Programs,
partners
# of journal
articles
# of
presentations
# prototypes
-initial
-intermediate
-commercial
Prototype cost
& performance
# and % of
commercialscale
technologies
validated
# codes and
standards,
# personnel
trained,
# audits,
# state
programs
# of tech’s
purchased by
gov’t,
# of materials
disseminated,
# of website
hits
R&D
community
R&D Community,
Industry
Relevant
industries
Relevant
markets
Potential
purchasers
# journal article
citations
# of potentially
commercializable
technologies
# of innovative
or advanced
commercial
products with
improved cost &
performance
# of
recommendations for using
advanced
commercial
products and
practices
# and % of
advanced
commercial
products by
adoption stage
# of technology spinoffs
Energy saved
(quad. Btu),
GW of add’l
RE capacity,
Expenditure
savings ($)
Carbon saved
(mmtce)
Electricity
prices (c/kWh)
Coal prices
($/ton)
GDP (billion
1996 $)
RE production
tax credit
(c/kWh)
EE/RE tax
credits ($)
CAFÉ
standards
(mpg)
8
Vehicle &
power plant
emission
standards
(varies by
pollutant)
Example 3.
Research, Technology/ Development and
Deployment Programs
New York Energy R&D Authority
Albert, Scott, Victoria S. Engel, Gretchen Jordan, Lori Megdal, Jane Peters, 2004. Using Program Theory
And Logic To Improve Design and Likelihood of Real Market Change - Experience With A State Public
Benefits Program,” accepted for presentation and publication at the ACEEE Annual Summer Buildings
Study. http://aceee.org/files/proceedings/2004/data/papers/SS04_Panel6_Paper01.pdf
9
Inputs:
Funds, staff, allies,
market knowledge
New York Energy $martSM Portfolio Basic Logic
Activities
NYSERDA
Evaluate/select
technology
opportunities
Supply-side
Research &
develop new
technologies
Markets
Actors
Multiple
stakeholders
Researchers
Investors
Manufacturers
Markets/Infrastructure
Coordinate, provide
information, incentives to
businesses & institutions
Energy
businesses,
managers
Short and
Intermediate
term
Outcomes
Deliberate portfolio
mix
Progress tracked &
used in planning
Whole is bigger
than the parts
Credible data
Demonstrated
technology performance
More investment in
developing technology
Longer term
Outcomes
Efficient,
relevant
projects with
measurable
impact
Increased knowledge &
availability of energy and
environmental
technologies
External Influences:
Economic realities impacting new investment in
technologies & energy improvements in some sectors,
energy prices, effect of changing political climates,
legislation & regulation, cost & performance changes
in technologies that support or compete with those
targeted by NYSERDA, existence and activities of
numerous other public & non-profit organizations
promoting similar objectives
Other related
energy
initiatives
Increased knowledge, skills,
profitability, certification
Expanded delivery channels
Favorable standards, rules
Coordinated initiatives and
lower transaction costs
Sale, adoption, service
of technologies in a
profitable business and
favorable policy
infrastructure
DRAFT
3/25/2004
Demand-side
Promote, provide
incentives to users to
adopt new technologies
Low Income
Residential,
Small Business Commercial,
Municipal
& Industrial
Institutional
customers
Identified opportunities,
Increased awareness,
understanding, branding
Changes in behavior and
technology adoption
Increased perceived value
Sustainable widespread
demand for more efficient
energy services and
renewable technologies
Public Benefits
Reduced energy use for
all customer sectors
Increased system reliability
and reduced peak load
Reduced environmental
impact of energy
production and use
Increased competition and
consumers and businesses
saved money
10
NYSERDA R&D Logic - DRAFT
Inputs:
Funds, staff, NYSERDA
competencies, partnerships
Activities
Product Development
Research for Policy
Study to inform
policy & R&D
community
Outputs
Draft 07/29/2004
NYSERDA Select & Manage R&D Projects to:
-drive portfolio changes over time to respond to current needs, and
-Provide public benefits
White papers,
workshops;
Policy-relevant
research
Informed policies &
Outcomes programs;
R&D opportunities &
standards identified,
publicized
Develop new or
improved product
Test &
improve products
New knowledge:
-papers, articles
-data
-Intermediate scale
prototypes
- Performance/cost
specifications improving
- Data from tests
- Establish standards
- Hands on experience
(industry)
-Feedback to R&D
Dissemination
builds common
knowledge base
-Lab prototypes
-Future R&D &
product options
-Investment/interest
growing
-Commercial scale product
developed
-Potential demonstrated
Study, Prove
Concepts
Product proven/
introduced in
market
Demonstration
Pre-deployment
Demonstrate
products, inform
markets
Educate, provide
incentives to
supply & delivery
- Data from tests in
different context
- Feedback to R&D
& policy makers
- Visibility & data
from showcases
Producers,
consumers,
policy makers
see value
-Training, certification
-Production incentives
-Innovative designs
-Other barriers
lowered
Business
infrastructure supports
the product
Policy and Product development and pre-deployment process (5-10 years)
- New
- Accelerated
- Expanded
Knowledge for future R&D and products
Firms have credibility & market infrastructure is supportive
Products manufactured as replacement, stand alone, or part of system and purchased by early adopters
Environmental benefits
Energy benefits
- generation, energy/load management, efficient use
Economic benefits
-cost of compliance, NY jobs
External Influences:
Cost, Performance of existing technologies; Industry willingness to take risks; Uncertainty of R&D; Energy prices; Government policies
11
Example 4.
Logic of Technology (or Practice)
Deployment (Diffusion in Market)
Reed, John H., and Gretchen Jordan. 2007. Impact Evaluation Framework for Technology Deployment
Programs, U.S. DOE, July.
http://www1.eere.energy.gov/analysis/pdfs/impact_framework_tech_deploy_2007_main.pdf
See also
Reed, John H, G. Jordan. 2007. Using Systems Theory and Logic Models to Define Integrated
Outcomes and Performance Measures in Multi-program Settings, in Research Evaluation, Volume 16
Number 3 September.
12
EERE deployment programs typically undertake these activities
Fund and Promote Adoption
Build Infrastructure
Analyze
and Plan
Develop
Technical
Information
Assist Public
Entities
Assist
Businesses
Outreach
and Partner
Assist and
Fund
Purchases
Provide Tools
and Technical
Assistance
Reviewing
and
Reporting
Partnering with or targeting these audiences
Technical and
other personnel in
laboratories,
government, firms,
colleges,
universities
Federal, state, and
local agencies and
nongovernmental
organizations
Investors and
financiers,
manufacturers,
distributors,
retailers, architects,
engineers,
trades people
End user organizations, firms and
individuals
Create and
enhance products,
create and align
market channels,
enhance
marketing, and
develop installation
and support
infrastructures
Adopt, replicate, institutionalize, and
enculturate energy efficient and clean energy
supply practices and technologies
To achieve the following intermediate outcomes
Market and
product
knowledge
Create, advance,
and package
market and
technical
knowledge to make
energy efficiency
more accessible
and implementable
Change the
policies, structure
and operation of
public entities to
smooth the
advance of energy
efficiency and
clean energy
supply
New
knowledge,
alternative
institutional
arrangements
and processes,
new product
and service
ideas, new
opportunities,
That produce the following long-term outcomes or impacts
Reduced energy use and emissions, increased clean energy supply, and enhanced productivity and global security
13
Reed and Jordan
Generic Logic of Federal Technology Demonstration and Deployment Programs
Inputs:
Activities
Draft 8/23/05
Reed and Jordan
Federal funds, staff, state & local funds & staff, existing technologies, partnerships
Develop
Tech information
Analyze &
Plan
Fund & Promote Adoption
Outreach &
Assist & fund
Provide tools & tech
Partner
purchase
assistance
Build Infrastructure
Assist Public
Assist businesses
Organizations
Reviewing,
Reporting
Delivery Mechanisms Used: web site, mailings, one-to-one, partner networks, etc.)
Packaged information
- About gaps
- Filling gaps
Descriptions
Outputs of Markets,
Needs,
Plans
Programs
For or reflect dynamic
With markets
Short &
intermediate
term
Outcomes
- Experts, staff
- Stakeholders
- Program activities
Broadcast
Innovators
Early Adopters
- Guidance, models
- Funds
- Coordination
- Gov’t staff - Manufacturers
- NGOs
- Retailers, Utilities
- Intermediaries- Developers
Early and late
majority
Seeking more information / persuasion
Product value accepted
- Relative advantage
- Compatibility, Complexity,
- Trialability, Observability
Decide to participate
in a program
- Purchases
- Financial
assistance
- Builders
- Commercial, Industrial, Transportation
- A&E firms
- Residential, Low income groups
- Decision makers - Resource Sector, Electric Power
- Cities / states; Purchasing agents
- Performance data
collected, stored
- Products validated
Feedback to program
Consumers assured
of quality
Decide to implement
practices / measures
Changed practices directly
due to program
- Finance/ contract
- Design/ specification
- Implement
- Operation / maintenance
Value confirmed
Program participants
emulate changes
External households
/facilities/firms observe
& emulate changes
Widespread
acceptance of product
Ultimate
Outcomes
- Users trained
- Assessments
- Demonstrations
Contagion
Awareness of resources, program and
eere opportunities
[Many
interactions are
not shown.]
- Partnerships
- Messages out
Institutional change
• Incorporate into standard operating
procedures
• Increased knowledge, promotion &
advocacy
 Government policy, codes, programs
support adoption
 Product, Sales, Service are available,
high quality, profitable
 Production & transaction costs decline
Energy Saved; Clean Energy Used
Sustained Changes in Markets
Serving underserved, energy security, other non-economic benefits
External
Influences:
Low prices of
oil, gas and
electricity;
Unfavorable
regulations;
Age &
condition of
existing
equipment;
Existing
technology
infrastructure;
Utility
restructuring;
Related state
activities
13
A detailed deployment logic model
The Industrial Technology Delivery Program uses these resources
Budget
Efficiency and market knowledge
Complementary interests
Matching funds
Champions
Skilled practitioners
To conduct these activities
Plans and
analyzes
Creates and
Organizes Knowledge
and Infrastructure
Creates
Partnerships
Conducts
Outreach
Conducts
Training
Delivers Practices
and Technology
Tracks
Evaluates and
Reports
External
Factors
Funding
Producing these outputs
To identify:
Medium and
large industrial
users
User needs
and technology
requirements
Delivery
channels
Program
activities
By developing:
Software
Publications
Training
Case studies
ESA assessment
protocols
ESA Experts
Qualified specialists
IACs
EERE Info Center
With:
Manufacturing
Extension
Partnerships
Utilities
PGC
Organizations
Industry and
business
Others
Through:
ESA teams
IACs
Manufacturing
Extension
Partnerships
Utilities
Websites
Web casts
Mailings
Publications
ESA specialists
Plant personnel
Students at
Industrial
Assessment
Centers
Qualified
Specialists
Consultants
Utilities
Others
Through:
ESAs
EERE Info Center
assistance
IAC Assessments
Software downloads
MEP activities
EPACT Voluntary
Agreements
EPACT financial
assistance
IAC database
ESAMS
BTPS database
LEU database
Info Center tracking
Customer
information
Peer reviews
Metric reporting
Case studies
Outcome/impact
evaluations
State and
local
programs
Utility
programs
Partnering with and targeting
Staff
Consultants
Management
Researchers
Congress
Academics
National Laboratories
Manufacturing
Extension
Partnerships
Utilities
Public Goods Charge
Organizations
Regional efficiency
organizations
Industrial firms
Consultants
Students
A&E Firms
Contractors
To induce the following interim outcomes
Increased market
intelligence
Better
understanding of
market segments
Knowledge gaps
filled
More accessible
knowledge
More knowledge
providers and
producers
To achieve these ultimate
outcomes/impacts
Gas and/or Electric
Utilities and PGC
Promote ITDP training and
technical assistance
Expand electric efficiency
programs to include gas
Create new electric and
gas efficiency programs
Recruit customers
Use ITDP tools and
methods
Manufacturing Extension
Partnerships
Recruit clients
Increase resources focused
on energy efficiency
Offer efficiency programs
Support industry efforts to
become more efficient
Awareness of;
Program opportunities
ITDP tools
Publications
Efficiency opportunities
Efficiency solutions
IAC Graduates
Take relevant jobs in industry
and consulting firms
Use tools and techniques
learned at the IAC
Implement efficiency
measures and practices
Consultants
Promote ITDP
programs
Adopt ITDP tools and
approaches
Recommend
technologies and
techniques that
increase energy
efficiency
Participating
firms
Seek information
Decide to use
Implement
Confirm value
Replicate in
plants
Enculturate
Promote
Nonparticipating
firms
Observe
Decide
Implement
Confirm value
Replicate in plants
Enculturate
Reduced energy use intensity, reduced emissions, managed costs, moderated fuel price effects, and improved productivity benefits
John Reed
Energy
Policies
Fuel prices
Structure
changes to
the
economy
International
competition
Outsourcing
Emerging
products
Environmental
regulation /
policy
Capital
availability
15
Example 5.
Logic of Innovation (R&D, Launch and
Market Uptake)
Jordan, G. 2010. A Theory-Based Logic Model for Innovation Policy and Evaluation, Research Evaluation,
19(4), October 2010, 263-274.
16
Example 5. A Systems Logic Model of the R&D to Adoption
Life Cycle
Technology PUSH
Information
Infrastructure
Market
Research
Research
Agenda
Setting
Basic &
Applied
Research
R&D
Launch,
System influences,
effects
Business
Infrastructure
Adoption
End User
Demand
Production
Science
Base,
R&D
Capacity
Development
research
System influences,
effects
G. Jordan, November 2011
Manufacturing
& Quality
Research
Technical
Infrastructure
Government
Infrastructure
/ Policies
End
Outcomes
Market PULL
16
Example 5: DOE Renewable Energy
Do these groups transfer knowledge?
Research on adv.
Semiconductor and
nano-structured
materials
Basic
research
R&D on desired characteristics, then
R&D to increase reliability so 30 year
Utilization & warranties could be offered
Behavioral
research
Research on PV module
Quality
recycling, waste disposal,
Research,
usage of toxic materials to
Product
protect workers
refinement
R&D
Research on PV
devices utilizing semiconducting colloidal
nanostructured…
Science
Base,
R&D
Capacity
Knowledge of Nano
science & technology
Teamed research on generic,
industry wide R&D problems
Manufacturing
getting volume up, costs down
research
Applied
research
Development
research,
Validation
PV Pre-incubator project helps
small businesses transition from
concept verification to tech.
development
Generic &
Infratechnologies
Built production process off
existing wire saw technology;
Fund an independent testing
facility that also provides
18
reference cells
Example 5: DOE Renewable Energy
Do these groups transfer knowledge?
Developed curriculum and
installed PV on school
Information
roofs; codified & published
Infrastructure
best practices; Provide
model legislation
Develop proven
financing techniques;
training/certification for
installers
Business
Infrastructure
Product
characteristics:
Launch,
-relative advantage Production
(operating cost)
-compatability (use for
hot water, etc.)
Generic &
-complexity
Infratech(simple drop in to roof)
nologies
-trialability
(no harder than
conventional)
-observability
(green pricing)
End User
Demand
Fund showcases for
people to view real
application; Make
aware of benefits
Government
Policies
States require use of
renewables; Block
grants to states for
energy plans; Simplify
permitting; Consistent
interconnection
End
Outcomes,
System
effects
Reduced pollution and
dependence on imported oil.
DOE R&D accelerated
development of PV modules
19
by 12 years
Example 6.
Impacts of Health Research –
Canadian Academy of Health Sciences
Framework
Canadian Academy of Health Sciences, Panel on Return on Investment in Health Research. (2009). Making an
impact: A preferred framework and indicators to measure returns on investment in health research. Ottawa (ON),
Canada: Canadian Academy of Health Sciences (CAHS). Retrieved from: http://www.cahs-acss.ca/wpcontent/uploads/2011/09/ROI_FullReport.pdf
See also
Graham KER, Chorzempa HL, Valentine PA, Magnan J. . (2012).Evaluating health research impact: Development and
implementation of the Alberta Innovates – Health Solutions impact framework. Research Evaluation,21(5):354-367.
20
Example 6: The Canadian Academy of Health Sciences
Logical Framework for Understanding the Impacts of
Health Research
Global Research
Knowledge Pool
Interactions/Feedback
Health Research
•Biomedical
•Clinical
•Health services
•Population and
Public health
•Cross cutting
Health
Industry
Other
Industries
Government
Research
Agenda
Public
Information,
Groups
Healthcare
Appropriateness,
Access, etc.
(Population)
Health Status
and Function
For Prevention,
Diagnosis, Treatment,
and Post Treatment
Determinants of
health (social,
environmental, etc.)
Well Being
And
Economic
Prosperity
Research
Capacity
Modified from CAHS Report on ROI for Health research
available at www.cahs-acss.ca/making-an-impact-apreferred-framework-and-ind...
21
Example 6: Detail
Research results…
That Influence decisions
that change…
Canadian
Public/ private
Health Research
Health Status,
Well Being,
Economic
conditions
•Biomedical
•Clinical
•Health services
•Population and Public
health
•Integrated research
Research Capacity
-Human capital
---student & faculty career
paths
-Absorptive capacity
-Cross fertilization of
ideas/research (joint
planning, cross
functional/multi disciplinary
teams, levels of
communication )
Research Decision Making
At levels of Academic,
Institution,
-Funders (Industry,
Government, Foundations)
R&D agendas/investment
-Tackling harder more
complex problems
Knowledge Pool Consultation/ Collaborations
Global Research
Health Industry
-Products/Drugs
-Product/drug development &
testing
-Services, databases
-Practitioners’ behavior
-Clinical/managers guidelines
-Institutional Policies (hospitals,
etc.)
-Social care practices
Other
Industries
-Products/services
-Built environment
-Work environment
Government
Interventions at
multiple levels related
to health care, social
care, Public health
-Resource allocation
-Regulation
-Policy
-Intervention programs
-taxes & subsidies
-Education curriculum
Public Information,
Groups
--Advocacy groups
-Media
-Attitudes
-Knowledge of
-Confidence in
research data
That affect health risk
factors, healthcare, and
health status
Healthcare System
Quality for Cost (appropriateness,
acceptability, competence,
continuity, effectiveness) and
Accessibility
-Adherence to guidelines
-Reduced errors
--hospital accreditation
Intermediate Health
Outcomes
Prevention and Treatment
For disease, illness, injury, or
progressive condition
-Prevention
-Diagnosis/prognosis
-Treatment/palliation
-Post-treatment
(Population)
Health Status
and Function
(disease
prevalence
and burden)
Well Being
And
Economic
Prosperity
External Influences:
Population Health Risks
-Age & genetics
-Personal behavior
-Social determinants
(education, networks, etc.)
-Environmental factors
-Built environment
--social environment
22
Example 7.
Logic of Accelerating Technology
Introduction in U.S. Supply Chains
Jordan, Gretchen, Jonathan Mote, Rosalie Ruegg, Thomas Choi, and Angela Becker-Dippmann.
2014. A Framework for Evaluating R&D Impacts and Supply Chain Dynamics Early in a Product Life
Cycle: Looking inside the black box of innovation, prepared for the U.S. Department of Energy.
http://www1.eere.energy.gov/analysis/pdfs/evaluating_rd_impacts_supply_chain_dynamics.pdf
23
A Framework for Assessing Accelerated Product Innovation, Manufacturing, Early Market Growth
U.S. Global Competitiveness in Manufacturing Energy Technologies;
National Energy and Economic Benefits
Ultimate
Impacts
O3
Interim Effects Outcome
Objectives
Stronger product
value chain
C1
Added capabilities –
technical and market
Inputs
Growth in US
manufacturing
O1
Accelerated new
product commercialization, adoption
C6
Short &
Intermediate
Conditions for
Progress
O2
Capabilities
for continued
innovation
C2
C 5Added value to
characteristics of
new product
Available capital
for R&D, scale up,
production
C3
C7
Stronger product
supply chain
Supportive
business
practices, gov’t.
policies
External
Influences:
Technical.
Information/
Networks,
Economic,
Policy
C4
Stronger networks,
knowledge exchange
EERE Investments/Activities and Collaborations
(Technical, Information/Relationships, ,Business, Policy)
24
DOE/EERE 2013
Detailed Logic of Accelerating Technology Introduction in U.S. Supply Chains
Ultimate
Impacts
U.S. Global Competitiveness in Manufacturing Energy Technologies;
National Energy and Economic Benefits
Capabilities for Continuing Innovation
Intellectual property captured in an area.
Technical leadership in the area.
Broader
Companies and universities strong in R&D.
Intermediate R&D infrastructure, knowledge diffusion.
Outcomes Favorable standards, regulations.
Leverage capital available at multiple points.
Proximity of R&D, firms, collective action.
Stronger Product Value Chain
-small businesses are involved
-challenges such as retooling met
-new business models adopted
-firms add to/modify product line
Short &
Intermediate
Conditions
for Progress
Added Technical &
Market Capabilities
-existing research, tools,
techniques
-technical challenges
solved
-standards, test facilities,
-market knowledge,
strategies.
EERE Investments,
Inputs
DOE/EERE 2013
Technical
Fund R&D & test
facilities.
Develop & provide
measurement tools.
Growth in U.S. Manufacturing
Accelerated Commercialization,
Adoption
Domestic production of components, end
products in a supply chain.
Increased production due to advantages of
using a new process.
Emergence of new markets where U.S.
firms are competitive.
Sales, employment. market share.
New products, features available, including
energy efficient, environment friendly;
measure of their value added.
New production features available (e.g.,
mobile, flexible, lower costs of transport).
Faster time to development, market.
Added Value to a New Product or
Process
-adaptation of existing, scale up, volume
-new, improved performance, -cost,
compatibility
Availability of Capital
at Multiple Stages
-able to raise private
capital; use user
facilities
-early adoption by
government
Business
Support validation,
demonstration.
Co fund start up firms,
production facilities.
Stronger Supply Chain
-incentives to enter, to stay
-market, customer orientation
-maturity, ability to deliver, on time
-flexibility, adaptability, robustness
Supportive Business Practices,
Policies
-appropriate focus, network
connections
-remaining flexible
-checking potential market
regularly
-favorable policies (tax,
regulation)
Government
Supportive standards,
government policy.
Government procurement
(early adopter).
External
Influences
Stronger Networks,
Knowledge Exchange
-connectedness within value
chain (e.g., with sources of
capabilities), in Supply Chains
-network characteristics
(strategic partnerships,
structure, ties, roles)
Information/Relationships
Provide technical/ market
analysis, databases.
Facilitate networking, publicprivate partnership.
25
EERE Investment in Lithium-ion Battery Plants in the U.S.
U.S. Global Competitiveness in Manufacturing Energy
Technologies
National Energy and Economic Benefits
Ultimate Impacts
Growth in US
manufacturing
of li-ion
batteries
Capabilities
for continued
battery innovation
Broader
Intermediat
e
Outcomes
Accelerated battery
commercialization,
use in electric cars
Stronger value chain:
(materials, coating,
binding, container)
Short to
Intermediate
Conditions for
Progress
Added capabilities
(Fund R&D; alignment
of standards)
Inputs
Added value:
More power durability,
range of operating
temperature
Capital available
for R&D, scale
up, production
Stronger supply chain:
(U.S. manufacturers,
U.S. suppliers
operating)
Business practices:
(require manf. flexibility for inputs)
New partnerships
bid for funds, work
together
External
Influences:
Small
market for
EVs; new
firms with
few financial
reserves;
Imbalance in
relevant
global
policies
EERE Investments
Co-Fund R&D;
Co fund production facilities in U.S.
26
DOE/EERE 2013
Example 8.
Logic of Basic Science Program and
Projects
Related discussion:
Lee, Russell, Jordan, Gretchen, Leiby, Paul N., Owens, Brandon, Wolf, James L. 2003.
“Estimating the Benefits of Government-Sponsored Energy R&D,” Research Evaluation, Vol.
12 No. 3 (Dec.): 189-195.
27
Logic Model of a Program of Basic Research
(U.S. DOE DRAFT -Unofficial)
ACTIVITIES
Identify/ Direct/Redirect
resources to important
questions & needs
OUTPUTS &
OUTCOMES
Gather/ Build/
Maintain/ Provide
resources in select areas
Students work
with DOE or
elsewhere
Construct,
operate,
facilities
Perform or Have
Performed high quality
research
Facility use
- DOE &
others
Propose;
Experiment,
theorize; Collect
& analyze data
Prove, disprove; Theories,
techniques developed &
solutions generated
Robust
S&T
workforce
Robust S&T
Facilities &
Equipment
Capacity/Agility
New structure, new ideas,
tools, fields, Opportunities
for use by others
Disseminate/ Seek
Review/ Feedback
research plans, findings
Inform and be
informed by
collaborators,
peers, potential
users
Strong
communities of
practice
Transitions – findings used
Significant Contributions to DOE Mission, National Needs, Society
G. .Jordan
05/13/2002
28
The Logic of a Basic Research Project
Manage Resources: expenditures by types of activities, skilled staff, core competencies;
environment for quality research, soundness of research planning and evaluation, use
scientific method
Activities
Outcomes
and
Results
Identify and state
the problem
Growing consensus
on problems
Develop, test and
build research tools
New techniques to
research problems
Do research and
report findings
Growing
convergence on
solutions to
problems
Exchange
knowledge in
papers,
conferences, etc.
Apply ideas of
others in research
New disciplines
Reach targeted
partners and
customers; other
researchers,
laboratories,
students,
universities, applied
researchers and
technology
developers,
industry; attendees
at conferences,
readers of
publications
New insights and
knowledge
[Feedback loops
are not shown]
G. Jordan 1996
Potential impacts of
research
Use in R&D or
Commercialization
Actual impacts of
the research
29
Example 9.
National Science Foundation's
Experimental Program to Stimulate Competitive
Research (EPSCoR)
Evaluation of the National Science Foundation's Experimental Program to Stimulate
Competitive Research (EPSCoR): Final Report
Brian L. Zuckerman, Rachel A. Parker, Thomas W. Jones, Brian Q. Rieksts, Ian D. Simon,
Gilbert J. Watson III, Elaine A. Sedenberg, Sherrica S. Holloman, Ryan M. Whelan, Lucas M.
Pratt, Christopher T. Clavin, Abigail R. Azari, Mitchell J. Ambrose, Jessica N. Brooks, and
Pamela B. Rambow. IDA Paper P-5221, December 2014
30
31