Knowledge Value Mapping Approach
to Knowledge Translation: Its Relation
to Evaluation and Management of R&D
and Practical Applications
Juan D. Rogers
School of Public Policy
Georgia Tech
April 23, 2008
KT-KVM Workshop, NCDDR
1
Presentation Outline





General Problems of R&D Evaluation
Use of Knowledge as Key Component in
R&D Performance
Use-and-Transformation in Tech Transfer
Examples from Knowledge Translation
Applications
Developing Knowledge Value Maps
April 23, 2008
KT-KVM Workshop, NCDDR
2
General Problems of R&D
Evaluation


Political climate demands a “mechanical control” model
Incompatible with well known traits of science:




Focus should broaden from “how” to “what” we are
evaluating


More valuable results are the most unpredictable
Every so often changes its mode of operation to suit new
challenges
We have poor understanding of the socio-cognitive mechanisms
that cause many important things in the workings of research
Almost always happens in actual R&D evaluation projects
Normative analysis: what values are realized
April 23, 2008
KT-KVM Workshop, NCDDR
3
Knowledge Use and R&D Policy
Performance

Outcomes, impacts, ultimate results happen in a different
location from knowledge creation


Many dissimilar communities and social actors are
involved: E. Rogers - heterophily


Social systems that must reflect impacts and outcomes are not the ones
that receive resources to do research
Complex knowledge flows are involved
Creates difficulties for R&D policy design and
implementation and program management and evaluation

Accountability is hard to articulate:


Due to the displaced outcomes vis a vis research activities
Due to the poor understanding of the mediation processes
 Effectiveness of transfer is inherent in R&D performance
April 23, 2008
KT-KVM Workshop, NCDDR
4
Use-and-Transformation in Knowledge
Dynamics

Knowledge Use and its cognates





Beyond the “Linear Model”



Technology Transfer
Diffusion of Innovations
Knowledge transfer
Knowledge translation
But how far?
Analytical frameworks reflect normative concern for impact
rather than empirical knowledge of underlying phenomena
No pure use of knowledge in its flow




Knowledge is transformed as it takes each step in its flow
True of its process of creation
More serious and less understood as it crosses community boundaries
All relevant participants are hybrid user-producers of knowledge
April 23, 2008
KT-KVM Workshop, NCDDR
5
Examples of Complexity of
Knowledge Flows


Features of industrial networks affect change at
the industry level (Andersson et al. 2006)
Knowledge flow from universities to industry:



Some indicator uses encourage counterproductive strategies
(Langford et al. 2006).
Knowledge flow in internal networks for effective
team work (Zhuge 2006).
Enthusiasm about the potential of collaborations

Collaborative work can be very difficult to carry out in practice
(Dustdar 2002).
April 23, 2008
KT-KVM Workshop, NCDDR
6
A Knowledge Value Mapping
Approach: Aims


KVM focuses on the knowledge flows between knowledge
creation activities and the impact contexts
Management of R&D always includes concern for knowledge
flow

Internal to R&D process

Interdisciplinarity

Portfolio management

External to R&D process




Impact and outcomes
Mission requirements
Accountability to stakeholders
Knowledge flows are hard to manage


May be facilitated to certain extent
They are not uniform across relevant dimensions
April 23, 2008
KT-KVM Workshop, NCDDR
7
A Knowledge Value Mapping
Approach: Basis

Content-value structures affect knowledge flows


Current evaluation takes only goals and objectives as relevant
values



Knowledge doesn’t have all its consequences in itself
When stakeholders are included, their values are taken as added or
conditional objectives (i.e. explicit interests)

Process is substituted for value articulation
This perpetuates the fact-value dichotomy
Values must be investigated empirically in the entire domain of
knowledge flow



Interests and agendas matter, but
Epistemic values such as validity and rigor criteria are generally not
analyzed for their effect on outcomes (intra-content)
Values must be also pursued for different articulations
April 23, 2008
KT-KVM Workshop, NCDDR
8
A Knowledge Value Mapping
Approach: Procedure

Map the field

Identify creators and users (we call them “Knowledge-Value
Collectives)







Explicit references (publications, citations, etc.)
Documented relationships (advisory roles, collaborations,
consortia)
Nominations
Find the patterned dynamics and formal agreements
(organized actions, teams, collaborations, contracts, etc.; we
call them “Knowledge-Value Alliances”)
Find the communication patterns
Find evidence and infer the normative stances of all involved
Articulate relevant patterns of knowledge flow
April 23, 2008
KT-KVM Workshop, NCDDR
9
A Knowledge Value Mapping
Approach: Applications

Retrospective KVM:


Features of past or present knowledge flow
Examples:





Division of knowledge labor in interdisciplinary teams or inter-sector
teams
Emergence of new specialized roles in interdisciplinary research
Institutional innovation in technology transfer
Community valuation of hidden knowledge outcomes
Prospective KVM:

Given KVM and proposed intervention, predict likely effects

Determination of facilitating and hindering factors for knowledge
flows (relevant to evaluation and other policy concerns)
April 23, 2008
KT-KVM Workshop, NCDDR
10
Application of KVM to Knowledge
Translation in Health

Evidence of need for knowledge-value maps from the
literature:













Diversity of uses and contexts
Insufficiency of research results for policy making
Diversity of types and mechanisms of use of research results
Variety of incentives in the context of use
Multidimensional matching of interventions and contexts in AIDS and Substance
Abuse
Complexities in a single field application
Theoretical commitments in research
Costs of knowledge absorption
Professional identity of practitioners
Values at different levels of change
Contexts of practitioner decision making
Interdisciplinarity
Dissemination networks and effects on tech transfer
April 23, 2008
KT-KVM Workshop, NCDDR
11
Underlying Value Issues in KT: Diversity
of uses and contexts




Alignment with ideology facilitates adoption
KT begins with assessment of research and
user’s preferences
“hire a knowledge broker to help find and
develop those relationships” (Jacobson et al
2003)
Routine feedback of implementation lessons
from the diversity of application contexts is key
to effectiveness (Van Kerkhoff and Szlezak
2006)
April 23, 2008
KT-KVM Workshop, NCDDR
12
Underlying Value Issues in KT:
Insufficiency of research results





Complexity of the interface of research and policy making
Incentives for both communities are necessary for
managing the interface (Lavis et al 2004; Lavis 2006)
Introduction of new guidelines is costly and benefits
variable or ambiguous
Research results are not sufficient guidance for policy
design and implementation
Extra knowledge is not within health researchers’
competency to produce (Grimshaw et al 2004)
April 23, 2008
KT-KVM Workshop, NCDDR
13
Underlying Value Issues in KT: Diversity of types
and mechanisms of use of research results


There are three categories of uses: instrumental,
conceptual and symbolic.
There are many explanatory mechanisms on the
features of knowledge that facilitate its use:

Engineering explanations
Organizational interest explanations
Two Communities explanations

Interaction explanations



The category of use and the mechanism varies across
agencies and policy domains. (Amara et al 2004)
April 23, 2008
KT-KVM Workshop, NCDDR
14
Underlying Value Issues in KT: Incentives in
the context of use

Implementation of evidence-based practice (EBP) in the
field of occupational therapy (Welch and Dawson 2006):



Most practice change is aimed at individual practitioners
Numerous obstacles for this process to succeed:





Practitioners in this field are embedded in a hierarchy
perceived labor unfairness,
misalignment of the EBP materials with their form of knowledge
acquisition and application in the field,
lack of ownership of the content of EBP presented to them,
lack of structures of mutual support for engaging in learning new
practice guidelines.
“…learning framework that focuses on the intended users
of research as the most critical element of the process”
(Farkas et al. 2003).
April 23, 2008
KT-KVM Workshop, NCDDR
15
Underlying Value Issues in KT: Multidimensional
matching in AIDS and Substance Abuse Cases

Organizational development methods to introduce EBP in
substance abuse treatment (Amodeo et al 2006)



Substance abuse treatment programs (Bartholomew et al.
2007; Greener et al. 2007; Rowan-Szal et al. 2007)




Addresses multiple dimensions of change in the application context
Does not address all dimensions of absorption
Higher organizational functioning led to better client engagement
Also led to fewer barriers to training and greater staff satisfaction with
training
Successful training programs were ones that received support from their
organizations to do post-training trials
HIV prevention interventions (Eke et al 2006; Harshberger
et al 2006)


Preparation during the research design to include contextual factors
Intervention implementation requires ongoing proactive technical
assistance
April 23, 2008
KT-KVM Workshop, NCDDR
16
Underlying Value Issues in KT:
Complexities of a single field application

“Useful Field of View” technology for older drivers (Ball et
al. 2003)

move it to the field and application in a community setting.

What was involved:







Establishing collaborations with health organizations in the area;
Adaptation of the protocol to the field setting
Training field personnel
Finding consenting participants for field trials
Multiple claims to ownership of the database
Answering further questions of cost-benefits for the health care
system
Legislation changes for purposes of changing required tests at
MVA
April 23, 2008
KT-KVM Workshop, NCDDR
17
Underlying Value Issues in KT:
Theoretical commitments in research

Theoretical shift in mental health research (“recovery
vision”) has radical consequences for what counts as
evidence (Anthony et al. 2003)




Fundamental treatment outcome variables used in the EBM
approach become relatively unimportant
Others related to patients’ experiences and goals as indicators
of recovery are more important
Requires a commitment to develop evidence and explore its
implications
The very content of EBM will be different depending on the path
researchers decide to follow in their theoretical commitment.
April 23, 2008
KT-KVM Workshop, NCDDR
18
Underlying Value Issues In KT: More on
epistemic values


Criteria that guide evidence assessment and quality change
with field
EBM has its origin in two particular subfields of clinical medicine




General internal medicine and clinical epidemiology
Transferring evidence assessment criteria from them to other fields of
medicine creates problems (Upshur et al. 2001).
EBM brings along the emphasis on quantitative measurement and,
especially, of randomly controlled trials (Worrall 2002).
Outside of those fields these aspects of measurement and
evidence gathering may not be feasible or as important:

“Although values are regarded as central to health-care decision making,
the quantitative orientation to evidence proposed by advocates makes
normative discourse seem an afterthought.” (Upshur et al. 2001)
April 23, 2008
KT-KVM Workshop, NCDDR
19
Underlying Value Issues in KT: The cost
of knowledge absorption

Getting involved with new knowledge requires a significant
investment of capabilities and resources:


Nature of efforts to enable the use of external knowledge


Concept of “absorption capacity”
Firms have their own research capabilities with the objective of
being able to absorb external knowledge (Cohen and Levinthal
1990).
Expectation of significant payoff for the commitment of
resources to be made:

scientific research will be applied if it is perceived to add value to
the user (Landry et al. 2006)
April 23, 2008
KT-KVM Workshop, NCDDR
20
Underlying Value Issues in KT:
Professional identity of practitioners

Values grounding the professional identity of nurses
(Fagermoen 1997):


The actualization of other-oriented values is integral to
their provision of nursing care


“Professional identity is defined as the values and beliefs held by
the nurse that guide her/his thinking, actions and interactions with
the patient.”
They realize self-oriented values, such as the satisfaction and
reward in their work, through the realization of the other-oriented
values.
Contextual factors limit their ability to provide care making
their work less meaningful
April 23, 2008
KT-KVM Workshop, NCDDR
21
Underlying Value Issues in KT: Values at
different change levels

McGuire (1990) identifies 10 areas of difficulty for KT in nursing:











complexity of change process;
genesis of research programs;
formulation of research questions;
differences in theoretical approaches;
timescales and planning cycles;
information overload;
credibility;
applicability;
response to change;
management of change.
The target should not be at the individual level but at the
organizational level:

Critique of focus on behavior modification
April 23, 2008
KT-KVM Workshop, NCDDR
22
Underlying Value Issues in KT: More on
levels of change


Rushmer et al (2004) show lessons for changes in
practices in primary care come from the lessons
learned in organizational change and
organizational learning.
Many of the obstacles faced by KT are typical
organizational change issues:




features of organizational leadership;
commitment to the program;
organizational culture;
openness and trust; and so on.
April 23, 2008
KT-KVM Workshop, NCDDR
23
Underlying Value Issues of KT: Contexts
of practitioner decision making

Decisions to adopt clinical practices by physicians are
made in complex networks of consultation and
collaboration with colleagues (Hader et al. 2007)




EBM guidelines expected to be followed independently
Must trust the judgment of unknown experts
Guidelines acquire meaning for the doctors within the entire
network of relations and exchanges
Guidelines create questions about their ability to make critical
medical decisions




Need reinforcement by trusted specialists
Consistent with potential patient compliance
Sustained support for the new practices
Not sufficient for doctors to produce new medical
knowledge in the context of use.
April 23, 2008
KT-KVM Workshop, NCDDR
24
Underlying Value Issues in KT:
Interdisciplinarity

Empirical investigation in KT generally confined to the
measurement of outcome variables related to the desired
changes in behavior (Sudsawad 2007).



Numerous social science theories are relevant to KT
(Graham et al. 2006; Estabrooks et al. 2006).


KT has been defined normatively first
Not the natural knowledge frontier of any particular field of
research.
Education, diffusion of innovations, organizational behavior,
psychology, social communication, management, political science,
among others
Attempts at reduction to first principles or a small set of
dimensions,

virtual new social science field of “KT research”
April 23, 2008
KT-KVM Workshop, NCDDR
25
Underlying Values Issues in KT: Dissemination
networks and effects on transfer of technology
variations

Pringle et al. (2004) studied 5 health dissemination
networks in US (RADAR; CADCA; CSN; NTCP; The
Network)



Their ability to effectively disseminate federal-to-local information
depends on fine tuning in four normative dimensions: Synergistic
specialization; Brokerage; Coordination; Broad access
information systems
These networks are different from those in other content areas
and other institutional environments
Reisinger et al. (2007) evaluated three alignment
systems based on two different alignment theories for
fabrication of mono-limb prostheses


They had different performance outcomes, one theory faring
better than the other
The different required equipment and processes led to different
paths of technology transfer, i.e. different ease of transfer
April 23, 2008
KT-KVM Workshop, NCDDR
26
Concluding Remarks

Knowledge flows are inherent in R&D
evaluation




“Nobody’s land” between research and
impacts
Must be determined empirically in the
evaluation process
KVM is proposed to do so
Content-value maps of user-producers
of knowledge
April 23, 2008
KT-KVM Workshop, NCDDR
27