Knowledge Brokering: creating links between people for knowledge Institute for Knowledge Mobilization Facilitator: Peter Levesque 2014 © Peter Levesque 2007-2014 1 Workshop Format • Facilitated discussion • Hands-on activities • Group discussions • Case studies • Videos 2014 © Peter Levesque 2007-2014 2 Workshop Agenda Morning: Afternoon: Workshop objectives Introductions 1. What is knowledge? 2. What is brokering? 3. What are you doing? 5. – Knowledge broker exercise and discussion 4. Core skills of a knowledge broker 5. What are people doing? – 2014 Discussion of knowledge broker stories 6. 7. 8. 9. More what are people doing The growing role of intermediaries The care and feeding of knowledge brokers Tools and Methods Lessons learned – Failures – Things to avoid – Overcoming barriers 10. Monitoring and Evaluation © Peter Levesque 2007-2014 3 Workshop Objectives • An operational understanding of the role of the knowledge broker. • How knowledge brokers are being implemented as a role within organizations. • • 2014 An understanding of the core skills required of knowledge brokers. How to support the use of knowledge brokers within organizations • An understanding of the tools and methods needed to support knowledge brokering based on a range of case studies. • How to identify and overcome barriers to knowledge brokering. • How to monitor and evaluate progress of the strategy. • Meet a group of peers that become part of your community of practice. © Peter Levesque 2007-2014 4 Introductions • All about you: – Who are you? – Where do you work? – What are your areas of interest? – What do you hope to learn from this workshop? 2014 About me… Here is my contact info: Peter Levesque Institute for Knowledge Mobilization 1 Rideau Street, Suite 700 Ottawa, ON, K1N 8S7 T: 613-552-2725 E: peter@knowledgemobilization.net W: www.knowledgemobilization.net T: @peterlevesque © Peter Levesque 2007-2014 5 Preparation for Follow-up As you participate in the workshop, note the following: 2014 • 2 People: to follow-up • 2 Ideas: that inspire or intrigue you • 2 Improvements: to your work or work environment © Peter Levesque 2007-2014 6 What is Knowledge: Definition knowledge(know|ledge) noun 1. facts, information, and skills acquired through experience or education; the theoretical or practical understanding of a subject: – – – – – 2. 3. a thirst for knowledge her considerable knowledge of antiques the sum of what is known: • the transmission of knowledge information held on a computer system. Philosophy true, justified belief; certain understanding, as opposed to opinion. awareness or familiarity gained by experience of a fact or situation: – – the programme had been developed without his knowledge he denied all knowledge of the incidents archaic sexual intercourse. http://oxforddictionaries.com/definition/knowledge 2014 © Peter Levesque 2007-2014 7 Is this Knowledge? http://vimeo.com/8665381 2014 © Peter Levesque 2007-2014 8 Busy new world • Take a deep breath. • Think about your work. • What is your relationship with data, information, and knowledge? • What does it look like? http://www.designfloat.com/blog/2011/03/30/moleskine-art-legendary-notebooks-cool-sketches/ 2014 © Peter Levesque 2007-2014 9 Does it look like this? http://www.sohotastic.com/ 2014 © Peter Levesque 2007-2014 10 Or more like this? http://www.spareroom.co.nz/2006/07/24/i-have-it-right-here// 2014 © Peter Levesque 2007-2014 11 Or even like this? http://carbon-based-ghg.blogspot.com/2009/11/rising-sea-levels-to-impact-waste.html// 2014 © Peter Levesque 2007-2014 12 Exponential growth http://www.tgdaily.com/hardware-features/42499-digital-content-doubles-every-18-months 2014 © Peter Levesque 2007-2014 13 What about Research? (e.g. geology) http://jrp.icaap.org/index.php/jrp/article/viewArticle/128/106 2000 1945 2014 © Peter Levesque 2007-2014 14 Data to Info to Knowledge • Data: 1, 8, 24, 27 • Information: Birth dates of my family. • Knowledge: preferences for parties 2014 © Peter Levesque 2007-2014 15 Hunting and Gathering • Never in human history have we hunted for so much data, information and knowledge. • Never in human history have we gathered so much that is useful but not used. 2014 © Peter Levesque 2007-2014 16 Data is: • Data is raw material for processing. • Data relates to fact, event and transactions. 2014 © Peter Levesque 2007-2014 17 Information is: • Information is data that has been processed in such a way as to be meaningful to the person who receives it. • Information is any thing that is communicated – IT IS INFORMATIVE. 2014 © Peter Levesque 2007-2014 18 Knowledge is: • Knowledge is result of perception and learning and reasoning. • Knowledge is social. • Knowledge requires relationships. 2014 © Peter Levesque 2007-2014 19 What is knowledge - discussion • How is knowledge understood in your organization? • How do you support the creation of knowledge – the social life of data and information? 2014 © Peter Levesque 2007-2014 20 What is Knowledge Brokering? Knowledge brokering links researchers and decision makers, facilitating their interaction so that they are able to better understand each other's goals and professional culture, influence each other's work, forge new partnerships, and use research-based evidence. Brokering is ultimately about supporting evidence-based decision-making in the organization, management, and delivery of health services. http://www.chsrf.ca/keys/glossary_e.php 2014 © Peter Levesque 2007-2014 21 What is Knowledge Brokering? Knowledge brokering is a dynamic activity that is the human force behind knowledge exchange and adoption. It involves bringing people together, helping to build links, identifying gaps and needs, and sharing ideas. It allows information to be used to solve a problem or lead to a better way of doing things. It also includes assisting groups to communicate and understand each others’ abilities and needs, and assists with guiding people to sources of research. This may include summarising and synthesising research and policy into easily understood formats and transforming issues into research questions. Knowledge brokering encourages the use of research in planning and implementation and uses evaluation activities to identify successes or improvements. Source: Land and Water Australia http://www.lwa.gov.au/library 2014 © Peter Levesque 2007-2014 22 Knowledge Brokers are key people in the Knowledge Mobilization process Knowledge mobilization is about ensuring that all citizens benefit from publicly funded research. It can take many forms, but the essential objective is to allow research knowledge to flow both within the academic world, and between academic researchers and the wider community. By moving research knowledge into society, knowledge mobilization increases its intellectual, economic, social and cultural impact. http://www.sshrc-crsh.gc.ca/society-societe/community-communite/index-eng.aspx 2014 © Peter Levesque 2007-2014 23 2014 © Peter Levesque 2007-2014 24 Associated Terms: Knowledge Transfer & Knowledge Exchange “Knowledge transfer and exchange is collaborative problemsolving between researchers and decision-makers.” Canadian Health Services Research Foundation 2014 © Peter Levesque 2007-2014 25 Associated Terms: Knowledge Translation “Knowledge translation (KT) is defined as a dynamic and iterative process that includes synthesis, dissemination, exchange and ethically-sound application of knowledge to improve the health of Canadians, provide more effective health services and products and strengthen the health care system.” Canadian Institutes of Health Research 2014 © Peter Levesque 2007-2014 26 Associated Terms: Knowledge Translation “This process takes place within a complex system of interactions between researchers and knowledge users which may vary in intensity, complexity and level of engagement depending on the nature of the research and the findings as well as the needs of the particular knowledge user.” Canadian Institutes of Health Research 2014 © Peter Levesque 2007-2014 27 Many terms with similar meanings: A strategy to create value from knowledge 2014 © Peter Levesque 2007-2014 28 Knowledge Mobilization as a Value Chain Programs Policies Priorities Processes Practice Innovation Supporting Infrastructure Initiatives Value Creation Now What: Decisions, Directions, Actions So What: Meaning, Analysis, Interpretation Products Perspectives Procedures Possibilities People Skills Incentives to Share between Levels What: Data, Information, Description, Stories MULTIPLE INPUTS FROM RESEARCH, PRACTICE, EXPERIENCE, CULTURE 2014 © Peter Levesque 2007-2014 29 Brief History of Knowledge Mobilization 3500 BCE 200 BCE 1450 1858 1950s 2014 • Cuneiform language invented • Parchment now available • Johannes Gutenberg brings functional moveable type to Europe • Trans-Atlantic telegraph • Computers and Knowledge Management emerge © Peter Levesque 2007-2014 30 History of Knowledge Mobilization Passive push (until 1970s+) • Dissemination via traditional journals, conferences Push harder (1990s+) • Focus on implementation, e.g. performance feedback Partner & pull (2000+) • Linkage & exchange, e.g. joint production Greater role for knowledge brokers! 2014 © Peter Levesque 2007-2014 31 Roles of Knowledge Brokers: Appendix 1 2014 © Peter Levesque 2007-2014 32 Roles of Knowledge Brokers Informing Linking Matchmaking Focused Strategic Building Collaboration Collaboration Institutions Behaviour Change 2014 © Peter Levesque 2007-2014 33 Knowledge Broker Exercise • Go to Appendix 2 • Match the product or service to the role • Discuss your answers with your neighbour • What else are you doing? 2014 © Peter Levesque 2007-2014 34 Core skills of a knowledge broker The role of the broker depends on the organization, but there is a basic skill set: • bring people together and facilitate their interaction; • find research-based and other evidence to shape decisions; • assess evidence, interpret it, and adapt it to circumstances; • a knowledge of marketing, communication and the industry/sector they work in; and • identify emerging management and policy issues which research could help to resolve. http://www.chsrf.ca/migrated/pdf/Theory_and_Practice_e.pdf 2014 © Peter Levesque 2007-2014 35 2014 © Peter Levesque 2007-2014 36 Core skills of a knowledge broker • Personal Attributes • Evidence Gathering Skills • Critical Appraisal Skills • Communication Skills • Mediation Skills 2014 © Peter Levesque 2007-2014 37 Core skills of a knowledge broker Personal Attributes Knowledge Brokers should be inquisitive, enthusiastic, flexible, inspirational, imaginative, highly credible and keenly interested in learning. They should be skilled analysts, able to see the 'big picture' and be able to readily identify links between ideas and pieces of information. http://www.canchild.ca/en/canchildresources/knowledgebrokering.asp 2014 © Peter Levesque 2007-2014 38 Core skills of a knowledge broker Evidence Gathering Skills Knowledge Brokers should be aware of the best sources of synthesized evidence and original studies within their content area and have focused expertise in searching these sources for research evidence. They should also be skilled in searching for less formal contextual evidence such as policy documents and evaluation reports. The ability to evaluate the effectiveness of knowledge brokering activities is also a necessary skill for an effective Knowledge Broker. http://www.canchild.ca/en/canchildresources/knowledgebrokering.asp 2014 © Peter Levesque 2007-2014 39 Core skills of a knowledge broker Critical Appraisal Skills Knowledge Brokers should be adept at appraising evidence to evaluate its quality, importance, and applicability to a particular context. In addition to traditional critical appraisal skills, they should have knowledge of the sector, the broader industry environment, its key players and controversies - and use this to gauge the applicability and adaptability of new evidence to user contexts. http://www.canchild.ca/en/canchildresources/knowledgebrokering.asp 2014 © Peter Levesque 2007-2014 40 Core skills of a knowledge broker Communication Skills Knowledge Brokers should have strong oral and written communication skills and use a variety of methods targeted to the needs of the diverse stakeholders (e.g., researchers, practitioners, policy-makers, managers, and customers/clients/consumers/citizens). They should use active listening skills to gain insight into the interests, issues and innovations of their network members. http://www.canchild.ca/en/canchildresources/knowledgebrokering.asp 2014 © Peter Levesque 2007-2014 41 Core skills of a knowledge broker Mediation Skills To function as effective relationship builders, Knowledge Brokers should be skilled mediators. They assemble teams and foster collaboration amongst individuals and groups who would not normally work together. They reconcile misunderstandings, facilitate the identification of shared goals, and negotiate mutually beneficial roles for all group members. http://www.canchild.ca/en/canchildresources/knowledgebrokering.asp 2014 © Peter Levesque 2007-2014 42 Tasks of a Knowledge Broker The tasks of a broker include: • bringing people together to exchange information and work together; • helping groups communicate and understand each other’s needs and abilities; • pushing for the use of research in planning and delivering services; • monitoring and evaluating practices, to identify successes or needed changes; • transforming management issues into research questions; • synthesizing and summarizing research and decision-maker priorities; and • ‘navigating’ or guiding through sources of research. http://www.chsrf.ca/migrated/pdf/Theory_and_Practice_e.pdf 2014 © Peter Levesque 2007-2014 43 What are people doing? Go to Appendix 4 – Read Story 1: – Dr. Vicky Ward: The case of the accidental knowledge broker – Questions and Discussion • What does Vicky mean when she says “it’s a complex process? • What is an expert? • What are real-world settings? 2014 © Peter Levesque 2007-2014 44 What are people doing? Go to Appendix 4 – Read Story 2: – Dr. Alex Bielak: From Science to Science Communication to Knowledge Brokering – Questions and Discussion • Do you think knowledge brokers have a clear career pathway? • Why do you think networks are important? • How is knowledge brokering different than communications? 2014 © Peter Levesque 2007-2014 45 What are people doing? Go to Appendix 4 – Read Story 3: – Dr. Melanie Barwick: Tinker, Tailor, Soldier, Spy: Building the Science, Practice, and Profession of Knowledge Translation – Questions and Discussion • Why is training not a one-off endeavour? • Why does knowledge brokering draw from many disciplines? • Why is change difficult? 2014 © Peter Levesque 2007-2014 46 What are people doing? Go to Appendix 4 – Read Story 4: – Jonathan Green: “Me” as part of an innovative system – Questions and Discussion • Why are flexibility and adaptability important skills? • How will change, change over time? • Are knowledge brokers important for innovation? 2014 © Peter Levesque 2007-2014 47 The growing role of intermediaries 2014 © Peter Levesque 2007-2014 48 Case 1: Education Cooper and Levin at the Ontario institute for Studies in Education (OISE) have adapted the Lavis (2003) model for use with intermediaries (knowledge brokers) in moving Education research into practice locations. 2014 Their approach: • Adapt Lavis KT Model • 3 prong strategy • Provide selection of tools • Show potential impacts of activities © Peter Levesque 2007-2014 49 OISE/Levin Model of knowledge Mobilization http://www.oise.utoronto.ca/rspe/Publications_Reports_Conferences/index.html 2014 © Peter Levesque 2007-2014 50 Case 1: Education 5 Prong KT Organizing Framework 1. MESSAGE: What should be transferred to decision makers? (p.223) 2. TARGET AUDIENCE: To whom should research knowledge be transferred? (p.224) 3. MESSENGER: By whom should research knowledge be transferred? (p.225) 4. KT PROCESS: How should research knowledge be transferred? (p.226) 5. EVALUATION: With what effect should research knowledge be transferred? (p.227) (Lavis et al., 2003) 2014 © Peter Levesque 2007-2014 51 Case 1: Education Strategy Approach KM Products KM Events KM Networks (Cooper & Levin, 2010) 2014 © Peter Levesque 2007-2014 52 Case 1: Education KM PLAN Organized by Strategy PRODUCTS Research Reports Exec Summ Research Snapshots Policy Briefs Success Stories Multimedia EVENTS Panels & Talks Conference Workshops & Training Sessions Annual Meetings NETWORKS Listserv, RSS feeds E-Bulletins COPs Social Media Online Forum MEDIA Press Release Newspaper Articles Radio TV Blogs (Cooper & Levin, 2010) 2014 © Peter Levesque 2007-2014 53 Case 1: Education (Cooper , 2011) 2014 © Peter Levesque 2007-2014 54 Case Study 2: USA National Archives Social media is about community and conversations. Our social media strategy is based on six core values that will help transform NARA. We focus on three main communities that we are seeking to engage: our staff community, the government community, and citizen archivists. http://www.archives.gov/social-media/strategies/ 2014 © Peter Levesque 2007-2014 55 Case Study 2: USA National Archives Our Core Values for Social Media • Collaboration: Together as one NARA and as partners with the public to accomplish our mission • Leadership: Out in front among government agencies and cultural institutions • Initiative: An agency of leaders who are passionate, innovative, and responsible • Diversity: Making NARA a great place to work by respecting diversity and all voices • Community: Caring about and focusing on the government community, citizen archivists, and each other • Openness: Creating an open NARA with an authentic voice http://www.archives.gov/social-media/strategies/ 2014 © Peter Levesque 2007-2014 56 Case Study 2: USA National Archives Staff Community • Empower staff to use social media tools to work effectively • Develop a cadre of social media leaders at NARA, who are subject matter experts with social media savvy • Implement and encourage use of social media tools for collaboration • Implement and encourage use of social media tools for professional networking • Implement and encourage use of social media tools for information and status update sharing http://www.archives.gov/social-media/strategies/ 2014 © Peter Levesque 2007-2014 57 Case Study 2: USA National Archives Government Community • Create spaces and platforms for conversations with the government community • Participate in online spaces and conversations that engage federal records manager, declassification stakeholders, and citizens and public interest groups with a focus on records access • Develop and demonstrate best practices for social media records management • Implement best practices for e-records management at NARA http://www.archives.gov/social-media/strategies/ 2014 © Peter Levesque 2007-2014 58 Case Study 2: USA National Archives Citizen Archivists • Participate in online spaces where researchers, citizen archivists, and potential citizen archivists spend time online • Make our resources and services more findable and sharable • Find, encourage, and recognize citizen archivists • Create opportunities and platforms for researchers and citizen archivists to help us and for them to help each other • Turn our catalog into a social catalog and invite the public to contribute to the online information available about our holdings • Find, encourage, and recognize the support of the Foundation for the National Archives http://www.archives.gov/social-media/strategies/ 2014 © Peter Levesque 2007-2014 59 Case 3: Seniors (Appendix 5) The Seniors Health Research Transfer Network (SHRTN) created a collaborative with other networks focused on seniors health issues. They have produced substantial numbers of materials on knowledge transfer and exchange & knowledge brokering. 2014 Their approach: • Develop mutual understanding of goals and cultures. • Collaborate with knowledge users and producers to identify issues and problems. • Facilitate the identification, assessment, interpretation, and translation of evidence • Facilitate the management of information and synthesis of knowledge © Peter Levesque 2007-2014 60 Other: Leeds Knowledge Brokering Model http://www.leeds.ac.uk/lihs/psychiatry/kt/docs/Knowledge%20Brokering%20Final%20report.pdf 2014 © Peter Levesque 2007-2014 61 Other: SECI Model http://knowledgeandmanagement.wordpress.com/seci-model-nonaka-takeuchi/ 2014 © Peter Levesque 2007-2014 62 The care and feeding of knowledge brokers • To thrive, brokering needs a supportive organization — one where there is a collaborative environment, sufficient resources for the job, processes to identify and capture knowledge generated by both employees and outside parties, and a desire to build intellectual capital • People doing knowledge brokering need support; joint activities and a national network will build commitment to brokering and keep crucial energy from being wasted reinventing wheels. • Management must “understand the level and nature of resources” brokers require to do the job well (Earl and Scott 1999), but without recognition the function won’t be encouraged or properly supported and can’t be evaluated. 2014 © Peter Levesque 2007-2014 63 The care and feeding of knowledge brokers • According to the literature review, a supportive organization is prepared to “create and leverage intellectual capital” (AlBanna, 1999). Its working atmosphere is collaborative, and it has the necessary technology and human assets to allow knowledge to be developed and exploited. Smith (2001) says “supportive, interactive learning environments built on trust, openness and collective ownership definitely encourage knowledge acquisition and sharing.” • Supportive organizations are also prepared to give a free hand to knowledge brokers to do their work, recognizing that “valuable human and knowledge resources will be wasted unless management openly accepts and supports efforts to gather, sort, transform, record and share knowledge” (Smith, 2001). 2014 © Peter Levesque 2007-2014 64 The care and feeding of knowledge brokers • A supportive organization also trusts its brokers; Feldman and colleagues (2001) note that brokers are necessary in the first place because of a lack of trust among people who work together. In a supportive organization, brokers are not seen as an add-on or a burden, because management understands that by encouraging the use of evidence in decisions, they make the whole organization more efficient. Brokers will regularize knowledge transfer, from building relationships with the research community to checking for best practices to ensuring that knowledge is put into use. CHSRF: THE THEORY AND PRACTICE OF KNOWLEDGE BROKERING IN CANADA’S HEALTH SYSTEM 2014 © Peter Levesque 2007-2014 65 The care and feeding of knowledge brokers Knowledge brokering can be done by: • people called “knowledge brokers”; • organizations, agencies and networks; and • by persons integrating brokering into their existing role • Knowledge brokering occurs in the context of networks, projects, programs, issues and organizations • Knowledge brokering can reside and move across domains including: research project-based; network-based; field / program-based; topic / issue-based; and organization-based SHRTN: CANADIAN KNOWLEDGE BROKERING literature review 2014 © Peter Levesque 2007-2014 66 The care and feeding of knowledge brokers • Knowledge brokering is a relationship building process that takes time, as it requires significant informal and formal conversations to build iterations of a solution • Knowledge brokering as a facilitation strategy can address barriers to collaboration and partnerships by facilitating opportunities for reciprocity SHRTN: CANADIAN KNOWLEDGE BROKERING literature review 2014 © Peter Levesque 2007-2014 67 The care and feeding of knowledge brokers • Factors essential to knowledge brokering success include: • Formal external and internal supports and resources (embeddedness) • Opportunities for sustained, intensive engagement with stakeholders • Formal acknowledgement of the role to moderate fragmentation of responsibility and actions (legitimacy) • Facilitating stakeholders capacity to actively engage in the knowledge translation process; • person’s knowledge brokering cannot do so alone SHRTN: CANADIAN KNOWLEDGE BROKERING literature review 2014 © Peter Levesque 2007-2014 68 Failures & Barriers 2014 1. Attempting to apply information technology to tacit knowledge (what is in your head and not yet codified). Explicit knowledge that is codified is most susceptible to the application of information technology. (Complexity) 2. Forgetting that knowledge brokering initiatives must relate knowledge to people’s day jobs. (Incentives) 3. Management says they want it, but everything they do is opposed to it. (Culture) 4. The Field of Dreams trap: “Don’t assume that if you build it, they will come.” There was no incentive for anyone to invest time and energy to solve other people’s problems. (Infrastructure) © Peter Levesque 2007-2014 69 Failures & Barriers 2014 5. Expecting new technology and reengineering of processes to produce a collaborative, sharing culture, where the organization’s greatest need was not new technology but a culture modification program to prepare for a KMb initiative. (Culture) 6. Failure to understand the organization’s willingness to change and to manage people’s expectations appropriately. (Incentives) 7. No process to monitor the quality of contributions. (Management) © Peter Levesque 2007-2014 70 Failures & Barriers 8. The knowledge question is not pertinent to practice. (Incentives) 9. The knowledge is not timely. (Common purpose - Culture) 10. The knowledge is not communicated in ways relevant to users. (Culture – Communication) 11. Management pressures trump the use of evidence in decision-making. (Incentives – Culture) 12. Others? 2014 © Peter Levesque 2007-2014 71 Unknowns • Who should organize the knowledge brokering process and how can it be institutionalized? • How important are attributes of the knowledge brokering (person or agency) to the success of knowledge translation interventions? • Who should be involved in each part of knowledge translation? • In what settings and among different health care professionals is knowledge brokering most effective? • Is the combination of knowledge skills brokering different across settings and among different health care decision makers? • How can we ensure the next important health policy question be facilitated by knowledge brokering? SHRTN: CANADIAN KNOWLEDGE BROKERING literature review 2014 © Peter Levesque 2007-2014 72 Unknowns • What is the optimal preparation and training of KBs. • What are the knowledge brokering characteristics most closely associated with knowledge brokering effectiveness? • What combination of knowledge brokering activities is associated with optimal evidence-informed decisions-making outcomes? • Is there an optimal dose for knowledge brokering? • What are effective strategies to promote participant engagement in the knowledge translation process? • Is there a critical level of engagement between the organization and a person knowledge brokering that is associated with changing organizational culture? • How can people knowledge brokering traverse institutions with different values and cultures? SHRTN: CANADIAN KNOWLEDGE BROKERING literature review 2014 © Peter Levesque 2007-2014 73 Monitoring & Evaluation Read Appendix 6: Measures of Assessment: • More easily measured: – – – – – 2014 Media coverage Web site user stats Publication orders Feedback forms Invitations to speak © Peter Levesque 2007-2014 74 Monitoring & Evaluation • Less easily measured: – Level of community interest & awareness of the issue – Level of political awareness & support: are you on the “agenda”? – Are new policies or practices adopted? – Is funding increased or decreased? – Is there a measurable change in behavior, e.g. improved health outcomes; fewer traffic accidents. Or not? • 2014 Importance of “Buzz” © Peter Levesque 2007-2014 75 Monitoring & Evaluation • Less developed area of practice – emerging – Example: Skinner, K. Developing a Tool to Measure Knowledge Exchange Outcomes, The Canadian Journal of Program Evaluation Vol. 22 No. 1 Pages 49–73 2014 © Peter Levesque 2007-2014 76 Monitoring & Evaluation • 2014 Example: Skinner, K. Developing a Tool to Measure Knowledge Exchange Outcomes, The Canadian Journal of Program Evaluation Vol. 22 No. 1 Pages 49–73 © Peter Levesque 2007-2014 77 Monitoring & Evaluation • 2014 Example: Review and Conceptualization of Impacts of Research/Creation in the Fine Arts © Peter Levesque 2007-2014 78 Monitoring & Evaluation • 2014 Example: VA National Center for Patient Safety – Health care Failure Mode and Effect Analysis (HFMEA) Appendix 7 © Peter Levesque 2007-2014 79 Thank you – Merci Final Questions and Discussion 2014 © Peter Levesque 2007-2014 80