LEARNING THROUGH
EVALUATION
A CASE STUDY ON ORGANIZATIONAL
LEARNING WITHIN THE NONGOVERNMENTAL ORGANIZATION UYDEL
(UGANDA YOUTH DEVELOPMENT LINK) IN
KAMPALA, UGANDA
ELLEN FÜRST
ANNA MEITON
Master of Science in Social Work
91-120 hp
Social Work Programme
January 2013
ABSTRACT
LEARNING THROUGH EVALUATION
A CASE STUDY ON ORGANIZATIONAL
LEARNING WITHIN THE NONGOVERNMENTAL ORGANIZATION UYDEL
(UGANDA YOUTH DEVELOPMENT LINK) IN
KAMPALA, UGANDA
ELLEN FÜRST
ANNA MEITON
Fürst, E & Meiton,A. Learning through evaluation. A case study on organizational
learning within the non-governmental organization UYDEL (Uganda Youth
Development Link) in Kampala, Uganda. Master of Science in Social Work 30
hp. Malmö University: Faculty of Health and Society, 2013.
In a globalized social welfare market, where international and national NGOs
influence social service delivery in community’s worldwide, the demands on
mutual exchange, shared responsibilities, and transnational collaborations has
increased. However, there are some challenges that NGOs face in this work. Lack
in coordination of donor funded projects have enabled overlapping responsibility
in service provision and a rivalry between NGOs in retrieving funds, witch in it
turn has hindered the communication of knowledge between NGOs in the social
work field. To create good practice, the basic assumption is that NGOs have to be
able to learn from past experiences, a process known as organizational learning.
One way of realizing organizational learning, is through evaluation. The success
of an evaluation is further closely linked with the utilization of the knowledge
gained. The purpose of this study is to discover how knowledge gained through an
evaluation made by the Swedish organization IOGT-NTO, has been used within
the non-governmental organization of UYDEL, and how this knowledge is
communicated to the organizations stakeholders. To realize our purpose, we
conducted a field study within UYDEL in Kampala, Uganda. The material,
collected through interviews and observations, was later transcribed and analyzed,
in relation to our theoretical framework. The theoretical framework includes basic
theories on organizational learning, evaluation and communication. Our results
indicate that issues that initially introduced themselves in this evaluation, later
was reintroduced in our material, and thus still remained a problem within
UYDEL. What seemed to hinder organizational learning was the organizations
over-dependency on its donors, which affected its sustainability, and could be
seen as a consequence of the misplacement of recourses within the organization.
Keywords: Communication, Evaluation, International social work, NGO,
Organizational learning, Sustainability
2
ACKNOWLEDGEMENTS
In August 2012 we got the opportunity, thanks to SIDA1 and Malmö University,
to travel to Kampala, Uganda to conduct a minor field study within the
organization of UYDEL2. Minor field studies (MFS) is a SIDA financed
scholarship, which aim to contribute to Swedish student’s awareness about
developing countries and development work. This experience added to our
understanding of the complexity that is modern day social work, and the
similarities and differences in social work practices in Sweden and Uganda. We
would like to thank SIDA for providing us with the means to travel and work in
Uganda, and UYDEL staff, youth clients and others, for showing us great
hospitality and support in our work. We especially would like to extend our
thanks to our field supervisor Mr. Rogers Kasirye for allowing us to conduct our
field study within UYDEL and helping us better understand the Uganda society.
We also would like to extend our thanks to our Malmö University supervisor
Jonas Christensen for offering great support and leading us in trying to understand
the subject of social work, in an ever-changing environment.
Ellen Fürst & Anna Meiton, January 2013
1
2
Swedish International Development cooperation Agency
Uganda Youth Development Link
3
INTRODUCTION
With this study we hope to gain the curiosity of people that are interested in the
development of the social work field, but also social work students that are
interested in social work in an international context. Hopefully our work will also
be of interest of Swedish aid organizations working internationally, and the
organization of UYDEL, where we have conducted this field study.
We would like to highlight some of the problematic issues in international
development work, and especially in relation to NGOs. We find that social work
in a globalized world is an interesting topic, and we would like to understand what
kind of implications the internationalization of social work and social service
delivery has on the community. We want to contribute to a discussion about the
future of social work in an international context, but also make a contribution to
the organization of UYDEL, by sharing their experience of social work practice in
a changing environment.
We think it is important, as social work students entering the professional field, to
always persist in learning and reevaluating social work theory and practice. That
is why we think it is vital to also keep questioning conventional truths about our
profession. Reevaluating social work also means taking part of discussing the
future of the social welfare, both in Sweden and internationally. We recognize that
our society is changing, and we think it is important for all practitioners to
participate in a discussion about the future of social work.
4
ABBREVIATIONS
The following passage provides an overview of the abbreviations that are used
throughout this study, and that are further introduced within each chapter.
NGO: Non-Governmental Organization
SNGO: Southern Non-Governmental Organization
NNGO: Northern Non-Governmental Organization
HSO: Human Service Organization
UYDEL: Uganda Youth Development Link
UN: United Nations
SIDA: Swedish International Development cooperation Agency
UNICEF: United Nations Children’s Fund
WHO: World Health Organization
ILO: International Labor Organization
IOGT–NTO: Independent Order of Good Templars- Sobriety Movement
CAN: Swedish Council for Information on Alcohol and Other Drugs
PM&E: Participatory Monitoring and Evaluation
5
TABLE OF CONTENTS
ABSTRACT
ACKNOWLEDGEMENTS
INTRODUCTION
ABBREVIATIONS
1. BACKGROUND ..................................................................................... 8
1.1 Social work in an international context ............................................. 8
1.2 What are NGOs? .............................................................................. 8
1.2.1 Challenges in NGOs .................................................................. 9
1.3 Evaluation: definition and use ........................................................... 9
1.4 Introducing the Ugandan society .................................................... 10
1.4.1 Vulnerable children and youth in Uganda ................................ 11
1.4.2 Uganda Youth Development Link ............................................ 11
1.5 Knowledge gained .......................................................................... 12
2. PURPOSE AND RESEARCH QUESTIONS ........................................ 13
3. PREVIOUS RESEARCH...................................................................... 13
3.1 NGOs and international social work ............................................... 13
3.1.1
3.2
Spreading knowledge ........................................................... 14
The use of Evaluation in NGOs ................................................... 15
4. THEORY .............................................................................................. 16
4.1. Understanding organization through System Theory ..................... 17
4.2 Understanding the Human service organization ............................. 18
4.3 Network Theory .............................................................................. 18
4.4
Communication ........................................................................... 19
4.5 Knowledge and learning within the organization ............................. 20
4.6 A building block for organizational change ..................................... 21
4.7 A definition of evaluation................................................................. 22
4.7.1 The utilization of evaluation ...................................................... 23
5. METHOD.............................................................................................. 24
5.1 Field Study, preparation and process ............................................. 25
5.2 A Qualitative methodology and Hermeneutic approach .................. 25
5.2.1 Interviews and interview guide ................................................. 26
5.2.2 Observation .............................................................................. 27
5.2.3 Selection of respondents.......................................................... 28
5.3 Analyzing the material .................................................................... 29
5.3.1 Choice of Literature and Theory ............................................... 29
6
5.4 Ethical considerations ..................................................................... 30
5.5 Research weaknesses.................................................................... 30
5.5.1 Study Credibility ....................................................................... 31
5.5.2 The researcher role .................................................................. 32
6. RESULTS AND ANALYSIS................................................................. 32
6.1 Presentation of material .................................................................. 32
6.1.1 The IOGT-NTO evaluation ....................................................... 33
6.2 The use of evaluation ..................................................................... 34
6.2.1 Analysis: How does the organization of UYDEL use evaluation?
.......................................................................................................... 35
6.3 Use of knowledge gained ............................................................... 36
6.3.1 Analysis: Enabling organizational learning ............................... 39
6.4 Internal and external communication .............................................. 40
6.4.1 Analysis: Learning through communication? ............................ 43
7. CONCLUSION ..................................................................................... 44
7.1 How does the organization UYDEL use the knowledge gained? .... 45
7.2 Communication and exchange of knowledge and experiences ...... 45
8. DISCUSSION ....................................................................................... 46
9. REFERENCES..................................................................................... 49
9.1 Books, anthologies and other publications ..................................... 49
ATTACHMENTS
7
1. BACKGROUND
Promoting social protection and justice for vulnerable individuals or groups in
society can be seen as a vital task within the social work profession. According to
author Lynne M. Healy (2008) a prerequisite for achieving change in society is to
use knowledge gained within social work practice. Modern day social work
professionals operate in a globalized social welfare market, where international
organizations influence social work policy and practice. This means that
international organizations play an important role in shaping the future of social
welfare globally. This is also why it is of great importance for social work
professionals to understand not only this development, but the consequences it has
on social work practice on a national and international level (ibid).
1.1 Social work in an international context
Healy (2008) argues that in order to discuss social work in an international
context it is essential to understand the basic scope of globalization and global
interdependency. Globalization can be described as “A process of global
integration in which diverse peoples, economies, cultures and political processes
are increasingly subjected to international influences” (Healy, 2008 p.26). The
basic assumption is that a transnational exchange of production, information,
ideas, authority and people generates global interdependency. Globalization and
an increasing interdependency has specifically affected and shaped social welfare
in different societies nationwide, most importantly through a vast increase in
migration. In the field of social work this has led to greater demands on mutual
exchange, shared responsibilities, and transnational collaborations, both in
relation to organizations working internationally, but also on a national level
(ibid).
The idea of a globalized social work field, where the interventions and social
problems of one society affects other society’s internationally, contribute to the
perception that there is a shared social responsibility that cuts across national
borders, something that in short can be described as international social work.
There are several different explanations as to what international social work is. It
can be identified as professional exchange of knowledge and experience between
social workers working within an international programme, or as development
work on an international level. One basic assumption is that international social
work is carried out by intergovernmental agencies (such as the UN) and includes
governmental agencies and private or non-governmental organizations (NGOs)
which are in some way involved in developing social welfare internationally
through programmes and interventions (Healy, 2008).
1.2 What are NGOs?
David Lewis (2007) recognizes that the term non-governmental organization
originates from the international citizen organizations that were funded by United
Nations after the Second World War. According to the UN definition any private,
non-profit organization that is not part of a government can be stated as a NGO.
With this said, there is not only one definition of the term NGO, for example some
organizations that are in fact income generating can be defined as NGOs, as well
as organizations that are working closely with the government. NGO’s have
specifically been playing an important role in the growth process of developing
countries, including Uganda (ibid).
8
1.2.1 Challenges in NGOs
Roger Riddell (2007) explains that the merge of NGOs as the main contributors of
social service has been subdued to extensive criticism, particularly in relation to
international donors supporting community based NGOs in service delivery in
development countries. Lack in coordination of donor funded projects have
enabled overlapping responsibility in service provision and a rivalry between
NGOs in retrieving funds, witch in it turn has hindered the spread of good
practices between NGOs in the social work field. The fragmentation of the social
welfare due to the non-cooperation between donors and stakeholders can be seen
one of the major problems that is evident within development work. This has also
in many ways affected the quality of service that the beneficiaries in these
organizations receive, for example a shifting quality in service on a community
level, due to lack of standardization of social work practice within the sector
(ibid).
NGOs are often faced with specific challenges. One of them is that almost all of
these organizations are donor dependent; this means that the space that enables an
NGO to act is closely linked with the donors perception of what is needed to
create a successful project. The assumption is that the NGOs have a close link to
the local community, which makes them better suited for delivering quality
service, based on the actual needs in a community (Riddell, 2007). Anthony J.
Bebbington, Samuel Hickey and Diana C. Mittlin (2007) argue that since most
NGOs rely on donors for funding it is crucial that they maintain good
relationships with these partners and at the same time also keep the commitment
of delivering good quality of service to their clients. This may be problematic
because NGOs often have a tendency to focus mainly on maintaining strong
relationships with the donor community, which could divert the attention away
from the needs of their target groups (ibid).
In discussing issues on NGOs, it is central to also discuss the concept of
accountability in relation to NGOs commitment to its stakeholders, according to
author David Fishel (2008). The basic notion of accountability in relation to the
NGO context is that an organization should be held accountable for its actions
towards its stakeholders, including staff members, donors, clients and others that
are affected by the organizations interventions. To achieve accountability, the
organization has to make sure that their mission and goals are realized. The
process of creating accountability in an organization can be realized through
assessing an organizations intervention and spreading information about the
organizations past, present and current actions to its stakeholders. NGOs have
assumed an important role in the delivery of social service and policy making; this
also means that the demands on accountability in these organizations have
increased. Evaluation of projects and programmes within an organization can be
seen as an important step in creating accountability. With this said, many NGOs
often lack in an efficient evaluation and monitoring of programmes and projects,
which can become and obstacle in achieving accountability (ibid).
1.3 Evaluation: definition and use
James Cutt and Vic Murray (2003) define an evaluation as a method of assessing
the outcome of interventions executed in an organization, which should serve as a
base for planning and decision-making regarding future interventions. They
further argue that we do not always know what consequences a project will carry
in the long run, and the ideal evaluation would be able to show, without doubt,
9
that the projects goal has been attained. However, what has become evident is that
the impact of NGO projects and programmes are difficult to measure. The goals
are often too broad, diffuse and long-term, which make it difficult to transform
knowledge into practice (ibid).
Bruce Britton (2005) points out another problem that affects NGOs, which is that
evaluations seldom serve as a base for decision-making, implementation and
planning within these organizations. This seems to be a reoccurring problem even
though many NGOs put great resources into evaluating their activities, which
ultimately stands in the way of transforming information gained through
evaluation, into practice (ibid).
When evaluating, there are many different kinds of technical methods to use to
gain information, but not all of them are equally valid or reliable. In the case of
the NGO sector this could be a problem, regarding the fact that most NGO does
not have the time, money or expertise to develop and interpret these methods of
evaluation. Another problem in the NGO sector is that lack of standards makes it
difficult for the organizations to interpret data. This means that it would be
difficult for an NGO to understand whether an evaluation indicates success or
failure. In addition to this, there are almost no general standards or norms within
this sector, which makes it difficult to compare evaluation results with other nongovernmental projects and programmes. An evaluation is pointless unless it
cannot serve as a base in an organizations decision making. To make a decision,
the organizations have to be able to understand the results of their interventions
and the impact they have on their beneficiaries. This is why it is important that
every evaluation made include explicit models, which declare why the evaluation
is being made, who are involved, what is being measured, how the data will be
interpreted and how the knowledge gained can be used within the organization
(Cutt & Murray, 2003).
1.4 Introducing the Ugandan society
Located in East Africa, neighboring Tanzania, Congo, Rwanda, Kenya and South
Sudan is the country of Uganda3, a former British colony which has been tainted
by political turmoil and un-equality. Years of dictatorship and civil war have
affected the Ugandan society and have had a great impact on the social welfare of
its people. The future of Uganda, one of Africa’s fastest growing economies,
comes with a promise of democracy, freedom for individuals and increasing civil
participation in the development of the country. In spite of the countries recent
success, there are still some challenges that the country faces on its road to
development. Ugandan society suffers from lack of employment opportunities,
social inequalities, corruption and stagnation in health related indicators and
weakness in the social sector management and administration. A weak social
welfare system have had many implications on the Ugandan society, one
consequence of this is the merge of the third sector and NGOs in the delivery of
social service to the Ugandan people (African Economic Outlook, Uganda, 2011).
According to United Nations Population Fund´s report State of Uganda
Population Report (2011), almost half of the Ugandan population, seventeen
million people, is under the age of eighteen years. The increase in population
growth affects the country’s ability to secure the social welfare and protection for
3
Attachment 1, Utrikespolitiska Institutets web page: http://www.landguiden.se/Lander/Afrika/Uganda
10
young people (ibid). Sarah Okwaare and Jennifer Chapman (2006) recognizes, in
the book NGO accountability - Politics, Principles & Innovations, that as a
response to this population growth the numbers of NGOs working within the
social work field have gone from 160 in the mid 1980's to 5,200 in 2004. Today
the NGO sector in Uganda occupies more than 230,000 people (ibid).
The increase of NGOs can also, according to Richard Nuwamanya Butanmanya
(2012), be seen as a consequence the efforts that were put in by international
agencies during the beginning of 1980s, to provide social service in developing
countries. This can be explained by a paradigm shift in international aid
development efforts towards investing and building capacity in NGOs working on
a local level, which was something that was not offered by government during this
period of time. The idea was that organizations working within the communities
had a better perception of the needs of the beneficiaries, and thus could satisfy
these needs in a more sufficient way (ibid). According to Riddell (2007) some of
the important international donors and stakeholders that support NGOs in
developing countries include SIDA, UNICEF and WHO. These stakeholders also
embody a substantial contribution to the social welfare in the Ugandan society
(ibid).
1.4.1 Vulnerable children and youth in Uganda
Author and executive manager of UYDEL (Uganda Youth Development Link)
Rogers Kasirye (2012) explains that the Ugandan government, in an effort to
promote and protect the rights of vulnerable children and youth, has created an
enabling environment for NGOs to deliver social service within this field.
Vulnerability can be described as a situation where "lack of security, susceptibility
to risk and/or exploitation" (Kasirye, 2012 p.1) manifests itself in the life of an
individual. In Uganda, children and youth that are involved in child labor,
trafficking, without education or living in poverty can be defined as vulnerable.
With a growing population of children and youth there is a need to strengthen
social protection within these groups, a challenge that have been proven difficult
for various reasons, one being a lack in resources allocated towards these issues.
Also, lack in coordination and collaboration between social protection providers
have hindered this progress. Insufficient documentation in organizations working
within this field has made it difficult to grasp their capacity and impact on society.
This has also been standing in the way of collaboration, and thus the development
of social protection among vulnerable children and youth in Uganda (ibid).
1.4.2 Uganda Youth Development Link
With the purpose of empowering vulnerable children and youth, and raising
awareness on issues that affect young people in Ugandan society, the nongovernmental organization of UYDEL was founded. The organization is
managing four main projects; child rights protection, HIV prevention, alcohol and
substance abuse and adolescent sexual and reproductive health. The projects are
implemented through five drop-in centers and outreach posts located in Kampala.
The programmes target young people between ages 10-24 years as primary
beneficiaries as well as parents, local leaders, teachers and other community
members as key stakeholders in these different projects. UYDEL is a part of a vast
network of national and international social work organizations and has
implemented a number of projects financed by different international as well as
national organizations including: IOGT–NTO, UNICEF (The United Nations
Children's Fund), ILO (International Labor Organization), WHO (World health
organization) among others (UYDEL Annual Report, 2011).
11
UYDEL is a part of a bigger context, a vast international social work field, where
they can contribute to other organizations through communicating their
experience but also receiving knowledge from other actors within the field, and
especially those who serves as important contributors to the organizations future
and development. This was the case for the Swedish sobriety organization IOGTNTO, one of UYDELs former donors, contributing not only with funds, but also
with knowledge in the shape of evaluation and development workshops.
1.5 Knowledge gained
In October-December 2009 an evaluation4 on the UYDEL Street Children Project
was carried out by IOGT-NTO representatives Sven Persson and Mathias Kimiro
(one of UYDELs main stakeholders). To further understand the concept of
evaluation and monitoring, we have contacted the evaluators from IOGT-NTO,
read and analyzed the material and identified some important issues that could be
seen as relevant in this matter. The purpose of the evaluation was to collect
information about the UYDEL Street Children Project results and impact on other
prevention levels and help to form the future development of the project. The
suggestions made in the evaluation include developing a dissemination strategy to
spread the programmes to partners and other NGOs. To achieve this, they propose
that UYDEL form a functional monitoring and evaluation plan. The monitoring
and evaluation process is aimed at helping the organization in setting priorities,
providing baseline information, identifying problems when they appear,
eliminating unnecessary activities and redirecting resources. The IOGT-NTO
representatives recognized that this would allow UYDEL to replicate successful
projects and avoid making the same mistakes, to increase the accountability in
their projects (ibid).
In the mentioned evaluation (2009) the IOGT-NTO representatives also identify
that a sustainable organization plan and a review of the organization, including
management and board, could create a more transparent management system. This
means that evaluation and monitoring is important tool that should be used
throughout the whole of the organization. IOGT-NTO recognizes that UYDEL has
good credibility as an NGO, and that many people and other organizations come
to learn from their experiences. This is why UYDEL's activities could be extended
locally, nationwide and internationally to build strong networks and partnerships
with both national and international partners. To enable this, the organization has
to be able to understand and interpret the results of the evaluation. In order to use
the knowledge gained, the organization has to develop a functioning
communication strategy. This could also serve as an important base for exchange
of knowledge and experience between local and international actors (ibid).
In 2012 the IOGT-NTO evaluation was followed up by former IOGT-NTO
representative Sven Persson, in collaboration with Swedish Council for
Information on Alcohol and Other Drugs (CAN). A development work shop was
initiated by UYDEL in October 2012, with the purpose of assessing the
organizations capacity and progress.
4
Street Children Project Evaluation Report. IOGT_NTO Movement International Institute (III) Regional
Office East Africa.
12
2. PURPOSE AND RESEARCH QUESTIONS
The purpose with this study is to discover how the knowledge gained through the
IOGT-NTO evaluation has been used within the organization of UYDEL, in
relation to its staff members, beneficiaries and other external actors. Our aim is to
contribute to an understanding about how human service organizations
communicate knowledge and experience to their stakeholders, working within the
social work field. We want to find out how general recommendations from an
evaluation made, can be adapted and translated into practice within UYDEL. This
study is based on two research questions that we aim to answer, as seen below;
-
How does the non-governmental organization UYDEL use the knowledge
gained through the IOGT-NTO evaluation?
-
How does the organization communicate and exchange knowledge and
experiences with other organizations and stakeholders working within the
field of social work?
3. PREVIOUS RESEARCH
In the publication Non-governmental Organization and Development, David
Lewis and Nazneen Kanji (2009) explains how the NGO sector gradually became
subject for research in the late 1980's, as the NGOs role in development work
became more central. It is an interdisciplinary field of scholarship, where
especially economist, sociologists, political scientist and anthropologists have
been studying the whereabouts of the NGO world. Even though the NGO sector
have been discussed and researched on from so many different angels, there is still
a challenge to fully comprehend the phenomena that is non-governmental
organizations. As mentioned in the background, the diversity of the NGO
landscape makes it hard to generalize on this subject (ibid).
Keeping this in mind, the following passage aims to provide an overview of
previous research in the field of social work, with specific focus on the
development of NGOs in an international context. In presenting this material we
have chosen to present publications that can be directly related to the study
purpose and research questions, which include discussions on the use of
evaluation within the NGO sector and the spreading of knowledge across
organizational boundaries.
3.1 NGOs and international social work
As we have discussed NGOs, both national and international are playing an
important role in delivery of social welfare in Uganda. Even though these
organizations are providing vital services within the community, Issa G. Shivji
(2007) argues in his book, Silence in NGO discourse, that there has been an
unwillingness to criticize the impact that NGOs impose on society. He further
identifies some of disadvantages that follow the merge of these organizations;
firstly he establishes that many NGOs are donor funded, which means that their
independence is restrained by the donor and their scope for action is restricted.
This also contributes to the NGO responding to the donor’s opinion of what is
13
needed, rather than advocating for the needs of the beneficiaries. This adds to a
top-down perspective on social welfare, that demotivates the participation of
beneficiaries in the development of social policy and practice. This argument can
be put in contrast to the supposed advantages that support NGOs, which is the
ability to reach the most needy and vulnerable in society (ibid).
In the publication Nongovernmental organizations quest for development Richard
Nuwamanya Butamanya (2012) distinguishes some of the challenges that NGOs
often face in their work. Firstly, he points out a tendency in NGOs to be Urban
centric, which mean that they often based in urban areas, rather than marginalized
areas where service is often lacking. Since NGOs also often are restricted to a
specific area, they are unable to make a wider effect on the social welfare as a
whole. Other challenges in NGOs include inadequate technical capacity and
database, which affect the spread of information internally and externally
throughout the organization, and thus limit the spread of knowledge gained in the
NGOs different programmes and project (ibid).
3.1.1 Spreading knowledge
Since this study concerns itself with understanding how organizations
communicate and exchange knowledge and experiences with its surroundings we
want to highlight some previous research on this subject. When studying
interaction and collaborations in the NGO sector, the literature mainly focus on
partnership between northern NGOs (NNGO) and southern NGOs (SNGO), and
the relationship/partnership between NGOs and the donors. In the article Power in
partnership? An analysis of an NGOs relationships with its partners, Sarah Lister
(1999) discuss the relationship between an southern NGO and one of its donors
from a western country. Lister identifies some components that are important –
not only in relation to the donors - for a successful collaboration between an NGO
and its stakeholders. These include mutual trust and accountability, additive
strength and clear, joint goals followed by mutual decision making, as well as
both partners communicating and exchanging information. Furthermore, there
must be functional tools to evaluate and monitor the work conducted, as well as
transparency in relation to finances and reciprocal commitment to working
together long-term (ibid).
Lister (1999) further state that many NGOs face difficulties in partnering with
others. The most significant reason for this is the fact that donors have control
over funding. The basic notion is that an NGO cannot demand of the donors what
the donors can demand of the NGO. This asymmetrical relationship makes it
difficult to create and obtain functional collaborations and partnership between
these two. Lister also recognizes the importance of personal relationships in
NGOs collaboration with its stakeholders. These relationships are mainly
maintained by management. However, the importance of personal relationships
makes NGOs more defenseless to alteration and challenges in leadership and
management. The success of collaboration is closely connected to how strong the
organizations relationships are. This in turn means that many NGO partnerships
and collaborations are not formalized or institutionalized; instead they are built
upon personal friendships, which mean that the NGO is dependent on those within
the organization who maintain these relationships. Lister argues that these
personal relationships, as a base for interaction, are crucial when trying to
understand how NGOs interact and collaborate within the field (ibid).
14
With this said, the discussion on NGOs is wide in range and there are many
advantages as well as disadvantages in these types of organizations. As authors
David Lewis and Paul Opoku Mensah (2006) discuss in their publication Moving
Forward Research Agendas on International NGOs: Theory, Agency and Context
the subject of NGOs has become complex and controversial, which can be seen as
a result of lack in research within the NGO field. They argue that previous
research has been focusing on casework within organizations, with little or no
emphasis on the organizations context (ibid).
In the book NGOs and Civil Society Anne C Hudock (1999) provides a conceptual
framework for understanding NGOs and their role in the field of international
development. Hudock recognizes that the relationships most NGOs have with
their financial donors determine their capacity to a large extent, and minimize the
control that NGOs have over their own space of action. This relationship is
defined as an interdependent relationship. She further argues that most literature
on NGOs focus on these organizations internal aspects, rather than acknowledging
this problematic relationship. This ultimately leads away from the main question
about how the organizational context controls NGO capacity. The author further
argues that to realize the NGOs sector full capacity in delivering social welfare
internationally, there is a need to understand and question the relationship NGOs
have with external partners such as donors and other stakeholders, both
international and on a local level (ibid).
3.2 The use of Evaluation in NGOs
Evaluation is an essential concept in this study, and specifically the utilization of
evaluation within NGOs. This is why we want to further pin point some of the
challenges, advantages and issues acknowledged in previous research, concerning
this subject.
In the article Who Counts Reality?, Participatory Monitoring and Evaluation: A
literature review, Marisol Estrella and John Gaventa (1998) discuss how the
growth of the NGO sector during the last two decades have raised a discussion
about the need to identify the efforts made within the sector, through evaluation
and monitoring. As mentioned in the background, the NGO sector faces different
difficulties in relation to evaluation and monitoring. Even though the interest in
evaluation and monitoring NGOs has increased, there is still much to be desired
for in reliable evidence on the impact that NGO have on development work (ibid).
Bruce Britton (2005) identifies three main reasons for evaluating and monitoring
the activities of NGOs. Firstly, the purpose of evaluation and monitoring is to
improve the work of the organization, by showing how resources are being used
and acknowledge difficulties, improve management and ensure that the projects
goals and objectives have been achieved. Secondly, Britton identify
communication, between the NGO and its stakeholders, as a reason for evaluating
and monitoring. According to Britton communication is the key to overcome the
difficulties NGOs often face in translating information gained through evaluation
into practice. Thirdly, Britton argues that evaluation and monitoring should serve
as a base for learning, by spreading the knowledge gained through evaluation to
other organizations operating in the same field. This could help other NGOs
repeat the successful interventions, and avoid failures (ibid).
In contradiction to this, Roger C. Riddell (2007) argues that NGOs, as a result of
the liberalization of the NGO market, are forced to compete for donor funding to
15
survive, and therefore tend to be more unwilling to share gained information
between each other. Evaluation and monitoring is crucial in creating
accountability in relation to donors and clients. This is also noted by Estrella and
Gaventa (1998), who focus on the participatory monitoring and evaluation
(PM&E), which is based on the idea that the NGO sector is a central arena for
development work and therefore should involve all central actors in evaluation.
PM&E acknowledge the fact that an NGO have different stakeholders who
directly and in-directly are affected by the intervention implemented by the
organization, and because of this they should be able to express their needs,
interest and expectations and how these can be realized in a specific project. The
stakeholders include for example donors, staff members, government agencies and
beneficiaries (ibid).
In the article Accountability in Practice: Mechanisms for NGOs, Alnoor Ebrahim
(2003) distinguish between external and internal evaluation in relation to
monitoring and evaluation in NGOs. At the end of an intervention, project or
grant, it is common for the donor to do an external evaluation of the
achievements, often in relation to the programme objectives and goals. This kind
of evaluation often serves as a base for the donor in the decision whether to
sustain the programme with funds. Internal evaluation is also common within
NGOs, where staff members evaluate their own achievements in relation to the
goals or objectives of a certain project. Furthermore, how to use evaluation and
for what reasons, can sometimes differ between donors and NGOs. When an NGO
wants to evaluate processes of participation, impact or empowerment, the donors
is often more interested in evident facts and quantifiable results, for example how
many children have been reached or how many schools have been built (ibid).
Ebrahim (2003) also acknowledge the fact that many NGOs are skeptical to
evaluating and monitoring their activities. The idea is that evaluation and
monitoring is something that is being used, for example by donors, to control the
activities of an organization, and not something that could be a used as a
possibility to improve the work within the organization. There is also a tendency
within the NGO sector to act instead of analyzing. A common notion is that the
NGO staff see them self as “doers”, and they get their legitimacy, not by engaging
in time and money-consuming evaluations, but in being in the field and helping
the people they are set out to help (ibid).
4. THEORY
In the previous chapters we have highlighted some of the central issues that many
NGO face in their work. What we have seen is that these issues cannot solely be
attributed to organizations internal affair, but also to circumstances outside the
organization. With this said, the purpose and study questions are based on two
different levels; on one hand we are concerned about the use of evaluation within
the organization, and on the other hand we want to discover how the organization
communicates with its surroundings. These different levels have furthermore
played an important role in our choice, and demarcation, of theories.
Keeping our research questions and purpose in mind, we will start out in a system
theory approach to shed light on some basic ideas on how organizations interact
16
and exchange information. This will also enable an understanding, through
network theory, of how the surrounding context influence one organization, and
vice versa. Furthermore, we will highlight some of the basic definitions of a
Human Service organization. To create an understanding about how an
organization can use the knowledge gained from evaluation, we will study
theories concerning organizational learning and communication. Since we are
interested in organizational learning, we feel the need to recognize certain theories
that could help us explain how knowledge can be exchange across organizational
boundaries, through internal and external communication. We will conclude this
chapter with presenting theories on evaluation, and more specifically the use of
evaluation in Human Service Organization.
4.1. Understanding organization through System Theory
We recognize that in order to discover how organizations interact we need to
adapt theories that can explain this relationship. Kathrine Miller (2006)
establishes in the book Organizational Communication – Approaches and
Processes that the system theory derives from the field of biology and
engineering. Applied to organizational theory, the idea is that organizations
resemble a living organism, a living system. The basic concept is that
organizations – in similarity to organisms – are build up by components. In a
biological system, these components consist of cells and organs. In an
organizational system, the components are the people and departments that form
the organization. The organization, in turn, operates in a larger system; society.
The basic notion is that the organization affects its surroundings and vice versa.
Therefore every organization has to find a way of interacting, and communicating,
with its surrounding in order to survive (ibid).
According to the system theory, organizations are complex social system and
therefore we have to study them as a whole - separating one part from the system
will only reduce the effectiveness of the whole system. Paul Flaa, Dag Hofoss,
Finn Holver-Hoven, Torstein Medhus and Rolf Ronning (1998) argues in the book
Introduktion till organisationsteori (Introduction to Organizational Theory) that it
is important, in relation to organizations, to draw the attention to the exchange of
ideas, funding/money, services, information and human resources between an
organization and its surroundings. This is necessary because no organization is
self-sufficient in relation to information and resources. All organization is, to
some extent, dependent on others to attain their goals and secure their survival.
However, this could signify the development of dependent relationships and
imbalance between organizations, and give external actors the opportunity to
control and influence the organizations actions (ibid).
Lars Skyttner (2006) discuss in his book General System Theory – Problems,
Perspectives, Practice, that an organization is a product of the needs of the
community that they operate in. To survive and succeed, an organization must
know and keep on learning about the society that they work within, and at the
same time remain true to its identity. The organization has to be able to control its
surroundings in order to endure, due to the great influence the environment has on
the organization (ibid). Miller (2006) uses the conception of interdependence. The
idea is that one component of a system rely on other components in the system in
order to function. This applies both within an organization, and for relationships
between the organization and its surroundings. In a globalized world, the concept
of interdependence is apparent to describe the complex relationships between
organizations, and their environment (ibid).
17
Skyttner (2006) further discuss that all system have boundaries, separating them
from their surroundings. There are open systems where the boundaries are
difficult to define, and closed systems with tangible boundaries. Organizations are
mostly categorized as open systems, and the boundaries coincide with the
different events that cause a chain of reactions within the organization. Skyttner
points out that the important thing is not the actions of one individual within the
organization, but the arrangement of these actions. In other words, what we need
to study is how people integrate and work together (ibid). Miller (2006) uses the
term permeability boundaries to describe how organizations let information flow
in and out. How permeable the boundaries are depend on how closed, or open,
the system is – but in order to survive, all social system need to be somewhat
permeable to incoming information both within the system and in relation to its
surroundings (ibid).
4.2 Understanding the Human service organization
Since this study is conducted within an NGO, working with vulnerable children
and youth, we appreciate that we have to recognize the basic scopes of this type of
organization, which can further be defined as a Human Service Organization.
Ingela Thylefors (2007) recognizes in her book Ledarskap i human service
organisationer (Leadership in Human Service Organizations) that human service
organizations (HSOs) can be understood through several different perspectives.
HSOs can be defined as organization that contributes to the social welfare in
society, which main purpose is to influence and transform people into independent
well-functioning citizens. These organizations can be both under the control of
government, or non-governmental. The basic thing that these organizations have
in common is that they are dealing with people in one way or the other. One
perspective within organizational theory is the theory of network based
organizations. This theory focuses on the importance of what happens between
organizations, rather than concentrating mainly on actions taking place within the
organization. The collaboration between different professionals within different
organizations is crucial in the development of the network based organization. She
argues that the gap between needs and resources within these organizations, has
contributed to an increasing need to collaborate with other organizations working
within the same field (ibid). The question then remains, how we can determine
whether a HSO is equipped to collaborate or communicate with other
organizations?
4.3 Network Theory
To shed light on how communication can be used as a tool for interaction with
other actors within the field, and how these affect the work conducted within the
organization we will apply the network theory. According to George Ritzer
(2009), the network theory aims to explain how the behavior of individuals and
organizations can be derived to the structure of the social relationships they are
part of. The basic notion is that these structures have a greater impact on behavior
than norms and values. When organizations interact and network with other
organizations, they tie a connection with each other. This connection in turn
motivate organizations to develop and work together, hence they will reach
consensus (ibid).
Ritzer (2009) further argues that these connections are critical for constructing
communication channels between organizations. These connections can be strong
or weak, nevertheless they fulfill the same function; to communicate and
collaborate with each other. However, without strong connections, the
18
organization become weak and isolated due to the fact that they will not receive
enough information from its surrounding, and eventually the organization will
cease to exist. In order to maintain these connections all involved partners
contribute with information, as well as receive information from the others. When
this happen, there will be collaboration between organizations that is characterized
by the willingness of working towards the same goals, regardless of subjective
ideals (ibid).
To further understand collaboration between organizations and its context we need
appreciate how an organization can receive knowledge. With this said, we would
also like to recognize that the collaboration and exchange of knowledge must be
seen from an internal and external perspective, where the collective as well as
individual learning process holds great importance in the development of an
organization.
4.4 Communication
As mentioned in the background, NGOs faces some challenges that can be
attributed to their ability to communicate knowledge and experience, gained
through evaluation. Since we are interested in understanding how an organization
like UYDEL communicates and share knowledge, we will apply theories on
communication and organizational interaction.
Cindy Gallois, John Gardner, Elisabeth Jones and Bernadette Watson (2004)
declare in the article Organizational Communication: Challenges for the New
Century that there are as much theories about communication and organizations,
as there are theories about communication in general. Communication is a central
conception when analyzing organizations. They further argue that organizations
today are under constant pressure to adapt to economic and environmental
changes. To do this, modern organizations must be able to transform their
communication channels in order to make, and reflect on, new structures,
processes and relationships. As mentioned earlier, boundaries between
organizations and their surrounding have become more and more vague, hence an
increasing need of communication channels overlapping the organizational
boundaries (ibid).
André Jansson (2009) recognize in his book Kommunikation (Communication)
that communication have three essential components. Communication is a
process, not a state of things. Communication happens between individuals,
linking them together. Last, all communication have an object, it contains of
something. Jansson further acknowledge three levels, or structures, for
communication. First, he declares that we need some sort of “code” to be able to
understand each other. Language is the most important tool – or code - for
communication. The second levels consist of infrastructure, different technics we
use to exchange information. In a modern society this includes for example
computers, internet, telephones, radio and TV. The third structure is power.
Jansson claims that all communication takes place between individuals and often
is a reflection on our position in society. What one individual asset material- and
culture wise, determines to what extent that person is able to control their own
communication. At the same time, communication itself helps maintain and reproduce these structures. However when communicating, it is not always clear
where the power lies (ibid).
Flaa, Hofoss, Holver-Hoven, Medhus and Ronning (1998) claim that
communication is the key to organization; without it, there simply can't be any
19
organizations. However, it is not just a matter of communicating. To succeed it is
also important to create a functioning structure for communication. In order to
make a good decision, there is a need to attain adequate information. To avoid
miscommunication and misunderstandings Flaa, Hofoss, Holver-Hoven, Medhus
and Ronning recognizes some possible moves an organization can make in order
to reduce the risk of dysfunctional communication. For example, it is important
for an organization to carefully plan the events of the organization in relation to its
goals, distribution of work and assignments. This will contribute to a better
understanding of where information lack and where there is a need of increased or
decreased communication within the organization, as well as between the
organization and the environment it operates in (ibid).
Authors Jesper Falkheimer and Mats Heide (2007) present another theory on
organizational communication, where they identify three basic types of
communication within an organization. Communication can be hieratical in the
sense that communication is centralized. This means that all decisions are made
within management and eventually spread throughout the organization. This
means that its management who has to communicate information and help the
organizations members to interpret the information gained. Media communication
is defined as internal communication enabled through meetings, newsletters and
other forums for exchanging knowledge. The purpose of this type of
communication is to make information easy to access. Another type of
communication is the one that consist of informal meetings within the
organization and is therefore defined as informal communication. Informal
communication is important in spreading information throughout an organization
and enables the organizations members to create a better understanding about the
organizations goals and interventions. With this said, they also point out that
building structures for communication within the organization is often a low
priority. Organizations tend to access statistical results and the effects that
communication have might be hard to evaluate within an organization, this
ultimately means that creating structured communication within the organization
is often overlooked (ibid).
4.5 Knowledge and learning within the organization
Author Nick Gould (2004) argues in his book Social Work, Critical Reflection
and the Learning Organization that individual learning is important, but not
imperative in organizational learning, it is a collective process that comes about
within the organization. His interpretation of the organizational learning
experience is closely interlinked with a three stage process of learning which
include; elaboration, expansion, externalizing and a constant reevaluation of the
organizations standpoints. The Elaboration face is defined as organizational
learning through practice; The Expansion takes place when an organization
transforms existing knowledge into deeper understanding through research and
Externalizing is the face where gained knowledge and experience lead to a
development in the organization. He further argues that this theory combines
individual learning and collective learning, through focusing on the actual faces in
which members of the organization collectively move toward a mutual
understanding of the information that has presented itself. In short this means;
when groups come together and share their ideas and experience there can be
collective learning, which can contribute to the externalization of learning
between different organizations. Thus, collective reflection can be seen as a
prerequisite for organizational learning. The basic notion of collective learning is
20
that the gained knowledge will be translated into action, and that this
transformation will change the organization for the better (ibid).
Yeheskel Hasenfeld (2010) defines organizational change as “a process that that
occurs in an HSO as a result of external constrains imposed on it or as a result of
internal organizational pressures that cause alterations and modifications in the
organization’s core activity, goals, strategies, structure, and service
programmes” (Hasenfeld, 2010 p.456). The basic notion is that change is brought
about by internal and external factors. The individual influence can be defined as
the meaning that individual members in the organization attribute to the suggested
change or basically the attitudes towards change. The basis for creating attitudes
among the organizations members is access to information and knowledge. The
information and knowledge create a necessary base for interpretation of the
suggested change, without information the individual is prone to discard the
suggested change due to lack of understanding (ibid).
Hasenfeld (2010) introduces a perspective that put emphasis on the more
objective or external oriented perception of bringing about organizational change,
which focuses on measurable modifications. The basic notion is that the
organization can bring about change through changing structures, goals and
programmes within the organization. He further presents different theories that
might be useful in explaining organizational change through knowledge and
information. One approach that might be interesting in particular in this study is
the Stakeholders Approach. This approach recognizes that the HSO is dependent
on its context, which contain of donors, local authorities, governmental agencies
and so on. These stakeholders all have their set of demands and expectations on
the organization. Hasenfeld notes that the more these actors have to gain from the
organization, the more legitimation and recognition the organization receive. This
means that the organization must chart the interests of these actors and respond to
these interests. This can be problematic since stakeholders often have conflicting
interests in relation to the organization, which ultimately makes the process of
organizational change relatively complex since the organization has to satisfy the
needs of many different actors (ibid).
4.6 A building block for organizational change
Thylefors (2007) recognizes the use and spread of knowledge and information as
a key factor in developing an organization. She also establishes that there is a
need for structure in face of change in an organization, especially in relation to its
staff, which can be realized through offering set routines, instructions and the
access to information necessary to create adequate practice. Thylefors further
introduces a model for organizational change, as shown on the next page (ibid).
This model, or building block, can be useful in trying to understand the basics of
what motivates organizational change. With this said, organizational change in
this theory is seen as something positive and rewarding in the development in an
organization, even though Thylefors (2007) notes that change can often pass
through different faces of resistance, trial and error, unfreezing, actual change and
stabilization. Decisions that lead to change might have negative impact which,
and with this said, it is important that the organization try new solutions but also
reject ideas that does not work. Resistance, as we have discovered, can be a
consequence of lacking spread of information but also fear of trying new solutions
within the organization. The unfreezing-process can be defined as motivation
21
among members of the organization to provoke change, for an example higher
level of efficiency or bettering the organizations reputation. The actual change can
be brought about when organizational structures are changed and reevaluated in a
process of trial and error. Stabilization of change in return can be seen as the point
in time when the organization has adapted and accepted the changes made (ibid).
Thylefors (2007) presents five basic building blocks that are important in
changing an organization. Firstly, there is the need for a clear purpose or vision of
the intended change. The vision is the idea of the organization that can only be
realized through competence. Competence is important because it is the key factor
in translating an idea into practice, and to be able to realize this, an organization
needs to attract a fair amount of competence. This brings us to the next building
block which explains the need for motivation in attracting competence, or human
resource. Motivation can be realized through certain rewards, such as education or
wages that are meant to motivate members of the organization towards realizing
change. Another important building block is the organizations resources. Attaining
Resources is crucial in order for the organization to maintain projects running.
When resources are limited or phases out in short term projects it can be difficult
to bring about lasting change in the organization. Resources that are long term are
more lightly to bring about lasting change. Furthermore, a systematic plan of
action is needed to bring about change, which can be realized through identifying
what needs to be change. The lack of these basic building blocks can, as shown
above, lead to confusion among members of the organization, anxiety and
insecurity about the future, resistance, frustration and the feeling of not getting
anywhere or being stuck in the “treadmill” (ibid).
Vision
Competence
Reward
Resources
Plan of action
Change
Competence
Reward
Resources
Plan of action
Confusion
Reward
Resources
Plan of action
Anxiety
Resources
Plan of action
Resistance
Plan of action
Frustration
Vision
Vision
Competence
Vision
Competence
Reward
Vision
Competence
Reward
Resources
”Treadmill”
To create organizational change can be complex, as we have learnt, but the basic
realization of bringing about change is the notion of what actually needs to
change. To be able to recognize what needs to be changed within an organization
we are going introduce some theories about the basic concept of evaluation in
HSOs.
4.7 A definition of evaluation
As we have established previously, evaluation can be defined as a method of
assessing the outcome of interventions executed in an organization, and which
should serve as a base for planning and decision-making regarding future
interventions. Björn Blom, Stefan Morén and Lennart Nygren (2011) argues that
most social work practice concerns itself in realizing changes in the society, and
22
the underlying logic is that these changes are supposed to be positive. To
understand whether these changes are positive or not, is on the other hand almost
impossible to determine, because changes can often be attributed to its context.
The basis for this discussion can be understood through non-realistic theory,
which sees reality as socially constructed, and therefore impossible to interpret
without placing one’s own perspective as a mold for understanding. The basic
assumption is that in evaluation, it is critical that the evaluator realizes that the
interpretation of what is happening in an organization is not always the “correct”
interpretation. This means that looking at the organizational context is important
in evaluation of an organization. By understanding the context, one can give a
truer picture of reality through evaluation, which eventually means that this
knowledge can be translated into practice (ibid).
The use of knowledge and information is, as we have discovered, a complex
question. Organizational change is dependent on internal and external factors, and
is bound to its context in many ways. In order to reach a deeper understanding
about what it takes to go from gained knowledge to practice we are going to study
some basic theories about the use of evaluation.
4.7.1 The utilization of evaluation
According to Evert Vedung (2009), the utilization of evaluation can be described
as a vital part of any HSO. The basic notion is that evaluations can be used as a
tool for systematically examine an organization, with the purpose of discovering
its challenges and opportunities, and ultimately serve as a base for planning and
decision making. Vedung further argues that the underlying purpose of an
evaluation can affect the utilization and how it is received within the organization.
An evaluation can be used to develop the organization and also contribute to
positive change and organizational learning, but at the same time it can be used
with the purpose of controlling whether the organization is reaching its set goals
and standards (ibid).
Vedung (2009) argues that the concept of accountability is central in the
utilization of evaluation. Evaluations that have the primary purpose of controlling
an organization or create accountability, are basically performed because a
specific actor wants to be able to assess the success of an organization and being
able to control the outcome. The basic purpose of this type of evaluation is
creating accountability upwards and placing supervision downwards. Regardless
what the underlying purpose is, an evaluations success is closely linked with the
utilization of the knowledge gained and how the suggested change can be
implemented within the organization (ibid).
Vedung (2009) further identifies some possible ways in which an organization can
utilize evaluations. Firstly, evaluations can be used instrumentally, meaning that
the recommendations that emerge from the evaluation are implemented and used
as a basis for decision making within the organization. The basis for instrumental
use of evaluation is that all recommendations and results have been researched in
a scientific manner, often in relation to a so called engineer model. An engineer
model can be understood as a model or tool for interpreting the relationship
between research, decision making and implementation processes, in a general
sense. The basis notion for this model is that organizations are rational and that it
is possible to generalize processes that take place within organizations through
scientific research, which ultimately mean that organizations would be able to
adapt all knowledge gained. Vedung recognizes that the use of an engineer model
23
should be purely used as an ideal for understanding the utilization of evaluations,
because it goes against the basic idea that organizations are active participants in
organizational learning. Secondly, he identifies the conceptual use of evaluation,
which means that evaluations serve as a base for creating discussion about
interventions and methods, creating new terminology and encourage critical
thinking within the organization. In this sense the evaluation can be used as a tool
for organizational learning, but may not be used as a base for decision-making or
acting on the basis of knowledge gained (ibid).
Vedung (2009) further explains that evaluation can also be used for the purpose of
legitimizing an organization; in this case the evaluation plays an important role in
motivating specific interventions or methods used within the organization. This
means that the evaluation is used to promote existing interventions and the
organizations use the knowledge gained selectively in order to legitimize its
actions. A tactical use of evaluation means that the evaluation process is used as
legitimizing or with the purpose of delaying decision-making within an
organization. The organization, in this case, uses evaluations to display that
changes and development will materialize within the organization, even though
this might not be the case. Some evaluations can be recognized as nothing more
than a routine activity, which has little or no impact on the organization; this can
be defined as discursive use of evaluation. The basic notion is that evaluation is
something that an organization should do, and can correlate with the legitimizing
aspect of utilization of evaluation. With this said, Vedung points out that the same
evaluation may be utilized in different ways over time, which means that an
evaluation that is used to legitimize an organization can eventually also lead to
conceptual use and organizational learning (ibid).
Vedung (2009) acknowledges some of the difficulties that might hinder the
utilization of evaluation within an organization. Evaluations are largely based on
subjective assessments; this means that the actor conducting an evaluation sets the
purpose or criteria in the evaluation. The pursuit of general evaluation criteria or
models that can be transferred into operational guidelines can thus be questioned.
The use of assessments can also be affected by the lack of resources within the
organization or misused in political power struggles, which makes it even more
necessary to distinguish the utilization of evaluation. Organizational development
is based on active participation and dialogue between actors such as evaluators,
the organizations members and stakeholders. This means that the evaluation could
play an important role in learning and the spread of knowledge within an
organization (ibid).
5. METHOD
This study concerns itself with understanding of the structural aspects of the
organization of UYDEL, its work and surrounding context. To realize this we
traveled to Kampala, Uganda and conducted a field study within the organization.
In the following text we intend to describe the process, from planning to
execution, in which we collected the material. We would like to acknowledge that
the process of writing this study is based on a shared responsibility between the
two of us, from gathering the material and analyzing, to the finished product. This
24
means that all the text and different passages are jointly put together and written
by us, in collaboration.
5.1 Field Study, preparation and process
In the end of August 2012 we traveled to Uganda to collect material for our field
study, but the process of planning and preparing the study has been an ongoing
process tracing back to approximately seven months previous to our journey.
According to Ulf Jakobsson (2011) the concept of a field study derives from the
field of anthropology, and it is a method where the researchers are collecting the
material in a natural setting - in real life. What the field consist of is decided by
the research subject. Jakobsson defines a case study as method where the
researchers study a phenomenon (for example a process, a state of condition or an
event) in its natural context, during a limited period of time. However, it is not
only the phenomena itself that is being studied, but the relations and connection
between different variables. The collection of material can be done by using
different methods, such as interviews, observations and through journals (ibid).
We would like to establish that this study can be defined as a field study in the
sense that we are operating within the field, in the reality of the study object. We
have limited our study to a specific case or example which means that our study is
also likely to be labeled as a case study.
Regarding the process of our study, we initially wanted to find information on the
country of Uganda, its history and culture, in order to increase our understanding
about the context in which we would conduct our study within. Previous to our
field study we also did a literature review on the annual reports, previous
evaluations and steering documents of UYDEL, found on the organizations
website. In May (2012) we met with UYDEL’s executive managers in Stockholm,
Sweden, this gave us the opportunity to present our study and make arrangements
for our stay.
Previous to this field study the goal was primarily to identify reoccurring topics
and results that had presented themselves both in evaluations (such as the IOGTNTO evaluation) and annual reports. We wanted to create an understanding about
common dilemmas in the use of evaluation within UYDEL. After reading and
studying this material we started to put into words the basic purpose of our study.
After gaining more knowledge about the organization we realized that our purpose
might not be relevant in relation to the present and ongoing work within the
organization. This led to a reformulation of our purpose, including a broader
perspective on the primary problem. With this we started formulating question
guides and trying to compile as much material as possible previous to our travel to
Uganda. Arriving in Uganda we started collecting material through interviews and
observation, which later was transcribed and analyzed.
5.2 A Qualitative methodology and Hermeneutic approach
This study will take on a qualitative methodology with a hermeneutic approach.
Steinar Kvale and Svend Brinkman (2009) explain that qualitative method can be
used for creating a basic understanding about human nature, through looking at
different characteristic of a specific phenomenon. Understanding these
characteristics can be realized through analyzing material that can be retrieved in
text, interviews and observation. This can be put in contrast to a quantitative
method that involves itself in studying numeric data and identifying variances and
correlations between characteristics or variables, through statistic analyze
25
methods. The idea is that reality is objective and therefore measurable in some
sense. Qualitative methodology, on the other hand, interprets the material based
on how the respondents view the world, which means that it’s subjective. In
quantitative methods the researcher determines the central concepts of the
research to a greater extent. Subjectivity can be seen as the central concept of a
field study. The assumption is that reality is not something that can be objectively
observed; rather the goal is to question this very fact. Reality, according to a
qualitative approach, is based in an understanding that reality is full of
contradiction and individual perceptions; therefore reality cannot be studied
objectively, but through processes in society. The fact that reality is socially
constructed ultimately pinpoints the notion that society is always changing. This
makes it difficult to find an absolute truth. This is why the purpose of a field
study, with a qualitative approach, aims to describe, interpret and understand the
subjective perceptions of a certain community or social group (ibid).
The basis for this study is a Hermeneutic approach, which according to Jakobsson
(2011) can be defined as a scientific approach that involves itself in trying to
interpret, understand a phenomena. The basic notion is that it is possible, through
communication and observation, to understand and draw conclusions about
human behavior. Interpretation and understanding is based on a process where the
researcher’s preconceived perception of a problem is challenged by new
experience, which leads to a greater insight of study object. In trying to
understand our research questions we will further assume an inductive
perspective. Inductive research aims towards understanding and examining a
study object without having connected it to a specific theory, the idea is to study
the different empirical results and create a greater understanding through using a
more general theory. Working inductive provides us with an opportunity to look
at our study object in a relative unbiased way, and enables us to take in important
aspects or themes that spring from interviews and observations (ibid). The basic
assumption is that the results of this study will be able to highlight or create
greater understanding of different concepts of evaluation and organizational
learning. We recognize that we are depended on the respondent’s interpretation of
their own reality as a base for our analysis, but also our subjective understanding
about the problem at hand.
5.2.1 Interviews and interview guide
We have conducted 22 semi-structured interviews and one focus group discussion.
Semi-structured interviews allowed us to be flexible and open to new themes in
the interview situation. This is characteristic for semi-structured interviews; the
design of the interview aims to increase the researchers understanding for the
respondent’s situation through dialogue and discussion. The respondents are able
to reply the questions in their own manner, and the researcher can ask questions
that were not originally in the interview guide, as well as follow-up questions. Jan
Trost (2010) recognizes that the level of structure depends on the similarity
between every interview conducted; is the researcher asking the exact same
question to all respondents, in the same sequence, in the same manner? (ibid).
In this study, the level of structure is low, partly because we had four interview
guides; one for UYDEL staff, one for UYDEL management, one for the
beneficiaries and on for the external actors. This is because we acknowledge the
fact that our respondents have different roles within, and in relation, to UYDEL.
Our purpose was not to do a comparative analysis; our purpose was rather to
26
understand the organization and its context as a whole. To be able to do this, we
had to place our respondents, their role, their perception and angle of approach in
relation to each other and to UYDEL as an organization. In order to get the whole
picture, we needed to ask varied questions. Trost (2010) state that an interview
guide can vary between interviews, bottom line is that the interview guides has to
be comparable and revolving around the same topics.
All of our interview guides was formulated out of our study purpose and research
questions, and from the information we got from the evaluation made by IOGTNTO and UYDELs annual reports, and other documents found on the
organizations website. From this information, we drew up different topics that we
wanted to discuss in the interviews, and under each topic we formulated more
specific questions. These topics include the use of evaluation and communication
strategies within the organization. This way of constructing an interview guide is
also typical for semi-structured interviews (Trost, 2010). Furthermore, we were
open to the possibility that we had to revise our interview guides while in Uganda,
to enable new topics that we were not previously aware of, which might be
relevant to our study object. All interviews were conducted in English, recorded
and later transcribed. The average interview lasted for about 30 minutes. We
conducted all of the interviews together, which enabled us to confirm and discuss
what have been said in the interviews, which allowed us to create a broader
perception on the respondents reasoning’s. The purpose of conducting a focus
group interview with the organizations beneficiaries was to confirm some of the
topics that had arisen in previous interviews, and how these issues affected the
youth clients. Therefore we invited five beneficiaries to participate in a focus
group discussion. Victoria Wibeck (2000) explains that a focus group discussion
can be distinguished from group interviews by the fact that there is a set topic for
discussion. An advantage with conducting a focus group interview is that the
respondents get the opportunity to interact, discuss and ventilate their perception
of the topic at hand, which in this case enabled us to create a deeper understanding
on some of these issues (ibid).
5.2.2 Observation
In collecting the material we had planned to take part in some of the organizations
activities, and we decided to conduct two observations in an effort to contribute to
our understanding of the work that UYDEL does. We observed an information
meeting that UYDEL held in purpose of disseminating information on trafficking
among vulnerable children and youth. We also attended a workshop initiated by
representatives from CAN which involved staff members in discussing the future
of UYDEL. The observations gave us further insight in reoccurring themes that
had come up in interviews. We decided to use observation as a method of learning
because we wanted to add on to our understanding of what actually transpires
within the organization, creating a more diverse picture of what had been said in
the interviews.
Jakobsson (2011) explains that the basic idea of observation is that a process is
observed without direct influence of a researcher, underlining the basic notion that
the observation should remain unbiased and free from misinterpretation to some
extent. We found that observing some of the activities within the organization
allowed us to create a space where we could let the respondents decide which
topics was most relevant to their work. We then identified and retrieved the topics
that were important to the purpose of our study, thus these questions became
relevant for both us as researchers, as well as the respondents. With this said, we
27
recognize that we interpret these observations according to our understanding of
what is actually happening in the process.
Jakobsson (2011) further explains that; participatory observation acquires the
researcher to be present in the process of whatever is to be studied, which means
that the process ultimately will be affected in some way by the researcher and his
or hers presence. Our observations can be defined as participatory, in the sense
that we have actively participated in the context we aimed to study and
understand. Since the aim of this study is to understand some of the work that
UYDEL does, we wanted to gather as much information as possible, and therefore
we saw it as reasonable for us to participate and gain access to all the information
we might be able to receive. We created an observation guide in order to avoid
problems in analyzing and structuring the material and based this guide on the
purpose of our study, in order to recognize what material was relevant.
5.2.3 Selection of respondents
Since the purpose of our study is closely related to the work within the
organization of UYDEL, it was important that our respondents worked with or
had some kind of relationship, or interest in the organization. To be able to
retrieve a broad understanding about the whole organizations perceptions on our
study questions, we needed to interview staff throughout UYDEL; from board
members, to management, to staff members and beneficiaries. To create an
understanding about how UYDEL exchange knowledge and information with its
surroundings we also needed to include external stakeholders. The external
stakeholders include representatives from WHO, UNICEF, ILO, CSF, CAN and
Makerere University. Initially we didn’t intend to include UYDEL youth clients in
the study, but when in Uganda, we realized that it would be valuable for our study
to also include the people that are most important for the organization; the
beneficiaries. In selecting respondents for the focus group, it was important for us
that these respondents would be over the age of eighteen, part of a UYDEL
programme and fluent in English. We also tried to form a group which consisted
of both girls and boys, that could represent the demography of UYDEL
beneficiaries in general.
The initial step in choosing respondents was to analyze the organizational chart of
UYDEL5. We chose to interview at least one person from each level of the
organization, in order to gain an understanding about the work being conducted
on all structural levels. We realized that we were not going to be able to interview
every staff member, therefore this was the selection of respondents we did. Since
we were interested in understanding the organizations surroundings we contacted,
with the help of UYDEL management, external interested parties that had a
connection to the organization, or working in the same field. These people were
selected because of their relationship with UYDEL, but also depending on their
availability. Since our aim was to create a broad picture of our study object, we
remained open to other relevant collaborators and actors in UYDEL's
surroundings. This meant that it would be difficult to limit the selection of
respondents. To avoid this, the study purpose served as a guideline for the
selection of respondents. The selection of respondents can be characterized as
strategic in the sense that we chose specific respondents because of their role in
relation to UYDEL. We also used a method described by Karin Widerberg (2002)
as a snowball method in our selection of respondents, which, in short, means that
5Attachment
4. UYDEL organizational chart
28
the respondent’s pilots to other appropriate interviewees within the same field
(ibid).
5.3 Analyzing the material
Our analysis is based on reoccurring themes that have presented themselves in the
collected material. After completing all interviews, we transcribed the collected
material. Transcribing the interviews took us about one week. When transcribing
we divided the interviews among the two of us, which means that we have
transcribed around 11 interviews each. After we were done transcribing, we read
the interviews, and discussed our material from the observations, in order to
identify common themes in the interviews and observations, but also between the
interviews and the observation. The process of discussing, interpreting and
reflecting upon the material can be described as a process known as coding (Kvale
& Brinkmann, 2009). In coding the material, we looked for certain reoccurring
concepts, or codes, which would help us draw conclusions in relation to our study
questions. Furthermore, we categorized different concepts that could be seen as
related by each other, and these concepts would eventually constitute the basic
themes of our analysis (ibid).
As stated in the previous passage, our aim was to conduct interviews with
respondents from all levels within the organization as well as UYDELs
stakeholders. In presenting the material, we have categorized our respondents into
different groups. Since we were interested in studying the whole organizations,
rather than individuals, we decided that the respondents would be categorized
based on their professional role within, and in relation, to UYDEL. We
interviewed representatives from all levels of the organizations, based on the
UYDEL organizational chart. Each level would be represented by at least two
respondents, which would represent their level in the organization. The following
categorization was made; category A represents external actors, including donors
and Advisory board. category B represents UYDEL management. category C
represents social workers within UYDEL, and category D represents the
organizations beneficiaries. In the result, every theme and quote that has been
presented can be seen as the general perception found at each of these levels in the
organization, as well as among their stakeholders. The categorization was made in
an effort to be able to generalize the result, and draw conclusions about themes
that affect the whole organization.
5.3.1 Choice of Literature and Theory
In analyzing the material, we have applied theories in relation to the different
topics that we had previously recognized in the process of planning this study.
With this said, there are many differences between the Swedish and Ugandan
society, and we recognize that our perspective on the subject matter is closely
linked with our knowledge about social work in a Swedish context. A substantial
part of the literature that we have used in this study can be defined as “western
literature”, even though we argue that this information can be applicable to the
material that we have found. To create a broader perspective on NGOs work and
organizational theory we included literature on NGOs in East Africa. This
literature includes research conducted at Makerere University in Kampala,
Uganda, among other publications produced by East African authors. We
intentionally used relatively broad or general theories on evaluation,
organizational learning and interaction, human service organizations and
communication, to be able to apply these theories and explain organizational
29
structures that are not bound by their context.
5.4 Ethical considerations
To minimize the risk of the respondents being hurt by this study we considered
three basic aspects of research ethics, recognized by authors Henny Olsson and
Stefan Sörensen (2011). These include; demands on spreading information about
the research, retrieving informed consent from the respondents, confidentiality as
an insurance of anonymity and protection of the material, and also the principle
that all gathered material is used simply for the purpose of the study (ibid).
First and foremost, we got approval from the management to conduct our study
within the organization. This meant that we would gain access to information that
would be relevant for our purpose. Initially, we informed the respondents and all
involved on the purpose of the study, and their right to decline their participation
in the study, followed by an assurance not to use any personal information about
the respondent in the study, as well as their right to be anonymous. We also
informed them that we would not use the gained information for any other
purpose than the current study, with the purpose of ensuring the respondents
utility of the material. We will make sure that the respondent is protected, and that
they will remain anonymous throughout the whole study process including
presentation of our study. Furthermore, we are the only ones that have access to
the recorded material. After the study is completed we intend destruct the
transcribed material as well as the recorded, in order to make sure that no one can
use this data in a different context. In addition to this will all respondent gain
access to the published material.
We used consent forms6 as an introduction to our work, but more importantly for
the respondent to agree and consent on their participation. To participate in the
study the consent form had to be read and signed by the respondent. Regarding
the observation, we informed the staff about our presence and that we intended to
use gained information for our study. In the case that some respondents object to
our participation in the workshops, we would withdraw our participation. The
consent forms also included information on the purpose of our study. Our hope
was that this information would add to the staff’s perception of our research
questions, and therefore they would be able to speak more in depth on the subject.
All our respondents are over the age of 18 and can therefore be seen as adults,
which means they would be able to consent their participation. The method in
which we have collected the material have been reviewed and approved by the
Malmö University ethical counsel (registration number: HS60-501:2), with the
purpose of insuring that this study would not cause harm to our respondents or
other people involved in this study.
5.5 Research weaknesses
One thing that became evident during the process of this field study was that there
was a big difference in how the respondents expressed themselves in the
observation, versus the interview situation. In the observation we experienced that
the respondents would express themselves more freely, while during the
interviews we felt they would hold back in some way. We recognize that this is
affected by the choice of method, but also by aspects of formalized meetings and
organizational hierarchies. We experienced that using a consent form in
6
Attachment 5 Consent form
30
interviews would make the situation feel very formal, which might hinder the
respondents to speak freely. This was most apparent among the UYDEL staff
members. In relation to the stakeholders, we felt that the consent form in contrary
legitimized our purpose and our presence in some way. This difference in
attitudes towards the consent form made the organizational hierarchy more
evident to us.
Another thing that might hinder the respondents to expressing themselves freely,
could be that we spent a lot of time in the UYDEL head office, which made coworkers see us as part of the management, and therefore they would define us as
part of the organizational hierarchy. This could lead to a situation where they
portray only positive sides of the organization, in interviews. Our close, or
perceived close, relationship with management can be seen as both positive and
negative. Positive in that sense that we got the opportunity to gain access to
important respondents and information, but negative in that sense that we could be
seen as an extension of management that were set out to control staff members
and their work.
Henny Olsson and Stefan Sörensen (2011) argue that the base for a qualitative
study is that the researcher understands and makes accurate interpretations
regarding the study object. A key to understanding is communication and sharing
a common language, both cultural and linguistic (ibid). We appreciate that
English is not our mother tongue, and this might result in miss-interpretations.
This could also be the case for some of our respondents that might not speak
English as their first language. We also realize that we may have different
perceptions due to cultural references, which could affect our interpretation of the
data. To avoid misinterpretations we recorded the interviews so that we could go
back and listen to the material. This would minimize the risk of
misunderstandings due to language obstacles.
5.5.1 Study Credibility
Jakobsson (2011) explains that in qualitative studies, credibility can be defined as
the level of quality or accuracy that a research attains. This is determined by the
suitability of the chosen method, or the correspondence between the results and
the collected material. One way to ensure credibility is to use triangulation, which
means that a researcher uses different methods in collecting the material, both in
relation to theory, method and source of information (ibid). Therefore, during the
whole study process we have tried to confirm the information that we gained in
interviews, mainly through observation of workshops and feedback from our
respondents, but also by relating different theories and literature to the material
we collected. However, there are limitations to this method. It could for example
be that the respondents give "inaccurate" information, because they may not fully
understand the purpose of the study. Therefore, it is again important to ensure that
all respondents were informed about the purpose of our presence. In the
observation we got the opportunity to observe re-occurring themes that already
had been verbalized in the interviews. The observation gave us a broader
perspective, because we were given an opportunity to understand the respondents
own interpretation of what was important to them, deciding on their own agenda.
This meant that we could gain information that was not affected by our own
preconception of the issue at hand, which had been more the case in the interview
situation (ibid).
31
5.5.2 The researcher role
Steinar Kvale and Sven Brinkmann (2009) describe the researcher’s role as
characterized by the researcher’s moral integrity, their ethical values, knowledge
and experience. To attain a high scientific quality, it is crucial that the study result
reflect the general perceptions of the study object. To be able to realize this, the
researcher has to stay objective, but at the same time keep a close relationship to
its study object, in order to achieve a deeper understanding (ibid). We recognize
that our interpretations and values affect the very idea of this study. We come
from a society where the scope of social security and welfare differs from the
Ugandan society in many ways, and we realize that our perception of social work
practice is based on the society we live in, and have studied. This means that our
perception of social work practice in a Ugandan context will stand in direct
relation to our knowledge and experiences of social work in Sweden. With this
said, the result and analyze is based on the testimonies expressed by our
respondent, which means that they are the ones that have decided what is
important in relation to our research questions. In an effort to avoid our own
perception of social work to affect the result of this study, we have confirmed the
testimonies expressed by our respondents through using feedback.
Regarding the researcher role, we appreciate that the organization has helped us
with practical issues such as housing and transportation. This could lead to a
problematic situation where we were seen as "guests" of UYDEL, rather than
students who were there to study the organizations work. This could become a
problem since our intention was to remain objective, and not end up in a situation
where we felt required to pass one-sided positive images of the organization. With
this said, qualitative research, and specifically field studies, are based on the
researchers close relation to the study object, which makes it difficult to remain
objective as a researcher, and affect the research credibility (Kvale & Brinkmann,
2009). However, we realize that a qualitative methodology and a hermeneutic
approach main weakness lies in the difficulty of remaining objective in relation to
the study object. This of course has to be put in relation to the deeper
understanding of the object that these frameworks allow.
6. RESULTS AND ANALYSIS
6.1 Presentation of material
In the previous sections we have introduced some basic perspectives on
organizational theories, learning and different ways of utilizing evaluation. We
have discussed the value of using communication as a tool for learning but also
the importance that internal and external communication hold in an organization.
In the following passage we aim to give a picture of what has transpired in
interviews and observations within UYDEL. This information will clarify what
has happened since the IOGT-NTO evaluation in 2009, in what way the
organization has learned from past experience and how they exchange knowledge
within the social work field. The material consists of testimonies expressed by
staff-members, beneficiaries and external actors and is based on the purpose of
this study as seen above. As we have explained previously, each level of the
UYDEL organization as well as their external stakeholders, have been divided
into different categories: category A represents external actors, including donors
32
and Advisory board. Category B represents UYDEL management. Category C
represents social workers within UYDEL, and category D represents the
organizations beneficiaries. We will start of by presenting the respondents views
on the use of evaluation and internal and external communication. We will further
conclude every one of these themes by analyzing and discussing the material in
relation to our theoretical framework.
6.1.1 The IOGT-NTO evaluation
This is study is based on information that presented itself in the IOGT-NTO
evaluation, which was conducted within UYDEL in 2009. In the following
passage we would like to re-introduce some of this information and shed light on
what was said in the evaluation. First and foremost we would like to establish that
this evaluation is conducted by one of UYDEL external stakeholders, and
therefore one could argue that the evaluation doesn’t fully reflect the view of the
organizations members. We argue that the view of the organizations members is
in fact reflected in the material, because the evaluation is built on a participatory
design. As mentioned previously, participatory evaluation is based on the idea that
the evaluator should involve all central actors, who are directly or in-directly
affected by the organizations work, in the evaluation. This means that
stakeholders should be able to express their needs, interest and expectations and
how these can be realized in a specific project (Estrella & Gaventa, 1998).
The purpose of the IOGT-NTO evaluation was to collect information about one of
UYDELs projects, their impact and contribution to the development of the
organization. The information was collected from different levels of the
organization, such as beneficiaries, staff members, board members, management
and members of the community. In this evaluation UYDEL is described as one of
the strongest actors, working with drug prevention projects, in the East African
region. UYDEL is also a professional and strategic partner for many international
organizations, working within the field of substance abuse and street children
projects. This demarks the organizations important role in the social work field,
locally and internationally7.
In the evaluation made by IOGT-NTO, the evaluators recognizes that UYDEL has
good credibility as an NGO, and that many people and other organizations come
to learn from their experiences. This is why UYDEL's activities could be extended
locally, nationwide and internationally to build strong networks and partnerships
with both national and international partners. To enable this, the organization has
to be able to understand and interpret the results of the evaluation. The IOGTNTO evaluators recognize that in order to use the knowledge gained UYDEL has
to develop a functioning communication strategy. This could also serve as an
important base for exchange of knowledge and experience between local and
international actors. The suggestions made in the evaluation include developing a
dissemination strategy to spread the programmes to partners and other NGOs8.
What we have learned is that every evaluation should include explicit models,
which declare why the evaluation is being made, who are involved, what is being
measured, how the data will be interpreted and how the knowledge gained can be
used within the organization (Cutt & Murray, 2003). In the following passages we
7
Street Children Project Evaluation Report. IOGT_NTO Movement International Institute (III) Regional
Office East Africa.
8
Street Children Project Evaluation Report. IOGT_NTO Movement International Institute (III) Regional
Office East Africa.
33
will discover how UYDEL has entrusted knowledge gained from the IOGT-NTO
evaluation.
6.2 The use of evaluation
Within the organization of UYDEL there are many different perceptions of the
utilization of evaluation. Some of the respondents see evaluations as a base for
learning within the organization, and recognizes that the knowledge gained should
be used as a base for planning and making decisions within the organization. A
common notion is that UYDEL beneficiaries and staff should affect or be
involved in this process. One shared interpretation among UYDEL stakeholders
and staff is that evaluation can be used to create accountability, mainly in relation
to the organizations donors.One person from category C explains:
“The reason for using evaluation and monitoring is first and
foremost it is tool for accountability /…/ In terms of financial
accountability and other accountabilities /…/ We get to know
where we have succeeded and then we can flag that and tell people
that we are doing this kind of work and it works /…/ We also use it
for mobilizing resources, because then we know that we need to be
focused on a certain component. And it helps us like to keep us
focused on what direction /…/ But most importantly its
accountability toward our donors.”
According to our respondents accountability can be seen in relation to the donor,
but also in relation to the organizations beneficiaries. Accountability is recognized
by category A as something that NGOs attain through accounting for funds
through evaluation. The majority of the staff member’s points out the importance
of delivering adequate service to the clients based on informed decisions. One
person from category C says:
“So, while they are here my role is to see whether the project that
have been working in UYDEL are really achieving their objectives,
by monitoring the way they are being implemented, the tools being
used and make sure that the services are really, that the clients and
the services are reaching their needs, because we are giving
service but does the client enjoy the service? ”
Evaluations contain information on how UYDEL has reached the right target
numbers, organizational goals and displaying the accurate statistics. This type
information is used to show the donors where the funds have been used and how
the organizations have reached its goals for a project, and thus creating
accountability towards the donor community. This is why evaluation becomes an
important tool for resource mobilization. One person from category B states that:
“Decisions that have to do with increased resource mobilization
are all based on monitoring and evaluation. /…/ We are running a
project and we are targeting certain kind of young people, and the
target of numbers - statistics - let me say that we are targeting a
hundred people in a month, all young people, and we don’t reach
the target, which means we will not be able to access the required
resource.”
34
One interesting finding is the importance that evaluation holds in the relationship
with the organizations external stakeholders. One person from category C explains
that:
“Why I am using evaluation, it’s to keep the organization on track,
even to help the donor to realize where the funds have gone, that’s
when accounting comes in.”
The majority of the respondents also claim that the main purpose of evaluating
projects and programmes is to create a base for learning, and also making
informed decisions based on past experience, one person from category A explains
that:
“If I look at the action plan for this year it should be informed by
recommendations and the findings of the assessment of its work in
the past. If not, then there is a problem /…/ I mean our
organization is a learning organization that’s going to look at the
future based on what we are learning now and we are learning
from the past.”
The perception is evaluation of projects and programmes should serve as a base
for decision making within the organization, one person from category C says:
“I think the most important thing for me is to see that we are able
to improve /…/ Evaluations provides the things that a programme
officer needs to be able to make the right decisions, whether a
project is going well or needs to be changed or needs to be done
better. /…/ So we use it to plan for the next programme but also to
know which component has gaps, and that needs to be addressed,
or which component is over-dundent.”
The use of evaluation within UYDEL has several purposes; it is used as a base for
learning within the organization but also in creating accountability towards the
donor community. The basic notion seems to be that planning and decisionmaking should be based on informed decisions that answer to both the clients and
donor community’s needs.
6.2.1 Analysis: How does the organization of UYDEL use evaluation?
As we have established previously the underlying purpose of an evaluation can
affect the utilization and how it is received within the organization (Vedung,
2009). The main purpose for using evaluation within the organization is to create
accountability towards its stakeholder. This affects the information which is used
in reporting and documenting results shown in evaluations made within UYDEL,
which include reaching target numbers, organizational goals and displaying the
accurate statistics. The underlying purpose of evaluation is creating accountability
which means that the use of evaluation focuses on catering to a specific actor that
wants to be able to assess the success of an organization and control the outcome;
in this case the actor can be identified as the organizations stakeholders. The use
of evaluation within UYDEL can be identified as what Vedung (2009) has
described as legitimizing, because the organizations use the knowledge gained
through evaluation selectively in order to legitimize its actions (Ibid).
35
Legitimizing the organizations actions thought evaluation is crucial in the
organizations survival. This can be explained through a system approach
perspective in understanding the organization and its context. What we have
discovered is that UYDEL has what we would define as an interdependent
relationship with its surroundings, and ultimately in relation to its donors.
Interdependence is based on the idea that one component of a system rely on other
components in the system in order to function. All organization is, to some extent,
dependent on others to attain their goals and secure their survival (Miller, 2006).
Interdependency implies that there is an equal dependency between the different
organizations. What has become evident in this case is that the organization of
UYDEL is not only interdependent in relation to their context; they are also
dependent on the donor community to secure their survival. This imbalanced
relationship ultimately gives the donor the opportunity to control and influence
the organizations actions.
This imbalance in the relationship between UYDEL and its donors has different
implications on the use of evaluation within the organization. The majority of our
respondents say that evaluation ought to be used as a tool for organizational
learning. However, the organization faces some difficulties in using evaluation for
these purposes. This can further be explained by the dependency the organization
has in relation to their donors. As we have seen in the result many of UYDELs
projects are short term. Short term projects donor funded projects are evaluated
only in relation to a specific projects and this means that no funds are allocated
towards evaluating the whole organization and thus contribute to its development.
The donor community allocates funds towards service delivery activities that
generate short term results rather than sustainable organizational change. This also
means that the organization cannot make asses the outcome of all the
organizations interventions.
Vedung (2009) states that a basic notion is that evaluation should serve as a base
for decision-making and planning regarding the organizations future
interventions. With this said, we argue that using evaluation with the purpose of
creating accountability affects not only the type of information that is used in
evaluation but also impact the organizations beneficiaries to some extent.
Accountability, according to the respondents, exists in relation to all the
organizations stakeholders, the dependency on the donor community could mean
that the beneficiary’s needs interests and expectations might be hard to answer to
and incorporate in decision making and planning within the organization.
6.3 Use of knowledge gained
In order to create further insight into the use of evaluation within the organization
of UYDEL we will look at how UYDEL has used the knowledge gained in the
IOGT-NTO evaluation. In addition to interviews we have conducted an
observation in a follow-up initiated by CAN. The work shop addressed the same
issues that had previously been introduced in the IOGT-NTO evaluation.
One recommendation that was made in the IOGT-NTO evaluation was that
UYDEL should develop a functional evaluation and monitoring plan. In the work
shop staff members again called out for strengthening in documentation and
internal reflection, in relation to the organizations vision and strategic needs. One
person from category C further explains that:
36
“We have too many tools. /…/ It’s confusing. If I wanted to know
things about drug abuse I have to go to a coordinator on drug
abuse, and sometimes we have so many of these tools, and it’s very
difficult to comprehend or even make a proper analysis.”
One of the problems that are identified is the need for further training in
evaluation and monitoring skills among UYDEL staff. Lack of training might
result in difficulties for staff to assess the outcome of a project or programme.
Another person from category C says:
“You will not be able to analyze the data that you get. It’s very
hard to get sense out of the data that they collect, if a person
doesn’t have M and E skills /…/ It’s also very difficult for an
organization to learn from such data. Initially the M and E person
should be the key point in the organization, because this is a
person who will provide you with a basis of adjustment for your
interventions.”
Many of UYDEL staff members and external stakeholders point out one specific
issue that has affected the use of evaluation, which is that most of the UYDEL
projects are short term. The basic idea is that short term projects stand in the way
of using evaluations that aim at developing the whole of the organization, and
instead examining the effects of one specific project. A person from category A
describes that:
“I also know that there is always a problem that you find that an
organization like UYDEL might have four projects, so when there
is an evaluation, there is an evaluation of each project
independently and not the whole programme///So, it is difficult to
build the future based on an evaluation of individual
projects///each project has its own environment, its own
implementation team, its own objectives specific to it and therefore
it may not easily provide lessons for the future of the organization.
I don’t know how to approach it, but usually these organizations,
they may have some work that they want to be done, but the
challenge we discover is that they don’t figure out so well///I’m
talking about our research or evaluation, they don’t in-build it so
well within their programmes.”
One issue that adds to this problem seems to be that UYDEL does not receive
adequate funding for administrative expenses that could be used for evaluation the
whole organization. One person from category B further explains:
“If the donors could really understand our side of the story /…/ If
there is a way that we could maybe have some funds left for the
organization, even if it’s like five percent or two percent of the
funds which they can leave you with, for you to be able to do follow
up on the next six months and so /.../ For us that would be
important, but you find that when the projects end, so the money
for that programme is also ending, and it becomes difficult for you
to go and follow up with lack of resources.”
37
A person from category B sheds light on some of the reasons that further explain
the difficulty in developing a functioning evaluation plan:
“The follow up after have been a bit weak, I think partly because
of resources in one way or another, because donors only give you
money after the end of a project period. /…/ So how would you as
an organization, if you don’t have any sustainability or income
generating activity, how are you going to follow up?/…/ It’s
becomes a challenge, and you cannot mix funds for follow up, you
can’t do that.”
One thing that becomes evident is how the lack of resources has a negative effect
on the use of evaluation, in relation to the whole of the organization as well as its
beneficiaries. One thing that is brought up by Category D is that lack of resources
means lack in material at the centers, which demotivates learning within this
group. A person from category C further explains how this affects the respondents
in category D:
“Like for instance, in the last evaluation for this project there was
a question that said ´’what else would you want’? Are you satisfied
with the services? And then the young people said, yeah it’s good
to learn the skills at the center and to get the counseling and
information, but we don’t have food. So the center has not
provided lunch, and that is a big problem and even when we
argued it with a donor it wouldn’t work, because that was not their
funding priority.”
Another problem that becomes evident is that the information that emerges from
evaluations is difficult for staff to manage and process. Lack of training and staff
capacity adds to this problem and ultimately leads to insufficient documentation.
One person from category C further explains:
“The workload is too much /…/ You end up doing the work that is
not yours. For example, you may be cornered by a deadline /…/
We had a project and the donors could say ‘you have to report out
every three months’ /…/ And they say you should report like three
hundred children or young people, so when the time comes when
you have to report, you have only 250. And when you go to the
files, the files are not ready and yet as a monitoring and evaluation
officer you have to get the information from the files.”
One problem that becomes evident is not only the lack of recourses but the
misplacement of resources from the donor community. One person from category
A points towards a possible explanation to this problem:
“I think the resources they have to commit to research; they face
competition from other priorities. When they mobilize the funding,
the donors normally would require they commit money to direct
service delivery, and maybe the donors also have not prioritized
research and evaluation so much.”
38
One thing that has become apparent is that short term projects and funding
regimes that does not prioritize research, and this might have a negative effect on
documentation and internal reflection within UYDEL. This problem is recognized
by one person in category C as one major challenge within their work:
“If I don’t have a project running, this is the biggest challenge I’m
having right now. And UYDEL can actually not help out, because
the organization doesn’t have funds to take care of staff when the
project has ended. So that is a big problem with sustainability.”
6.3.1 Analysis: Enabling organizational learning
What has become evident is that organizational learning is hindered by different
aspects that include: lack or misplacement of resources, which further makes it
difficult for the organization to pay salaries and provide material at the different
centers. According to the model for organizational change provided by Thylefors
(2007), lack of resources can create frustration, which is a reoccurring theme in
the majority of the interviews. Another source of frustration is that the
organization is unable to allocate resources where they are needed, for example in
developing tools for evaluation and monitoring. Lack of resources also have had
great implications for the sustainability of projects and programmes, which
according to this model for organizational change will make it difficult to bring
about organizational learning and most importantly bring about lasting change
within the organization (ibid).
Vision
Competence
Plan of action
Reward
Frustration
According to Thylefors (2007), organizational learning can be accomplished
through motivating the organizations members with certain rewards. In UYDEL
many of the respondents are motivated by helping and changing the lives of
young people or making a change in society in general. What has proven difficult
is lack of reward in terms of salaries, which seem to demotivate organizations
members, mainly social workers, but also the beneficiaries. Even though many of
the social workers feel motivated, lack of reward can lead to resistance, which
means that staff members eventually might not be open to the possibility of
learning and change within the organization (ibid).
Vision
Competence
Resources
Plan of action
Resistance
What has become evident in the interviews is that UYDEL has the knowledge,
experiences and ideas that could contribute to lasting change within the
organization and eventually amongst its clients. We recognize that UYDEL has
gone through what Thylefors (2007) label as faces of elaboration and expansion,
where the organization has learned through practice and applied research. With
this said, organizational learning can be seen as a prerequisite for using evaluation
as base for practice and future interventions. Not being able to transform
knowledge into continuous organizational development means that the
organization are not able to realize what can be identified as the externalization
face. Our perception is that UYDEL are able to learn through practice but
organizational learning that leads to lasting development is not achieved because
of a negative spiral where the misallocation of resources leads to a lacking
39
evaluation systems, which eventually leads to inadequate information on
interventions and practices.
We recognize that if the organization adopts, what has been recognized by
Hasenfeld (2010), as an external oriented approach to organizational change,
where they transform the organizational structure towards being less dependent on
external financial support for projects and programmes, this would eventually
enable organizational development. However, the organization is dependent on its
donors and is ultimately tied to something we categorize as Hasenfeld´s (2010)
stakeholders approach to organizational development, which makes it difficult for
them to satisfy their own need for learning in favor for the needs of the donors.
Our understanding is that UYDEL need support to provide basic things such as
administrative budgeting or finances that they can control themselves, for follow
up on the whole of the organization, which also means they will have a bigger
space to act and answer to the needs of the clients.
6.4 Internal and external communication
The IOGT-NTO evaluation established the need for UYDEL to develop a
functioning communication plan. This was later reconfirmed in the workshop
were staff members indicated that a communication plan was still lacking. The
perception among staff was that UYDEL management should orient them in
communication structures and define who should be responsible for
communicating what. They further saw a need for communicational training
among staff to enable standardized communication formats. Another issue that
emerged was need for improvement in social networking, website development
and a budget for communication tools, such as internet access at the different
centers. In relation to UYDELs external communication, a need for formalized
referral systems and MOUs (Memorandum of Understanding) was identified, the
reason being to strengthen external relationships. Staff suggested that UYDEL
should hold regular meetings in the different centers and create a forum that
would make it easier for the clients to communicate their needs and thoughts to
staff and management.
Another thing that was suggested was that UYDEL should strengthen their
network systems. UYDEL communicates with different stakeholders through
different lines of communications, a person from category B explains:
“We share knowledge using annual reports and through
newsletters, and then UYDEL has a website. We share information
through the website, where there is a lot of information that
everyone is free to access. We have a posters and brochures, for
sharing information /…/ Or when we go for trainings and
workshops we acquire material and some good information.”
The organization has a lot of communication externally, both with international as
well as local stakeholders. What has been proven difficult is maintaining and
formalizing external and internal relationships, another person from category B
explains:
“We actually increased the number of NGO's that we network with
/…/ When you see UYDEL now, and the number of people we are
collaborating with, or network we are a part of, you will actually
40
see a big improvement. The challenge is that an effective refuel
system is not yet there. Sometimes it is even difficult to partner
without know the profiles of all the partners, but also to keep in
touch with each and every partner, given the fact that most of the
partner contacts are made by the coordinators, the directors// So,
we seen a lot of improvement in our networking. The only thing
that we do not have is the formalization of this network. /…/ We
have not yet fully developed the database of our local partnership
and our international partnership.”
The majority of the respondents are concerned about the lack in funding which
means that it becomes difficult to afford to produce or purchase information and
material to hand out to the organizations stakeholders. One person from category
B explains that this is not only a problem within UYDEL, but something that is
evident in other organizations that they collaborate with as well:
“I think we still are working with other NGO's, but we are more or less in
the same basket, we are all donor dependent and we have the same
challenges more or less. Yes, we work together as a team; this is a
partner, this one does this and that, this one does work on slum youth,
street children out of school /…/ But you find that you use the same
resources, so now you find that you can only share what's coming out from
the other side of the story /…/ We also encouraging other organizations,
since UYDEL now have the center in Masooli which other interventions
don't have, and most organizations we are partnering with are now able to
send their children or young people from their centers to come and train
at Masooli /…/ I think networking among NGOs is going on because I
know when we are involved with stakeholders forums in trafficking and
task forces, we are seeing people sharing, but we come back to the same
issues of resources”
Another reoccurring theme is the duplication of service among other organizations
working in the field, there is a need to create formalized relationships to enable
the organizations to refer to other organizations working with similar issues, and
thus saving resources. A person from category C explains:
“We have organizations that we work with and we sign MOUs,
memorandum of understandings with them, were you cannot go
they go, where you cannot reach they reach, where you cannot
support, they can support.”
Internal communication among staff and management is provided mostly by
giving and receiving feedback, which has been positive for all involved. In spite
of this there are some problems in communication that staff comments. There is a
need to develop structured channels of communication, mainly through
technology, in form of internet access and telephone privileges, and clear
communication between staff and management. One main issue that arises is how
lack of funding prohibits this development.
UYDEL is an organization that communicates and exchange knowledge and
experiences both externally and internally. One person from category C explains
41
how UYDEL uses networking as a base for learning and in extending their web of
collaboration partners and donors:
“When we collaborate with stakeholders, first and foremost we get
networks, we get partnerships. We partner with those stakeholders
in implementing the projects, they also provide information on the
specific problems in the areas of operation and they can also offer
knowledge about the different methods that we can use to solve the
problems, and even the methodologies to solve the issues that are
in their communities.”
UYDEL has increased the number of partners by using the website as a platform
for spreading knowledge and information on their work. A person from category
C state:
“I think around 2000 read our website, and this has helped us get
more coverage of our services and what we do, and this has led
many people to want and come and partner with us.”
Another person from category C also recognizes that UYDEL can serve as a base
for connecting national and international organizations and donors:
“UYDEL can offer a platform for collaborations and network, in
the country. It can be a focus point for international donors in the
country. It can also provide resource, information on the country,
what works, what doesn’t work.”
However, a majority of the respondents claim that most of UYDELs network are
not formalized, the roles are not clear, which makes it difficult to communicate
and collaborate in an efficient way. One person from category C explains:
“Most of the collaboration is not formalized /…/ So that’s usually
a very big challenge, and sometimes you are not sure about what
your roles are in a collaboration, and once that happens it’s very
difficult for you to collaborate fully.”
UYDEL communicate externally and exchange knowledge through workshops,
seminars, training and capacity building, initiated by the organization, or by other
organizations working with similar issues. This is one way of getting to know the
social work field, exchanging knowledge and learn about the capacity of other
organizations working with vulnerable children and youth. The basic notion is that
this in turn would benefit the clients. One person from category C states:
“Sometimes they call for workshops for organizations that work
with similar issues /…/ You can just go to an organization, found
out what they do, how they do it, and then look at what we do. So
when I get a client that needs help that I actually not can give, I am
able to refer them to some other place.”
Workshops, capacity building and training are also held for the UYDEL staff
members. This is a forum for the staff members to discuss and learn from each
other’s experiences from the field. In the interviews, several staff members also
speak about the workshops as an arena for receiving feedback from their
supervisors and colleagues. A person from category C says:
42
“If I want feedback I can get it. Sometimes I get feedback from my
supervisors, I get feedback through capacity building, through
different workshops that we have.”
Apart from workshops and such, the social workers of UYDEL get feedback from
their supervisors on a regular basis. The kind of feedback they get mainly concern
their weekly reports, which contain figures on how they have reached the projects
goals and target numbers. In the interviews, many of the staff members describe
how this kind of feedback makes them motivated to work harder. One person from
category C states:
“We have quarterly meetings to give us feedback, give an example
on how many young people we have reached in the given quarter
and what the target was before. We get feedback on how we are
doing in the different areas; say information, dissemination, in
reaching out to the community and the young people. Sometimes
we realize that we actually reached a bigger number then the
target. Feedback also encourage us, motivates us. If you are
reaching more young people and impacting them then it's going to
keep us moving.”
6.4.1 Analysis: Learning through communication?
One of our purposes was to shed light on how UYDEL communicates and
exchange knowledge and experiences with other organizations, stakeholders and
international actors working within the field of social work. The basic idea that
motivates our purpose is that communication is the key to overcome the
difficulties that organizations often face in translating information gained through
evaluation into practice as described by Britton (2005). What has become evident
is that the organization still is lacking functional communication structures. This
can be seen as problematic since creating organizational learning, according to
Flaa, Hofoss, Holver-Hoven, Medhus, and Ronning (1998) is based on good
decisions relies on gaining access to adequate information. As we have explained
previously there are three basic levels of communication; described by Jansson
(2009) as language, infrastructure and power. In this case the importance of
infrastructure in communication becomes the most evident. Lack in infrastructure
can be recognized as the lack of technology, such as computers, internet access
etc., something that becomes evident in the case of UYDEL. What becomes even
more evident is that the lack of a functioning communication plan means that it
becomes difficult for the organization to communicate knowledge gained in
evaluation, both internally and externally.
UYDEL uses different forums to communicate knowledge and experience both
externally and internally. Their communication mainly contain of what can be
interpreted as Falkheimer and Heide´s (2007) definition of media communication,
which entail spreading information through the UYDEL website as well as other
information materials like brochures. This means that the organization is able to
spread knowledge and information that is easy to access to all stakeholders as well
as staff members. However, without a structure for communication it is difficult to
appreciate who, how many and to what extent they actually have reached and
affected those whom they want to communicate with. Since media communication
plays a big part in spreading the organizations goals and experience, not having a
43
structure for communication jeopardizes this. Hierarchical communication is
based on the idea that all communicational structures are centralized, and derives
from management (Falkheimer & Heide, 2007). Therefore we argue that lack of
structure therefore can be seen as a management problem. What has become
evident is that management should better support staff members and provide them
access to information and further develop forums for communicating and
exchanging knowledge.
Lack of structure also means that the need for informal communication increases,
according to Falkheimer and Heide (2007). This is also the case in UYDEL,
where informal communication plays an important role. One thing that has proven
to be a challenge is that informal communication, not only in relation to the
organizations internal communication, but also towards the organizations external
stakeholder have led to a uncertainty about the roles in collaboration. UYDELs
collaborations are in many cases not formalized; therefore they can be interpreted
as informal in a sense. Lack in communication structures in this case affects the
way that the organization interacts and communicates knowledge within the field.
An isolated organization will not survive, they must collaborate with others to be
able to change the situation for the clients, as both system theory and network
theory suggest. In this case, we recognize that UYDEL do collaborate and
exchange knowledge and information with many other organizations within the
field, as well as the donor community. The issue, as mentioned above, is the lack
of structure for these collaborations, and this makes it difficult to create a mutual
understanding about the task at hand. The consequence will ultimately be that it
becomes difficult for the involved actors to work towards the same goal. As we
have seen in the result, UYDELs collaboration with other NGOs on a local level
becomes difficult due to the fact that they all are donor-dependent and competes
for the same resources. According to the network theory, the connection that
organizations tie with each other serve as a base for creating functional
communication structures between organizations (Ritzer, 2009). What we have
seen is that networking and spreading information is of great importance for an
NGO like UYDEL to survive, to get funding and to create legitimacy within the
field. The issue, in this case, is not that UYDEL have weak connections or that
they don't exchange information with other organizations within the field, rather
they face difficulties in transforming these connections in to functional
collaborations, due to misplacement of resources that hinders them from develop
administration and structures for communication. This ultimately means that it
becomes difficult for these organizations to fully develop the collaborations and
by that; reach consensus.
7. CONCLUSION
In studying the collected material there are different reoccurring issues that
initially introduced them self in the IOGT-NTO evaluation and later was
reintroduced in the workshop, interviews and observations. First of all many of
the problems that were addressed in the IOGT-NTO evaluation still remain a
problem in the organization. With this said, there are some challenges that the
organization faces. The main issues include the organizations over-dependency on
its donors, which affects its sustainability and which ultimately can be connected
with a lack or misplacement of resources.
44
7.1 How does the organization UYDEL use the knowledge gained?
Our conclusion is that UYDEL uses knowledge gained through evaluation with
the main purpose of communicating with its external stakeholders, and
specifically the donor community which consist of both local and international
organizations. The funding is often project based which affects the organization
negatively in various ways. This problem is well known within the organization
and can be seen as a general problem that affects many NGOs. The fact that
UYDEL relies on donor support is something that is in-built in the structure of
being an NGO. One problem that reveals itself is that short term projects hinder
the organization to create a sustainable future, both in relation to its staff and its
stakeholders. To become self-sufficient, the organization – paradoxically - has to
rely on donor support, which makes it difficult for them to plan for the future.
This means that the organization must chart the interests of these actors and
respond to these interests. This can be problematic since stakeholders often have
conflicting interests in relation to the organization, which ultimately makes the
process of organizational change relatively complex since the organization has to
satisfy the needs of many different actors.
UYDEL has the ambition, knowledge and experience to change the lives of their
clients. Using evaluation as a base for creating accountability rather than
contributing to internal organizational learning hinders the organization from
reaching their full capacity. To some extent this cannot be changed unless more
effort or targeted funds from the donor community are put into development of
the organization as a whole, rather than through specific projects.
7.2 Communication and exchange of knowledge and experiences
To better sustain the organization it is important to maintain an on-going and
effective networking, communication and collaboration with other organizations
within the social work field. However, most of the organizations, mainly on a
local level, that UYDEL collaborate with are also donor dependent NGOs. This
means they are sharing the same resources and because of that they all have
limited possibilities to communicate and share knowledge. Lack in
communication structures and defined roles also add on to difficulties to
communicate both externally and internally. How the organization communicates
is closely related to how they use evaluation within the organization. In this case
the misplacement of donor funds has meant resulted in a lacking evaluation
system which means that inadequate information is communicated back to the
donors, which in turn lead to further misplacement of funds in a vicious spiral that
hinders organizational development.
Sustainability is ultimately connected to resources which mean that UYDEL are
not able to support and empower the community in a sufficient way, because of
project based funding regimes. Frustration is not only evident in relation to work
being done in the community but also among staff members that are affected by
short term projects. Regarding sustainability, UYDEL are in need of long term
programmes in order to enable best possible support and service to the clients.
The assumption is that long-term projects would enable the organization to learn
and understand the impact their interventions have, not on only the clients but the
community as a whole. Being able to evaluate the impact over a longer period of
45
time would contribute to organizational learning and eventually organizational
development.
8. DISCUSSION
In the following passage we would like to discuss some of the topics that have
been introduced previously, as well as issues and subjects that we have discussed
during the process of this study. We will conclude this chapter with suggestions
on further questions, which could be seen as an addition to the conclusions made
in previous chapters.
The fundamental idea of international social work is that the knowledge and
practical experience of social work professionals is shared internationally within a
globalized social work field. The basic assumption that supports this idea is that
organizations that operate within this field are able to share knowledge gained
through practical experience. Being able to use knowledge gained through social
work practice is actually a prerequisite for achieving change in a society. We
recognize that international social work subsequently indicates a shared
responsibility in providing social welfare globally. We argue that the overdependent relationship that UYDEL have in relation to their donor community,
and specifically to its international donors, should serve as an example of the
challenges NGOs faces in general and that this will ultimately hinder the
organization from sharing its knowledge and experience, one thing that became
evident in the results of this study.
As we have stated previously, the basic scope of the NGO as contributors of
social welfare in society is based on the idea that these organizations have a better
perception of the clients and community’s needs, and thus could satisfy these in a
more sufficient way. What we have found is that UYDEL like many other NGOs
has limited capacity to actually use the knowledge gained and translate it into
practice. This makes us question the fact that this type of organization would in
fact be more aware of what the needs of the clients are, but what we have realized
is that the capacity of the organization is to a great extent determined by its
context. UYDEL operate in an organizational field that is characterized by
competitive search for funding, where the donor community sets the agenda
according to what is perceived as the needs of society, as formulated by these
organizations. The problem is not that the organization of UYDEL does not have
information and experience about their clientele; rather it is their environment that
makes them unable to use this knowledge.
Even though NGOs and donors may share the same perceptions on what problems
needs to be addressed in society, the imbalance relationship between these two
forces the NGO to adapt to the donors idea of reality in order to create
accountability and receive funding. This is a minor problem when NGOs and their
donors agree on the needs of the clients, but it becomes problematic when the
NGO face and communicates issues among the clients to the donor, that the donor
in turn doesn’t want to address. This means that one NGO could have a lot of
information and knowledge about what is needed in the community, which they
are not able to translate into practice – at least not if they want funding for their
46
projects - because of this dependent relationship to the donors and to the financial
support they could offer.
Looking at our results there are a few things that have become clear to us, it is
impossible to understand an organization without learning about its context,
which supports the idea that organizations are in fact dependent on their
surroundings. The problem is that this organizational context is difficult to grasp
since most of the challenges within the organization are closely interlinked and
can be seen as a part of an ongoing structure or spiral. We wanted to find out how
UYDEL uses knowledge gained through evaluation, what we found out was that
this is a complex question. What we found interesting is that almost all aspects of
using evaluation and communicating knowledge within UYDEL were closely
linked with funding regimes. This makes us question the idea of NGOs as the type
of organizations that are best positioned to serve the needs of the society. This has
nothing to do with the capabilities of one NGO such as UYDEL. As we have
discovered, UYDEL have plenty of capability in terms of knowledge and
experience. It has to do with the fact that they are not able to act on that
experience that could contribute to organizational learning and development.
We recognize that it is not possible to solve these problems through understanding
an organization as separate from its context; rather there is a need to look at the
bigger picture, to use evaluation to create sustainable organizational change
instead of searching for instant outcomes from short term projects. We would like
to point out that this is an important issue, not only for the Ugandan society. For
different reasons, Uganda among other countries, have a social welfare market
that is mainly occupied by private initiatives and NGOs. These in turn survive on
funding from national and international donors and aid agencies; without these
most NGOs would not live very long. We have established that the context is
crucial to understand in order to comprehend the whereabouts of an organization,
but we also recognize that the problems these local NGOs face cannot be solved
only by studying and understanding the conditions on a local level. The context of
UYDEL extends the borders of Uganda and East Africa, and to find solutions to
the donor-dependency, we need to study the relationships international donors and
local NGOs. What implications does this have for young sex workers, people
infected by HIV/AIDS and slum youth in Kampala that the organizations that are
set out to support them are not able to fully do so, due to complex and overdependent relationships to international donors? As we see it, it is important to
find ways of shifting this dependent relationship to an interdependent relationship
that is characterized by balance and equality, contributes to effective social work
practice and sustainable services to the clients.
Based on this discussion we acknowledge the need to ask further questions on this
subject. In this study, we have been looking at the organization of UYDEL and
the structures that this organization is built upon. We have seen that donordependency is central in understanding and explaining many of the issues
regarding evaluation and organizational learning that UYDEL, along with other
NGOs, faces in their daily work. We realize that what we have presented in this
study has provided a small insight to a study field that is rather extensive. With
this said, we would like to raise further questions about how the donordependency further affect the clients, and how this in turn affect their rights as
members in society, based on human rights and the participatory of the clients in
developing social work practice. Furthermore, we would like to discuss how the
47
donor community could enable organizational learning within the local NGO's
they support, and how aid agencies and the donor community could contribute to
a sustainable future for social service organizations. The underlying question is;
who benefits from a social welfare system where organizational learning and the
needs of the clients are set aside for the needs and wishes of the donor
community?
48
9. REFERENCES
9.1 Books, anthologies and other publications
African Economic Outlook (2012). Uganda 2012.
http://www.africaneconomicoutlook.org/fileadmin/uploads/aeo/PDF/Uganda%20
Full%20PDF%20Country%20Note.pdf. 2012-11-26
Anheier, H K & Seibel, W (1990) The Third Sector- Comparative Studies of
Nonprofit Organizations. Berlin: Walter de Gruyter & Co.
Bebbington, J, Hickey, S & Mitlin, D C (2007) Can NGOs Make A Difference?The Challenge of Development Alternatives. New York: Zed books LTD.
Blom, B, Morén, S & Nygren, L (2011) Utvärdering i socialt arbete. Stockholm:
Natur & Kultur.
Britton, B (2005) Organizational Learning in NGOs: Creating the Motive, Means
and opportunity. The International NGO Training and Research Center,
INTRAC, Praxis paper No.3.
Cutt, J & Murray, V (2003) Accountability and Effectiveness Evaluation in NonProfit Organizations- Routledge Studies in the Management of Voluntary Nonprofit Organizations. London: Routledge
Ebrahim, A (2003) Accountability in Practice: Mechanisms for NGOs. UK:
World Development Vol 31pp.813-821.
Estrella, M & Gaventa, J (1998) Who Counts Reality? Participatory Monitoring
and Evaluation: A Literature Review. UK: Institute of Development Studies.
Fishel, D (2008) The Book of the Board- Effective governance for non-profit
organizations. Sydney: The federation Press.
Flaa, P, Hofoss, D, Holver-Hoven, F, Medhus, T and Ronning, R (1998)
Introduktion till organisationsteori. Lund: Studentlitteratur.
Gould, N & Baldwin, M (2004) Social Work, Critical Reflection and the Learning
Organization.UK: Ashgate Publishing Group
Hasenfeld, Y (2010) Human Service as Complex Organizations. US: Sage.
Healy, L M (2008) International Social Work: Professional Action in an
Interdependent world. USA: Oxford University Press.
Jakobsson, U (2011) Forskningens termer & begrepp- en ordbok. Lund:
Studentlitteratur AB.
Jansson, A (2009) Kommunikation. Malmö: Liber AB.
49
Jones, E,Watson, B, Gardner, J &Gallois, C (2004) Organizational
Communication: Challenges for the New Century. Journal of Communication,
VOL 54, issue 4.
Jordan, L &Van Tuijl, P (2006) NGO accountability – Politics, principles and
Innovation. London: Earthscan.
Kvale, S & Brinkman, S (2009) Den kvalitativa forskningsintervjun. Lund:
Studentlitteratur.
Lewis, D (2007) Management of Non-Governmental Development Organizations.
New York: Routledge.
Lewis, D & Kanji, N (2009) Non-Governmental Organization and Development.
UK: Routledge.
Lewis, D & Opoku-Mensah, P (2006) Moving Forward Research Agendas on
International NGOs: Theory, Agency and Context. Journal of International
Development J. Int. Dev. 18, 665–675 (2006)
Published online: Wiley InterScience
Miller, K (2006) Organizational Communication- Approaches and Processes.
USA: Thomson, Wadsworths
Nuwamanya Butamanya, R (2012) Nongovernmental Organizations Quest for
Development. Uganda: Xypro.
Olsson, H & Sörensen, S (2011) Forskningsprocessen- kvalitativa och
kvantitativa perspektiv. Stockholm: Liber AB
Payne, M (2006) The Origins of Social Work- Continuity and Change. New York:
Palgrave Macmillan
Persson, S & Kimiro, M (2009) UYDEL (Uganda Youth Development Link) Street
Children Project Evaluation Report. IOGT_NTO Movement International
Institute (III) Regional Office East Africa.
Riddell, R (2007) Does Foreign Aid Really Work? New York: Oxford University
Press Inc.
Ritzer, G (2009) Sociologisk Teori. Malmö: Liber AB
Shivji, I (2007) Silence in NGO discourse- The role and future of NGOs in Africa.
Kenya: Fahamu Kenya
Skyttner, L (2006) General Systems Theory- Problems, perspectives, Practice.
USA: World Scientific
Thylefors, I (2007) Ledarskap i human service-organisationer. Stockholm: Natur
och Kultur.
Trost, J (2010) Kvalitativa intervjuer, Lund; Studentlitteratur AB
United Nations Population Fund (2011) State of Uganda Population Report 2011.
http://mail2.unfpa.or.ug/pub/2011/SUPRE2011.pdf 2012-11-27
50
Utrikespolitiska institutet(2012) Landguiden,Uganda.
http://www.landguiden.se/Lander/Afrika/Uganda. 2013-01-03
Vedung, E (2009) Utvärdering i politik och förvaltning. Lund: Studentlitteratur
AB
UYDEL (2011) UYDEL annual report 2011.
http://www.uydel.org/downloads/UYDEL%20annual%20report%20201120121114-085855.pdf, 2012-11-23
Wibeck, V (2000) Fokusgrupper: om fokuserade gruppintervjuer som
undersökningsmetod. Lund: Studentliteratur
Wideberg, K (2002) Kvalitativ forskning I praktiken. Lund: Studentlitteratur.
51
ATTACHMENTS
ATTACHMENT 1: Introducing Uganda, Country facts
Area km2: 241551,00 (2010)
Time: Sweden + 2 hours
Highest Mountain: Margherita (Mount Stanley) (5110 masl)
Major rivers: White Nile
Largest lakes: Lake Victoria, Lake Albert, Lake Edward, Kyoga Lake
Neighboring country / countries: the DRC, South Sudan, Kenya, Tanzania,
Rwanda
Capital, number of inhabitants: Kampala, 1.7 million (2011)
Other major cities, number of inhabitants: Kira 180 000, 154 000 Gulu, Lira
108 000, Mbale 92,000, Nansana 90 000, 90 000 Jinja, Mbarara 84,000 (2011)
Government: Presidential Republic
President: Yoweri Museveni
Social Indicators:
Infant mortality rate (per 1000 live births): 79 (2009)
HIV/AIDS estimates (per cent of total population): 6, 5 (2009)
HIV-estimates among young women (per cent): 4, 8 (2009)
HIV-estimates among young men (per cent): 2, 3 (2009)
Public expenditures in health care as per cent of GDP: 1, 6 (2007)
Expenditures for health care per person (in USD): 1, 6 (2007)
Number of population that have access to clean water (per cent): 67, 0 (2008)
Number of women in Parliament (per cent): 30, 7 (2009)
52
ATTACHMENT 2: Interview guides
Interview guide, UYDEL stakeholders:
Can you tell us something about your background, education and work?
What is your role in relation to UYDEL?
Can you tell us about your collaboration with UYDEL and other NGO´s in
Uganda? What are some of the challenges in these collaborations?
Do you feel that you are involved in the work UYDEL does?
How do you exchange experience and knowledge with local organizations in
Uganda?
Do you have any specific demands or expectations on your collaboration with
UYDEL?
What is your view on NGO´s as the main contributor of social welfare in Uganda?
What kind of impact do you feel that international stakeholders have on social
welfare in Uganda?
Most NGO´s operate in a changing environment, where networking and the
spread of information is a big part of their work. How do you go from talking
about change to actually making a change in a community?
Do you follow up or get feedback on the work you do with UYDEL?
What are some of the challenges in your work right now?
Interview guide, UYDEL staff:
Can you tell us about your role in UYDEL?
What motivates you in your work? What frustrates you?
Do you feel that you are able to influence the different programmes that are
applied within UYDEL? In what way?
Do you get feedback on the work you do within the organization? In what way?
Can you tell us something about how you take part of evaluations made within
UYDEL?- How do you use this information in your work?
53
How do you go from planning a project to executing a project?- Do you involve
the youth in this process? In what way?
Can you tell us something about how you determine the success of a programme?
What tools do you have for evaluating programmes?- Are these tools sufficient?
How do you use evaluation?
What are the reasons for using evaluation?
Do you take part of all the evaluations made within UYDEL?
How do you spread the information gained through evaluation?
Do you feel that UYDEL's collaboration partners and stakeholders can offer
knowledge that is useful in your work? In what way?
Do you feel that you can influence projects that are initiated by your different
stakeholders?
Do you have any expectations on your stakeholders?
Can you tell us something about your collaboration with organizations working
with similar issues on a local level?- Do you share knowledge gained? In what
way?
What are the needs of your clients, according to you?- Do you feel that UYDEL
can meet these needs?
How do you motivate your clients?
What are some of the challenges you are facing in your work right now?
Interview guide, UYDEL beneficiaries:
How did you come in contact with UYDEL?
What motivates you to come here?
What would be the main reason for you not coming here?
What are your expectations on UYDEL?
How can UYDEL improve their work?
Do you feel that UYDEL is listening to your needs? Do you feel that UYDEL are
listening to your opinions?
How do the social workers involve you in different activities? How do the social
workers encourage you?
54
Interview guide, UYDEL management:
What are your thoughts on the workshop at Masooli last week?
According to the evaluation that was done in 2009, the number of children
seeking UYDEL services is bigger than the resources availed. How have you been
working with this problem since then? How can you stay in touch with the target
group?
The evaluation also recommend that UYDEL should support and work with other
NGO's that want's to work with street children programme - how can this be
done? What are the challenges?
In the workshop, they were talking about motivation and throughout our
discussions with staff one common theme is how to keep young people from
dropping out from the programmes. How can this be prevented? What are the
challenges?
One suggestions from the workshop was to improve communication and
technology (through internet, staff support etc.), how can this be done?
Another thing that was mentioned in the workshop, and also in the evaluation
from 2009, was the lack of sustainable programmes. What has been done in this
matter? What have been some of the challenges?
What are some of the challenges in monitoring and evaluating the programmes?
What kind of contribution do you feel that UYDEL can give to your stakeholders,
both national and international? What are some of the challenges in this
collaboration?
How do you use evaluation?
What are the reasons for using evaluation?
Do you take part of all the evaluations made within UYDEL?
How do you spread the information gained through evaluation?
55
ATTACHMENT 3: Observation guide
Workshop observation:
What are some of the main topics that concerns staff members in the workshop?
Are the staff members able to speak freely about the subject at hand?
Are the staff members involved in choosing the subjects?
What questions do the staff members have?
Which one of these topics can be connected to organizational learning, the use of
evaluation and communication?
How do they communicate their experience from the filed amongst each other?
What are some of the attitudes among staff towards the workshop?
56
ATTACHMENT 4: UYDEL organizational chart
The following chart aims to provide a picture of UYDELs organizational
structure. The chart does not reflect all professional titles and staff members
within the organizations, but rather respondents of each level of the organizations
which have participated in this study. The following categorization has been
made:
External Stakeholders
Category A
Advisory Board
Executive Director
Deputy Executive Director
Category B
Senior Programme Officer
Senior Accountant
Programme Officers
Accountants
Monitoring & Evaluation Officers
Category C
Social Workers
Category D
Beneficiaries
57
ATTACHMENT 5: Consent Form
Form
Informed consent
Appendix 2
(submitted along with Appendix 1 to the participant joining the project for signature)
Project title:
Date:
Learning through evaluation – a case study on
organizational learning within the non-governmental
organization UYDEL (Uganda Youth Development
Link) in Kampala, Uganda
Study manager:
Anna Meiton and Ellen Fürst
Your e-mail as a student at Malmö University:
Anna Meiton: hso09037@student.mah.se
Ellen Fürst: kf06830@student.mah.se
30/8-1/11 2012
Studying at Malmö
University, Faculty of Health
and Society, 206 05 Malmö,
Tfn 004640- 6657000
Education: Social Work
Programme
Level: Bachelor
I have been verbally informed about the study and read the accompanying
written information. I am aware that my participation is voluntary and that I, at
any time and without explanation, can withdraw my participation.
I hereby submit my consent to participate in the above survey:
Date: ……………………………………………………………………………..
Participant’s signature: …………………………………………………………
58