Water Implementation Projects In
Rural North India: Evaluating
implementation “on the ground”
Katharine Owens, University of Hartford,
Politics and Government
Marcia Hughes, University of Hartford,
Center for Social Research
Overview of Presentation
Project efforts to date
 Evaluation, implementation, and
collaboration
 Assumptions of this evaluation work and
inherent challenges of project efforts
 Evaluation model and methodology:
Contextual Interaction Theory
 Analyses, findings, and uses of the model
 Recommendations and conclusion

Projects in Abheypur, India

University of Hartford student-faculty
service-learning teams

Professional and student chapters of
Engineers without Borders

Interdisciplinary projects: engineering,
social science, and education/outreach
projects focusing on water.
“On the ground” in Rural India
Abheypur
Delhi
Gurgaon
Pathways
Abheypur
Abheypur, India Project Team
Abheypur, India
The Projects




January 2008: Installation of a solar powered well
pump and water storage system
August 2008: Extension of the water supply to a poor
section of the village
January 2009: Implementation of a rainwaterharvesting system
October 2010: Installation of a potable water supply
well and water storage tank for poorest community
January 2010: Installation of
water drainage improvement
for the “Potter’s Village”
 January 2011: Installation of
water drainage improvement
on main road of the village
a grey
project
a grey
project
Abheypur, Haryana
October 2009 tank location
• Well
• Tanks
• 2 tanks
• Piped to girl’s school
2010 Soak Pit
2011 Soak Pit
•
•
•
•
Solar panels
2 wells
4 tanks
Rainwater harvesting
Main Road Drainage Problem
Design for Soak Pit Implemented in 2011
Gate
Gutter: 6” wide by 6” Deep
Brick Wall
Driveway
New Drain
Continue Drain Across Road
Drain Covered by Grate to Allow Vehicles & Carts to Pass
New Drain
Right-of-Way
Private Farm
Soak Pit
No Gutter – Grade
so swale takes water
along side of road
About 3% grade
from west side
of road to soak
pit
W
N
S
E
Potters’ Village Drainage Problem
BEFORE
Potters’ Village - Soak Pit Design Jan 2010
Two layers (6”) of bricks are used for soil
structural support to keep soak pit open.
Thus, excavated diameter is 1’ wider and
longer than internal dimensions listed.
4’
4’
Greywater
Inflow
Forebay
4’
Soak Pit
Overflow to swale
during for rain events
Stones and
rocks to
reduce water
velocity
Brick Weir
(Width = 3”)
Forebay Depth = 1.5’
Soak Pit Depth = 4’
Forebay and Soak Pit covered with slate
AFTER
Implementation

When political ‘actors’ desire change in
society, they develop and implement
programs to address a perceived problem.

Researchers focus on the implementation
stage of the policy process as a point
where intentions and outcomes diverge

Two different processes: Developing a
program/policy versus implementing a
project/program/policy
Merging evaluation and implementation

Program evaluators seek to quantify and
catalog the successes and failures of
programs and policies.

Implementation policy analysts evaluates
this stage in the process to better
understand participants’ roles and any
impediments to change.

We seek to infuse evaluation with
implementation analysis methodology.
Merging the two

Evaluation researchers find it’s not enough
to know whether outcomes are achieved,
it’s important to understand how.

Using an implementation analysis tool in
evaluation can answer questions about
how people come together and work
towards achieving a project (or fail to).

Scholars call “for responsive, contextual,
flexible, adaptive, multidisciplinary, and
mixed-methods approaches,” particularly
when considering environmental programs
and policies (Preskill, 2009, p. 99).
Collaboration is an imperative
(Gadja, 2004)

Being solo does not work for most of the
problems that need addressing

Need shared efforts to achieve goals that
would not otherwise be attainable working
independently.

More and more, collaboration is becoming
the method for addressing complex social
problems
Understanding collaboration
Wostl et al. (2007) –
In managing water resources in
particular, a paradigm shift is needed,
one that recognizes the importance of
stakeholder involvement and
collaboration.
Water: communal and imperative
Understanding collaboration
 One
goal: To develop communities of
practice.
 Would
we recognize if we were not
working collaboratively with our
Indian partners?

Should we trust our own
conceptions, or evaluate the process
to confirm our goals of collaboration?
What does collaboration look like?
The Practice of Collaboration
Gajda and Koliba (2007) describe key
characteristics of interpersonal
collaboration to be:
 a shared purpose
 dialogue
 decision-making
 action, and
 evaluation (p. 29).
Water Implementation Projects in India:
Inherent Challenges







Working across cultures
Across languages
Across disciplines (engineers and social scientists)
Across class (American middle-class and Indian
impoverished villagers)
Caste distinctions remain present in the village
Gender distinctions (American women in positions
of leadership, which may be atypical)
There is short period to build trust among
partners, implement new projects, and manage
and evaluate previous projects.
This evaluation work

We work to accomplish project
implementation as well as outcome
evaluation in India.

We find it important to engender true
collaboration with India partners.

We use an implementation analysis theory
to evaluate these water projects.
Contextual Interaction Theory

finds project participants and other
stakeholders can influence implementation
in critical ways.

analyzes collaboration between
participants based on motivation,
knowledge, and power, to shed light on
how projects are implemented.

produces a prediction about interaction
based on these core participant
characteristics
Contextual interaction Theory

allows the emphasis on implementation
(the how) as called for by evaluation
researchers.

can provide evidence to those “on the
ground” about their effectiveness in
implementation.

allows large-n studies that enable
researchers to find patterns in large data
sets and enables multi-year studies.
Contextual Interaction Theory

The motivation, knowledge, and power of
the implementer and target are the focus
of the assessment.

Implementer: the student-faculty team,
and a loose coalition including Navjoti, and
the professional Engineers without Borders
chapter

Target: local villagers, including those
residing near the project and those taking
an interest in the implementation of the
project.
Research Questions
How does motivation, knowledge, and
power among the different players affect
implementation of the project?
 What is the relationship between these
characteristics among the people ‘on the
ground’ and the outcomes of the
implementation process?
 How is the project plan or design
implemented through collaboration
processes?

Methods

Faculty-student trip to Abheypur, India in
January 2010.

14 interviews using a semi-structured
interview instrument.

Questions investigate the core elements of
the theory (motivation, knowledge, and
power)

We asked slightly different questions of the
American and Indian counterparts as
reflected their roles in the process.
Independent variables
 Motivation
 Knowledge
 Power
Dependent variables
Active cooperation
 Passive cooperation
 Forced cooperation
 Opposition
 Obstruction
 Joint learning
 No interaction
 Policy learning

[1]
Analysis
Role
M scale=((proportion)-.50) *2, scale of -1.0 to +1.0
[2] I and P scales= (proportion), scale of 0.00 to 1.00
[3] n/a : participant did not answer questions on this concept.
K scale[2]
Motivation
M
scale[1]
(+) 5 of 5
1.00
positive
(+) 4 of 4
(+) 2 of 4
0.00
neutral
(+) 2 of 2
1.00
(+) 1 of 2
Power
P scale
1.00
(+) 2 of 4
0.50
(+) 3 of 6
0.50
(+) 0 of 2
0.00
positive
(+) 4 of 6
0.67
(+) 0 of 2
0.00
0.00
neutral
(+) 4 of 5
0.80
(+) 1 of 2
0.50
(+) 10 of 13
0.538462
positive
(+) 15 of 21
0.7142857
(+)3 of 10
0.30
(+) 3 of 3
1.00
positive
(+) 2 of 2
1.00
(+) 0 of 3
0.00
(+) 2 of 3
0.33
positive
(+) 1 of 1
1.00
(+) 0 of 2
0.00
(+) 2 of 2
1.00
positive
(+) 2 of 2
1.00
(+) 0 of 2
0.00
(+) 3 of 3
1.00
positive
(+) 2 of 2
1.00
(+) 0 of 1
0.00
(+) 1 of 2
0.00
neutral
n/a[3]
n/a
(+) 0 of 1
0.00
(+) 1 of 2
0.00
neutral
(+) 1 of 1
1.00
(+) 0 of 2
0.00
Target: Villagers
(+) 2 of 2
1.00
positive
n/a
n/a
n/a
n/a
Target
aggregate
(+) 14 of 17
Implementer:
Faculty-student
team members
Implementer
aggregate
0.647059
Category
positive
Knowledge
(+) 8 of 8
1.00
(+) 0 of 11
0.00
Analysis
Qualitative assessment
Implementers have some doubts about
the work, their role, and the role of the
target participants in the process.
"…but you know sometimes I wonder maybe
there might be actually better things we
could be doing with our time."
 "I think [with] this project we weren't
communicating with them before we got
there, they didn't know our plan.”
 New understandings of caste subgroups
and inter and intra group dynamics

Analysis
 What
is the potential for other scenarios to
emerge?
 The
strong imbalance of power does not
come into consideration when both actor
groups are in favor of a project.
 But
how might this impact implementation if
one actor was no longer in favor of
implementation?
If the target lacked motivation
If the implementer lacked motivation
Evaluating our collaboration
Recommendations (February 2010)

Community-driven projects based on a
formalized decision-making process.


establish a shared purpose/role among EWB
interdisciplinary team
Always consider the
caste/subcaste/gender/poverty issues and
related power/resources, knowledge, and
motivation among different groups
Recommendations (February 2010)

Make collaboration a goal





Are all stakeholders involved?
Is there a shared purpose and understanding
of the project?
Is there equal decision-making? Has the
community been involved and/or surveyed for
their feedback?
Is there local leadership for the project (i.e., in
the village)?
Expect and plan for disagreement and conflict.
Recommendations (February 2010)

Streamline the collaborative process




Develop project ideas with community leaders
and Navjyoti.
Navjyoti can survey/scan community for
input/feedback on project ideas.
Navjyoti can serve as link between EWB and
village and district leadership (i.e., for
communicating project design development,
and planning).
Fine tune plan and develop budget-tasks and
responsibilities (prior to starting project).
Recommendations (February 2010)

To make previous projects sustainable




Monitoring
Planning
modifications
Monitor systems to ensure proper use and
satisfaction with



system(s),
community leadership, and
overall quality of water
Conclusions

It is both a goal and an obligation to work
in a collaborative way with local groups in
India.

Using the theory we can gain insight into
not only the outcome of implementation
(cooperation), but also provide context for
how and why participants are cooperating.
Conclusions and Next Steps
 Our
analysis also sheds light on how
the people involved in this project
influence it.
 It
allowed us to set meaningful goals
for collaboration and partnership
 Next
Steps: Analysis of 2011 data