Design Development and School Improvement:
Bridging Research and Practice with
Equity-Relevant Interventions in Local Contexts
(working title)
A Practical Guide
Rick Mintrop
with Mahua Baral, Elizabeth Zumpe, and John Hall
Leaders committed to educational equity and justice need to be guided by creativity, moral
purpose, and idealism; yet, evidence plays a large role. Equity-relevant improvements are
about what we value, but also about what works. We need philosophy and deliberate
reasoning to clarify what we value.
Fortunately, as professionals, we can also draw from a developed knowledge base on
effective schools and teaching that reveal factors associated with better student outcomes.
This body of knowledge, however, is relatively unhelpful in telling educators what to do
about specific, but enduring problems of practice. Equity-relevant design development
marries the values and ideals of visionary leaders with the evidence that their efforts
worked. Against the idea of implementation fidelity, it assumes the autonomy of
participants. Less concerned about universal effectiveness or broad generalizations, it
holds that context sensitivity is the base of all successful improvement, but that design
principles can travel.
This book is meant to be a practical guide for educational leaders who strive to contribute
to this knowledge.
The theoretical foundations of design-based leadership in education
Many of us associate the term ‘design’ with the shape of objects (for example, designer
clothes, designer furniture, industrial design, or the design of your smart phone). We often
find these objects pleasing. Harbingers of design-based thinking like Tim Brown (2009)
and Tom and David Kelley (2013) argue that, in our work and our organizations, we all are,
or can be, designers. Designers do not only design objects; they can also design forms of
human interaction that may elicit similar satisfaction among users and observers.
Following this concept of design, in the year 2006, the Graduate School of Education at the
University of California, Berkeley, started a new doctoral program for working
professionals, called the Leadership for Educational Equity Program (LEEP), which
Prospectus of an Upcoming Book Published by Harvard Educational Press
1
experimented with design development methodology as the signature pedagogy for the
practice-oriented educational doctorate. In creating this signature pedagogy, we consulted
an extensive literature on design experiments, design studies, and design development.
The logic of design development
A design connects an actionable problem to an intervention. The task is two-fold. On one
hand, design developers support schools or districts by responding to urgent problems of
practice that pose a design challenge in a specific context. On the other hand, they work in a
systematic fashion towards designs that can be transferred from one organizational unit to
another. Practicality, impact, and transferability are the three most important quality
criteria.
Figure 1 below summarizes the design development logic. First, the problem of practice
(PoP) is identified and a design challenge formulated. To address this PoP, design
development requires a sound theory of action, a sort of model or conceptualization that
predicts how to move from a problematic state to a desirable one. A theory of action is then
used to develop an intervention, which is then implemented and evaluated to see if the
predictions contained in the theory of action were correct.
Figure 1. Design development logic model
This logic model only displays one iteration of conceptualizing, designing, implementing,
and evaluating an intervention – the likely extent of what is feasible for an education leader
within a single academic year. The call for continuous improvement in education
organizations is akin to understanding that a corroborated design would likely take several
iterations, over a longer period of time. Across the iterations, the design can generate a set
of practices that seem to work for an identified problem of practice, practices that give
insight to leaders in other organizations wanting to address a similar problem of practice.
Decisions and design
While many types of decisions are small and made in a flash of recognizing past
experiences in present situations, some are bigger. Big decisions require leaders to ask,
"What is the behavior I am trying to change; what causes the behavior; what learning or
Prospectus of an Upcoming Book Published by Harvard Educational Press
2
unlearning needs to occur for the behavior to change; what kinds of activities, tools, or
structures elicit that kind of learning; how do I know that it has worked?” Decision-making
for these bigger decisions rests on the problem-solving skills that are evoked in a design
process.
The daily flow of work rarely allows practitioners to address the complexity that designbased problem solving requires. Leaders usually only have time and energy for the kind of
problem-solving that is done quickly, using shortcuts: “Strategy X has always worked for
me when I have encountered situation Y, let’s do it.” Or when under pressure, “People who
I trust tell me that Strategy X or Program Y have promise or have shown to work, let’s try
it.” These kinds of shorthand decision rules, called heuristics, are necessary when making
decisions quickly, but they are not good guides for important strategic decisions about new
directions, new programs, new organizational structures, or new professional development
initiatives, because they can often be misleading.
For the most part, educational leadership in disadvantaged contexts is severely constrained
by decisions made elsewhere. Thus, leadership at the local level often ends up being about
making the best out of rather adverse circumstances. But this does not mean that we
cannot make advances towards more equitable services or learning opportunities, in the
action space we control or in the sphere we influence.
Identifying an actionable problem of practice
Just as we have many questions, only some of which are researchable, we are faced with
many problems, only some of which are actionable. In finding a problem of practice (PoP),
the question is not: what am I interested in or what captures my fancy, but it is: what are
the most urgent development problems, encountered in the organization where I work, for
which we need new solutions or improved approaches. Therefore suitable problems of
practice are burning issues in your organization for which people in leadership positions
want answers. Problems of practice specifically deal with practices that are entrenched in
leaders’ own organizations. With a focus on leadership, problems of practice deal with the
way adults deliver services.
In addressing a problem of practice, we should capitalize on all the forces moving towards
humanizing education that we can, but we must also be realistic. Some big problems may
be worthy causes of a professional career, but are not actionable within the span of timeconstrained improvement efforts. This means we need to focus on a problem within certain
feasibility constraints. For example, if the big problem is how to make work teams in
schools commit to closing achievement gaps, then the actionable PoP may be how to move
a ninth-grade mathematics team to develop a program that articulates grade-level
instruction with remediation. Or, if the big problem is how to make instructional
supervision and leadership more effective, the actionable PoP may be how to help
principals give good feedback after informally observing an instructional sequence of
fifteen minutes.
Prospectus of an Upcoming Book Published by Harvard Educational Press
3
Upfront, most changes take additional investment of time, energy, and intellectual and
material resources. Design-based thinking guards against overload and churn and asks of
the professional to think hard about the connection between our intentions and the human
and material resources we have at our disposal to meet our goals. When we do this, and
people around us notice that not only do we have strong values and ambitious goals, but
also a keen understanding of how to get change done given people’s resources, knowledge,
and energy, we can become transformative.
Developing a theory of action
All design begins with intuition. All of us have some pretty good intuitions about our work;
otherwise we would not have had the success that has come our way. We have experienced
repetitive cause-effect relationships in our work. We have experienced what works and
what does not. When we see a behavior that poses a problem, we just know by association
what might work as a remedy.
A theory of action rests upon a diagnosis of a problem, a change process that is predicted to
address the problem, and a clear picture of the desired outcome. We have intuitive ideas for
each of these elements, but in order to make our improvement process more exact, we
must move beyond the intuitive space into a sharper and more conscious analysis.
Challenging our intuitive theory of action (and intuitive problem diagnosis, change process,
and desired outcome) requires the help of knowledge that has been accumulated through
educational research and professional practice.
Local needs assessment
Before we jump into spending time on exploring the knowledge base for our PoP and
thinking about designing interventions, we need to check if our assumptions are correct.
Local needs assessments, often exploratory in nature, aim to question assumptions
regarding the problem of practice; and, local needs assessments can also be used, along
with consulting the knowledge base, to think about remedies or solutions.
Our PoP should focus on beliefs, attitudes, or practices of adults that are members of, or
associated with, our organization. Many of the beliefs, attitudes, and practices (in short,
behaviors) that we think to be problematic are perceptible. We can observe them or we can
hear about them through conversations. So in a first step of the problem diagnosis, we need
to look for these perceptible behaviors - we look at symptoms. In this first round of
problem diagnosis, we stay with low-inference descriptions of these behaviors (i.e., beliefs,
attitudes, practices). We are not making high inferences yet, and we refrain from
associating causes as of yet.
But we will run into limits of perceptibility that social scientists are all too familiar with.
Practices are observable, but we often do not know what they mean without knowing the
actors’ intentions, which are sometimes unacknowledged, subconscious, or concealed. For
example attitudes (e.g., being inspired, motivated, committed, sympathetic, fearful, hostile,
not liking certain people, pretending to agree, etc.) can only be inferred through speech or
actions, and given the uncertainties with speech and action, attitudes can often only be
Prospectus of an Upcoming Book Published by Harvard Educational Press
4
ascertained through relatively higher inferences. In everyday life, we help ourselves with
these high inferences by trusting our intuitions and by reading subconscious cues, such as
facial expressions. But in design thinking, we will need something more robustly verifiable
for our problem diagnosis. So, for all those high-inference states of mind we need clues of
more low-inference behavior that people can display without directly disclosing what is
going on in their minds. This is called operationalizing – making concrete these abstract
high-inference constructs (e.g., motivation, creativity, engagement).
Consulting the knowledge base
Problem diagnosis moves from low-inference evidence of observable behaviors to highinference descriptions of patterns of behavior and the causes of behavior. This shift
involves investigating the underlying or root causes of the problem, or the factors that
explain what contributes to the recurrence of a behavioral pattern or what holds it in place.
Understanding these factors requires consulting the knowledge base to develop some
theoretical knowledge of the macro-, meso- and micro- level dynamics that contribute to
human behavior within schools.
Having consulted the literature to get a better sense of the causal factors underlying the
problem, there is now a need to find out whether these causal factors are evident in the
organization at hand before you begin to think about what kind of change process or
intervention is needed. This is because this second round of local needs assessment data
may shift one’s the understanding of the problem, which will mean that the search for
solutions will therefore need to move in a different direction.
Diagnosing a problem is one part of a theory of action, but a diagnosis alone does not mean
that we know how to treat or cure the disease. From the knowledge gained about the
problem, we can move beyond our intuition in order to construct a theory of change that
predicts how the problem can be remedied.
Theory of change
A theory of change spells out how we believe people will change beliefs, attitudes, or
practices in the desired direction. It specifies the learnings that need to take place, the main
drivers that we believe will motivate the intended change dynamic, and the problemspecific activities in broad strokes. It also addresses the issue of feasibility, since activities
are carried out in specific contexts that afford assets and pose obstacles for
implementation. As with the problem diagnosis, we need to gather ideas from professional
sources, research, and professional experiences that help us understand what to do to get
from Point A to Point B.
Changing the behavior of those who provide services in educational organizations is highly
complex. It involves organizational structures and resources that distribute learning
opportunities. It involves knowledge, skill, and cognitive understanding. It involves
attention, priorities, concentration of work energy, motivation, goals, and demands. But
most of all, it involves shared meanings, interpretations, expectations, norms, values,
rituals, and routines that to a large part are tacit and subconscious and hold together an
enterprise that is at its core full of ambiguity, inconsistency, and contradiction.
Prospectus of an Upcoming Book Published by Harvard Educational Press
5
At the core of change, people are unlearning unwanted beliefs, attitudes or practices, and
learning new ones. Even when we change a rule, use a new instrument, create a new
organizational structure, or allocate new resources, at the core, people need to learn what
to do with these new conditions. From the problem diagnosis we already know what the
unwanted behaviors are. But we do not know how to dislodge them, what to replace them
with, and what learning processes need to unfold in people. Note that we are not yet talking
about activities. We first need to know what learnings or unlearnings need to take place.
But knowing what needs to be unlearned and learned is only part of the equation. You also
have to know how to get that process off the ground within an organization, and this is why
you need to think about drivers. Drivers are the levers or catalysts that can make things
happen within the organization. Some drivers are related to resources – reorganizing roles,
tasks or procedures, allocating resources differently, using new technologies, and
introducing incentives. Other drivers may be related to the power of leadership – goals,
visioning or command and control. There may be professional drivers, as well – training,
collegial reflection, and inquiry.
Once the problem is diagnosed, learnings and unlearnings are identified, and suitable
drivers of change are selected, activities are planned accordingly. These activities work in
specific contexts and rely on local knowledge and ideas. This means that the intervention
must be designed in a way that takes the contextual conditions into account. This ensures
that the design is feasible to carry out within a given time frame of the intervention. But it
also means that, once a theory of change is broadly conceptualized, the design process
needs to shift the focus back to the local context in order to check if the assumptions, now
more thoroughly refined through consulting the literature, still hold.
Baseline
After formulating the theory of action – the problem diagnosis, theory of change, and
activities – but before planning the details of the intervention and getting underway for
implementation, another round of needs assessment data is essential. In this case, the
needs assessment data allows us to test our problem diagnosis at the beginning of the
design experiment by providing a baseline assessment.
The baseline assessment is a much more succinct form of data collection than the previous
rounds of a needs assessment, and is standardized so that the same observations, surveys,
interviews, document analyses, or task analyses can be conducted at the beginning and at
the end of the intervention. The baseline instruments should focus on low-inference
behaviors (operationalizations) that indicate the current state of affairs on the main
components of the design. As a needs assessment, the baseline assessment succinctly
corroborates whether our previous thinking was correct. It also enables us to assess
growth when we repeat the same collection of data at the end of the design.
Prospectus of an Upcoming Book Published by Harvard Educational Press
6
From theory of action to intervention design
An intervention design consists of a sequence of activities that together, or in combination,
intervene in existing knowledge, beliefs, dispositions, or routines in order to elicit new
learning leading to new practices. Activities could revolve around new inquiries, materials,
tools, rules, procedures, or resources that foster opportunities for new practices to take
hold. Interventions are designed. That is, interventions are formal activities planned for a
defined period of time and for a specific organizational setting. Intervention designs are
surface artifacts of underlying theories of action. They are not unlike a teacher’s lesson plan
that lists surface instructional activities, but implies the teacher’s underlying idea of how
students learn the new content.
Planning the intervention activities requires keeping distinct the proximal and distal goals.
Bridging these two kinds of goals speaks to the value of incremental changes in addressing
systemic social problems. While the systemic social problem may be the student
achievement gap, this is a distal goal that cannot be met by a short-term design in a
particular organization. But a designed intervention can take an incremental step towards
the distal goal. It is this proximal goal of the intervention that needs to drive its design,
implementation, and evaluation of impact. For example, in service of the distal goal of
closing the achievement gap, a leader may design an intervention for the proximal goal of
creating an organizational focus around the need for student academic engagement in the
classroom. The impact of the intervention is not measured in its effect on the achievement
gap, but its progress towards the proximal goal. But a sequence of interventions over a
longer period of time should make a dent in the achievement gap.
Moving into the method space
There are many different ways to perceive reality. When we read a good novel, we know
that the characters in that novel are fictitious, yet they teach us something about ourselves.
They reveal a deeper reality to us that fascinates us, yet they are fiction. How can fiction
reveal reality? A good writer uses techniques or methods of narration that make us identify
with characters in her story. We identify because the characters and situations are
recognizable to us, even when the scenery is making the familiar strange. The narration
resonates with us at a deeper intuitive and emotional level. It is perceived as trustworthy,
and its trustworthiness opens us up to the deeper insights that the narrator’s characters
reveal: catharsis, the cleansing of emotions through higher discernments. Narrative
techniques are the appropriate method for a novel.
This is not what we are after in design research. At the core, we as researchers are cool and
detached. We really want to know what works, and it is the beauty of robust findings that
makes us quiver. Narration and trustworthiness are part of the picture, but our “plot”
revolves around a dispassionate connecting of logical and procedural dots. Our goals are
not identification with heroes and catharsis, and we want to avoid fiction at all cost. Rather,
we aim at visibility, verifiability, and transferability.
Prospectus of an Upcoming Book Published by Harvard Educational Press
7
Impact and process data
Design development methodology consists of an ordered sequence of fixed steps that result
in a researchable intervention design. An intervention design becomes researchable when
it consists of a set of planned, prepared, and pre-arranged formal activities around which
an accurate and efficient search for evidence of change and effect can be organized. The
search for evidence proceeds with established or prescribed practices or methods of
collecting, analyzing, and interpreting data that connect baseline, process, and outcome.
Thus, one characteristic of design development methodology is that it requires us to be
mindful of the precision of both the intervention design (a stream of change activities) and
the research design (a stream of data collection activities) that may run parallel. Precision,
however, does not mean inflexibility.
To read about the full design process, look out for the published book with chapters on:
 Implementing Designed Interventions
 Design Principles That Can Travel
Conclusion
Leaders who engage in design-based thinking must themselves be open to learning,
experiment, and critique. They need to base their interventions on testable assumptions
based on evidence that can be disputed by participants. A design connects the dots
between a problem of practice, an intervention that is constructed with a testable logic, and
metrics that indicate to what degree the implemented design helped participants to
express or fulfill identified needs. A design-based approach is appropriate in a work
environment that increasingly demands of leaders to show how their decisions are based
in research and evidence, and that they have produced results. These results need to be
confirmable by participants and verifiable by a community of equity-relevant reformers.
Prospectus of an Upcoming Book Published by Harvard Educational Press
8