1
CCCC 2014 – Panel || The Order in Method
Opening Slide
Why bother writing a methods section? Many published articles do not have one.
But the answer becomes clearer when we consider where research starts.
It starts with hunches or guesses about what is interesting and why. What aspects of
the phenomenon are worth examining and what questions do we want to ask?
It continues with observations of phenomena in the world in rich, 3-dimensional
detail, full of significance, detail, and complexity.
It continues further with reflection on our embodied experiences. Although the
world is rich in potential meaning, we have disciplined ways of experiencing it. We
make the world smaller by shrinking it with our paradigmatic ways of
understanding the world, through our instruments, techniques, and taxonomies for
collecting and processing information that can be taken up into venues of
publication and distribution that expect that kind of output.
In other words, we make the world esoteric, knowable but less accessible to others.
Fleck Slide – Esoteric to Exoteric
One way to see this point is with work of Ludwig Fleck (Genesis of a Scientific Fact)
who traces out the philosophical and material basis of paradigms. In the center of
the diagram, the esoteric circle (aptly named) are scientists that the hard core
journals that they use to circulate information among themselves.
As work on a phenomenon spins out, centrifugally, it gets captured in more general
terms, attached to a wider variety of practices and applications all in order to appeal
to more generalized audiences who then use the research.
First, we take the world and make it esoteric but then we need to make it moveable
into the exoteric for those who do not share our ways of perceiving.
Latour Cycles of Abstraction Slide
The process is cyclical, as Latour has suggested. We start with the big, messy world
and make abstractions of it that strip away some details, turn the world into data
2
that is easier to manipulate and put into papers that can then circulate and
reconnect with the world. Distant planets are easier to study as spectral arrays;
diseases as collections of lab values, writing as a finite collection of observable
practices.
The world is also much smaller (fitting on a notecard) and lighter (weighing no
more than paper) in these forms.
What the methods section represents is the learned ways that we have adopted for
making the world into something that we understand. Methods are sets of rules for
making observations, reducing their complexity, making sense out of them, and
recirculating them for uptake.
Methods and Their Importance Slide
So why bother with methods? We can be more precise about this now.
Smagorinsky Slides
Peter Smagorinsky offers that there are three central components to a methods
section in which this rhetorical work is done.
Go through the slides
But let’s come back to a couple of points running through these guidelines. A
methods section is supposed to communicate what we did and also communicate a
sense of how valid and reliable the outcomes are going to be. As you know, validity
refers to the match between what we observe and how we observe it to the thing in
the world (how true is the match). Reliability refers more to consistency and the
discipline of application.
Of course, validity and reliability are a balancing act. We increase one at the expense
of the other. So while we make more reliable observations of phenomena if we
control where they occur (e.g., in a lab) or if we reduce their complexity in order to
make identification or analysis simpler, we increase reliability but decrease validity.
Likewise, if we observe a phenomenon “in the wild,” we have to give up on the
ability to control what we see and how we might collect data about it that would be
comparable from one observation to the next.
When we present our methods, we are always telling a story about data collection,
reduction, and analysis and making claims about trust and confidence by showing
where our choices are valid, reliable, and (in most cases) limited.
To drive at that point, let’s consider what the practice looks like in our field.
3
Spectrogram Slides
Most of these proceed in a manner that you would expect. They often start with
information about data collection, proceed to data reduction, and sometimes end
with data analysis.
Sprinkled throughout are claims about validity and reliability. Assuring readers that
the data collection choices result in sufficiently valid representations of the
phenomena being analyzed. Or that choices about data reduction and coding are
made reliable through double checking or other verification processes. If
researchers are forthcoming about them, there are also comments about the limits
that one can take an analysis. We see aspects of all in these three articles.
And we can look more closely (time allowing) at some of the techniques used to
make the methods work:
Data Collection and Validity Slide
Claims about the validity of data collection procedures, assure that the participants
for the study are representative of the populations or regions about which the
analysis makes claims
Data Reduction and Reliability Slide
And here the purpose is to show that the choices made about data reduction follow
a set of accepted principles, like Grounded Theory, or that the process of reduction
itself was verified independently by another researcher.
Data Analysis and Validity Slide
In these passages, the point is to show that the analysis was verified by comparison
to the world of data from which the sample was taken and sometimes even actively
so, by the people being studied.
4
Limitations Slide
Finally, some methods sections offer some perspective on the limitations of the
collection, reduction, and analysis. All studies have limitations to them, but the
studies differ in terms of how directly they note these limitations.
Closing Slide
Although there is more nuance to the method section than we have time to cover in
the short time that we have here today, it is our hope that this overview clearly lays
out some of what is at stake, rhetorically.