‘Blue remembered hills’? –
A qualitative appreciation
of Quantitative
Methodology
Ian Shaw
PhD Summer School, 6 June 2013
A position

there are real and significant differences amongst the s/w research
community that cannot and should not be dissolved, either into some abstract
level of agreement, or into a pragmatic consensus of one or other variety.

I should always assume - unless there is what seems good evidence to the
contrary - that those who hold a view I happen not to share do so based on
effort to think carefully and authentically about it, and that they have as
much right as me to have their voice heard.

social work research cannot be understood if it is sealed off from wider social
science.

Past and present
Past and present?

Into my heart an air that kills
From yon far country blows:
What are those blue remembered hills,
What spires, what farms are those?

That is the land of lost content,
I see it shining plain,
The happy highways where I went
And cannot come again.
A E Housman
Not an overview of the merits of
quantitative methods

Though I have done several substantial quantitative studies, I am not a
quantitative methodologist.

A qualitative appreciation of quantitative methods

The origins of this presentation – and session.
References

Shaw, I. ‘Serendipity, Misfires and Occasional Patterns: A career in social work
research’ in I Shaw (2012) Practice and Research Aldershot: Ashgate
Publications.

Shaw, I. (2012) ‘The Positive contributions of quantitative methodology to
social work research: a view from the sidelines’ Research on Social Work
Practice 22 (2): 129-134.

Shaw, I. and Norton, M (2008) ‘Kinds and Quality of Social Work Research’ Br
Journal of Social Work 38 (5): 953-970.

Shaw, I. (2008) ‘Merely Experts? Reflections on the History of Social Work,
Science and Research’, Research, Policy and Planning 26 (1): 57-65.
Gifts and Borrowings

By ‘‘gifts’’ I have in mind those
occasions—by no means universal—
when quantitative scholars take
positions on the nature and priority
of different sources of evidence
that permit reciprocity in
exchanges between quantitative
and qualitative scholars.

By ‘‘borrowings’’ I have in mind
ways in which design solutions and
fieldwork methods in qualitative
research may gain from adopting
aspects of the logic of quantitative
comparison studies.
Gift – modesty of claims for
experimental designs

The research design used in the Kingswood study was signally unsatisfactory
on a number of counts. Not only did ethical difficulties inherent in the design
contribute to the premature closure of the project, but even if it had been
completed as planned, the results would have been of doubtful scientific
value since they would have provided a poor basis for any generalisations
about effective treatment methods. In addition the research disrupted many
aspects of the administration and life in the school and because it took such a
long time (and would have taken even longer if it had gone according to
plan), it is doubtful that decisions about treatment policy could have been
suspended until its completion. (Clarke & Cornish, 1972, p. 19)

C.f Cronbach et al on the notion of evaluation as a summative single standalone study in which ‘the program is to “play statue” while the evaluator’s
slow film records its picture’ (1980, p.56).
modesty of claims for experimental
designs - 2

Bill Reid – on what we would call RCTs: ‘It is like trying to decide which horse
won a race viewed at a bad angle from the grandstand during a cloudburst’
(Reid, 1988).

The question, ‘Does it work?’, is a sceptical question and ‘functions as an
exclusionary gatekeeper’ (Bogdan & Taylor, 1994).

a ‘program that appears superior to a rival program in isolation may be
inferior when each program is embedded in the regular sequence of school
experience’ (Cronbach et al, 1980).
Lee Cronbach

He complains of evaluation research being geared to justification rather than
discovery. ‘Investigators...should sniff around the phenomenon and probe
unsystematically for a long while before they mount a wrap-up study intended
to “establish” what they have perceived” (1986: 102).

He also casts doubt on the value of the notion of replication. ‘A program
evaluation is so dependent on its context that replication of it is only a figure
of speech’ (1980: 222).
Gift - restraint on the nature of
scientific knowledge

Reid again, saying to me and Nick Gould that we ‘might consider some
reference to the long-held notion that science is, indeed, as Dewey once put
it, “common sense, writ large”’

…and acknowledging that theoretical assumptions are implicit in scientific
practice when saying ‘it may make sense to construe scientific practice as a
“perspective” on intervention’, and his willingness ‘to accredit client ideas
about measurement, data collection and the like that might not fit
conventional research notions’.

Kirk and Reid went as far as to say that this understanding of a scientific
practice perspective ‘could be used with advocacy research, even though the
practitioner researcher might need to forgo his or her “neutrality” ’ (Kirk &
Reid, 2002: 89).
two kinds of concession
1.
acknowledgement to advocacy
research
2.
conceding that conventional
scientific logic may not be the only
player in the game.

Byrne expresses it thus ‘measurement is a process of
interpretation, no less than the
processes of interpretation which
underpin qualitative research
practice’ (Byrne, 2011: 32).

‘In my opinion, social science is
cumulative, not in possessing evermore-refined answers about fixed
questions, but in possessing an everricher repertoire of questions’
(Cronbach, 1986, p.91).
Borrowings

Ian Sinclair

qualitative methods are in many
ways ‘more adapted to the
complexity of the practitioner’s
world than the blockbuster RCT’.

quantitative evaluation can assess
causality ‘as it actually plays out in
a particular setting’ (Miles &
Huberman, 1994)

qualitative proxy for control within
a natural setting.

E.g. the simulated client – Wasoff
and Dobash. Those who evaluate
the process of professional practice
come face to face with the
invisibility of practice. How may
we learn the ways in which
lawyers, teachers, general medical
practitioners, or social workers
practice? How would different
professionals deal with the same
case?
Patton

the creation of qualitative matrices for exploring linkages between process
and outcome: Suppose we have been evaluating a juvenile justice program
that places delinquent youth in foster homes…. A regularly recurring process
theme concerns the importance of “letting kids learn to make their own
decisions”. A regularly recurring outcome theme involves “keeping the kids
straight” … By crossing the program process (“kids making their own
decisions”) with the program outcome (“keeping the kids straight”), we
create a data analysis question: What actual decisions do juveniles make that
are supposed to lead to reduced recidivism? We then carefully review our
field notes and interview quotations looking for data that help us understand
how people in the program have answered this question based on their actual
behaviors and practices. By describing what decisions juveniles actually make
in the program, the decision makers to whom our findings are reported can
make their own judgements about the strength or weakness of the linkage…’
(Patton, 2002)
Borrowings 2 - structured methods
within ethnography

Systematic Self Observation - Rodriguez and Ryave, (2002)

training informants ‘to observe and record a selected feature of their
everyday experience’, so that participants ‘go about their lives while alertly
observing’ the matter of interest (p.2). The focus is on understanding the
ordinary, in particular the covert, the elusive and the personal. In an effort to
overcome the ‘numbness to the details of everyday life’ (p.4) respondents are
asked to observe ‘a single, focused phenomenon that is natural to the
culture, is readily noticeable, is intermittent...is bounded...and is of short
duration’ (p.5) and also to focus on the subjective.
Doing systematic self observation

In observing they are instructed in no way to act differently than usual, to
never produce instances nor to judge the propriety of the action – ‘do not
judge it, do not slow down, do not speed up, do not change it, do not
question it – just observe it’
End note

Nothing said in this presentation supports naïve consensual synthesizing of
quantitative and qualitative methods, sometimes apparent in the current fad
for mixed methods. But it may enrich both qualitative and quantitative
inquiry.
References

Byrne, D (2011). Applying Social Science. Bristol: The Policy Press.

Clarke, R & Cornish, D (1972). The Controlled Trial in Institutional Research.
London: HMSO.

Cronbach, L. (1986). Social inquiry by and for Earthlings. In D W Fiske & R A
Shweder, Metatheory in Social Science. Chicago, University of Chicago Press.

Cronbach, L., Ambron, S., Dornbusch, S., Hess, R., Hornik, R., Phillips, D.,
Walker, D., & Weiner, S. (1980) Toward Reform of Program Evaluation. San
Francisco: Jossey-Bass.

Kirk, S. A. & Reid, W. J. (2002). Social Science Work: A Critical Appraisal. New
York:
References - continued

Patton, M. Q. (2002). Qualitative Research and Evaluation Methods. Thousand
Oaks: Sage Publications.

Reid, W. J. (1988). Service Effectiveness and the Social Agency. In R. Patti, J.
Poertner & C. Rapp (eds) Managing for Effectiveness in Social Welfare
Organizations. New York: Haworth.

Reid, W. J. (2002). ‘In the Land of Paradigms, Method Rules.’ In Qualitative
Social Work 1, 291–95.

Reid, W. J. & Zettergren, Pam (1999). A Perspective on Empirical Practice. In
I. Shaw & J. Lishman (eds). Evaluation and Social Work Practice,.Pp. 41–62.
London: Sage.

Rodriguez, N & Ryave, A (2002). Systematic Self Observation. Thousand Oaks:
Sage Publications.