Theresa Hammond
12th Annual Coaching & Mentoring Conference
Oxford Brookes University
January 14th 2016
• Context – Pastoral Ministry
• My personal experience
• Beneficial environment of coaching & mentoring:
 Missing
 Patchy
 Design
•
•
was questionable
lacking & haphazard
delivery mostly via current hierarchal structures
• Increasingly, Coaching & Mentoring are offered
• However, it has been noted that it is the exception and
these relationships are often neglected
• No standardized practice
• Making monitoring and evaluation effectiveness difficult
Literature Review
• Confirmed the need and call for Coaching & Mentoring in the
pastoral ministry for:
 Training
 Support
 Ongoing
•
Development
However, of the handful of empirical studies of Coaching &
Mentoring in the pastoral ministry, none focused specifically
on the design of the Coaching & Mentoring
Aim of Research
• To identify what different coaching and mentoring designs are
currently in use in the pastoral ministry
• To evaluate the efficacy of the different coaching and
mentoring designs
Evaluation of Coaching & Mentoring
• Notoriously difficult
• Not-for-profit settings – difficult to establish desired outcomes
• Minister’s own sense of efficacy arising out of his/her
learning, growth and development
• Long term nature of growth and development – difficult &
less tangible
What and How to Assess
• Evaluation can be approached in two ways (Ely, et al., 2010):
Outcomes
 Process

– summative evaluation
– formative evaluation
• Context & aim of research call for a process or formative
evaluation
• To do so is to ask:
Does the coaching/mentoring environment provide the
conditions that either promote or hinder growth and
development & therefore his/her sense of efficacy & resilience?
What and How to Assess
• We might be able to define the conditions necessary to facilitate
growth & development i.e. Rogerian conditions, etc
• However, it is not what is believed to be delivered, but what is
subjectively experienced and can therefore be accessed
• This necessarily calls for a subjective assessment
Research Methodology
Due to the requirement of subjective data
• Qualitative research methods were considered – however,
although this would have produced rich data in terms of the
experiences of individual participants it could not necessarily
link this data with coaching and mentoring designs
Instead
• Q Methodology was chosen – a combined research method
coined a qualiquantological approach (Stenner & Stainton,
2004)
Q Methodology
. . a science of subjectivity, introduced in 1935 by psychologist/physicist
William Stephenson
. . it offers a ‘a qualitative analysis through a quantification of patterned
subjectives’ (Shemmings, 2006)
At the heart of a Q investigation
A person is presented with a set of statements (Q-Set) about a specific
topic and asked to rank-order them, an operation referred to as Qsorting (Brown, 1993)
These statements are subjective and are ranked from the participant’s
point of view onto a grid of ‘prearranged frequency distribution’
Sorting Grid used for this study
Most Disagree
-4
-3
-2
-1
0
+1
+2
Most Agree
+3
+4
The forced ranking of the statements result in Q-Sorts:
configurations of items (statements), as statements relative to
one another
These are:
whole configurations, patterns of items, which reveal
viewpoints as a whole, and can be subjected to factor analysis
In factor analysis the Q-sorts are correlated with each other to
identify the most common subjective viewpoints
. . . those which are highly correlated may be considered to have a
family resemblance (Brown, 1993), and are referred to as Factors
The aim of this study being to identify these Factors (patterns of
subjective experiences and viewpoints) and how these might
correlate with coaching & mentoring designs
Broader Q Methodological study involves
The following steps:
1. Definition of the Concourse
2. Development of the Q-Set
3. Selection of the Participant Set
4. Q-sorting
5. Analysis and interpretation
(Van Exel & de Graaf, 2005)
Definition of Concourse
First step is defining the concourse
The flow of communicability surrounding any topic is referred to as a
concourse
The concourse is usually drawn from a wide range of sources, ‘the
ordinary conversation, commentary, and discourse of everyday life’
(Brown, 1993, p.94)
Interviews, casual conversations, newspapers, magazines and
documents are all possible sources for a concourse
Statements can also be drawn from academic and theoretical literature
Definition of Concourse
In this study, due to time constraints, collecting subjective statements
was unrealistic
Instead, a concourse was created using theoretical underpinnings
Guided by:
 conditions & environment that promote or hinder growth and
development which could mediate the minister’s efficacy and
resilience
 the outcomes that would indicate the minister’s efficacy and
resilience
Definition of Concourse
To evaluate the subjective experience of coaching & mentoring
environment statements were drawn from:
Professional competencies as outlined by ICF (ICF, 2015)
 Roger’s conditions for growth (Rogers, 1967)
 Heron’s interventions (Heron, 2001)
 Heron’s degenerative interventions (Heron, 2001)
 Kram’s mentoring functions (Kram, 1985b)
 Eby et al’s Meta-themes of negative mentoring (Eby, et al.,
2000)

Definition of Concourse
To evaluate the impact of coaching & mentoring statements were
drawn from:
Objectives & outcomes of coaching/mentoring (Allen, et al.,
2004; Eby & Lockwood, 2005)
 Burnout ↔ Engagement Dimension (Maslach, et al., 2001)
 Areas of Worklife (Maslach, et al., 2001)
 Resilience CD-RISC Scale (Conner & Davidson, 2003)

Development of Q-Set from Concourse
Final concourse resulted in 61 statements:
 36 statements for the experience of coaching/mentoring
 25 statements for the impact of coaching/mentoring
Recommendation – to generate a large concourse, which can be
refined and reduced to produce a manageable Q-Set
The Q-Set is the set final set of statements presented to participants
that they rank order onto the sorting grid
Two characteristics make a good Q-Set:
 Broadly representative – of all possible subject positions
 Balanced – not biased
Development of Q-Set from Concourse
Paring down of Concourse is done by piloting the statements in a QSort procedure
Piloting highlighted the pertinent issues concerning the concourse
statements and aids with reducing and refining a Q-Set:
 Statements with double or multiple propositions
 Statements in essence repeating others
 Statements with negative proposition
 Procedure too long and cumbersome
The final Q-Set consisted of 25 statements covering both the
‘experience and impact’ of coaching/mentoring, which were to be
sorted onto the grid shown earlier
Selection of Participant Set
Selected on basis of typical case sampling (Bryman, 2012)
Research question demanded the sample was restricted to those in
pastoral ministry or training for pastoral ministry
Three contexts:
 Elim Pentecostal Church
 London School of Theology
 Personal contacts
Q-Sorting Procedure
To be able to reach these geographically dispersed participants I created
an online procedure
Consisted of three elements of data collection:
 Demographic information of recipient & coach/mentor
 Design and delivery of coaching/mentoring
• Matching
• Context
• Configuration
• Who commissioned C/M
• Training or orientation
• Length of alliance
• Clarity of purpose, location, frequency, etc
• Ending
 Q-Sorting procedure
Analysis
45 respondents producing 45 Q-Sorts
Q-Sorts subjected to factor analysis to identify the most common
subjective viewpoints
 Done by entering Q-Sorts into a computer programme –
PQMethod 2.35 (Schmolck, 2014)
 A centroid analysis is performed to extract factors
 Further probing done by rotating the data to further sharpen
and focus each factor
In final analysis: three-factor solution seemed a best fit
Two of these factors were bi-polar – bi-polar opposite experience
Factors identified
• Factor 1+
Skilled, Challenging & Transformative
• Factor 1-
Unskilled & Manipulative
• Factor 2
Directive, Heavy-handed & Educational
• Factor 3+
Supportive & Restorative
• Factor 3-
Unsupportive, Manipulative yet Educational
Interpretation
 For the majority (75%) coaching & mentoring was beneficial and
satisfactory (Factors 1+ and 3+)
 For some (8%) coaching & mentoring delivered some benefits, but
was not necessarily satisfactory (Factors 2 and 3-)
 For one participant (2%) coaching & mentoring was neither beneficial
nor satisfactory (Factor 1-)
In Q post-sorting interviews can be used to add depth & richness to the
Factors
Unrealistic in this research – most not face-to-face – even though
comments were invited
Instead attending to the demographic and design data identified the
pertinent issues relating Factors and design
Relevant themes to Factors & Design
 Continuum of Learning: instrumental ↔ transformative
 Continuum of Ownership of Learning: Organisational ↔ Individual
Ownership
 Support for stress amelioration & resilience building
 Skill of Coach/Mentor: expertise as minister ↔ skill as coach/mentor
 Context & Purpose
 Unsatisfactory coaching/mentoring & toxicity
Design Implications
 Clarity on context & purpose
 Nuanced approach to matching – taking into account:



Context and purpose
Continuum of learning
Continuum of ownership of learning
 Nuanced approach to delivery of coaching/mentoring:
 Formal C/M - yet imitating informal C/M to foster self-directedness
 Informal C/M - yet aided & facilitated by organisation to maximise
effectiveness and protect from toxicity
 Training – themes and issues identified in this study could be used to
foster effective coaching & mentoring programmes and culture
Q Methodology as Evaluation Tool
• Research demonstrates Q Methodology as a useful evaluation tool
• Although significant preparation is required to design the Concourse
and the Q-Set, I found the Q-Sort elegant and easy to administer
• Approximately 10 minutes to perform the Q-Sort
• Procedure elicits rich &informative qualitative data in minutes
• Development of computer software means calculations possible
Implications for Practice
The Q-Set developed for this research could be used to evaluate
coaching/mentoring interventions
Other Q-Sets could be designed for different purposes and foci,
for example, designed to evaluate experiences of coaches/mentors
A singular Q-Sort could be undertaken and evaluated on its own
merit
A coach/mentor could use a Q-Sort procedure as an evaluation
tool for their own practice
Limitations
• The findings of Q Methodology are not considered to be
generalizable (but are considered stable)
• Due to time constraints the concourse was produced theoretically
rather than by collecting subjective statements which may have
influenced the factors identified
• Some participants found the forced ranking difficult and this may
have put off those who had conflicted experiences
• This study only considered the perspective of the recipients of
Coaching & Mentoring and did not include the perspectives of
Coaches and/or Mentors or the sponsoring Organisations
Further Research
Longitudinal study could be undertaken to appreciate how these
coaching/mentoring environments might correlate with objective
outcomes
Useful to create a concourse sourcing subjective statements in the
more traditional manner of interviews, conversations, etc – might
identify other factors
Different Q-Sets could be generated to evaluate different aspects i.e.
coach/mentor behaviours, or from coach/mentor perspective