Improvement or accountability, what is the true purpose of
performance management within the public sector?
Examining the effectiveness of performance management within
South Wales Fire & Rescue Service.
Stephen Alan Rossiter
MSc Management
2009
Improvement or accountability, what is the true purpose of
performance management within the public sector?
Examining the effectiveness of performance management within
South Wales Fire & Rescue Service.
Stephen Alan Rossiter
01056085
September 30th 2009
This dissertation is submitted in part fulfilment of the
Master of Science (MSc) Degree in Management.
I declare that this dissertation is the result of my own
independent investigation and that all sources are duly referenced.
Signed:____________________________________
University of Glamorgan
1
Abstract
This study examines the effectiveness of the performance management system currently
in use within South Wales Fire and Rescue Service, whilst challenging the assertion that
targets do lead to improved performance from the customer’s point of view. This paper
recognises the importance of performance management in the delivery of an
organisations strategic intent at the same time questioning its true worth under the
current target regime.
The approach taken is that of a multi-method case study, utilising quantitative
questionnaires on which to base, deeper qualitative enquiry in the form of semistructured interviews, thus developing a theory in the process. The research is limited to
a single public sector organisation, the focus of the research being restricted to a section
of business planning managers within the host organisation. The purpose being to
critically evaluate the important role performance management has within the wider
aims of an organisation. Much of the research is based on the results of a
comprehensive literature review, relating to public sector performance management.
The findings of this research take a similar view to the academic argument, suggesting
that performance management in the public sector tends to be more to do with control
and accountability than with learning and improving performance. Furthermore there is
clear evidence to suggest that centrally driven targets can have unintended
consequences on actual performance in the form of gaming and dysfunctional
behaviour. Moreover this study concludes that a bureaucratic, top-down approach
actually impedes rather than improves performance, suggesting a more holistic
approach to management should be considered.
It is hoped that this paper not only benefits the development of the author, but
additionally the host organisation, by highlighting how the positive use of information,
in the form of measurement can better inform managers in their decision making, whilst
concentrating on providing a sustainable quality service in place of arbitrary targets, that
tend to remove critical recourse away from frontline services.
2
Acknowledgements
The past three years studying on a part-time basis have proven difficult. Balancing a
busy work schedule with the high level of self-discipline, commitment and
determination have required a personal sacrifice to my family and social life. Although
difficult at times, this study has been very enjoyable and has taken me on a personal
developmental roller coaster, being exposed to and having a greater understanding of
the strategic management issues facing organisations. I have learned so much over this
period and I am indebted to all who have assisted, motivated and supported me. I wish
to acknowledge this continued support, without which none of this would have been
possible.
I begin with a number of key academics in the field who provided me with many
articles, published work and direction to study material, which made the task that little
easier. I would therefore like to thank: Mike Kennerley, Bernard Marr, Zoe Radnor and
John Seddon for their assistance, contribution and inspiration. I would like to take this
opportunity to thank everyone at the University of Glamorgan who has helped me on
my journey, none more so than John Batten, who has for the past twelve months held
my hand, refocused my direction, maintained my enthusiasm, provided me with support
and encouragement and believed in me and my ability.
I owe so much to my wife and best friend Jackie and my two daughters Terri and
Sophie, who have for the past three years, had to endure many mood swings, increased
stress levels and late nights at my computer, whilst always being there to offer support
and at times, much needed reality checks. Always encouraging and reassuring me
through the difficulties. I would like to offer them a very special thank you.
Finally a close friend and work colleague Laurence Edwards, who have been by my side
throughout the journey, and who has spent many hours painstakingly trying to make
sense of all my work. His advice, guidance, constructive feedback, and honesty have
motivated and steered me throughout the year and I would like to personally thank him.
A very personal thank you, to you all.
3
Table of Contents
1
Introduction
1
2
Literature review
7
2.1
Targeting improvement or commanding control
7
2.2
Managing performance - the true purpose
12
2.3
Delivering the strategy - how do we know?
18
2.4
Is there a better system?
23
3.
Adopting the right approach
29
3.1
Belief’s and values
30
3.2
Strategic intentions
32
3.3
Ethical and credible research
36
4.
Organising and analysing the data
40
4.1
Making sense of the numbers
43
4.2
Analysing the rich side
44
5.
Presenting the evidence
46
5.1
Delivering the vision
46
5.2
Managing performance
47
5.3
Getting to where we want to be
49
5.4
Box ticking or quality service
51
5.5
A different way of thinking
54
6.
Reflection
57
7.
References
63
8.
Bibliography
70
9.
List of Appendices
82
4
1.
Introduction
“My aim is to put down on paper what I see and what I feel in the
best and simplest way”. Hemingway, (1899 – 1961).
Improving public services has been high on New Labour’s modernisation agenda since
they came into power in 1997. One of the first objectives of the new Government was to
launch a Comprehensive Spending Review (CSR), from which to prioritise public
sector spending plans. This led to Government departments being tasked with setting
their own targets to form part of the then, new Public Service Agreements. This was the
Government’s approach to ensuring public sector organisations were clearly held to
account to the tax payer for the services their deliver and to bring some transparency to
the system. The Audit Commission (1999:3) articulated that:
“there are at least two key reasons why Government would want to
set performance measures. First, to improve public services; and
second to reinforce accountability”.
Moreover to clarify the purpose behind this performance measure culture, the Audit
Commission (1999:6) stated that:
“performance measurement is not an end in itself. Rather, it is a
means to generate valuable management information, partly for ongoing monitoring purposes but crucially to focus management
attention on areas where change is sought”.
The Local Government Act 1999 placed a requirement on local authorities to secure
improvement in the way they exercise their functions, having regard to a combination of
economy, efficiency and effectiveness. In 2002 the Welsh Assembly Government
(WAG) established the Wales Programme for Improvement (WPI), which is the current
performance measurement framework for public sector organisations in Wales (Welsh
Assembly Government 2005). In 2005 the responsibility for the Fire and Rescue Service
(FRS) in Wales was devolved from the Office of the Deputy Prime Minister (ODPM) to
5
the WAG. Since then the FRSs within Wales fall under the scrutiny of the Wales Audit
Office (WAO), in relation to performance of services, “ensuring Welsh public bodies
provide good value for money in the delivery of their services to the public” (Welsh
Audit Office 2005:3). With this in mind, the following research dissertation will focus
on improving performance within a public sector organisation, concentrating on South
Wales Fire and Rescue Service (SWFRS). This paper will examine the affect that a
bureaucratic (command and control) approach can have on improving performance
within a public sector organisation, and whether such performance management systems
can assist or impede improvement. The main aim of this research is to critically
examine and measure the effectiveness of the current performance management system
within SWFRS, whilst answering the key question “can, a change in management
thinking, lead to the development of a more effective continual improvement culture
with SWFRS?” This paper also seeks to explore the current Government driven target
culture as suggested by Hood (2007:97) as “public management by numbers” and their
intended and unintended effects on delivering a better service, thus challenging the
assertion that ‘targets do lead to improved service delivery from the customer’s point of
view’.
Through his current academic studies the author finds himself questioning and
challenging management norms within his organisation, none more so than the real
purpose and worth of performance management. The rationale in choosing this research
topic is the author’s fascination and desire to study the relationships between strategic
direction, business planning, performance management and learning and development,
and their reliance on each other to deliver organisational success, Marr (2009:14)
explains:
“Too many performance management approaches assume that the
strategy and business models are well understood by everyone in the
organisation. From my experience, this is not the case and this is
often a key contributing factor to the failing of performance
management initiatives”.
There is a further motive for selecting this area of management to study, currently
responsible for performance management and the business planning process within the
6
fire safety department the author is keen to develop a meaningful performance
management system. A system that is driven by the strategic direction of the
organisation, forming part of the business planning process and concentrating on what
in essence is arguably the key intention of performance management – improvement
through learning, Marr (2009:8) suggests that to create a performance driven culture
“learning and performance takes centre stage”. Hume and Wright (2006:191) concur
stating: “a performance framework is underpinned by a learning and development
strategy”, which should be based on evidence of effective practice. This is echoed by
Senge (1990:4) who states:
“the organisations that will truly excel in the future will be the
organisations that discover how to tap people’s commitment and
capacity to learn at all levels in an organisation”.
As for all research studies, the more one delves into the literature, the deeper and richer
the knowledge becomes and the drive to learn becomes some what of a passion, this
dissertation is no exception. There is an abundance of literature available on the subject
of public sector performance, targets and continual improvement; according to
Wisniewski (2008), as cited by Adcroft and Techman, (2008:2) “there has been on
average a new paper on performance management written every five hours of every
working day since 1994”. This research however intends to evaluate a different
approach to performance management that of a systems thinking approach, where
organisations function as a ‘whole’ holistic system rather than individual silos. Seddon
(2008) argues that systems thinking, unlike a top down functional hierarchy approach
operates from an outside-in perspective, where the design and management of work is
based around what the customer wants, which is primarily what public services are
intended to deliver. Systems thinking according to Chapman (2002:26) “provide a
holistic approach to understanding and managing complexity”. In the publication ‘A
Systematic Approach to Service Improvement’ – ‘Evaluating Systems Thinking in
Housing’, the ODPM (2005:4) suggest that systems thinking:
“allows the organisations to look at itself as a whole. This creates
organisational development as sections discover that their role is
part of the delivery of the overall service and not an end in itself”.
7
This research intends to examine present management thinking in relation to the
statement above, by questioning managements understanding of how their individual
actions (decisions, departmental business plans and departmental performance indictors)
can affect the wider (whole) organisation in their drive for continual improvement.
Lebcir (2006:6) argues that:
“it is critically important that the decision-makers understand and
appreciate that they are working within systems that include many
interconnected and interdependent elements”
The research will unfold taking the reader on a journey, setting the scene with a
background history of the current Government “something-for-something” target
culture as maintained by Broadbent (2007:194) and the impact it is having on public
sector organisations and their drive for continual improvement. The following sections
will unpick the performance management systems presently being practised in SWFRS
including a critical evaluation of the relationship between the strategic direction of
SWFRS and that of the externally driven targets, while examining the present systems
used to measure, manage and report on performance. Furthermore this paper will
determine whether the true purpose of performance management is, the delivery of a
value for money service to the communities of South Wales, “with an emphasis on
increased efficiency and effectiveness of service delivery”, as put forward by Marr,
(2009:2), or is the focus on meeting the Key Performance Indicators (KPI) set centrally
by Government departments, thus suggesting what Hauser and Katz (1998) argues as an
organisation becoming what it measures. In a global survey on strategic performance
management in government and public sector organisations, Marr (2008a:2) found that:
“there were too many performance indictors and a lack of clear
strategic direction. Performance management should provide the
strategic direction for everyone in the organisation, and the
performance measures should be used to check whether the
strategic journey is on track or not”.
The journey will continue by drawing on the rich literature available from both
government bodies and academics alike, identifying the real impact that performance
8
targets are having on actual service delivery. A comprehensive examination of how
public sector organisations are measuring, managing and reporting on performance, will
be undertaken, whilst considering the impact of how targets can lead to organisational
gaming. To conclude the literature review, an alternative approach to the traditional top
down, command and control management of performance will be considered. Offering a
systems thinking methodology, this focuses on a holistic view of an organisation and
one which manages performance from an outside-in perspective.
The research development began with the formulation and clarification of the research
topic, as proposed by Saunders, Lewis and Thornhill, (2003). This element was clear
and straightforward as the authors motives above suggest; furthermore SWFRS being
the main sponsor of this academic research had a major influence on the research topic.
The research structure commenced by identifying core themes associated with
performance management, searching for key academics in the field, in order to carryout
a critical analysis of what other authors have written, as offered by Jankowicz, (2005).
A ‘relevance tree’ was development in order to determine what subjects were relevant
to the research topic (see appendix.1 on page 83 above). The journey then unravels the
literature, whilst evaluating it against current management thinking within SWFRS in
relation to performance management from the view of the practitioners. By gathering
valid and reliable data that are relevant to the research question and objectives as
recommended by Saunders et al (2003) a selection of personnel at various levels within
SWFRS will be interviewed. This will be achieved through a systematic investigation;
the methods used will be covered in more detail in ‘adopting the right approach’ section
below. The final leg of the journey will discuss the detail gained from literature review
combined with the findings from the investigation, which will hopefully provide
SWFRS with current, up to date information on, not only how performance is presently
being managed within the host organisation, but also how the current system is viewed
by those involved in it and by it. The findings will hopefully confirm whether the
current management thinking actually assists or impedes the process.
There are limitations to this study however, firstly being based on a single organisation
may suggest that wider applicability of the specific findings from this study should be
questioned. Secondly the target audience for the research was confined to a small group
of managers with responsibilities for business planning, as the researcher intends to
9
focus on performance management in the context of an organisations strategic intention.
There are various levels of performance management within the host organisation and
although this study makes certain reference to individual performance, this study
concentrates on the strategic element, again suggesting that further study may be
required if adopting the findings from this research into individual performance issues.
The author does however propose, that many of the underlying principles can indeed be
utilised within a performance management framework.
Therefore the following chapters intend to critically examine and measure the
effectiveness of the current performance management system within SWFRS. Through
a comprehensive literature review and a detailed investigation with practicing managers
within the host organisation, this research will compare two differing approaches to
performance management, a bureaucratic, “command and control” approach and a
systems thinking approach, whilst answering the key research question “can, a change
in management thinking, lead to the development of a more effective continual
improvement culture with SWFRS?” Finally it is intended to review the current
Government target culture and its association with performance management,
challenging the assertion that “targets do lead to improved service delivery from the
customer’s point of view “.
10
2.
Literature Review
“When you take stuff from one writer its plagiarism; but when you
take it from many, its research”. Mizner, (1876 – 1933)
The availability of literature on the subject of performance management is vast and
wide ranging; therefore this review will concentrate on performance management within
the public sector with particular focus on the FRS. The subsequent sections will
consider the implications that the current Government performance management and
reporting culture is having on public sector organisations, the researcher will seek to
question the assertion that targets do lead to improved performance, whilst examining
the unintended consequences of such targets as forwarded by Pidd, (2005) and Hood,
(2004). The review will commence with an introduction to the background behind the
inception of this regime, also know as New Public Management (NPM), as presented by
Verbeeten (2008) and Adcroft and Techman (2008). Taylor, (2008:1) suggests that
organisations implementing such a culture may in fact be ‘hitting the target but missing
the point’ in terms of actual service delivery to the customer. The debate then focuses
on what Marr, (2009): Radnor, (2008) and Collier (2006) suggest is the true purpose of
performance management, improvement and learning. Finally the review will consider
an alternative approach, a systems thinking approach, which focuses on the customer as
the driver for improvement. Although systems thinking can be dated back to the 1920’s,
there has for many years been an increase in interest of this approach to the design and
management of work. From the Toyota Production System (TPS) to more recent
publications such as Senge’s (1990) The Fifth Discipline, there appears to be credibility
to this approach, which is slowly influencing politicians to sit up and take note. We
begin however at the birth of the current target culture.
2.1
Targeting improvement or commanding control
“The only thing that saves us from the bureaucracy is its
inefficiency”. McCarthy, (1916 – 2005).
Even before Margaret Thatcher’s arrival at No.10 Downing Street in 1979, Adcroft and
Techman (2008:2) concur that “attempts to boost the effectiveness of public services”
was well underway and saw a performance measurement revolution. A more managerial
11
approach intending to increase efficiency and focusing on results was the order of the
day according to Flynn, (1995). The conservative Government’s image of public
services back then was one that mirrored the private sector approach to managing its
resources, focusing on the 3E’s, effectiveness, efficiency and economy, exposing
authorities to tighter scrutiny through the National Audit Office. In 1997 the 'New
Labour' Blair regime came in on a tidal wave of public sector expectation of better times
and a more humane and equitable approach to resource allocation. However there was
no major shift in emphasis and the Labour Government continued with what Radnor
and McGuire, (2004:1) contend as a “drive to improve the effectiveness of public
services through the use of private sector principles”. From Thatcher’s 3E’s came
Blair’s’ 4C’s of Best Value (BV) challenging, comparing, competition and consultation
(with/against similar authorities). BV became the answer, this approach required whole
authorities to produce a vast range of documentation as evidence it was delivering
services to set standards. The implications of BV on the capacity of authorities were
considerable and included the emergence of new departments whose sole task was to
prepare for the inspections. Trickery (2003:2) in his metaphorical labelling of public
sector workers under the Conservative Government as “galley slaves” who, looking for
calmer waters under the New Labour regime, only to find Blaire’s response as:
'We think you have been doing a really excellent job, we really
value your contribution, sadly we cannot increase your rations and
could you please sing in harmony as you row?”
The author feels it appropriate to capture the political development of performance
management over time in order to lay down the foundation to allow for a sequential
evolution within the literature review. The target culture from the past decade has
evolved from Best Value, Public Service Agreements, Comprehensive Performance
Assessments, and from April 1st this year (2009) the new Comprehensive Area
Assessment. This suggests that improving public services through such a culture is here
to stay. But does it work, does it do what is says on the tin, are services improving as a
consequence of such tight scrutiny, or are unintended consequences emerging from the
rubble of what Bevan and Hood (2006:517) imply is “the governance of public services
that combine targets with an element of terror”. There is an abundance of academic
interest in this subject area and supportive evidence to suggest that the target culture
12
does not provide the improvements intended, in fact many proponents such as Edwards
(2007); Pidd (2005) and Hood (2004) suggest that it does quite the opposite and
encourages dysfunctional behaviour, cheating and gaming. In the global study of
strategic performance management in government and public sector performance
organisations, Marr (2008a:22) concludes that government and public sector, “with all
of the targets under which it must operate, is becoming lost in a sea of measurement
data it is generating”. Marr (2008a:1) goes on to claim that there is also evidence to
show that public sector performance management initiatives are “so mechanistic and
number-focused” that it prevents organisations from achieving their desired aim.
McKevitt and Lawton (1996) suggest that performance measurement has tended to be
developed to provide legitimacy within the ‘institutional environment’ rather than to
inform organisational change and service improvement. Senge in his forward in Johnson
and Broms (2000:xii) concur, that:
“it has become almost an unquestioned tenet of contemporary
management that bosses set quantitative targets and then create
control systems to assure that management goals are met”.
Further evidence that targets can lead to dysfunctional behaviour such as target fixation
and cheating was published in a book entitled “Stitched Up - The Highs and Lows of
Life as an A&E Doctor” where Edwards, (2007) suggests that within the NHS this
behaviour was endemic. The author will strive to determine whether this is the case
within the host organisation.
The debate is not one sided and there appears to be support for targets, in his Times
column Parris (2008:2) argues that “when public money is spent it is governments duty
to get value for it” suggesting that if we cannot measure something, then we should not
ask the tax payer to fund it. This is echoed by the Commons Pubic Administration
Committee who highlights Kenerley’s (2004:1) argument that “if used appropriately
measurement can play a vital part in improvement efforts” The HM Government
(2007:7) further supports this in their publication ‘Cutting bureaucracy for our public
service’ by stating that:
13
“it is important that Government collects data from the front-line to
provide accountability to the public for money spent and the quality
of service provided”.
Gershon (2004:3) in his forward to the Independent Review of Public Sector Efficiency
– ‘Releasing resources to the front line’ agrees, stating he was mindful that his
recommendations needed to “ensure the savings are not delivered at the expense of
impacting on service delivery”. Radnor and McGuire (2004:2) argue:
“that it is importance to remember that the performance
management function within an organisation is an overhead that
drives resources away from the production of front line services”.
Pidd (2005:483) concurs that “those planning a performance measurement system
ought to ensure that its benefits outweigh the cost”, likewise the author has witnessed
an expanding performance management department within the host organisation, known
as the Performance Management Unit, the department collects and analysis data, and
manages all the business plans within the organisation. The most current literature and
media coverage for what Hood (2007:97) describes “as a disease that other countries
should strenuously try to avoid” suggests there is a paradigm shift from leaders within
certain public sector organisations to move away from the target culture, where Hume
and Wright, (2006: Title) argue that “you don’t make a pig fatter by weighing it” The
debate focuses on actual service delivery to the customer rather than hitting centrally
determined targets. Hume and Wright (2006:190) articulate “the continuous
improvement approach is longer term and is unlikely to create headlines rather, seeking
to achieve sustainable improvement.” This again suggests that quick fixes can take
precedence over longer term benefits. Further target perversity, is that of the ambulance
service’s target for response times; being measured for the time they take to respond to
certain life threatening incidents. This particular indicator requires an ambulance to
respond to an emergency incident within 8 minutes in 75% of cases (DoH, 2005).
Whilst time can be critical to any life threatening incident, once in attendance at the
incident the clock stops and the paramedics are no longer measured. If the paramedics
are to be measured at all, surely it should be on what many perceived as their most vital
role, “emergency life support and stabilizing the patients condition, sufficiently for
14
rapid transportation to hospital” as presented by DoH, (2005:8). This mirrors the turn
out times for fire appliances, where the clock stops when the fire crews arrive at the
incident, there is no monitoring or recording of firefighting activities, such as
preventing injuries, minimising property and environmental damage, as suggested by
Elkan and Robinson (1998). The main areas for criticism of targets have tended to focus
on the National Health Service (NHS), education and policing, a further educational
example as put forward by Clancy, (2008:1) concludes that:
“an obsession with assessment and meeting goals and targets is
misconceived and damaging, and leads to students achieving more
but learning less”.
Chief Constable Thornton of the Thames Valley Police Service, in a statement to The
Telegraph (Edwards, 2008) offers:
“Since I became chief I have realised how important it is to get
police providing the service that the public wants. You do not
improve policing by setting lots of targets from the centre. You
improve policing by giving a better service to the people, to improve
confidence and satisfaction. In this day and age we cannot ignore
what the people want in policing”.
Although there was support for targets within the literature reviewed, the majority of
writers were critical of centrally driven targets. The main thrust of their argument
being that targets destroy accountability to customers. There was also a consensus that
targets damage professional morale and judgement, taking away autonomy and
decision making from the people who have the skills and experience to deliver to the
communities they serve. Finally there appeared to be agreement that a target culture
prevents organisations from learning on the basis of experience and mistakes made,
whilst increasing levels of bureaucracy as argued by Taylor (2008). The author
proposes to examine how the people of SWFRS view the current target culture and
question whether the political versus professional battle exists in the FRS. There is
strong evidence to suggest that targets lead to unintended consequences as the previous
section has touched upon, that said if targets do not assist in improving performance,
15
then what does? In developing academic argument the author of this research will
investigate performance management, measurement and reporting, whilst making a
link with the overall strategic aim of an organisation.
2.2
Managing performance - the true purpose
“It is not strange that desire should so many years outlive
performance” Shakespeare (1564-1616).
Having examined current thinking behind the Government target regime; the researcher
intends to dissect performance management, its true purpose and worth, whilst looking
at the bigger picture and why attempts are made to manage performance, what are
organisations trying to achieve, and how do they know when they have achieved it?
Radnor and Barnes (2007) argue that it is important to differentiate the terms so often
used under the all encompassing umbrella of performance management. The terms have
their own meaning and purpose and it is important to separate these before
understanding how they interact with each other. The three main terms used with the
context of this study are; performance management, performance measurement and
performance reporting. Marr (2009:1) encompasses all these terms into one definition of
performance management (see fig.1) and suggests:
“ first, you need to agree and clarify what matters in your
organisation; second, you need to collect the right management
information to understand whether you are delivering performance
in accordance with your plans; and third, you need to gain insights
from the information, which in turn helps you deliver better
performance going forward”.
Fig.1. Managing performance
Identify and
agree what
matters
Collect the right
management
information
.
Marr, 2009: Managing and Delivering Performance
16
Learn and
improve
performance
The author cites the above as a simple illustration of what Radnor, and Barnes, (2007);
Seddon, (2008) and Marr, (2009) see as the main purpose of performance management.
Firstly an organisation (not central body) must be clear as to its purpose, why does it
exist, then based on its current performance, gather management information (otherwise
know as performance measurement) to identify whether it is on track to succeed in
delivery of its purpose. Then analyse the information to gain insights from it, i.e. learn
and modify (systems, processes and people) according to the information as suggested
by Radnor and McGuire (2004) and Seddon, (2008). According to Broadbent (2007)
performance measurement has its roots in accounting, at a simple level it provides an
account of a series of events and is concerned with measuring the success of a given
outcome. Radnor and Barnes (2008:393) define performance measurement as “a
quantitative or qualitative value of the input, output, outcome or level of activity of an
event or process”. According to Marr (2009) measurement information allows us to
understand the world we operate in, to make sense of it, and to use this information to
guide our decision making and learning. Parmenter (2007:22) concurs arguing that
“performance measures are meaningless unless they are linked to the organisations
strategic objectives”.
Marr (2009:7) concurs “without the right management information we can understand
nothing and know nothing”. Wheeler (1993) agrees, stating that measures should never
be arbitrary; figures that are required for planning should always be based on previous
data, plus current action plus the possible future conditions. Very often we take for
granted some of the measurement tools we use in every day life, such as; clocks,
calendars, speedometers, sell-by dates etc. Therefore measurement is important to all
organisations to understand how it is operating, whether progress is being made and
learning from mistakes. Marr (2009:137) further argues that we measure performance
for three main reasons, controlling behaviour, external reporting and compliance and
learning and empowering, suggesting that the third reason is “the most natural way of
using performance indictors and that will lead to the biggest performance
improvements”. It could be argued that performance reporting is not restricted to
external bodies; in fact it is vital for internal information to review and amend systems
and processes. Verbeeten, (2008:428) agrees, proposing three managerial purposes for
performance management:
17
“a communications purpose, defining clear organisational aims,
thus helping employees understand their purpose; transparency and
accountability purpose, reassuring the public for what purpose their
money is being used; and the learning purpose indicating where the
organisation excels and where improvements are necessary”.
The author here however, will examine the main purpose of performance management
within SWFRS, whilst suggesting that SWFRS employ all three, the question to be
answered is “how successfully? The relationship between the above three terms is vital
for continually improvement, Radnor (2008:318) however suggests:
“the importance that too much focus on measurement and reporting
means that performance measures and indicators are not so much
used as a tool to improve the performance of an organisation but
become and ends in themselves”.
A wealth of performance management literature commentates on what Marr, (2009);
Pryor, Anderson, Toombs and Humphreys, (2007); Radnor and McGuire (2004) argues
is the critical link between organisational strategy and performance management.
Performance management is a fundamental element of any organisational strategy;
performance results should be feed back into the strategic management process to
provide valuable insights on current direction, which will add leverage to the decision
making process as suggested by Humphreys (2004). Australian Public Services
Commission (2001:2) claims that:
“an effective management framework that integrates organisational
business and individual planning with performance is what
produces an effective performance management system”.
The 5P’s model of strategy (purpose, principles, people, processes and performance) as
produced by Pryor et al (1998) provides a diagrammatic view of where performance sits
within a strategic framework (see fig.2). This model depicts the relationship between
strategy; i.e. the purpose of an organisation, and performance. Strategy drives structure
(organisational principles and processes), which drives (people’s) behaviour and
18
behaviour drives results (performance). The arrow as suggested by Pryor et al (1998)
from performance to purpose represents the continual feedback mechanism essential for
learning and improving decision making. In citing this model, the author recognises how
each component impacts on each other; however the people element in any strategy
should follow purpose as the next most important element.
Fig.2. The 5 P’s Paradigm
Purpose
Principles, Processes
People
Performance
Pryor et al 1998: A strategic systems approach to continuous improvement
Many proponents, such as Leavit, (1965); Pettigrew and Whip, (1991) and Radnor,
1999) argue that (public sector) organisations must first understand all the elements and
facets of an organisation and their relationship with each other. Building on Leavitt’s
(1965) model (see appendix.2 on page 84, above) Radnor (2004:257) argue that the
main facets requiring consideration for performances management within the public
sector are “strategy, process, people and system”, this is not too dissimilar to the 5P’s
model above. Radnor (2004:257) continues to dispute that there exists an interdependent
relationship between all the facets, which therefore impact on each other, suggesting
that “a change in any one of the facets usually (or should) result in a change in the
19
others”, further suggesting a holistic (systems thinking) approach should be adopted.
The researcher intends to expand the development of a systems thinking approach to
performance management, as he believes the host organisation would benefit from
adopting such principles to facilitate delivering a quality FRS to the communities of
South Wales.
Continuing with the strategic theme, in his Global Survey of Strategic Performance
Management in Government and Public Sector Organisations, Marr’s (2008a:8) key
findings clearly highlighted that “many public sector organisations have an obsession
with measuring performance, but fail to manage it”.
Marr (2008:2) continues
suggesting that a common problem in the public sector is:
“organisations are drowning in data but thirsty for information. By
measuring what is easy to measure, rather than what is strategically
important to the organisation, the chance for driving efficiency and
performance improvement can be dramatically reduced”.
There appears to be a link here with the previous section on Government driven targets,
Greiling (2006:449) argues that the majority of measurement captured in public sector
organisations is required for annual reports to auditors rather than for improvement,
suggesting that this type of measurement is “an accountability tool”, not a learning tool.
The literature provided many examples of how Marr (2009); Seddon (2008) and Heath
and Radcliffe (2007) argue that organisations are using measurement as a reporting tool
rather than an improvement tool, furthermore many critics such as Boland and Fowler
(2002); Holloway (1999) and Rouse (1993) highlight the fact that a new industry has
developed within the public sector, which is concerned with, collecting, reporting and
apprising organisational performance. Given the above statement it would be interesting
to delve a little deeper into Radnor and McGuire’s (2004:2) acknowledgement:
“that it is importance to remember that the performance management
function within an organisation is an overhead that drives resources
away from the production of front line services”,
20
to identify actual costs associated with such measurement systems and to establish
whether such an industry (under the current target culture) is in fact cost effective,
whilst confirming Gershon’s (2004) stance that front line services should not be affected
by such processes, would require further investigation, however this argument is outside
the scope of this research.
The literature implies that many organisations apply the old what gets measured gets
done adage, whereas many critics such as Kennerley, (2004); Heath and Radcliffe,
(2007) and Hume and Wright (2006); suggest that measurement should be more about
reflecting the vision and objectives of organisations and should move away from
compliance with external standards towards checking whether an organisation is
achieving its objectives and improving in areas of poor performance. Einstein’s (18791955) simple but famous statement typifies the current measurement issues faced by
public sector organisation; “Everything that can be counted does not necessarily count;
everything that counts cannot necessarily be counted”. Collier (2006:165) concurs that:
“the development of performance indicators has been primarily topdown with a dominant concern for enhancing control and upward
accountability, rather than promoting learning and improvement”.
This statement is again typical throughout the literature where critics, for example
Seddon, (2008); Wheeler, (1993), and Deming, (1982) are opposed to arbitrary targets
as a measurement function; they do however recognise the importance that measuring
the right things are vital for any performance management systems. Seddon (2008:97)
argues:
“there is no value in having a target, since it is an arbitrary
number, by its nature it will drive sub-optimisation (distortion) into
a system, allowing parts to ‘win’ at the expense of the whole. It is,
however, vital to know how the system actually performs – its
capability”.
Many proponents such as Marr (2009); Seddon, (2008), and Radnor, (2004) agree,
maintaining that when measures are used for learning, then people should be in control
21
of measurement and, that the data collated by them will inform their decision making
process. The author here, whilst using the host organisation as a vehicle, will develop
academic argument to investigate how targets are being used and whether the current
performance management system is effective in delivering the organisational strategy.
2.3
Delivering the strategy – how do we know?
Would you tell me, please, which way I ought to go from here? That
depends a good deal on where you want to get to. Said the Cat. I
don't much care where - Said Alice. Then it doesn't matter which
way you go, said the Cat. Carroll, (1832-1898).
The strategic direction of an organisation is its intended purpose, why it is in existence;
according to Hambrick and Fredrickson (2005:51) it is “an integrated overarching
concept of how an organisation will achieve its objectives”. Whereas performance
management is the mechanism that informs the organisation of its position in relation to
its strategic intentions. Therefore their interactions are critical. The author here
recognises the importance of this relationship and intends to analyse SWFRS’s
performance management system, to examine its association with the strategic aims and
objectives. Furthermore it is intended to evaluate the organisation’s position on target
setting, data gathering and monitoring performance, whilst also identifying the
relationship between performance management and learning and development.
Sanderson (2001:298) suggests the weakness of many performance management
systems “is a failure to promote understanding and learning”.
Strategy within the FRS is based on a systematic, formalised approach to strategy
formulation as suggested by Grant, (2003). Short, medium and long term plans are set,
based mainly on externally (WAG) driven targets, a “calendar driven ritual” as
indicated by Hamel (1996:69). Some of the main elements of the all encompassing
improvement system for Fire and Rescue Authorities (FRAs) in Wales, otherwise know
as ‘Wales Programme for Improvement’ (WPI) are business planning, performance
reporting, risk reduction planning and annual improvement plans all of which occur on
an annual cycle and aligned with budgetary and accounting processes). Not adequately
integrating performance management with mainstream budgetary and management
processes has, according to Palmer (1993) been a fundamental failure on local
22
authorities. Farnham and Horton (1993:47) suggest that the current performance
management systems, which are the birth child of what Hood (1991) contends as being
‘New Public Management’ are linked to ‘managerialism’ and are:
“founded upon economistic, rationalistic and generic assumptions
all of which increase the emphasis on measuring performance,
mainly in terms of costs, productivity and efficiency, in the context
of planning systems driven by objectives and targets”
The Local Government Act 1999 places a requirement on local authorities to:
“secure continuous improvement in the way in which they exercise
their functions, having regard to a combination of economy,
efficiency and effectiveness”.
and in doing so, each FRA must prepare a performance plan for each financial year, as
required by the WAG, (2005). The logic of this according to Pollitt (1993:44) has been
“one of cutting the size of the public sector and increasing the efficiency of what was
left”.
The key constituent elements of SWFRS’s performance management framework
include; an integrated planning framework, a performance measurement framework,
personal appraisal systems, risk management framework, monitoring framework, whilst
integrating with the requirements and principles of the WPI (see appendix.3 on page 85,
above). The planning process begins with the Risk Reduction Plan (RRP), which is the
host organisation’s key strategic plan; it sets out the “major strategic challenges facing
the organisation in delivery of its statutory duties” as set out in SWFRS (2009:4). Some
FRS’s allocate this plan with an evolving ‘shelf life’ of anything between one to five
years; SWFRS’s current RRP runs from 2010 to 2011 and allows all stakeholders the
“opportunity to review and comment on the aims and objectives of the plan” through a
consultation process as per SWFRS (2009:4). The RRP provides the strategic direction
of the organisation and is the “overarching plan that governs everything that SWFRS
does” as highlighted in SWFRS, (2007:8). As in most organisations the planning
process takes the form of a ‘top-down’ process, where the organisational plan is
23
cascaded down the various levels within the hierarchal structure. McKevitt and Lawton
(1996) suggest that such an approach tends not to be integrated into the overall strategy
and management processes and fail to engage the commitment of middle and junior
management.
Every directorate (7), department (11) and station (50) within SWFRS produce business
plans, totalling 70 plans throughout the organisation including the RRP and the annual
improvement plan. There are 48 strategic and core Performance Indicators (PI’s), which
are set centrally by the WAG, the main thrust being risk reduction, reducing the
incidence of fires, fire deaths and injuries as indicated in table.1 below:
Table.1: No of strategic and core performance indicators
Strategic Indicators
No’s Core Indicators
No’s
Risk Reduction and Community Safety 15
Risk Reduction and Community Safety 15
Effective Response
Corporate Health - Workforce
15
Corporate Health - Finance
1
2
Furthermore SWFRS has identified 17 corporate objectives based around six themes;
community protection, developing our people, effective use of resources, technology,
organisational improvement and sustainability. Not withstanding the many directorate,
department or station local PI’s, this totals 65 PI’s. Is SWFRS ‘missing the point’ as
alluded to earlier, and is this amount of planning ‘effective use of resources’ or is it
duplication of work, concurring with Radnor and MaGuire (2004) and Pidd’s (2005)
earlier argument that any performance management systems must be cost effective?
Marr (2009:189) argues that the “estimated cost and efforts required to introduce and
maintain PI’s” should be a serious consideration for management. With 70 business
plans being produced within SWFRS, the costs associated with producing these plans,
collection of data, time and effort required to analyse the data as well as to report on
each plan on a quarterly basis can, according to Austin (1996) incur a considerable
financial outlay. It could however be argued for example by Gray, (2005) that such costs
are in fact seen as investment, when the information provided by such indicators are
meaningful and relevant, which aids decision making and learning. Gray (2005) goes on
to argue that the planning process is a corporate necessity that sets the course for how
the organisation can deliver its vision. However, is it required to be so resource
24
intensive and is it the most cost effective method available, the author here intends to
examine this?
Many business (action) plan objectives have generic statements with quantitative goals,
e.g. “reduce the number of accidental fires by 5% compared to the 2008 – 2009 levels,
from 42 to 40” see Fire Station Service Plan (2009/2010: 33). Marr (2009:188) suggests
that organisations often base their targets on previous performance figures and just
suggest a target that looks “a bit better” than last year. Seddon (2008) argues that there
is no reliable method for setting targets and simply by increasing last year’s
performance by 5% may not actually improve performance, Wheeler (1993:18) concurs,
arguing that “the uncertainties of extending the data from the past into the future makes
these values unsuitable as targets”. The recommended measure of success of the above
example is the quarterly review of the indicator, suggesting that limited valuable
information is being provided, such as how do we know if our community safety work
is actually reducing fires? The author here however suggests that the host organisation is
missing the opportunity to gather such vital insights from the process, rather than
chasing the target. Furthermore he believes that the principles of performance
management are not fully understood by many managers and is keen to examine this
concept during the research.
After close examination of a selection of business plans within the host organisation, a
common theme emerges and there appears to be a need for a greater understanding of
the relationship between organisational strategy, objective setting, measurement, data
gathering, analysis, reporting and learning. A typical example is one of fire safety’s
local performance indicators for the number of fires in non domestic premises as set by
WAG, (2009), which scrutinises the success of the FRSs inspection, enforcement and
reduction policy. Improvement is based on actual figures compared to past performance
figures, Adcroft and Teckman (2008:5) refer to organisations that use such an approach
as “performers, who improve their performance by reference to their own past
achievements and internal processes”. This leads to focusing only on last years figures
as a ‘yardstick’ rather than, seeking to identify valuable insights such as actual causes of
fires, so as to learn from this critical information and review the action plan to deliver
genuine results. Adcroft and Teckman (2008:4) further concur that the public sector can
25
learn much from the private sector, offering a competitive oriented mindset as opposed
to a performance oriented mindset, suggesting that:
“Performance orientation is about the development of potential for
excellence while competitive orientation is about translating that
potential into action”.
De Bruijn (2007:141) provides an analogy of what the true purpose of measuring
performance should be about:
“performance measurement is not fitted with dials from which
performance can be readily read but rather with ‘tin openers’ that
invite further investigation and interpretation”.
De Bruijn, (2007) therefore suggesting that it is the tin openers and the further
investigation that provides the true value and knowledge that eventually leads to
improvement. This type of measurement known by Torres, Preskill, and Piontek,
(1996:2) as “informational measurement” can be used for learning and decision-making,
rather than for controlling people’s behaviour, which Ethin (2000) argues does not work
in today’s organisations. Moreover performance measurement within the public sector,
including SWFRS tends to relate to costs, volume of service, targets times and
productivity, whereas measures such as service quality, customer satisfaction and goals
of achievement are uncommon. In fact “performance is rarely evaluated from the
customers’ perspective” Kanji and Moura (2007:50) and this in essence is what
improving performance for a service organisation must surely be, or is it?
Systems thinkers, such as Seddon, (2008) contend that if service organisations want to
improve their performance, they must therefore consult with the people for whom they
provide a service; this is known as managing the organisation from an ‘outside-in’
perspective, from the customers’ point of view. To enable public sector managers to
grasp this concept, they first have to change the way they think about the organisation in
order for them to see the bigger picture, to see the organisation as a holistic whole rather
than individual silos. Chapman, (2004:13) maintains that such an approach requires
managers to have a “willingness to work jointly with those who have other
26
perspectives…., whilst being able to reflect on the outcomes of their actions on the basis
of reflection”. The next section intends to discuss systems thinking and examine
whether such an approach to performance management can improve service delivery.
2.4
Is there a better system?
“It is not the strongest of the species that survives, nor the most
intelligent, but the one most responsive to change”
Darwin (1809-1882).
The terms ‘the big picture’, ‘birds eye view’, ‘helicopter view’ and ‘blue sky thinking’
are commonly used in meeting rooms throughout many organisations. It is however,
questionable how many managers using such terminology actually understand the
concept of what is otherwise know as systems thinking? This section intends to examine
this very thought, whilst evaluating if it exists within SWFRS. Brown and Lerch (2007)
offer a simple explanation of the systems thinking approach, it is the ability to ‘see the
wood for the trees’, understanding your organisation as a whole rather than individual
parts. More importantly systems thinking require managers to see how these
interconnecting parts impact and interact with each other, particularly when you apply
certain parts of the system to a change process. Zemke (2001:42) as cited by Reed
(2006) states:
“For every problem there is a solution that is simple, neat and
wrong. This maxim has been attributed at various times to Mark
Twain, H,L. Mencken, and Peter Drucker as a wake up call to
managers who mistakenly think that making a change in just
part of a complex problem will cure the ails of an entire system.
Everyday
management
thinking
too
often
looks
for
straightforward cause and effect relationships in problem
solving that ignores the effect on, and feedback from, the entire
system”.
There are many approaches to systems thinking; however what they all have in common
according to the ODPM (2005) is examining issues from a ‘whole system’ approach,
focusing on the whole and not a collection of the separate parts. System thinkers
27
recognise the critical need to apply a holistic approach when analysing an organisation,
concentrating efforts on parts of a system (departments/teams), not understanding the
interconnections between these parts can lead to silo mentality and organisational
problems such as resistance to change as maintained by Lebcir (2006). Ten Have, Ten
Have and Stevens (2003:77) cites Senge (1990) when describing the principles of
systems thinking:
“systems thinking provides a way of understanding practical
business issues in terms of particular cycles and levels of detail
or complexity. The essence is to see interrelationships between
processes, rather than status quo ‘snapshots’ and linear causeand-effect chains”.
Checkland, (2004) suggests the world is a complex adaptive system, as are
organisations; this means they have the ability to adapt to changing circumstances.
Chapman (2004, 52) argues that the principle feature of a complex adaptive system is its
“ability to survive significant changes in its environment through changes in behaviour
and internal processes”. Public sector organisations (under the New Public
Management regime) are typical examples of complex adaptive systems, given the
amount of change they’ve endured over the past two decades, however the question
arises, ‘are they being managed as such’? Moreover organisations not only need to
adapt to their changing environment but they need to learn to from it and evolve, thus
becoming complex evolving systems, or a learning organisation as proposed by Senge
(1990).
Richards (2003) argues that such public sector reform has taken control from
professionals and given it to managers with a more strategic, efficiency-orientated
responsibility. An organisation’s ability to respond to such change is dependant on its
ability to understand itself as a system; do managers within SWFRS understand this
concept? Taking a holistic view of problems requires systems thinking as argued by
Senge, (1990), the greatest leverage for change according to Ohno, (1988), is achieved
when one understands an organisation as a system. The author (through this research)
aims to examine if such SWFRS posses such a holistic view of itself. Perri 6 (1997) as
cited by Bundred (2006:125) suggests that the core problem with government is:
28
“that it has inherited from the nineteenth century a model of
organisation that is structured around functions and services
rather than solving problems. Budgets are divided into separate
silos for health, education, law and order and so on. The vertical
links between departments and agencies in any one field and
professional groups such as the police, teachers, doctors and
nurses are strong. The horizontal links are weak or non-existent”.
From a performance management point of view, system thinkers such as Marr (2007),
and Seddon (2008) argue that too many organisations are using performance data to
control behaviour rather than informing the decision making process and direction of
organisational learning. This is further disputed by the quality theorists, who imply that
the majority of performance problems are attributed to the way an organisation is
designed and managed. Deming (1982) and Juran and Godfrey (1999) suggest that
between 85% and 95% of the variation in performance is attributed to the system,
arguing that the performance of anyone is largely governed by the system that he/she
works in. Command and control management methods focus on managing the people
and their behaviour and not the system, therefore in keeping with Deming, (1982) such
managers tend to concentrate on the 5%, which can be argued is not very cost effective.
System thinking theorists for example Senge (1990), Checkland, (1999), and Seddon,
(2008) take the view that a good system with average people will perform, whereas a
poor system with excellent staff will produce poor quality. This concurs with Juran
(1988) and Deming (1982) who agree that the main variable within an organisation is
the design of the system.
The world has endured many changes since the days of the ‘stop watch’ management
theories created by the likes of Fayol, Taylor and Ford, the principles of scientific
management have no place in such a dynamic changing, complex world. Having gone
full circle and returned to basic principles, system thinkers such as Peters, (1992) claim
that the ‘new organisation’ regimes puts expertise back (to where it belongs) close to the
action and back in the hands of the professionals as suggested by Crainer, (2000:189) as
being “as it was in craft-oriented, pre-industrial revolution days”. Peters (1992) argue
that command and control is a thing of the past and organisations require a more
‘curious’ workforce who are creative, imaginative and innovative, which is slightly
29
apposed to the current Government control management regime with similarities to that
of scientific management. One of the current systems thinking activists, Seddon is
credited with translating the world renowned Toyota Production System (TPS) into a
system for service organisations, and who is currently campaigning Government to be
rid of the perverse target regime and adopt a systems thinking approach to managing
public sector organisations. One of the main principles of the TPS otherwise labelled as
‘lean production’ by Womack, Jones and Roos (2007) is seeing an organisation “as a
continuous and uniform whole, a stream including supplies as well as customers” as
cited by Crainer, (2000: 201). According to Liker (2004) the quality guru Deming
influenced the Japanese after the Second World War, which led to Taiichi Ohno paving
the way and developed ‘The Toyota Way’. Fujio Cho the then President of Toyota
Motor Company in a rare interview with Liker (2004:xv) suggested that the uniqueness
of Toyota’s success was:
“The key to the Toyota Way and what makes Toyota stand out is not
any of the individual elements…. But what is important is having all
the elements together as a system. It must be practiced every day in
a very consistent manner – not in spurts”.
Can this way of management thinking allow public sector organisations to deliver a
better service to the communities they serve; is it not possible to change our
management thinking from a top down, to an outside-in perspective, focusing on
customer needs? The WAG (2004:3) vision in their publication ‘Making the
Connections: Delivering Better Services for Wales’ would suggest yes, stating:
“our vision brings citizens and communities into the centre of the
way public services are designed and delivered. Services should be
more responsive to the needs of the users…”
Radnor and Boaden (2008) argue that such a change in management thinking can lead to
greater efficiencies, suggesting that many public sector organisations who are looking to
adopt lean management thinking, focus mainly on the tools and techniques, rather than
embracing the whole ‘lean philosophy’. Radnor and Walley (2008:14), concur that lean
works best if driven by all the people, suggesting that “developing a culture that creates
30
the involvement of everyone in an organisation is critical for the implementation of the
‘lean’ philosophy”. Without such a culture, changing management thinking alone will
not pave the way for continual improvement. SWFRS do possess a culture of shared
values and beliefs, which is based around the moral aspects of some of its core
functions, saving life and property and protecting the environment, as described by
Handy (1993) as task culture. However the task has evolved over the past decade, under
the ‘modernisation agenda’ much of the FRSs work has taken a more preventative
approach and has undergone considerable change, moving from a reactive emergency
service to a more proactive risk reducing service. It is questionable however whether the
culture has changed in line with these transformations, the author here, would suggest it
has been slow and is a little out of step of what Welsh Assembly Ministers and senior
managers would like to see. Organisational culture is a research area in its own right and
therefore falls outside the boundaries of this study.
According to Seddon (2005, 2008) today’s public sector command and control, topdown, hierarchal management structure does not fit with the current dynamic, ever
changing complex world. This concurs with Bosch et al (2006:218), who argue that
“new ways of thinking are required (if not essential) to manage complex problems”. A
different approach is required, one of synthesis and alignment as forwarded by the
University of Missouri (2008). Synthesis means to look at the organisation as a whole
(system) and focusing on what is important in terms of its purpose, and alignment
means linking the key strategies with the key processes and aligning the resources to
improve overall performance and satisfy customers (see appendix.4 on page 86, above).
Joiner (1994:25) agrees, suggesting that “to manage an organisation effectively, we
must begin to think more in terms of relationships than independent components”. One
of the objectives of this research is to examine if such a bureaucratic organisation can be
transformed into a ‘whole system’ organisation that understands itself as such and
manages itself as such. This is a challenge for the host organisation, or as systems
thinkers would suggest a ‘no brainer’. De Bono (2002:129) argues that “we can seek to
do better and better at what we are doing now, or we can change the way we do
something”. Einstein (1879-1955) further suggests that “everything has changed except
our way of thinking”. Deming (1997) makes the case that seeking to do better and better
is just ‘tinkering’ with the system; one has to change the system for continual
improvement, but first one has to understand an organisation as a system..
31
There are many examples of the extent to which public sector organisations are
becoming serious about adopting a systems thinking methodology, the ODPM
sponsored project ‘A Systems Thinking Approach to Service Improvement’ is one and
the results of the which were found to be extremely positive as suggested by the ODPM,
(2005). The above project involved three local authority organisations from the North of
England, Tees Valley in Middlesbrough, Leeds South East and Preston City, all of
which identified positive findings, which were designed from an outside-in perspective
so as to ‘keep the customer satisfied’. Having researched a wide variety of literature and
developed a greater understanding of the differing academic’s views on such a very
topical and interesting subject area, the author now intends to examine the extent to
which SWFRS staff understand and manage performance, whilst measuring its
effectiveness. Through a comprehensive investigation of SWFRS personnel who hold
responsibilities for business planning and performance management, the author will
identify whether a change in management thinking will lead to improved performance.
Furthermore the research will identify current management thinking on performance
whilst trying to establish if the current performance management system is too
bureaucratic and focused on targets rather than concentrating on improving service
delivery.
32
3.
Adopting the right approach
“Science cannot solve the ultimate mystery of nature. And that is
because, in the last analysis, we ourselves are part of the mystery
that we are trying to solve”. Planck, (1958 – 1947).
Having completed a comprehensive review of the literature, and possessing a greater
understanding of ‘what the academics say’, the author now intends to identify the most
appropriate research methods and techniques, which will ensure the research aim is
achieved, whilst recognising their limitations. Saunders et al (2007) suggest research
design is about developing of a plan of how to approach answering the research
question. Furthermore this section will provide critical reasoning why alternative
approaches were not selected. Bell (1999) argues that the initial approach should be to
firstly appreciate what information is required and why, before deciding on how to
collect and analyse that information. Embarking on a research project, one must be
highly motivated, disciplined, exercise good judgement, have intelligence, imagination,
determination, and a little luck. One of the most important qualities in doing research is
to ask the right questions at the right time (Flick, 2009).
‘What is research’? Drew (1980:4) concedes that research “is conducted to solve
problems and expand knowledge, but stresses that research is a systematic approach to
asking questions”. Research according to Leedy and Ormrod (2005) originates from the
need to answer a question or solving a problem, the aim of this research, is to answer the
question ‘can, a change in management thinking, lead to the development of a more
effective continual improvement culture with SWFRS’? Furthermore this research
intends to enhance the authors understanding of performance management, in order to
establish its current effectiveness and to disseminate the findings to the senior managers
within SWFRS, whilst discussing the appropriateness of targets. Leedy and Ormrod
(2005:2) concur with the above by suggesting that research is:
“the systematic process of collecting and analyzing information
(data) in order to increase our understanding of the phenomenon
with which we are concerned or interested”.
33
The author has already confirmed his interest and research rationale in relation to the
chosen research area, whilst seeking to examine an alternative (systems thinking)
approach to performance management. In relation to a systematic approach or research
methodology, this study will follow the Saunders et al (2007) ‘research process onion
model’, peeling off the political, theoretical and philosophical layers of the onion to
uncover the core information required to answer the research question as proposed by
Seale (1998). The following sections will expose each layer of the onion at a time,
beginning with research philosophy as described by Saunders et al (2007:101) as being
“the overarching term that relates to the development of knowledge” whilst linking the
chosen philosophy with the authors more natural choice of an ‘inductive’ approach of
developing a theory. The research strategy and data collection types and methods will be
discussed, debating the advantages and disadvantages of gathering qualitative and
quantitative data include where they can be incorporated in a research. The chapter will
close with discussing the ethical issues associated with research, whilst examining the
credibility of the research findings to ensure the evidence and conclusions will stand up
to ‘close scrutiny’ as suggested by Raimond (1993).
3.1
Beliefs and Values
“Remember that what you believe will depend very much on what
you are”. Porter, N (1811-1892).
Before commencing any research project the researcher has to be aware of his/her own
beliefs and values. Having an appreciation of one’s philosophical standpoint will
underpin the research strategy acting as a foundation from which to build. As indicated
above the author’s native position is the importance of researching people’s (rich) views
and opinions of the research topic, as the people (in the eyes of the researcher) are the
life blood of any organisation. Attempting to evaluate the current performance
management system within SWFRS without speaking to (real) people is like trying to
understand a book just by reading the front cover, one has to delve much deeper.
SWFRS (2009) according to their core values, value all their employees by practicing
and promoting; fairness and respect, recognition of merit, honesty, integrity and mutual
trust, personal development and co-operative and inclusive working, the researcher
therefore feels strongly about their involvement in the research, by capturing their views
and opinions.
34
Research philosophy according to Saunders et al (2003:83) is about “the way you think
about the development of knowledge”. Denzin and Lincoln (1994) suggest there are a
basic set of beliefs that direct investigation, which consist of three main elements,
ontology, epistemology and methodology. Ontology is “the branch of philosophy
concerned with the nature of existence and the relations between things” as provided in
the Longman Dictionary of Philosophy, (1999). It relates to the question of whether or
not a reality independent of knowledge exists, where as epistemology as maintained by
Marsh and Stoker, (2002) is the study of knowledge. This research follows the
subjective view of ontology, in line with Remenyi, Williams, Money and Swartz
(1998:35) who suggests, “it is important to study the detail of the situation to
understand the reality or perhaps a reality working behind them”, thereby taking an
interpretivist view of the world, believing that the world is “far too complex to lend
itself to theorising by definite laws in the same way as the physical sciences” see
Saunders et al (2007:106). This according to Saunders et al (2007:108) “is often
associated with social constructionism”, which corresponds seamlessly with the
author’s philosophical viewpoint. This viewpoint is important; given the research topic
of performance management clearly requires the investigation and involvement of
people (social actors). People’s understanding of work, performance and learning,
requires “abstract entities such as thoughts” as detailed by
Johns and Lee-Ross
(1998:16), as apposed to the positivist, objective, scientific approach that Remenyi et al
(1998:32) suggests:
“prefer working with an observable social reality and that the end
product of such research can be law-like generalisations similar to
those produced by a physical and natural scientist”
Management problems as stated by Chapman, (2002), and Johns and Lee-Ross, (1998)
are inherently complex and very often unique, and as such require a holistic (systems
thinking) approach to their study. A scientific, positivist approach is unable to provide
this, as science is concerned with breaking down situations into elements which can be
verified one by one. Given the research aim is to measure the effectiveness of
performance management within SWFRS, and paying cognisance to what has been
written in the literature, it is unlikely that a deductive approach to this study will
provide the required depth of information, as a deductive approach, as indicated by
35
Johns and Lee-Ross (1998:7) by its very nature deduces a “conclusion from something
known or assumed”. The author here acknowledges the researcher’s intrinsic
involvement, common to this research approach and the possibility of bias, however
every attempt will be made to ensure any bias or distortion is avoided. It is therefore
intended to apply an inductive approach, by developing a theory through careful data
collection and analysis before inducing a conclusion, which has been based on people’s
interpretation and understanding of the research topic. This phenomenological approach
according to Johns and Lee-Ross (1998) tends to be rich and complex, unlike the
scientific, positivist approach which relies on one methodology and one world view.
3.2
Strategic intentions
“If you are planning for a year, sow rice; if you are planning for a
decade, plant trees; if you are planning for a lifetime, educate
people” Chinese Proverb.
Although tradition suggests the interpretivist approach lends itself to adopting a
qualitative research method, it is widely acknowledged for example Jankowicz, (2005)
and Flick, (2009) that indeed both quantitative and qualitative methods can be used
within the interpretivist context. The researcher’s intention is to capture “rich
descriptions of the social world”, as suggested by Denzin and Lincoln (2000:10) as
these are seen as important and will allow the researcher to get closer to SWFRS
personnel, capturing their individual thoughts and views. Qualitative researchers argue
that quantitative research will rarely provide such perspectives due to the scientific
empirical methods used, this research does intend to utilise both quantitative and
qualitative methods, where the intention is to build upon the initial quantitative
conclusions by exploring a smaller sample in greater depth as proposed by Johns and
Lee-Ross (1998). The author is more concerned with people’s views and thoughts on the
current performance management regime with SWFRS and how (if at all) it can be
improved, as described by Hakim (1987:28) as the “worm’s eye view” as apposed to the
“birds eye view” of the quantitative researcher. Furthermore Gummesson (1991) as cited
by Connell, Lynch and Waring (2001:3) maintains that:
36
“due to the uniqueness of human beings in business situations,
there is considerable likelihood that existing theory will prove
inadequate for new investigations”.
This process is regarded by quantitative researchers such as Denzin and Lincoln, (2000)
as being unreliable. Each method of data gathering be it qualitative or quantitative, not
only brings its own individual strength to the research but also compensates for
weaknesses or limitations in other methods, thus providing a more rounded outcome.
Triangulation as suggested by Henn, Weinstein and Foard (2006) also overcomes any
bias. The aim to this research is to examine performance management from a variety of
different perspectives, providing a thorough research study from which to extract
valuable learning thus making the study both worthwhile and valid. The author is aware
of the tendency of researchers to drift away from what the research is striving to
achieve, frequently capturing interesting information at the expense of valid
information, as warned by Frankfort-Nachmias and Nachmias (1996). This research is
no exception and many times the author needed to refocus on the research aims to
ensure ‘it does what it says on the tin’, constantly reaffirming what the research has set
out to examine. It is important to note, that the author has made certain preliminary
assumptions, based on the literature review, in order to grasp a better understanding of
the research topic. He has not however, developed a hypothesis and does not intend to
influence or distort the data and will endeavour to maintain complete integrity
throughout. The proposal for this research is to collect valuable data from a variety of
methods in order to develop a theory rather than commencing with one.
Development of a research strategy became a little easier having identified the author’s
ontological and epistemological standpoint, including the author’s motivation to “probe
deeply, uncovering subtle and complex issues” in relation to performance management
within the host organisation, as suggested by Johns and Lee-Ross (1998:121.) Denzin
and Lincoln (2000:371) contend that “strategies of inquiry connect researchers to
specific approaches and methods for collecting and analysing empirical materials”,
taking a qualitative approach, whilst proposing to adopt an evaluation type study,
through which the researcher can judge the current effectiveness of the performance
management system within the host organisation as proposed by Leedy and Ormrod
(2005).
37
Reviewing the main research strategies available for an inductive approach, removed
both experiment and survey options. These methods are associated with a deductive
approach and arguably will not be suitable for the purpose of answering the research
question, as indicated by Saunders et al (2007). Leedy and Ormrod (2005) argue that
ethnography is rooted in the inductive approach and is especially useful for gaining an
understanding of the complexities of a particular, intact culture, however such studies
require the researcher to “enter into a close and relatively prolonged interaction with
people in every day life”, as indicated by Denzin and Lincoln (2000:456) involving
detailed field observation, therefore this approach was deemed too time consuming for
this study.
Glaser and Strauss (1967) suggest that grounded theory, arguably the best example of
the inductive approach aims to systematically collect, and analyse data in order to build
theory. Data analysis begins almost immediately, from where the researcher groups the
data into categories, requiring coding to extract meaning, which requires certain skills
and can be very time consuming. Furthermore researchers according to Strauss and
Corbin (1990) are not encouraged to review the literature before embarking on a study,
which prevents the author from gaining an appreciation of the research subject and the
academic’s view prior to gathering data, therefore this approach was also dismissed.
According to Stake in Denzin and Lincoln (2000) ‘Handbook of Qualitative Research’,
case study is not only a process of inquiry but also the product of that study, providing
the researcher with a better understanding of a particular case. As alluded to in the
introduction, the author holds a healthy fascination for performance management and its
association with an organisation’s strategic intent. Particularly how these two crucial
management functions perform together in order to deliver the required outcomes.
Therefore the author wishes to develop an enhanced awareness of the effectiveness of
the performance management system within SWFRS. Consequently, what Stake in
Denzin and Lincoln (2000:437) identifies as an intrinsic case study approach has been
adopted, which according to Morris and Wood (1991:245) allows the researcher to
“tease out the stories of those living the case” in order to obtain a rich insight of the
research and processes being enacted. Case study as defined by Robson (1997:53) is:
38
“a strategy for doing research which involves an empirical
investigation of particular contemporary phenomenon within its
real life context using multiple sources of evidence”.
Case studies allow for a range of data collection methods to be used, thus the methods
employed in this research consist of a combination of questionnaires and semistructured interviews. The purpose of the questionnaire was to act as a litmus test, so as
to identify key personnel and their initial thoughts, before embarking on the deeper data
gathering interview process. In fact the questionnaire doesn’t actually ask questions it
merely provides statements, which participants can either agree or disagree with, thus
removing the loaded question possibility and therefore could be described as a
statementaire. It is acknowledged that adopting such a multiple data gathering approach
will require confirmation of information through triangulation, thus substantiating initial
conclusions gathered from questionnaire with those from the semi-structured interviews,
“clarifying meaning and verifying the repeatability of an observation or interpretation”
as put forward by Denzin and Lincoln (2000:384). Academic argument exists, for
example Bryman (1989); Hammersey (1996); Strauss and Whitfield (1998) that
triangulation increases the strength of the research findings and validity. Triangulation
means ‘making triangles’, derived from land surveying, suggesting you can find an
unknown point; from two know points, thus providing overlapping information. Johns
and Lee-Ross (1998), maintain this technique is often the most effective way to evaluate
a new management imitative.
Great care and focus was taken in the design and wording of the statements, so as not to
confuse to mislead the participants (see appendix.5 on page 87, above). As promoted by
Johns and Lee-Ross (1998) a small pilot group was utilised to trial the questionnaire to
ensure the statements were appropriate and relevant to the research aim. The author here
however wanted to be sure the target audience understood the subject area, particularly
terminology, and was conscious that his own understanding and knowledge on the
subject area had developed substantially during the study. Bell (1999:84) contends that
such an exercise will “get the bugs out”, thus minimising any difficulties for the
participants and hence responses. As a consequence the questionnaire was amended
several times. The researcher was also mindful that eight colleagues carrying out similar
research projects within the host organisation could also be using questionnaires as part
39
of their study, in addition the timing was inconvenient, as the distribution of
questionnaires was carried out during the summer leave period. This potentially could
affect the return rate, which is clearly acknowledged as being a particular weakness for
this method of data collection by Leedy and Ormrod (2005). However the
questionnaires allowed the researcher to target a wider audience in the first instance,
thus ‘testing the water’.
The selection of statements used in the questionnaire was intended to get a feel for the
participant’s attitudes and perceptions in relation to the research area. The technique
utilised to evaluate and quantify the findings was a Likert type rating scale. There
appears to be a debate as to whether to include a mid-point in such scales, which Leedy
and Ormrod (2005:187) maintain provides respondents with the option to “straddle the
fence” by the inclusion of an undecided, neutral response. The author is aware that fence
straddler’s responses can muddy the waters when searching for definite outcomes.
Furthermore this is the reason why the author sought to use a qualitative research
approach so as to confirm (or otherwise) the findings of the survey. Each respondent
was required to rate each statement on a five point scale, ranging from strongly disagree
at one end to strongly agree at the other. The subsidiary role of the questionnaire, in
accordance with Connell et al (2001) served to provide broad quantifiable background
data from which to develop the case studies and assist in designing the format of the
semi-structured interviews. Voice recording the interviews, as suggested by Henn, et al
Foard (2006), provides the author with a number of advantages over note-taking, firstly
to capture a full transcript of the interview, which is seen as being more difficult when
relying exclusively of note-taking, secondly to allow the researcher to take additional
notes during the interview to enhance the data and thirdly to increase data reliability.
3.3
Ethical and credible research
“In law a man is guilty when he violates the rights of others. In
ethics he is guilty if he only thinks of doing so”.
Kant , (1724-1804)
Henn et al (2006) suggest that research in the social science has greater potential to be
more intrusive than that of the natural science and as such, the ethical implications of
such research have been at the fore front of the authors mind throughout. Being
40
conscious of and making distinctions between how Henn et al (2006:68) describes as
“what is right and just in the interest of those who are the focus of the research”. Barnes
(1979:16) defines such ethical factors as those which:
“arise when we try to decide between one course of action and
another not in terms of expediency or efficiency but by reference of
standards of what is morally right or wrong”.
Every attempt will be made to avoid any such wrong doing throughout the whole
process and at no stage will the author distort or influence any research findings. It
would however be wrong to suggest that the author does not posses any bias towards the
research topic, as by its very nature (and as explained earlier) this research topic is one
that the author finds very interesting and therefore holds views and opinions of his own.
It is also fair to suggest that in having such interests and views on the research topic the
author has in many ways impacted on how and under what circumstance the research
has been conducted. When embarking on such a study, one sets off on a development
journey, not only researching a given subject but also learning and developing one’s
own knowledge. This will inevitably lead to certain personalised conduct; however the
author has, at every juncture of the research been conscious of how and where such
behaviour can affect the study and has endeavoured to guarantee impartiality
throughout. The author feels strongly about research and the way in which it should be
conducted and mirrors the thoughts of Henn et al (1996:69) that “no information is so
valuable that it should be obtained at the expense of eroding an individual’s personal
liberty”. The challenge for the author here is to balance the needs of gathering rich
information from the interviewee whilst respecting their own personal needs.
The use of voice recordings as a means of capturing important information during
interviews produces its own inherent ethical concerns. Henn et al (2006:192) argue that
such methods can in fact “dissuade frankness or increase nervousness to unacceptable
levels”. The author here was also conscious that such interviews, which tend to be more
free-flowing than those totally reliant on note making, can go off track and will require a
little more control, which could be seen as directing the interviewee. Moreover when
transcribing the recordings into the written word, the author will endeavour to give a
41
true account of all that was discussed, as suggested by Arksey and Knight (1999) as
being good practice. Silverman (2001:230) warns that:
“when people’s activities are tape-recorded and transcribed, the
reliability of the interpretation of the transcripts may be gravely
weakened by failure to transcribe apparently trivial, often crucial
pauses and overlaps”.
The researcher’s rationale for using voice recording was not an attempt to hold the
interviewee to ransom but an attempt not to miss vital information and this is how it was
explained to all participants prior to the interviews. It was however highlighted during
pilot interviews that the presence of the recording device posed some what of a threat to
the interviewee and once relocated under the desk, the interviewee felt much more at
ease and obliging. The positive aspect of the voice recordings, as suggested by Henn et
al (2006) was to accurately harvest the true voices of the interviewees rather than any
manipulation or bias that the interviewer may attempt during the interview. In an
attempt to eliminate such bias, the author produced a list of open questions based around
the research aim, (see appendix.6 on page 92, above) with a view of asking each and
every participant the same question, in the same way, without any leading or steering,
but applying a consistent approach throughout, in an attempt to minimise variation.
Initial authorisation was sought from the Chief Fire Officer to carryout the research,
stating clearly the methods to be employed and the purposes of the research,
furthermore a covering letter was produced, explaining that participants had a choice
whether to take part or not. Additionally prior to inviting participants to take part in the
interviews, the author sought agreement from them to record the interviews, on the
understanding that all tapes would be erased once the data had been captured and
transcribed, whilst offering each individual the right to withdraw at any point, clearly
indicating this research was purely voluntary.
A further concern was the author’s position within the organisation, particularly in
relation to the interviewees. Any power differential between the interviewer and the
interviewee can influence the environment in which the interview is taking place.
Moreover the interviewees may only offer social desirable information that they think
42
the interviewer wants them to provide, as described by Saunders et al (2007:149) as
“participant bias”, particularly if they feel intimidated, rather than empowered, this
according to Henn et al (2006) is where validity is threatened by ‘reactivity’. This can
be further compounded by the wearing of service uniform, when interviewing nonuniform personnel. In an attempt to minimise the possibility of affecting or distorting
the information, the researcher sought early dialogue with all interviewees and by
clearly explaining the purpose of the study, whilst offering total confidentiality was able
to craft an environment that would eliminate the power struggle, thus developing
professional honesty between the interviewer and the interviewee, as put forward by
Leedy and Ormrod (2005). Building a rapport with the interviewees, as discussed by
Finch (1993:168) as a process involving the development of “an identification” and a
“relationship” with participants, can reduce such effects.
Being conscious of the ethical issues is vital in social research, as is providing reliable
and valid conclusions. Reliability according to Saunders et al, (2007:149) is “the extent
to which the data collection analysis procedures will yield consistent findings” where
as, validity is concerned with whether what is being measured are the things that need to
be measured. Cano (2009) offers a simple term; reliability and validity address issues
about the quality of the data and appropriateness of the methods used in carrying out a
research project. Therefore by combining different methods of data collection and
employing triangulation can according Denzin and Lincoln (1998) potentially provide
more credible findings. Brewer and Hunter (1989:17) agree suggesting that “mixing
methods is all about trying to attain validity in research”, this corresponds with the
author’s desired outcome. The various methods of data gathering also required different
methods of analysing, interpreting and presenting the data in order to provide a
meaningful conclusion. The following chapters will discuss the data analysis techniques
used, whilst presenting the findings from both quantitative and qualitative methods.
43
4.
Organising and analysing the data
“Reason is the slow torturous method by those who do not know the
truth discover it”. Pascal, (1623 – 1662).
This research was undertaken to evaluate the effectiveness of the performance
management system currently being employed within SWFRS. A case study
methodology was adopted, applying a multi-method approach to data gathering, using
two individual methods in which to capture the information. An initial questionnaire to
provide the researcher with an early representative overview of peoples understanding
of the subject area was used, as suggested by Johns and Lee-Ross (1998). This primary,
mainly quantitative data acted as a foundation for the design and development of open
questions, to be used in the semi-structured interviews. Such an approach allows the
author to dig deeper and capture richer qualitative data, at the same time developing
triangulation of findings. Furthermore the information gathering from the initial
questionnaire aided the author in refining the research; as suggested by Henn et al
(2006). The statements within the questionnaire were based around the research aims
and placed under five main headings; performance management, business planning
process, measurement, communications and learning and development. Each section
consisted of four carefully structured statements, taking care not to unintentionally steer
the participant in a certain direction, as proposed by Bell (1993). Distribution of the
questionnaires was via the internal e-mailing system, Anderson and Gansnder, (1995)
suggest that this medium can provide a higher rate of return than that of a postal
medium. Each questionnaire was accompanied with a covering letter (see appendix.7
on page 93 above), clearly explaining its purpose including return date, also an
‘additional comments’ section was included to encourage qualitative feedback, in line
with Moser and Kalton (1971) suggestion, which the author believed would assist in
development of the interview questions. Furthermore each respondent had the
opportunity to undertake further involvement, by indicating their intention in a box
provided. Almost everyone involved offered to take further part in the process if
required, one individual required confirmation of confidentiality before returning the
questionnaire.
44
SWFRS employ approximately 1850 people in various roles (South Wales Fire and
Rescue Service, 2009), the author was conscious that the sample size was chosen
carefully, as Schram (2003:97) suggests “you cannot be everywhere at once or take
every possible viewpoint at the same time”. The population size was therefore
determined by the number of personnel responsible for business planning. Although
many more personnel have responsibilities for performance issues within the host
organisation, the author is keen to concentrate on the relationship between the strategic
intent of the organisation, business planning and performance management, hence the
reason for “talking to certain people rather than others” (Schram, 2003:97).
A pilot interview was also set up with a senior manager from the host organisation, to
test the format, question type and understanding, as well as to determine the likely
required duration. The rationale for doing so was similar to that of piloting the
questionnaire. The pilot interview provided the researcher with a greater appreciation of
how to manage the interview, a number of issues were raised from the pilot e.g. the
format was too rigid and required a more flexible approach. From this feedback the
researcher’s intention was to conduct the interview with far more flexibility, so as to
allow the interviewer to tease out valuable information, without being too rigid. A
senior manager acting as the guinea pig suggested reducing the number of questions, to
focus conversation around fewer key points, thus yielding what Silverman (1993)
suggests as deeper information as apposed to more shallower data. A further point of
note was the time that participants would have to give to the process, i.e. their
availability, within an already busy schedule. The format was subsequently reviewed
and amended to take account of all the issues raised in the pilot. A positive outcome
from the pilot was the quality of information offered from the interviewee, it was
stressed from the outset that the information required was the personal thoughts and
opinions of the interviewee and not what the interviewee sees as the corporate stance on
such questions, again attempting to minimise any social desirability. The researcher
claims that providing such a comfortable, amenable environment, relaxes the
interviewee, encouraging open honest discussions, which lead to rich quality data.
The study population of 52 is taken from the number of personnel responsible for
producing business plans within the host organisation. It was acknowledged that access
to each of the 52 members represented the population to take part in semi-structured
45
interviews would be very time consuming, difficult and wholly dependant on good will,
as put forward by Bell (1993). Therefore the researcher, in an attempt to gather a wide
range of views, sought to sample a diverse a group as possible, from the total
population. Furthermore, even though the researcher’s intention was to paint the most
realistic picture from this information, reaching an ultimate truth was always going to be
impossible, as Schram, (2003) suggests. Leedy and Ormrod (2005: 145) advocate that
“qualitative researchers are intentionally non-random in selection of their data
sources”, the researcher decided therefore to adopt a purposeful sample. This method of
sampling was selected on the basis that the individual’s chosen would yield the most
information regarding the research topic; as pout forward by Leedy and Ormrod (2005),
Jankowicz (2005) and Flick (2009). A 10% sample of personnel from different
directorates, departments and stations, at various levels of management, including
uniform and non-uniform personnel, were selected and contacted via internal e-mail.
Venues, dates and times were arranged, including confirmation of the use of recording
equipment. The approach throughout the research was open and honest, gaining
people’s approval and agreement the whole time. Not only seeking authority from the
Chief Fire Officer, but also heads of department and the line managers of those
involved. A brief telephone conversation with all participants of the semi-structured
interviews was conducted before requesting their involvement. Clear explanation as to
the purpose of the research and the value it may bring to the organisation was given to
all interviewees, whilst offering assurance of total confidentiality of all information
gained during the process.
The researcher built a trustful relationship with all personnel, expressing understanding
of personal accountability and individual values, as suggested by Collins (1990).
Furthermore by applying such a sympathetic stance, the researcher was able to establish
an environment that provided rich quality information to come to the surface, suggesting
that a less open, covert approach, as argued by Bulmer (1982:217) would be “neither
ethically justified, nor practically necessary, nor in the best interest of sociology as an
academic pursuit”. Each contributor was offered feedback on the findings, so as to
confirm the value of their involvement, at the same time engendering joint ownership of
the research, as argued by Guba and Lincoln, (1989:236):
46
“When participants do not own the data they have furnished about
themselves, they have been robbed of some essential element of
dignity, in addition to having, been abandon in harm’s way”.
4.1
Making sense of numbers
“You can use all the quantitative data you can get, but you still
have to distrust it and use your own intelligence and judgement”
Toffler, (1928 – To date)
The main challenge for the researcher was making sense of the data, pulling together
both the quantitative and qualitative data so as to bring some common meaning and
significance to the study, in order to make the whole process worthwhile. With regard to
the quantitative data provided by the questionnaire, the main purpose of gathering this
data was to provide a snapshot or a very basic summary of the responses, so as to
provide a good basis from which to build further qualitative research and analysis. In
keeping with what Dey (1993) poses as a very important element of any research, the
recording, managing and analysis of data is made all more straightforward if the data
has been collected in an organised and structured way.
As previously indicated the questionnaires were distributed and retuned via the internal
e-mailing system, each individual questionnaire was then saved to a dedicated folder
with a unique reference number as suggested by Johns and Lee-Ross (1998), making
analysis easier, whilst providing a robust audit trail. All data was entered into a
spreadsheet, which allowed swifter analysis and aided the researcher in producing the
chosen presentation method. Each response to each question was ranked using a Likert
type ranking scale to enable the researcher to collect opinion data, in relation to how
each respondent viewed each statement. The resultant nominal data provided the
researcher with descriptive statistics, providing the most frequently occurring or modal
group value in a data set for any given statement. Saunders et al (2007:437) argue that
“the mode is the only measure of central tendency that can be interpreted sensibly”.
This allowed the researcher to identify the most common view of the participants for
each statement within the questionnaire, so as to compare with the literature. During the
questionnaire process of the study, one of the population group retired and the position
47
is currently vacant, this reduced the population size down to 51. From of the 51
questionnaires distributed, 45 were returned, equating to an 88 % return rate, the
researcher here considers this to be a significant return, allowing for a meaningful
evaluation. The analysis of the data from all twenty four questions can be found in
table.2, appendix.8 on page 94 above, including individual graphs for each question.
4.2
Analysing the richer side
“The quality of an organisation can never exceed the quality of the
minds that make it up”. Eisenhower, (1890-1969).
The analysed data from the questionnaire allowed the researcher to develop a
meaningful research strategy for capturing the valuable qualitative element of the
research. Qualitative data gathering is very time consuming and requires good data
management, therefore the researcher adopted the Creswell (1998) ‘data analysis spiral’
from which to manage all the data (see appendix.8 on page 107, above). The
management of the data is vital from the outset, identifying what data is required, how
to go about gathering it, and most importantly how to manage the information when it’s
been collected so as to ease analysis and presentation. As previously stated the semistructured interviews were voice recorded to capture the whole conversation, this was
then played back numerous times and transferred to the written word, prior to being
word processed. The researcher had the opportunity to utilise audio-typists from within
the host organisation, however the use of such a resource would have compromised any
confidentiality and therefore the use of this facility was dismissed. For each interviewee
a separate folder was produced and field notes as well as the transcriptions were saved
into these folders, thus completing the organisation section of the spiral.
The data captured underwent numerous cross-examinations in order to make sense of
what it was trying to say, whilst teasing out the valuable information that will support
(or otherwise) the academic debate. Information which was deemed irrelevant was
placed to one side. This initial process took a holistic view of the qualitative data,
however a more focused view was required and the data was then categorised, searching
for key themes, looking for meaning and getting a general sense of emerging patterns as
promoted by Creswell (1998). The categorisation of data was influenced by the research
aims and that of the academic literature reviewed. The researcher in developing the
48
format for the semi-structured interviews did so with a view of pre-empting
categorisation of the responses in a sequential format, which would allow the research
aims and objectives to be debated and the discussion to flow accordingly. This
theoretical framework as posed by Saunders et al (2007) assisted the researcher in the
formulation of units of data into manageable, related bits of information to be analysed
further. For example, in order to establish an understanding of what the interviewees
perceived as being the host organisations strategic intent, the opening question posed
was: ‘what do you think is the organisations purpose, why are we in existence, what are
our goals’? The data extracted from the discussion that followed was, primarily
‘ensuring a red fire engine is readily available’ for four of the five responses, thus
produced a recurring pattern and one which will be discussed in depth later. This
process as proposed by Miles and Huberman (1994) is ‘data reduction’, where the data
was transformed and condensed into selective chunks of information in such a way as to
ease analysis. Each unit of information was batched according to the pattern identified
and analysed to determine commonalities and trends, so as to later compare with the
academic literature and that provided by the questionnaire.
49
5.
Presenting the evidence
“Iraq is free of weapons of mass destruction and I challenge anyone
who claims that we have them to come forward with their evidence
and present it before public opinion”. Hussein, (1937 - 2006).
This research was commissioned to critically examine and measure the effectiveness of
the current performance management system within SWFRS whilst also examining to
what affect a bureaucratic (command and control) approach can have on improving
performance within a public sector organisation, and whether such performance
management systems can assist or impede improvement. By pulling together the
information from the literature review, and the research data, this chapter will present
the findings, thus developing a theory, whilst answering the research question “can, a
change in management thinking, lead to the development of a more effective continual
improvement culture with SWFRS”. Additionally this research addresses the assertion
that “targets do lead to improved service delivery from the customer’s point of view”.
The findings will be presented in a similar sequence to that of the questionnaire format,
beginning with what the researcher sees as an important element of performance
management, employee’s understanding of the strategic aim or purpose of the host
organisation.
5.1
Delivering the vision
“Give to us clear vision that we may know where to stand and what
to stand for - because unless we stand for something, we shall fall
for anything”. Marshall (1926 – To date)
The underlying assumption that every member of SWFRS are on the same journey,
striving for the same aims in an attempt to deliver the corporate vision does not appear
to support either the academic argument or the thoughts and opinions of the managers
involved in the research. Even though 71.1% of respondents to the questionnaire, either
agreed or strongly agreed that they understood how their business plan impacts on the
strategic aims of the host organisation, this however was not supported during the
discussions with the interviewees. Four of the five interviewees suggested that the
primary role of the FRS is to provide and emergency response service and to make
50
readily available the ‘red fire engine’ to the people of South Wales. It must be stressed
that the interviewees were all middle managers responsible for business planning and
performance; the findings therefore poses a further question; if these people are unclear
as to the core business aims of the organisation, the likelihood is the lack of clarity will
continue down the hierarchal chain to the very people attempting to deliver the not so
clear vision. As Marr (2009) explains above, one of the key contributory factors of
failing performance management initiatives in public sector organisations is people’s
lack of understanding of the strategic aims of their organisation. This is further
supported by Pryor et al (2007); Humphreys (2004) and Radnor and McGuire (2004)
perhaps suggesting that SWFRS should commission further research into this area to
establish if the employees of SWFRS are all singing from the same song sheet, as this
research suggest they are slightly out of tune. The researcher, in an attempt to delve a
little deeper, probed the interviewees to establish where they thought Community Fire
Safety (CFS) sat within the strategic goals. There were positive responses as to the
worth and success of this element of FRS duties, but apart from one interviewee, the
responses stated that this was a secondary role and providing an emergency response to
the communities was first and foremost. When considering the wider strategic
organisational implications, the survey concluded that 88.9% of business planners
believe that a more collaborative approach to developing business plan will lead to
greater organisational improvement. This statement concurs with the system thinkers
Seddon (2008), Radnor and Walley (2008) and Womack et al (2007) who suggest that a
more holistic, joined-up approach is required in today’s complex world. This first
interview question dovetails into many of the questionnaire statements, such as business
planning, communications and learning and development, the author felt it prevalent to
engage on such an important topic, recognising the broader implications on the research.
5.2
Managing performance
“When performance exceeds ambition, the overlap is called
success”. Hightower, (1923 – To date)
In essence measuring the effectiveness of the current performance management system
with the host organisation is this researches core aim, and in attempting to so, the
researcher had to firstly establish how managers viewed performance management,
whether they were in agreement with Radnor and McGuire (2004); Verbeeten, (2008)
51
and Marr (2009) as to the true purpose of performance management. The questionnaire
provided the following statistics: in relation to those managers who has received the
relevant training and development in performance management, a mere 57.8% had
undergone certain development, where as 35.6% hadn’t. Out of the 35.6% of managers
that had not received any such training, over half were managers responsible for either
department or directorate planning. Therefore, as performance management is seen an
integral part of the overall strategic mechanism of an organisation, it would be expected
that such levels of management would have received the appropriate skills, knowledge
and experience to manage such an important element of any organisation. This statistic
supports further evidence that suggests performance management within SWFRS is not
given enough status in terms of its importance to the overall management of an
organisation. Furthermore evidence obtained from the questionnaire in relation to
people’s perceived purpose of performance management confirmed that 77.8% felt it
was more of a control and accountability tool, where a similar question produced a
lesser 53.3% that agreed or strongly agreed that the purpose of performance
management was to learn and improve as argued by Marr (2009). This is further
supported by the interview data, where interviewees suggested that “performance
management within SWFRS is about accountability and nothing else”, “people pay lip
service to the whole process”, one interviewee stated performance management was
seen as more a “big stick rather than a carrot” and on occasions there has been “naming
and shaming” of failures within certain management meetings, which is apposed to the
academic argument, where Marr (2009); Hume and Wright (2006) and Senge (1990)
claim that a performance driven culture, should be underpinned by a learning and
development strategy. This approach to performance management however, does fall in
line with what the Audit Commission (1999) argues as being the second key reason for
setting performance measures, so as to reinforce accountability. 75.8% of respondents
felt that the current performance management system was an effective management
tool; half of those saw performance management as being a dual tool, as both an
accountability and learning tool. One respondent argued that “performance
management is done after the event or because we’ve been told to do it, rather than
using the tools available to plan and manage effectively”. There were opposing views
as to what if anything is done when business plan objectives are not meet, evidence
suggests there is no consistency and that in certain circumstances a small narrative
within the business plan quarterly update if suffice for non delivery, otherwise known as
52
moving the goal posts, where others stated they have undergone “finger waving”
exercises in the company of peers, when failing on certain business objectives, very
often when the failure is outside of their control. This would suggest that SWFRS are
using neither, the learning or accountability tool to any great effect or with any
consistency; suggesting that SWFRS, whilst committing huge quantities of resources
into performance management are missing a great opportunity to learn and improve, as
according to Franklin (1706 – 1970) “the only thing more expensive than education is
ignorance”.
There was also a strong indication from interviewees that they felt the whole
performance management process was too time consuming, labour intensive, agreeing
with Boland and Fowler (2002); Holloway (1999) and Rouse (1993) that behind the
organisations performance management system there was a “huge machinery of systems
and people”, a further statement suggests “it appears to me right now, that it’s getting
more and more bureaucratic, whole empires being built on the back of performance
management system”. This would go against Gershon (2004) and Radnor and
McGuire’s (2004) argument that such systems should not drive resources from front
line services.
5.3
Getting to where we want to be.
“Listen to the desires of your children. Encourage them and give
them autonomy to make their own decisions”
Waitley, (1933 – To date)
The business planning procedure within SWFRS is a cascading process, where
everything flows down the hierarchal chain of command to the teams and individuals
who then transform the vision into reality. During the research there is clear evidence to
suggest that many of the business plan objectives are influenced by others, 95.6% of
respondents imply this is the case. Furthermore evidence from interviewees intimate
that managers feel they have no real control over much of their business objectives,
stating that many objectives are imposed from above with little if any discussion. Again
there is clear evidence from interviewees to suggest that the whole process is very time
consuming,
53
“a bit of a monster, the business planning process doesn’t deliver a
better service, it’s a mile off, when I complete my quarterly returns,
there’s no statistical analysis on my behalf, because I just don’t
have the time”.
Another interviewee suggested that:
“only 40% of business plans are working properly, 60% are
effectively paying lip service to the process; I don’t think we
objectively set our objectives. Our BP process ticks the right boxes,
but if you could put hand on heart and say we get clear direction
and strategy from senior managers, we’d be kidding ourselves”.
Therefore the statistics taken from questions 7 and 8 should be taken in context of these
statements, where 84.4% of managers indicate their business plan is a working
document and directs their activities, and 93.4% stated that they monitor their business
plans on a regular basis. Given the number of managers involved in producing,
monitoring and recording on the business planning process, there is sufficient evidence
to suggest that SWFRS are driving resources away from front line services as warned
by Radnor and McGuire (2004). Business planning was included in the research, as the
researcher needed to highlight the important relationship between strategic direction and
performance management and suggests that the business planning process within the
host organisation acts as this critical link. A further important relationship examined
was that between setting business objectives (targets) and performance management,
known as performance measurement. The quantitative data collected from the
questionnaire provided strong evidence that managers are familiar with performance
measurement and its association with managing performance, with a high 86.6% of
participants stating that they collect and analyse relevant data in order to deliver on their
business plans.
A further finding was that 95.6% of returns implied that they meet regularly with their
teams to discuss progress on their business plans. The survey also produced a 77.7%
return of managers who either agreed or strongly agreed that they understood what
measures they require to monitor progress on their business plans, there appeared to be
54
some contradiction between this result and that offered in the interviews. As stated
above there is evidence (from interviewees) that approximately 60% of managers
involved in the planning process pay lip service to it, furthermore one interviewee stated
that due to the finger wagging manner of managing performance, they will only
concentrate on “ticking the right boxes”, whether this delivers a better service or not. A
further interviewee alleged that “measures are not explored if the targets are not met”.
Moreover the researcher has found during his own experience of business planning that
there is a great amount of data capture from where realistic objectives could be set,
however managers are not using this data when producing business plans, confirming
with the published academic work such as Marr, (2009) who argue that public sector
organisations are drowning in data but thirsty for information. One debate as to whether
measuring core activities are difficult proved inconclusive, 44.4% agreed that core
activities are difficult to measure where as 28.9% disagreed, leaving a higher than
average number (26.7%) of fence straddler’s, who were unsure. In fact this statement
would have probably produced better results if amended to read: measuring quality core
activities is difficult. Fortunately due to the survey results the researcher was able to
adjust the interview questions to capture this information, where much of the discussion
was around targets, which is discussed in the next section. A further respondent from
the questionnaire provided the following additional comments, “I am of the view that
there is an over reliance on PI’s for the sake of statistics and their usefulness can be
exaggerated in some instances”.
5.4
Box ticking or quality service
Objectives are not fate; they are direction. They are not commands;
they are commitments. They do not determine the future; they are
means to mobilize the resources and energies of the business for the
making of the future. Drucker, (1909 – 2005).
This section of the research was the most contentious and brought about deep and
emotive discussions on the subject of targets, whilst the survey produced clear results as
to how the managers of SWFRS see the worth of targets. 83.4% of participants agreed
or strongly agreed with Heath and Radcliffe (2007); Hume and Wright (2006) and
Kennerley’s (2004) argument that what gets measured gets done, this was further
55
confirmed during the interviews where an overwhelming opinion as to the real worth of
targets came to the surface. Below are some of the views of the interviewees on targets,
and although the written word provides sufficient evidence in agreement with the
academic debate on targets, the text in this dissertation does not capture the passion and
emotion exhibited in the voices of the participants during the interviews. “Managers
will fixate on the target, there’s no question about it”, giving an example of how
perverse the targets are, one interviewee explained that because giving CFS talks to
certain groups within the community is not a stipulated target, station commanders are
not focusing their activities at these groups, as they produce “zero ticks in the box”, this
coincides with Wheeler’s (1993:85) argument that “measures of actual activity will
generally be more useful than simple counts of how many times a goal has been met”.
That said they went on to say:
“the reality is, I’m not getting a tick in the box for it, but guess
what, those people are my customers, therefore I’m directed
according to the target rather than the needs of my customer, that’s
not right, targets make people focus specifically on the targets”.
This is in line with what the academics are saying, such as Seddon (2009); public sector
organisations are hitting the target but missing the point, another interviewee stated that
“we don’t measure the stuff that really matters” arguing that SWFRS “has a big
department measuring what we don’t do, where as they should be measuring what we
actually do”, suggesting again that targets are less meaningful in terms of delivering a
quality service.
One of the most contentious areas throughout the whole interview process was that of
the Home Fire Safety Risk Assessment (HFSRA) which are carried out as part of CFS
targets. The consensus is that this target is, as Seddon (2008) argues, purely arbitrary
with no real meaning, where each station is given a number of HFSRA to carryout in a
year, the limited amount of science attached to this target is mainly dependant on the
number of staff per station, a very quantitative target, that does little if anything for the
most needy, as evidence suggests people are just chasing the numbers game rather than
concentrating on the more important quality issues of where the risk is situated and
those people most in need of HFSRA. The survey results prove that people’s perception
56
on whether hitting targets doesn’t necessarily lead to the delivery of a quality service
was clear, as 93.3% agreed with this statement. One respondent suggested that because
such targets are set from the centre and not based on local knowledge of risk, then the
people of South Wales are getting a different level of service dependant on where they
live, suggesting a “post code lottery” of service. The following statements are from
respondents who provided additional comments as part of the questionnaire:
“individual station business plan targets can some times be reset
centrally with no discussion/input from the station manager who
needs to use it…. I’m conscious that in my present role, I seem to be
judged purely on quantities and therefore do feel held back in that I
must concentrate all my efforts on hitting quantitative targets with
the inevitable effect of impacting on focused quality work…. No
consideration is taken into the inputs to attain a certain output. We
as an organisation focus on targets (have we met them or not)
without considering the hours spent striving to meet such targets”.
The majority of the academic debate placed around targets is alive and kicking in the
fire service, even gaming. There was conclusive evidence from participants that targets
can cause unintentional consequences such as cheating, otherwise know by Hood (2007)
as gaming, 80% either agreed or strongly agree with this statement and all interviewees
gave an example of such behaviour within the host organisation, suggesting that “if you
are measuring the wrong things you drive people’s behaviour the wrong way”. Another
interviewee stated:
“Hitting the target keeps the wolf from the door, I will do my best to
hit the target, I won’t however exceed it I’ll stop, as there is peer
pressure not to exceed because we know the target will be raised
next year. Gaming, never heard of it but I’ve done it, it’s been going
on for years. I agree that what gets measured gets done but it
inevitably leads to manipulation and falsifying figures to hit
targets”.
57
These statements clearly demonstrates that such behaviour is happening within the FRS,
supporting the academic evidence, as put forward by Edwards, (2007); Pidd, (2005) and
Hood, (2004) that such a culture doesn’t provide it’s intend improvements, in fact it
does quite the opposite and drives resources away from where they are most needed in
order to tick the target box, at the same time introducing dysfunctional behaviour to
organisations. Seddon and Caulkin (2007:20) concur with the above statement and argue
that this “demonstrates the extent to which well-meaning regulation can become a major
barrier to improvement”. There were however counter argument to the process by a
limited number of respondents, who argue that:
“In the planning of what we deliver to reduce risk, performance
management systems have undoubtedly made us critically review
performance data and sharpen our planning processes and also
made us far more publicly accountable”
5.5
A different way of thinking
“The world we have made as a result of the level of thinking we
have done thus far creates problems we cannot solve at the same
level of thinking at which we created them”. Einstein (1879 – 1955).
Systems thinking is not fully understood within SWFRS, certain managers have an
appreciation for this less common style of management, but it’s fair to say that the
traditional command and control style of management is still the preferred method. It’s
also fair to raise at this point that the recent relocation into a new headquarters intended
to seriously review at the way SWFRS worked, with a view of adopting a different
method, however due to the magnitude of the task, this opportunity was missed,
suggesting that a problem does actually exist within the host organisation. There are
however little pockets of enthusiasts within the organisation who are looking to a
different approach, but there appears to be limited understanding within SWFRS of
systems thinking. Much of the systems thinking debate have been captured during the
other areas of the research, however the researcher wanted to capture sufficient data of
people’s assumptions of systems thinking, in order to establish if there is potential in
adopting such a system and how much (if any) support there would be for it. Due to this
lack of understanding the researcher, therefore used such terms as communications and
58
collaborative working within the questionnaire, however the systems thinking question
was broached during the interviews and the following findings are presented. The
responses from the questionnaire were not totally convincing, in fact only 51.1% were
in agreement with the statement, ‘I consult with both internal and external stakeholders
when developing my business plan’, where as 46.6% stated that they did not, however
88.9% agreed that a collaborative approach to business planning produces greater
organisational improvement, suggesting, perhaps this is what they should be doing, and
if not then why not?. Systems thinkers such as Seddon (2008) and Joiner (1994) also
agree with the previous statement, and argue that when an organisation optimises
separate pieces (departments for example) it destroys the effectiveness of the whole
(organisation), therefore for an organisation to function effectively as a whole, the
components must work together, this doesn’t appear to be the case according to the
evidence. Furthermore for a citizen centre organisation, as presented by WAG (2004)
only 48.9% of respondents stated that they speak to their customers in order to gauge
their satisfaction of the service they receive from SWFRS, a staggering 37.8% stated
they did not, further confirmation that SWFRS are top-down rather than an outside-in
organisation, this again is captured in the academic argument, by Radnor and Walley
(2008); Seddon (2008); Lebcir (2006) and Deming (1982), posing the question, how
does SWFRS really know how they are doing in relation to customer satisfaction?
The rich data presented from the interviews mirrored that of the questionnaire
responses, having given all interviewees a brief definition of systems thinking, the
following statements were offered: “
“Yes, I’ve got a number of areas I’m responsible for and generally I
don’t care about anyone else, I just want to make sure mine are OK.
Generally from day to day I don’t care about anyone else”.
Another interviewee explained that he still doesn’t know what all departments currently
do, stating that “fire stations felt they were in isolation, and who operate as individual
silos”. A further interviewee suggested that a great deal of duplication of work goes on,
due to the fact that people concentrate on their own tasks, sometimes at the determent of
the wider organisation, mirroring the 88.9% of responders who suggest a greater
collaborative approach should be adopted. One interviewee suggested that if he was
59
given autonomy of his business plans, many of the targets that are set by the hierarchy
would indeed be in his plan, however the approach he would adopt would be to dictate
where, when and how to deliver on those targets, based on local need, rather than just
chasing meaningless numbers. The systems thinkers suggest such an approach focuses
on what the customer wants and will lead to continual improvement, where culture
change, according to Seddon (2008) comes free of charge. Four of the five interviewees
suggested that SWFRS would be better placed with one all encompassing business plan
that everyone works to, taking very much a holistic approach to the organisation. Some
of these views will be expanded on in the next section.
60
6.
Reflection
“The only possible conclusion the social science can draw is: some
do, some don’t”. Rutherford, (1872 - 1937).
This research began, what now seems like a very long time ago, at a time when the
research aims and objectives were slightly fuzzy. The journey however, has been a little
bumpy a long the way, causing the aims and objectives to be amended as progress was
being made and fresh knowledge unearthed. The information gathered during the
research has to a greater extent provided the researcher with sufficient insights, to
enable the research question to be answered, whilst also assisting to deliver the research
aims and objectives. The main aim of the research was to critically examine and
measure the effectiveness of the current performance management system within the
host organisation of SWFRS. In order to do that the researcher had to question and
challenge a number of associated areas such as the current management thinking and the
affect that a bureaucratic approach can have on improving performance, whilst
answering the question “can, a change in management thinking, lead to the development
of a more effective continual improvement culture with SWFRS”? Furthermore this
research set out to challenge the assertion that targets do lead to improved service
delivery from the customer’s point of view, at the same time evaluating whether a
systems thinking approach would provide a better alternative in delivering such
improvements.
The researcher is satisfied that the positive findings from this research has, to a greater
extent supported the academic viewpoint of performance management in the public
sector, particularly in relation to its impact on actual service delivery. Furthermore the
chosen methodology adopted for this research has allowed a deeper more valuable
insight into what the people (within the host organisation) are thinking in relation to the
current performance management system, and this was as important to the researcher as
the actual outcomes. As suggested above in Pryor’s, (1998) 5P model of strategy,
people are the lifeblood of an organisation and the researcher believes that their views
and opinions are pivotal to SWFRS if it is to deliver its vision and continually improve
and provide a quality service to the communities of South Wales. It is important that
now all the information has been captured, it is time to reflect on the journey, to
61
examine what the research has or has not achieved and how the research findings
confirm or contradict with what the academics say, so as to allow SWFRS to share in
this learning.
In relation to targets and business planning, the evidence is clear, many of the targets set
by senior managers and or the political paymasters are not (in certain circumstances)
delivering their intended purpose and, to a greater degree influencing managers to
concentrate on the targets at the detriment of delivering a quality service, thus
confirming the academic viewpoint, for example Edwards (2007), Pidd, (2005) and
Hood (2004), that targets encourages dysfunctional behaviour. There are a number of
issues which fall out of this; firstly the research has provided strong evidence that
suggest the people doing the work should manage the work, setting their own gaols and
objectives in line with local needs. Many of the people involved in this study have
strong views on what they should and should not be focusing on, stating that if given
autonomy in setting their own business objectives without interference from senior
managers, the likelihood is that many of the current targets will in fact be met, as argued
by Seddon (2008), whilst adding some much needed quality to the approach. This could
provide managers with a sense of achievement, providing then with a measure of
progress towards organisational aims, rather than focusing on numbers plucked out of
the air as suggested by some of the interviewees. Secondly SWFRS mirror many other
public sector organisations that have, at their disposal a wealth of data, the research
suggests that SWFRS (as a whole) do not currently make the best use of such
information when developing business plans and setting targets. By understanding what
information is required when setting and managing real performance, can indeed focus
peoples minds on the task ahead as Marr (2009) suggests, whilst increasing motivation
and ownership, which is lost when people struggle with the concept of arbitrary
numbers. There is a further link here between understanding measurement and a
systems thinking approach, by removing the barriers associated with functional silos,
and by focusing organisational wide, this can lead to a better understanding of the
bigger picture of what is happening inside SWFRS and hopefully developing a more
cohesive organisation. Thirdly, there was ample evidence to suggest that the whole
business planning process is too resource intensive, time consuming and appeared to be
a box ticking exercise, there was overwhelming confirmation that manager’s pay lip
service to this highly demanding process, often distracting them from focusing on what
62
they suggest is their core business. Finally the research has provided sufficient evidence
that supports the academics argument that targets do not lead to improved service
delivery from the customer’s point of view and in fact internal targets were not only not
helping people to do good work, but by focusing them away from the customer, they
were actually preventing improvement.
In relation to the effectiveness of the current performance management system, this
research has identified that great amounts of time, effort and resources are being
allocated to business planning, whereas there is little evidence to suggest that the
performance of this very resource intensive process is paying dividends. This may be
seen as a criticism and in fact when attempting to justify the management hours that are
invested in the process, only to find there is little if any performance management
enquiry undertaken, then yes this is a criticism. That said the problem doesn’t exist with
the people involved in the process, the problem is the process itself as already
highlighted by Deming, (1982) in so far as it is too big, too disjointed and lacks true
vision, and potentially too expensive, thus concurring with Radnor and McGuire (2004)
that as an overhead, potentially it does drive resources away from front line services.
The importance of the correct use of information in the decision making process has
been touched on, similarly the correct use of measurement in managing performance is
also an important factor and if this data was closer aligned with the strategic direction of
the organisation, then it would make SWFRS both data rich and information rich, thus
quenching our thirst in accordance with Marr (2009). There is sufficient evidence that
suggests SWFRS do not posses clarity and organisational wide agreement about its
strategic aims, if the overriding strategy is clear everyone will pull in the same direction
and will more likely focus on what matters. If people don’t know what they are
supposed to be doing, or why, there is likelihood the organisation’s strategic aims will
not be meet, again there is a paradox between what the vision and mission statements
purpose and the target setting process. This finding is critical for the long term success
of SWFRS, without a clear strategy (purpose) then chaos will ensue and people will do
their own thing, pulling in different directions, duplicating work, causing waste,
therefore not delivering value for money. This is not too dissimilar to the example of a
station manager giving talks to certain groups outside his station jurisdiction, because
there are no such targets. The strategic intentions of the host organisation must be clear
63
as per Marr’s (2009) performance management model above, SWFRS therefore need to
agree and clarify what matters and what to focus its efforts on, this will prevent
ambiguity and provide direction, suggesting a little further research is required in this
area.
Performance management should be about defining, assessing, implementing and
continuously refining an organisation’s overall business strategy, in other words, clearly
identifying what matters to the organisations, what is its strategic intent. Being too
output focused, and concentrating on the numbers game, can lead to perverse and
dysfunctional behaviour, and there is evidence this is happening within SWFRS. This
follows the academic argument, that when performance management is mainly used to
control behaviour it tends to eliminate organisational learning and drive gaming and
other undesirable behaviour as suggested earlier by Bevan and Hood (2006).
Furthermore a successful organisation is one that learns from its performance and
experiences more in line with a complex evolving system, constantly learning and
feeding the learning back into the decision making pot. SWFRS therefore, must ensure
it evolves and becomes what Senge (1990) suggests is a learning organisation,
otherwise it will remain in a vortex, constantly going around and around reacting to the
same issues time after time, not making any significant progressing.
The author however agrees that measurement used correctly is vital cog in the
performance management mechanism, he also concurs that the public should get value
for their hard earned money, particularly in the current economic climate. However the
author also feels that many centrally driven targets, do not take account of local need. It
is therefore important that the current performance management system within SWFRS
does provide such information, and that this information does supply management with
insights from which to base decisions, whilst learning from the whole process, helping
to deliver better performance. This therefore confirms that it is essential for managers
who make the decisions, have at their disposal the appropriate knowledge and
information at all times, otherwise many of the decisions being made are based on
what? It is also critical that the performance management framework within SWFRS
should promote learning opportunities, as stated earlier by Hume and Wright (2006); it
should be underpinned by learning and development. To increase its knowledge base,
an organisation (so as to make better informed decisions) must be a learning
64
organisation; providing a performance management system that first and foremost
facilitates and supports such learning at every junction of the process, ignorance is no
excuse. Aiming for long term sustainable improvement rather than short term box
ticking targets, this again is a challenge to the senior managers of the host organisation.
Everyone comes to work to do a good job, the FRS by its very nature has an inherent
high work ethic, ‘they get the job done’, can they however, get the job done better more
efficiently? In the current economic climate with increasing pressures on public sector
organisations the answer should be yes and without delay. A greater emphasis as argued
by the academics should be on outcomes and not adherence with processes and targets,
such as the current business planning process. By focusing on real qualitative service
delivery and what matters to the people of South Wales rather than the quantitative,
target driven measures is a step in the right direction. This however, requires a change
in management thinking, which is a far more difficult proposition for SWFRS. Working
as a holistic system, recognising that each directorate, department, team or individual
are striving for the same outcomes is quite a step change for the FRS, this may be
happening at the top table but is not mirrored or being cascaded down through the
organisation. Many managers think this is the case, but like the business planning
process, there is evidence to suggest that SWFRS are paying lip service to it. By
adopting a more joint up (systems thinking) approach, this will begin this much needed
journey of change for a real purpose and not change for change sake.
The author here however suggests that changing the current management thinking and
adopting a systems thinking approach will lead to greater efficiency, improved use of
resources, increased motivation and hence a better service to the communities of South
Wales at a lower cost, therefore perhaps the question should be when rather than how.
In conclusion this study have identified that the current performance management
system doesn’t do what it says on the tin and is a very resource intensive process with
limited governance on actual performance. This according to Marr, (2009) and Seddon,
(2008) and much of the research findings, is due to the top-down management style
found in FRSs throughout the country. The author acknowledges that all FRSs has (to a
greater extent) political paymasters who direct, and very often dictate what they can and
can’t do, raising yet again the debate about who best knows how to run a FRS, the
professional or the politician? It will take a very brave Chief Fire Officer to stand up
65
and be counted in the name of true quality service delivery, who can find ways of
circumventing the draconian target regime to do what they believe to be right. Very
much like the Chief Police Officers, Head Teachers and Chief Executive Officers who
indicated earlier in the literature review, that they have had to change the way they see
the wood for the trees and have modified their approach to concentrate on what the
customers needs. Surely this is what service delivery is about, to provide a quality
service for the communities in which these organisations operate, so is the balance right,
do the politicians have too much say in the name of accountability and ensuring a value
for money service, or should the balance be returned to the professional, who are closer
to the local needs and more familiar with the nuts and bolts of how to run a FRS?
This research concludes that the current bureaucratic approach does actually impede
rather than improve performance, the use of arbitrary targets from the centre, lack of
focusing from what the customer and local communities need, the continual auditing of
public sector organisations all contribute to a less than effective service, which is
unquestionably the wrong approach for what service delivery stands for. The dilemma
is: ‘is this bureaucratic system fit for purpose, is the current target regime about
improving performance or control and accountability, do, what the people think really
drive the system?’ The author, akin to the academics believes that this current system is
based purely on economy, accountability and control with limited focus on real
improvement and learning, the vast amounts of finances that have been poured into the
public sector under the umbrella of best value etc over the past two decades have not
been money well spent and have left such organisations confused as to their real
purpose and losing their focus for the sake of chasing random targets. This therefore
answers the research question that ‘yes’ a change in management thinking can and will
lead to the development of a more effective continual improvement culture within
SWFRS, but only if there also exists a desire to do so.
66
7.
References
Adcroft, A and Techman, J. (2008), Whose standards are they anyway? The need for
competitive spirit in public sector management. The Ashbridge Journal, Autumn 2008.
Audit Commission. (1999), Performance measurement as a tool for modernising
government. December 1999.
Australian Public Service Commission. (2001), Performance management in the
APS: A strategic framework. Australian Government.
Barnes, J. (1979), Social Sciences, Privacy and Ethics. Harmondsworth: Penguin.
Bell, J. (1993), Doing Your Research Project – A guide for first-time researchers in
education and social science. 2nd ed. Open University Press, Buckingham.
Philadelphia.
Bevan, G and Hood, C. (2006), What’s Measured Is What Matters: Targets And
Gaming In The English Public Health Care System. Public Administration, Vol.84,
No.3, 2006, pp. 517-538.
Bosch, O.J.H, King, C.A, Herbohn, J.L, Russell, W.I and Smith, C.S. (2006).
Getting the Big Picture in Natural Resource Management – Systems Thinking as
‘Method’ for Scientists, Policy Makers and Other Stakeholders. Published online in
Wiley InterSciences.
Brewer, J and Hunter, A. (1989), Multimethod research: A Synthesis of Styles.
Newbury Park, CA.
Broadbent, J. (2007), If you Can’t Measure It, How Can You Manage It?
Management and Governance in Higher Educational Institutions. Public Money &
Management, June 2007.
Bulmer, M. (ed) (1982), Social Research Ethics. London: Macmillan.
Carrol, L. (1832 – 1898). Taken form Alice in Wonderland. www.thinkexist.com
accessed 14th September 2009.
Chapman, J. (2002), System Failure. Demos, London.
Chapman, J. (2004), System Failure. 2nd ed. Demos, London.
Chinese Proverb. (Unknown). www.thinkexist.com. Accessed 14th September 2009.
Clancy, J (2008) Ministers urged to ‘back off’ – An important new study says students
in colleges are ‘achieving more but learning less’. guardian.co.uk, accessed 6th March
2009.
Collier, P.M. (2006) In Search of Purpose and Priorities: Police Performance
Indicators in England and Wales. Public Money & management June 2006.
67
Crainer, S. (2000), The Management Century. A Critical Review of 20th Century
Thought & Practice. Jossey-Bass Publishers, San Francisco.
Cumberland Lodge. (2008), Hitting the Target, Missing the Point? Conference
Report, April 25th – April 27th 2008.
Darwin, C.R. (1809 – 1882). www.thinkexist.com. Accessed 14th September 2009.
De Bono, E. (2000), New Thinking for the New Millennium. Penguin Books, England.
De Bruijn, H. (2007), Managing Performance in the Public Sector, 2nd ed. Routledge,
London.
Denzin, N.K and Lincoln, Y. S. (2000), 2nd Ed. Handbook of qualitative research.
Sage Publications Ltd.
DoH. (2005), Taking Healthcare to the Patients: Transforming NHS Ambulance
Services (London)
Drew, K. (1980), Developing Interactions Inclusive Environment. Bethlehem: Lehigh
University Press.
Edwards, R. (2008) Police chief slams Government target culture.
www.telegraph.co.uk/news. Published: 12:12. AM, BST 09 Jun 2008. Accessed 1st
March 2009.
Einstein, A. (1879 – 1955). www.thinkexist.com. Accessed 14th September 2009.
Eisenhower, D.D. (1890 – 1969). www.thinkexist.com. Accessed 8th September 2009.
Farnham, D. and Horton, S. (1993), Managing the new public services. Basingstoke:
Macmillian.
Finch, J. (1993), Great to Have Someone to Talk to: Ethics and Politics of
Interviewing Women. In: M. Hammersley (ed). Social Research: Philosophy, Politics
and Practice. London: Sage, pp166-180.
.
Flick, U. (2009), An Introduction to Qualitative Research. Sage: London.
Gershon, P. (2004), Releasing resources to the front line. Independent Review of
Public Sector Efficiency HMSO.
Greiling, D. (2006), Performance measurement: a remedy for increasing the efficiency
of public services? International Journal of Productivity and Performance
Management. Vol. 55, No.6, pp 44-465.
Guba, E and Lincoln, Y. (1989), Fourth Generation Evaluation. Newbury Park, CA:
Sage.
68
Gummesson, E. (1991). Qualitative methods in management research. Thousand
oaks, CA: Sage Publications
Hakim, C. (1987), Research design: Strategies and Choices in the Design of Social
Research. London: Allen Unwin.
Hambrick, D.C and Fredrickson, J.W. (2005), Are you sure you have a strategy.
Academy of Management Executive, Vol. 19, No.4, pp 51-62.
Hamel, G. (1996), Strategy as revolution, Harvard Business Review, Vol. 74, No. 4,
pp. 69.
Hemmingway, E.M. (1899 – 1961). www.thinkexist.com. Accessed 14th September
2009.
Henn, M, Weinstein, M and Foard, N. (2006), A short introduction to social
research methods. Sage: London.
Hightower, C. (1923 – To date). www.thinkexist.com. Accessed 14th September
2009.
HM Government. (2007), Cutting bureaucracy for our public services. June 2007.
HM Treasury. (2008), Budget 2008. Stability and opportunity: building a strong,
sustainable future. Economical and Fiscal Strategy Report and Financial Statement and
Budget Report March 2008. HM Treasury.
Hood. C. (2007), Public Service Management by Numbers: Why Does it Vary? Where
Has it Come From? What Are the Gaps and the Puzzles? Public Money &
Management. April 2007.
Hume, C and Wright, C. (2006), You Don’t Make a Pig Fatter by Weighing It –
Performance Management: The Experience of the Youth Justice Board. Public Money
& Management, June 2006.
Hussein, S. (1937 – 2006). www.thinkexist.com. Accessed 14th September 2009.
Johns, N and Lee-Ross, D. (1998), Research Methods in Service Industry
Management. Caswell, London.
Joiner, B.L. (1994), Fourth Generation Management; The New Business
Consciousness. McGraw-Hill: New York.
Kanji, G and Mora, S.A. (2007), Performance Measurement and Business
Excellence: The Reinforcing Link for the Public Sector. Total Quality Management,
Vol. 18, No.1-2, pp 49-56, January – March 2007.
Kant, I. (1724 – 1804). www.thinkexist.com. Accessed 8th September 2009.
69
Lebcir, R.M. (Dr.) (2006), Health Care Management: The Contribution of Systems
Thinking.
Leedy, P.D and Ormrod, J.E. (2005), Practical Research – Planning and Design. 8th
ed. Prentice Hall.
Liker, J.K. (2004), The Toyota Way. 14 Management Principles from the World’s
Greatest Manufacturer. McGraw-Hill, New York.
Longman. (1999). Dictionary of Philosophy
Marr, B. (2008a), Strategic Performance Management in Government and Pubic
Sector Organisations – A Global Survey. Advanced Performance Institute.
Marr, B. (2009), Managing and Delivering Performance: How government, public
sector and non organisations can measure and manage what really matters. Elsevier
Ltd.
Marsh, D and Stoker, G. (2002), Theory and Method in Political Science. Palgrave
Macmillian 2nd ed.
Marshall, P. (1926 – To date ). www.thinkexist.com. Accessed 14th September 2009.
McCarthy, E. (1916 – 2005). www.thinkexist.com. Accessed 8th September 2009.
Mizner, W. (1876 – 1933). www.thinkexist.com. Accessed 14th September 2009.
Morris, T and Wood, S. (1991),’Testing the survey method: continuity and change in
British industrial relations’. Work Employment and Society. Vol.5, No.2, pp.259-282.
ODPM. (2005), Measurement of Output and Productivity of the Fire and Rescue
Service. A Conceptual Framework.
ODPM. (2007), A systematic Approach to Service Improvement. Evaluating Systems
Thinking in Housing. ODPM Publications.
Parris, M. (2008) Think twice about attacking ‘target culture’. The Times online.
www.timesonline.co.uk Accessed 1st march 2009.
Parmenter, D. (2007), Key Performance Indicators – Developing, Implementing and
Using Winning KPI’s. John Wiley and Sons, Inc, USA.
Pascal, B. (1623 – 1662). www.thinkexist.com. Accessed 14th September 2009.
Perri6. (1997), Holistic Government. Demos, London.
Planck, M. (1859 – 1947). www.thinkexist.com. Accessed 14th September 2009.
Pollitt, C. (1993), Managerialism and the public services. 2nd Ed. Oxford: Blackwell.
70
Porter, N. (1811 – 1892). www.thinkexist.com. Accessed 8th September 2009.
Pryor, M.G, Anderson, D, Toombs, L.A and Humphries, J.H. (2007) Strategic
Implementation as Core Competence. 5P’s Model. Journal of Management research.
Vol.1, No.1, April 2007.
Radnor, Z & McGuire, M. (2004), Performance management in the public sector:
fact or fiction? International Journal of Productivity and Performance Management.
Vol.53, No.3, 2004, pp. 245-260.
Radnor, Z.J. and Barnes, D. (2007), “Historical analysis of performance
measurement and management in operations management”. International Journal or
productivity and Performance Management. Vol. 56, No 5/6
Radnor, Z and Boaden, R. (2008), Lean in Public Services – Panacea or Paradox.
Public Sector. Public Money & Management February 2008, pp 3-7.
Radnor, Z and Walley, P. (2008), Learning to Walk Before We Try to Run: Adapting
Lean for the Public Sector. Public Money & Management February 2008, pp 13-20.
Radnor, Z. (2008), Muddled, massaging, manoeuvring or manipulated? A typology of
organisational gaming. International Journal of Productivity and Performance
Management. Vol.57, No.4, 20088, pp. 316-328.
Remenyi, D, Williams, B, Money, A and Swartz, E. (1998), Doing Research in
Business and Management: An Introduction to Process and Method. Sage, London.
Robson, C. (1997), Real World Research. London: Blackwell Publishers Ltd.
Rutherford, E. (1872 – 1937). www.thinkexist.com. Accessed 14th September 2009.
Sanderson, I. (2001), Performance Management, Evaluation and Learning in
Modern@ Local Government. Public Administration Vol. 79, No.2, pp 297-313.
Saunders, M, Lewis, P and Thornhill, A. (2007), Research Methods for Business
Studies 4th ed. Prentice Hall.
Seddon, J. (2008), Systems Thinking in the Public Sector – the failure of the reform
regime and the manifesto for a better way. Triarchy Press.
Seddon, J and Caulkin, S. (2007), Systems thinking, lean production and action
learning. Action Learning: Research and Practice Vol.4, No.1, April 2007, pp9-24.
Senge, P.M. (1990), The Fifth Discipline, The Art & Practice of The Learning
Organisation, Doubleday, New York.
Schram, D.L. (2003), Conceptualizing qualitative inquiry: Mindwork for fieldwork in
education and social sciences. Upper Saddle River, NJ: Merrill/Prentice Hall.
Shakespeare, W. (1564 – 1616). www.thinkexist.com. Accessed 14th September 2009.
71
Silverman, D. (2001) Doing Qualitative Research: A Practical Handbook. London:
Sage.
South Wales Fire and Rescue Service. (2007), Performance Management
Framework. Issue 1, May 2007.
South Wales Fire and Rescue Service. (2009), Risk Reduction Plan 2010-2011. June
2009.
South Wales Fire and Rescue Service. (2009), Caerphilly Fire Station Service Plan
2009/2010.
South Wales Fire and Rescue Service. (2009), http://www.southwales-fire.gov.uk
Accessed 2nd August 2009.
Taylor, A. (2008) Hitting the Target, Missing the Point? A Conference Report – The
Cumberland Lodge 27th Annual Conference. Friday April 25th to Sunday April 27th
2008.
Ten Have, S, Ten Have, W and Stevens, F with vand der Elst and Pol, Coyne, F.
(2003), Key Management Models: the management tools and practices that will
improve your business. Prentice Hall.
Thomas Johnson, H and Broms, A. (2000), Profit beyond measure: extraordinary
results through attention to work and people. The Free Press New York.
Toffler, A. (1928 – To date). www.thinkexist.com. Accessed 14th September 2009.
Torres, R.T., Preskill, H.S. and Piontek, M.E. (1996), Evaluation Strategies for
Communicating and Reporting: Enhancing Learning in Organisations. Sage,
Thousand Oaks, CA.
Trickery, K. (2003) When Best Value may not say what it means: a reflection on
measuring quality in public services in the United Kingdom. The Australian Library
Journal, April 2003.
Verbeeten, F.H.M. (2008), Performance management practices in public sector
organizations. Accounting, Auditing & Accountability Journal Vol. 21, N0.3, pp 427454.
Waitley, D. (1933 – To date). www.thinkexist.com. Accessed 14th September 2009.
Wales Audit Office. (2005), The Auditor General’s Code of Practice of Audit and
Inspection Practices. 11th May 2005.
Wheeler, D.J. (1993), Understanding Variation. The Key To Managing Chaos. SPC
Press, Inc, Tennessee.
Wisniewski, M. (2008), Performance management in the public sector: why is it so
difficult? Presentation at The Open University 12th March 2008.
72
Womack, J.P., Jones, D.T and Ross, D. (2007), The machine that changed the world
– How lean production revolutionized the global car wars. Simon Schuster: London.
Zemke, R. (2001), "Systems thinking - Looking at how systems really work can be
enlightening -- or a wake-up call." Training Vol.38, No.2, p p.40-47.
73
8.
Bibliography
Adcroft, A and Techman, J. (2008), Whose standards are they anyway? The need for
competitive spirit in public sector management. The Ashbridge Journal, Autumn 2008.
Anderson, S.E. and Gansneder, B.M. (1995), Using electronic mail surveys and
computer monitored data for studying computer mediated communications systems.
Social Science Computer Review. Vol.13, No.1, pp.33-46.
Arksey, H and Knight, P. (1999). Interviewing for social scientist. London: Sage.
Armstrong, M. and Baron, A. (2004) Managing performance: performance
management in action. London: Chartered Institute of Personnel and Development
Audit Commission. (1999), Performance measurement as a tool for modernising
government. December 1999.
Audit Commission. (2000), On Target: The practice of performance indicators.
Audit Commission. (2006), A manager’s guide to performance management: the
performance, management, measurement and information project (2nd ed).
Audit Commission. (2008), In the know: Using information to make better decisions:
a discussion paper.
Australian Public Service Commission. (2001), Performance management in the
APS: A strategic framework. Australian Government.
Austin, R.D. (1996), Measuring and Managing Performance in Organisations. Dorset
House Publishing, New York.
Barnes, J. (1979), Social Sciences, Privacy and Ethics. Harmondsworth: Penguin.
Behn, R.D. (2003). Why Measure Performance? Different Purposes Require Different
Measures. Public Administration Review. September/October 2003, Vol. 63, N0.5, pp.
586-606.
Bell, J. (1993), Doing Your Research Project – A guide for first-time researchers in
education and social science. 2nd ed. Open University Press, Buckingham.
Philadelphia.
Bevan, G and Hood, C. (2006), What’s Measured Is What Matters: Targets And
Gaming In The English Public Health Care System. Public Administration, Vol.84,
No.3, 2006, pp. 517-538.
Bolton, M. (2003), Public sector performance measurement: delivering greater
accountability. Work Study, Vol.52, N0.1, 2003, pp. 20-24.
74
Bosch, O.J.H, King, C.A, Herbohn, J.L, Russell, W.I and Smith, C.S. (2006).
Getting the Big Picture in Natural Resource Management – Systems Thinking as
‘Method’ for Scientists, Policy Makers and Other Stakeholders. Published online in
Wiley InterSciences.
Bowland, T & Fowler, A. (2000), A Systems perspective of performance management
in public sector organisation. International Journal of Public Sector Management,
Vol.13, No.5, 2000, pp 417-446.
Brewer, J and Hunter, A. (1989), Multimethod research: A Synthesis of Styles.
Newbury Park, CA.
Broadbent, J. (2007), If you Can’t Measure It, How Can You Manage It?
Management and Governance in Higher Educational Institutions. Public Money &
Management, June 2007.
Brown, S.E, and Lerch, D.C. (2007), Systems Thinking: A tool for Municipalities.
Laurel Nov 20, 2007. www.postcarboncities Accessed 12th June 2009.
Bryman, A.E. (1989), Quantity and Quality in Social Research. London: Sage.
Bryman, A.E. (1989), Research methods and organisation studies. London: Unwin
Hyman.
Bulmer, M. (ed) (1982), Social Research Ethics. London: Macmillan.
Bundred, S. (2006), Solutions to Silos: Joint Up Knowledge. Public Money &
Management, April 2006.
Cano, V. (2009). Reliability and Validity in Qualitative Research.
http://www.qmu.ac.uk/psych/Rtrek/study_notes/web/sn5.htm. Accessed1st August
2009.
Carrol, L, (1832 – 1898). Taken form Alice in Wonderland. www.thinkexist.com
accessed 14th September 2009.
Chapman, J. (2002), System Failure. Demos, London.
Chapman, J. (2004), System Failure. 2nd ed. Demos, London.
Checkland, P. (1999), Systems Thinking, Systems Practice. Including a 30 year
retrospective. John Wiley and Sons Ltd, Chichester.
Chinese Proverb. (Unknown). www.thinkexist.com. Accessed 14th September 2009.
Collier, P.M. (2006) In Search of Purpose and Priorities: Police Performance
Indicators in England and Wales. Public Money & management June 2006.
Collins, P.H. (1990), Black Feminist Thought: Knowledge, consciousness, and the
politics of empowerment. New York: Routledge.
75
Conti, T. (2006), Quality thinking and systems thinking. The TQM Magazine, Vol.18,
No.3 pp 297-309, 2006.
Clancy, J. (2008) Ministers urged to ‘back off’ – An important new study says students
in colleges are ‘achieving more but learning less’. guardian.co.uk, accessed 6th March
2009.
Crainer, S. (2000), The Management Century. A Critical Review of 20th Century
Thought & Practice. Jossey-Bass Publishers, San Francisco.
Creswell, J.W. (1998), Qualitative inquiry and research design: Choosing among five
traditions. Thousand Oaks, AC: Sage.
Connell, J, Lynch, C and Waring, P. (2001). Constraints, Compromises and Choice:
Comparing Three Qualitative Research Studies. The Qualitative Report, Vol.6 No.4,
December 2001.
Cumberland Lodge. (2008), Hitting the Target, Missing the Point? Conference
Report, April 25th – April 27th 2008.
Darwin, C.R. (1809 – 1882). www.thinkexist.com. Accessed 14th September 2009.
De Bono, E. (2000), New Thinking for the New Millennium. Penguin Books, England.
De Bruijn, H. (2007), Managing Performance in the Public Sector, 2nd ed. Routledge,
London.
Deming, W. Edwards. (1982), Out of a Crisis. MIT Press, Massachusetts.
Deming, W. Edwards. (1994), The New Economics - For Industry, Government,
Education. MIT Press, Massachusetts.
Denzin, N.K and Lincoln, Y. S. (1989), Collecting and Interpreting Qualitative
Materials. Thousand Oaks, CA: Sage.
Denzin, N.K and Lincoln, Y. S. (1994), Handbook of qualitative research. Sage
Publications Ltd.
Denzin, N.K and Lincoln, Y. S. (2000), 2nd Ed. Handbook of qualitative research.
Sage Publications Ltd.
Dey, I. (1993), Qualitative Data Analysis: A User-Friendly Guide for Social Scientist.
London: Routledge.
DoH. (2005), Taking Healthcare to the Patients: Transforming NHS Ambulance
Services (London)
Drew, K. (1980), Developing Interactions Inclusive Environment. Bethlehem: Lehigh
University Press.
76
Edwards, N. (2007), Stitched Up: The Highs and Lows of Life as an A & E Doctor.
Friday Books, London
Edwards, R. (2008) Police chief slams Government target culture.
www.telegraph.co.uk/news. Published: 12:12. AM, BST 09 Jun 2008. Accessed 1st
March 2009.
Einstein, A. (1879 – 1955). www.thinkexist.com. Accessed 14th September 2009.
Eisenhower, D.D. (1890 – 1969). www.thinkexist.com. Accessed 8th September 2009.
Elkan, R and Robinson, J. (1998), The use of targets to improve the performance of
health car providers: a discussion of government policy. British Journal of General
Practice, August 1998, pp 1515-15-18.
Ethin, C. (2000), Unleashing Intellectual Capital. Butterworth Heinmann, Boston.
Farnham, D. and Horton, S. (1993), Managing the new public services. Basingstoke:
Macmillian.
Finch, J. (1993), Great to Have Someone to Talk to: Ethics and Politics of
Interviewing Women. In: M. Hammersley (ed). Social Research: Philosophy, Politics
and Practice. London: Sage, pp166-180.
.
Flick, U. (2009), An Introduction to Qualitative Research. Sage: London.
Flynn, N. (1995) The future of public sector management. International Journal of
Public Sector management. Vol. 8, No.4, pp 59-67.
Frankfort-Nachmias, C and Nachmias, D. (1996), Research Science in Social
Sciences, 5th ed. New York: St Martin’s Press.
Gershon, P. (2004), Releasing resources to the front line. Independent Review of
Public Sector Efficiency HMSO.
Glaser, B.G and Strauss, A. (1967), The discovery of grounded theory. Chicago:
Aldine.
Grant, R.M. (2003), “Strategy planning in a turbulent environment: evidence from the
oil majors”, Strategic Management Journal, Vol. 24, No. 6, pp. 491-517.
Gray, D.J. (2005), A Multi-Method Investigation into the costs and into the Benefits of
Measuring Intellectual Capital Assets (unpublished Ph.D. thesis). Cranfield School of
Management, Cranfield.
Greiling, D. (2006), Performance measurement: a remedy for increasing the efficiency
of public services? International Journal of Productivity and Performance
Management. Vol. 55, No.6, pp 44-465.
77
Guba, E and Lincoln, Y. (1989), Fourth Generation Evaluation. Newbury Park, CA:
Sage.
Gummesson, E. (1991). Qualitative methods in management research. Thousand
oaks, CA: Sage Publications.
Hains, S.G. (2007), Strategic and Systems Thinking – The Winning Formula. Systems
Thinking Press.
Hakim, C. (1987), Research design: Strategies and Choices in the Design of Social
Research. London: Allen Unwin.
Hambrick, D.C and Fredrickson, J.W. (2005), Are you sure you have a strategy.
Academy of Management Executive, Vol. 19, No.4, pp 51-62.
Hamel, G. (1996), Strategy as revolution, Harvard Business Review, Vol. 74, No. 4,
pp. 69.
Hammersey, M. (1996), The relationship between qualitative and quantitative
research: Paradigm loyalty versus methodology eclecticism. In J.T.E Richardson (Ed),
Handbook of Qualitative research methods for psychology and social sciences. The
British Psychological Society, pp.159-174.
Handy, C.B. (1993), Understanding Organisations, 4th ed. Penguin.
Harrison, S. (1991), From here to perversity.
www.healthmatters.org.uk/issue7/perversity. Accessed 15th November 2008.
Hauser, J.R. and Katz, G. (1998), "Metrics: You Are What You Measure!" European
Management Journal, 16, 5, (October), 516-528
Heath, G and Radcliffe, J. (2007), Performance Measurement and the English
Ambulance Service. Public Money and Management, June 2007.
Hemmingway, E.M. (1899 – 1961). www.thinkexist.com. Accessed 14th September
2009.
Henn, M, Weinstein, M and Foard, N. (2006), A short introduction to social
research methods. Sage: London.
Hightower, C. (1923 – To date). www.thinkexist.com. Accessed 14th September
2009.
HM Government. (2007), Cutting bureaucracy for our public services. June 2007.
Holloway, J. (1999), “Managing Performance", in Rose, A, Lawton, A (Eds), Public
Services Management, Prentice-Hall, Harlow, pp238-259.
Hood, C. (1991), ‘A public management for all seasons’ Public Administration,
Vol.69, No.1, pp 3-19.
78
Hood, C. (2004), Gaming in Targetworld: The targets Approach to Managing British
Public Service. Public Administration Review, July/August 2006.
Hood. C. (2007), Public Service Management by Numbers: Why Does it Vary? Where
Has it Come From? What Are the Gaps and the Puzzles? Public Money &
Management. April 2007.
Hume, C and Wright, C. (2006), You Don’t Make a Pig Fatter by Weighing It –
Performance Management: The Experience of the Youth Justice Board. Public Money
& Management, June 2006.
Humphreys, J. (2004) The Vision Thing. Sloan Management Review Vol. 45, No.4,
pp 96.
Hussein, S.(1937 – 2006). www.thinkexist.com. Accessed 14th September 2009.
Jackson, M.C, Johnson, N and Seddon, J. (2007), Evaluating systems thinking in
housing. Journal of the Operational Research Society, November 2007
Jankowicz, A.D. (2005), Business Research Projects, 4th ed. London, Business Press
Thomson Learning.
Jarrar, Y. (2007), Measuring performance in the public sector: challenges and trends.
Measuring Business Excellence. Vol.11, No.4, 2007, pp. 4-8.
Jensen, M.C. and Meckling, W.H. (1996), “Specific and general knowledge, and
organisational structure”, in Meyers, P.S., Knowledge Management and
Organisational Design, Butterworth.
Johns, N and Lee-Ross, D. (1998), Research Methods in Service Industry
Management. Caswell, London.
Joiner, B.L. (1994), Fourth Generation Management; The New Business
Consciousness. McGraw-Hill: New York.
Juran, J.M. and Godfrey, A.B. (1999), Juran’s Quality Handbook (5th ed), McGrawHill Professional.
Kanji, G and Mora S.A. (2007), Performance Measurement and Business Excellence:
The Reinforcing Link for the Public Sector. Total Quality Management, Vol. 18, No.12, pp 49-56, January – March 2007.
Kant, I. (1724 – 1804). www.thinkexist.com. Accessed 8th September 2009.
Kennerley, M. (2004). Measuring performance in the public sector: Learning the
lessons. Cranfield, Bedfordshire, UK: Centre for Business Performance.
Kunc, M. (2008), Using systems thinking to enhance strategy maps. Management
Decision, Vol. 46, No. 5, 2008.
79
Leavitt, H.J. (1965), Applied organizational change in industry, in March, J.G (Eds),
Handbook of Organizations, Rand McNally, Chicago, IL.
Lebcir, R.M. (Dr.) (2006), Health Care Management: The Contribution of Systems
Thinking
Leedy, P.D and Ormrod, J.E. (2005), Practical Research – Planning and Design. 8th
ed. Prentice Hall.
Liker, J.K. (2004), The Toyota Way. 14 Management Principles from the World’s
Greatest Manufacturer. McGraw-Hill, New York.
Longman (1999). Dictionary of Philosophy
Marr, B. (2008a), Strategic Performance Management in Government and Pubic
Sector Organisations – A Global Survey. Advanced Performance Institute.
Marr, B. (2008b), Performance Management in the Public Sector - What’s the score?
www.ap-institute.com. Accessed 17th October 2008.
Marr, B. (2008c), Performance Anxiety. www.PublicFinance.co.uk. The Internet
Magazine for the Public Sector. Accessed 19th October 2008.
Marr, B. (2009), Managing and Delivering Performance: How government, public
sector and non organisations can measure and manage what really matters. Elsevier
Ltd.
Marsh, D and Stoker, G. (2002), Theory and Method in Political Science. Palgrave
Macmillian 2nd ed.
Marshall, P. (1926 – To date). www.thinkexist.com. Accessed 14th September 2009.
Mays, N. (2006), Use of Targets to Improve Health System Performance: English NHS
Experience and Implication for New Zealand.
McCarthy, E. (1916 – 2005). www.thinkexist.com. Accessed 8th September 2009.
McKevitt, D. and Lawton, A. (1996), The manager, the citizen, the politician and
performance measures: Public Money and Management, Vol. 16, No.3, pp 49-54.
McQuade, D. (2008), New Development: Leading Lean Action to transform Housing
Services. Public Money & management, February 2008.
Miles, M.B. and Huberman, A.M. (1994), Qualitative Data Analysis (2ed). Thousand
Oaks, CA, Sage.
Mizner, W. (1876 – 1933). www.thinkexist.com. Accessed 14th September 2009.
Morris, T and Wood, S. (1991),’Testing the survey method: continuity and change in
British industrial relations’. Work Employment and Society. Vol.5, No.2, pp.259-282.
80
Moser, C.A and Kalton, G. (1971). Survey Methods in Social Sciences. Aldershot:
Gower.
Nef. (2006), Hitting The Target, Missing The Point: How government regeneration
targets fail deprived areas. Measure What Matters.
ODPM. (2005), Measurement of Output and Productivity of the Fire and Rescue
Service. A Conceptual Framework.
ODPM. (2007), A systematic Approach to Service Improvement. Evaluating Systems
Thinking in Housing. ODPM Publications.
Ohno, T. (1988), Toyota Production System – Beyond Large-Scale Production.
Productivity Press, New York.
Palmer, A. (1993), Performance measurement in local government. Public Money and
Management. Vol.13, No.4, pp, 31-36.
Parris, M. (2008) Think twice about attacking ‘target culture’. The Times online.
www.timesonline.co.uk Accessed 1st march 2009.
Parmenter, D. (2007), Key Performance Indicators – Developing, Implementing and
Using Winning KPI’s. John Wiley and Sons, Inc, USA.
Pascal, B. (1623 – 1662). www.thinkexist.com. Accessed 14th September 2009.
Perri6. (1997), Holistic Government. Demos, London.
Peters, T. (1992), Liberation Management. New York: Knopf.
Pettigrew, A.M and Whipp, R. (1991), Managing Change for Competitive Success,
Blackwell, Oxford.
Pidd, M. (2005) Perversity in public service performance measurement. International
Journal of Productivity and Performance Management. Vol. 54, N0 5/6, pp 482-493.
Planck, M. (1859 – 1947). www.thinkexist.com. Accessed 14th September 2009.
Pollitt, C. (1993), Managerialism and the public services. 2nd Ed. Oxford: Blackwell.
Porter, N. (1811 – 1892). www.thinkexist.com. Accessed 8th September 2009.
Pryor, M.G, Anderson, D, Toombs, L.A and Humphries, J.H. (2007) Strategic
Implementation as Core Competence. 5P’s Model. Journal of Management research.
Vol.1, No.1, April 2007.
Pryor, M.G, White, J.C and Toombs, L.A. (1998). Strategic quality management, A
strategic systems approach to continuous improvement. Thomson Learning. Mason
OH.
81
Radnor, Z & McGuire, M. (2004), Performance management in the public sector:
fact or fiction? International Journal of Productivity and Performance Management.
Vol.53, No.3, 2004, pp. 245-260.
Radnor, Z.J. and Barnes, D. (2007), “Historical analysis of performance
measurement and management in operations management”. International Journal or
productivity and Performance Management. Vol. 56, No 5/6
Radnor, Z and Boaden, R. (2008), Lean in Public Services – Panacea or Paradox.
Public Sector. Public Money & Management February 2008, pp 3-7.
Radnor, Z and Walley, P. (2008), Learning to Walk Before We Try to Run: Adapting
Lean for the Public Sector. Public Money & Management February 2008, pp 13-20.
Radnor, Z. (2008), Muddled, massaging, manoeuvring or manipulated? A typology of
organisational gaming. International Journal of Productivity and Performance
Management. Vol.57, No.4, 20088, pp. 316-328.
Radnor, Z. (2008), Hitting the target and missing the point? Developing an
understanding of organisational gaming in Van Dooren, W and Van de Walle, S (ed.)
in Performance Information in the Public Sector: How it is used, Palgrave, Hampshire,
94.
Raimond, P. (1993), Management Projects: Design research and presentation.
Chapman and Hall, London.
Reed, G.E. (Col). (2006), Leadership and Systems Thinking. Defence AT&L: May –
June 2006.
Remenyi, D, Williams, B, Money, A and Swartz, E. (1998), Doing Research in
Business and Management: An Introduction to Process and Method. Sage, London.
Richards, S. (2003), Leadership – An Exploration of Issues Relating to Leadership in
the Public Sector Domain. Northern Ireland Review of Public Administration, Belfast.
Robson, C. (1997), Real World Research. London: Blackwell Publishers Ltd.
Rouse, J. (1993), "Resource and performance management in public service
organizations", in Isaac K, Painter, C, Barnes, C (Eds), Management in the Public
Sector, Challenge and Change, Chapham & Hall, London, pp.59-76.
Rutherford, E. (1872 – 1937). www.thinkexist.com. Accessed 14th September 2009.
Sanderson, I. (2001), Performance Management, Evaluation and Learning in
‘Modern’ Local Government. Public Administration Vol. 79, No.2, pp 297-313.
Saunders, M, Lewis, P and Thornhill, A. (2007), Research Methods for Business
Studies 4th ed. Prentice Hall.
Seale, C. (1998), Researching Society and Culture. London: Sage.
82
Seddon, J. (2005), Freedom from Command and Control – a better way to make the
work work. Vanguard Education Ltd, Buckinghamshire.
Seddon, J. (2007a). Public sector targets: doing less of the wrong thing is not doing
the right thing. A paper to Ruth Kelly, Minister for Communities and Local
Government. January 2nd 2007.
Seddon, J. (2007b). Citizen-centred services: a discussion of the aims and methods of
the White Paper. Vanguard Consulting.
Seddon, J. (2008), Systems Thinking in the Public Sector – the failure of the reform
regime and the manifesto for a better way. Triarchy Press.
Seddon, J and Caulkin, S. (2007), Systems thinking, lean production and action
learning. Action Learning: Research and Practice Vol.4, No.1, April 2007, pp9-24.
Senge, P.M. (1990), The Fifth Discipline, The Art & Practice of The Learning
Organisation, Doubleday, New York.
Schram, D.L. (2003), Conceptualizing qualitative inquiry: Mindwork for fieldwork in
education and social sciences. Upper Saddle River, NJ: Merrill/Prentice Hall.
Shakespeare, W. (1564 – 1616). www.thinkexist.com. Accessed 14th September 2009.
Silverman, D. (2007). A very short, fairly interesting and reasonably cheap book
about qualitative research London: Sage.
Silverman, D. (2001) Doing Qualitative Research: A Practical Handbook. London:
Sage.
Silverman, D. (1993). Interpretive qualitative data: Methods for analysing talk, text
and interaction. London: Sage.
Smith, A. (pseudonym of Goodman, G.J.W) (1973), An interview with Daniel
Yankelovich. Supermoney. Michael Joseph, London, p.286.
Smith, P. (1995), Performance Indicators and Outcome in the Public Sector. Public
Money and Management, December 1995.
South Wales Fire and Rescue Service. (2007), Performance Management
Framework. Issue 1, May 2007.
South Wales Fire and Rescue Service. (2009), Risk Reduction Plan 2010-2011. June
2009.
South Wales Fire and Rescue Service. (2009), Caerphilly Fire Station Service Plan
2009/2010.
South Wales Fire and Rescue Service. (2009), http://www.southwales-fire.gov.uk
Accessed 2nd August 2009.
83
Strauss, A.L and Corbin, J. (1990). Basics of qualitative research: Grounded theory
procedures and techniques. Newbury Park, CA: Sage.
Strauss, G and Whitfield, K. (1998), Research methods industrial relations. In K.
Whitfield and G. Strauss (Eds), Researching the world of work: Strategies and methods
in studying industrial relations. Ithaca: Cornell Press.
Taylor, A. (2008) Hitting the Target, Missing the Point? A Conference Report – The
Cumberland Lodge 27th Annual Conference. Friday April 25th to Sunday April 27th
2008.
Ten Have, S, Ten Have, W and Stevens, F with vand der Elst and Pol, Coyne, F.
(2003), Key Management Models: the management tools and practices that will
improve your business. Prentice Hall.
Thomas Johnson, H and Broms. (2000), Profit beyond measure: extraordinary
results through attention to work and people. The Free Press New York.
Toffler, A. (1928 – To date). www.thinkexist.com. Accessed 14th September 2009.
Torres, R.T., Preskill, H.S. and Piontek, M.E. (1996), Evaluation Strategies for
Communicating and Reporting: Enhancing Learning in Organisations. Sage,
Thousand Oaks, CA.
Trickery, K. (2003) When Best Value may not say what it means: a reflection on
measuring quality in public services in the United Kingdom. The Australian Library
Journal, April 2003.
University of Missouri. (2008), Criteria for Performance Excellence. The foundation
for the SMART Self-Assessment. The Curators of the University of Missouri – 4/2008.
Verbeeten, F.H.M. (2008), Performance management practices in public sector
organizations. Accounting, Auditing & Accountability Journal Vol. 21, N0.3, pp 427454.
Waitley, D. (1933 – To date). www.thinkexist.com. Accessed 14th September 2009.
Wales Audit Office. (2005), The Auditor General’s Code of Practice of Audit and
Inspection Practices. 11th May 2005.
Welsh Assembly Government. (2004), ‘Making the Connections: Delivering Better
Services for Wales’.
Welsh Assembly Government. (2007), The Wales Programme for Improvement –
Guide for Fire and Rescue Authorities in Wales.
Welsh Assembly Government. (2007), The Wales Programme for Improvement –
Guidance for Locale Authorities 2005. Circular 28/2005.
84
Welsh Assembly Government. (2009), Wales Fire and Rescue Service Circular WFRSC (09) 12. Performance Indicators for Fire and Rescue Authorities 2009-2009
onwards. March 2009.
Wheeler, D.J. (1993), Understanding Variation. The Key To Managing Chaos. SPC
Press, Inc, Tennessee.
Wisniewski, M. (2008), Performance management in the public sector: why is it so
difficult? Presentation at The Open University 12th March 2008.
Womack, J.P., Jones, D.T and Ross, D. (2007), The machine that changed the world
– How lean production revolutionized the global car wars. Simon Schuster: London.
Zemke, R. (2001), "Systems thinking - Looking at how systems really work can be
enlightening -- or a wake-up call." Training Vol.38, No.2, p p.40-47.
85
9.
List of Appendices
1
Relevance map to determine key research areas
83
2
Organisational diamond for performance management in the public
sector
84
3
South Wales Fire and Rescue Service Performance Management
Framework
85
4
Criteria for performance excellence framework.
86
5
Performance Management Questionnaire
87
6
Performance Management Interview Questions
92
7
Internal Memo Requesting Employee Participation
93
8
Presentation of Questionnaire Findings
94
9
The data analysis spiral
107
86
Relevance Map
Fire Service
Performance
measurement
Systems thinking
Performance
management in the
public sector
Government
targets
Hood
Senge
Deming
Performance
management
Public sector
Checkland
Key theorists and
academics
Ohno
Seddon
Marr
Radnor
Designed to determine key subject areas and academics in the field.
87
Organisational diamond for performance management in the
public sector
Strategy
Process
People
System
By Radnor (2004,) Performance management in the public sector: fact or fiction?
88
South Wales Fire and Rescue Service Performance
Management Framework
Risk Reduction Plan
Annual Improvement Plan
Vision
Core values
Budget Setting
Project Management
Corporate Risk register
Audit Action Database
Corporate Objectives
Directorate Plans
Department Plans
Personal Development & Review System
Performance Management Framework, Issue.1 May 2007 (SWFRS).
89
Wales Programme for Improvement
Mission
Criteria for performance excellence framework.
Organisational Profile:
Environment, Relationships, and Challenges
Strategic
Planning
Human
Resources
Business
Results
Leadership
Customer &
Markets
Process
Management
Information and Analysis
Provided by: The Curators of the University of Missouri – 4/2008
90
Performance Management Questionnaire
The following questionnaire intends to gather honest opinion from a wide range of
personnel throughout the Service on performance management system(s) currently
being used within South Wales Fire & Rescue Service (SWFRS). Your contribution will
provide a valuable insight into performance management, whilst helping the
organisation assess and monitor its effectiveness.
An essential element of continual improvement is to objectively question what we are
currently doing, in order to identify areas for improvement; therefore the collection of
thoughts and opinions of our people on such important issues are essential. Your views
will be highly valued and help improve future performance management systems.
Please complete and return by August 3rd to: Group Manager Steve Rossiter, Fire
Safety Department, FSHQ. sa-rossiter@southwales-fire.gov.uk
You
Name (Optional)
What is your current role/grade within the organisation?
In what directorate do you currently work?
Are you responsible for producing business plans?
Are you responsible for performance management?
I would be happy to take part in any further interviews on the subject of performance
management.
Yes
No
This questionnaire consists of a series of statements, please copy and paste a into the
most appropriate box according to your view on the statement.
91
Performance Management
1.
I have undergone training and development in performance management.
Strongly Agree
5
2.
Agree
4
Undecided
3
Disagree
2
Strongly Disagree
1
I believe that performance management within SWFRS is an effective
management tool
Strongly Agree
5
3.
Agree
4
Undecided
3
Disagree
2
Strongly Disagree
1
Performance management as applied to my current role is about learning and
improvement.
Strongly Agree
5
4.
Agree
4
Undecided
3
Disagree
2
Strongly Disagree
1
Performance management as applied to my current role is about control and
accountability.
Strongly Agree
5
Agree
4
Undecided
3
Disagree
2
Strongly Disagree
1
Business Planning Process
5.
I know exactly how my business plans impact on the strategic aims of SWFRS
Strongly Agree
5
6.
Agree
4
Strongly Agree
Agree
4
Strongly Agree
Undecided
3
Agree
4
Disagree
2
Strongly Disagree
1
Undecided
3
Disagree
2
Strongly Disagree
1
I revisit and monitor my business plans on a regular basis.
Strongly Agree
5
Strongly Disagree
1
My business plan is a working document, which directs my activities.
5
8.
Disagree
2
I feel that many of my business objectives are influenced by others.
5
7.
Undecided
3
Agree
4
Undecided
3
Disagree
2
92
Strongly Disagree
1
Targets
I am conscious that “what gets measured gets done”, when setting business
9.
objectives.
Strongly Agree
5
Agree
4
10.
Strongly Agree
Agree
4
Strongly Disagree
1
Undecided
3
Disagree
2
Strongly Disagree
1
Hitting the target do not necessarily lead to delivering a quality service.
Strongly Agree
5
Agree
4
12.
Disagree
2
Targets can cause unintentional consequences, such as cheating.
5
11.
Undecided
3
Undecided
3
Disagree
2
Strongly Disagree
1
I believe targets are important in managing my business plans.
Strongly Agree
5
Agree
4
Undecided
3
Disagree
2
Strongly Disagree
1
Communications
13.
I consult with both internal and external stakeholders when developing my
business plans
Strongly Agree
5
Agree
4
14.
Undecided
3
Disagree
2
Strongly Disagree
1
I feel that departments develop their business in isolation, thus causing
unnecessary duplication of work.
Strongly Agree
5
Agree
4
15.
Undecided
3
Disagree
2
Strongly Disagree
1
I speak to customers in order to gauge their satisfaction of the service they
receive/ we provide.
Strongly Agree
5
Agree
4
16.
Undecided
3
Disagree
2
Strongly Disagree
1
A collaborative approach to business planning produces greater organisational
improvement.
Strongly Agree
5
Agree
4
Undecided
3
Disagree
2
93
Strongly Disagree
1
Measuring and Monitoring
17.
Measuring core business activities is difficult.
Strongly Agree
5
Agree
4
18.
Strongly Agree
Agree
4
Strongly Disagree
1
Undecided
3
Disagree
2
Strongly Disagree
1
I meet regularly with my team to discuss progress of our business plan
Strongly Agree
5
Agree
4
20.
Disagree
2
I collecting and analyse relevant data in order to deliver my business plan.
5
19.
Undecided
3
Undecided
3
Disagree
2
Strongly Disagree
1
Before I set business objectives, I understand what measurements are required in
order to monitor progress
Strongly Agree
5
Agree
4
Undecided
3
Disagree
2
Strongly Disagree
1
Learning & Development
21.
Performance management is about learning and feeding that learning back into
the decision making process.
Strongly Agree
5
Agree
4
22.
Undecided
3
Disagree
2
Strongly Disagree
1
The feedback I get from managing performance allows me to identify training
needs for my staff.
Strongly Agree
5
Agree
4
23.
Undecided
3
Disagree
2
Strongly Disagree
1
Personal Development Review’s (PDR’s) are a fundamental element of any
performance management system.
Strongly Agree
5
Agree
4
24.
Disagree
Strongly Disagree
1
Disagree
Strongly Disagree
1
2
Change is about improvement through learning.
Strongly Agree
5
Undecided
3
Agree
4
Undecided
3
2
94
If you like to elaborate on any of the above statements, please feel free to include them
in the additional comments section below.
Additional Comments
I would like to take this opportunity in thanking you for your honest views and
assistance in compiling such much needed information.
95
Performance Management Interview Questions
Organisational Strategy
What do you think is the organisations purpose, why are we in
existence?
Where do you see CFS sits in terms of organisational aims?
Performance Management/Business Planning
What is your opinion on the current business planning process?
Give me your appreciation of the purpose of performance management and where it fits
within the overall strategic plan?
Do you think we have an effective PM system?
Target Setting
What is your view on target setting?
In relation to our strategic aim, what is your view on the statements – “hitting the target
missing the point” and “what gets measured gets done”?
What is your understanding of gaming in relation to targets and do you believe this goes
on within SWFRS and if so why?
Systems Thinking
In your opinion do you think SWFRS operates as a holistic organisation or do you thing
a silo mentality exists within SWFRS?
What are your thoughts on communicating with other stakeholders when developing
your business plan?
What is your view on engaging with the people of SW to ensure we are delivering a
good service, do you think this is happening and if not why?
Learning & Development
Where do you see L&D fits within the PM framework?
Within any PM framework, there should be a feedback loop, where we learn and modify
our decisions through our experiences, what are your views on this?
How does PDR fit within you PM system and your business planning process?
96
Internal Memo
To:
All Business Planners
My Ref:
SAR/PMS
Copy:
CFO. A Marles
Tel Ext:
2700
Your Ref:
PMQ/6.09
Date:
June 2009
From:
GM Steve Rossiter
Fire Safety
Fire Service Headquarters
Llantrisant
Re: Request to participate in the attached Performance Management Survey
Dear Colleague,
Attached is a questionnaire relating to your personal opinions regarding performance
management systems within SWFRS. Completion of this will allow me to gather vital
thoughts and opinions from a wide range of personnel. All responses are highly valued
and will be analysed and fed into the Brigade Management Team to support the
development and continual improvement of SWFRSs system, processes and people.
Please could you complete the questionnaire and return to myself at FSHQ by the date
indicated? Authorisation from the Chief Fire Officer and your line manager has been
sought and all responses will be treated with total confidentiality. The findings will be
used to support my current study area where upon completion they will be destroyed.
Should you wish not to take part, would you please respond as such, so as to allow me
to manage the return statistics?
Can I take this opportunity to thank you for you cooperation and honesty in completing
this questionnaire and if you have any queries relating to this matter or would like
further involvement please do not hesitate to contact me on. 01443 232700.
Kind regards
Steve Rossiter
97
Presentation of Questionnaire Findings
Question
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Strongly
agree
No
%
3
6.7
5
11.1
6
13.3
7
15.6
10
22.2
25
55.6
10
22.2
12
26.7
7
15.6
10
22.2
24
53.3
5
11.1
3
6.7
8
17.8
7
15.6
12
26.7
2
4.4
6
13.3
9
20
6
13.3
8
17.8
3
6.7
8
17.8
8
17.8
Agree
No
23
21
18
28
22
18
28
30
26
27
18
25
20
19
15
28
18
33
34
29
26
28
13
25
%
51.4
46.7
40
62.2
48.9
40
62.2
66.7
57.8
60
40
55.6
44.4
42.2
33.3
62.2
40
73.3
75.6
64.4
57.8
62.2
28.9
55.6
Undecided
No
3
11
6
3
7
1
6
1
4
4
1
9
1
13
6
5
12
5
0
8
5
9
13
7
%
6.7
24.2
13.3
6.7
15.6
2.2
13.3
2.2
8.9
8.9
2.2
20
2.2
28.9
13.3
11.1
26.7
11.1
0
17.8
11.1
20
28.9
15.6
Disagree
No
12
8
15
6
4
1
1
2
8
4
2
6
20
5
17
0
10
1
2
2
6
5
11
5
Table illustrating questionnaire returns
98
%
26.7
17.8
33.3
13.3
8.9
2.2
2.2
4.4
17.8
8.9
4.4
13.3
44.4
11.1
37.8
0
22.2
2.2
4.4
4.4
13.3
11.1
24.2
11.1
Strongly
disagree
No
%
4
8.9
0
0
0
0
1
2.2
2
4.4
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1
2.2
0
0
0
0
0
0
3
6.7
0
0
0
0
0
0
0
0
0
0
0
0
0
0
1.
I have undergone training and development in performance management.
Strongly agree
3
Agree
23
Undecided
3
12
Disagree
Strongly disagree
4
0
2.
5
10
15
20
25
I believe that performance management within SWFRS is an effective
management tool
Strongly agree
5
Agree
21
Undeided
11
8
Disagree
Strongly disagree
0
0
5
10
99
15
20
25
3.
Performance management as applied to my current role is about learning and
improvement.
Strongly agree
6
Agree
18
Undecided
6
15
Disagree
Strongly disagree
0
0
4.
2
4
6
8
10
12
14
16
18
20
Performance management as applied to my current role is about control and
accountability.
Stronlgy agree
7
Agree
28
Undecided
3
6
Disagree
Strongl disagree
1
0
5
10
15
100
20
25
30
5.
I know exactly how my business plans impact on the strategic aims of SWFRS
Strongly agree
10
Agree
22
Undecided
7
4
Disagree
Strongly disagree
2
0
6.
5
10
15
20
25
I feel that many of my business objectives are influenced by others.
Strongly agree
25
Agree
18
Undecided
1
Disagree
1
Strongly disagree
0
0
5
10
15
101
20
25
30
7.
My business plan is a working document, which directs my activities.
Strongly agree
10
Agree
28
Undecided
6
1
Disagree
Strongly disagree
0
0
8.
5
10
15
20
25
30
I revisit and monitor my business plans on a regular basis.
Strongly agree
12
Agree
30
Undecided
1
2
Disagree
Strongly disagree
0
0
5
10
15
102
20
25
30
35
I am conscious that “what gets measured gets done”, when setting business
9.
objectives.
Strongly agree
7
Agree
26
Undecided
4
8
Disagree
Strongly disagree
0
0
10.
5
10
15
20
25
30
Targets can cause unintentional consequences, such as cheating.
Strongly agree
10
Agree
27
Undecided
4
Disagree
4
Strongly disagree
0
0
5
10
15
103
20
25
30
11.
Hitting the target do not necessarily lead to delivering a quality service.
Strongly agree
24
Agree
18
Undecided
1
2
Disagree
Strongly disagree
0
0
12.
5
10
15
20
25
30
I believe targets are important in managing my business plans.
Strongly agree
5
Agree
25
Undecided
9
6
Disagree
Strongly disagree
0
0
5
10
15
104
20
25
30
13.
I consult with both internal and external stakeholders when developing my
business plans
Strongly agree
3
Agree
20
Undecided
1
20
Disagree
Strongly disagree
1
0
14.
5
10
15
20
25
I feel that departments develop their business in isolation, thus causing
unnecessary duplication of work.
Strongly agree
8
Agree
19
Undecided
13
5
Disagree
Strongly disagree
0
0
2
4
6
8
105
10
12
14
16
18
20
15.
I speak to customers in order to gauge their satisfaction of the service they
receive/ we provide.
Strongly agree
7
Agree
15
Undecided
6
17
Disagree
Strongly disagree
0
0
16.
2
4
6
8
10
12
14
16
18
A collaborative approach to business planning produces greater organisational
improvement.
Strongly agree
12
Agree
28
Undecided
5
Disagree
0
Strongly disagree
0
0
5
10
15
106
20
25
30
17.
Measuring core business activities is difficult.
Strongly agree
2
Agree
18
Undecided
12
10
Disagree
Strongly disagree
3
0
18.
2
4
6
8
10
12
14
16
18
20
I collecting and analyse relevant data in order to deliver my business plan.
Strongly agree
6
Agree
33
Undecided
5
1
Disagree
Strongly disagree
0
0
5
10
15
107
20
25
30
35
19.
I meet regularly with my team to discuss progress of our business plan
Srongly agree
9
Agree
Undecided
34
0
2
Disagree
Strongly disagree
0
0
20.
5
10
15
20
25
30
35
40
Before I set business objectives, I understand what measurements are required in
order to monitor progress
Strongly agree
6
Agree
29
Undecided
8
2
Disagree
Strongly disagree
0
0
5
10
15
108
20
25
30
35
21.
Performance management is about learning and feeding that learning back into
the decision making process.
Strongly agree
8
Agree
26
Undecided
5
6
Disagree
Strongly disagree
0
0
22.
5
10
15
20
25
30
The feedback I get from managing performance allows me to identify training
needs for my staff.
Strongly agree
3
Agree
28
Undecided
9
5
Disagree
Strongly disagree
0
0
5
10
15
109
20
25
30
23.
Personal Development Review’s (PDR’s) are a fundamental element of any
performance management system.
Strongly agree
8
Agree
13
Undecided
13
11
Disagree
Strongly disagree
0
0
24.
2
4
6
8
10
12
14
Change is about improvement through learning.
Strongly agree
8
Agree
25
Undecided
7
5
Disagree
Strongly disagree
0
0
5
10
15
110
20
25
30
The Data Analysis Spiral
The Final Report
Synthesis
Offering hypothesis or propositions
Constructing tables and diagrams
Classification
Grouping the data into categories or themes
Finding meanings in the data
Perusal
Getting an overall “sense” of the data
Jotting down preliminary interpretations
Organisation
Filing / creating database
Breaking large units into smaller ones
The Raw Data
Creswell (1998)
Qualitative inquiry and research design: Choosing among five traditions
111