Invitation to Tender
Evaluation of PQASSO in Wales Project
Sue Blackmore
Interim PQASSO Programme Lead
Charities Evaluation Services
June 2014
sueb@ces-vol.org.uk
020 70789379
Charities Evaluation Services, 4 Coldbath Square, London EC1R 5HL
Registered charity number 803602 in England and Wales and SCO40937 in Scotland.
A company limited by guarantee registered in England & Wales. Company registration no: 2510318
Contents
1.
Summary
1
2.
Background information
2.1. About CES
2.2. About PQASSO
2.3. PQASSO in Wales
1
1
2
3
3.
The project outline
3.1. Comprehensive evaluation project plan
3.2. Set up project monitoring systems
3.3. Evaluation
3.4. Reporting
3.5. Timeframe
3
4
4
5
5
6
4.
Consultant requirements
6
5.
Application requirements and process
5.1. Project proposal
5.2. Application Process
7
7
8
6.
Appendices
8
PQASSO in Wales Evaluation, May 2014
1.
Summary
Charities Evaluation Services (CES) is one of the leading charities advising the
voluntary sector about quality management, evaluation and impact. Since 1997 roughly
17,000 charities have used CES’ quality standards – PQASSO.
In 2012, a working group convened by the Big Lottery Fund Wales and involving the
Welsh Government, Wales Audit Office, Charity Commission, county voluntary councils
(CVCs) and Wales Council for Voluntary Action (WCVA), explored potential options for
developing a distinctive Welsh initiative to support good governance within the charitable
sector. The group agreed that the preferred approach was to be based on Charities
Evaluation Services’ PQASSO quality standards.
PQASSO is a CES flagship product. Currently in its 3rd edition, it is the leading off-theshelf quality system in use within the UK voluntary sector. The 3rd edition of PQASSO
was developed in 2008 and, although the document is still relevant and useful to VCOs,
it is now due for revision. The aim is to launch a revised version with an online data
management tool in March 2015 and to translate and tailor this for the Welsh audience.
CES is inviting proposals from consultants wishing to evaluate the project. The project
will include the following strands of work:



2.
A comprehensive project plan.
Setting up a project monitoring system.
Evaluation of the programme.
Background information
2.1. About CES
CES supports third sector organisations and their funders so they can focus on
doing what they do best, and achieve more for the causes they serve. Our work
directly helps organisations become more effective and better equipped to
demonstrate their value.
CES was established in 1990, as a joint funder and government initiative, following
calls for improved practice in the sector. New government policy laid out a
framework for third sector reporting mechanisms and set expectations that
charities should in future behave more like businesses. Research at that time
showed that this new emphasis required better monitoring and reporting practice in
the sector. CES’ vision was, and remains, a more effective third sector with strong
and credible mechanisms for accounting for its work and learning from it.
PQASSO in Wales Evaluation, June 2014
1
Each year we directly work with over 1,000 organisations to develop performance
management and to improve quality in their work. We do this by providing training,
support, advice, tools and independent evaluations.
CES influences policy and develops practice using knowledge directly gained
through our work with diverse organisations and our understanding of their needs.
The practical free guides on our website are downloaded over 30,000 times each
year.
2.2. About PQASSO
PQASSO is the UK award-winning quality standard used by about 17,000 voluntary and
community organisations. The PQASSO standards have been developed specifically by
and for the third sector, addressing the needs of the sector.
PQASSO covers all aspects of an organisation, from governance to service delivery and
monitoring outcomes. PQASSO’s flexibility means it can be used by all types of third
sector organisations, including charities, social enterprises, community interest
companies and community groups, and organisations of all sizes.
PQASSO offers a flexible approach to quality which allows an organisation to work at its
own pace as it is self-assessed through the 12 quality areas. Three levels of
achievement provide an incremental approach to quality improvement.
Organisations can also apply for the PQASSO Quality Mark, a nationally recognised
award endorsed by the Charity Commission that offers users as well as commissioners
and funders external verification of the quality and credibility of an organisation.
The 1st edition of PQASSO was produced by CES in 1997 as a workpack and was
based on the Kids Club Network’s pioneering quality assurance scheme Aiming High.
The 2nd edition of PQASSO was produced in 2000 and involved a major overhaul of the
standards and criteria of the 1st edition. A 3rd edition was produced in 2008. A PQASSO
CD ROM was produced in 2001 to complement the workpack. The CD ROM essentially
acts as a database for storing evidence and improvement plans.
CES also has experience of tailoring and adapting PQASSO. We have worked with
membership and umbrella organisations to adapt PQASSO under licence to meet their
specific needs.
CES runs a range of PQASSO training courses, the most significant being the two-day
Implementing PQASSO course to assist groups in implementing PQASSO. Using a
‘cascade’ or ‘train the trainer’ approach CES also runs a PQASSO licensed mentor
scheme to train staff from infrastructure organisations and local government, as well as
independent consultants, to assist them in providing their own PQASSO-related training
and support to VCOs.
2
PQASSO in Wales Evaluation, June 2014
Please see the CES website and enclosed literature for more information about
PQASSO and related products and services.
2.3. PQASSO in Wales
In reviewing options for quality standards for the third sector in Wales, the Big Lottery
Fund specifically chose to use PQASSO because it is holistic – addressing all aspects of
quality management – and is tried and tested.
Research shows that PQASSO is the lead quality standard in the UK third sector, and
that in some parts of the UK, local authorities and other commissioning bodies now cite
PQASSO as a recognised system alongside other non-third sector specific standards
such as ISO 9001. Yet its take up in Wales remains low: sales in Wales are estimated at
around 100 organisations. There are only five PQASSO mentors based in Wales and
two Peer reviewers who are trained and commissioned by CES to carry out external
assessments. Six organisations have achieved the PQASSO Quality Mark, and a further
10 have applied and are going through the process.
In recent years, there has seen increasing demand from our PQASSO customers for an
online tool. This is from a wide range of organisations, but particularly those operating
from more than one site. An online tool will enable multiple users to work with PQASSO
at the same time, doing self-evaluation work, and storing evidence simultaneously.
There is also a particular need for an online product in relation to PQASSO in Wales to
facilitate the choice of language, which is less practical and cost effective in the current
paper-based workbook. The development of an online rather than a paper-based tool
provides an opportunity to review and refresh the standards themselves.
The project outline
3.
CES wishes to contract independent consultants to carry out an evaluation of the
PQASSO in Wales project.
The evaluation will be both formative and summative, and will have two main aims:
1. To assess the implementation of the programme, providing reporting and
management information against performance indicators. This will help to keep
the programme on track, improve delivery and provide learning about how to
ensure that the initiative is sustainable.
2. To assess against programme outcomes, evaluating the benefits to third sector
organisations in the short, medium and longer term.
Evaluation questions will include:

How far does PQASSO for Wales meet the specific requirements of the Wales
environment and Welsh stakeholders?
PQASSO in Wales Evaluation, June 2014
3





How functional is the online tool?
What are the drivers and barriers to implementing PQASSO, using the online
tool, and to achieving the Quality Mark?
To what extent has PQASSO for Wales reached smaller and hard-to-reach
groups, and has the bursary scheme helped?
To what extent has the cascade model been appropriate in Wales and helped to
develop good practice and the adoption of quality standards?
What is required to maintain sustainability of the initiative?
In completing this work the consultants will be expected to carry out the following tasks:
1. Develop a comprehensive evaluation project plan
2. Set up project monitoring systems
3. Undertake evaluation.
3.1. Comprehensive evaluation project plan
In consultation with CES, the consultants will be expected to develop a comprehensive
project plan showing how s/he will undertake the evaluation within the specified
timeframe and budget. As well as showing how s/he will undertake the main tasks of the
project, the plan must also include:



An explanation of evaluation methodology.
Fully-costed and time-bound milestones showing when particular pieces of work
will be completed (and against which the consultant will be paid).
Details of days and rates for each evaluation team member.
3.2. Set up project monitoring systems
The consultants will firstly review the outcome and output indicators for the project. They
will then work closely with the PQASSO Programme Lead to set up a monitoring system.
This may include, but is not limited to, the following:




4
Centralised tracking and collation of:
o key stakeholder contacts (eg, local authorities, influencers, commissioners,
networks)
o organisation contacts (eg, contact type, organisation size, location,
organisation type)
o achievement of milestones.
Contacts/events tracking tool – to be completed monthly by Mentors.
Baseline data collection tools, to be completed by/for:
o stakeholder contacts
o PQASSO users at initial take-up of PQASSO/Mentor support.
Knowledge/awareness tracking tool – for use at PQASSO events.
PQASSO in Wales Evaluation, June 2014

Web-based feedback tools for PQASSO online.
3.3. Evaluation
We welcome ideas from the consultants about which evaluation methods to use, but
they may include the following:







Questionnaire/interviews with PQASSO Mentors.
Interviews with members of the implementation team.
Survey/short telephone interviews with stakeholders.
Outcomes questionnaire, to be sent to PQASSO users.
Case studies - these will also serve to promote and communicate the benefits of
PQASSO.
Long-term tracking tool.
Face-to-face/telephone/email interviews with key stakeholders.
3.4. Reporting
We will require a final report to be completed by the end of December 2018. End of year
reports will also need to be submitted by December 2014, 2015 and 2016.
The end of year 1 report will report on Year 1 implementation including development of
PQASSO for Wales; achievement of first year milestones, including scoping,
recommendations for online tool and early development; lessons learned and
recommendations for the second year’s implementation.
The end of year 2 report will include year 2 milestones; recommendations for third year
implementation and future sustainability of PQASSO for Wales; and short-term
outcomes for PQASSO users and Mentors/Peer Reviewers.
The end of year 3 report should include achievement against performance indicators;
outcomes for third sector organisations using PQASSO, those attaining the PQM and
mentors and third sector infrastructure; lessons learned from the introduction of
PQASSO for Wales and from the introduction, use and functioning of an online tool; and
extent of wider awareness of PQASSO by stakeholders in Wales and perception of its
benefits.
PQASSO in Wales Evaluation, June 2014
5
The end of year 5 report will include longer-term outcomes for PQASSO users and the
sustainability of PQASSO for Wales.
We are planning to publish the year 3 and 5 reports.
3.5. Timeframe
CES expects the evaluation to be fully completed by the end of December 2018. To
meet this deadline, the consultants are expected to set fully-costed and time-bound
milestones as part of their project plan (which will need to be agreed by CES).
Throughout the project the consultants will be expected to meet the agreed milestones
on time, and to regularly discuss their progress with CES. The consultants will be paid
against the successful completion of these milestones.
4.
Consultant requirements
CES will expect the consultants to:
 Work from their own premises (although there is no office space available at CES for
the consultants to be permanently based, there may be availability from time to time).
 Be prepared to travel to Wales, spending nights away from home when necessary.
 Work independently, while at the same time keeping CES regularly updated as to
their progress through email and telephone correspondence.
 Report on the project’s progress quarterly, either by meeting with CES or by email.
 Keep within the project plan’s timeframe (as agreed in advance with CES).
 Work flexibly where necessary, responding to unexpected obstacles and
opportunities which could arise during the course of the project. Any potential
deviations from the project plan and/or time-frame will need to be discussed in
advance with CES.
CES will support the consultants by:
 Facilitating access to CES stakeholder contacts.
 Clear communication and reporting through the Programme Lead and Programme
Officer.
A sum of £32,400 (inclusive of VAT) is available for consultant fees. This is broken down
over five years as follows (please note there is no budget for 2017):
 2014: £6,600
 2015: £7,800
 2016: £12,000
 2018: £6,000.
Further budget will be available to cover project costs (focus groups, questionnaires,
travel, accommodation etc).
6
PQASSO in Wales Evaluation, June 2014
5.
Application requirements and process
5.1. Project proposal
CES is inviting applicants to submit a full project proposal with detailed costings.
CES will accept joint project proposals from more than one consultant working together
on the project. However, such proposals must identify one lead project manager who will
take responsibility for the overall project and will be responsible for liaising with CES.
CES will not directly manage two or more different consultants working on separate
pieces of work. In addition, workplans must highlight days and rates for each team
member.
Please demonstrate clearly your suitability for the work addressing:

How you will meet the tasks and requirements set out in the above Project Outline.

Your understanding, skills and experience in carrying out evaluations, and your
experience of evaluating quality standards.

Your understanding of the voluntary sector in Wales and its current policy
environment.

Clear and concise written and oral communication skills.

An excellent track-record of successfully managing similar high-level evaluation
projects.
Please provide full details of the following information:

The individuals who will be working on the project, with their qualifications and
experience.

How different consultants working on different aspects of the project will ensure their
work is fully coordinated (this is only relevant in the case of joint-proposals from two
or more consultants).

Predicted timings and costings related to the various stages of the project (please
make clear your proposed start date).

Any conflict of interest they may have with the work of CES or this project. Applicants
who believe they may have a conflict of interest would be advised to discuss this
informally with Sue Blackmore (sueb@ces-vol.org.uk) before submitting their
application.

Email and telephone numbers of two recent referees who can confirm suitability and
reliability (preferably from similar work or another large consultancy project) and who
can be contacted at short notice.
PQASSO in Wales Evaluation, June 2014
7
Please complete your submission by signing a declaration stating that all information
provided is, to the best of your knowledge, true.
5.2. Application Process
Applicants should submit proposals by email to Sue Blackmore (sueb@ces-vol.org.uk)
no later than 26 June 2014.
Following a short-listing process, successful applicants will be invited to attend an
interview at CES, 4 Coldbath Square, London EC1R 5HL on 10 July 2014.
It is anticipated that this interview will involve a short presentation by the consultant on
their project proposal, followed up by questions from the panel.
A starting date for the successful applicant will be discussed at interview.
Appendices
6.


8
PQASSO brochure: http://www.ces-vol.org.uk/PQASSO/pqasso-the-basics
PQASSO case studies: http://www.ces-vol.org.uk/examples-of-work/examplework-charities/examples-pqasso-quality-charities.htm
PQASSO in Wales Evaluation, June 2014