Intelligent Targets 22 April 09
Welsh Assembly Government
A Standardized Improvement Method and Consistent
Evaluation of Intelligent Targets in NHS Wales
Purpose
 To propose a standardised improvement method for implementing
Intelligent Targets based on experience and evidence
 To propose an evaluation framework for the Intelligent Targets
proposed by the four core groups
 To set out a monitoring strategy for ensuring progress in
implementation of the targets
The rationale for a change of approach heralded by Intelligent Targets is
supported by this quote from Professor Trisha Greenhalgh1:
“Getting evidence into practice” is an intuitively appealing notion but at an
operational level it can be a can of worms. The rationalistic, linear model of
‘produce evidence-based guideline  disseminate guideline  implement guideline
in clinical practice  evaluate outcome’ often proves impossible to apply at the
bedside, in the outpatient clinic or around the policymaking table. Research shows
that the failure of evidence to flow neatly into practice cannot be overcome
merely by addressing the ‘knowledge gap’ or the ‘behaviour gap’ amongst
clinicians, generating and distributing National Service Frameworks, or
restructuring healthcare organisations. The need for whole-systems, multi-level
change is widely recognised. But work on innovation in healthcare is often heavy on
management buzz-phrases such as “leadership”, “engagement”, “culture change”,
“accountability” and so on but light on theory-driven approaches to these
challenges. The time is overdue for planners and policymakers to join forces with
researchers so that new interdisciplinary conceptual and theoretical models can
illuminate and inform the complex challenges involved in spreading and sustaining
best practice.
Scope
Intelligent Targets are intended to deliver improvement through change in
the delivery of health services in Wales. The distinctive features of the
approach are:
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
1
Intelligent Targets 22 April 09
1. Clinicians will be engaged in the process of setting targets so that
they represent a consensus view of what should be achieved and in
improvement work at a local level
2. Targets will be based on evidence and linked to outcomes as far as
possible so as to ensure that they do not result in perverse
outcomes.
The additional criteria drawn from the terms of reference are:
3. The targets should be capable of demonstrating change at a high
(presumably Health Board and National) level
4. The development process should be sustainable and transferable
5. Development work for 2009-11 will focus on:
a.
b.
c.
d.
Cardiac Services
Stroke
Unscheduled Care Services
Mental Health and well-being
6. The targets developed must address the complete care pathway,
recognising the different roles that primary, secondary and tertiary
and rehabilitation services play in delivering safe, high quality and
effective services which achieve the optimum outcome for the
people using them.
The current restructuring of the NHS in Wales creates an unprecedented
opportunity for delivering the benefits across sectors previously divided by
the purchaser/provider split.
Achieving change
Greenhalgh2 has summarised the characteristics of an effective change in
a healthcare setting.
a. It must have clear relative advantage
b. It must have compatibility with the user’s values and ways of
working
c. Complexity must be minimised
d. Users will adopt more readily if innovations allow trialability
e. There must be observability, that is it must be seen to deliver
benefit
f. Reinvention is the propensity for local adaptation
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
2
Intelligent Targets 22 April 09
National Service Frameworks do not, of themselves, drive change
because they fail almost all of these characteristics. Previous work on
integrated care pathways has not delivered substantial change in Wales
largely because it too has not met these criteria: in particular, ICPs have
often been complex rather than focussing on priorities. However, there is
good experience of change building in Wales from a number of initiatives.
The WAG Quality Improvement Plan required the development of an
evidence based clinical change model appropriate to the needs of Wales.
The result – the 1000 Lives Campaign – is based on:
 The Model for Improvement including clear outcomes, evidence
based interventions, good process measures and small cycle
change to encourage trialability and local adaptation
 A secure platform of evidence support (through NPHS/WCH) and
Welsh professional guidance to ensure fit with national context
(through the Healthcare Improvement Faculty)
 A national campaign underpinned by a breakthrough collaborative
project structure to support teams in achieving changes.
The Health Foundation funded Safer Patients Initiative (SPI) has also
delivered fundamental change in service delivery using the Model for
Improvement and evidence based interventions.
On a smaller scale and using the same principles, the Welsh Critical Care
Improvement Programme has achieved major and sustained changes in
clinical outcome by using evidence based care bundles, local monitoring
and peer supported training through a collaborative structure.
All of the programmes listed above have been characterised by high
levels of clinician involvement and enthusiasm.
A Consistent Approach to Clinical Change in Wales
Setting targets will not be enough. Their usefulness will be gauged by
how well they drive improvement. Wales has accumulated extensive
experience of clinical improvement work and there is a basis to propose a
consistent approach to the Intelligent Targets work. That approach can be
based on what has worked, capitalise on the reusable improvement
infrastructure which has developed and reinforce the philosophy and skills
sets which are required to sustain and spread change.
Such a consistency (and simplification) of method will deliver other
benefits. It will allow the Steering Group to monitor progress of the four
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
3
Intelligent Targets 22 April 09
work strands and to understand whether relative success in one or other
area is due to the design of the targets or to their implementation. It will
support continued learning about what transferrable approach to advocate
in the future.
That simplicity will also allow those responsible for parallel initiatives to
plan appropriate links with Intelligent Targets. Examples include the Map
of Medicine, System Level Measures, evolution of Chairs’ Dashboards
and Organisational Development frameworks,
Thus it is proposed that a consistent approach is owned by the Steering
Group and that all work strands follow the same approach. The approach
should include the characteristics set out in Table 1 below.
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
4
Intelligent Targets 22 April 09
Characteristic
Agreed care pathway*
based on evidence and
Wales consensus
Driver diagram** based on
agreed care pathway and
featuring key interventions
Outcome measure
preferably available at local
and timely level and
building to system level
Process measures linked
to interventions and
outcome, simplified using
Care Bundle approach
where appropriate
Use of PDSAs to support
local adaptation
Rationale
Clear relative advantage
Precedent
1000 Lives, WCCIP,
Stroke services
Complexity must be
minimised
1000 Lives, Stroke
services, SPI
Observability
Clear relative advantage
WCCIP, Stroke services
Complexity must be
minimised
Reinvention
Observability
1000 Lives, WCCIP,
Stroke services, SPI
Trialability
Reinvention
1000 Lives, WCCIP,
Stroke services, SPI
Data tool for use by local
teams and reportable
nationally
Trialability
Observability
Stroke services, 1000
Lives, SPI
Integration with existing
networks, learning sets,
groups etc
A stretch target as
opposed to a fixed target
Compatibility
1000 Lives, WCCIP,
Stroke services
Clear relative advantage
1000 Lives, WCCIP,
Stroke services, SPI
*Needs to be consistent. It is suggested that the core and reference group are the mechanism to
achieve professional consensus. Each core group will need also to be assured that the care pathway is
evidence based.
**An example of a driver diagram is given at Appendix A.
Table 1. Characteristics of the shared approach to Intelligent Targets.
There is a direct read across to the brief set for the Core Groups but the
approach will impose some additional tasks to further standardise the
products. For ease of reference, that brief required the following
deliverables:





Mapped care pathways;
Identifying a series of care pathway improvement measures and
targets;
Identifying a series of clinical quality and outcome measures;
A set of high level system measures for the health care system
initially; and
A documented development process for the production of future
intelligent targets post 2009/2010
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
5
Intelligent Targets 22 April 09
The two lists match well. The main differences are that the table proposes
that all Care Pathways are simplified down to a Driver Diagram to reduce
complexity and highlight key interventions. Driver diagrams were first
developed by Joe McCannon and colleagues at the Institute of Healthcare
Improvement to support the 100,000 Lives Campaign3. The diagrams are
logical, simple and well received by busy clinicians. The table also
envisages that process and outcome measures will be scalable to the
national level. Put the other way: system level measures need to be
applied to the local context.
A further point of comment is the stretch target. This is an ambitious goal
which will require real changes in performance and approach. It should
match consumer expectations. A conventional target which requires a set
and usually arbitrary target (90% achievement, 50% increase etc) lacks
face validity (it incorporates failure) and produces the wrong conversation.
For those who achieve it, ambition evaporates; those who don’t are
performance managed until they say they have achieved it. Nowhere is
there a conversation about improvement. That contrasts with
conversations from SPI for example where the ambition is to avoid all
failures.
Evaluation of Progress
It is suggested that the Steering Group use the same progress monitoring
criteria as are used in the 1000 Lives Campaign. These are based on
work by Dr Sarah Fraser4 and focus on spread. They use three domains:
uptake, process change and outcome change. They all rely on clarity
about the value of the denominator. For example, if 14 sites offer stroke
services in Wales, then the measures of uptake are all x/14.
Uptake indicators should respond immediately a programme is started
and should remain in place. They are effectively organisational
preconditions and will need to be monitored both by local teams and by
the Centre. Process changes will take longer to appear but are the heart
of Intelligent Targets. These are the care processes which, according to
evidence from experimental conditions, will deliver improved outcomes
Outcome change typically takes even longer: a year is not unusual,
sometimes more. Outcome measures are not targets and cannot be
performance managed. They are consequences of improvements.
Examples of possible progress indicators are set out in Table 2 below.
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
6
Intelligent Targets 22 April 09
Domain
Uptake (organisational conditions)
Process change (Intelligent Targets)
Outcome
process)
change
(consequence
Examples of indicators
Identified management lead
Identified clinical champion
Intranet sign up
Data submitted
Teams trained
Local communication strategy in place
Bundle compliance
Uptake of new practice (specific to driver
diagram)
of Reduced morbidity
Reduced mortality
Reduced dependency
Reduced hospital stay
Table 2. Examples of Progress Indicators.
Taking this forward
It is suggested that the Steering Group use the characteristics in Table I
to review the submissions from the four Core Groups and seek further
revision if appropriate. It is then suggested that each of the work streams
is required to collect monthly data from all sites in Wales using the
framework set out in Table 2 and that these are summarised and reported
to the Steering Group on a quarterly basis.
Creating the Climate
Beyond evaluation, publicity and consensus building have been critical
success factors in all of the programmes listed in this paper. That is
because, even when simplified and standardised, the change process
requires work to gather measures and to achieve change. It is suggested
that the Steering Group give urgent thought to this aspect of the Intelligent
Targets work. At present, the work is not widely known about and still less
understood by staff in the NHS in Wales. The shared methodology
approach will make this vast communication task more manageable.
The Steering Group will also want to consider how the Core Groups’ role
will be continued as the need for professional oversight and revision of
pathways and driver diagrams evolve over time.
Conclusion
The Assembly has instigated work which, if carried out successfully,
offers the opportunity to deliver real change in clinical services in Wales
by harnessing the commitment of clinical staff. On the basis of research
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
7
Intelligent Targets 22 April 09
evidence and recent experience in Wales, it can adopt a shared model for
this work which will allow it to monitor the success of individual
workstreams and to capitalise on experience, education and training
which already exist in the service.
There is a further need to launch a comprehensive communication drive
to ensure understanding and ownership of the Intelligent Targets work in
Wales.
Recommendations
The steering group is asked to agree
 Adoption of a consistent, evidence based approach to implementing
and evaluating Intelligent Targets across the four workstreams
 Use of the proposed characteristics to assess the proposed targets
 Use of the proposed progress indicators to shape quarterly
evaluation reports from the core groups
 Urgent consideration of a communication strategy to support the
development of Intelligent Target work in NHS Wales.
 Consideration of the Core Groups’ role in the future management of
the Intelligent Targets process
Bibliography
1. Greenhalgh T (2007) Personal communication
2. Greenhalgh T, Robert, G & Bate, P. (2004) How to Spread Good Ideas: A systematic
review of the literature on diffusion, dissemination and sustainability of innovations in
health service delivery and organisation. Report for the National Co-ordinating Centre
for NHS Service Delivery and Organisation R & D (NCCSDO) [Available online]
www.sdo.lshtm.ac.uk/files/project/38-final-report.pdf
3. www.IHI.org/Programs/Campaign/100kCampaignOverviewArchive.htm
4. Fraser SW (2007) Undressing the elephant. Why good practice doesn’t spread in
healthcare www.lulu.com 100pp
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
8
Intelligent Targets 22 April 09
Appendix A. An example of a driver diagram
Improving Critical Care
(Trusts Only)
Gwella Gofal Critigol
Content Area
Drivers
Reduce complications from:

Ventilators

Central lines

Severe Sepsis

Healthcare Associated
Infections
Rapid response to acutely ill patients
(Shared with medical /surgical
improvement and leadership)
Improving
Critical
Care
Provide patients and family driven care
Interventions
Reliable




processes of care:
Ventilator management
Central line management
Identification and treatment of severe
sepsis
Hand hygiene
Reliable processes are contained in the NICE
guidance (50) on identification and treatment of
acute illness and include:

Establishment of and training for a whole
hospital early warning system

Development of and training in graded
risk based response to acute illness

Audit process and outcomes

Inclusion of Trust board management,
referring medical teams and ward staff in
audit feedback process
Processes

Inclusion of patient/ public representation
on local critical care improvement team

Integrate patient/family into improvement
work

Promote open communication among team
and family
Create an environment of collaboration
and culture of safety
Processes

Multi disciplinary rounds and daily goal
setting

Ensure staff have knowledge and expertise
in improvement work

Ensure communication and collaboration
within a multi disciplinary team

Appropriate infrastructure: intensivist led
model
Involve Leadership in safety
Integrate leadership into improvement efforts
Standardized method and consistent evaluation, IT Steering Group, 22 April 2009
Alan Willson
9