Evaluation of the National Infarct Angioplasty Project Pilots
(NIAPP): summary of proposed research.
Background, aims and objectives.
The Department of Health is currently funding a number of pilot sites as part of the
National Infarct Angioplasty Project Pilots to test the feasibility of implementing a
countrywide primary angioplasty service for patients with acute ST-elevation
mycocardial infarction (STEMI). We aim to evaluate the pilot sites and address the five
elements of the brief produced by the NHS Service Delivery and Organisation (SDO)
R&D Programme.
The specific objectives will be:
 To gain an in depth understanding of patient and carer experience of receiving
care at the pilot sites, and measure differences in satisfaction between the pilot
sites and sites providing standard, thrombolysis-based care.
 To assess the workforce implications of setting up an angioplasty service at the
pilot sites, and measure the effect of providing the service upon the angioplasty
team, support staff, and related staff groups.
 To describe the models of service delivery established at the pilot sites in terms
of their setting (geography, population, transport and communications), structure
(hospitals, referral networks, transfer and access points) and components (staff,
facilities, and equipment).
 To explore implementation and feasibility issues by describing the processes
involved in establishing primary angioplasty (particularly the development of
teams and leadership roles), identifying facilitating factors and barriers to
implementation, and assessing the implications of establishing primary
angioplasty for cardiology and other services.
 To compare the costs and outcomes of providing angioplasty and thrombolysis,
and estimate the incremental cost-effectiveness of a comprehensive primary
angioplasty service, compared to standard, thrombolysis-based care, for the
different organisational service models adopted in the pilot sites.
Plan of investigation
We plan to compare the pilot (intervention) sites to four control sites that have not
formally established a primary angioplasty service. Evaluation will take a whole-system
approach. We anticipate that even pilot sites that successfully implement 24-hour
primary angioplasty will still use thrombolysis for at least a residual proportion of
patients. Likewise, control sites that do not develop a primary angioplasty service may
still use angioplasty opportunistically for some patients arriving during working hours.
Our evaluation will compare the overall effect on all patients with AMI at each site, rather
than focussing upon those who received the intended or principal service.
Patient and carer experience
We propose to use a phased approach of developing an instrument specific to the
patient and carer experience of primary angioplasty. We will use the instrument to
describe experiences in a range of pilot and control sites.
Phase 1 will involve two sites with designated special responsibility for the evaluation of
the patient and carer experience. Qualitative interviews post-discharge will be based on
a maximum-diversity sample of patients and carers attending the pilot sites to include
different ages, genders, arrival routes and living distance from pilot sites. We will
interview between 15 and 25 people in their own homes using the critical incident
technique to determine positive and negative aspects of the process, including issues
such as communication and quality of care. We will also use these interviews to assess
the feasibility of using a postal questionnaire in phase 2, specifically addressing the
issue of identification of carers.
The interviews will be used alongside other published instruments to develop a NIAPPspecific instrument to measure patient and carer satisfaction. A generic questionnaire
will also be developed for patients in the context of emergency cardiac care.
Questionnaires will be reviewed and piloted by the project consumer group.
Phase 2 will involve a questionnaire developed from the interviews as well as other
generic instruments for assessing patient satisfaction. We will send a postal
questionnaire one week after hospital admission to every patient with ST-elevation AMI,
and their main carer, attending four of the pilot sites over a nine-month period, ensuring
that the samples are suitably screened for severe adverse events. The patient
questionnaire will include the NIAP-specific and generic instruments, the carer
questionnaire will include the NIAP-specific instrument only. We expect a minimum of 75
patients from each site in this time period, totalling at least 300 patients, and 200 carers
(given that not all patients may have carers). We will aim for a response rate of 75% with
two reminders.
To provide data for comparison, we will send questionnaires to 300 patients with STelevation AMI in four control hospitals. We will only post the generic instrument to
patients at control sites for direct comparison with pilot-site patients. We will aim to
select the four pilot sites with the highest proportion of patients receiving angioplasty so
that this aspect of the evaluation maximises the contrast between intervention and
control care. This sample size will have 80% power to detect a 0.75 point change on
each 5-point patient satisfaction question (alpha=0.05).
Patients will be identified by staff within the cardiology department at the hospitals.
Patients will need to consent to their postal details being sent to researchers at the
University of Sheffield. Questionnaires and reminders will then be administered by
researchers at the University.
Phase 3 will involve up to 20 qualitative interviews with patients and carers from the two
original pilot sites to expand upon results from the survey and identify possible solutions
to issues identified. Patients will be identified from the questionnaire respondents.
Workforce implications, implementation and feasibility
We will use a mixture of quantitative and qualitative methods to address the questions
outlined in the brief. Staff involved with providing angioplasty services will be identified
by prior contact with the Trust.
Six of the pilot sites will be selected to take part in the workforce assessment. It is
anticipated that specific material will be produced for this study. These will be developed
on sites working closely with the teams who are collecting economic data to produce a
holistic model of investigation, and will involve close collaboration with researchers
working at two other sites to develop the patient and carer satisfaction evaluation. Once
piloted these materials and methods will be developed, edited and then used with the
other intervention sites.
We will administer self-complete questionnaires to staff who work with patients that have
treatment for AMI. We will send the questionnaire to relevant staff by internal mail
suggesting completion during work time. The questionnaire will examine the changing
aspects of their work role, flexible working and managing a 24 hour service, support and
training, teamworking and work factors (e.g., autonomy, role clarity, role conflict,
influence in decision-making, peer and leader support).
Questionnaire data will be complemented by the use of focus groups and semistructured interviews with key stakeholders to examine the impact of the angioplasty
service on the workforce and explore the wider implications of providing an emergency
angioplasty service, such as cardiologists not being available for the general medical
rota and changes to emergency referral patterns. Additional focus groups will examine
the impact of angioplasty on other associated workforces (e.g. radiography, ambulance
services and the emergency department). These focus groups and interviews will also
explore issues of implementing the service and factors perceived to have facilitated or
opposed effective delivery. Observational data will be collected, and where this is not
possible it will be collected retrospectively through discussion of cases after treatment.
A one-day site visit will be used as the principal vehicle to arrange and conduct
interviews and focus groups. We will try to arrange this around an elective PCI session
to minimize any inconvenience to staff. The visit will be supplemented, as necessary by
telephone interviews, if staff are unavailable at the time of the visit. The visit will also be
used to undertake the specialist teamworking interviews and ethnographic observation
(or where not possible retrospective case discussion) that will address the
implementation and feasibility issues.
Teamworking interviews will examine the exact nature of the service delivery team, how
team members work together and how their specific roles integrate. The model for the
site visit will be piloted within the first six months of the project (phase one) to enable
any practical adjustments to be made. It is anticipated data will be collected on 6 sites
during the period September 2006 to March 2007.
Data will be analysed using quantitative and qualitative methods. This material will be
presented in an anonymous manner and integrated into a final report looking at various
aspects of pilot project evaluation. A summary of this report and access to the final
document will be made available to all who take part in the study
Description of models of service delivery
We will use data from routine sources (NIAPP, MINAP, BCIS, CCAD, HES), a telephone
survey and site visits to describe the models of service delivery developed by the pilot
sites. We will seek information on the following parameters: Service setting: urban or
rural; area covered; catchment population (number, age and gender, coronary heart
disease morbidity and mortality rates, ethnicity and social deprivation); transport and
communications networks. Service overview: participating hospitals; teaching status;
referral networks between hospitals; ambulance services; other prehospital care; access
route to the service. Service infrastructure: staff numbers, grades, profession and
specialties; interventional cardiology skills; rostering and on-call arrangements; facilities
and equipment; bed availability (general, coronary care and intensive care); cardiac
surgery services. Service activity levels: number of AMI treated (prehospital
thrombolysis, in-hospital thrombolysis and angioplasty), numbers of emergency and
elective admissions, revascularisation procedures, and cardiac operations performed.
Economic evaluation
Mark Sculpher and colleagues at the University of York have already developed a costeffectiveness model to compare angioplasty to thrombolysis. It considers costeffectiveness from the perspective of the NHS and uses data from the literature and
routine sources to provide an estimate of the incremental cost per QALY. The analysis
also looks at the variation in cost-effectiveness according to the time delay to
angioplasty. The value of the model for the pilot evaluation is that it can be used to
explore how the cost-effectiveness of primary angioplasty might vary between the
different types of service configuration seen in the pilot centres. In particular, it can use
centre-specific data on costs and times to reperfusion to assess variation in costeffectiveness across different models of service delivery.
We will increase the robustness and scope of the cost-effectiveness analysis by
undertaking a microcosting study to measure the true costs of providing angioplasty in
different models of service delivery and check the validity and completeness of MINAP
data. Working alongside the workforce and feasibility evaluations, we will identify factors
that determine the true cost of providing angioplasty.
We will collect resource use data from four intervention and four control sites for 60
patients at each site, giving a total of 240 patients in each arm. At each of these sites,
data will be collected on staff costs, equipment, consumables, drugs and procedures
associated with the intervention (angioplasty or thrombolysis). Care will be taken to
stratify patients by day and night time admission in order to capture any differences that
may arise due to the timing of the admission.
The resource use data will be valued using local unit costs where available or national
unit costs. We will use, as far as possible, centre-specific cost data, estimated in
collaboration with local finance departments using a standardised protocol. Staffing
costs will be assessed in conjunction with cath lab management. The staffing
arrangements will be costed according to whether procedures take place in or out of
hours and the specific trust arrangements regarding out of hours staff including
sensitivity analyses that assume different costs of knock on cancellations from out of
hours work.
It is anticipated that much of the data can be collected retrospectively using routinely
collected data (MINAP / NIAPP / BCIS datasets). Where required, we will supplement
this with detailed examination of patient and ambulance notes. Data will be gathered on
these patients individual characteristics, including time to needle/balloon, for use as
covariates in the cost analysis.
We also intend to ask staff to complete a short questionnaire immediately after
completing a sample of PCIs in order to record staff involved and their perception of
knock on effects the following day. An additional adjustment that must be included is to
identify the proportion of patients that are referred as ST- elevation AMI and therefore
may trigger the out of hours team but are subsequently found not to be “true” cases.
All the new data collected will be synthesized with the data in the existing costeffectiveness model and an incremental cost per QALY will be estimated. Probabilistic
sensitivity analysis will be used to address the uncertainty in the input parameters of the
cost-effectiveness analysis. The decision uncertainty will be described using a costeffectiveness acceptability curve.
We will develop a regression model using the patient-level cost data and MINAP data to
determine how much of the variation in cost can be explained by differences in the
models of service delivery used, having controlled for case mix. Findings from the
workforce and feasibility evaluations will be used to explain how different models of
service delivery may be associated with different costs. The different costs for each
potential model of service delivery will then be used in the cost-effectiveness model to
explore the relative cost-effectiveness of the different approaches to service delivery.
Summary of research elements and timescales.
Element of
research
No. sites
involved
What is involved for each site?
Timescales
Patient & Carer
evaluation:
phase 1
(development
interviews)
2-3
intervention
sites
Recruiting around 10 patients and
their carers at each site for a faceto-face interview.
May-August
2006
Patient & Carer
evaluation:
phase 2 (postal
questionnaires)
4 intervention
sites and 4
control sites
Recruiting @75 patients and their
carers (where possible) in each site
to be sent a questionnaire.
Checking patient status for any
patients to whom we send a
reminder
October
/November
2006 – July
2007
Patient & Carer
evaluation:
phase 3
(exploratory
interviews)
Any of the
sites involved
in phase 2.
Patients and carers will be recruited
from the questionnaire study. Staff
may be asked to check the status of
a patient before us contacting them.
November
2006-July
2007
Description of
service models
All
intervention
and control
sites
A researcher will undertake site
visits and telephone questionnaire
with PI/key staff.
March 2006January 2007
Workforce
evaluation: staff
survey
6 intervention
sites
Staff working with STEMI patients
(in variety of roles) will be asked to
complete a questionnaire around
workforce issues.
September
2006- April
2007
Workforce,
implementation
and feasibility:
focus groups &
interviews
6 intervention
sites
Some of the above staff and other
staff in roles affected by PPCI may
be asked to participate in focus
groups or semi-structured interviews
to discuss teamworking,
implementation and feasibility
issues.
September
2006- April
2007
Economic
evaluation:
identification of
cost data
4 intervention
sites and 4
control sites
A health economist will discuss
economic issues with finance and
cardiology staff.
August 2006
– June 2007
Economic
evaluation:
survey of staff
time.
4 intervention
sites and 4
control sites
We will require access to routine
data sources (MINAP, CCAD,
NIAPP etc)
We will require access to routine
data and patient notes for 60
patients at each site.
For a subset of patients, staff will be
asked to complete a short survey
about time spent on PCI procedures
and consequences of on-call.
August 2006
– June 2007