Masterclass in Writing RfPB Grant
Applications
Carole Fogg
Sophie Hyndman
Helen Lloyd
28th January 2011
Presented January 2011; updated and adapted for web,
January 2012
Welcome!
Introductions
Aim and Objectives of the Day
RDS South Central:
Research Design Support for NHS
and Social Care Researchers
Who are we and what can we do for you?
Some of our A-Team:
Multi-disciplinary team:
•
•
•
•
•
•
•
Statisticians
Clinical Trialists
Epidemiologists
Qualitative
Researchers
Health Economists
Patient/Public Experts
Health Psychologists
What can the RDS SC do to help you?
For all aspects of grant preparation, advice on:
•
•
•
•
Sources of funding
Literature search and review
Developing and refining research objectives
Study design, including health economics & statistics
•
•
Planning teams and budgets
Patient and Public Involvement (PPI)
•
•
•
Research Advisor, with access to specific experts as required
Research Design Advisory Panels
Potential for collaboration
Communications: e-mail, newsletter, website
Training: research master classes
Question: In what way is a research grant application like a general
election campaign?
Answer:
They both require the right balance between
Aim
To support researchers in developing successful
research grant proposals, specifically, Research for
Patient Benefit applications
Objectives
To provide:
• An overview of funding applications, and how they relate to research
governance and ethics applications
• An overview of NIHR funding schemes and the purpose and scope
of RfPB
• An understanding of what reviewers expect to see in an RfPB grant
application
• Some guidance on identifying what makes ‘good’ and ‘bad’ grant
applications
• An introduction to the structure and content of the RfPB form
• An understanding of the importance of patient and public
involvement in RfPB research
• An understanding of methodological issues important to reviewers
• An understanding of the resources and time required to undertake a
research project
• An introduction to other sources of research support
Part 1
Research in the NHS: Funding
Applications in Context
Important initiatives
•
•
•
Good Clinical Practice (GCP) and the UK
Medicines for Human Use (Clinical Trials)
Regulations 2004
Responsibilities and accountabilities for
clinical trials of investigational medicinal
products
Research Governance Framework for Health
and Social Care (2005)
Informed by above, but for research more
generally
Best Research for Best Health (2006)
Infrastructure and systems
Salvatore Vuono.
http://www.freedigitalphotos.net/i
mages/view_photog.php?photog
id=659
The purpose of these initiatives
Research Governance
•
•
•
•
•
•
To protect the safety, dignity, rights and wellbeing of research participants
To promote useful, ethical, valid, safe and affordable research
To prevent futile, unethical, invalid, dangerous and extravagant research
To provide research standards
To define responsibilities and accountabilities
To set out delivery and monitoring systems to ensure standards
Best Research for Best Health
•
•
•
•
To make the UK a recognized centre of healthcare research excellence
by providing a good research infrastructure
To build research capacity
To fund research to improve health and social care
To improve and manage knowledge and information systems to simplify
research processes and maximise usefulness of research
The result of these initiatives
Among other things…
•
•
A range of funding schemes for health research
A system of research approvals to ensure that research
is well regulated and ethical
What are the main ‘hoops’
to jump through?
All research carried out in the NHS requires:
• A sponsor
• Ethics approval
• R&D approvals
Most research will require:
• Funding
1 ) Hula hoop moment from Circus Smirkus
performance 2006. http://smirkus.orgImage
provided courtesy of Wikipedia. Permission is granted
to copy, distribute and/or modify this document under
the terms of the GNU Free Documentation License,
Version 1.2
Some research will require
• Other approvals (e.g. Clinical Trial Authorisation)
• Honorary contracts or research passports
Where do I start?
Sayan Samana
http://www.freedigitalphotos.net/images/view_photog.ph
p?photogid=2067
Steps to take1....
* IRAS
1. May not always be in this order!
Image from Office Online
Steps (1) Write a research protocol and (3)
Apply for funding
Q. What is the difference between a research protocol and a research
grant application?
A.
A research protocol is the definitive document that will inform the
process of the research. Some R&D offices will insist on a GCP
approved protocol, and provide templates for this (regardless of
whether the project is a Clinical Trial of an Investigational Medical
Product (CTIMP) or not)
The research protocol can be used as the basis for applying for ethics
and funding, and there will be an overlap, but these latter documents
will have different emphases, so it is not simply a matter of cutting
and pasting
See http://www.oxfordradcliffe.nhs.uk/research/downloads.asp
Downloads 16 & 17 for GCP approved templates
Image from Office Online
Step (2) Find a sponsor
Q. Why do you need a sponsor?
A. You need an organisation to be accountable for the
research
i.e.:
•
•
•
•
•
Responsibility for the initiation and overall management, including quality
assurance (research protocol, team, research environment)
Responsibility for the finances associated with the study
Ensure all approvals obtained, e.g. ethics, clinical trial authorisation
Arrangements for handling of any Investigational Medicinal Products
(IMP)
Ensure Good Clinical Practice (GCP), monitoring and reporting
(including safety reporting)
Image from Office Online
Q. Who can be a sponsor?
A.Usually:
•
•
•
•
Chief Investigator’s employing institution, e.g. a university or NHS
trust
A funding organisation
Where research is for training purposes, the research supervisor (on
behalf of their employer)
Private sector e.g. pharmaceutical industry
Step (4) National Research Ethics Committee
(NRES) approval
http://www.nres.npsa.nhs.uk/
Role:
•
•
Protect rights, safety, dignity and well-being of research participants
Facilitate and promote ethical research of potential benefit to
participants, science and society.
Advice on: information sheets and consent forms, GCP, procedures,
use of personal data etc.
Apply: online via Integrated Research Application System (IRAS)
https://www.myresearchproject.org.uk/
Image from Office Online
Step (5) Local trust research and
development office (R&D) approval
Role:
•
•
•
To give permission to conduct research in the site
If sponsor, to undertake that role; if hosting research (sponsored by
others), to undertake audit of a proportion of studies
To ensure all relevant research-related contracts are in place
Advice available:
•
Variable and dependent on personnel/ time: NRES applications,
financial aspects of study from trust perspective, more general research
advice
Image from Office Online
When do you meet them?
•
Variable and dependent on personnel/ time: for high risk/ drug
related/ device related studies, as soon as you have a protocol;
some offices may be interested in these and other studies at the
development stage.
Apply:
1. Via IRAS Coordinated System for gaining NHS Permission (CSP)
for research qualifying for the NIHR Clinical Research Network
Portfolio
2. Directly to R&D Office for all non portfolio research.
http://www.crncc.nihr.ac.uk/about_us/processes/portfolio
Step (6) Other approvals
IRAS captures the information needed by:
• Administration of Radioactive Substances Advisory Committee
(ARSAC)
• Gene Therapy Advisory Committee (GTAC)
• Medicines and Healthcare products Regulatory Agency (MHRA)
• Ministry of Justice (National Offender Management Service)
• HSC research offices
• NRES / NHS / HSC Research Ethics Committees
• National Information Governance Board for Health and Social Care
(NIGB).
• Social Care Research Ethics Committee
Image from Office Online
Step (7) Research Passports and Honorary
Contracts
Q. What is a research passport?
A. A set of signatures and checks from your employing organisation
required prior to issuing an honorary contract or letter of access.
Q. What is an honorary contract?
A. A contract with an NHS trust for a researcher whose employer is not that
NHS trust.
Q. What is a letter of access?
A. A letter permitting a researcher whose employer is not that NHS trust into
the trust
Image from Office Online
Q. When are honorary contracts needed?
A. When: (a) the research is hosted in the NHS and (b) the researcher has no
contract with the NHS and (c) the researcher’s activities in the NHS site will
involve interacting with individual patients in a way that has a direct bearing
on the quality of their care.
Q. When are letters of access needed?
B. When (a) and (b) above are present, but not (c)
Always check with trusts what level of access they
consider you need
http://www.nihr.ac.uk/systems/Pages/systems_research_passports.aspx
Q. Why are honorary contracts needed?
1. Patients in receipt of research procedures come under the NHS duty of
care and the NHS indemnity scheme applies to the researchers and their
activities.
2. The researcher is accountable to the NHS organisation for their activities,
explicit supervision arrangements are in place, and the researcher is made
aware of the relevant NHS policies with which they must comply in their
activities.
3. The researcher’s substantive employer understands the research activities
and has undertaken all the necessary checks in relation to the researcher’s
suitability to carry out those activities.
Q. Where do I apply to get an honorary contract?
A. Usually Human Resources Department of NHS Trust where you wish to do
the work.
Steps you MIGHT have to take:
(1) Involving a Comprehensive Local
Research Network (CLRN)
Can work with investigators to:
•
•
•
•
•
Provide NHS Support Costs for research projects
Facilitate permissions
Help with recruitment of sites and patients
Help with data collection
General advice on the research process in their area of expertise.
http://www.ukcrn.org.uk/index/networks/comprehensive/clrns.html
Image from Office Online
(2) Involving a Clinical Trials Unit (CTU)
Can work with investigators to:
•
•
Develop new trials (design, collaboration, funding, communication
with Clinical Research Network, regulatory and governance issues)
Manage funded trials (questionnaire design, set-up, recruitment,
pharmacovigilance, data management, monitoring, analyses, report
preparation)
http://www.ukcrc-ctu.org.uk
Image from Office Online
Part 2 – Research for Patient Benefit
(RfPB) & The Importance of Sc pe
Image courtesy of Wikimedia commons (Public Domain)
NIHR Research Programmes
•
•
•
•
•
•
•
Research for Patient Benefit (RfPB)
Programme Grants for Applied Research (PGfAR)
Invention for Innovation (i4i)
Health Technology Assessment (HTA)
Efficacy and Mechanism Evaluation (EME)
Public Health Research (PHR)
Health Services and Delivery Research (NIHR HS&DR)
-
formerly 2 programmes: Health Services Research (HSR) &
Service Delivery and Organisation (SDO)
http://www.ccf.nihr.ac.uk/RfPB/Pages/home.aspx/
Research for Patient Benefit offers…
The opportunity to conduct research that:
•
•
•
•
•
•
•
arises from daily practice in the NHS
is developed between NHS and other partners
is open to topic and method
shows tangible health and health care benefits for patients
amplifies the patient voice
that encourages collaboration
closes the loop between research and practice – will be
implemented or drive change
Aims / sc pe
•
•
To support projects in health services and public health
research
Qualitative, quantitative or mixed method research that
will:
– study the way NHS services are provided and used
– evaluate whether interventions are effective and
provide value for money
– examine whether alternative means for providing
healthcare would be more effective
Aims / sc pe
RfPB will also fund research that will:
•
•
pilot or assess the feasibility of projects requiring major
applications to other funding bodies
formally assess innovations and developments in healthcare
Proposals which have emerged from interaction with patients/service
users and public are particularly welcome
Systematic reviews may be funded as part of a larger project or as
stand alone studies. For more details, see the RfPB FAQ on
systematic reviews.
http://www.ccf.nihr.ac.uk/RfPB/Pages/FAQ.aspx
OUT of sc pe:
•
Lab-based or basic science research, including
research based on animals
•
•
•
Setting up or maintaining research units
Proposals which are solely service developments
Proposals which are solely audit, surveys, needs
assessment, technology development (although may
be included as part of an integrated research study)
Images from Office Online
Read the guidance...
Each RfPB competition has associated Guidance for
Applicants that covers:
• aims
• scope
• eligibility etc
For general enquires
call 0208 843 8057
Or email rfpb@nihr-ccf.org.uk
Or visit http://www.ccf.nihr.ac.uk
Image from Office Online
Selection Criteria and Eligibility
Applications judged on:
•
•
the quality of the research proposed
the significance and potential benefit of the research to the NHS
All NHS organisations and other providers of NHS services in England can
apply (For joint NHS/university applications funding is awarded to the NHS
partner)
•
•
•
Probably with academic partners
Potentially with strong component of service user involvement.
Applications with collaborations in social care and third sector
providers of health and social care are also encouraged
If strong case, can include academic partner outside England.
Image from Office Online
The Application
Process
•
Full application deadline
•
Preliminary scrutiny and is it in scope?
•
External peer and lay review
•
Full committee assessment
•
Final ratification
•
Notification of outcome
Images from
Office Online
Overall, a wide range of RfPB projects
funded
•
•
•
Based in a diversity of settings and care groups
Concerned with clinical and non-clinical topics
Using a diversity of methods and research techniques
See the detailed report on the first three competitions –
Prioritising Patients
Part 3 RfPB APPLICATIONS
A grant application, like a
picture, should be:
•
•
•
•
Clear
Unambiguous
Easy on the eye ☺
Easy on the brain
Photo by Remy Steinegger; Copyright : World Economic Forum.
Licensed under the Creative Commons Attribution-Share Alike 2.0
Generic license (http://creativecommons.org/licenses/by-sa/2.0/deed.en)
Source: Wikipedia
It should not be :
•
•
•
•
Jumbled up
Unclear
Difficult to decipher
Confusing
Or indeed something
you didn’t intend it to
be
Original Author: David Angell (adapted by Odder).
Licensed under Creative Commons Attribution 2.0 Generic
Licence (http://creativecommons.org/licenses/bysa/2.0/deed.en). Source: Wikipedia
Part 3 RfPB APPLICATIONS
(a) What would you expect to see
in an RfPB grant application?
Part 3 RfPB APPLICATIONS
(b) Examples of ‘good’ and ‘bad’
grant application sections
Group/interactive work
For examples of good & bad
grant application sections,
see [weblink]
Simon Howden
www.freedigitalphotos.net
http://www.freedigitalphotos.net/images/view_photog.php?photogid=404
Part 3 RfPB APPLICATIONS
(c) What a reviewer expects to see
in an RfPB application form
Overall style
Limit use of overly technical language, buzzwords and
acronyms
•
•
•
Tech Speak (defined by Edward Tenner)
“Post colloquial discourse modulation protocol for user status enhancement.
It's a referential system for functional-structural, microscopically specific
macroscopic-object redesignation. It's a universal semantic transformation
procedure. It's a holophrastic technocratic sociolect. It's a meta-semiotic
mode for task specific nomenclature." Get the idea?
Buzzword
An important-sounding word or phrase connected with a specialized field or
group that is used primarily to impress laypersons: “‘Sensitivity’ is the
buzzword in the beauty industry this fall.”
Alphabet soup
Language laced with acronyms. "An RFP was issued by DOE; ORNL
responded for the ORR."
Reference: http://www.designsensory.com/pws/lesson4/index.htm,
‘Tech Speak’ as defined by Edward Tenner (1986) Tech Speak, Random House, in above website.
1.
Research details
11. Case For Support – Part 3
2.
Lead Applicant Details
12. Management and Governance
3.
Lead Applicant CV
13. Intellectual Property and Innovation
4.
Joint Lead Applicant(s)
14. NIHR Infrastructure
5.
Co-applicant(s) details
15. Finances
6.
Research & Development Office
16. Monitoring information
7.
History of the Application
17. Suggested reviewers
8.
Patient and public involvement
18. Supporting Documentation
9.
Case For Support – Part 1
19. Declarations and signatures
10. Case for Support – Part 2
(1) Research details
•
•
•
•
•
•
•
Title
Host organisation
Region
Duration
Proposed start date
(Application type) – Pilot, Feasibility, or Main Study*
(Theme Call Name) – Only fill this in if you are applying for a specific
theme call, e.g. dementia
*For definitions, see: http://www.netscc.ac.uk/glossary/
Research Title
•
Descriptive, yet concise
•
Contains keywords relevant to the research
•
Study question and main outcome measure to be included
•
If a trial, also include intervention and population
•
State if pilot/ feasibility
•
Clear from title that study is in scope – e.g. for RfPB, has shortmedium term patient benefit
Title: What can go wrong
•
Whilst you may think some titles are genuinely funny:
Parasites as weapons of mouse destruction.
Journal Animal Ecology. 2008 Mar;77(2):201-4.
•
They may be viewed by some as inappropriate:
Targeting promiscuous signaling pathways in cancer:
another notch in the bedpost
Trends in Molecular Medicine. 2004 Dec;10 (12):591-598 .
Title: What can go wrong
•
And some can appear very strange indeed:
Who Moved My Cheese by Dr Spencer Morgan
Putnam Adult (1998)
Image from Office Online
(2-5) Applicants
•
•
•
•
Lead applicant
Current Research Commitments and how this application will fit within
current research activities
Lead applicant’s C.V.
Joint lead, Co-applicants. N.B. There is now a system for co-applicants to
formally confirm involvement, add C.V.s, and approve the final application.
The latter must be done before submission.
Consider:
• Kudos associated with being lead
• Not necessarily a big-wig but need proper support if junior
• Tends to be person initiating the research
• Contract duration (especially universities), i.e. if your contract expires before
project, can you be lead?
Q. What is the difference between ‘co-applicants’ and ‘research team’?
A. A research team may consist of more people than simply the applicants, but
N.B. the application will be judged primarily by those appearing on the form
(6) Research and Development
Office
•
Host must be an NHS organisation
•
Host R&D office contact details
Image from Office Online
(7) History of This Application
•
If previous RfPB, NIHR may well look to see what the initial
feedback was and how the current application has changed, BUT
always viewed as a NEW application (no resubmission policy)
•
If previously submitted elsewhere, just for their information
•
Details of related applications
•
Details of concurrent applications are not asked for, but RfPB would
obviously want to know if a submitted project gets funded from
another source
(8) Patient and Public Involvement
Image courtesy of healthcaregovernancereview.wordpress.com
What is Patient and Public
Involvement?
•
An active partnership between the patient/public and researchers,
NOT the use of patients/public as ‘subjects’ to be studied (e.g. to be
interviewed or assessed) by researchers
For example:
•
•
Working together with patients to develop a good research proposal
Commenting on the methods or processes of the
research, being actively involved in the research process
Image from Office Online
Examples of PPI
•
•
•
•
•
Being part of the research team
Helping design the research
Being a member of project reference group
Writing lay summaries and patient
information sheets
Carrying out questionnaires with the
participants
More power in the decision making process of
the research
Image from Office Online
Nature of PPI in Research
Consultation:
Asking people who use your
services for their views on your
research to help inform decision
making
E.g. Commenting on the
ability to understand lay
summaries
Collaboration:
Ongoing partnership with
E.g. A member of public
members of the public throughout on the project steering
the course of the research project. group.
User-Led/ User
Controlled:
Members of the public lead the
research and are in control of the
research. This is often, through a
community or voluntary
organisation
E.g. Service users may
decide on the research
question; or manage the
research itself
Proper PPI in Research IS NOT:
•
Asking the patient or the public to answer research
questions and provide data, i.e. asking them to be a
‘guinea pig’ in your research – enrolling them into the
study
X
•
Listening to patients to design your service better – to
elicit service design suggestions
•
Asking for feedback about a web based intervention – to
evaluate an intervention
Image from Office Online
Why Involvement?
•
•
•
To prevent researchers making mistakes that are obvious to
patients
To guide finances
For greater transparency, making research with patients,
for patients, that is more relevant to patients.
•
Obligation to the tax payer
•
To meet the requirements of the NHS Constitution
•
To meets the requirements of funders
Can you spot Involvement?
The following are all extracts taken from National
Institute of Health Research grant proposals
Which are Involvement and which are not?
Image from Office Online
"We plan a steering group for the project with four patient
representatives as equal members of the research team."
1.
2.
3.
4.
Involvement
Participation
None of these
Unsure
"Individual service users who have been referred ... via this and
other routes will be interviewed regarding their experiences."
1.
2.
3.
4.
Involvement
Participation
None of these
Unsure
"Our patients agreed with us that the study would deliver
important information and that the research was timely and
important."
1.
2.
3.
4.
Involvement
Participation
None of these
Unsure
What can the RDS Offer?
•
•
•
•
We provide a lay review service as part of our pre-submission
review panel and as a separate activity
We can advertise your project through our PPI network and help to
identify patients to work with you
We can provide financial support to patients or members of the
public working on projects supported by the RDS up until the point of
submission.
We can provide PPI advice from our staff on the PPI team.
http://www.rds-sc.org/public-patient-involvement/
What can the RDS do?
•
•
•
Give guidance on payment structures.
Provide assistance in writing lay summaries usually by getting lay
members to write them either with you or for you.
We have useful guidance helping you consider what involvement to
use to get the best out of involvement in your project.
David Castillo Dominici
www.freedigitalphotos.net
http://www.freedigitalphotos.net/images/view_photog.php?photogid=3062
How can we work together?
•
•
•
Maybe you are part of, or know of, a group that wants to get
involved in research, if so register the group via the RDS website
Researchers can advertise their projects through our systems
If you would like to get involved individually as a patient or a
member of the public register with us via the RDS website
http://www.rds-sc.nihr.ac.uk
Image from Office Online
What reviewers expect to see:
Patient & Public Involvement (PPI)
•
•
•
•
•
•
•
•
Have the needs of the service users been considered?
Have you made any assumptions about service users or the cooperation of other involved parties which may not actually be met?
Are there lay co-applicants?
Are there lay people on the project steering group?
Have the relevant national / local organisations been contacted for
their input? Are there plans to involve them in the study design /
study management / in dissemination of the results?
Is there evidence of consulting the INVOLVE guidelines on PPI?
State if PPI input at study design stage was not undertaken and
say why this was the case (N.B. all efforts must be made...)
Is the PPI element ‘tokenistic’, or has real thought and planning
been put into it?
(9) Case for Support - Part 1
•
Aims and objectives
•
Plain English (Lay) and Scientific Summaries
•
Background and rationale
•
Why research is needed now
Aims and Objectives
•
•
•
•
•
Need absolute clarity of objectives / primary outcome / secondary
outcomes, and ensure that these relate to RfPB scope and are not
contradicting the title
Is this definitive / exploratory / preliminary work? What would the
upshot be if it is successful in the case of the latter?
If there is a null hypothesis, is it clear and appropriate?
Do any qualitative aspects of the research have clearly stated
objectives / questions / intentions?
If a pilot or feasibility study, make this clear, and also how it would
lead on to full trial
Q. What is the difference between a feasibility study and a pilot study?
A.
Feasibility Studies “Can this study be done?” Used to estimate important
parameters needed to design the main study, e.g. standard deviation of the
outcome measure; willingness of participants to be randomised and
clinicians to recruit them; number of eligible patients; characteristics /
design of proposed outcome measure; follow-up rates, response rates,
adherence/compliance rates
Pilot studies are a version of the main study that is run in miniature to test
whether the components of the main study can all work together, e.g. to
test recruitment, randomisation, treatment, and follow-up assessments
(http://www.netscc.ac.uk/glossary/#glos6)
Abstract in plain English
(Lay Summary)
Concise, convincing summary of:
•
•
•
•
•
Problem (e.g. disease burden)
Research question and rationale
Design and method
Expected outcomes
Expected impact
BUT absolutely understandable to the lay person:
•
•
No clinical or research jargon
No difficult words
Involve a PPI representative!
(Scientific) Summary
Concise, convincing summary of:
•
•
•
•
•
Problem (e.g. disease burden)
Research question and rationale
Design and method
Expected outcomes
Expected impact
A ‘Mini-Me’ of the main protocol!
Background / Rationale
This section needs to
PACK a PUNCH
!!!
Using the literature:
•
•
•
•
•
Define the research issue
Justify WHY the research is necessary – clear case for clinical importance and
impact - E.g. Disease / condition burden (local / national figures)
Back up logic of aims and objectives
Provide evidence of best design approach
Highlight if study fits into a desired but under-represented area in the funding
stream - E.g. social care, minority groups
Image from Wikimedia Commons (Public Domain).
Relevance of proposal to RfPB
Look at RfPB scope in Guidance
Think of the:
•
•
•
Likely benefits to patients
Implications for health services
Potential impact on policy / service delivery
Make sure you are showing how your work fits in with
the RfPB remit
Image from Office Online
Why is this research needed now?
•
•
What evidence is available to justify the study at this time: make a case
Is the research addressing the right questions in the subject area at the
right time in relation to, for example, the availability of knowledge, or
the development of technology, Government priorities?
Are you presenting a convincing argument?
Don’t forget this is a competitive process!
(10) Case for Support
Part 2: Research Plan & Methodology
Image: NASA
You should include:
• Study design – appropriate to meet aims and objectives
• Setting (location, population)
• Sample – sampling frame, eligibility criteria
• Sample size
• Intervention (if relevant)
• Details of randomisation if RCT
• Appropriate data types and sources
• Outcomes/ other measures – validated/appropriate for the population
• Procedures
• Data analysis methods
Consider:
• How will potential sources of bias be avoided/ taken care of? Who is
blinded and how will this be achieved?
• Feasibility
• Bear in mind what will be reported (e.g. CONSORT, STARD etc)
Study population
•
•
Clearly defined and consistent with study title/objectives
Eligibility criteria – how generalisable are the results to the NHS?
?disease severity
?comorbidities
?location
?new diagnosis
?previous treatment
?concomitant medications
?exposure
Image by Xedos4, freedigital photos.net
http://www.freedigitalphotos.net/images/vie
w_photog.php?photogid=1539
Intervention (i)
•
Clear description of intervention
?product / technology/therapy
?timing
?dose
?length of exposure
?number of sessions/treatments
?staff administering treatment (training, quality assurance, experience,
preferences)
•
•
•
•
Allocation - randomisation method – blocks / stratification / matching
Blinding – who, how and why?
Balance between standardisation and tailoring to the patient
Will it be affected by patient/ carer adherence?
Intervention (ii)
Why was it chosen?
•
•
Evidence of piloting / testing or feasibility/ effectiveness from the
literature
Clear description and evidence-base for control / comparison
group(s)
For Example,
• Why placebo / alternative treatment chosen?
• Why 3 weeks treatment rather than 6?
Intervention (iii)
•
•
•
Feasibility and other benefits
Provision for gathering information on the patients / staff view of the
research, whether it works or not
Patient benefit of intervention
– Recovery
– Empowerment
– Independence
– Increased confidence in their outlook
– Benefits added on existing standard of care
Outcomes
•
•
•
•
•
•
Measurements must accurately reflect the impact of the intervention
If surrogate / proxy measures are used, give evidence that these
can accurately predict long-term outcomes
WHO?
HOW?
WHEN?
WHY?
Follow-up:
• Appropriate length of follow-up to see the
effect on outcome measures
• Motivation of patients to complete study!
Image in Public domain, courtesy of Wikimedia
Statistical Considerations
Statisticians are useful
for much more than just a
sample size calculation!!!
Prof. David Spiegelhalter playing with Arco
Iris samba band, by Tamara K.
This file is licensed under the Creative
Commons Attribution-Share Alike 3.0
Unported license
(http://creativecommons.org/licenses/bysa/3.0/deed.en)
Defining study
objective
Defining a
study design
Defining the
primary
outcome
Sample size
calculation
Statistical
analysis




Cohort study
Cross sectional study
Case-control study
Randomised controlled trial
 Type of outcome
o Continuous
o Binary
 Clinically relevant and relevant to patients
 Measurable
Which
patients to include in the analysis
Effect
size
 Accurate
E.g. principle
of intention to treat in
o oClinically
significant
randomised
controlled
trials
o Results
in a change
in practice
Account
for confounding
 Based
on pilot
study/previous evidence
Missing
data
 Correct
and
appropriate for the study design chosen
Subgroup
analysis
 Account
for loss
to follow-up
 Multiplicity
 Sensitivity analysis
Sample size
•
•
•
•
•
State null hypothesis and choice of statistical test
Is the detectable effect size likely to be clinically significant and
result in a change in practice?
Is the sample size based on previous evidence / pilot results?
Are the sample size calculations correct and appropriate for the
study design chosen?
Has loss to follow-up been accounted for in the sample size?
Common statistical errors in applications
•
•
Statistical analysis does not meet objectives or type of data
collected
Section is too ‘general’ ‘continuous data will be analysed using ttests’
•
Copied from other/ previous grant application without tailoring
•
No costing for statistical support
Outcomes / Analysis
•
•
•
•
•
•
•
Measurements accurately reflect the impact of the intervention or
are surrogate / proxy measures (and if so, evidence that these can
accurately predict long-term outcomes)
Appropriate length of follow-up to see the effect on outcome
measures
If there is a long period of follow-up, how will the patients be
motivated to complete the study?
Protection of outcome measurement from bias
Sub-analysis clear and documented – rationale and design
Listing of confounding factors – measured / factors for stratification
with rationale
Set out analysis plan logically – descriptive tabulations, approach to
analyzing associations among variables, qualitative analytic
methods
What is Health Economics?
•
•
The NHS operates in a world of scarce resources (e.g. MONEY, labour,
medicines, hospital beds)
Health economics informs how scarce resources may be allocated
among treatment, prevention & diagnosis of different diseases. At the
extreme:
Karl Marx:
State allocates resources
Adam Smith (man in £20 note):
Free market
Images in Public Domain. Courtesy of
Wikimedia.
How do we prioritise resources to treat &
prevent disease?
•
Need (how do you measure it / competing needs)
•
Discrimination (e.g. by age, smokers …)
•
Personal merit and social esteem
•
Lottery
•
Ability to pay (free market approach)
Do you think any of these methods would work in the NHS consistently?
In the NHS, new interventions prioritised if:
•
•
they can be shown to be effective and fit for purpose
AND GOOD VALUE FOR MONEY – i.e. the intervention is likely to
be cost-effective (mainly through NICE)
Image from Office Online
Health economics can help us understand the costs
and benefits of different interventions:
For example:
• If the health benefit generated by a new intervention is worth the
additional cost
•
•
If an intervention is more effective than its alternative, but its related
costs are exorbitant, this intervention will represent poor value for
money and not a good use of NHS resources
If one intervention provides more health outcomes for the same level
of cost this intervention should be adopted
UK research funders increasingly value applications that answer health
economic questions
Planning a project
“Beagle 2 could have crashed into
Mars because the atmosphere on the
planet was less dense than expected”,
explained Professor Pillinger. “The
probe may have been going too fast
for its parachute and airbags to bring
about a soft landing.”
Photograph: ESA
But contrary to belief, the mission was
not a failure. The technology used for
the project could play a major part in
managing the tuberculosis pandemic.
(Open University WebSite, March 2010)
The people you need
The materials, equipment and space you need
The time you need
How you organise the above
Who?
What?
When?
Where?
For how long?
Think about what will need to be done on a day to day basis
Think about maximising benefits to all involved
The people you need
Photograph: NASA
Various roles, for example:
•
•
•
•
•
•
•
•
•
Collaborators
Steering committee
Day to day project manager/ co-ordinator
Research assistants
Administrative staff
Support staff
Consultants
Data Safety Monitoring Board
Patients/ public
•
•
•
•
•
Collaborators
Bring the right skill mix to the research team
Varying degrees of involvement
Steering committee
Overseeing the work, ensuring that the study happens as
intended
Day to day project manager/ co-ordinator
Managing the study on a day to day basis
Research assistants
Data collection
Collection and processing of samples
Administrative staff
Administrative support
•
•
•
•
Support staff
People external to the study, but whose help may be needed.
E.g. staff on wards, library, ethics committee, local data protection
officer
Consultants
For specialist help and advice
E.g. statisticians, economists, computing
Data Safety Monitoring Board
Independent ; periodically check on data being collected for patient
safety purposes; broad-ranging monitoring of data quality and
integrity
Patients/ public
Different levels of involvement
E.g. Consultation, collaboration, user control
Organising the personnel
•
Who is in charge of what?
•
Must be clearly designated at the start
•
Everyone involved must be committed in terms of interest
and time
Staff management: very important
Logistical issues in recruitment
Photograph: NASA
Staff
•
•
•
•
•
Liaison with personnel
Development of job descriptions
Advertisements
Interviewing
Getting proper references
Sites
•
•
•
Give
yourself
time!
Liaison with management
Liaison with health professionals
Information about the study, what it will entail for the study site,
the information being collected etc.
Participants
•
•
Look at throughput/ numbers to give you an idea about how long it will take
First contact, informed consent
Recruiting Participants
•
•
Clear recruitment plan and sampling strategy (note any key groups
of patients likely to be excluded)
Realistic timing - use local data and apply eligibility criteria, refusals,
variation in caseload etc
Image: ddpavumba
freedigitalphotos.net
http://www.freedigitalphotos.net/images
/view_photog.php?photogid=2888
•
Make effort to ensure inclusion of minority / disadvantaged groups /
range of patients across all socioeconomic
groups
– describe provision for overcoming
poor literacy and language barriers
Image from Office Online
The importance of communication
Photograph: NASA
Before the start of the study
•
•
•
•
Who should know about the study?
Who should be involved?
Formal agreements (e.g. sponsorship, financial
arrangements, data ownership and authorship)
External communication (e.g. Websites, Press Office)
Examples of who needs to know:
•
•
•
•
•
•
•
•
•
Trust R&D offices
Other senior managers
Ethics committee(s)
Clinicians caring for patients involved
Nursing staff if wards involved
Comprehensive Local Research Networks
Medical records officer
Clinic/ward clerks
GPs
On-going communication
•
•
•
•
•
•
•
•
•
•
•
Clearly defined roles between researchers – ‘chain of command’,
appropriate support
To ensure data quality, reaching of project milestones
With ethics committee (any change in protocol, methods, dilemmas
etc.)
With participants: contact person for everyone, whenever required
(e.g. everyday, or ‘on-call’, as necessary)
Back-up information
With study sites and personnel: politics and facilitation
Clearly identified researchers on wards etc.
General information exchange
Query-answering
Trouble-shooting
Feedback and acknowledgement
The other resources you need
Photograph: NASA
Postage
Stationery
Tests
Equipment
Telephones
Travel
Photos: Wikimedia
Printing
Miscellaneous, e.g. staff training
Logistical considerations in study design
Photograph: NASA
•
•
Can your protocol be operationalised as a guidance document
for staff?
Sample selection and monitoring recruitment
If applicable:
•
Randomisation
•
Arranging an intervention
Assigning responsibility and ensuring time for these
Logistical considerations in data
collection & data management
Data collection on Mars
Photograph: NASA
Types of tools
•
•
•
•
•
Measuring outcomes – tools need finding (N.B. permissions) or developing
(N.B. validity, reliability)
Lay and Non-English speaking issues
Collecting other information, e.g. process, administrative, contextual
Coding for the computer, development of database
Only collect what you need to answer the question
Logistics
•
•
•
•
•
•
•
•
How will the data be collected: By whom? Where? When?
Holidays, bank holidays (postage)
Data checking and quality control
Confidentiality/ data protection issues
Procedures for reporting adverse events
Data management and storage (e.g. double blind entry, restricted access, backup, encryption etc)
Computing – coding and data entry
Data cleaning
Piloting
Photograph: NASA
The importance of piloting
Always pilot!
It checks that everything works:
•
•
•
•
•
•
•
•
•
Management and personnel
Methods
Feasibility – technically, politically, logistically, methodologically
Training of fieldworkers
Sample
Timing and content of questionnaires/ interviews
Validity of tools and data sources (feedback from participants)
Overall time-scale
Data entry, cleaning and analysis
Logistical issues in data analysis
Cray 2 Supercomputer, NASA
Photograph: NASA
Planned before the study has started:
•
Equipment (hardware, software)
•
Statistical methods
•
Statistics and computing support
•
Other analysis issues (e.g. qualitative, health economic)
TIME
(11) Case for Support – Part 3
Dissemination, Outputs & Expertise
Pigeon post
Source: Wikipedia (Public Domain)
Routes for Outputs/ Dissemination
Via:
•
•
•
•
•
•
•
•
•
•
Involved patients / lay members & local interest groups
Research network meetings
Targeted conferences for presentations / posters
Targeted peer-reviewed journals - maximum readership within field
Other media, as appropriate
Use of workshops / training packages
Guidelines which could be altered by the results
Targeted groups of specialist / professional bodies
Inclusion of results in existing meta-analyses / Cochrane reviews
Members of the team involved in teaching
Not forgetting…
•
•
•
Participant patients and staff
Report for funding body
Any reports for Ethics Committee
And logistically…
•
•
Time
Personnel
And importantly…
•
How will findings be translated into health care to produce
patient benefits
Expertise: have you got your A-team?
3
4
1
2
Hypatia (Ὑπατία)
1 ) Mad Scientist drawn by JJ, Image provided courtesy of Wikipedia. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free
Documentation License, Version 1.2
2) The School of Athens - fresco by Raffaello Sanzio. Image in the public domain supplied courtesy of Wikipedia.
3) Marie Currie ca. 1898. http://www.mlahanas.de/Physics/Bios/MarieCurie.html Image in the public domain supplied courtesy of Wikipedia.
4) This image is available from the United States Library of Congress's Prints and Photographs Division under the digital ID cph.3b46036. Image in the public domain supplied
courtesy of Wikipedia.
Questions a funder will ask:
What are the specific roles of members of the research team?
Does the team:
•
•
•
•
•
Have appropriate track records in research?
Demonstrate appropriate skills mix - clinical and research
experience?
Include, where appropriate, a statistician, health economist?
Include a PPI representative?
Have genuine interest and time to commit to your research
Overall, why is this group of people best placed to do the work?
(12) Management and Governance
•
Research timetable: milestones and deliverables
•
Management arrangements
•
Previous relevant work
•
Measures of success, risks and how to manage them
•
Ethical issues
Ethics
Image from Office Online
•
•
•
Does the research involve specific areas with particular
ethical implications?
Ethics committee approval required?
Has the proposal already been granted a favourable
opinion from relevant regulatory authorities?
(13) Intellectual Property and
Innovation
•
•
•
•
•
•
•
•
Current IPR held by applicants and relevance to current application
Has a search for existing IPR been undertaken?
Related/ similar IPR which may affect freedom to operate?
Will IPR be produced or improved through the research?
Management of any IPR
Competitive devices, practices and technologies and advantages
over these of any likely to be produced through the research?
Benefits of any likely IPR
Regulatory considerations and risks to be considered for any new
technology to reach the market?
(14) NIHR Infrastructure
•
Links to NIHR networks - look at www.ukcrn.org.uk
•
RDS input
•
Involvement with other NIHR partners
Image from Office Online
(15) Finances
Source: nuttakit, freedigitalphotos.net
http://www.freedigitalphotos.net/images/view_photog.
php?photogid=1556
Value for money
•
Justification for major sources of expenditure, e.g. staff
•
Justification for how research costs have been allocated
•
Justification for how NHS Support and Treatment costs will be
allocated.
The people you will need to
involve
•
•
•
The local NHS R&D Finance Officer of your employer
If you are collaborating with other organisations, the appropriate
Finance Officer in each one
Your local Comprehensive Local Research Network (for any
NHS Support Costs required)
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
The RfPB Finance Form
Always: Consult your Finance Office at an early stage
Be aware:
•
•
•
•
•
•
That costs are broken down by project year
Of timings for university/ trust/ CLRN approval
Of which organisation is taking responsibility for the project – their
authorised signatory will have to sign off the form.
That if the study is a collaboration between organisations, a ‘system’ will
not necessarily exist to sort out respective costs
That the participation/ costs of other collaborators will also have to be
signed off by them
That, if successful, formal collaboration agreements will need to be in
place before funds are distributed
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
Directly Incurred Costs
Q. What is a directly incurred cost?
A. Costs of people/ other things that are being
employed/ used exclusively for the project
•
•
•
•
•
•
Research, technical and other staff
Travel and subsistence
Equipment
Consumables
PPI
Other, e.g: Conference fees, room hire, publication, training
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
HEI Indirect Costs –Estates
Charges
Q. What are HEI Indirect Costs – Estates Charges?
For universities only, the estate costs for staff, i.e. physical environment
required for general staff support.
•
E.g. For Oxford University, for 2010/11, the estate charges, per full
time member of staff per year, regardless of grade, are £3,367
(£12,020 for lab. (scientific) departments)
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
HEI Indirect Costs – Other
Indirect Costs
Q. What are HEI indirect Costs –Other Indirect Costs?
For universities only, (i) the overheads associated with employing
someone, (ii) the costs of people already employed taking ‘time out’ for
the purposes of the study (e.g. supervision), & (iii) university-based
technical support (e.g. laboratory costs)
•
E.g. For Oxford University, for 2010/11, the additional costs (or
‘overheads’) associated with employing a full time member of staff,
regardless of grade, is £44,124.
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
Commercial/ Other Partner
Organisation Indirect Costs
Q. What are ‘Commercial/ Other partner Organisation Indirect Costs’?
A. The costs of resources used by the research that are shared by other
activities.
•
Seek advice from Finance Department
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
NHS Overheads
Q. What are ‘NHS Overheads’?
A. Costs of resources used by a project, but shared
by other activities
E.g.
•
Estates charges
•
Telephone rental etc.
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
NHS Support Costs
Q. What are ‘NHS support costs’?
A. The additional patient care costs that result from the
research being run in an NHS organisation. These
costs would end when the project ends.
•
•
•
Extra patient tests
Extra in-patient days
Extra staff time, e.g. consultants or nurses taking consent, extra
attention due to the research
This funding would usually come via the Comprehensive Local
Research Network, and is not included in the total of the grant
application figure
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
NHS Estimated Treatment Costs
Q. What are ‘estimated treatment costs’?
A. The treatment costs that relate to the research
intervention.
•
•
You are asked for information on the research treatment
(intervention), and the normal standard treatment, so the additional
costs for running the research intervention can be calculated.
These costs would normally come from the NHS as part of normal
patient care commissioning
Different treatment, or treatment administered in a new way, or a new
place.
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
Costing people
•
•
•
•
•
•
•
•
•
Researchers involved with the study (but already in post)
Researchers employed exclusively for the study, e.g.
experienced day to day study/ clinical trial manager/ coordinator/ primary researcher, research assistants
Collaborators
Steering committee
Administrative staff
Support staff
Consultants
Data Safety Monitoring Board (DSMB)
Patients and public involved with research (PPI)
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
Salaries:
Regular staff (internal) Consultancy (external)
Basic salary (check
scales)
National insurance
Superannuation
Increments/ pay rises
Indirect costs (including
Estate costs)
Daily rate
Rate per sample etc
Have you considered dates for each major phase of
the study, with staff responsible
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
Costing other resources
Stationery
E.g. How many questionnaires, how long, how many times; invitations;
letters, information sheets, how to represent study to public/ patients
Printing
E.g. How many questionnaires, how long, how many times; invitations;
letters, information sheets
Postage
E.g. How many mailings, follow-ups and reminders
Travel
E.g. researcher to participant, participant to researcher, meetings to
sites to set up and run project, dissemination, PPI, Steering Committee
meetings
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
Tests
E.g. Clinical tests, laboratory tests
Equipment
E.g. monitoring instruments, computing equipment (including software), desks,
chairs
Telephones
E.g. Office and mobile; standard rates for institutions?
Miscellaneous
E.g. Staff training, running costs of equipment, advertisements for staff,
incentives (within ethics guidance)
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
Overall budget
Have you considered:
•
•
•
•
•
•
•
How the resources requested will be used
How realistic and accurate the costings are
Whether the ‘best price’ has been found for tests etc?
Whether anything appears ‘over-budgeted’?
Whether the requested resources are justified?
Does the proposal represent the most efficient use of resources?
Is the research good value for money?
Give yourself some leeway
Photo: renjith krishnan, www.freedigitalphotos.net,
http://www.freedigitalphotos.net/images/view_photog.php?photogid=721
(16) Monitoring information
•
Information used by DoH and NIHR to monitor and categorise
applications
•
RDS use and satisfaction with service
•
Carbon guidelines
(17) Suggested Reviewers
•
Suggest three potential peer reviewers who have relevant expertise
•
May or may not be used
(18) Supporting Documentation
E.g.
•
Flow diagram
•
Gantt Chart
•
•
Bibliography - ensure that all papers cited are listed clearly and
logically
Letters of support
The inclusion of a Gantt Chart is
mandatory
31st FW Balkan
Operations Gantt
chart. United States
Air Force, 2000
Image in public domain courtesy of Wikimedia
(19) Declarations and Signatures
•
•
•
Lead Applicant
Host Organisation (finances)
Host Organisation (Head of Department/ Senior
Manager)
Give yourself TIME to get these!
renjith krishnan, freedigitalphotos.net
http://www.freedigitalphotos.net/images/view_photog.php?p
hotogid=721
Part 4 Top Tips:
Optimising your chances
of success
(Inspired by RfPB feedback)
Make sure:
•
•
•
•
•
•
•
•
•
Your study is in scope
You submit your proposal on time
It is complete!
It is a ‘well-crafted’ proposal
Your proposal is understandable to a diverse audience
You have shown the relevance of the study to the NHS
You have had adequate lay involvement
You have the right skill mix
You have made the necessary research partnerships
Make sure:
•
•
•
•
•
•
•
•
•
You have demonstrated a need for the study, and your knowledge
of the area
Any necessary pilot work is undertaken or planned appropriately
Interventions (e.g. for RCTs) are clearly described
You have an appropriate study design
Other aspects of the methods and procedures are clear
You have consulted appropriate experts and incorporated their
suggestions
You have given yourself a realistic timetable
Your study is sensibly funded
Don’t be over-ambitious!
Any questions?
Evaluation
MANY THANKS!
Photo: Simon Howden, FreeDigitalPhotos.net
http://www.freedigitalphotos.net/images/view_p
hotog.php?photogid=404
FreeDigitalPhotos.net staff
FreeDigitalPhotos.net