Grant Writing Workshop for
Exploration Projects
Allen Ruby
National Center for Education Research
Getting Started
Request for Applications
http://ies.ed.gov/funding/11rfas.asp
Future RFAs: sign up for newsflash
http://ies.ed.gov/newsflash/
IES Grants.gov Application Submission Guide
http://ies.ed.gov/funding/11rfas.asp
Application Package
http://www.grants.gov/
Education Research Grant
Program (84.305A)
Special Education Research Grant
Program (84.324A)
Education Research Grant Program (84.305A)
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Reading and Writing
Mathematics and Science Education
Cognition and Student Learning
Teacher Quality (Reading & Writing; Math & Science)
Social and Behavioral Context for Academic Learning
Education Leadership
Education Policy, Finance, and Systems
Postsecondary Education
English Learners
Early Learning Programs and Policies
Education Technology
Adult Education
Organization and Management of Schools and Districts
Analysis of Longitudinal Data to Support State & Local
Education Reform
Special Education Research Program
(84.324A)
•
•
•
•
•
•
•
Early Intervention and Early Childhood Special Education
Reading, Writing, and Language Development
Mathematics and Science Education
Social and Behavioral Outcomes to Support Learning
Transition Outcomes for Special Education Secondary Students
Cognition and Student Learning in Special Education
Professional Development for Teachers and Related Service
Providers
• Special Education Policy, Finance, and Systems
• Autism Spectrum Disorders
Solution Driven Research
Research intended to contribute to the solution
of practical education problems
–
–
–
–
–
Exploration
Development
Efficacy Evaluation
Scale-up Evaluation
Measurement
Before Development Work Begins...
Need to understand the problem:
• Examine the underlying processes that affect
education outcomes
• Look for malleable factors that might be
targets for interventions
• Identify what distinguishes between effective
and less effective practices
Before Evaluation Begins...
Identify education interventions or components
of them that appear promising due to their
association with improved student outcomes
before committing resources to evaluation
In a Nut Shell...
Development work and some
evaluations begin with
Exploration Projects that
examine underlying
processes for the purpose of
informing the development
of new interventions,
modifying existing
interventions, or identifying
promising interventions for
evaluation
Exploration Goal
• Explore the association of education
outcomes with malleable factors
• Examine factors or conditions that mediate or
moderate the relationship between malleable
factors and education outcomes
• Small primary data studies, secondary
analyses, and meta-analyses
Malleable Factors
• Malleable factors must be under the control
of the education system – something that can
be changed by the system
• Examples
–
–
–
–
Student characteristics: behavior, skills
Teacher characteristics: practices, credentials
School characteristics: size, climate, organization
Education interventions: practices, curricula,
instructional approaches, programs, and policies
Exploration Projects Should
• Identify underlying processes that enhance or
inhibit learning. These may contribute to the
development or modification of interventions
• Identify education interventions that may
deserve rigorous evaluations
• Identify factors that mediate or moderate the
underlying processes or the interventions
• Generate hypothesis concerning causal
relations between factors and outcomes
• Contribute to theories of action
Exploration Projects Should NOT
• Test the efficacy of education interventions
• Examine non-malleable factors
• Examine malleable factors outside the control
of the school system
Research Narrative includes 4
sections
•
•
•
•
Significance
Methodological Requirements (Research Plan)
Personnel
Resources
Significance Section
• Describes the overall project
– Research question to be answered
– Variables to be examined
• Provides a compelling rationale for the project
– Theoretical justification
• Logic Models, Change Models
– Empirical justification
• Related work supporting the project
– Practical justification
• importance of the variables (malleable factors,
mediators, moderators, outcomes) examined
Significance Section
• How work will lead to useful next step
– Development or modification of interventions to
address the identified malleable factors or
underlying process to improve student outcomes
– Identification of interventions for more rigorous
evaluation
• Overall importance
Significance – 2 Problem Areas
• Unclear theory of change
– Why should the malleable factor be related to
improved outcomes
– A well laid out theory of change makes clear what
is expected to happen and in what order
– Easy for reviewers to understand research plan –
why measure certain outcomes
– Graphic can be helpful – e.g. a logic or change
model
Significance – 2 Problem Areas
• Unclear Description of Intervention
– What the intervention is
• Many components and may be applied at different
times – how fit together – Graphic may help
– Intervention not shown to be strong enough to
expect an impact
• Informational
• Ensure fidelity
– Overly focused on actions not content
• Ex.: 20 hours of PD held over 10 weeks but no
detail on what is to be covered in the sessions
Research Plan
• Describe the work you intend to do
– How you will answer your research question
• Make certain Research Plan is aligned to
Significance section
– All research questions should have justification in
Significance.
• Step-by-step process
– Timeline to show when everything will be done
Methods will vary by research
question and design
• Secondary data analysis of existing datasets
• Original data collection with appropriate
statistical analyses
• Secondary data analysis complemented by
primary data collection
• Meta-analyses designed to determine
moderators or moderators of effects
All Research Plans Should Include
• Clear, concise hypotheses or research
questions
• Well-specified relations between hypotheses,
measures, and independent & dependent
variables
• Clear description of data sources
• Detailed descriptions of data analysis
procedures
Secondary Data
• Database(s)
–
–
–
–
–
Sample characteristics
Variables to be used
Missing data – how it will be handled
Access to and permission to use the data
Links if multiple datasets used
• Measure of malleable factor
• Primary outcome measures to be used,
including their reliability and validity
• Mediators & moderators
Primary Data Collection
• Describe sample characteristics
• Describe measures to be collected, including
reliability & validity
• Procedures proposed for data collection
• Describe links to secondary data if relevant
If observational data are collected
• How will inter-observer reliability be
maintained
• How will data be coded
• How will data be quantified to support
prediction of relation between what was
observed and outcomes of interest
Meta-analysis
• Criteria for including or excluding studies, and
rationale
• Search procedures used
• Coding scheme & procedures for extracting
data from studies
• Procedures for ensuring reliability of the
coding
Meta-analysis
• Demonstrate that sufficient numbers of
studies are available to support meta-analysis
• Demonstrate that relevant information is
reported frequently enough that a database
can be constructed
• Clearly describe effect size statistics,
associated weighting functions, procedures
for handling outliers, and any other
adjustments
Data Analysis
• Detailed description of data analysis
–
–
–
–
–
Describe method to analyze data
Show model
Link method to all research questions
Include analyses for mediators and moderators
Sensitivity tests to address assumptions
Personnel
• Show expertise in every aspect of project:
– Relevant content domain
– Appropriate methodological expertise for work to
be done in project
– Working with schools, or other education agencies
(if appropriate)
• Analysis of Longitudinal Data to Support State and
Local Education Reform requires SEA/LEA personnel
For Each Key Personnel Describe
• Link each person and their expertise to their
role in the project
• Qualifications
• Roles
• Responsibilities
• Percent of time devoted to the project
Show every aspect of project has person with
expertise and time to do it
Personnel Strategies for PI
• Senior Researcher
– Show adequate time to be PI
– Make credentials clear: not all reviewers may know
• Junior Researcher as PI/PD
– Show you have adequate expertise not only to do
work but to manage project
• Continuation of graduate research
• Management skills as graduate student
– Reviewers more comfortable if you have senior
person(s) on project to turn to for advise
• Co-PI, Co-I, contractors, advisory board
• Have them on for enough time to be taken seriously
Resources
• Show the institutions involved have the
capacity to support the work
– Don’t use university boilerplate
• Show that all organizations involved
understand and agree to their roles
– What will each institution, including schools,
contribute to the project
– Show strong commitment of schools and districts
– Have alternatives in case of attrition
• Appendix A should back this up with
– Detailed Letters of Support from research
institutions, States, districts, schools
Resources
• Data issues
– Document permission to use and access to
confidential data
– Show familiarity with data – show that it can be
used to do the proposed work
– If merging datasets, show that it can be done
Award Information
• Secondary data analysis or meta-analysis:
– $100,000 to $350,000 per year total cost (direct +
indirect)
– Up to 2 years
• Primary data collection analysis (with or
without secondary analysis):
– $100,000 to $400,000 per year total cost (direct +
indirect)
– Up to 4 years
Write from Reviewers’ Perspective
•
•
•
•
•
Write clearly and concisely
Address the points described in RFA
Organize information in logical sequence
Label sections and number pages
Make it easy for reviewers to find and
understand the information
Submitting a Proposal
• All proposals must be submitted electronically
to:
– http://www.grants.gov
• By 4:30:00 pm Washington DC time on
the date listed in the RFA for the competition
to which you are submitting
Key Dates
Application
Deadline
9/16/10
4:30:00 PM
Letter of
Intent
Application
Package
iesreview.ed.gov
www.grants.gov
7/19/10
7/19/10
Start
Dates
7/1/11
to
9/1/11
Final Application Submission
• On-line forms are complete
• PDFs of proposal contents have been
uploaded
• Use “Check Package for Errors” button on
grant application package
• Authorized representative has completed the
final step of the electronic process.
• You have received email acknowledging
receipt of your application.
Peer Review
• Proposal is reviewed for compliance.
• Proposal reviewed for responsiveness
• Compliant and responsive proposals are
assigned to a review panel.
• Two or three panel members conduct primary
review of each application.
• At panel meeting, the most competitive
applications are reviewed by full panel.
Peer Review Process Information
http://ies.ed.gov/director/sro/peer_review/index.asp
Notification
• All applicants will receive email notification of
the status of their application.
• All applicants receive copies of reviewer
comments.
• If you are not granted an award the first
time, plan on resubmitting, and talk to your
program officer.
Final Reminders
•
•
•
•
Start early
Read the Request for Applications
Talk with the program officer
Start the online submission process early
ies.ed.gov
Allen Ruby
allen.ruby@ed.gov