Exploration Projects
within the Education Research Grants
Program (84.305A) and Special Education
Research Grants Program (84.324A)
Allen Ruby
National Center for Education Research
Information for Applying
 Requests for Applications
 Letter of Intent
 IES Grants.gov Application Submission Guide
 Application Package
Requests for Applications (RFA)
• A separate RFA for each grant program
(84.305A and 84.324A)
• Describes the requirements for an application
• Requests for Applications are available on:
http://ies.ed.gov/funding
• To be informed about the release of future
RFAs, sign up for the IES Newsflash:
http://ies.ed.gov/newsflash/
Letter of Intent (LOI)
• A short description of your intended application
– PI, institution, collaborators
– Budget – rough estimate
– Up to 1 page abstract describing the work
• Purpose
– Used by program officers to discuss your idea
– Used by IES Office of Standards and Review to plan
for peer review process
– Not used in the peer review process – superseded
by your application
• Submitted on http://iesreview.ed.gov
Application Packages
• Contains the forms to be filled out and submitted as
your application
• Available at www.grants.gov
– Help: support@grants.gov or 1-800-518-4726
• For the June 23, 2011 application deadline,
packages will be available starting April 21, 2011
• For the September 22, 2011 deadline, packages will
be available starting July 21, 2011
• Packages are specific for grant program and
deadline
IES Grants.gov Application Submission Guide
• Instructions for completing and submitting the
application package
• Available on http://ies.ed.gov/funding
Key Dates
Application Letter of Intent
Deadline
Due
iesreview.ed.gov
Application
Package
Available
Start
Dates
www.grants.gov
6/23/11
4/21/11
4/21/11
3/1/12
to
9/1/12
9/22/11
7/21/11
7/21/11
7/1/12
to
9/1/12
Education Research Grants
Program (84.305A)
Special Education Research
Grants Program (84.324A)
Grant Topics
• All applications to 84.305A and 84.324A must
be directed to a specific topic
• Identify the appropriate topic for your work
– Read the Topic sections of the Request for
Applications (RFA)
– Discuss with the appropriate program officer
(listed in the RFA)
Education Research Topics (84.305A)
•
•
•
•
•
•
•
Reading and Writing
Mathematics and Science Education
Cognition and Student Learning
Social and Behavioral Context for Academic Learning
Education Technology
Effective Teachers and Effective Teaching
Improving Education Systems: Policies, Organization,
Management, and Leadership
• Postsecondary and Adult Education
• Early Learning Programs and Policies
• English Learners
Special Education Research Topics (84.324A)
•
•
•
•
•
•
•
•
•
•
•
Early Intervention and Early Learning in Special Education
Reading, Writing, and Language Development
Mathematics and Science Education
Social and Behavioral Outcomes to Support Learning
Transition Outcomes for Special Education Secondary Students
Cognition and Student Learning in Special Education
Professional Development for Teachers and Related Service
Providers
Special Education Policy, Finance, and Systems
Autism Spectrum Disorders
Technology for Special Education
Families with Children with Disabilities
Grant Research Goals
• All applications to 84.305A and 84.324A must
be directed to a specific research goal (1 of 5)
• The goal describes the type of research to be
done
• As a result, every application is directed to a
specific topic/goal combination
– Note topic and goal on your application (e.g., at
top of abstract)
The 5 Research Goals
•
Exploration
–
•
Development and Innovation
–
•
Evaluate an intervention under ideal conditions
Scale-up Evaluation
–
•
Develop a new or modify an existing intervention
Efficacy and Replication
–
•
The focus of this webinar
Independently evaluate an intervention under routine
conditions
Measurement
–
Develop and/or validate a measure
The Exploration Goal
Before Development Work Begins...
Need to understand the problem:
• Examine the underlying processes that affect
education outcomes
• Look for malleable factors that might be
targets for interventions
• Identify what distinguishes between effective
and less effective practices
Before Evaluation Begins...
Identify education interventions or components
of them that appear promising due to their
association with improved student outcomes
before committing resources to evaluation
In a Nut Shell...
Development work and some
evaluations begin with
Exploration Projects that
examine underlying
processes for the purpose of
informing the development
of new interventions,
modifying existing
interventions, or identifying
promising interventions for
evaluation
Exploration
• Explore the association between malleable factors and
education outcomes
– A malleable factor can be changed by the education
system be it a characteristic of students (e.g., skills,
behaviors), teachers (e.g., credentials, practices) or school
(e.g., climate, size) , or an education program or policy
• Underlying processes that enhance or inhibit learning
• Aspects of a school, district, or community associated with
beneficial education outcomes
• Education interventions associated with beneficial education
outcomes (e.g., professional development, curricula, policies)
• Explore factors that mediate or moderate the relationship
between malleable factors and student outcomes
Education Outcomes are for Students
Preschool
• School readiness
• Developmental outcomes for infants and toddlers with
disabilities
Kindergarten through Grade 12
• Academic outcomes in reading, writing, math and science
• Behaviors, interactions, and social skills that support learning in
school and successful transitions to post-school opportunities
• High school graduation
• Functional outcomes that improve educational results,
transitions to employment, independent living, and
postsecondary education for students with disabilities
Education Outcomes (continued)
Postsecondary
• Access, persistence, completion
• Achievement in gateway math & science courses
• Achievement in introductory composition courses
Adult Education
• Reading, writing, and math for basic and secondary
education and English Language Learners
Exploration Projects Should
• Identify underlying processes that enhance or inhibit
learning. These may contribute to the development
or modification of interventions
• Identify education interventions that may deserve
rigorous evaluations
• Identify factors that mediate or moderate the
underlying processes or the interventions
• Generate hypothesis concerning causal relations
between factors and outcomes
• Contribute to theories of action
Exploration Projects Should NOT
• Test the efficacy of education interventions
• Examine non-malleable factors
• Examine malleable factors outside the control
of the school system
The Application’s Research Narrative
• Key part of your application
• 4 Sections
–
–
–
–
Significance
Research Plan
Personnel
Resources
• Each section scored and an overall score given
• Requirements vary by topic & research goal
• 25 pages, single spaced
Significance Section
• Describes the overall project
– The research question to be answered
– The malleable factors, moderators, and mediators
to be examined
• Rationale for the work
– Theoretical justification
• Logic Models, Change Models
– Empirical justification
• Related work
– Practical justification
• importance of the variables (malleable factors,
mediators, moderators, outcomes)
Significance Section
• How work will lead to useful next step
– Development or modification of interventions to
address the identified malleable factors or
underlying process to improve student outcomes
– Identification of interventions for more rigorous
evaluation
• Overall importance
Significance – 2 Problems Often Seen
in Applications
• Unclear theory of change
– Why should the malleable factor be related to
improved outcomes
– A well laid out theory of change makes clear what
is expected to happen and in what order
– Easy for reviewers to understand research plan –
why measure certain outcomes
– Graphic can be helpful – e.g. a logic or change
model
Significance – 2 Problem Areas
• Unclear Description of Intervention
– What the intervention is
• Many components and may be applied at different
times – how fit together – Graphic may help
– Intervention not shown to be strong enough to
expect an impact
• Informational
• Ensure fidelity
– Overly focused on actions not content
• Ex.: 20 hours of PD held over 10 weeks but no
detail on what is to be covered in the sessions
Research Plan
• Describe the work you intend to do
– How you will answer your research question
• Make certain the Research Plan is aligned to
Significance section
– All research questions should have justification in
Significance.
• Step-by-step process
– Timeline to show when everything will be done
All Research Plans Should Include
• Clear, concise hypotheses or research
questions
• Well-specified relations between hypotheses,
measures, and independent & dependent
variables
• Clear description of data sources
• Detailed descriptions of data analysis
procedures
The Research Plan Should Describe
• Setting
• Population and sample
– Sampling plan: inclusion and exclusion criteria
– Size (power issue) and attrition
– External validity
• Measures
– Outcomes
• Proximal and distal
• Sensitivity
– Other measures
– Reliability and validity
Work May Include
• Original data collection with appropriate
statistical analyses
• Secondary data analysis of existing datasets
• Secondary data analysis complemented by
primary data collection
• Meta-analyses designed to determine
moderators or moderators of effects
Primary Data Collection
•
•
•
•
•
•
Sampling strategy
Sample characteristics
Variables to be measured
Procedures for data collection
Procedures for coding data
Describe links to secondary data if relevant
If observational data are collected
• How inter-observer reliability maintained
• How data be coded
• How will data be quantified to support
prediction of relation between what was
observed and outcomes of interest
Secondary Data Use
• Database(s)
–
–
–
–
Sample characteristics
Variables to be used
Access to and permission to use the data
Linking if multiple datasets used
• Malleable factor
• Outcome variables
• Mediators & moderators
Exploration Work is Non-Causal
• Descriptive analysis
• Statistical correlational analysis
– Can include more complex methods to address
selection issues
• Mediation analysis
Meta-analysis
• Criteria for including or excluding studies, and
rationale
• Search procedures used
• Coding scheme & procedures for extracting
data from studies
• Procedures for ensuring reliability of the
coding
Meta-analysis
• Demonstrate that sufficient numbers of
studies are available to support meta-analysis
• Demonstrate that relevant information is
reported frequently enough that a database
can be constructed
• Clearly describe effect size statistics,
associated weighting functions, procedures
for handling outliers, and any other
adjustments
Research Plan: Analysis
•
•
•
•
•
•
•
•
Describe how analysis answers research questions
Show your models
Address clustering
Describe how missing data will be handled
Check for equivalency of groups at start of study
Track attrition and check for attrition bias
Describe sensitivity tests of assumptions
Describe analysis of qualitative data and links to
quantitative analysis
Personnel Section
• Describe key personnel
– Link each person and their expertise to their role in
project - show that every aspect of project has
person with expertise to do it
•
•
•
•
Methodologists: show expertise in particular method to be used
Substantive person for all issues addressed
Do not propose to identify and hire key people after grant awarded
Project management skills
– Give time contribution for each - show that every
aspect has enough time from expert
• Orient CVs same way – specific to project
– 4 pages plus 1 page for other sources of support
Personnel Requirements
• Publication record and projected publications
from this grant are considered
• Developers should discuss past success
getting developed interventions evaluated
• If previous IES grant, discuss results
• Evaluations require attention to objectivity
should a developer or persons with financial
interest be involved
– Efficacy projects: address how objectivity maintained
– Scale-Up: Independent evaluation: developer can
provide routine implementation support
Personnel Strategies for PI
• Senior Researcher
– Show adequate time to be PI
– Make credentials clear: not all reviewers may know
• Junior Researcher as PI
– Show adequate expertise not only to do work but to
manage project
– Reviewers may be more comfortable if you have
senior person(s) on project to turn to for advice
Resources
• Show the institutions involved have the
capacity to support the work
– Do not use university boilerplate
• Show that all organizations involved
understand and agree to their roles
– What will each institution, including schools,
contribute to the project
– Show strong commitment of schools and districts
– Have alternatives in case of attrition
Resources (continued)
• Appendix C should back this up with
– Detailed Letters of Support from research
institutions, States, districts, schools
• Data issues
– Document permission to use and access to
confidential data (letters in Appendix C)
– Show familiarity with data – show that it can be
used to do the proposed work
– If merging datasets, show that it can be done
Appendices
• Appendix A (15 page limit)
– Figures, charts, and tables
– Examples of measures
– 3 pages to address past reviewer comments or to
argue that a proposal is a new submission
• Appendix B (10 page limit)
– Examples of materials used in an intervention or assessment
• Appendix C (no page limit)
– Letters of agreement (districts, schools, data providers, other
partners, consultants)
– Clearly state responsibilities of the writer
Budget and Budget Narrative
• Provide a clear budget and budget narrative
for overall project and each sub-award
• IES Grants.gov Application Submission Guide
describes budget categories
• Check RFA for specific budget requirements
for Research Goals and Grant Programs
• Ensure agreement among Research
Narrative, Budget, and Budget Narrative
Award Information
– Secondary data analysis or meta-analysis:
• Typical $100,000 to $300,000 per year (direct
and indirect)
• Maximum 2 years and $700,000
– Primary data collection and analysis (with
or without a secondary data analysis):
• Typical: $100,000 to $400,000 per year
• Maximum 4 years and $1,600,000
Grant Submission
•
•
•
•
•
Make sure your institution is registered on grants.gov
Complete your online forms and upload PDFs
Authorized representative completes the process
Submit by 4:30:00 EST on deadline – earlier is safer
If problems uploading
–
Contact Help Line 1-800-518-4726 and get a case number
• You should receive four emails
–
–
–
–
Grants.gov: assigns you a number that starts with GRANT
Grants.gov: your application is validated or rejected due to
errors. If the latter, correct and resubmit until validated.
Dept. of Ed: retrieved your application from Grants.gov
Dept. of ED: assigns you a number that starts with R305 or R324
Application Review)
•
•
•
•
Carried out by the IES Office of Standards and Review
Compliance screening for format requirements
Responsiveness screening to topic and goal requirements
Assigned to review panel
– 2-3 reviewers (substantive and methodology)
– If scored high enough, application is reviewed by full panel
• Many panelists will be generalists to your topic
• There will an expert in every procedure you use
– Overall score plus scores on Significance, Research Plan,
Personnel, and Resources
• Resubmissions encouraged: address comments
Peer Review Process Information
http://ies.ed.gov/director/sro/peer_review/index.asp
Notification
• All applicants will receive e-mail notification
of the status of their application
• All applicants receive copies of reviewer
comments
• If you are not granted an award the first
time, plan on resubmitting and talk to your
program officer
Final Reminders
•
•
•
•
Start early
Read the Request for Applications
Talk with the program officer
Start the online submission process early
ies.ed.gov
Allen Ruby
allen.ruby@ed.gov