1
WRITING PROPOSALS WITH
STRONG METHODOLOGY
AND IMPLEMENTATION
Kusum Singh, Virginia Tech
Gavin W. Fulmer, National Science Foundation
Goals
2



Encourage you to seek funding from NSF for your
research.
Help you develop rigorous methodology, data
collection and analysis plans that will make your
proposal competitive.
Help you consider the level of detail appropriate
for implementation projects.
3
Describing Your Project’s
Methodology
Expectations for Methods in DRL
4





The DRL Programs welcome research using a variety
of evidence.
The program is open to qualitative, quantitative, and
mixed methods.
Methods must be rigorous and appropriate to the
proposed research questions or hypotheses.
Design, methods, and analytic techniques should have
a coherent and logical link.
Research methods should be described in adequate
detail.
Details of Methods to Include – 1
5

Provide a rationale for your research design
 Make
it clear how the research design and analyses
answer the research questions (RQs)

Include a description of study population and
sampling method, sample size, expected effect size
 Power
analysis should inform sample size decision
Details of Methods to Include – 2
6

Instruments or protocols to be used
 Validity,
reliability, and triangulation of measures
 Reviewers are cautious about development of new
measures

Data analysis plans
 Statistical
Models, procedures for analysis of
text/video/observation data

All of these need to have a rationale for them that
connects to your RQs
Quantitative research
7



Research design (e.g. experimental, quasiexperimental and non-experimental designs, issues
of internal & external validity)
Measurement (e.g. data to be collected, constructs,
measures, validity & reliability of measures)
Data analysis (e.g. statistical decisions, models &
procedures)
Qualitative Research
8





Identify the methodology as a systematic research
design (e.g. case study, discourse analysis etc.)
Describe how and what data will be collected
Consider issues of validity, and triangulation
Include plans for analysis of textual data (coding
scheme, themes etc.)
Find good balance between planned approach to
analysis and flexibility to respond to findings
Find the Expertise You Need
9

Content experts are not necessarily methods
experts; so partner with research methodologists
 Sooner
is better than later (in proposal writing stage)
 Especially necessary if design is complex or you use
innovative methods

Find a colleague
 As
co-PI or as consultant
Common Missteps in Methods -1
10

Overly generic language and description
 “We
will use constant comparative methods.”
 “We will use HLM.”

Lack of consistent link between the theory, the RQs,
the data collected, and the analyses
 Reviewers
will notice.
 Methods and planned analyses inadequate to answer
RQs.
 Try developing a matrix of RQs, data/measures, and
analyses – even if only for you during planning
Common Missteps in Methods -2
11

Too little or too much data without clear analysis
plan
 Reviewers

will wonder if you understand the task.
Method is novel and not well understood in field
 Needs
more detail, examples and citations to justify
that it is appropriate
Summary of Main Points
12






Articulate clearly your research questions or research
hypotheses
Think about the most appropriate and rigorous methods
to answer your research questions
Give a clear and concise description of the research
methods
Include your rationale for research design decisions
Include a research methods expert in your team
Articulate clearly why your research is important and
how it would contribute to theory and practice
13
Describing an Implementation
Details of Implementation
14

There are important implementation issues that
need to be addressed if your project includes
 Curriculum
development
 Professional development
 Interventions
For All Implementation Projects
15

Consider the method(s) used to gauge the quality of
the implementation
as “Fidelity of Implementation” (FOI),
Intended/Enacted Curriculum, or other approaches
 Whether


Be specific on the STEM content, ages/grades,
settings
Be clear on the roles of the team
 Who
will lead PD or curriculum, who will oversee
implementation?
 Who will collect evaluative data on implementation?
Issues for Curriculum Development
16



Specify the STEM content of interest and age
range(s) for which you are developing curriculum
Specify the role(s) of the PI team, outside experts,
participating teachers, or others
Identify the process for development, revision, and
field-testing
 Provide
justification for the design process you will use
 Make sure the measures match the materials/curriculum
under development
Issues for Professional Development
17

Be specific on the professional development (PD)






Identify the model for PD you will use




STEM content, grades, and school settings
Role(s) of the PI team, outside experts, participating teachers, or others
Format of professional development (e.g., online, workshops)
Duration and location of PD
Evaluation
Train-the-trainer
Master teacher
Professional Learning Community
Provide justification for the model, the format, and your team’s
expertise
Issues for Intervention
18

Describe development history and its prior use
 Provide
evidence, if any, for intervention’s potential
effects

Describe in detail:
 Population
and sample;
 Setting, duration, and content;
 Design process, if the intervention will be revised
iteratively
Consider Generalizability
19

If you are developing a new curriculum/PD model:
 How
will the intervention, curriculum, or the professional
development developed in your setting apply to new
settings that may differ from the study?

If you are applying an intervention, PD model, or
curriculum adopted from another setting:
 How
well does that intervention apply to your setting?
 Will promising prior results be replicable in this project?
Evaluation Plan
20

Evaluation should be useful for improving the
research project
 Design
and content of the plan should be appropriate
to what would enhance or benefit the project



Formative or summative, internal or external may
be appropriate, depending on the project.
For example, advisory committees are appropriate
for the evaluation of projects.
Go to specific session on Project and Program
Evaluation later in the conference for more details.
21
Any Questions??
Don’t be shy.
22
THANK YOU!
Feel free to contact Kusum Singh for follow-up and tips for finding a good methodologist:
ksingh@vt.edu