Research Design. The second research question will be addressed through a combination of quantitative
and qualitative research methods. A quantitative, quasi-experimental design will be used to determine
the overall effectiveness of teaching CS concepts under three types of game conditions: (a) games alone,
(b) games with fictional settings, and (c) games with stories. At each research site in Arizona and New
Jersey, approximately 10 participants will be recruited for the pilot and 35-40 participants recruited for
the final study. Participants will be randomly assigned to one of the three treatment groups. Each group
will attend six sessions of game play, each lasting approximately 1- 1.5 hour, for a total of 6-8 hours of
instruction and assessment. The sessions will be held over six weeks at the respective university campus
or at afterschool sites in the local community. A trained facilitator will lead each session and a second
team member will be responsible for data collection. All participants will complete an initial
questionnaire to collect background information and a pretest, a measure of computational thinking
that will developed specifically for the study (assessments are described in more detail below). At the
study’s conclusion, all participants will complete a post-test of computational thinking and a summative
survey of engagement in the sessions overall. The pre and post testing may be done at a separate time
or integrated with the game play sessions (See Table 1 below).
Table 1
Group A
Game Only
Preliminary
Data
Collection
Game Play:
Concept 1
Game Play:
Concept 2
Game Play:
Concept 3
Final Data
Collection
Background
questionnaire
Concept
Application
Concept
Application
Concept
Application
Post test of
computational
thinking
Pretest of
Computational
thinking
Engagement
survey
Engagement
survey
Engagement
survey
Game Play:
Concept 1
Game Play:
Concept 2
Game Play:
Concept 3
Group B:
Game +
Fictional
Setting
Background
questionnaire
Pretest of
Computational
thinking
Group C:
Game +
Story
Background
questionnaire
Pretest of
Computational
thinking
Concept
Application
Concept
Application
Concept
Application
Engagement
survey
Engagement
survey
Engagement
survey
Game Play:
Concept 1
Game Play:
Concept 2
Game Play:
Concept 3
Concept
Application
Engagement
survey
Concept
Application
Engagement
survey
Concept
Application
Engagement
survey
Summative
engagement
survey
Post test of
computational
thinking
Summative
engagement
survey
Post test of
computational
thinking
Summative
engagement
survey
Schedule with Major Project tasks & Dissemination Strategies
Table 2
Term
Fall 2014
Spring 2015
Project Goals
Identify, elaborate and evaluate
potential game concepts
Design story and game prototypes
Summer 2015
Develop final version of games,
fictional settings, and stories
Conduct pilot test to finalize research
methods and create data collection tools
Fall 2015 - Spring
2016
Conduct full testing as detailed in
Section D
Begin data analyses
Summer 2016
Complete data analyses
Prepare final report
Prepare recommendations for
integrating story and games, with
revised story/context-embedded
games
Plan and conduct teacher workshop
Dissemination
Create website and initiate blog
posts
Publication #1: Issues and
strategies associated with
integrating story and educational
games
Potential paper submission to
International Journal of GameBased Learning
Weekly update of website and blog
Publication #2: Designing CS
games for girls
Potential paper submission to
International Journal of Gender in
Science, Engineering, and Technology,
Weekly update of website and blog
Publication #3 - Pilot test results
Potential paper submission to
ACM ACM Technical Symposium
On Computer Science Education
Weekly update of website and blog
Publication #4 - Final study
findings
Potential paper submission to
Computers and Education
Weekly update of website and blog
Teacher Workshop materials
available for distribution
Game materials available for
distribution
NSF Grant Award
NSF-AISL Grant# 1421806 was awarded to this proposal in August 2014. The team has been working
on the project and is on target with the planned research. However , this work is very time consuming.
NSF clearly states that it expects the university to grant the faculty responsible, with time to do this
work, as part of their normal responsibilities. Teaching a full 12 credit hours every semester is not in
keeping with this expectation by either NSF or this faculty person. The NSF grant specifically does not
allow to pay for faculty time during the semester, but only for a Summer research stipend.
The NSF Proposal and Award Policies and Procedures Guide (Chapter II, page 14).
(a) Senior Project Personnel Salaries & Wages Policy
“NSF regards research as one of the normal functions of faculty members at institutions of higher
education. Compensation for time normally spent on research within the term of appointment is deemed
to be included within the faculty member's regular organizational salary. As a general policy, NSF limits
salary compensation for senior project personnel to no more than two months of their regular salary in
any one year. This limit includes salary compensation received from all NSF-funded grants. This effort
must be documented in accordance with the applicable cost principles. If anticipated, any compensation
for such personnel in excess of two months must be disclosed in the proposal budget, justified in
the budget justification, and must be specifically approved by NSF in the award.23 These same general
principles apply to other types of non-academic organizations. NSF award funds may not be used to
augment the total salary or salary rate of faculty members during the period covered by the term of faculty
appointment or to reimburse faculty members for consulting or other time in addition to a regular full-time
organizational salary covering the same general period of employment. Exceptions may be considered
under certain NSF programs, e.g., science and engineering education programs for weekend and
evening classes, or work at remote locations. If anticipated, any intent to provide salary compensation
above the base salary must be disclosed in the proposal budget, justified in the budget justification, and
must receive the prior written approval of the cognizant NSF Program Officer.”