ALT-C
2012
Learning Technology
Research Group
Online Learning
Environments and the
Pedagogy of Computer
Programming
Paul Neve, David Livingstone,
Gordon Hunter, James Orwell
Learning Technology Research Group
Faculty of Science, Engineering and Computing
Kingston University
Computer programming is
painful for everyone…!
 Students
 "Programming is boring…"
 "Programming is difficult…"
 "I don't understand
programming…"
 "I'm no good at
programming…"
 "Can I choose a combination
of modules for my
computing course that
doesn't involve
programming…?"
ALT-C
2012
Learning Technology
Research Group
 Lecturers
 "They just don't get
programming no matter
what we teach or how we
teach it"
 "Some students just aren't
equipped to be
programmers"
 "They might have passed
the programming modules
but they've no idea how to
tackle the programming
component of their final
year project"
Practical Programming
ALT-C
2012
Learning Technology
Research Group
 A first year second semester programming module at Kingston
University
 Follows a first semester module that uses* Java to teach
fundamental programming concepts to beginners
 Sets out three aims:
 To develop students' enthusiasm for practical programming
 To enhance students' experience with programming environments
 To develop students' confidence in their ability to write programs
 Two flavours, one for Computer Science students, one for
Information Systems students
 In 2011 almost half the IS cohort failed even after resits had
been taken into account 
* or rather, attempts to use…
Enter NoobLab
ALT-C
2012
Learning Technology
Research Group
 NoobLab: an online learning environment designed
specifically for teaching programming
 Several "angles":
 An immersive, engaging and flexibly available learning experience
 Deliver some some of the feedback that a tutor might give in a
programming workshop
 Monitor progress, learn about how they learn, and inform the
pedagogy of programming
NoobLab and Practical
Programming
ALT-C
2012
Learning Technology
Research Group
 A holistic approach:
 Module redesign – content, activities and assessment used
NoobLab as the primary delivery tool
 All summative assessment involved a practical programming
activity within the environment:
 Four "Small Tests": simple programming tasks
 Three coursework assignments: Hangman, Tic Tac Toe and Connect 4
 One "Big Test": making live changes to the games
 New features added to the environment as content dictates
 Tail and dog in perfect harmony!
NoobLab demo
If you want to follow along…
http://nooblab.kingston.ac.uk
Username: anything you like!
Password: altc2012
ALT-C
2012
Learning Technology
Research Group
The results…
ALT-C
2012
Learning Technology
Research Group
 78% of students who completed a majority* of summative
assessment tasks achieved a passing grade
 Significant correlation between time spent in the NoobLab
environment and final grade (R = 0.749, p < 0.001)
 Students' end of module survey
 An average 90% satisfaction factor+ about the NoobLab environment
 "I did better on this module than I expected" - 83%+
 NoobLab environment frequently complimented in free-text
feedback, e.g. "what was the best thing about the module?"
* 4 or more of the 8
assessment tasks
+ A "satisfaction factor" was determined by giving the student a positive
statement e.g. "I did better on this module than I expected to". Students
assigned a mark where 1 indicated strong disagreement with the positive
statement, and 5 strong agreement – i.e. higher numbers representing a
better response. A maximum possible score for each based on number of
responses was calculated, and the final score actually achieved is
expressed as a percentage of this maximum possible.
Building on the
experience…
ALT-C
2012
Learning Technology
Research Group
 Teaching the skill of "thinking like a programmer is key…"
 …make the learning activities more visual
 …avoid abstractions and examples that don't speak students' own
language (i.e. maths!)
 Carol the Shopping Trolley Wielding Robot
 Inspired by Logo and Karel the Robot [1, 2]
 New materials coming up for Practical Programming and other
modules
[1] R. E. Pattis, Karel the Robot: A Gentle Introduction to the Art of Programming, 1st ed. John Wiley & Sons, Inc. New York, NY,
USA, 1981.
[2] B. W. Becker, “Teaching CS1 with Karel the Robot in Java,” SIGCSE Bulletin, vol. 33, no. 1, pp. 50–54, ACM, New York, USA,
2001.
Patterns in the chaos…
ALT-C
2012
Learning Technology
Research Group
 Big Brother NoobLab is watching you; logs…




Navigation through learning material
Interactions with learning material, e.g. responses to quiz questions
Any code executed – and any errors that result
Exercises attempted – and whether the criteria for the exercise was
met
 Common patterns between students are already evident in this
data…
 The "SOS" pattern
 The "Rosetta Stone" pattern
 We expect to find more given further analysis
 Patterns can be used as triggers for feedback
The "Rosetta Stone"
Pattern
#2 Student goes
back to lesson 2
and lesson 3
material
#1 Student fails
first attempt at
Small Test 2
#3 Student goes
all the way back
to the lesson 1
material
#3 Still can't
crack it… 
ALT-C
2012
#4 Student advances to
lesson 4… particularly
hovering around part 4
of this lesson…
Learning Technology
Research Group
#5
EUREKA!

The "Rosetta Stone" pattern happens when a student who is
struggling with an exercise succeeds after examining a certain
piece of content

If several students exhibit the same pattern, it might be used
as a possible feedback trigger…


"You seem to be having problems with this exercise… perhaps
you should check out Lesson X…"
If the "Rosetta Stone" pattern is unexpected, it might also be
a trigger for a content author to re-examine the material
For more information
ALT-C
2012
Learning Technology
Research Group
 Email Paul Neve
 paul@kingston.ac.uk
 Source code (no documentation yet, though…!)
 svn://wlab.paulneve.com/NoobLabPP
 KU Learning Technology Research Group blog
 http://ltrg.kingston.ac.uk
 Papers
 HEA STEM Conference 2012 – Developing Virtual Programming
Laboratories to Inform the Pedagogy of Programming
 http://www.heacademy.ac.uk/assets/documents/stemconference/Computing1/Paul_Neve_David_Livingstone.pdf
 IC12: International Symposium on the Intelligent Campus – Nooblab:
An Intelligent Learning Environment for Teaching Programming
(forthcoming, December 2012)
ALT-C
2012
Learning Technology
Research Group
Questions…?