Computing Education Research
Anders Berglund
Informationsteknologi
Uppsala universitet
Uppsala
anders.berglund@it.uu.se
Informationsteknologi
Computing education research at
Uppsala University

How do our students understand computer
science concepts?

How to teach computer science?
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
This talk

What can we say about how grading in an
internationally distributed project-based course?
and

Why and how to do research in computing
education?
Two parallel
stories
Anders Berglund, Department of Information Technology
My secret agenda:
Inspire to Computing
Education Research
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
What does it mean to learn
something?
Unfortunately (?)….
 General
case:
A “meaningless” question
 It
all depends on
“what you mean by learning”
or “how you see things”
Anders Berglund, Department of Information Technology
Informationsteknologi
Quantitative/Qualitative research

Quantitative research is grounded on
•

“… the assumption that features of the social environment
constitute an objective reality … collecting numerical data on
observable variables”
Qualitative research is grounded on
•
“… the assumption that individuals construct a social reality in
the form of meanings and interpretations. … studying …
intensively in natural settings”
Implications for the role of the researcher, the concept of
evidence, interpretation etc.
(Gall, Borg & Gall, 1996)
Anders Berglund, Department of Information Technology
Informationsteknologi
A research
approach/methodology/
framework

Offers a way to perform research in
learning

Organizes “ways to see things”

A lens with a certain focus


With a specific research approach:
Some issues get clearer, others blurred
Offers theoretical stand on learning,
ways to see possibilities and limitations,
opens to communicate with other
researchers etc.
Anders Berglund, Department of Information Technology
Research approach/methodology/framework
Phenomenography
Informationsteknologi
(Marton & Booth, 1997)


Takes the learners’ perspective.
Aims at analysing and describing the variation
in students’ experience (understanding,
learning).

A empirical, qualitative research approach

Data is often collected through interviews

Outcome: A few qualitatively different ways,
in which something is understood within a
student cohort
Examples: TCP, experience of being graded
Anders Berglund, Department of Information Technology
Informationsteknologi
Phenomenography
The students
study TCP
TCP
The researcher studies
the different ways in
which the students
understand TCP
Researcher
Students taking a course
Anders Berglund, Department of Information Technology
Informationsteknologi
Why the learners’ perspective?

A teacher’s understanding of how her
students understand and learn about
something (CS concepts, for example
TCP) is a good tool for improving
teaching.

A change that is not perceived as “good”
by the students does not improve
learning.

Example: Grades are not the driving force for
most students who take (a certain) project
course.
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
Background: The Runestone initiative
A project course in computer systems




A project-based course in distributed systems,
real-time programming and computer
communication.
Third/fourth year students majoring in CS.
International collaboration for students who do
not go on exchanges.
Experience of collaboration over ICT tools.
Anders Berglund, Department of Information Technology
Informationsteknologi
Project course in computer systems
The Runestone project
USA
Communication
by e-mail and chat
• 3 + 3 advanced CS students per team
• 16 teams in total
• No lectures
• Tutoring by e-mail and chat
Anders Berglund, Department of Information Technology
Sweden
Informationsteknologi
Student project



Student project: Produce a software system to control a
(modified) Brio labyrinth from any Web-browser.
The task demands computer communication solutions.
The task requires collaboration within the team of 6.
Anders Berglund, Department of Information Technology
Informationsteknologi
Grading in Runestone



Both the process and the project are graded, in
relation to the teams’ own plan
Process grade is based on weekly meetings
Components of grade:
•
•
•
•


Team performance, in relation to the team’s own plan
Individual contribution
Peer evaluation
The instructor’s decision.
Team members are graded by “their” instructor
Different grading schemes in Sweden and US
•
•
Sweden: pass/fail
US: A to E
Anders Berglund, Department of Information Technology
Problematic???
Informationsteknologi
Analysing the grading in Runestone



Teachers’ distribution of grades
(quantitative)
Peer evaluation
The students’ evaluation of each others’
contributions
(quantitative)
Students’ experienced purpose of being graded
(qualitative)
Anders Berglund, Department of Information Technology
Informationsteknologi
Grades

Grades awarded by the instructors, according to
the Runestone scheme (Max = 100, Pass ≈ 60)
To all students
83,61
To Americans
81,55
To Swedes
85,05
Anders Berglund, Department of Information Technology
Informationsteknologi
Peer evaluation

Each student awarded USD 120.- to his
team-mates
From Swede to Swede
22,25
From Swede to American
18,79
From American to American
20,07
From American to Swede
20,07
Then, what is the driving force?
Anders Berglund, Department of Information Technology
Informationsteknologi
The experienced purpose of being
graded
Cat
Getting a
good grade …
Focus is on
1.
… has a value on its
own
The grade per se
2.
… is a tool to reach
other aims
The benefits of a
good grade
3.
… is sub-ordinated to
other aims
Anders Berglund, Department of Information Technology
Me and the team
My team and
other teams
Informationsteknologi
Results on grading

Getting a good grade is not the driving force for
most students in this project.

”Me in the team” or ”My team in front of other
teams” is often important.

How generalizable are these results?
•

Research in computing education research are normally
situationally bound
How can we use this in our teaching?
•
Both ”hard” results and insights gained by doing the
research are useful.
Anders Berglund, Department of Information Technology
Informationsteknologi
Computing Education Research
1.
Background
•
•
2.
Results concerning grading
•
•
•
•
•
3.
A research approach?
Phenomenography
The setting
The teachers’ grades
The students’ peer evaluation
The experience of being graded
Results concerning grading
Summary
•
•
•
Computing Education Research
Some research results
Choosing a research approach
Anders Berglund, Department of Information Technology
Informationsteknologi
On Computing Education Research



Theoretically sound research in students’
learning in Computing can serve to improve
teaching.
Different research approaches offer various
contributions to our understanding of students’
learning.
The perspective on ”reality”, what can be
studied, what can be known, what the
researcher’s role is, how research is performed
etc. varies.
This talk: Some examples of
qualitative research
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach
Constructivism


A family of traditions
Jean Piaget, 1896 - 1980
•
•
•
•

Reality is rejected or irrelevant
Knowledge is constructed by each individual
No firm methodology
Passive learning will fail
Extremely influential in school teaching
Anders Berglund, Department of Information Technology
Informationsteknologi
Empirical results from constructivism
in CS Education



Students construct rules for parameters. They
are only sometimes successful. (Fleury, 1991)
Students construct their own understanding of
variables. (Paz, 1996; and others)
Software visualization in itself does not help
students understanding (Mulholland, 1997)
Anders Berglund, Department of Information Technology
Informationsteknologi
Applications of constructivism in
Computer Science Education






Think twice when using visualizations
Explicitly teach the model of the computer
Don’t start with abstractions
Teach planning, teach to avoid “bricolage”
Don’t run to the computer
Organize “closed labs”
(from Ben-Ari, 2001)
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach: A
socio-cultural perspective
Thinking/Learning is not influenced by the
environment
⇔
Thinking/Learning is an interaction between the
individual and the environment


A family of traditions in research into learning
Common “source of inspiration”: Lev Vygotsky
(1896 – 1934)
Anders Berglund, Department of Information Technology
Informationsteknologi
Applications of the soci-cultural
tradition



Why do teams of students interpret a
programming task so differently? (Holland &
Reeves, 1996)
Why do our students hand in “incorrect”
programs? (Ben-David Kolikant, 2005)
The example: Open source community •
Linux
Anders Berglund, Department of Information Technology
Informationsteknologi
Example, research approach:
Critical enquiry

Critical enquiry (research with “a mission”, often
to address power imbalances)
•


Feminist research
Cannot be defined in ontological stand or
research methodology
Instead, it questions fundamental principals and
values and aims to implement changes.
Anders Berglund, Department of Information Technology
Informationsteknologi
Feminist research, an example of
critical research

Are there factors within computer science itself,
that preserves the currently dominating gender
structure? (Björkman & Trojer, 2002)
“We consider it of vital and decisive importance that
gender research is done from within computer science.”
“Such research on the core of computer science and its
knowledge production would serve to enrich computer
science as well as education within computer science.”
Anders Berglund, Department of Information Technology
Informationsteknologi
Selecting a research approach
Why phenomenography?
A personal view

Appropriate for the research questions
•
•

“Close to” learners
•
•
•

•
•
Results “talk to” computer scientists
Supports deployment of results in teaching
Relevant for recognition of CSE research within CS
Competence at hand/tradition
•

Students’ perspective
A way for students to “talk” to teachers/organisers
Data stems from individuals
Computer science is in focus
•

“Complex” “answers” desirable
Statistical “answers” hard or impossible to get
Shirley Booth
I like it 
Anders Berglund, Department of Information Technology
Read!
Create a network!
Informationsteknologi
Literature
Read!

Clancy, M., Stasko, J., Guzdial, M., Fincher, S., & Dale, N.
(2001). Models and Areas for CS Education Research. Computer
Science Education, 11(4), 323-341

Fincher, S., & Petre, M. (2004) Computer Science Education
Research, London, UK: Taylor & Francis

Berglund, A., Daniels, M. and Pears, A. (in press). Qualitative
Research Projects in Computing Education Research: An
Overview. To appear in the Proceedings of the 8th Australasian
Computer Science Education Conference, Hobart, Australia.

Berglund, A. 2005. Learning computer systems in a distributed
project course: The what, why, how and where. Acta Upsaliensis
Universitatis. Uppsala Dissertations from the Faculty of Science
and Technology 62
Anders Berglund, Department of Information Technology
CeTUSS
Informationsteknologi


CeTUSS ska verka för anpassning av
teknikutbildning till samhällets och
studenternas behov.
Genom att utveckla och sprida
information om lärandemiljöer som är
•
•
•
•

Create a network!
Personligt meningsfulla
Socialt relevanta
Gränsöverskridande
Baserad på samarbete (lokalt och internationellt)
Workshops, kurser etc.
Anders Berglund, Department of Information Technology
Informationsteknologi
Anders Berglund, Department of Information Technology