Paper on the 17th Improving Student Learning For the Twenty-First Century Learner.
Imperial College, London, UK 7-9 September 2009.
Problem based learning in Higher Education and new approaches to
assessment as a consequence of new formal regulations
Lone Krogh, Associate Professor (lkr@learning.aau.dk)
Annie Aarup Jensen, Associate Professor (aaj@learning.aau.dk)
Department of Education, Learning and Philosophy, Aalborg University
September 2009
Introduction
Aalborg University has more than 30 years of experiences in educating students through
the use of problem based learning strategies. The pedagogical model is described in
keywords as problem-oriented, project-organised cooperative learning in groups. Within a
curricular framework based on scientific or professional fields (often with a considerable
degree of interdisciplinarity) the members in the study groups choose and formulate problems and questions, which are investigated and analysed by the students and they take
steps towards resolving, making use of existing sources, methods and theories. This work is
documented in a project report drawn up by the group and it becomes the basis for oral
examination at the end of each term and semester. In general this work process has
similarities to research work, but it is also related to work processes such as development
projects, project management and in relation to organisational learning. It is therefore
considered a useful preparation for the candidate’s working life, both as regards the
knowledge dimension and the development of collaborative competences.
Until recently examinations of the project work were held in groups on the basis of the
project report in order to ensure alignment between goals, learning activities and assessment
form. However, in 2006 the government announced that group examinations would no
longer be permitted. As a result students are now allowed to do study work and write
reports in groups, but they are to be examined and assessed individually, i.e. without the
presence of other students from the group.
In this paper, we will investigate some of the consequences of these new regulations for
assessment. The research questions will address the question of alignment between group
study and individual examination; the pros and cons of the individual examinations
1
including the aspect of validity; the influence on student behaviour, if any; and finally the
academic results of the new regulations. The results of a qualitative survey among study
leaders will be compared to previous investigations made since the ban of group exams.
Based on selected theoretical approaches to teaching, learning and assessment we wish to
discuss the result of our research, the consequences of the changes in assessment forms, as
well as the measures taken at the university in order to obtain new valid assessment forms.
Finally, suggestions will be made to some alternative assessment forms.
The principles for problem-oriented, project-organised cooperative project
work in Denmark
We will start with a short description of the context for the problem of this article. The
didactics of Aalborg University have been developed from the original principles
formulated around principles such as: problem orientation and inter-disciplinarity,
exemplarity, open curriculum and experience-based learning, peer learning, and
cooperative learning in groups (Enemark and Kjærsdam, 1994), (Kolmos, Fink, Krogh,
2004).
Problem based learning, PBL, project work etc. are concepts used widely and with very
different meanings, integrated in varying educational designs and with different
instructional goals. The original idea and theoretical foundation of the problem oriented
project work in a Danish context was formulated by Knud Illeris in 1974 in his seminal
book “Problem orientation and participant direction. An introduction to alternative
didactics.”1
The founding principles of problem oriented project work
Based on an analysis of society’s need for education Illeris lists three categories of
qualifications which appear to be necessary for the development of society: 1) Skills which
can be defined in direct relation to a given task or work process, 2) Adaptive qualifications
of a general character and comprising attitudinal characteristics (e.g. diligence,
perseverance, vigilance etc.) – combined with a willingness to apply these characteristics
in relation to work, to accept and subject to the existing work processes, 3) Creative /
innovative qualifications which may be divided into qualifications for scientific,
innovative work and qualifications for continuous renewal and being able to cooperate.
(Illeris: 32-35). Referring to Piaget’s theory of learning Illeris understands accommodative
learning processes as the prerequisite for creativity. With this point of departure Illeris
describes an expedient learning process which allows for the development of skills,
adaptive ability and creativity as a process which alternates between accommodative
1
The PBL model, implemented in Denmark was a result of needs for developing new more critical
approaches to teaching systems.
Theoretically the Danish PBL approach originally was based on ideas from the two German authors Negt
and Kluge (1975) and with references to C. Wright Mill (1959), Dewey (1933) and Freire (1970).
2
processes (the creation of new cognitive structures) and assimilative processes (the
incorporation of new material in the individual’s existing structures). Such alternating
processes are a precondition for student’s ability to acquire a holistic competence which
comprises the skills, the adaptive ability and the creative qualifications which according to
Illeris’ analysis was needed by society (Illeris: 76-77).
The alternative didactic concept – the problem oriented project work – was and is
characterised by:
 Problem orientation, which means that the point of departure is the subject related
knowledge, methods and theories relevant to the specific problem. This means that
interdisciplinarity becomes a leading principle.

Participant direction, which means that both the definition of the problem and the choice
of work methods must lie with the students as this is the only way to create the
possibilities for accommodative learning processes necessary to develop creativity and
flexibility.
“Because if you determine beforehand politically which problems should be the point of
departure of problem oriented education and how you are supposed to work with the problems,
you may well have transgressed the traditional borders between disciplines, but have at the same
time established new limitations which in the same way as the old ones will hinder the
accommodative learning which is the prerequisite for developing the desired creativity and
flexibility.” (Illeris: 82)
Furthermore, the problem in question has to be and be experienced as relevant for the
individual student in order to ensure the appropriate learning process,
“Accomodative learning is a demanding process which depends on commitment. You only
accommodate in situations which have significance for you to manage. Why else ‘take the
trouble’? In less significant situations you dismiss the problem or assimilate it, i.e. integrate it
into already established cognitive structures.” (Illeris: 82-83)
As a consequence of this the principle of participation in decision making becomes
important.

The principle of exemplarity which means that working with the important and
representative aspects exemplify the area of the discipline in question.

Group work. The students collaborate in groups about problem solving. In that way
they learn the difficult art of collaboration and leaderships of projects.
3

The communicative didactic, with primary focus on the forms of work and cooperation
within the group and the considerations regarding the choice of content.
“The communication in the group, the group’s reaching an understanding of an experienced
problem, is the second main factor. It is a democratic process and therefore significant partly as
a subject for negotiation, and partly as a source of insight.” (Illeris: 177)
The work process in practice
Ideally a problem oriented project work would go through the following phases,
 Selection of subject and the first problem raising
 Formulation of project (corresponds to ‘problem formulation’ often used in the PBL
terminology)
 Methodological reflections and decisions on how to solve the raised questions in
the problem formulation
 Project work (i.e. theoretical and empirical work, perhaps involving experiments)
 Production (of project report) (sometimes involving descriptions of work processes)
 Product evaluation and if necessary – product adjustment
As mentioned above problem oriented or problem based project work may be interpreted
and implemented in a number of different ways according to educational institutions,
disciplines, subjects, and learning goals. There may be varying degrees of free choice
regarding the specific problem, subject area, and method, and the project work may differ
in size, i.e. the students’ workload per semester. Furthermore, there may be vast
differences in resources allocated to the project work in terms of hours of supervision as
well as study rooms for the groups. Even though the facilities and resources for project
work may vary, the following model illustrates the elements which generally forms part of
the problem oriented project work at Aalborg University,
4
Fig. 1. Example of project-organised problem solving processes (inspired by Kjærsdam and
Enemark, 1994)
The project work is closely combined with lectures, seminars or laboratory work on
relevant subject matters. The actual organisation of the study programme depends on the
didactic analyses, performed by the study board and the teachers. The intentions are that
all sources and methods aim at supporting the students in their working processes. The
students’ work is facilitated by university teachers supervising their project work. It is
generally expected that students work in groups of 6 - 8 persons during the first year of
study, which later in their studies often shrinks into 3 or 2. Individual project study is
accepted, but the students are told that this minimizes the peer-learning.
The organisation of the studies can be illustrated as in this model:
5
Project related
courses
Project
Study courses and
other activities
Until 2007 group
examination
Now individual
examination
Fig. 2. General organisation of the Aalborg PBL model
As mentioned above there are many different kinds of project work, mirroring differences in
fields, subjects and study environments. Students in philosophy or anthropology may find it
difficult to recognise the kind of project study undertaken by for instance engineering
students. Therefore the model varies considerably in practice.
The teacher as supervisor
Each group is assigned a supervisor2, who through the process helps, challenges,
supervises, teaches and discusses with the students and assesses them. The supervisor’s
assistance is, however, important at several occasions, for instance in both the selection
and formulation of problems, methodological reflections and decisions, facilitation during
the working process and in explaining how exams will take place.
Group examination
In accordance with the founding principles behind the PBL-approach examinations have
been carried out in groups, in the form of a combination of oral presentations from each
study member, questions from examiner and external examiner and discussions on the basis
of the project report, thus forming an alignment (Biggs and Tang, 2007) between goals,
learning activities and examination form (i.e. evaluation of students learning outcome),
2
At the Natural – and Health sciences groups are assigned 2 supervisors at the first year. One supervisor
represents the subjects and one is primarily acting as process- supervisor.
6
where the study form and activities correspond closely with the form of examination and
objectives to be met.
As we have discussed earlier in this paper the intentions of the problem-based, project
organised group work are that the students learn together and from each other.
During the group exams the group and its members were tested in their abilities of
presenting and showing the problem definition analysis in the project, as well as their
abilities in both performing individually and cooperating with others during the exam
situation. The examiner (who has also acted as supervisor/facilitator for the group) and the
co-examiner or external3 examiner challenged and tested the students’ knowledge and
experiences, making sure that all students in the group got the opportunity to demonstrate
the knowledge and the understanding of essential relevant topics in the curriculum.
Effects of synergy were often displayed during group examinations when the students,
prompted by the questioning of the examiner and the co-examiner or external examiner,
were mutually inspired to respond in very qualified ways.
The students were assessed individually in the groups. Exams might often last several hours
dependent on group size4. This meant that the exams often became qualitatively very valid
exams.
Aalborg University has accumulated much experience in group examinations. A crucial
part of this is the awareness of the necessity of assessing the performance of both the
group and the individual students in the group. The obligation to assess the individuals
has always been emphasized in the official regulations for examinations in Denmark.
There are different ways of doing this. If the examiners were in doubt about the skills of an
individual student they might pose supplementary questions exclusively for him/her, and
in the Humanities at Aalborg University students have always been required to indicate
which parts and chapters etc. they have contributed to the project report in order to ensure
individual assessment regarding the written report. It has not been uncommon for
students in a group to be graded differently.
Ban on the group examination
In 2006, however, the liberal-conservative government announced that group examinations
would no longer be permitted at any educational institution in Denmark. Students were
welcome to do study work and write their reports in groups, but they were to be examined
and assessed individually, without the presence of the other students in their group5.
3
In the Danish educational system it is possible to operate with either internal or external censorship. The ratio between
internally and externally censored examinations is laid down by the Ministry of Science, Technology and Innovation,
and the study order for each education determines the rules regarding the specific examination.
4
For instance, 30 minutes per student would mean a 3 hour examination of a group of 6 students.
5
This in itself is a breach of the general principle in Denmark that all examinations are public.
7
The announcement provoked resistance from many quarters, especially from students,
their organisations, and many educational institutions, but, perhaps more importantly,
also from industry and business, thus underlining that the skills, knowledge, and
competencies claimed to be an outcome of the problem-based, group-organised project
work are highly sought after and valued by the employers.
Before continuing to present the results from the survey and discuss some of the
consequences from the ban on group examinations we will briefly illustrate some of the
previously documented learning outcome from the Aalborg PBL model.
Potentials and added bonus from the Aalborg PBL model
As mentioned cooperative study aims at strengthening certain aspects of the learning process
in higher education. It demands that students learn to cooperate on tasks and develop a plan
for the division of labour without losing sight of the basic questions they are investigating. It
means that much learning takes place through discussions and knowledge sharing where the
ideas and perspectives of each individual student are presented, confronted, discussed, and
perhaps integrated in a common understanding. Furthermore, cooperative study means that
students are trained to formulate and present the results of their studies and learning, most
often in the form of written reports. Study tasks like this establish a collective responsibility.
The work processes, which are going on during project work bear some resemblance to
research processes, and they are also related to types of work processes as they may be
seen in business, in particular in project management and in combination with
organisational learning. In this context there are similarities to the principles of Modus
Two research (Gibbons, 1994, Wenneberg, 2000). Especially in the last terms of a master
programme it gives students the possibility of transcending the barrier between the
cognitive and the affective learning when they experience the relevance of research in
relation to society and needs from the working places. This is achieved by asking the
students to investigate problems in theory and practice. This type of work can also be seen
as types of/or pre-forms to research-processes. However, the intention is not to turn all
students into researchers but to make it possible for them to develop skills that are useful
within their future professional and academic working life whether it takes place at
universities or in other workplaces nationally or internationally.
A comprehensive survey which was carried out among 10,000 former students at Aalborg
University and Roskilde University Centre, which is also working with a PBL-approach,
indicated that the pedagogic form and the cooperative studies seem to have supported the
Masters (Candidates) in developing skills and abilities of for instance communication,
problem solving, creativity, innovation and cooperation in solving workplace related
tasks, i.e. competencies, which have proven useful in job situations. This is documented
both in earlier studies (for instance Rasmussen 1991) and as mentioned in two surveys
(October 2002 and April 2003) among Masters graduated from Aalborg University and
Roskilde University and among 150 employers from public and private firms (Kandidat-
8
og aftagerundersøgelsen, 2003); (see also Krogh and Rasmussen 2004). A new survey
among Masters graduated from Aalborg University during the period 2001 to 2007 has
just been ended (www.Karrierecentret.aau.dk). This survey confirms the results from the
surveys in 2002/2003 in relation to the development of the above mentioned skills.
It is seen from these surveys that some of the intentions behind using this particular
pedagogic model seem to have been met according to the Candidates/Masters and their
employers. This can be said about the development of theoretical knowledge and
methodological knowledge and to some degree the ability to cross the traditional
disciplinary boarders. In this respect the pedagogical principles in the pedagogic model
seem to meet some of the demands rising from the development of society for knowledge
transcending the boundaries of the strict disciplinary knowledge, without loosing the
traditional disciplinary knowledge – still according to the employers and their employees.
These demands thus seem to be fulfilled at least in the problem based didactic as it has
been performed at Aalborg University and Roskilde University Centre.
Educational arguments for discussing exam forms
There may be many good reasons for discussing the forms and the functions of
examinations in education. In international educational research there are continuing
critiques about whether established methods of examination actually test the aspects of the
students’ learning which are intended and relevant (see for instance the overview in
Lauvås and Jakobsen, 2002) and Sadler (2005). It is also well known that assessment
methods and requirements probably have a much greater influence on how and what
students learn than any other single factor – see for instance Boud (1988), Gibbs (1999)
Lauvås og Jacobsen (2002), Cowan (2003). Boud argues that if you get the assessment
right, then you are likely to direct students’ activities appropriately. He states,
“If we want to encourage them (the students) to take a deep rather than a surface approach to the
development of practical skills, we need to design practical assignments intelligently. We need to
think not just about the assessment criteria but also about weighting, timing, agency and ‘fitness
for purpose’, with imaginative consideration of methods and approaches that can challenge
students, be inclusive and suit the topic, context, cohort and level” (Boud in Pickford and
Brown, 2006)
Actually it is this kind of research, which lead to much concern from many Danish
educational researchers, developers and practitioners because there potentially seemed to
be so much alignment between learning goals, the teaching and learning model and the
assessment form in group examination related to PBL, so valid exams were a possibility
and at the same time motivated the students to work collaboratively in groups.
9
Some consequences from the shift in exam forms – students’, teachers’ and
examiners’’ perspective
In the press release quoted above as well as in the ensuing debate the two ministers6
offered little in the way of arguments for the ban. They mainly insisted that “we want to
know which skills the individual has”, implicitly claiming that this can only be assessed
through individual examinations (without the presence of the other group members). The
fact that there at that time were not much systematic evidence about how group
examinations actually had been working (although there were much practical experience
in some educational institutions) probably made it easier for the ministers to avoid serious
debate and arguments about their claims. Some of the existing evidence can be read for
instance in a survey made by the Danish Evaluation Institute (2005) – see Krogh and
Rasmussen (2007).
An ongoing research project at Aalborg University, established in connection with the ban
on group examination, aims at comparing the impact of group examination and individual
examination on different aspects of the teaching and learning process. As mentioned
above teaching and study at this university has generally followed the principles of
problem-based project learning and group organisation, and the shift from group
examinations to individual examinations gave the opportunity to investigate the
consequences for study work and learning. The decision about individual examination
was implemented by the Ministry of Science, Technology and Innovation (which has the
responsibility for universities) by fall of 2007. However, a limited number of diploma level
students in engineering did in fact make the shift already in 2006, and for these students it
was possible to compare the experiences of individual and group examinations. Some
results from this first analysis (Kolmos and Holgaard 2007, p 10) were that,



Compared to students taking group exams, students taking individual exams found
that their academic skills were questioned and examined to a much lesser degree
Examiners find that individual exams give much reduced possibilities for asking
questions related to deeper academic understanding, and they find that the basis
for assessing individual performance is better in group exams.
Both students and examiners find that a number of skills are less likely to be assessed
in individual exams than in group exams. These include arguing for choices of
theories and methods, discussing different solutions to problems, transferring
knowledge from the study project to other contexts, engaging in dialogue and
teamwork.
This research project has been followed up by investigations among students,
supervisors/internal examiners and external examiners in 2008 (Kolmos and Holgaard,
6
The Minister of Education and the Minister of Science, Technology and Innovation
10
2009). It concludes that it to a certain degree is not possible to test students’ skills
concerning project competences such as analytical, methodological and communicative
skills in solving technological problems. It is to a larger degree easier to test the individual
students’ basic knowledge. In that way there seems to be only a minimum of alignment
between goals, learning method and assessment form and consequently there is probably
problems concerning the validity of many exams.
Furthermore, the majority of examiners (internal as well as external) and students claim
that they prefer group based exams to individual exams. It should, however, also be
mentioned that the investigation shows that still more students seem to be positive
towards the individual exam in 2008 than they were in 2006. An explanation to this might
be that some of the students who participated in the survey have never tried group exam
and thus have no basis for comparison.
One argument brought forward a few times from Government to support the ban on
group exams was that students taking group exams tend to obtain grades resembling
those of the other students in the group, whereas grades obtained in individual exams
were more dispersed. This was presumably seen as an indication that no individual
assessment had taken place. There is, however, another more plausible and learning
oriented interpretation of this fact. When students cooperate intensively on a common
project in much of their daily study work they will learn from each other and the levels of
skills and knowledge among the students in the group may approach each other. If the
grades of the group members approach each other there is no need to assume that this is
due to a lack of validity in group examination. However, in another respect the argument
about levelling takes on a different meaning. Experience with project work points to the
fact that in a given student population, students will tend to form groups that are fairly
homogeneous in skill, motivation and knowledge levels. This means that the differences in
ability and achievement will manifest themselves not in, but between groups, which again
would account for the seemingly homogenous character of the individual group. It might
also indicate that there may not be any levelling effect for the student population after the
first years of study and selection of group members.
Results from recent research – the study leaders’ perspective
Since the ban of group exams in 2006 many efforts have been made in the educational
system and among study leaders and teachers in order to minimize the more negative
consequences of the government decision. To supplement the above mentioned surveys an
investigation among study leaders within the Humanities and the Social Sciences was
undertaken in May/June 2009.
11
In the Danish university system study leaders are responsible for implementing the
decisions made by the local study board7 regarding the study programmes for which it is
responsible, and are in charge of the day-to-day management of the studies. Study leaders
thus are in a unique position to form an overall opinion/impression/picture of the situation
both as regards positive and negative consequences, if any. A questionnaire was sent to 22
study leaders and they were given the option to either respond to the questionnaire in
writing or to participate in an interview based on the questionnaire. 15 study leaders
participated. The overall research questions in the questionnaire was related to the
following overall themes 1) Individual project exams (description of the actual exam
situations; advantages and disadvantages of the new form); 2) Group work and study
strategy (ratio of group work compared to before; group size; measures taken to ensure
group work, if any; 3) Results of the individual examinations (quality of the projects;
validity and reliability of the examination)
The results of the investigation show that there is both complexity, conflicting experiences
and diversity in the understandings of the consequences of changing exam forms and the
function of individual exams. Some of these consequences and concerns are presented
below. However, it is important to be aware of the fact that in Denmark there are various
opinions among university teachers and leaders about the importance of the pedagogy
compared to research and more narrow subject related issues and the role of exams, - and
of course these different opinions and understandings influence on the statements, which
may be seen in the data-material.
In the following the results from the investigation will be presented and statements from the
informants, the study leaders, will be cited to illustrate the points made.
The question about Alignment
Generally from the answers, study leaders express concern regarding the question of
alignment in terms of coherence between what has to be learned, the working and study
forms and the validation of students’ learning outcome. As one study leader points out the
lack of alignment is evident to the students,
“It is difficult to argue for the students, that we have an assessment form which do not correspond
with the working form”
Another study leader is in addition to this concerned with what you might call the
‘alignment’ which reaches beyond the educational framework and into the future work
situation of the students. Referring to the original ideas behind the PBL approach one of
the intentions was to prepare the students for their future working life. Whereas the group
exam was a continuation of the group work processes of the project period and as such in
7
A study board has representatives from the academic staff as well as from the students.
12
many ways is thought to mirror the work processes of the work life in companies,
organisations, institutions etc. the individual exam represents both a discontinuation of
these processes and a decontextualisation,
“Assessment of the individual student happens without any kind of relation to the collaborative
process which has been the basis for the project. And it is quite unnatural in relation to the kind of
future in which the students will have to act. It makes the exam situation unnatural instead of
natural. You also spend too much time on exams.”
The latter comment refers to the general outline of the individual examination 8 compared
to the group examination. In the group exam examination time was a function of the
number of students in the group supplemented with time for group discussion. This
would give the examiner and the co-examiner/external examiner ample time to probe in
depth into the knowledge of each student, and get at good overall impression of the
competencies of the individual as well as of the group as a whole. Compared to this the
individual examinations have a limited amount of time, on an average approximately
between 20 and 30 minutes. As each student has to start from scratch so to speak
regarding the presentation of the project, the problem, the method, the choice of
theory/ies, etc. there is not much time to discuss in depth the project, the implications and
consequences of the choices made, possible alternatives etc. Furthermore, the examiners
must take care not to make a difference between the examinations of each member of the
group, which is also the concern expressed by this study leader,
“You do not gain any insight into the group’s interaction and you have to repeat the same
questions time after time when there are many students in a group”
Some of the issues addressed here regarding the group’s interaction refer to the group’s
collaborative competencies in terms of professional dialogue, critical thinking, project
management and division of labour within the group.
As previously mentioned the group examination had the potential of being a learning
encouraging situation due to the generous time frame of the exam, as is also advocated by
some assessment researchers. Lauvås and Jakobsen (2004) for instance advocate a form of
summative evaluation that is conducive to the students’ learning rather than merely a
control of the knowledge acquired. With the shift to individual examinations the control
aspect will outweigh any intentions of creating a learning scenario.
Validity
From the surveys undertaken up till now it can be concluded, that the exams risk
becoming primarily about ‘checking’ the students’ rote learning and abilities to reproduce
theories, used in the project. This also means that the enormous amount of work
undertaken by the students during the project work does not seem to have a platform for
assessment in the individual exam. You might say that it is only the tip of the iceberg of
8
See also Kolmos and Holgaard (2009) for a presentation of the group examination of project work versus the
individual examination of project work.
13
what has been learned that is actually assessed, which has consequences for both
alignment and the validity of the exam, as is expressed in the following statements,
“I also estimate that the validity has deteriorated as the result of the exam today is more dependent
on the 20 minutes’ questions and answers and do not involve the result of the thousands of working
hours in the assessment of the total performance and of the individual’s contribution to it.”
Most of the study leaders find that the basis for assessment has become poor because of
less time for the examinations (each individual student about 20 minutes). They value it as
very unsatisfactory professional situations, where the students’ project is only discussed
and treated on the surface compared to the ‘old’ group exams, where you could discuss
the projects in depth and obtain a much better impression of the individual student’s
competences and the scope of their knowledge based potentials.
“It is not easy to guarantee validity because of the time span”
“Less validity! The new individual exam forms do not quite catch the talent”
The last statement here addresses yet another aspect of the assessment, namely that the
short time span interferes with the student’s possibility to ‘shine’ and show it if he/she is
particularly talented.
“It is a less challenging exam form. If the wish was to develop talents then you have shot yourself in
the foot. Now you get standard articles!”
On the other hand this scenario entails another risk, namely, that students, who may not
have participated whole-heartedly in the project work, may be able to pass the
examination solely based on having read the report prepared by the other group members,
but without having been a part of the learning process. Since the examination will be
checking the ‘surface’ of the student’s knowledge and he/she is not challenged in a
discussion with the rest of the group as well as the examiners, this new examination form
might aggravate the very thing, that the ministry feared and wanted to avoid, namely that
an individual is granted a grade that is not a true reflection of his/her actual skills,
knowledge and competencies.
“The new exam form represents poor quality. There are no possibilities for assessing the students’
abilities (in relation to time, depth and the possibilities of confronting/challenging the students to
each other (positively meant).”
Other study leaders, however, express a more positive attitude towards the individual
project exam even if it does mean that there is no alignment,
“The individual student has the possibility to defend the product without taking into consideration
that others are present and without influences from others. It makes it easier to assess the individual
14
student correctly” … The more negative aspect is, that the work process is not consistent with the
assessment of the product”
“It has become easier now to focus on the individual student” or
“I cannot see any negative consequences in individual exams” and
“It has become easier too to evaluate the individual student – who is expected to be able to give
answers to everything”
The last statement emphasises the question and answer procedure of the individual exam
as opposed to dialogue and discussion which was generally the ideal for the group exam.
Academic results
During the last years a new qualification framework system and a new grading system
have been implemented in Higher Education in Denmark. An essential aspect of these
new systems has been the focus on the importance of the formulation of skills and abilities
in terms of competences as well. As a consequence of this there has been a process at the
university where learning goals have been re-formulated in terms of abilities, skills and
competences in the curriculum and in course/programme regulations.
In addition to this several study leaders claim, that clarification of learning goals to a
certain degree has ‘saved’ the group exams, because learning goals have become more
explicit than before the ban of group exams. It means that the teachers/supervisors as well
as students are more conscious about the intended learning outcome, which in a way can
clarify the exam situation. But it does not necessarily mean that the validity of exam is
present.
A few study leaders say that they have no problems with the validity of exams and, some
even find that a raise in professionalism/subject related knowledge can be seen.
“The students seem generally to be better prepared”
In general the results in terms of grades show that no changes can be seen compared to the
situation before.
Group work and student behaviour
The university generally still believes very strongly in the group organised problem
oriented project work, and one concern following the lack of consistency between the ideal
of group work and the individual exam is that the students will to a larger extent wish to
work alone. Students do not always find it easy to have to work in groups, and several of
15
the study leaders also claim that more and more students actually want to work
individually.
“We talk about a thoroughly collective product and the quality of this product depends on the team
work in the group. The exams form partly block the way for motivation as well as collective
commitment in relation to the group work”
Consequently, some study leaders (not all) report that their study board and the teachers
have to make special efforts to convince the students that group work is important for the
learning processes and the learning outcome. Some of the measures reported by the study
leaders were for instance, during the process of establishing the project groups teachers
and supervisors may point out the problems related to data gathering, analysis and
knowledge sharing if students wish to work and write individually. Some study leaders in
fact report that in some semesters students are told that it will be impossible for them to
complete the project, if they do not work in groups. In one particular case this has resulted
in a situation where 80-90% of the students therefore work in groups of often 5 students
(which are actually larger groups than before). Another incentive to promote group work
is allocating relatively more supervision hours to large groups.
Furthermore, some study leaders have noticed changes in students’ behaviour in terms of
on the one hand a less social and collectivist attitude in the students,
“Solidarity has been replaced by elbows”
“Students keep their cards closer to their chest. You may see students accusing each other for
plagiarism”
And on the other hand a marked lack of independence in the students is beginning to
show,
“Tendency of lack of independence among students is seen, but the students have also become
younger”9
According to the Danish qualification framework professional independence is a
competence which must be developed during the study at both bachelor and master level;
it is therefore a potential problem that students now seem to be less independent.
9
Students in Denmark have for many years been older than students at other universities in the world. Now
new regulations have been decided by the Minister in order to encourage them to start their studies sooner
after finishing the Gymnasium.
16
Finally, some study leaders report that another negative side effect is that among
especially first year students study leaders and teachers have experienced that many
students are much more nervous in the exam situation than earlier, and generally that they
are not being able to argue as reflectively and open-mindedly on questions from
examiners as earlier (unless they are a very trained and self-assured students).
This last statement would appear to be in opposition to what some students expressed
during an interview in summer 200710 regarding the transition from group examination to
individual examination, where one of the sentiments involved seemed to be relief,
“It is far easier to go to exam alone than being roasted by the intern and extern examiner
for several hours.” (Information, 28th June, 2007)
In this case the wider time frame of the group examinations was perceived as a negative
factor by the students.
In conclusion, it appears from the investigation that there is still a considerable amount of
group work taking place, in some cases even a larger number of students work in groups
and sometimes even bigger groups than before. It would thus appear that in spite of the
ban on group exam the teaching system has to a certain degree succeeded in convincing
the students about the importance of working in groups in relation to learning outcome
even when they find it difficult to work in groups.
Conclusion and suggestions for new /additional exam forms
Group work as a study form is under pressure in Denmark and although the shift in
student behaviour towards a more individualistic orientation may not be directly linked to
the problems of alignment regarding the group work, the tendency requires special
attention among the study leader, teachers, supervisors etc. in order to maintain the
learning potentials of the group work (cf. Illeris). A number of the basic principles in the
Aalborg PBL, model such as problem orientation, participant direction and the principle of
exemplarity, may still be applicable when the student works alone, but apart from the
obvious fact that a one man project may only cover a limited subject area compared to a
group project due to the resources (man hours) which are at the student’s disposal.
Working alone also means that the student is not challenged in his/her understanding of
the problem, is not trained in the democratic process involved in group work, does not
train and practice his/her verbal negotiation skills and so on. The possibility of learning
through dialogue, discussion, challenge of point of view etc. is reduced. It is therefore
necessary continuously to be very articulate about the pedagogic/learning theoretical
rationale behind the study form, and to give the students positive experiences with group
work so that they experience first-hand how much more they can achieve and learn when
they cooperate with their fellow students. It is necessary to continue creating a learning
10
In an article in the Danish newspaper ‘Information’
17
culture that does not question the ‘wisdom’ of the group work processes as an effective
and motivating form of study.
However, solving the problem with group work does not solve the problem with the exam
form. Based on the above results it is difficult to argue that the solution for the problem
with alignment between the group work dimension of the PBL-model and the individual
exam has been found so far. The individual oral examination has the traits and
characteristics of the ‘traditional’ subject exam, as regards both the limited time frame and
the control aspect of the examination. The scope of what it is possible to control or test for
during this type of examination is consequently reduced accordingly. Thus the complexity
of competencies gained through group work, for instance the ability to work together as a
team also during a stressful examination, can not be assessed. Neither can the depth and
the complexity of knowledge and skills acquired by the students during the project work.
The question of the validity of the exam form must consequently be of high priority – and
new/other exam forms must be created to capture this.
One consequence of the alteration of the exam forms could be a reduction in the demands
for knowledge, skills and competencies to be assessed through the examination. However,
due to the impact of the Bologna-process on the Danish educational system all the learning
goals are described in terms of knowledge, skills and competencies, and generally this
detailed description has resulted in an increase of the demands. Consequently, other
measures have to be taken to ensure validity of the assessment of the problem oriented
group based project work. One suggestion could be that part of the examination takes
place as formative assessment during the semester:
 Written tests regarding selected subject-related issues central to the project work
in question. The tests are prepared by the group’s supervisor. Answers may be
worked out collaboratively among the students, but presented individually.
It appears to be difficult to assess group processes without observing the individual
student within the group. However, some suggestions could be:
 A video recording of a group session – and a meeting where the individual
student describes and reflects upon his/her role in the group, his/her actions and
reactions, perceptions of group dynamics etc. It could be supplemented with a
recording of a group session with the supervisor, as it is sometimes a very
different situation regarding interaction and distribution of responsibility and
time.
 Log books or portfolios where the individual student describes the group
processes and reflect on his/her role in the group as well as what the learning
outcome has been regarding group work, project management etc.
18
To introduce such assessment initiatives regarding for instance group dynamics it would
be necessary that this dimension was explicitly stated as a learning outcome with very
clear assessment criteria in the study regulation, and that resources for the assessment
processes were allocated accordingly.
References
Barrows H.S. (1986). A Taxonomy of Problem-based Learning Methods. Medical Education, 20:
481-486.
Biggs J. and C. Tang (2007). Teaching for Quality Learning at University. The Society for
Research into Higher Education.
Bologna Declaration (1999), www.esib.org.
Boud, D. (2001). Introduction: making the move to peer learning. In D. Boud et al (eds).
“Peer Learning in higher education- learning from and with each other.” London: Kogan Page
Boud D. (1988). Unpublished conference presentation
Cowan J. (1998). On Becoming an Innovative University Teacher. Refledtion in Action. The
society for Research into Higher Education.
Danmarks Evalueringsinstitut (2005), Prøver og eksaminer. Brugerundersøgelse. København
Dewey, J. (1933). How we think: a restatement of the relation of reflective thinking to the educative
process. Boston: Heath,
Freire, P. (1970). Cultural Action for Freedom. Boston.
Gibbons, M. et al. (1994). The New Production of Knowledge. The Dynamics of Science and
Research in Comtemporary Societies, Lonson. Sage.
Gibbs G. (1999). Using Assessment stratetically to Change the Way Students Learn. I S.
Brown & A. Glasner (eds.). Assessment matters in Higher Education (pp. 41-54). The Society
for Research into Higher Education and Open University Press.
Information, 17th June, 2007 (Danish Newspaper)
Illeris K. (1974 ). Problem orientation and participant direction. An introduction to
alternative didactics.
www.Karieerecentret.aau.dk
Kjærsdam, F. and Enemark, S. (1994). The Aalborg Experiment – project innovation in
universitiy education, Aalborg University Press, Aalborg, Denmark.
Kolmos, A., Fink, F. and Krogh, L. (eds.). 2004. The Aalborg PBL model. Progress, Diversity
and Challenges. Aalborg University Press
Kolmos A. and J. Holgaard (2009). Gruppebaseret eller individuel projekteksamen – fordele og
ulemper i Dansk Universitetspædagoagisk Tidsskrift. Årgang 4. Nummer7.
Kolmos, A. and Holgaard, J. E. (2007), Sammenligning af gruppebaseret og individual eksamen
sommer 2006 ved fakultet for Ingeniør, Natur og Sundhedsvidenskab. Aalborg University:
Centre for Engineering Education Research and Development.
Kolmos, A. and Holgaard, J.E. (2009), Gruppebaseret eller individual projekteksamen – fordele
og ulemper. Dansk Universitetspædagogisk Tidsskrift nr. 7, 2009
Mills, C. Wright (1959). The Sociological Imagination. Oxford University Press. New York
Negt, O. (1975). Sociologisk fantasi og eksemplarisk indlæring. Roskilde Universitetsforlag
19
Pettersen, R. C. (1997). Problemet først: Problembasert læring som pedagogisk idé og
strategi.Oslo: Tano Ascheoug.
Krogh, L. og Gulddahl Rasmussen, J. (2004). Employability and problem-based learning in
project-organized settings at universities. I: Kolmos A., Fink F. K., Krogh, L., The Aalborg PBL
model - Progress, Diversity and Challenges, Aalborg University Press
Krogh L. and Rasmussen P. (2007). The Ban on Group Examinations in Danish Higher
Education. Department of Education, Learning and Philosophy, Aalborg University
Lauvås, P. & Jakobsen, A. (2004) Exit eksamen – eller? Former for summativ evaluering i høgre
utdanning.Oslo: Cappelen Akademisk Forlag. 2. opplag
Ministerial Order on University Examinsations (The Examination Order) no. 867 of 19
August 2004
Pickford R., S. Brown (2006). Assessing Skills and Practice. Routledge
Ramsden, P. (2004). Learning to Teach in Higher Education (second edition). Routledge,
London and New York
Rasmussen, P. (1991), De studerendes oplevelse af undervisningen ved Aalborg
Universitetscenter. Aalborg.
Sadler, D.R. (2005). Interpretations on criteria-based assessment and grading in higher education.
Assessment and Evaluatio in Higher Education 30.
Wenneberg, S. (2000), ”Forskningsledelse som arketypisk eksempel på vidensledelse – hvordan
håndteres den anvendte faglighed?”i Christensen, Per Holdt (red.) (2000); Viden om ledelse,
viden om virksomheden. Samfundslitteratur
Østergaard, P. (2002) Uddannelsessystemets behov for kvalitetsmålinger. Handelshøjskolen i
Århus
Aalborg Universitet og Roskilde Universitet (2003), Kandidat- og Aftagerundersøgelsen 2002.
Towards a Danish qualification key for higher educations. Qualifications Framework. Final
Report, approved by the Bologna Expert Monitoring group, January 15th 2003.
20