Student assessment
Research findings
Zadar – Tempus project
Reform of Croatian legal education
Research methodology
Ana Barić, LL.B. (in spe)
• basic framework
• questionnaire with 80 questions
• 17 exams with written and oral part of
the exam
• 12 oral exams
• 10 students conducting research
• 76 professors included in the research
• exam terms in January and February
• around 300 student working hours
altogether
Provisional and incomplete
results arising from the questionnaire
Prof. Dr. Alan Uzelac
Elements of the research
• Methods
– analysis of the
annual statistics
(24.602 exams)
– analysis of one
exam term (February)
(3.760 applications,
1.809 exams)
• Parts of evaluation
–preparation for
exams
–applications for
examination
–techniques of
examination
• oral/written
–applied methods of
examination
–publicity
–results
COMPULSORY SUBJECTS
ELECTIVE SUBJECTS
Roman private law **
Legal psychology **
General history of law and state ##
International maritime law ##
Croatian legal history in European context
English I
**
General theory of law and state **
English II
**
Sociology
German I
##
German II
##
**
Political economy
**
Economic policy
**
Family law
**
**
Criminal procedure **
Criminal law
**
Statistics and information science **
Civil procedure
**
Civil law
##
International law
##
parts of the exam
only oral exam
written and oral
exam
Financial law and financial science **
Administrative science **
Labour law and social security ##
Commercial law
##
Private international law
Administrative law
**
**
Company law ##
Maritime and general transport law##
Constitutional law
##
17
**
12
##
Preparation for the exams
Is teaching useful at all?
Frequen
cy
Perce
nt
Seminars are recommended and greatly
facilitate success in exams
9
42,9
Seminars are useful but literature is more
important
10
47,6
Seminars aren’t useful, it is better to
learn from the literature
1
4,8
Seminars don’t take place
1
4,8
21
100,0
Total
Freque
ncy
Percent
4
19,0
Lectures are useful but literature is
more important
17
81,0
Lectures aren’t useful, it is better to
learn from the literature
0
0,0
Lectures don’t take place
0
0,0
21
100,0
Lectures are recommended and
facilitate pass
Total
Seminarists’ privileges?
1
Mean
0,8
0,6
0,4
0,2
0
earlier term of the exam for seminarists
seminarists can choose for exam
professor who was on the seminar
grade from the seminar have an
influence on the final grade
Techniques of examination
Writing v. Orality
Methods of written tests
• Written test: 17 courses (v. 12)
• Anonymous: none (at the term)
• Multi-part (test and essay): 5 (yes) v.
12 (no)
• Quitting without fall: 5 (no) v. 12 (yes)
• Types of written tests:
– multiple choice (28);
– filling-in the form (22);
– essays (10)
– longer questions (7)
– shorter questions (20)
Methods of oral examination
0,8
Mean
0,6
0,4
0,2
0
professors
spontaneous
questions
prepared
questions by the
professors choice
prepared
questions by the
random choice
roughly chosen
theme, without
subquestions
roughly chosen
theme, with
subquestions
precise question
from the particular
theme
Methods of oral examination
How do students take oral
examination?
One student alone in the room
One student, others listen
Oscillating between students
who take exam
TOTAL
Freque Perce
ncy
nt
7
9,7
45
62,5
20
27,8
72 100,0
Relation of written and oral part
• Common elements
– pass at written part necessary for taking of the
oral examination
• minor exceptions
– (new) statutory option for examination boards
– borderline cases
• The meaning of pass at written exam?
– possible options
•
•
•
•
only ticket to oral part
correction of grading
great influence on grading
automatic pass
– with the grade from the written part
– with only the passing grade
– differences even within the same course
Publicity of exams
Announcing the written test results
0,6
0,5
0,3
0,2
0,1
1
0
oral notice of the
results
notice of the results
on the notice board
0,8
notice of the results
on the Internet
Mean
Mean
0,4
0,6
0,4
0,2
0
notice with the
name of those
who have
passed
notice with the
notice with the
notice with the
notice with the
name of those number of points number of points grade of those
who have failed those who have those who have
who have
passed
failed
passed
Publicity of oral examination
• generally, exams are made in public
– first layer: students who take the exam
• group exams – 7 (individual); 65 (other
students who take the exam present)
– second layer: other interested students
• 5 reported cases of no public at the oral
examination
• reasons stated:
– request of the teacher (1)
– request of the student (1)
– other (alleged lack of space) - 3
student doesn't
know the grade
before
professor
enters it in his
index
allways
often
rare
professor
explains why he
gave particular
grade
allways
often
rare
never
never
professor
proclaim the
grad before
entering it in
student's index
student can
argue the grade
given by the
professor
allways
allways
often
often
rare
rare
never
never
it is offered to
the student not
to take the
grade and to
came again on
the oral exam
it is offered to the
student not to
take the grade
and to continue
answering the
questions
allways
allways
often
often
rare
rare
never
never
Results
First to last year of study – average success
percentage of
unsufficient grades
during the year
40,00
percentage of sufficient
grades during the year
percentage of good
grades during the year
30,00
percentage of very good
grades during the year
Mean
percentage of excellent
grades during the year
20,00
10,00
0,00
1
2
3
year of studying
4
Comparison: annual and monthly statistics
Compatibility grades during the year to the grades on this term
percentage of excellent grades on the
exam
percentage of excellent grades during
the year
percentage of very good grades on
the exam
percentage of very good grades
during the year
percentage of good grades on the
exam
percentage of good grades during the
year
percentage of sufficient grades on the
exam
percentage of sufficient grades during
the year
percentage of unsufficient grades on
the exam
percentage of unsufficient grades
during the year
0,00
10,00
20,00
Mean
30,00
40,00
Grades: percentage of success
percentage of
unsufficient
grades during
the year
50,00
1
percentage of
sufficient grades
during the year
Median
40,00
30,00
5
2
20,00
10,00
4
2 3
3
percentage of
very good
grades during
the year
percentage of
excellent grades
during the year
4
5
1
0,00
compulsory
percentage of
good grades
during the year
elective
compulsory/elective subjects
Success/methods
Grades regarding if the exam is only oral or written and oral
percentage of
unsufficient grades
on the exam
40,00
percentage of
sufficient grades
on the exam
Mean
30,00
percentage of good
grades on the exam
percentage of very
good grades on the
exam
20,00
percentage of
excellent grades on
the exam
10,00
0,00
only oral exam
written and oral exam
parts of the exam
percentage of very
good grades during
the year
percentage of
excellent grades
during the year
60,00
Mean
Success in individual courses
percentage of
unsufficient grades
during the year
120,00
percentage of
sufficient grades
during the year
100,00
percentage of good
grades during the
year
80,00
40,00
20,00
0,00
Statistics and information science
Sociology
Roman private law
Private international law
Political economy
Maritime and general transport law
Legal psychology
Labour law and social security
International maritime law
International law
German II
German I
General theory of law and state
General history of law and state
Financial law and financial science
Family law
English II
English I
Economic policy
Croatian legal history in European co
Criminal procedure
Criminal law
Constitutional law
Company law
Commercial law
Civil procedure
Civil law
Administrative science
Administrative law
subject
Reliability of comparison – negative grades
withdrawal from the
written exam after
exit, without fall
no
yes
percentage of
excellent grades
during the year
60,00
Mean
Success in individual courses
percentage of
sufficient grades
during the year
100,00
percentage of good
grades during the year
percentage of very
good grades during
the year
80,00
40,00
20,00
0,00
Statistics and information science
Sociology
Roman private law
Private international law
Political economy
Maritime and general transport law
Legal psychology
Labour law and social security
International maritime law
International law
German II
German I
General theory of law and state
General history of law and state
Financial law and financial science
Family law
English II
English I
Economic policy
Croatian legal history in European conte
Criminal procedure
Criminal law
Constitutional law
Company law
Commercial law
Civil procedure
Civil law
Administrative science
Administrative law
subject
Diploma (final exam) – success story
60,00
50,00
Mean
40,00
30,00
20,00
10,00
0,00
percentage of excellent
grades on the final exam
percentage of very good
grades on the final exam
percentage of good grades
on the final exam
percentage of sufficient
grades on the final exam
Number of defended final papers
OTPD
120
RPP
OPP
PHPD
100
SOC
PolEk
UST
KAZ
80
KPP
ObP
Pinf
60
EkPol
GP
GPP
40
FPFZ
UZ
MP
20
UpP
TrgP
Pdru
PomOP
0
MPP
# diplomskih
Rsoc
EJP
Diploma (final exam) – success story
The final paper was defended
before…
3 teachers, continously
Mainly 3, sometimes 2.
Rarely 3, more often 1 or 2.
Total
Frequenc
y
Percent
10
52,6
4
21,1
5
26,3
19 100,0
Output: number of graduated students
Year
Inscr.
Grad.
600
561
203
92/93
811
272
93/94
1123
237
94/95
1231
242
95/96
1496
268
96/97
1440
294
97/98
1552
366
98/99
1427
354
99/00
1145
396
00/01
1133
426
01/02
956
453
02/03
1111
453
03/04
969
445
04/05
1006
497
05/06
1050
216
TOTAL
1800
Inscribed
1600
500
1400
400
1200
1000
300
800
200
600
400
100
0
91
/92
92
/93
93
/94
94
/95
95
/96
96
/97
97
/98
98
/99
99
/00
00
/01
01
/02
02
/03
03
/04
04
/05
05
/06
91/92
Graduated
200
0
Final Questions
for
not so Final Solution
Prof. Dr. Hrvoje Sikirić
Open questions ???????????
•
Teaching (lectures, seminars) / Exams
•
Number of exam terms per year, plan of terms
–
Avoiding overlapping of terms
•
Distribution of exams into subsections
•
Techniques of examination – writing v. orality
•
Written exams – Standards – unanimity v. diversity
–
–
–
–
•
Oral exams
–
–
•
Anonymity
Structure – questions, case analysis, essays
Evaluation – grading, selectiveness
Duration
Ensuring objectivity?
Publicity
Diploma exam
–
–
Writing standards
Evaluation
Thank you for your attention!
Series1
1.1.1900
2.1.1900
3.1.1900
4.1.1900
5.1.1900
6.1.1900
7.1.1900
8.1.1900
9.1.1900
10.1.1900