Overview of Student Assessment
and its Guidelines in the College
Nadia Al Wardy
Chairperson, College Examinations Committee
Assessment orientation
5/29/201
6
Standards of “Sound
Assessment”
• Continuous assessment
• Evidence of linkage to course
objectives
• Diversity of assessment methods
• Assess different levels of
achievement
• Criterion referenced
• Marking accurate and consistent
• Feedback meaningful and helpful
• Should enhance student learning
• Students’ awareness of criteria
• Proportional to the workload of
the course
http://www.squ.edu.om/Portals/6/docs/Docss/Assessment%20Policy%20Regulations%20&%20Guidelines-Oct%202009.pdf
http://www.squ.edu.om/Portals/20/PDF/Acad%20Regulations/english%20regulation.pdf
The assessment process

Specifying the purpose

Determining what is to be measured and how

Measuring it

Setting standards of expected performance

Reporting results

Making a judgement
Specifying the Purpose
Formative

Diagnosis
Summative

Promotion

Certification
Determining what is to be measured and how

Blueprinting

A matrix of:

What to assess?


knowledge, skills, attitudes
How to assess?

Different assessment tools









MCQ
EMI
MEQ
SAQ
OSPE/OSCE
Mini-CEX
CbD
OLC
….
Guidelines

Vetting: For all exams, questions /
stations / cases etc. should be reviewed
and emended by the relevant
committees before the examination
takes place

Relevance (Scenarios)

Types (at least 2 different modalities)
IDEAL Office

International assessment bank

Over 31000 questions available
Formats of assessment in Phases I, II and III
Domain/Outcome
Phase I
Clinical Skills
Health Promotion &
Disease Prevention
Phase II
Phase III
OSCE
Mini-CEX, CbD, OLC, OSCE,
Seminars,WRP, BSP, SC, OSPE
MCQ, Reports
Seminars, OSCE, OLC, CbD
Communication Skills
TBL
Present
Seminars, OSCE, OLC, CbD, MCEX
Medical Informatics
CBA
MCQ, CBA
MCQ, CBW
MCQ,
Spotter
Lab rep,
SAQ
MCQ, SAQ, CBP
Quiz, problems,
Spotter, EMI, OBE,
OSPE, MEQ,Viva
MCQ, EMI, MEQ, KFQ, SAQ
Assignment, Project,
Proposal,Thesis, SAQ
MCQ, Mini-CEX, CbD, OLC
TBL,
OBE
Assignment, Project
Poster,Thesis,TBL,
Present
Portfolio
Knows
Knows How
Basic, Social & Clinical
Sciences
Attitudes, Ethics & Legal
Responsibility
Personal & Professional
Development
Miller’s Pyramid
Shows How
Does
Principles of assessment (Reliability)
Sampling

No. of assessments


(min of 3 components)
No. and quality of questions

(at least 30 MCQs or
equivalent/credit hour)

No. and quality of examiners

Weighting of components


(no single component to
exceed 60% of total marks)
Time per question

(1.5 - 2 min/MCQ)
Example of assessment in courses of Phase
I, II, and BMS
Course/Module
No. of
No. of
credits components
Weighting
Format
Intro to
Anatomy
4
4
20%, 20%,
20%, 40%
MCQ, Spotter,
online
Locomotor
system
3
3
30%,10%,
60%
MCQ, OSPE
Integrated
Modules
3
3
30%, 20%,
50%
MEQ, OSCE
Molecular
Diagnostics
3
3
20%, 20%,
60%
MCQ, practical
Assessment in Phase III
Stage
Block
Continuous (50%)
Final (50%)
Stage I
Pre-Clerkship
3 Mini-CEX
MCQ and EMI
Junior Clerkship:
Med/CH & allied block
Surg & allied block
BM/Fam block
Mini-CEX, OLC, KFQs (40%)
Portfolio (seminars, ward round
presentations, bedside procedures,
case write-ups) (10%)
80 MCQ & EMI,
20 SAQ (30%)
Senior Clerkship:
Med/Famco & allied block,
Surg & allied block,
Women & Child block
Mini-CEX, OLC, (CbD), KFQs (40%)
60 MCQ & EMI,
Seminars,Ward round presentations, 20 SAQ (30%)
Bedside Procedures, Case write-ups
(10%)
4 OSCE (20%)
Stage II
8 OSCE (20%)
Elective
Selective
Pre-Internship
FMD
120 MCQ (20%)
120 SAQ (20%)
15 OSCE (20%)
8 Short Cases (40%)
Pre-clerkship
(10 weeks)
Ophthalmology
(3)
Junior Clerkship (48 weeks)
Stage 1
Behavioral Medicine, Communication Skills, History Taking,
Physical Examination Skills, Ethics, Prescribing, Medical Records, HIS
Surgery (6)
Child Health
(6)
Orthopedics,
Rehab and Pain
Management
(4)
Medicine (7)
SLT (Lab)
(1)
Clinical
Genetics &
Counseling
(2)
MED/CH
FAMCO (5)
Behavioral
Medicine (7)
Dermatology(2)
SLT (Anes) (1)
SLT (Stats) (1)
SLT (Geriatrics)
(1)
SLT Forensic
Medicine (1)
Study Break (1 week)
SURG
Oral Health (1)
BM/Fam
Junior Clerkship Exams (1 week)
Child Health
(3)
Surgery (3)
3 block exams
A&E (4)
ICU (1)
Obs/Gyne (8)
Medicine (7)
SLT (Therap)
FAMCO (3)
SLT (Lab) (2)
Elective (8)
ENT (2)
Hematology (2)
Radiology (2)
Selective (2)
(1)
MED/Fam
Study Break (1 week)
SURG
Senior Clerkship Exams (1 week)
Women & Child
Buffer (3 weeks)
Pre-Internship
(12 weeks)
Stage 2
Senior Clerkship (48 weeks)
Buffer (3 weeks)
Pre-Internship
Medicine
(3 weeks)
Surgery
(3 weeks)
Child Health
(3 weeks)
Study Break (2 weeks)
FINAL MD EXAM (2 weeks)
Obs/Gyn
(3 weeks)
3 block exams
Junior Clerkship
Week
A
B
C
D
E
F
1
Med
x
CH
x
Surgery
x
Ophthal
x
BM
x
FAMCO
x
2
Med
x
CH
x
Surgery
x
Ophthal
x
BM
x
FAMCO
x
3
Med
x
CH
x
Surgery
x
Ophthal
Mini-CEX
BM
x
FAMCO
Mini-CEX
4
Med
OLC
CH
Mini-CEX
Surgery
Mini-CEX
Orthopedics
x
BM
Mini-CEX
FAMCO
Mini-CEX
5
Med
Mini-CEX
CH
Mini-CEX
Surgery
Mini-CEX
Orthopedics
x
BM
x
FAMCO
OLC
6
Med
Mini-CEX
CH
OLC
Surgery
OLC
Orthopedics
Mini-CEX
BM
Mini-CEX
Dermatol
x
7
Med
OLC
Genetics
x
Anesthesia
Mini CEX
Orthopedics
Mini-CEX
BM
SAQ
Dermatol
KFQ
8
Lab
SAQ
Genetics
SAQ
EBM
MEQ
Forensic
MCQ
Geriatric
SAQ
Oral Health
KFQ
Marking

MCQ and EMI:


MEQ, SAQ and Essays:



if not done on-line, then optical
marking.
model answers
double marking is encouraged
OSCE/OSPE, Mini-CEX, OLC,
Presentations, Reports…


checklists and rating scales
several examiners should be
available, but one examiner
should mark the same question
for all students for consistency.
Standard Setting
Standard setting
method
Angoff
Hofstee
Borderline
group
Fixed
percentages
(eg 60%)
Grade descriptions
Grade
Description
A
Exceptional performance; all course objectives achieved; objectives
met in a consistently outstanding manner.
B
Very good performance; significantly more than the majority of
the course objectives achieved (majority being at least two-thirds);
objectives met in a consistently thorough manner.
C
Satisfactory performance: at least the majority of course
objectives achieved; objectives met satisfactorily.
D
Minimally acceptable performance: less than the majority but
more than the minimum required course objectives achieved;
objectives met at a minimally acceptable level.
F
Unacceptable performance: minimum required course objectives
not met; objectives not met at a minimally acceptable level; no
credit earned.
FW
Failure to meet the attendance requirement, no credits earned
FSP
Failure with Supplemental Privilege
Programme for conversion of scores
to grades
Item analysis and test statistics

IDEAL Office

Determination of
difficulty,
discrimination and
reliability indices

Results of these
analyses should be
discussed by the
relevant committees
to assist in making
informed decisions
Reporting results

What?

Grade sheets filled
online to be submitted
to the Examination
Office before and after
confirmation of grades
(timeline)

Separate spreadsheet
showing scores of all
ICAs and final
accompanied by course
report form

Grade report form

To whom?

Examinations
Committee through
Examinations Office
Feedback
 ICAs:


within 1 week

timetabled

should not involve
release of questions
but a discussion of
points of weaknesses
Finals not discussed
Management
College Board
College Examinations Committee
Phase II Exam
Subcommittee
Written group
Phase III Exam
Subcommittee
OSCE group
IP and BMS Exam
Subcommittee
SC group
Management
College Board
College Examinations Committee
• Chairman
• ChairmanPhase
(CC) III Exam
Phase II Exam
IP and BMS Exam
Subcommittee
Subcommittee
Subcommittee
• Assistant Deans (Pre-clinical and Clinical)
• Phase Coordinators
• Senior staff members
Written group
OSCE group
• Examinations Officer
SC group
Management
College Board
College Examinations Committee
Phase II Exam
Subcommittee
• Course/module
coordinators
Written group
Phase III Exam
Subcommittee
IP and BMS Exam
Subcommittee
• PC, JC and SC Block coordinators
• Chairs/Reps Dept Exam Committees
OSCE group
SC group
Examinations Office
•
•
•
•
•
•
•
•
•
•
Examination timetables
Examination booklets, optical mark sheets, etc
Administrative forms related to examinations
Copy, store (electronically), and ensure security of all
examination question papers
Assignment of invigilators
Prepare, print, and distribute information relating to
assessments
Coordinating the visit of External Examiners
Collection of grades
Liaising with A&R in matters related to assessment
…..