Tests and Quizzes Tool
(aka SAMigo)
Marc Brierley, Stanford University
T&Q Lead Designer
http://bugs.sakaiproject.org/confluence/display/SAM
www.sakaiproject.org
1
Agenda
• Design Origins
• Feature Overview
– Focus on pedagogical flexibility using
templates
• Short Demo
• Futures
www.sakaiproject.org
2
Where did the design originate?
• Formative Assessment (Homework)
– Stanford Assignment Tool for Human Biology
– CourseWork CMS with Language Tools
(Andrew W. Mellon Foundation funded)
– OKI Tools Workshop to Specify Functionality
(Andrew W. Mellon Foundation funded)
– OKI Formative Assessment Tool
(Andrew W. Mellon Foundation funded)
• Summative Assessment (High-Stakes tests)
– OnCourse Quiz Tool
www.sakaiproject.org
3
Where did the design originate?
• OKI Tools Workshop
– Many schools met to define needs for a CMS
– One group specified the needs for an
Assessment Tool
– Several hundred functional requirements
identified
www.sakaiproject.org
4
Where did the design originate?
• Assignment and Assessment Manager
– Supports teaching and learning in
large lecture science courses and in language
instruction
– This OKI Tool was developed at Stanford
– Andrew W. Mellon Foundation funded
www.sakaiproject.org
5
Where did the design originate?
• AAM design combined with OnCourse
Quiz tool design “Navigo” (2003)
• Stanford and Indiana join Sakai project,
Stanford rewrites tool to Sakai Style Guide
TPP, tool renamed SAMigo (2004-present)
www.sakaiproject.org
6
What are T&Q’s features:
•
•
•
•
•
•
Assessment Authoring
Assessment Publishing
Assessment-taking Management
Assessment Grading
Question Pools Management
Template Management
www.sakaiproject.org
7
Organize & Publish Assessments
• Core (pending) Assessments for a course
– Starting page for assessment authoring
– Can be ex/imported in QTI XML format
– When ready, published via settings
• Published Assessments for student
assessment-taking
– When students respond, scores link appears
– Can change release, due, retract dates
www.sakaiproject.org
8
www.sakaiproject.org
9
Question Types
•
•
•
•
•
•
•
•
Multiple Choice (single and multiple correct)
Multiple Choice Survey
True/False
Matching
Essay/Short Answer
Fill in the Blank
File Upload
Audio Recording*
* in future Sakai 2.x release
www.sakaiproject.org
10
www.sakaiproject.org
11
Rich-Text Everywhere
• WYSIWYG editor
Fonts, style, layout, images, HTML, links
• In section headers, question texts, selections
for responses, and feedback
www.sakaiproject.org
12
www.sakaiproject.org
13
Pedagogical Flexibility
•
•
•
•
•
•
•
•
Online test
Self-study questions
Homework, problem-sets
Paper, code, or project submission
Language drills
Quick knowledge probes
Surveys
Others???
www.sakaiproject.org
14
Templates
• Create assessment “types” for different
disciplines, different uses
• Hide complexity by selecting what which
settings are visible to the instructor.
• Pre-set defaults by making setting choices
for the type (e.g., Self-study always has
immediate feedback)
www.sakaiproject.org
15
Template Workflow
www.sakaiproject.org
16
www.sakaiproject.org
17
www.sakaiproject.org
18
www.sakaiproject.org
19
www.sakaiproject.org
20
www.sakaiproject.org
21
Online Test
• Unique tests for each student
– Randomized order of questions
– Randomized draw from question pools
• Timed test-taking during access window
– Auto-submit at end of timed period
• One submission only per student
• Higher Security
– IP Addresses restricted
– Secondary Password (proctored)
• No late submissions accepted
• Scores transferred to gradebook
www.sakaiproject.org
22
www.sakaiproject.org
23
Self-Study Questions
• Immediate feedback
• Random access to Questions
– Table of contents
– Marked for review list
• No record of score in gradebook
• No due date; always available to student
• Unlimited submissions allowed
www.sakaiproject.org
24
www.sakaiproject.org
25
Homework/Problem Sets
• Published with multiple release and due dates
for different sections
• Multiple methods for handling late submissions
• Students can save work during assignment
period.
• Auto-grading with methods for diagnosing
learning problems
• Quick-reviews of student aggregate performance
by instructors viewing histograms, statistics.
• Pools for organizing questions for reuse
www.sakaiproject.org
26
Essay and Project Submission
• Use file upload question type for submitting
any type of document
• Grader can download assignment; upload
marked-up file to return to student
• Grades are recorded in gradebook
www.sakaiproject.org
27
Question Pools
• Organize questions into multiple subpools
• Author Questions from within Pool Mgr.
• Randomly draw from pools for an
assessment
www.sakaiproject.org
28
www.sakaiproject.org
29
www.sakaiproject.org
30
Demo
www.sakaiproject.org
31
www.sakaiproject.org
32
www.sakaiproject.org
33
www.sakaiproject.org
34
www.sakaiproject.org
35
www.sakaiproject.org
36
www.sakaiproject.org
37
www.sakaiproject.org
38
www.sakaiproject.org
39
www.sakaiproject.org
40
www.sakaiproject.org
41
www.sakaiproject.org
42
www.sakaiproject.org
43
www.sakaiproject.org
44
www.sakaiproject.org
45
www.sakaiproject.org
46
www.sakaiproject.org
47
www.sakaiproject.org
48
www.sakaiproject.org
49
www.sakaiproject.org
50
www.sakaiproject.org
51
www.sakaiproject.org
52
www.sakaiproject.org
53
www.sakaiproject.org
54
www.sakaiproject.org
55
www.sakaiproject.org
56
www.sakaiproject.org
57
www.sakaiproject.org
58
www.sakaiproject.org
59
www.sakaiproject.org
60
T&Q 2.1 Highlights
• “Section-Aware” grading - assessments
filtered by section (e.g. discussion, lab)
• Workflow improvements (based on 2.0.1
usability testing)
• Code refactored to expose tool API
– looser coupling to Sakai (or other CMS)
– clean stand-alone build
www.sakaiproject.org
61
T&Q post-2.1 Future
• Audio-recording question type (Stanford)
• Others
– Export assessment results
– Release assessments to specific classes and
sections
– Department/school-level templates
– Question Pool sharing by group, import/export
www.sakaiproject.org
62
NEW! Assessment Tools DG
• First BOF
– Time: Thursday 7:30-8:30am
– Location: Salon J
• Possible Topics
– current usage and best use recommendations for assessment tools
within Sakai
– assessment pedagogy (both formative and summative)
– user interaction and tool design
– assessment tool interoperability with other tools (e.g. Tests & Quizzes
with Gradebook) and standards compliance (e.g. IMS-QTI)
– process for feature requests and prioritization
– future plans for assessment tools
www.sakaiproject.org
63
Questions?
www.sakaiproject.org
64
Contacts
Marc Brierley, Stanford University
T&Q Lead Designer
marc.brierley@stanford.edu
Lydia Li, Stanford University
T&Q Project Manager
lydial@stanford.edu
Homepage:
http://bugs.sakaiproject.org/confluence/display/SAM
www.sakaiproject.org
65