Technology Enhanced Learning
Test and Quizzes in Sulis
Facilitator: Angelica Risquez
Index












Benefits of T&Q
T&Q key features
T&Q homepage
Question Pools
New assessment: Quick create
New assessment: Slow create
Changing and previewing assessments
Publishing assessments
Working with scores and answers
Retracting a published assessment
Exporting/importing assessments
Assessment types
Benefits of online formative T&Q
 Learners develop understanding progressively and have
opportunities for feedback and reinforcement
 Allows for scalable formative testing and feedback
provision that would not be possible without automatisation
 Available anytime, anywhere
 You can delegate on students to create questions
 If well used, may reduce teachers’ workload and increase
students’ responsibility for their own learning
 It may increase participation in face-to-face sessions
Useful case: http://www.elearning.ac.uk/effprac/html/casestudy5.htm
Test & Quizzes in Sulis: Key Features
Automatic grading: True/false, matching, multiple choice, fillin-the-blank, numeric response, survey (grades automatically
transferred to the Gradebook tool)
No automatic grading: short answer/essay and file upload
Question Pools to re-use questions and share them
Import assessments from other sites you own and import
previously created tests/quizzes/surveys (QTI -compliant files)
Create questions with online forms (standard approach) or
single text box (quick create approach)
Customisation: location for taking a test, IP addresses,
availability of test, time limits, number of submissions allowed,
and late submissions
Attachments (by uploading local files, specifying URLs, and
selecting files from Resources) to any question.
Templates (or “assessment types”) to speed customisation
T&Q Homepage (instructor view)
1.
2.
3.
Core Assessments (unpublished) and Published ass’s.
You can return to this screen with the reset arrow!
New assessment options
1.
2.
3.
Quick Create
Create (recommended)
Import (xml file)
Question Pools: create





In the T&Q tool, click the Question Pools button. In the bar near
the top of the screen, click Add New Pool
You can also import an assessment as a question pool, after
exporting an assessment from another site:
In the T&Q home page (instructor view) click Question Pools.
In the bar above the list of pools, click Import and follow
instructions to import your assessment
You can also create subpools, move and delete pools
To add questions to a pool or subpool, click the “Add” link (right
hand side of the screen, opposite to “Questions”), and follow steps
To Copy a question from a Question Pool to an assessment, you
use the pulldown list on the Questions screen (see slide 6) to select
the type of question to add. Choose the option “Copy from
Question Pool”
New Assessment: “Quick” Create
Only for multiple choice (single or multiple answer), true/false,
survey and fill-in-the-blank questions.
1.Name the assessment and click “Quick Create” (don’t choose
assessment type for now). Sulis will display the Questions screen
2.Click the links in for each type of question to read specific
instruction and examples
3.Review each question on the Validate Assessment/Question
Pool screen
4.Click “create assessment”: Sulis displays the T&Q home page
and the new assessment listed under Core Assessment (you can
create a Question Pool instead)
When you finish with the process, your new quiz will appear under
“Pending assessments”. Note: the quiz is not yet available to
students!
New Assessment: “Slow” Create
To be used for other type of questions, to create parts
(needed for random assessments), and to reuse questions
from Question Pools
Name the assessment and click “Create” (don’t choose
assessment type for now). From menu “Add question”, add
questions one by one or bring from Question Pool
When you finish with the process, your new quiz will appear
under “Pending assessments”. Note: the quiz is not yet
available to students!
New Assessment: “Slow” Create (2)
Create questions








You can either create new questions in your assessment or add
questions from a Question Pool.
You may add question to any unpublished assessment.
Under the appropriate part, select the type of question you want to add with
the “Add Question” pull down menu
Answer point value: If questions have two correct answers, the points for
each correct question accumulate (e.g. 10+10=20). For ungraded
questions enter 0 (zero). Incorrect answers are worth negative points.
Using the Assign to Part pulldown list, select the part (that is, the section)
of the assessment to which you want to assign this question
To assign a question to a question pool, select the name of the desired
pool from the Assign to question pool pulldown list.
To provide question-level feedback enter text in the boxes under
“Correct/Incorrect Answer Feedback.” (voluntary)
You can require participants to provide a reasons for choosing an answer
under Required Rationale
New Assessment: “Slow” Create (3)
Create questions (continued)
Multiple choice questions

To indicate the correct response, click the Correct Answer radio button for that
response (multiple correct answers allowed)

To display the possible answers in random order, und Randomize Answers
Survey choice questions

You do not have the option of randomizing answers or requiring a rationale.
Short answer/essay questions

Answer point value: enter the point value you want to assign to this question. For
an ungraded question, enter 0 (zero). (MANUALLY GRADED)
Fill the blank questions

Place curly braces { } around words requiring a blank For example: {gravity}.

If more than one response is correct for a given blank, or to allow alternate
spellings, use commas to separate the alternate responses. For example: Darth
Vader actually was Luke Skywalker's {father, dad, poppa, pater, pa, daddy}.

For questions with more than one blank, each blank must contain a unique
response (mutually exclusive). For example: The colors of the Irish flag are
{orange, white, green}, {orange, white, blue}, and {red, white, blue}. To answer
correctly, the participant must respond with some combination of orange, white,
and green.
New Assessment: “Slow” Create (4)
Create questions (continued)
Numeric Questions

Place curly braces { } around numbers that require a blank (for example: 3*3=
{9} y 2+2= {4} requires a blank before the digits 9 and 4).

If a range of values is acceptable, insert a pipe between the highest and lowest
acceptable values (for example: 5|8 accepts answers of 5, 6, 7, and 8).
Matching Questions

In the Choice box, enter a choice. In the Match box, enter the correct match
for the choice you specified.

Click the Save Pairing button. Sulis saves the pair you have created and
displays it under the Add/Edit Pairing and Optional Feedback header
File Upload Questions

In the Answer Point Value box, enter the point value you want to assign to
this question. For an ungraded question, enter 0 (zero). (MANUALLY
GRADED)
You may change questions in any unpublished assessment. Display the
Questions screen for the assessment, find the question you want to change
and click the Edit option for it.
New Assessment: “Slow” Create (5)
To create a random quiz from a question pool, you need to
edit the settings of a part of the test. The part should be empty
of questions, if it is now, you need to create a new part (click
“Add Part”). You can use the Default part as long as no
questions have been added yet: click “Edit” for Default Part and
under Type, choose “Random draw from question pool” (choose
pool).
Changing and Previewing Assessments
 You may change the settings, parts, and questions for an
assessment at any time BEFORE it is published (unpublished
assessments are listed under Core Assessments on the T&Q home page)
 After an assessment has been published, you may change only the
following attributes:
 Delivery, Due, and Retract dates
 Gradebook options
 Feedback Delivery options
 Feedback Components options
 To preview an assessment before publishing it, display the Questions
screen for the assessment and click the Preview Assessment link
 By default, only students or site members with the appropriate
permissions may submit an assessment. Instructors and site organizers
may preview an assessment but may not submit an assessment.
Publishing Assessments
When you publish an assessment, you make it available to your site's
participants. After the assessment has been published, its contents
CANNOT be changed.
1. In the Core Assessments section, click the Settings link under the
title of the assessment you want to publish.
2. Work through the sections by opening them and choosing the
appropriate options for this assessment.




To open all sections, click the Open link under the title of the screen.
To close all sections, click the Close link under the title of the screen.
To open a section, click the triangle icon (►) for it.
To close a section, click its triangle icon (▼) again.
Publishing Assessments (2)
3.
4.
Make sure all settings are configured as you want them. Be
sure to click Save Settings and Publish. If you click Save
Settings, the quiz will NOT be published.
Sulis displays a confirmation screen. Click Publish.
Working with Scores and Answers
In the Published Assessments section, find the assessment you
want and click the Scores link under its name. Sulis displays the
Total Scores screen
 To adjust a student's total score and add an overall comment, enter an
adjustment to the score in the Adj box. To add a comment, enter it in
the Comment box. After entering changes for all appropriate
participants, click Update. Sulis calculates and displays the final scores.
Sulis generated
this total
You entered this info
Working with Scores and Answers (2)
 To see the submission status of an assessment for all
participants, click the Submission Status button
 If the Date entry is blank for a student, s/he has not submitted the
assessment.
 You may contact any student who is expected to submit an
assessment by sending him or her an email (Click the Email link
under the student’s name)
 To allow one or more students to retake an assessment, click the
Allow retake? link under the student’s name
Working with Scores and Answers (3)
 To see individual questions and all responses to it to quickly
compare all participants' answers to any question, click the
Questions option
 The Question section displays the question, the number of points allotted to
it, all possible answers, and the correct answer(s)
 The Responses section lists all participants by user name, date the
assessment was submitted, score on the question, student's response, and
a box for comments. You may view responses, view the highest scoring
submission for each student or all submissions, and view each response.
You can also download all files uploaded by students in response to file
upload questions
 You can also adjust a student's score and add a comment for one or more
individual questions
Working with Scores and Answers (4)
 To see statistics for the assessment as a whole and for each
question and its responses, click the Statistics option
 The Total Score section shows the total number of submissions and the
total score possible on the assessment, along with statistical information
such as the mean, median, mode, and range of the scores for the
assessment as a whole.
 Below, Sulis displays statistics for each question and its possible answers
(with a check mark for the correct answer
), a histogram showing the
number of students who selected each response, the total number of
responses to the question and the percentage of participants who gave
the correct response.
Working with Scores and Answers (5)
 You can also see an item analysis for each question
 … and also download all questions and scores into Excel
Retracting a Published Assessment
To remove a test from students’ view, you need to retract it.
 After you have retracted a published assessment, you can
change the settings, parts, and questions in it, as needed,
and then republish the assessment.
 If you change a retracted assessment and republish it, the
newly published assessment will have a different URL
1.Find the assessment in the Published Assessments
section
2.Click the title of the assessment and click Edit to retract
Exporting Assessments
 You may want to export an assessment to upload in another
course or share with a colleague
1. In the Pending Assessments section and click the Export link under
the title of the quiz, then select QTI v 1.2 and click Export
2. Your browser displays a window with downloading instructions (click
Download; after the browser displays the XML file, click File/Save
Page As, enter a name for the new file, and save it in your pc)
Importing Assessments
You can share quizzes among instructors with the
export/import features
1.
2.
Lecturer A follows instructions to export quiz (previous slide),
and sends the file to Lecturer B
Lecturer B goes to T&Q home page (instructor view). In the
New Assessment section, clicks the Import Assessment
button and follows instructions.
Note: you can use this to reuse quizzes from another course,
but using Question Pools is a preferred solution
Assessment Types
Assessment Types are collections of settings (no questions) that are useful
if you are going to have publish several of the same type of test
 Predefined types (choose existing assessment type list). You can edit
them if you want. Please check their settings carefully before choosing
one! (e.g. grading options?)
 You may create your own:
1. Click Assessment Types button
2. Type a name for new assessment type, click create.
3. Choose appropriate options in each section (click ►to open
each section). Be careful with “can edit default value”
Questions?
Angélica Risquez
Technology Enhanced Learning Advisor
Centre for Teaching and Learning
Millstream Courtyard
University of Limerick
Limerick, Ireland
[T] +353-61-202 580
[F] +353-61-338 044
[E] angelica.risquez@ul.ie
[W] www.ul.ie/ctl