UNIVERSAL STUDENT
RATINGS OF
INSTRUCTION (USRI)
COURSE EVALUATION
Jean Gomes,
Office of Institutional Analysis
October 27, 2015
Take note for 2015-2016:
 Amanda
 Stephen
 Successfully re-using returned blank forms in the
spring and summer evaluations
 Inactive instructors – results can no longer be
viewed by students
 IE browser requires setting USRI link as a trusted
site
 Please help to ensure a class is not evaluated
twice – on paper and online
VP Planning & Resource
Allocation
Office of Institutional
Analysis manages ….
Academic
Programs/Depts/Faculties
VP Teaching & Learning
USRI Course
Evaluation
project
IT, Enterprise
Reporting
General
Faculties
Council
(GFC)
Imagine Printing
- Cognos reports
(maintained by ER-James)
- Course timetable (ES)
- Instructor data (HR, ES)
Admin-verified course data
file is finalized
- Online surveys: student
- Cognos
access via portal (IT
reports
Help,IT-Quoc)
(maintained
- Hope to use D2L (ITby ER-James)
Heather & Darcy)
- Loading (IT- Communications:
Quoc)
Course and student
data Relations,
University
files are loaded into
SES, StdSuccessCtr, SU,
GSA, Residence,
ClassClimate
- ClassClimate VividForms
Reader scanning issues
(IT-Denver)
TriMediaAlliance, D2L,
depts.
Paper USRI surveys are
scanned by Imagine Printing
- Cognos reports
(maintained by ER-James)
Datamart (ER-James)
Admins return USRI
Results reports (ER-James)
envelopes
USRI Login
(IT-Hayes) to Imagine
to USRI Team
Access toPrinting
results (IT,orES):
portal, USRI website, Student
Centre
Scanned and online results - Level of access/security (HR, ES,
IT): job codes,
sessionals,
student,
are checked/tested/reCheck
for USRI
(paper)
instructor
who
was
a
student
scanned if necessary
surveys-with-missing-data
USRI
PROCESS
Surveys (etc.) are
generated. Imagine
Printing prints and delivers
forms to Admins
Class evaluations (paper or online) are
conducted. Admins coordinate paper,
in-class evaluations. OIA conducts
online evaluations.
Export data from
ClassClimate
USRI reporting system creates Section Rating Reports (USRI results)
RELEASABLE RESULTS
Online Faculty
Survey results
emailed to Admins
by USRI Team
NON RELEASABLE RESULTS
- Class Climate software upgrades
(IT-Karen, Ivan)
- IT Support for help
USRI Admins view
results via USRI
website
Deans/Dept Heads
may view own
faculty/dept results
via USRI website
Students may view
results via ‘Class
Search’ in their
Student Centre
Instructors/current sessionals may
view results (only their courses) via
My UofC portal
USRI Timeline
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Spring
Summer
Winter
Fall
Winter
= USRI evaluations are conducted in classroom and online
= USRI project tasks and deadlines
Jan
Feb
 About 80 different units
 About 70 USRI Administrators
 About 2200 classes evaluated – about 25% of
surveys are online
 About 20 different Faculty Surveys (student
comments)
USRI Reporting System
USRI Course Evaluation Results Release Dates:
1) Fall results – January 15
2) Winter results – May 15
3) Spring, Summer results – September 15
What happens on these dates?
- USRI results for the latest term become available
- Official summary reports are generated and
Department Heads and Deans may access these via
the USRI reporting system.
Outline
I. USRI Course Evaluation Process
• Validating course data
• Coordinating in-class paper evaluations
— Handouts for Admins; Coversheets; Blank/generic USRI form
• Faculty Surveys (student comments)
• Online Surveys
— Accessed through student portal; “How Students Complete their Online Surveys”
II.
•
•
•
•
•
Accessing USRI Results (“Section Rating Reports”)
a) Instructors
b) USRI Admins
c) Deans/Dept Heads
d) Students
USRI Admin request: adding and deleting a person’s login access to USRI reporting
system
• USRI Admin request: exemption of USRI results due to special circumstances
III. Communication Plan and Advertising
IV. Imagine Printing
•
Printing forms, supplies, delivery, pick-up
V. Resources - USRI Website
I. COURSE EVALUATION PROCESS
Validating Course Data
Why do we ask Admins to review the list of classes?
• To make any needed corrections to Student Records
(BAS PeopleSoft Student Administration)
• To confirm the list of classes to be evaluated. Surveys
are generated for each class on the list.
• To inform us of any classes to be added/deleted from
evaluation.
• To inform us of anything needing special attention
(e.g., an online survey instead of paper; instructor
name change, class end date does not reflect actual
course instruction, etc.)
• To produce a “cleaner file” -> reduces surveys with
missing data
Some of the criteria for selecting course data
extracted (using Cognos Query) - example
 Instruction Role: AL - Alternating Instructor, PI Primary Instructor, SI - Simultaneous Instructor
 Session Code: -, 1-Regular Academic, 6W1-First 6
Weeks, 6W2-Second 6 Weeks, BW-Block Week,
GSP-Group Study Programs, NS-Non-Standard, PAMulti-Term: Part A/Part B, UCQ-Qatar Nursing
Program
 Instruction Mode: BL-Blended Learning, EPEnglish Language Program, P-In Person, WW-WebBased Instruction
 Enrolment: 8 or more registered students
Coordinating In-class Paper
Evaluations
 Two handouts are given to USRI Admins
1. HOW TO CONDUCT USRI IN-CLASS ADMINISTRATION
Notes for Faculty/Department USRI Administrators -- In-class Administration




—
—
—
—
—
—
—
—
—
—
—
At the agreed upon time, the Department USRI Administrator/designate should arrive at the classroom with the paper USRI forms,
and Faculty Survey forms (if applicable).
It is important to keep the USRI forms and the Faculty Survey forms in separate envelopes.
If possible, bring extra blue or black ink pens to be given out to students that do not have a black or blue ink pen. (NOTE: do not use
red ink.)
The Department USRI Administrator/designate should complete each of the following:
If the course and instructor information is pre-printed on the survey, ask the instructor to verify course name, number and their name
as printed.
Ask the instructor to leave the room, and agree upon a location that the Administrator will find the instructor when the evaluation is
completed. Inform the instructor that administration will take approximately 15-20 minutes.
Read the prepared statement below explaining the purpose of the evaluation and instruct the students on how to complete the
evaluation forms. (See second handout.)
Have student volunteers in larger classes help hand the evaluation forms around the room. Students should not start filling in the
evaluation until instructions have been explained.
Collect the completed and incomplete USRI and Faculty Surveys. It is important that the two types of surveys are kept in separate piles.
Place the completed USRI evaluations in the USRI envelope.
Place the completed Faculty Surveys in the Faculty Survey envelope.
Unused forms should be collected, but not placed, in the envelopes. Keep unused forms in your office for future use.
Seal each envelope, and have the seal signed by the last student submitting or a volunteer.
The completed forms should be securely stored by the Department USRI Administrator until completion of the survey period.
Starting Winter 2011: coversheets not delivered inside survey envelopes. Admins asked to place the coversheet into envelope
before returning survey to Imagine Printing for scanning.
2. PREPARED SCRIPT TO READ TO THE CLASS

The University of Calgary is currently working on improving the administration of the Universal Student Ratings of
Instruction (USRI) with the goal to increase student response rates.

For this course, the USRI is being completed in class. You may be asked to complete online evaluations in some of your
other classes. You may or may not receive an email. All students should check for online surveys via their portal.

You may use either a black or blue ink pen. DO NOT USE RED INK PLEASE. Mark responses with an X inside the box. If
you need to make a correction, black-out the box with the incorrect X and then enter an X in the correct box.

The information you provide on this rating instrument is intended for use by students to aid them in selecting courses, by
Deans and Department Heads to assess instructors, by instructors to assist them in improving instruction, and by the
University for administrative research purposes. Therefore it is essential that you answer the questions carefully. If you
have concerns about the course or instruction that are not addressed by this instrument, please speak to the Department
Head or Dean.

Your participation in this evaluation process is gratefully acknowledged by the Students' Union, Graduate Students’
Association, Instructors, and Administration of the University.

When you have finished please hand in your completed surveys. If you are doing USRIs and Faculty Surveys please put
them into separate piles. I will ask for a volunteer to initial the sealed envelope with all of the completed evaluations
before I leave the room. I will also sign the envelope, and will then take them to be processed. Your individual evaluation
will not be shown to your instructor. Your instructor will get a report on the aggregated results from this class after the
final grades have been officially submitted.

Please begin to fill out the evaluation now. Thank you.
Sample of Pre-printed Coversheet (24 lb paper)
Sample of Manual Coversheet
 Blank/Generic
USRI Form.
 Starting in Winter
2011 USRI
stopped asking for
student id #
Some reminders:
1. Always complete the manual coversheet information:
— Print identical name for instructor, as entered into BAS (PeopleSoft)
— Print the course name/title, as listed in BAS (PeopleSoft)
2. In the pre-printed coversheet, the barcode contains the information. You cannot
change the information manually, (e.g. crossing-out text and writing in the
correction) because your changes are not recognized by the scanner.
**Do not write on the USRI questionnaire, especially over a checkbox and barcode.
Please advise USRI team if there are changes/corrections needed for this class.
3. Per GFC: When there are 4 or more multiple instructors in the course, one survey
is conducted to represent all the instructors. Admin should select one of the
instructors to ‘hold’ the data.
- In classes with 3 or fewer instructors, each instructor is surveyed.
- Combined courses with multiple instructors (error in calculating response rate)
4. USRI forms are printed on 24 lb paper
- Forms cannot be photocopied.
- Contact USRI team if more copies are needed. “Normal” paper will not scan.
Faculty Surveys (student
comments)
•
•
•
Faculty Surveys are printed on yellow-coloured normal paper as they are
not scanned
We do not support a class evaluation if it uses only the Faculty Survey
Responses from online Faculty Surveys will be emailed to Admins, as
soon as possible following the release date – we try for within 2 to 3
weeks.
•
ALL online Faculty Survey responses will be emailed to Admins
Online Surveys
 Used for classes held off campus, web-based classes
 except for Environmental Design, and Werklund School of
Education
 USRI Admin may request an online survey if there are special
circumstances in which paper cannot be used
 USRI Admins should notify course instructor
 USRI Admins should send mass emails to their students asking them to
check for online surveys to complete
 USRI team may send out reminder emails to students who have online
surveys to complete.
 Students access via MyUofC Portal (see following screenshots):
“How students complete their online surveys”
How students complete their online surveys
 Login to My UofC portal (top right)
Once logged in, click on the ‘Academic’ tab
Click on “Evaluate my courses” which will take you to the USRI
site with the list of available surveys. Click on Survey to start the
survey.
(See next page for example)
How students complete their online surveys
(continued)
II a) Accessing USRI Results
- Instructors
 From MyUofC portal
Step 1: Click on the “Academic” tab
Step 2: Click on “View Course Evaluations”
Accessing USRI Results
(Instructors) continued
Step 3: Click on “View USRI Results” tab
Step 4: Click on “Section Rating” to view the
report.
II b) Accessing USRI Results
- USRI Admins
1. Go to USRI web site http://www.ucalgary.ca/usri/
- Click on Administrators tab
Accessing USRI Results
(USRI Admins) continued
2. Click on “Go to Results”
Accessing USRI Results
(USRI Admins) continued
3. Sign in with secureID
Accessing USRI Results
(USRI Admins) continued
4. Select Term, Course Career
*** Admins are able to access only releasable results
Accessing USRI Results
(USRI Admins) continued
5. Select parameters:
Faculty
-> Department
-> Program/Subject
-> Course (Optional)
(Remember to refresh the next
section after each parameter is
filled out)
e.g. Faculty of Arts -> Refresh
Department
Accessing USRI Results
(USRI Admins) continued
6. Search for instructor name.
-
-
Scroll to search area in the bottom
half of the window
Type a name in the search window
(e.g. Hunter)
Click Options
- You will get a list of items, choose
“Contains any of these keywords”
Accessing USRI Results
(USRI Admins) continued
 Result of the Name Search:
— The Instructor name(s) and ID(s) will display in the
Results window. Select and click ‘Insert’ to bring to the
choices window. Click Select all, then click ‘Finish’.
Accessing USRI Results
(USRI Admins) continued
7. Choosing Section Rating Report
In the grey box below …
- All blue categories (e.g. Term, Program/Subject, Course Career,
Course, Faculty, Instructor, Department) can be edited to view different
reports
(e.g. Winter 2014 can be changed to Spring 2010)
Accessing USRI Results
(USRI Admins) continued
Your role determines which Overall Rating reports you can view:
Department Admin -> Can view only Department Level Report
Faculty Admin -> Can view Faculty Level Report, and Department Level Report
Super user -> Can view ‘Error Reports’ (see example above)
Accessing USRI Results
(USRI Admins) continued
8. Click on ‘Section Rating’ to get the report
Accessing the USRI Results
(USRI Admins) continued
Section Rating Report example (page 1)
Accessing USRI Results
(USRI Admins) continued
9. Click on ‘Instructor Rating’ for all reports for the
instructor for up to the last 10 years
II c) Accessing USRI Results
- Deans and Department Heads
1. Go to the USRI web site:
http://www.ucalgary.ca/usri
- Click on Administrators tab
Accessing USRI Results
(Deans, Department Heads) continued
 Click “Go to Results”
Accessing USRI Results
(Deans, Department Heads) continued
3. Sign in with SecurID
Accessing USRI Results
(Deans, Department Heads) continued
4. Select Term, course career
-
Note: As of September 2013: USRI Admins can access only releasable
results
Accessing USRI Results
(Deans, Department Heads) continued
5. Optional Parameters
Note: Only dept/faculties linked to your courses will display in the
window.
Selections may be made on the following:
Department(s)
->Refresh Program/Subject(s)
->Select Program/Subject(s)
->Refresh course
-> Select course
-> Finish
II d) Accessing USRI Results
- Students
1. From the U of C’s website, click on MY U OF C
Accessing USRI Results
(Students) continued
2. Enter UCID and password
Accessing USRI Results
(Students) continued
3. Once signed in, the Student Centre should
automatically open. Click on the ‘Search For Classes’
button on the right hand side of the screen
Accessing USRI Results
(Students) continued
4. Enter search criteria, and once finished, click
‘Search’ at the bottom-right hand side of the page
- Additional search criteria can be added to the search if wanted
E.g. Search for Psych 433 course in Winter 2014
Accessing USRI Results
(Students) continued
5. Using the previous Psych 433 example, the sections
are displayed here.
- If there are USRI results available for the class, a link will be
displayed under the course name and number.
Click on the
USRI link
Accessing USRI Results
(Students) continued
6. The Usage Advisory screen will open. If you agree
to the user conditions, click “I AGREE”.
Accessing USRI Results
(Students) continued
7. You can choose whether you want to look at
reports for your course for all the terms it was
taught, or only the reports for the latest term. In
this example, ‘USRI results for all terms’ is
selected
Accessing USRI Results
(Students) continued
8. Following the previous example, you will then see a list of all the terms
in which the course you selected was taught. The USRI Reporting
System contains reports since 1999. Selecting ‘Section Rating’ will give
you the results for the specific class and instructor. Selecting
‘Instructor Rating’ will give you a list of all the different courses this
specific instructor has taught since 1999.
II. Adding/Deleting a Person’s Access
to USRI Results
USRI REPORTING SYSTEM
 A USRI Admin must contact the Office of
Institutional Analysis, who will in turn submit a
request to IT (Enterprise Reporting)
 Please notify OIA for:
— changes in USRI Admin
— changes in Dept Head, Associate Dept Heads
— changes in Dean
 Need to limit the number of users
 Sessional instructors have 1 year grace period
II. Exemption of USRI Results
EXEMPTION
 A USRI Admin makes the request, on behalf of the
course instructor, to remove results due to special
circumstances.
— Evaluation would not be fair assessment of instruction
— Instructor was trying out new course material/format
— Instructor was unable to teach most of the course
III. Communication Plan and
Advertising





USRI Admins promote USRI within their own units
Course instructors
Students’ Union
Graduate Students’ Association
Student and Enrolment Services, Residence Services;
Orientation; New Student Registration Assistance;
Service Stop
 UCIT (future - portal messages, Course Desire2Learn)
 University Relations (e.g., website messages)
 Various communications offices & media (e.g., NUTV)
IV. Imagine Printing
-
Printing USRI and Faculty Survey forms




Labelled envelopes
USRI forms (white color and 24 lb special paper)
Faculty Survey forms (yellow color and normal paper)
Pre-generated coversheets (white and 24 lb special paper for USRI
and yellow normal paper for Faculty Survey).
 We do not print Faculty Survey coversheets
-
Delivery:
 Based on the delivery list provided by OIA (updated by Admins)
USRI/Faculty Survey forms, pre-printed coversheets and envelopes
(with course information labels on the envelopes) are delivered to
every department/faculty.
 For each class, all of the coversheets will be placed in the class’s
envelope.
 After the survey is done, it is the Admin’s responsibility to insert
the coversheet into each envelope.
IMAGINE PRINTING SERVICES
(continued)
 Contact for Pick up and Delivery:
 Please phone Imagine Printing Services at 220-7497 to
make arrangements. Let them know your location and
contact information.
 Carol Fitton: cfitton@ucalary.ca; Carol.Fitton@xerox.com.
 Faculty Survey forms will NOT be scanned. Do not send
any faculty survey forms to Printing Services. Please
keep the completed forms in your own
department/faculty.
V. Resources
www.ucalgary.ca/usri
Internet Explorer is the preferred browser. If you get
an error message, try adding the USRI link as a
trusted/secure site.
- Instructions posted on USRI website
Other important contacts:
 Course and instructor timetabling: Barb McCutcheon,
bvanderk@ucalgary.ca, Enrolment Services
 IT Support: problems logging into the USRI reporting system
 Imagine Printing: deliveries/pick-ups
PREFERRED EMAIL: usri@ucalgary.ca
Office of Institutional Analysis: Jean Gomes, Amanda Chan, Stephen Childs
Thank you to all the USRI Admins for your ongoing
work on the USRI course evaluations!