Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
1
Issue #1a
Email exchanges
Created on Jan 10, 2016 5:23 PM by Tim Payer
So much of teaching Statistics is about prompting the student to express their findings with notation
and numeric values paired together.
This means a mixture of two data types: Perhaps a multiple choice pull down of possible notation
answers paired with a numerical value.
Canvas prevents this for several reasons:
1.) Text and numeric data can not appear together in a grouped question.
Okay, how about if we treat the numeric value as "text"? That is a numeric "word" spelled with
integers?
2.) Okay this is a good work around. But then there is another problem: Unless the student gets all
the grouped problems correct or all of them wrong the feedback given to the student will be "Partial".
The "Partial" indicator is all that Canvas is prepared to give in terms of feedback to the student. The
"Partial" result will only sum the total number of correct points out of the total that is possible. "7 out
of 12" would mean 7 out of 12 points have been awarded. But there will be no indication of which
problems are correct and and incorrect. Flag correct answer in green and incorrect answers in red
so that the student can focus on their specific mistakes, rather than remain bewildered as to which
entry was made in error.
And another thing.
If you can ask our school to pay Canvas with six figures after a dollar sign, how come our instructors
can’t ask Canvas for a numerical tolerance that exceeds four figures after a decimal point?
Tim Payer
Renee Carney
Jan 12, 2016 9:16 AM
Thank you so much for taking the time to visit the Community and submit a feature idea. Can you
help us narrow and focus your idea? We're having a little bit of trouble deciphering the issue you are
seeking to resolve.
There are great tips for constructing a feature idea, including use cases and screenshots, in How do I
create a new feature idea?.
Thank you.
Tim Payer Jan 12, 2016 9:37 AM (in response to Renee Carney)
Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
2
Issue #1a
Email exchanges
Hi Renee,
Could I send you a pdf attachment that spells the issue out in excruciating detail with screen
captures and comparative displays with Moodle vs Canvas?
There are several issue but let’s focus on the primary one:
Grouped questions cannot give the student specific feedback.
This venue of "Like" and text reply does not permit attachments.
Who can I send this attachment to (8 pages) that would actually look at it?
Thanks, Tim
Scott Dennis
Jan 12, 2016 10:14 AM (in response to Tim Payer)
Hi Tim,
If you would like to email me your document directly, I'd be happy to pass it along to the appropriate product
manager.
Thanks,
Scottd at Instructure
Renee Carney
Jan 12, 2016 10:24 AM (in response to Tim Payer)
We are all in this together, and we are very sincere in reading all feature ideas and helping them move
forward. There is no need to be rude in our communications.
You can send your document to Scott (above), and/or me (rcarney@instructure.com). I'm not sure that we
need to look at the Moodle vs Canvas, yet. It is a fact that different LMS features function differently. Let's
explore how the feature in Canvas is and is not functioning as you expect, first.
Thank you for specifying the focus of your issue: Grouped questions can not give the student specific feedback.
I am assuming that you are creating a quiz, then within that quiz you create a question group? Within the
question group you create questions? Each question allows for specific feedback. Are you seeking a feedback
option that provides text to a student based on their score for the group only?
Hi Renee,
(Tim Payer response at 9:33 am, Tuesday, January 12th, 2016)
Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
3
Issue #1a
Email exchanges
I am Sorry Renee, I should not have been rude.
I am frustrated at the prospect of losing a comprehensive feature offered in Moodle that apparently Canvas
does not offer.
I should not have been so snippy with you, Sorry.
Could I send you the attachment to better explain the issue I am having?
I appreciate your willingness to converse despite my rude comment.
Tim
(Renee Carney response at 10:40 am, Tuesday, January 12th, 2016)
Thank you so much for the apology. That means a lot!
Yes, I am going to look at your PDF now and see what we can do to help you out!
(Renee Carney response at 11:42 am, Tuesday, January 12th, 2016)
Thank you for sending your document. We're looking at it now!
Good Afternoon Tim. (Renee Carney response at 12:59 pm, Tuesday, January 12th, 2016)
I spent some time with some of our Canvas Coaches looking over your document. We worked to see how we
could help address the hurdles you were running into.
First, thank you for the doc. It was
very well thought out. I could tell you
put a lot of time and thought into it. I
could feel your pain whenever I came
to a point in your process where you
found a hurdle.
One of our Canvas Coaches
(Kona) is a Canvas Admin and
teaches Statistics. She said
she uses very similar question
types. She took a screenshot
that demonstrates where to
input custom feedback per
answer, but she did say their
team has opted not to go that
extra step because of the
amount of time for setup (1300
question bank).
Here is the screenshot she shared.
Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
4
Issue #1a
Email exchanges
Here is an example of
what the student would
see upon the quiz
review. (Review does
have to be open for the
student to see the
feedback)
I think those two
screenshots may help
with one of the issues
you were running into?
There are 4 other hurdles that we do not have a solution for, yet. (Two of which I found feature ideas for. I
thought I had seen the others before. If I cannot find them, these may be the 2 we want to submit)
 Differentiated points for multiple answer questions
 Mathematical answer variations: 0.20 = 0.2
 Fill in the blank questions allowing mathematical answers
o Allow Content editing for the fill in the blank
 A retake that allows students to correct only their mistakes
o Quiz retake work on only incorrect response
I think a few may be covered in a new Quizzes.2 project that is just kicking off. I have sent your document on
to our Quizzes Product Manager and he has let me know that he will take a look.
I will add a summary of this email to the idea thread, in case anyone is following it. Would it be ok with you if I
also shared your document there, or do you prefer it to stay private?
I look forward to hearing back from you and I'll send you more info when I hear back from the Product
Manager.
Hi Renee,
(Tim Payer response at 2:19pm, Tuesday, January 12th, 2016)
Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
5
Issue #1a
Email exchanges
Thanks for responding so quickly. I appreciate the effort and detail you went through to explain this response.
Yes please do show the document to any and all that are interested. I actually have many more documents that
I am currently sharing with our Campus LMS group. I will send more if they are desired. But the one I sent you
was the key issue.
But I think this key issue of feedback is still lacking.
When you mention in the last screen shot that "Review does have to be open for the student to see the
feedback".
By "Review" you mean that the instructor must check "Let Student See Answer" for a given quiz? Which is
problematic: The student wants to make more attempts on the quiz, and the only way that the specific green
and red flag feedback can be given is if the instructor lets the student "see" ALL the answers to the quiz the
student is taking?
The written feedback notes are a nice feature, and this I have seen, however
the core problem is not resolved.
Could I try and specify the issue one more time?
Ideally what I would like is for those red and green flags to pop up with every student submission within a
grouped question set.
But it is important here that the correct answers are NOT revealed to the student. With unlimited attempts the
student can then know which problems are wrong and then work on correcting those wrong answers on
subsequent attempts. If the answers are given (as in Review Mode) I am sure that you are aware that many
students will simply copy the correct answer on their future attempts. (The written comments are nice but
secondary.)
My current understanding of how Canvas runs quiz feedback now is as follows:
1.) Grouped questions must use all the same question type:
Either all "Multiple Fill in the Blank", or all "Multiple Drop-down".
It is not possible to mix data types of numeric and text.
It is not possible to weight individual cells within a grouped question with different point values.
2.) "Partial" is the only feedback that a student is permitted for a grouped question. The student will be given a
point tally of corrected points submitted as in: "7 of 12". But the student will not know which her entries are
correct and which are incorrect. (This will frustrate the student, and this is the key issue.)
3.) If each and every cell of a grouped question is separated into individual questions then an instructor will
have control of the data type used and the points assigned. A student will have their incorrect answers red
flagged, and correct answers will not have a green flag. The blank non-response is the indication of a correct
answer. (A green flag would be nice, but again this is secondary.)
So it seems there is no way around Issue number 2. Does this seem to be the case?
I can work around all the other issues but that "Partial" feedback issue is going to be a major problem.
Please let me know if you can think of any other work-around solutions.
I appreciate your efforts Renee.
Sincerely, Tim
Tim!
(Renee Carney response at 4:18pm, Tuesday, January 12th, 2016)
Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
6
Issue #1a
Email exchanges
We'll keep trying until we get this
Thank you for the permission to share the
document.
There is one small setting that I think may help
you when it comes to the review. You can
allow students to see if their answers were
correct or incorrect, without giving the correct
answers! I took a quick screenshot for you.
"Let Students See The Correct Answers"
should be deselected if you want the correct
answers hidden. Does this help with the main
issue?
Responses to your numbered questions/issues:
1) You are correct that the multiple answer questions (or as Moodle calls them, grouped questions) are either
dropdown select, or text. There is no mixing and matching. You are also correct that answers cannot be
weighted within those question types. I think these two ideas would make perfect feature idea submissions!
2) If you allow the student to review their quiz, but turn off the correct answers feature, they will be able to see
which answers were incorrect without giving them the answers!
3) You taught me something new today! I guess I never realized that the correct answers did not have a flag.
Below is a screenshot that demonstrates 1) what the review looks like with correct answers turned off. 2) using
the feedback tool to tell the student when a multiple answer - answer is correct. (don't mind the
cows,
when you build a test course, you use what you know!)
Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
7
Issue #1a
Email exchanges
Let me know if we're getting closer!
Hi Renee,
(Tim Payer response at 4:18pm, Tuesday, January 12th, 2016)
I appreciate all the detailed screen shots as it helps communicate the point, but Sorry no, we are not any
closer.
Many students will be making multiple submissions (between 4 and 50 submissions per student) on a quiz
module that has been adapted for homework assignment submissions.
We want the student to check their notes, talk to class mates, come in to office hours, struggle and search the
web in an effort to find the correct process for a calculation or a text answer before the due date. Revealing the
answers to the student would only make sense After the assignment is due. Otherwise the student will take the
correct answer and submit it in their next entry.
So I am afraid we are back to square one on the main point of specific feedback on grouped questions.
Thanks for the persistence though, Tim
Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
8
Issue #1a
Email exchanges
P.S. Cows are sweet. I live next to a pasture for grazing cows and they sometimes wander up to the barbed
wire fence that separates our back porch and the pasture just five feet away from our porch couch. If you
remain perfectly still they wont spook.
Hi Again Renee,
(Tim Payer response at 4:38pm, Tuesday, January 12th, 2016)
Let me take another look at your cow problem.
I misunderstood what you were implying with the "correct answer" response.
You are saying that correct submissions within a multiple answer problem will be flagged with a "correct
comment" and the incorrect replies will not be flagged with a response?
This helps, yes, I think.
So lets say I had a multiple answer question with 9 cells for the student to enter. If the student made 5 correct
entries of those 9 cells, that student would see 5 "correct comments" next to the summary of their submitted
answers? And no comment will be next to their incorrect answers letting the student know that these
submissions are in error? Provided of course that I have created such a comment for each submission?
I am going to give it a try on some quiz assignments that I had separated into single question single responses.
It will take me a while to do and test but I will get back to you tomorrow.
Thanks so much for your persistence,
Sincerely, Tim
(Renee Carney response at 6:55am, Wednesday, January 13th, 2016)
The grazing cow pasture next to you sounds amazing! We raise a few cattle. Ours are pretty tame and love to
come to the fence for a treat and a good scratch!
In your last email, it looks like you are on the right track for testing. Looking forward to hearing how it works.
Hi Renee,
(Tim Payer response at 8:48am, Wednesday, January 13th, 2016)
Okay, I have made a test run with my first Homework problem, and I think that the solution you offered of letting
comments for correct feedback function as a flag will work! Or at least we are closer.
Thank you for going the extra mile to offer a solution.
It is going to be a lot of extra work though. Numeric values in particular when entered in the "Multiple Fill in the
Blank" mode will require the creation of many possible "correct" answers.
For example if a correct answer is numeric : "0.1234" this would have to be considered as text with a particular
numeric spelling. And if I want to give a tolerance of plus or minus 0.0005, this would mean writing up 21
possible numeric words of accepted spellings:
For example:
The following would all need their own comment to be able to be flagged as correct:
0.1229
0.1230
Canvas Issues for the Conversion of Moodle Quiz Modules
Stat 109 Trial, Spring Semester 2016
9
Issue #1a
Email exchanges
0.1231
0.1232
0.1233
0.1234
0.1235
0.1236
0.1237
0.1238
0.1239
And another 11 "spellings" for students who clip the leading zero:
.123
.1229
.1230
.1231
.1232
.1233
.1234
.1235
.1236
.1237
.1238
.1239
It will mean at least 4000 to 5000 entries of the comment of "Correct!" for 30 plus assignments, but this is
preferable to the lack of feedback for the students.
I better get to it.
Thanks for your help,
Tim