STLF Report to CWSEI and Mathematics Department
STLF: Costanza Piccolo
Period: 05/03/10 – 31/03/10
Submitted: 31/03/10
Specific activities performed by STLF
1) Professional development
 Attended the weekly Reading Group discussions
 Attended weekly STLF meetings
 Presented at the Math TA accreditation seminar series (jointly with Warren): the seminar was
meant to be a brief introduction of the Calculus Workshop Program for the graduate TAs.
 Attended talk by R. Shavelson on March 25.
 Attended PHYS 304 on March 29 and EOSC 355 on March 30: none of these lectures were
particularly inspiring, probably due to the pending midterms; will visit more classes after the
Easter break, if time allows.
2) MATH SEI general meetings/activity
 Met with the Math-CWSEI group to plan workload for next year.
 Attended monthly meeting with Carl and Math-CWSEI group on March 5th
 Met with Carl and Warren to discuss students’ difficulties with translation problems and
relative teaching strategies, also discussed how to avoid overlap of my projects for next year
with Warren’s.
 Met with Myriam, my TA, for an update on her work (she has been helping me collect
student from past exams, among other things)
 Attended meeting with Carl, Warren and Sandi to discuss Sandi’s new project.
 Met with Depart. Head to discuss structural changes within the Math-CWSEI group.
3) Course-specific meetings/activities
MATH 180/184 Workshops Project
1. Completed the analysis of workshop data: performed correlation analysis between workshop
scores and course grades, compared results with last year’s data, analysed attendance data to
look for possible factors that may reduce student attendance. I wrote a detail summary of my
results and sent it to the workshop coordinator. The file is too big to be sent as attachment (I
haven’t figured out how to keep file size small in Word and still be able to include graphs!), so
I’ll summarize the main points here:
a) There is a strong correlation between workshop scores and (adjusted) course grades
(r = 0.67, workshop scores are the combined attendance+participation+quiz scores, adjusted
course grades are final course grades without the 10% workshop contribution). Even if you
look at attendance alone, this year the correlation is much higher than last year: in 2008 in
Math 184 the correlation coefficient for workshop attendance vs final course grade was r =
0.37, this year is r = 0.52. This is clearly an effect of the new and more sophisticated
assessment system (participation+quiz) implemented in the workshops this year.
b) The distribution of workshop marks and course grades don't match, which is not surprising
considering the heavy weight on attendance/participation on the workshop scores and the fairly
easy workshop quizzes. However, engaging consistently in the workshops (measured by a
workshop score of 80 or higher) corresponds to a very high percentage of passing grades (only
~5% of students with a workshop score of 80 or above failed the course, overall failing rate
was 19% in Math 180 and 17% in Math 184).
c) Overall attendance was good, and comparable to last year. Only 7% (Math 180) and 9%
(math 184) of students attended 5 or less workshops. Attendance was lower on Wednesday and
Friday, with the 8am workshops being the least attended sessions. Course instructor and GTA
don’t seem to be variables that affect attendance. We’ll use this information to improve
workshop schedule for next year.
d) Normalized, average (by section) quiz scores are almost all above 2 (out of 3), showing that
the level of difficulty of the quizzes was adequate (the normalization was based on attendance).
2. Reviewed workshop schedule for next year and helped with the selection of suitable
classrooms for a workshop.
MATH 307 – Applied Linear Algebra
1. Edited lecture notes to update learning goals, developed Matlab-specific learning goals and
compiled a list of minimum goals that all students taking this course should achieve, regardless
of the type of applications covered by the instructor. Document is attached.
2. Richard asked the other faculty member who is currently teaching the course (and has taught it
often in the past) to provide feedback on our list of minimum goals, we are waiting for his
response.
MATH 152 – Linear Systems
1. Completed data analysis of the Mech 2 August 2009 Entry Quiz and the Math 152 April 2009
Final Exam, and compared results with the corresponding 2008 data set. The comparison with
the 2008 data set can give us a measure of the effectiveness of the new computer labs
developed in 2009. Briefly, I collected data to measure a) student learning of Matlab syntax
and “for” loops in Math 152 (final exam data) and b) how much of that knowledge is retained
after 4 months (entry quiz data). I found that:
a) Student learning of Matlab syntax seems to have improved when compared to the 2008
data. Unfortunately, over the two years the exam questions changed sufficiently to
make a straight comparison difficult, however, the percentage of correct answers nearly
doubled in 2009 (44% in 2008, 86% in 2009), suggesting a possibly significant change
in learning. A more direct measure of improved learning was possible using the Mech 2
quiz data: the percentage of students who responded correctly to the same question
involving a “for” loop increased from 45% in 2008 to 63% in 2009 (these data were
presented in my previous report). To confirm students’ understanding of “for” loops,
the 2009 quiz had a second question on this topic, which 79% of students answered
correctly.
b) Retention of knowledge of Matlab syntax doesn’t seem to have improved from last
year. In 2008, retention was measured on 3 questions: on average, 55% of students who
answered these questions correctly in April retained their knowledge in August. In
2009, retention of syntax knowledge was measured with only one question: 57% of
students who answered the question correctly in April did so again in August. Retention
of “for” loops however showed a much higher rate: 89% of students who answered a
question involving “for” loops correctly in April did so again in August. This is not
surprising, I think, because mastery of Matlab syntax is based on memorization and
regular practice, while for loops require more conceptual understanding. We expect
retention of Matlab syntax to increase this year thanks to the more frequent testing of
this topic in the weekly homework assignments than in the past.
I wrote a detail report on this, with sample of exam questions, and sent it to the instructor and
Warren for their information.
2. Analysed performance on Matlab questions in the April 2009 Final exam. The exam had
several short answer questions, 5 of them that had some Matlab. The average score on these
questions was 71%, the average score on the remaining short answer questions was 65%, so
students seem to have learned Matlab just as well as the rest of the material. Also the
correlation between the total scores on these 5 questions and the total exam score was quite
good, r = 0.61, showing that the students who did well on this questions did also well on the
rest of the exam. Again, this information was communicated to the instructor and Warren.
MATH 257/316 - Partial Differential Equations
1. Prepared final version of learning goals. Document is attached.
2. Developed and administered an online student survey to collect data on whether students have
studied/used the numerical techniques/tools learned in this course in subsequent courses, and
collect their opinions on the usefulness of the these techniques/tools.
MATH 256 (Mech 221)
1. Analysed students’ performance on the Math questions of the December 2009 exam (Mech 2
students write a single exam that tests all subjects in the program, so only a few exam
questions are Math related). The exam contained three Math questions: two short questions [5
marks each], one including Matlab (Q1) and one including only analytical computations (Q2),
and a long-answer analytical question (Problem 1) [25 marks, no Matlab]. The three questions
tested different math concepts. Average scores on these questions are all similar, suggesting
similar degrees of mastery of both analytical and numerical tasks.
Exam question
Mean Scores out of 100 +- st dev
Q1(a) [interpreting the output of a “for” loop]
Q1(b) [identifying and correcting errors in the code]
Q1 (a) +(b)
Q2 [analytical question, no Matlab]
Problem 1 [analytical problem, no Matlab]
77.6 +- 33.9
74.8 +- 35.6
76.5 +- 29.3
71.3 +- 16.4
75.9 +- 26.3
The correlation between scores on the Matlab question and the other exam questions is
small, and so is the correlation with the lab scores:
Correlation coeff. between Q1 scores and Q2 scores
r = 0.26
Correlation coeff. between Q1 scores and Problem 1 scores r = 0.23
Correlation coeff. between Q1 and final lab score
r = 0.15
2. Collected most common students’ errors on Q1 to help designing multiple choice questions for
future assessment; sent this information to instructor and Warren.
3. Last term it was reported that the section attending the computer labs on Mondays had often
difficulties to complete the labs in the allotted time, while students in other sections were
observed to be working through the lab tasks with the help of other students’ solutions. It was
suggested that this behaviour may affect learning, so I looked at student performance in the
Mech 221 December exam (section average score on all Math questions and score on the single
Matlab question), and compared it with performance in the Computer Labs, the Mech 220
Software module given in September (this is the score on questions involving both Matlab and
another software), and the Mech 220 Entry quiz (score on questions that do not include
Matlab) given in August. Results are below.
Mech 221 Exam (All Math)
100%
Mech 221 Exam (Matlab Q)
90%
CLabs
80%
Mech 220 Software Module
70%
Mech 220 Entry Quiz
60%
50%
40%
30%
20%
10%
0%
Mon
Tue
Wed
Fri
The Monday section performed significantly worse in the Mech 221 final exam (Math portion) then
the other sections, however it also started with a weaker Math & Physics background knowledge as
shown by the Entry quiz, which tests 1st year math & physics knowledge. So we cannot conclude that
attending the labs on Monday (when supposedly students may have spent more time on fixing syntax
bugs than understanding the underlying concepts) affected their overall learning of math. However, if
we look at how students performed on the single Matlab question in the Mech 221 final exam and use
the Mech 220 Software module scores as a baseline, then it appears that the Wednesday and Friday
sections displayed higher levels of mastery of Matlab. The Wednesday section is reserved for
Mechatronics students – a special option in Mech 2 for students with an interest in mechanical
engineering, electrical engineering, and computer science. This may explain the considerably higher
marks on the software question. The Friday section is more interesting: they performed slightly worse
than anybody else in the Mech 220 software module, but did better than the early week sections in the
Mech 221 Matlab question, however the differences in the Mech 221 scores across the Mon and Fri
sections are not statistically significant. In conclusion, I don’t think these data show any effect on
learning due to the day of week when students attended the labs.
All these results were summarized into a report and sent to the instructor and Warren.
Current Project Status (material was prepared by either STLF or other members of the MATH
SEI group)
MATH 100/180 –104/184
Learning Goals: Course-level Learning Goals are completed for both courses. Overarching goals for
the workshops are completed. Workshop-specific learning goals have been developed.
Assessments: regular quizzes in the workshops; Midterm survey, end of term survey.
New Methods/Materials: All 24 workshops, 12 per course, have been reviewed.
MATH 257/316
Learning Goals: Topic-level learning goals are completed.
Assessments: student survey
New Methods/Materials: Set of Excel Spreadsheets for students to implement finite difference
schemes for PDEs: spreadsheets are used both as in-class simulations and homework assignments.
MATH 307
Learning Goals: Overarching course-level and topic-level learning goals have been completed.
Assessments: Online surveys
New Methods/Materials: Lecture Notes have been updated.
Plan for immediate future work
MATH 180/184 Workshops:
1. Run a few student interviews to investigate students’ abilities in solving calculus problems and
try to identify which of the steps commonly listed in the literature as key elements of problemsolving are most challenging for our students.
2. Review Math 180 workshop problem sets to improve quality of problems and include problemsolving strategies, possibly based on results from student interviews.
3. Continue the literature search on group work.
4. Survey students with low workshop attendance to poll on their motivations for not attending.
5. Write up results of student interviews that were conducted last year.
6. Prepare workshop observation checklist for future Head TAs.
MATH 307
1. Prepare more assessment tools (lists of questions that can be asked in class, challenging
homework assignments, online pre-reading quizzes and surveys), especially those involving
the use of Matlab.
2. Collect students’ marks on the Matlab questions included in the final exam to assess their
learning and collect most common errors.
MATH 257/316
1. collected data from final exams: the final exam has one question covering numerical methods,
which is part of a larger question that involves analytical calculations. I’d like to compare how
students did on the numerical part vs the analytical part.
MATH 152
1. compare student performance on the Matlab components of the Mech 2 program of the
students who took Math 152 the previous year vs. transfer students. This may give us some
indication of the effectiveness of the labs in Math 152.