Maple TA piloting in School of Social Sciences - Economics 2012-13
Faculty of Humanities eLearning Team
1. Pilot summary and recommendations
2. Key Maple TA features
3. 2012-13 pilot
4. 2013-14 pilot proposal
5. Appendix 1: Economics evaluation report
6. Appendix 2: Maple TA potential beyond the pilot: Disciplines with maths/stats related
courses
1. Pilot summary and recommendations
In 2012-13, the Faculty funded the piloting within Economics (School of Social Sciences) of
Maple TA, web-based software offering enhanced capabilities for assessing mathematical
and statistical material.
Maple TA forms one component of an overall trio of Maplesoft products that also
encompasses Maple Academic (for generating teaching materials) and Maple Sim (for
generating mathematical/physical models).
The pilot ran in four Economics courses (764 students) over two semesters, the first
semester testing basic system requirements and usability and the second semester
deploying Maple TA for formative assessment.
Project review meetings were attended by an EPS representative, Niels Walet, who
expressed interest in assessing the potential of Maple TA for assessment in ‘service’
teaching of mathematics in STEM subjects (teaching mathematics to non-maths specialist
students); this coinciding with their own evaluation of Maple Academic and Maple Sim for
specialist mathematics teaching.
During the pilot, Len Gill in Economics undertook an evaluation of Maple Academic and
reported its value in integrating with and extending the capacity of Maple TA.
An evaluation report of the Economics pilot was produced in July by Ken Clark and Len Gill
(see Appendix 1) identifying clear benefits in deploying Maple TA for both formal and
practice assessment purposes, highlighting its facility for testing of both conceptual ideas
and practical skills, and that positive feedback was received from students.
Their report also requests further/better training opportunities and resources, and fuller
integration with Blackboard to realise significant benefits.
EPS report that “integration with Blackboard would give an enhanced user experience that
would make it of interest…” “…if the cost is not too high, we should definitely have it in our
toolbox.” “To code questions for Maple TA, we need access to Maple [Academic] to test
and try the relevant coding.” (Niels Walet, 9 July 2013 email)
EPS are distributing communication about Maple TA to their Schools to establish interest.
Economics will continue to pilot Maple TA in 2013-14 (via Faculty funding for a further
year) and wish to request University-wide licensing for subsequent years (see
Recommendations below).
1
Further developments following the evaluation report from Ken Clark and Len Gill:





Over the summer, Maplesoft have set up a UK base, offering direct and extended
support (rather than through resellers);
Maplesoft offered Maple TA training for the Faculty of Humanities pilot, for instructors
and for technical administration, to take place before Start of Year 2013;
A Maple TA software upgrade in August 2013 addresses interface/navigation aspects;
EPS initiated and agreed a contract for both Maple Academic and Maple Sim, with
University-wide licensing (and Maplesoft training support);
An invitation to participate in further piloting is being sent out by the Humanities
eLearning team to other disciplines in Humanities that have maths/stats based
component (an initial invitation went out during summer vacation, to be followed up
after Start of Year). [See Appendix 2 for listing of courses where Maple TA could be of
benefit]
Recommendations
From the outcome of piloting in SoSS and additionally with reference to take up by EPS for
the provision of Maple Academic and Maple Sim at University-wide level, Humanities
eLearning and The School of Social Sciences recommends University-level provision of
Maple TA including enterprise wide licensing, integration with Blackboard and a University
level support model.
Both SoSS and EPS have stressed the importance of Blackboard integration for
administrative, teaching and security benefits, particularly with regards to efficiencies in
marking processes. This integration would be achieved through deployment of the
Blackboard/Maple TA Building Block centrally, giving authentication security, single sign-on
(once signed into the Blackboard course, student access to Maple TA becomes automatic)
and automatic transfer of class registers from Blackboard to Maple TA and transfer of
assessment results from Maple TA to the Blackboard Grade Centre (and subsequently to
Campus Solutions where appropriate).
Neil Briggs of the eLearning Applications Team advised that Blackboard Building Block
testing can only take place once a request for change is submitted and approved centrally.
Costings: Maplesoft identify that annual pricing for University-wide provision of Maple TA
is available through eduserv CHEST agreement
(http://www.eduserv.org.uk/lns/agreements/maplesoft-2013#pricingandordering).
Manchester is rated as a large institution, so based on current pricing, annual costs would
be £10945:
Maple TA:
BB connector:
£7295.00
£1825.00
Hosting (organised separately to CHEST):
£1825.00
There is no additional cost attached to support, which will be provided through a
combination of UK-based application engineers and Canada-based Product managers.
The 2013-14 pilot in Humanities gives opportunities to evaluate integration of workflow
between Maple Academic/Maple Sim and Maple TA, working in collaboration with EPS.
2
2. Key Maple TA features
Maple TA allows accurate automatic marking and feedback for a wide range of mathsbased discipline areas. Maple TA allows students easily to enter algebraic or other symbolic
notation, and to sketch graphs or identify graph areas; it recognises mathematically
equivalent responses (e.g. an answer of “y + 2x” would be recognised as equivalent to “2x +
y”); questions can be set up to generate random numerical values into a specific
mathematical equation, giving a unique numerical problem each time the question is
deployed; and allows ‘part marking’ options, whereby if a student is struggling, guidance
can be given on initial formulation allowing the student to continue to complete the
remaining part of the question.
These and other features are identified in more detail in the Economics Evaluation Report
(see Appendix 1).
Maple TA also has a proctor facility for online exams to ensure the browser is locked down;
however this feature was outside the ‘formative’ scope of the 2012-13 pilot.
3. 2012-13 pilot
For the 2012-13 pilot, students were directed to use the Maple TA website via links in
Blackboard. To track results, academics accessed Maple TA’s own grade book facility.
The Blackboard/Maple TA Building Block was not tested or deployed. Course
administrators were not involved at this stage.
The project review team comprised Ken Clark and Len Gill, Economics; Niels Walet,
Faculty of Engineering and Physical Sciences; David Buckley, Humanities IT Services for
administration, supplier liaison, contracts and licensing; Neil Briggs, Assessment officer,
central eLearning Applications Team; Cath Dyson and Cath Booth, Humanities eLearning
team.
4. 2013-14 pilot proposal
 Continuing formative assessment in Economics
 Testing summative assessment processes
 Extending the pilot to selected other maths/stats related disciplines within the
Faculty of Humanities
 Maintaining liaison with EPS (opportunity to collaborate on investigation and sharing
of results re the suite of Maplesoft products)
 Investigating benefits of ‘Maple workflow’ through pilot deployment of Maple
Academic as a teaching tool in Economics
 User-testing the Maple TA/Blackboard Building Block, if feasible*
(*Testing/deployment can only be undertaken at University level, i.e. dependent on
approval having been given for University-level provision of Maple TA)
Report Produced by Cath Booth, Assistant eLearning Manager, Humanities eLearning team, cath.booth@manchester.ac.uk
For queries or comments please contact Cath Dyson, eLearning Manager, Humanities eLearning team, cath.dyson@manchester.ac.uk
Friday, September 06, 2013
3
5. Appendix 1
Draft Maple TA Evaluation
Ken Clark and Len Gill, 24th June 2013.
1. From a teaching and learning perspective, Maple TA offers academic staff teaching mathematical
and statistical material an enhanced ability to offer online assessment and practice material to
students. Here, “enhanced” is understood in relation to the standard tools available within the
current version of the Blackboard VLE. In what follows we provide an evaluation of our experience
within the Economics DA of the School of Social Sciences of using Maple TA on some undergraduate
and postgraduate courses in mathematical or statistical/econometric approaches to economics.
2. Maple TA was used on the following modules (student numbers in parentheses):
 ECON10071 Advanced Mathematics (484)
 ECON60611 Introduction to Econometrics (84)
 ECON10132/20292/60632 Statistics for Economists (144)
 ECON20092/30312 Operational Research IB (52)
3. The extent of the use of Maple TA, in terms of the proportion of the module material tested using
the system, increased over the academic year with the first semester modules piloting the system
while those in the second semester integrated it more fully into formative (not summative)
assessment.
4. Approaches to teaching mathematical and statistical material, whether in Economics or in other
areas including pure mathematics or the sciences, traditionally proceed in a similar manner.
Lectures cover the theory/methods of solving a particular type of problem and are typically
presented by the lecturer together with a limited set of examples or applications of the method.
The real learning, however, takes place when the students actively try to apply the methods to new
problems in example or exercise sheets which are often covered in tutorials. Students, if they are
lucky, may hand in answers to selected exercise problems and have them marked thus receiving
feedback on their work. More typically, and especially where SSRs are high, they are expected to
“learn where they went wrong” from the tutor going through the examples on the board in a
tutorial, or may seek one to one help from their tutor in a drop in session or office hour.
5. The potential advantages of moving this kind of learning online are clear: students can access a
potentially large number of examples from wherever they are, whenever they find convenient and
can obtain instant feedback on their work. The problem is that the typical VLE such as Blackboard
does not deal well with mathematical material and offers only a limited subset of the kinds of
questions which teachers would like to ask in order to properly assess the taught material. For
example while it is possible to test some kinds of mathematical knowledge using standard multiple
choice or true/false type questions, this may place limits on the types of question asked or material
tested. For example, it limits the students’ engagement with the use of mathematical notation; it
may not make it easy for students to enter the results of calculations that they have performed; it
does not allow students to interact with diagrammatic material.
6. Maple TA affords a potential solution to a number of these issues. Its documentation promises that
“Maple T.A. is an easy-to-use web-based system for creating tests and assignments, and
automatically assessing student responses and performance. It supports complex, free-form entry
of mathematical equations and intelligent evaluation of responses, making it ideal for science,
technology, engineering, mathematics (STEM), or any course that requires mathematics." “Maple
4
T.A. supports the use of standard mathematical notation in both the question text and student
response, so students can focus on their assignment, not on the software." (Maplesoft Maple TA
web pages).
7. In practice Maple TA allows instructors to set a wider range of questions than those available in
Blackboard. As well as the standard types of question (numeric answer, multiple choice, fill in the
blank, true/false) which Blackboard covers reasonably well, Maple TA allows, for example,
 Algebraic answers to questions (i.e. the student enters mathematical notation)
 Sketch Graph questions where students are expected to draw a diagram onscreen
 Clickable Image questions where students are asked to click the answer on a picture presented
by the instructor.
8. This list is non-exhaustive but these are three important types of question for the teaching of
mathematical and statistical material since they test important conceptual ideas (the use and
meaning of mathematical notation which is the fundamental building block of all mathematical
knowledge) and practical skills (the construction and interpretation of graphical and diagrammatic
presentation of mathematical and statistical material). The importance of mathematical and
statistical material to modern economics makes testing these types of skills important to teaching
economics too.
9. As well as the alternative question types the other major advantage of Maple TA is the ability of the
Maple TA system to interpret a student’s answer for its correctness in mathematical terms. It is
clear that, mathematically, 2x+y is equivalent to y+2x however Blackboard has no way of evaluating
these two expressions and could mark a question incorrect if it was merely expressed in a different,
but mathematically equivalent, way. This is a non-trivial issue – an important concern when
marking quantitative material in economics (and other disciplines) involves giving students due
credit for answers which are mathematically correct but may be expressed differently by different
students.
10. Like Blackboard, Maple TA is able to provide sketch answers – either quite short statements of the
general principle behind a question, or a full answer which provides details of any numerical
calculations.
11. Questions in which the numerical values are chosen randomly by Maple TA are also available. Such
randomised questions have a number of virtues. First, they capture the idea of students actively
trying out the methods – as noted above this is a fundamental element of learning quantitative
material - and learning as they practice. Secondly, one randomised question can be equivalent to
several specific questions which are designed to capture specific features of the problem or method,
and thus is a more efficient method of assessing key quantitative skills.
12. During the pilot in Economics question types used included: True/False, Multiple Choice, Clickable
Image*, Maple Graded*, Multipart, Formula List*, Key Words, Inline, List, Blanks, Formula Mod C*,
Formula*, Matrix*. In addition, one can utilise the facilities of the Maple mathematical
computation engine to extend the coverage of mathematical topics. (Asterisks indicate question
types not available in Blackboard). This therefore represented a genuine assessment of the
additional capabilities offered by the system.
13. From the perspective of instructors, the main impressions of using Maple TA were that:
 the system allows the testing of a wider range of mathematical knowledge than Blackboard;
5






the system appeared to be robust and secure;
integration with Blackboard is an essential requirement of a wider roll-out of Maple TA;
the Maple TA interface is not user-friendly and the online manual is difficult to use;
students were able to successfully use the system and appreciated its potential enhance their
experience of studying quantitative material in Economics;
there was appreciation that there is no need for instructors to mark student work in a VLE
manually, as is currently required in Blackboard;
the availability of Maple itself within the institution is necessary for an instructor to make full
use of the capabilities of Maple TA.
14. Feedback was also gathered from students. Verbatim comments from students included:
 “Much superior to Blackboard, where Java often breaks down and answers are limited to
multiple choice.”
 “Really like Maple TA. ... allowed more flexibility than Blackboard"
 “The navigation system is a bit tricky. It's a good system though.”
 “The equation editor is a good improvement to Blackboard.”
 “Maple site user friendly"
 Some students said they preferred Blackboard, “because it is there" -no need to go to another
website.
The overall impression from the instructors who used it is that Maple TA has the potential to improve
the teaching of quantitative material in a range of disciplines at the University and in so doing to
improve the experience of students. However it is clear that the interface to Maple TA discouraged
some staff from engaging with the system and also put off some students. This suggests a need for
further/better training opportunities and materials and/or engagement with Maplesoft to improve the
interface. Fuller integration with both the Maple system itself and Blackboard are also necessary to
realise all of the potential benefits of Maple TA.
6
6. Appendix 2: Maple TA potential in other disciplines/Faculties
compiled by Humanities eLearning, Summer 2013
Academics who have expressed interest in finding out more about Maple TA
(NB Italics indicate academics not yet contacted)
 Education: Howard Bond, Michael Wigelsworth, Diane Harris
 Law: Jo Deakin, Juan Medina-Ariza
 SED, Geography: Jen O’Brien, Laura Edwards, Philip Hughes
 SoSS, SOST/SOCY/SOCH: Kingsley Purdam, Ian Plewis, Mark Brown, Nick Shryane and others;
 ALC: Laurence Brown, Ina Berg, Francesco Maria Ciconte, Wendell Kimper, Stuart Campbell
 MBS: James Freeman; plus Sarah Taylor (MBS eLearning) forwarded communication to other
lecturers in MBS
Faculty of Humanities courses with Stats, Quantitative Methods, Maths, Economics
content (NB not exhaustive list)
Statistics






EDUC60651 Quantitative Methods and Statistics in Psychology 2012-13 1st Semester – Michael
Wigelsworth
LAWS70821 Introduction to Statistics & Data Analysis 2012-13 1st Semester – Jo Deakin
BMAN20400&20741 Statistics for Management 2012-13 Full Year – Emanuel-emil Savan, James
Freeman
BMAN71791 Applied Statistics and Business Forecasting 2012-13 1st Semester – Nadia
Papamichail, Madeleine Ryan, Lucy Filmore, Noor Kamileen, Helen Derbyshire, James Freeman,
Hilary Garraway
BMAN80161 Foundation Statistics for Business Research 2012-13 2nd Semester – Lynne BarlowCheetham, Daniel Wheatcroft, Claire Faichnie, Charles Cui, Claire Smith
GEOG10092 Researching Manchester: Environmental Methods for Geographers 2012-13 2nd
Semester – Peter Ryan, Jennifer O’Brien, Angela Harris, Claire Goulsbra, Caroline Turner (stats,
graphs/curves, etc)
GIS (check whether Maple TA would be useful)




GEOG20382 Remote Sensing in Action: Mapping and monitoring land cover changes 2012-13 2nd
Semester – Karl Hennermann, Angela Harris
GEOG30011 The Frozen Planet, Satellites and Climate Change 2012-13 1st Semester – Laura
Edwards, Philip Hughes – **uses Matlab**
GEOG60941 Environmental Remote Sensing 2012-13 1st Semester – Julia McMorrow, Karl
Hennermann, Andrew Stimson
GEOG60951 GIS and Environmental Applications 2012-13 1st Semester – Sarah Lindley
Economics-related (not ECON)







BMAN10001 Economic Principles: Microeconomics 2012-13 1st Semester – Reza Salehnejad,
Mohammad Hajhashem, Abdulaziz Alosaimi, Ronald Ramlogan, Amir Haghayegh khorasani
BMAN10612 Business Economics 2012-13 2nd Semester - Davide Consoli, Ronald Ramlogan
BMAN20320&20701 Managerial Economics 2012-13 Full Year – Carlos Ramos Perez, Reza
Salehnejad
BMAN20340&20691 Intermediate Macroeconomics 2012-13 Full Year – Terry Robinson
BMAN31212 Investment Economics and Innovation 2012-13 2nd Semester – Hugh Cameron
BMAN31391&30461 The Economics of Sport 2012-13 1st Semester – Terry Robinson
BMAN31722 Macroeconomics Analysis 2012-13 2nd Semester – Terry Robinson
7





BMAN70191 Economic Analysis of the Multinational Enterprise 2012-13 1st Semester – David
Holman, Madeleine Ryan, Helen Derbyshire, Svetlana Budyakova, Rudolf Sinkovics, M Yamin
BMAN71122 Financial Econometrics 2012-13 2nd Semester - Stuart Hyde, Edie Shemilt-griffiths,
Pauline Bradshaw, Sungjun Cho, Helen Derbyshire, Madeleine Ryan
BMAN71441 International Macroeconomics and Global Capital Markets 2012-13 1st Semester –
Michael Bowe, Norman Strong, Helen Derbyshire, Madeleine Ryan, Pauline Bradshaw, Sungjun
Cho
BMAN71562 Multinationals and Technology Transfer in Economic Development 2012-13 2nd
Semester – Faith Hatani, M Yamin, Madeleine Ryan, Svetlana Budyakova, Helen Derbyshire
BMAN71751 Financial Appraisal and Investment Economics 2012-13 1st Semester – Hugh
Cameron, Edie Shemilt-griffiths, Helen Derbyshire, Madeleine Ryan, Hilary Garraway, Matthew
Przysiecki, Shirley Snelson
Mathematics-related


EDUC20552 Teaching and Learning Mathematics 2012-13 2nd Semester – Diane Harris, Julian
Williams, David Swanson
EDUC30882 Primary Mathematics Education 2012-13 2nd Semester – Diane Harris
Data Analysis







BMAN70302 Research Methods and Data Analysis (IBM) 2012-13 2nd Semester – M Yamin,
Madeleine Ryan, Umair Choksy, Yong kyu Lew, Kalindi Maheshwari, Noemi Sinkovics, Helen
Derbyshire, Rudolf Sinkovics, Eva Alfoldi, Svetlana Budyakova
BMAN80141 Qualitative Data Analysis 2012-13 1st Semester – Lynne Barlow-cheetham, Daniel
Wheatcroft, Claire Faichnie, Catherine Cassell, Claire Smith
BMAN80542 Qualitative Data Analysis with NViVO 2012-13 2nd Semester – Eva Alfoldi, Claire
Faichnie, Claire Smith, Daniel Wheatcroft, Lynne Barlow-Cheetham
EDUC60562 Qualitative Data Analysis 2012-13 2nd Semester – Kirstin Kerr, Andy Howes (+HKP,
AG)
GEOG70602 Digital Image Processing & Data Analysis 2012-13 2nd Semester – Angela Harris
LAWS20452 Data Analysis for Criminologists 2012-13 2nd Semester – Deborah Jump, Rosemary
Broad, Juan Medina-Ariza, Judith A Aldridge, Katie Benson
SOST70022 Longitudinal Data Analysis 2012-13 2nd Semester – Kingsley Purdam, Ian Plewis,
Albert Varela Montane
Quantitative Methods








SOST70511&70511 Introduction to Quantitative Methods 2012-13 1st Semester – Mark Brown
POLI60251 (combined course: SOCY60201, POLI60251, POLI70021, SOAN60231, SOAN60892,
SOAN61001, SOCY60422, SOCY70722, SOST71022) Qualitative and Quantitative Research
Methods 2012-13 1st Semester – Adi Kuntsman, Ian Fairweather, Angela De Souza Torresan,
Kevin Morgan, David Richards, Elisa Bellotti, Wendy Olsen, Sophie Woodward
SOCH70162 Applying Quantitative Methods 2012-13 2nd Semester – Marta Cantijoch Cunill, Nick
Shryane, Maria Pampaka
ARGY20082 Data Processing & Quantitative Analysis 2012-13 2nd Semester – Stuart Campbell, Ina
Berg
LELA30101 Quantitative Research Methodology 2013-14 1st Semester – Wendell Kimper
BMAN10732 Quantitative Methods for Business & Management 2 2012-13 2nd Semester – Julia
Handl
BMAN10750 Quantitative Methods for Accounting and Finance 2012-13 Full Year – Julia Handl,
James Freeman
BMAN10821 Quantitative Methods for Business and Management 1 2012-13 1st Semester –
Ludmil Michailov
8


BMAN70211 Introduction to Quantitative Methods 2012-13 1st Semester – Massimo Guidolin,
Hening Liu
SING25000 Quantitative Methods For Management 2012-13 Full Year – Ludmil Mikhailov, Sarah
Mountford
(NB Academics in Economics teaching Quantitative Research Methods indicated that Maple TA is
likely to be a most useful tool for both practice and more formal assessment across the subject area)
Other Faculties:



EPS: During pilot: Niels Walet; Colin Steele (Head of Service Teaching); Stuart Anderson,
Learning Technologist.
For 2013-14, Niels Walet will be communicating to academic staff across EPS to encourage
further interest, particularly in light of EPS take up of Maple Academic and Maple Sim.
MHS: NURS60016 Quantitative Design and Analysis (and contact MHS eLearning for other
potential contacts)
FLS: Claire Faichnie; plus contact FLS eLearning for other potential contacts
9