Department of Economics and Related Studies Assessment Feedback Policies and Procedures (for first year undergraduates studying under the modular scheme) 2010/11 1 Overall aims The Department of Economics and Related Studies (DERS) aims to provide timely and useful feedback to undergraduate and taught postgraduate students on their progress in relation to both formative assessments (i.e., those which do not contribute to the student’s final degree result) and summative assessments (i.e., those which do so contribute). This feedback is provided in a number of different ways which are appropriate for the context in which they are given. 2 Formative Assessment The term “procedural work” is often used to describe coursework which students are required and/or expected to complete within a module, but which does not count in the summative assessment on that module, or therefore in the student’s final degree result. Modules provided by DERS cover a wide range of topics and modes of analysis. The procedural work required across those modules is correspondingly varied in nature. However a broad distinction can be made between work involving formal problem solving, with answers that are either correct or incorrect, and work involving discussion and the weighing of evidence. (Many of our modules include both kinds of work, of course.) Where the work involves formal problem-solving, a common device is the whole-group problem class, where the module lecturer works through the set problem(s) in a session attended by all students on the module. The students are normally expected to have attempted the problem(s) for themselves in advance of the class. The class therefore provides students with an opportunity to self-assess their own work, and to ask questions of clarification which are important forms of feedback. Formal problem-solving can additionally be the subject of small-group seminars, used especially in our undergraduate modules. The format of seminars can vary, even within a given module, but normally the tutor, or nominated students, will work through the set problem(s) in front of the group. As in the whole-group classes this enables students to selfassess. Iin the small-group seminar there is additional scope for questioning and reinforcement. Furthermore, each student will normally be required to submit, to the seminar tutor, a selection of completed procedural work, to be returned to the student with marks and (where appropriate) annotations. Where the work involves discussion and the weighing of evidence, the focus is normally on the small-group seminar. Again the format of seminars can vary, but normally includes a mixture of presentation by students and discussion led by the tutor. Again, this provides some scope for students to self-assess their own understanding of the material, here the tutor’s assessment of presentations and submitted work is the main form of feedback. To facilitate this process the Department provides two forms – one for seminar presentation and one for submitted essays – which can optionally be used by the tutor in providing the feedback. DERS Assessment Feedback Policies and Procedures 10/11, p1 In addition to the tutor’s assessment of a student’s submitted procedural work (of either broad type), small-group seminars may also enable the tutor to make an informal assessment of each student’s progress and level of effort/understanding, based on the student’s participation in the seminars themselves. Along with the recorded marks from submitted procedural work, this information is fed back to the student, via the supervisor, on the end-of-term seminar report form. Each student receives such a report in each module. The reports are made available to supervisors prior to the (mandatory) supervision meeting at the start of the following term, so that they can be reviewed and discussed by the supervisor and student at that meeting. From Spring 2008 onwards the Economics Board of Studies has agreed a requirement that all of our undergraduate courses give at least once piece of procedural work per 10 credits of the course, and provide feedback to students on this procedural work. The feedback may take the form of (i) an overall mark or grade for the procedural work, and comments on the work submitted, (ii) the provision of model answers or guidance on what a model answer should contain, so that students can understand how their answer compares with a model answer, (iii) the circulation of examples of good answers, and advice on examples of common mistakes and how answers may be improved, (iv) VLE-based tests that generate marks for each student, and include suggested remedial actions for students with moderate or low scores, and/or (v) detailed feedback on individual or group student presentations, including the use of relevant pro formas on their performance which are returned to them. Since individual modules vary considerably in their subject matter, orientation and numbers of students, appropriate practice is also likely to vary across modules. For each module within the Department, the Module Organiser should include within the Module Outline a section on the feedback and assessment arrangements for the course, for both formative and summative assessments, and for responding to student evaluations. This should include a description of how numerical marks/grades are to be interpreted as indicators of student learning and academic progress, or a link to a relevant webpage that provides this information. Each Module Outline should contain an explicit deadline for the return to students of marked formative assessment work, where this deadline would normally be no more than three weeks after the deadline for the submission of the assessment work. Where specimen answers or solutions are given in place of the detailed marking of individual student procedural work, the specimen answers or solutions should be made available to students within a similar time-frame. Each Module Organiser should give students on the module guidance on what is expected from them in writing good answers to essays, examination questions and other formative and summative assessment work. They should also give clear guidance to Teaching Fellows on the module of what is expected of students’ work on the course and the criteria which Teaching Fellows should apply in marking their work for formative assessment. Essay and presentation feedback forms The Board of Studies has recommended the use of two forms (copies in Appendix A) to provide feedback to students on classwork essays and seminar presentations respectively. One copy should be handed to the student when the essay is returned or the seminar presentation is completed, and a second copy retained by the seminar tutor for information in completing the Tutorial Report Forms. DERS Assessment Feedback Policies and Procedures 10/11, p2 3 Summative Assessment Summative assessment in DERS modules takes a variety of forms, but is usually either an unseen exam or an essay to be submitted by a given deadline. DERS policy is that students will receive the mark from each summative assessment by the agreed university deadline of six weeks after the assessment date. These marks will remain subject to final approval by the External Examiner and the Board of Examiners. The Department provides feedback to individual students on their summative assessment in the form of the overall marks awarded, according to the following marking scheme: Degree Class 1 2.1 2.2 3 Compensatable Fail Outright Fail Percentage Mark 70%+ 60-69% 50-59% 40-49% 30-39% 0-29% Appendix B gives more detailed information on the Class Descriptors, with interpretative criteria for each class, in the context of both essay-type work and problem-solving work. Feedback The department offers 3rd year students the chance to review their 2nd year scripts during supervised feedback sessions during the Autumn Term. The candidate will be supervised by a member of admin staff and will be provided with their examination script, the exam paper and any solutions. Students will not be permitted to take any copies or make any notes from the script. The sessions will be allocated on a first come first served basis and details of how to book a session will be emailed to the relevant students at the start of the Autumn Term. In addition, cohort-based feedback on summative assessment is provided. For each module a Module Report is produced, at the same time as the individual marks for that cohort become available. This gives information on the aggregate distribution of those marks, i.e., the distribution over degree classes and also the mean and variance. For second-year modules, the Module Report also includes supplementary information intended to help each student understand why they achieved the mark they did, and thus to help them in preparation for subsequent summative assessments on other modules. This will report, question-by-question, how well or badly that question was answered, in general, and note especially any common strengths or weaknesses in the answers. In addition, it may include the mean and standard deviation of the marks by question across all students answering the question. An example of such a Module Report is given in Appendix C. DERS Assessment Feedback Policies and Procedures 10/11, p3 4 Student evaluations of DERS modules Students are invited to evaluate the delivery of each DERS module, via the university’s Online Feedback System. There are two standard online forms used by DERS, one for lecture evaluation and the other for seminar evaluation. These standard forms appear in Appendix D. All students are emailed with a link to the forms relevant to modules they have taken, and asked to complete them. One email is generated for each module to be evaluated. The student has the option to remain anonymous when leaving feedback. There is also a button which students may use if they are generally satisfied with the course and do not wish to complete the form and leave comments. Online evaluation forms are sent out to students, via email, at the beginning of the last week of teaching for a period of 2 weeks. This allows for at least one week during term time and possibly one week outside of term time. The exposure period for students to complete the forms is always over by the time of the assessment/examination. The Seminar Evaluation Form requires the student to identify the tutor they are evaluating. The Lecture Evaluation Form is specific to the module and tutor. In cases where more than one tutor took the lectures that term, one evaluation form will be sent out for each tutor. Evaluation Forms are deployed for all the modules taught during that term. Two-term modules are evaluated at the end of each term that they are taught. Module organisers are required to produce a response to the feedback they receive. The responses are posted on the DERS webpages for students to access: For Undergraduate modules: www.york.ac.uk/economics/current-students/ug-information/feedback/ For Postgraduate modules: www.york.ac.uk/economics/current-students/gsp/pg-feedback/ DERS Assessment Feedback Policies and Procedures 10/11, p4 Appendix A DEPARTMENT OF ECONOMICS AND RELATED STUDIES COURSEWORK ESSAY FEEDBACK STUDENT’S NAME ……………………………………… YEAR TERM DEGREE MODULE GROUP ……... ……… ………… ………… ……….. TUTOR ………………………………. STUDENT’S SUPERVISOR ………………………………. UNDERSTANDING OF MATERIAL Poor 1 2 3 4 5 Excellent LEVEL OF ECONOMIC ANALYSIS Poor 1 2 3 4 5 Excellent ANSWERING THE QUESTION Poor 1 2 3 4 5 Excellent USE OF RELEVANT LITERATURE Poor 1 2 3 4 5 Excellent STRUCTURE OF ESSAY Poor 1 2 3 4 5 Excellent PUNCTUATION, GRAMMAR AND SPELLING Poor 1 2 3 4 5 Excellent STANDARD OF ACADEMIC ENGLISH Poor 1 2 3 4 5 Excellent OPTIONAL COMMENTS SIGNED …………………………………… DATE ……………………….. DERS Assessment Feedback Policies and Procedures 10/11, p5 DEPARTMENT OF ECONOMICS AND RELATED STUDIES SEMINAR PRESENTATION FEEDBACK STUDENT’S NAME ……………………………………… YEAR TERM DEGREE MODULE GROUP ……... ……… ………… ………… ……….. TUTOR ………………………………. STUDENT’S SUPERVISOR ………………………………. SEMINAR PERFORMANCE PRESENTATIONAL SKILLS Poor 1 2 3 4 5 Excellent QUALITY OF ANALYSIS Poor 1 2 3 4 5 Excellent COMPREHENSION Poor 1 2 3 4 5 Excellent INDEPENDENT THOUGHT Poor 1 2 3 4 5 Excellent PREPARATION Poor 1 2 3 4 5 Excellent USE OF RELEVANT LITERATURE Poor 1 2 3 4 5 Excellent GENERAL COMMENTS SIGNED …………………………………… DATE ……………………….. DERS Assessment Feedback Policies and Procedures 10/11, p6 Appendix B DERS Class Descriptors As indicated in the table, the interpretation of any given mark can differ according to whether the assessed work takes the form of an essay (or similar) or problem-solving. But even under these two broad headings there can be different contexts or types of work, so not all of the interpretations apply in each case. class 80-100 1 70-79 1 60-69 2.1 50-59 2.2 40-49 3 35-39 Compensatable Fail mark 1-29 Fail 0 Fail essay or similar Excellent. Clear and sustained demonstration of mastery of the subject. Unusually well organized answers showing proficient use of independently obtained information, e.g. highly original critical analysis. Very good. Showing consistent high-level insight. Wellorganized and presented, including original analysis. problem-solving Unusually clear insight and/or exceptional sophistication in approach. Faultless answers may be given 100% when appropriate. Accurate derivation of answers to all parts of question, including parts requiring significant independent thought. Clear explanation and thorough understanding of the reasons for the method employed, and the intuition behind the results. Largely correct solutions of unseen problem. Good. Accurate answer to most parts of Evidence of high-level thought. Confident, question, with incomplete answers only to critical analysis of the relevant issues and sections requiring significant independent clear understanding of their implications, solution. but with some lapses. Well presented, Clear explanation of methods, results and clearly organised and effective intuition Competent work. Accurate answer to all parts of question Signs of organisation, thought and insight, requiring standard or routine approaches, but with lapses in argument. with some explanation and intuition, In Economic History, reliance on narrative. reproducing lecture or textbook material on the reasons for methods employed and their interpretation. Errors should not betray significant misunderstanding of standard material Work shows some familiarity with the Displays some surface knowledge of the subject, but is ill formed or erroneous, subject matter of the question, but there poorly expounded or developed, and are errors in some routine parts of the wholly or in part off the point. question. Inadequate or inaccurate explanations Evidence of some effort but lacking a Evidence of some competence but lacking sound understanding of the subject. Barely a sound understanding of the methods adequate, with a number of factual and required to address the question. Showing interpretive errors. some evidence of logical reasoning. Inadequate or inaccurate explanations Failure to understand the question, or to Little or nothing of relevance in answer to identify and resolve the issues it poses. question. Work showing inadequate ability and Comprehensive mistakes, failures and understanding. Errors of fact and misunderstandings, showing that little or interpretation. nothing of value has been understood from module material. Work not presented, or mark minus time penalty (for submission) is negative. DERS Assessment Feedback Policies and Procedures 10/11, p7 Appendix C Example of Module Report with cohort feedback REPORT: DEFENCE ECONOMICS 06/07 1. The percentage distribution of marks was: First class: Upper second class Lower second class Third class Total number of students: 19% 54% 25% 2% 104 Overall Comments 2. Overall, a very good high standard of essay answers. Good standard answers showed understanding of relevant theory/models and literature; they were well-researched; the questions were answered and critically evaluated; and the answers were supported with relevant diagrams/maths. 3. Some scripts showed the candidate’s name; others used very small fonts and extensive footnotes; some wasted valuable space by presenting an Abstract; and some presented Appendices which exceeded the word limit (such lengthy Appendices were not given any credit; nor were answers which considerably exceeded the word limit). Others showed that not much thought had been given to the structure of the answer within the word limit. Some failed to answer the question as set. Not all answers provided a word count. Some diagrams were poorly presented with errors or parts missing. Some answers were presented in note form rather than sentences. Comments on Questions Q 1. Some answers contained much irrelevant material, failing to grasp the nature, definition and significance of outsourcing (e.g. PFI/PPP). Answers needed to refer to in-house units as public monopolies not subject to competition. Relevant analysis/ critique included transaction costs; problems/costs of writing long-term contracts; risk transfer and efficiency savings. Q 2. A good answer required knowledge of Smart Acquisition, its analytical basis, the use of NAO Reports and a critique of the policy and its success/failure. Few students attempted this question. Q 3. A popular question. Good answers applied economic principles of comparative advantage; scale and learning economies with supporting diagrams. The relevant alliance needed to be specified; scenarios examined with supporting data on cost savings. Barriers include nationalism; reluctance to depend on allies; and desire for work-sharing. Some answers digressed on economics of alliances rather than defence industrial policy within an alliance; others were less good on section (ii). Q 4. Another popular question. Some very good answers; but some failed to focus on causes and the role and contribution of economics and other disciplines. Economics explanations include use of military force to achieve resource reallocation; cost-benefit analysis; game theory; arms race models. Consider other explanations: religion; ethnic; grievance; poverty; human rights; revenge; mistakes/errors; desire for nation state. For section (ii) consider ending arms race; but relate to problems identified by Intriligator model. On terrorism, consider applications of choice models and income/substitution effects (there were often errors with diagrams of substitution effect). Q 5. Demand for military expenditure model needed with applications and critique of results. Some answers failed to deal with section (ii). Q 6. This required the use of UK Defence Statistics, 2004 and needed to answer all parts of the Question. Some answers failed to use data from UK Defence Statistics 2004. There was a need to define and critically assess definitions of UK DIB. Opportunity cost issue had to be addressed. There were some errors on current UK policy. Future problems include maintaining DIB during troughs in orders; maintaining competition; costs of UK DIB and willingness to pay. DERS Assessment Feedback Policies and Procedures 10/11, p8 Appendix D Module Evaluation Forms LECTURE EVALUATION Department of Economics and Related Studies To help us improve the course in future years please evaluate the following statements. Please complete this section carefully. 1. The lectures were well organised Strongly Disagree 1 2 3 4 5 Strongly Agree N/A 2. The presentation skills of the lecturer(s) were good Strongly Disagree 1 2 3 4 5 Strongly Agree N/A 3. The explanation of concepts was clear Strongly Disagree 1 2 3 4 5 Strongly Agree N/A 4. The module content matched the declared objectives Strongly Disagree 1 2 3 4 5 Strongly Agree N/A 5. The classes/seminars reinforced the lecture material Strongly Disagree 1 2 3 4 5 Strongly Agree N/A 6. I found the module intellectually challenging Strongly Disagree 1 2 3 4 5 Strongly Agree N/A 7. The reading lists and study guides were helpful Strongly Disagree 1 2 3 4 5 Strongly Agree N/A 8. It was easy to get hold of the reading material Strongly Disagree 1 2 3 4 5 Strongly Agree N/A very heavy 1 2 3 4 5 9. Compared with other York modules of the same credit weighting, I found the workload for lectures, tutorials and private study .... very light Overall Comments 1. What did you like most about the module? 2. What did you like least about the module? 3. What suggestions have you for improving the module? 4. State your degree course: Close this window to end the preview Help page (opens in new window) Queries? Comments? Mail the feedback system administrator DERS Assessment Feedback Policies and Procedures 10/11, p9 N/A SEMINAR EVALUATION Department of Economics and Related Studies To help us improve the course in future years please evaluate the following statements. Please complete this section carefully. 1. The tutor was well prepared and knowledgeable Strongly Disagree 1 2 3 4 5 Strongly Agree 2. The tutor has good rapport with the students (s/he is friendly and easy to get on with) Strongly Disagree 1 2 3 4 5 Strongly Agree 3. The explanation of the exercises was clear Strongly Disagree 1 2 3 4 5 Strongly Agree 4. There was adequate opportunity to take an active role in the learning process (e.g. class discussion, interaction with tutor etc.) Strongly Disagree 1 2 3 4 5 Strongly Agree 5. The tutor provided helpful, sufficient and prompt feedback (this tutor keeps me informed about how I'm doing) Strongly Disagree 1 2 3 4 5 Strongly Agree 6. How many sessions did you attend? Overall Comments 1. What did you like most about the seminars/tutorials? 2. What did you like least about the seminars/tutorials? 3. What suggestions have you for improving seminars/tutorials? 4. State your degree course Close this window to end the preview Help page (opens in new window) Queries? Comments? Mail the feedback system administrator DERS Assessment Feedback Policies and Procedures 10/11, p10