Making Assessment Easier and More Meaningful on the New Assessment Calendar Amanda Udis-Kessler, Director of Assessment and Program Review, September 2, 2015 Two years ago, the Assessment Committee worked hard to get out the message that while all departments and programs had to do assessment they should make it as easy on themselves as they could while still meeting the requirements. Last year the Committee focused on the message that assessment should be as meaningful as possible given that assessment is required. Now we are working on developing a process that will make assessment both easier (much shorter reports) and more meaningful (built in time for reflection and program modification in response to findings). Even so, as we roll out the new calendar, we want to make sure that your department or program is engaging with the new calendar in ways that are both as easy as possible and as meaningful as possible. We don’t believe that ease and meaningfulness have to exist on opposite ends of a continuum. Here are some suggestions for how to go through the four semesters with both as much ease as we can imagine and as much usefulness as you want. For departments/programs currently in semester 1: Semester 1 (planning): The chair or assessment representative (hereafter referred to as the rep) should remind department/program members of the department/program’s learning outcomes in an email and ask them to email the primary assessment representative with two to three learning outcomes that represent an area where the faculty member thinks the department may have room for improvement. Emails should include a suggestion for how to assess the learning. Have the emails due shortly before a department/program meeting in block 2 or 3. The rep will organize comments before the meeting and bring ideas to the meeting. Carve out half an hour in the meeting to agree on the two to three learning outcomes you want to consider. If the department/program is strong in most of its learning outcomes most faculty members are likely to agree on where it is less strong. Ideally, the outcomes will be related to each other in some way and thereby focus on some systematic aspect of the department/program curriculum. After addressing the learning outcomes of choice, determine how you will collect information in semester 2. Capstone projects? One or more required courses? Some other way? This part of the discussion should be brief and does not need to include developing an actual assignment, only information about when and how the assessment will take place. The semester 1 report is due on February 1 (early semester 2), meaning that departments and programs have two weeks after the beginning of block 5 (2016) if they still have planning or writing to do. Regarding all reports, the Committee presumes that the rep will write the report but will support any department/program member who does this. Semester 2 (gathering information): As with semester 1, start with an email ahead of a meeting. Collect ideas for the assignment that will be used to demonstrate learning in the areas of the selected outcomes. The assignment should ideally already exist as part of a course, a set of courses, or the department 1 or program’s capstone; existing assignments should be sent as attachments, even if in draft form. The rep will sort through ideas (and if the assignment is taken seriously, people won’t assume the capstone is the only option) and bring an organized list to a department/program meeting along with any submitted assignments. Roughly half an hour will be reserved at the meeting, hopefully in block 5 or 6, to come to agreement about what the assignment will be. If the assignment does not already exist, additional time will be required to create it before it is implemented, but if the assignment would have been created anyway for grading purposes (as the Assessment Committee suggests) this part of the process does not add time to the second semester assessment process. The rep designs a rubric for assessment if one does not already exist. The semester 2 report is due on July 1 (after semester 2), meaning that departments and programs have time after the end of the 2016 school year if they still have planning or writing to do. Semester 3 (evaluating, reflecting, adjusting): Ideally, all department/program members read through the student work that is being used as a demonstration of learning, preferably using a rubric to normalize the evaluation. After reviewing the materials faculty members should draw up a short list of conclusions about how well students overall met each learning outcome. If the assignment was substantial and/or if many students were involved the department/program may need to divide up pieces of the work and review different sections of the assignment, ideally with no fewer than two people reviewing each part of the assignment. If possible, the conclusions should be sent to the rep ahead of the meeting. Department/program members should also generate ideas for addressing concerns that they see before the meeting, if possible, and send those to the rep as well. The department/program should dedicate at least an hour at a meeting to discussing findings from the demonstration of learning as well as determining changes to be implemented to improve learning in the weakest area or areas. This may be the one area where saving time will interfere with meaningfulness. Even so, if people have been thoughtful in their premeeting work the work of the meeting itself may not take much time. We recommend that this meeting take place in block 2 or 3 of the second year of the process. The semester 3 report is due on February 1 (early semester 4), meaning that departments and programs will likely have two weeks after the beginning of block 5 (2017) if they still have planning or writing to do. Semester 4 (implementing changes): Changes planned in semester 3 are implemented in semester 4, ideally not adding much time to semester 4 preparation or teaching (as professors often modify courses or capstones for improvement apart from assessment). In block 7 or early block 8 department/program professors send information about changes implemented and any preliminary observations about their success to the rep. Ideally half an hour or so can be set aside at the block 8 meeting to discuss faculty experiences implementing changes to make writing the report easier. If a department or program elects to begin planning for the next cycle at this point some additional time will need to be built into the meeting for discussion. 2 After the meeting the rep pulls together the materials required for appendices, which can be saved from former reports unless they have changed in some way: o Assessment history o Mission statement o Curricular goals o Learning outcomes o Assignments used for assessment during this cycle o Rubrics used for assessment during this cycle o Curricular map NOTE: As pointed out in the earlier “Rethinking the Assessment Process” document we assume that many departments and programs will be able to carry out this process over the course of two academic years, starting in the fall of the first year and concluding at the end of the spring semester of the second year. However, we can imagine situations in which a department wants to gather its information in a fall semester or over the course of multiple semesters, and our intent is to develop processes to work individually with such departments to determine how to modify their two-year cycles to enable them to do the assessment that is most meaningful to them. For departments/programs currently in semester 3: The process, which will only occur in this way for one cycle, is different from the above process because three semesters need to be addressed within a single semester. The most effective way to do this will probably be to work quickly to determine what assignment from one or more classes or involving the capstone from 2014-2015 can be mined for assessment use. Rather than develop a single strategy for doing so, the Assessment Committee will work with each department or program in this situation to bring them through the equivalent of semester 3 by February 2016. Semester 4 will then follow as above in spring semester 2016. Please contact Jim Parco or Amanda Udis-Kessler with questions. 3