STUDENT LEARNING ASSESSMENT PROGRAM SUMMARY FORM AY 2012-2013 Degree and Program Name: MA History Historical Administration Option Submitted By: Nora Pat Small Please use size 10 font or larger. Please complete a separate worksheet for each academic program (major, minor) at each level (undergraduate, graduate) in your department. Worksheets are due to CASA this year by June 14, 2013. Worksheets should be sent electronically to kjsanders@eiu.edu and should also be submitted to your college dean. For information about assessment or help with your assessment plans, visit the Assessment webpage at http://www.eiu.edu/~assess/ or contact Karla Sanders in CASA at 581-6056. PART ONE What are the learning objectives? How, where, and when are they assessed? What are the expectations? What are the results? 1. Students will incorporate the theories and principles of Historical Administration into their written work, oral presentations, and major projects, thus demonstrating their ability to think critically and analytically about these concepts, as well as demonstrating effective oral and written communication skills. Four major capstone experiences: 1.) installed class exhibit 2.) written comprehensive exam 3.) revised seminar paper (chosen by the student) for defense at oral comprehensive exam 4.) oral comprehensive exam (which covers the gamut of their experience as a student and an intern, as well as the scholarship demonstrated in the seminar paper). Rubric for assessment of oral and written comprehensive exams is attached. 1. Class Exhibit Exhibit plan and design will demonstrate ability to deliver complex messages in simple manner Exhibit plan and design will respond to, address, or demonstrate an awareness of multiple perspectives or arguments Exhibit plan and design will demonstrate an awareness of multiple audiences Exhibit will respond to front-end analysis conducted by students Students will critique the exhibit through summative evaluations or final reports 2. Written Comprehensive Exam Students will respond in a 1. Class Exhibit, 2012-13 Students researched , designed, fabricated, and installed “Experiences of the Illinois Civil War Soldier: Reflections in Art and Artifact.” Central to the exhibit design process is translation of complex topics into a presentation that is readily accessible to visitors with no knowledge of the topic, while remaining interesting to those with more familiarity. The students accomplished this through careful choice of artifacts for display, concise label copy, and gallery design. Objects in the exhibit required that students Committee/ person responsible? How are results shared? 1. Instructor for exhibit design course. Results shared through the exhibit itself; through discussion among the HA faculty, guest exhibit curator, and exhibit site client; and through discussions between faculty and students. While the exhibit is public, and therefore the results are technically shared with anyone who visits, the analysis of the outcome remains confined to those who participated in the process. 2. Two to three members of the HA faculty read each written comprehensive exam and share impressions of clear and organized fashion to a total of four questions Students will employ relevant theories and/or principles to support their answers Students will incorporate their own projects, research, and perceptions gained from the fall and spring study trips into their answers 3. Revised Seminar Paper Paper Paper clearly states and defends a thesis Paper incorporates and explains relevant theories and/or principles Paper demonstrates ability to use primary and secondary sources critically 4. Oral Comprehensive Exam Students will be able to succinctly state seminar paper thesis Looking back on their coursework, students will analyze the ways in which relevant theories and principles were incorporated into their various projects and papers, as well as into their written comprehensive exam answers Students will discuss the theory and principles of Historical Administration consider the multiple historical perspectives of those who created the artifacts, as some were created during the war on the war front, some on the home front, some after the war. Students reached multiple audiences through the use of digital media on-line, as well as through in-gallery activities and multi-media explorations of various exhibit themes. The exhibit will continue to exist in digital form after the physical exhibit is dismantled. Students did not conduct a front-end analysis this year, but participated in numerous meetings with HA faculty, the guest curator, and Tarble staff to devise and refine the exhibit’s ‘Big Idea.’ Students critiqued the exhibit during their final walk-through with TAC staff prior to the exhibit opening, making alterations requested by the client (TAC) prior to opening. Students also completed individual reflections of the exhibit process in their exhibit log books 2. Written Comprehensive Exam All members of the class of those exams prior to the oral comprehensive exam. Comments are either written on the exam or shared with the coordinator. Students are also informed of the assessment of their written exams at their oral exams (assuming there were no serious problems that would have required retaking the exam— which has never happened to my knowledge), and are sometimes asked to delve more deeply into a topic they addressed in their writtens. 3. One to two members of the HA faculty read each seminar paper submitted for the student’s permanent file. The student then responds to suggestions for revision. The revised paper is read by two-three members of the HA faculty, who share their critiques prior to and during the oral comprehensive exam. All faculty are informed of the student’s performance after the oral exam. 4. Two or three members of the HA faculty participate in each oral comprehensive exam. Results of the exam are discussed immediately in the context of their internships 2011-2012 passed their written exams. As always, each exam was read by twothree faculty members (including the coordinator) who provided feedback to the coordinator on whether or not the students had achieved the previously stated expectations. Written comprehensives demonstrate that the strength of the program lies in its applied aspects. All students answered the process parts of the questions very well, incorporating examples from coursework and the study trips, and justifying their answers with reference to professional standards. Few chose to incorporate literature from the field that helps to contextualize (providing underlying philosophy or historiography) those professional standards and practices. following administration, prior to deciding whether the student has passed, or achieved a pass with distinction. The student is informed of the decision immediately. 3. Revised Seminar Paper Students are required to revise their seminar paper in accordance with one faculty member’s suggestions. Seminar papers are read by two-three faculty members prior to oral comprehensive exams. Eight of the 11 fulltime members of the class of 2011-12 have submitted their papers and passed their oral exams. (The part-time Finally, the HA coordinator, at multiple points throughout the year reviews with the students (as a group) the expectations for the written and oral exams, the internship reports, and their revised seminar papers. The students are reminded to employ their HA Student Handbooks as reference guides. The HA Program Committee meets at least once a year to discuss overall results of written and oral comprehensive exams, as well as papers and projects that have been or will be incorporated into class work. The HA faculty continue to evaluate and discuss student performance and stress levels throughout the year. In this way we are able to detect and address particular strengths and weaknesses of the group and of individuals. student in that class will be counted in the class of 201213, having finished his coursework this academic year.) All eight of those students who have completed their oral comps have successfully fulfilled all of the expectations for their seminar paper, which becomes a permanent part of their file. While all papers were solid, only 2 were particularly insightful. In general, they demonstrate a weakness in the broader contextualization of their chosen topics. We believe this is because of time constraints. (See Part 3 for the way we are addressing this.) 4. Oral Comprehensive Exam Eight of the eleven full-time members of the class of 2011-12 have taken and passed the oral comprehensive exam. (The other three are expected to complete their oral comps in Fall 2013.) All were able to clearly and concisely restate their seminar paper’s thesis, and were able to discuss/ defend their paper topic. All reflected on their coursework and affiliated projects, as well as their internships, in the process considering the principles and theories that underlay that work. 2. Students will apply the theories and principles of Historical Administration in accordance with professional standards. Successful application will demonstrate the ability to think critically and analytically about the application of those concepts In order to achieve Objective 1, students need the opportunity to apply the theory and principles they are attempting to master. They have multiple opportunities throughout the academic year and during their 6-month internship to apply these theories and principles. Specific projects vary from year to year but include such things as writing grants for area historical and cultural agencies, creating public programming for local historical and cultural organizations, designing, fabricating and installing an exhibit in a local venue. These projects are completed as part of the course requirements for HIS 5010 Administration of Historical Organizations, HIS 5030 Archival Methods, HIS5090 Care and Management of Historic Artifacts, HIS 5110 and 5111 History Museum Exhibits, HIS 5112 and 5113 Digital Applications in Museums and Archives, and HIS 5060 Historic Preservation. The assessment measure for this objective is simply the attempt to incorporate theory and principle into practice through these discrete projects. Students’ overall success in applying the theories and principles of the multiple disciplines incorporated in the field of Historical Administration is assessed Applied projects will explore the relationship between theory or principles and practice. During the oral exam, students will explain the application of theory and principles in their own applied projects, or as observed on study trips to various institutions. APPLIED PROJECTS Multiple courses throughout both semesters require students to complete projects in which they must translate readings and theoretical knowledge into practice. Among those projects for the class of 2012-13 were digitization of Tarble artwork, the re-evalutation of Lincoln Log Cabin’s 5th Grade Live-In program, the creation of public interpretive programming “Food Preservation and Herbs” for Lincoln Log Cabin’s Harvest Frolic, creation of an interpretive plan for the CCC at LLCSHS, creation of a CCC website for LLCSHS, researching and writing a National Register nomination, the creation of the annual exhibit, collections care at Dudley House, an inventory of archival records relating to the Tarble’s annual traveling student show, a conservation plan for Coles County Circuit Court Records held at IRAD. Undertaking these projects required reflecting on the intersection of best practices and theory with onthe-ground conditions. Students will receive a pass or pass with distinction on written STUDENT EXHIBIT The students’ annual exhibit Student exhibit will demonstrate familiarity with a variety of media and presentation methods. Internship special project report will explain both the practical and theoretical principles involved in the project. Internship supervisor will express satisfaction with the knowledge and skills the student brought to the internship. Written exams will incorporate students’ own projects, research, and perceptions gained from the fall and spring study trips into their answers, and will explain the role of theory and principles in their projects and/or in the projects and practices of the institutions visited. through the written and oral comprehensive exams (the written exam administered at the end of the academic year, the oral exam administered at the completion of the internship) and through the required internship report, internship special project report, revised seminar paper, and internship supervisors’ letters of evaluation. and oral comprehensive exams. A pass will constitute a demonstration of knowledge commensurate with an emerging professional: 1. knowledge of professional standards 2. some experience applying those professional standards 3. knowledge of principles and theories that underlay museum and cultural institution practice 4. some experience applying those principles and theories incorporated a wide variety of media and presentation methods. In addition to creating an official exhibit website, the students also created a Facebook page and tumblr and twitter accounts. They loaded I-pods in the exhibit with images, music, and poetry. They created educational materials available through their website. INTERNSHIP AND SPECIAL PROJECT REPORT All of the members of the class of 2011-12 who have completed their internships and filed their reports, wrote thoughtfully about their projects, their overall internships, the conditions they encountered, and how and why they proceeded as they did with their various projects. They incorporated references to course materials, professional standards, and professional literature/ resources as appropriate. All internship supervisors were completely pleased with the students’ preparation and knowledge. WRITTEN COMPS As stated above, the written comprehensives demonstrate that the strength of the program lies in its applied aspects. All students answered the process parts of the questions very well, incorporating examples from coursework and the study trips, and justifying their answers with reference to professional standards. ORAL COMPS Students seem better able, or more inclined, to discuss the principles and standards that underlie all decision-making in the museum field during their oral comprehensives than in the writtens. This may be because they have recently completed their internships and internship reports where they are required, and have had time, to reflect on these issues. All members of the class of 2011-12 who have completed their oral comps were able to do this. 3. Students will demonstrate advanced scholarship through achieving a competency in the historiography of the disciplines integral to Students’ grasp of relevant historiography is assessed through their use and discussion of that literature in their research papers and major projects in HIS In the aggregate, a student’s research papers and comprehensive exams will demonstrate awareness of multiple historiographies COMPREHENSIVE PASS All members of the class of 2011-12 passed their written comps. 8 members have attempted and have succeeded in passing their oral comprehensive exams, in accordance with the standards laid out in the previous column. Members of the class of 2011-12 grappled with the literature and historiography of the field throughout the curriculum. Collectively, historical administration. Competency requires familiarity with the literature of the respective disciplines and the incorporation of that literature into their research papers and projects. 5010 Administration of Historical Organizations, HIS 5020 Historical Interpretation and Research Methods in Local History, HIS 5050 History of American Architecture, HIS 5330 Material Life in America, and in their US History elective. Again, gaining a competency in this literature requires repeated exposure and practice in its use. Students’ overall success in achieving a competency in the relevant historiography is demonstrated through use of and discussion of that literature in written and oral comprehensive exams, and in the revised seminar paper turned in for their permanent file. 4. Students will demonstrate a competency in formulating, conducting, and presenting research. Competency in presenting research includes the ability to articulate, in writing and orally, such things as key concepts that drive their research, research questions , thesis statements, Assessed throughout the academic year in papers and projects completed for HIS 5010 Administration of Historical Organizations, HIS 5020 Historical Interpretation and Research Methods in Local History, HIS 5050 History of American Architecture, HIS 5330 Material Life in America. associated with the various disciplines within historical administration. Papers and comprehensive exams will demonstrate familiarity with specific authors’ viewpoints and arguments. Papers and comprehensive exams will demonstrate an awareness of the ways in which politics, society, and culture shape perceptions of the past and museum policy. Formulating: Students’ papers or projects will be based on a research question or thesis derived from course readings and discussions or from curiosity about a particular aspect of the material studied in that course; seminar papers will reflect wide reading in both primary and secondary they demonstrated their understanding of the literature associated with exhibit design and public programming through their public exhibit and programs. Individually, all students achieved all course goals, and so have demonstrated familiarity with each area’s literature and/or historiography. All members of the class of 2011-12 who have completed their oral comprehensive exams have demonstrated familiarity with various authors’ viewpoints and arguments. Examining the field through the lens of history and historiography in order to better comprehend the social, cultural, and political influences that have shaped museum policy remains the most difficult task for most students. We hope that our revised curriculum will more directly address this area. (See Part 3) Members of the class of 2012-13 have produced a variety of papers and projects throughout the academic year. In the process they have articulated key concepts, devised research questions and thesis statements, and marshaled evidence to support their and supporting arguments. As with the previous learning objectives, this is a skill that requires practice. The assessment measure for this objective is the attempt, in various formats, to articulate key concepts, research questions, thesis statements, and supporting arguments. Students’ overall success in achieving this objective is assessed through the seminar paper submitted for the oral comprehensive exam and for their permanent file. Students’ overall success is also assessed during their oral comprehensive exam, where the examiners are looking particularly for the ability to verbally articulate and present their research sources in the subject area. Conducting: Students will seek out resources in a variety of formats and from a variety of sources, both secondary and primary; student research will ensure that the student has a strong grasp of the scholarship/ historiography in the field of research. Presenting: Seminar papers offer a clear thesis; supporting argument is clearly laid out; argument is based on accurate and valid evidence; paper provides synthesis and analysis of arguments from secondary sources and demonstrates that student has interpreted primary sources critically; oral presentations or public presentations of any sort (exhibit, public programming, paper presentation) demonstrate command of subject and ability to engage and inform audience. arguments or procedures. They have presented this research publicly in the form of an exhibit, in gallery talks, and through the creation of websites, and less publicly through informal class presentations. One member of the class of 2012-13 has successfully completed his oral comprehensive exam, and has therefore defended his seminar research paper. Eight members of the class of 2011-12 have successfully completed their oral comprehensive exams where they have presented and defended their research papers to two or three HA faculty members. (Continue objectives as needed. Cells will expand to accommodate your text.) PART TWO Describe your program’s assessment accomplishments since your last report was submitted. Discuss ways in which you have responded to the CASA Director’s comments on last year’s report or simply describe what assessment work was initiated, continued, or completed. We continue to conduct informal exit interviews after students have passed their oral exams. Each student after passing the oral exam sits down with the exam committee to discuss various aspects of the program. Students continue to be satisfied with their preparation, and they continue to offer constructive suggestions for the program. As a result of one of our exit interviews, we are looking into adding discussions with the business or development office of the museums we visit on the Fall and Spring Study Trips. Students often mention specific computer programs they wish they had had more practice with. While we cannot do it all in one year, we continue to look for ways to incorporate introductions to and use of various computer programs and digital resources into all courses. The standalone Digital Applications in Museums course has been removed as we have come to realize that each area has specific digital applications that we incorporate into our course-specific projects or papers. Our new HA faculty hire is also a ‘digital humanist’ and will be working with all of us as we continue to develop this important aspect of the program. All students felt prepared to step into their internships or jobs. The students can always, and do, come up with things they would like to see added to the curriculum, but also note that the one-year curriculum is one of the program’s strengths and attractions. The informal exit interview may be revised in light of a recent AASLH Technical Leaflet on “Skills Most Valued for Entry-Level Professional Museum Positions.” We are considering creating a checklist of competencies that the students feel they possess and/or that they used in their internships. PART THREE Summarize changes and improvements in curriculum, instruction, and learning that have resulted from the implementation of your assessment program. How have you used the data? What have you learned? In light of what you have learned through your assessment efforts this year and in past years, what are your plans for the future? Our continual assessment of the program (as opposed to assessment of the students which, of course, is part of what drives the assessment of the program) and how it prepares students for the museum world resulted in a 5-year effort to re-examine and revise the curriculum. That examination involved all HA faculty, formal and informal discussions with alumni and other museum professionals, consultations with current students, and consideration of the results of our assessments. The curriculum and course revisions were all approved by early Spring semester 2013. Although we were unable to devise a way to offer our courses in blocked format (see the assessment summary form for 2011-12), we have reconfigured how courses are grouped over the course of the two semesters, removed some courses, added a couple of new courses, and revised all syllabi to reflect new goals. (See attached curriculum and explanation.) As a result of students tending to excel at the concrete and applied, and to continue to be less strong when it comes to demonstrating critical and analytical thinking skills (Learning Objective 1), we will be reducing the numbers of research papers and projects overall to give students more time to concentrate on more in-depth papers and projects. We anticipate that the new curriculum will enable students to become more familiar with the professional and historical literature and to be more thoughtful in their application of that knowledge to their projects and papers. We have long struggled with a very full curriculum and believe that these revisions will enable students to more fully comprehend the interdisciplinary nature of museum work, the importance of historiography, and the differences between, and the intersections of, historical writing and public history, and to apply that in their academic work as well as at their internships. They will also be creating professional portfolios comprised of their major research papers and projects, and presenting those portfolios to our program applicants and faculty during interview weekend in March, thus providing them with another opportunity to practice and demonstrate effective oral communication skills (Learning Objectives 1, 4). Several students in recent years have suggested finding a way to incorporate the creation of professional portfolios into the academic year. They will be able to refer to these portfolios as they seek out internships and apply for jobs. The coordinator has begun the task of revising our program’s assessment tools in light of the curricular revisions that will be in place beginning Fall 2013. These revisions will appear in the new HA Handbook for 2013-14, as well as in next year’s assessment program summary form.