STUDENT LEARNING ASSESSMENT PROGRAM SUMMARY FORM AY 2013-2014 Degree and Program Name: M.A. History Historical Administration Option Submitted By: Nora Pat Small Please use size 10 font or larger. Please complete a separate worksheet for each academic program (major, minor) at each level (undergraduate, graduate) in your department. Worksheets are due to CASA this year by June 14, 2014. Worksheets should be sent electronically to kjsanders@eiu.edu and should also be submitted to your college dean. For information about assessment or help with your assessment plans, visit the Assessment webpage at http://www.eiu.edu/~assess/ or contact Karla Sanders in CASA at 581-6056. PART ONE What are the learning objectives? How, where, and when are they assessed? What are the expectations? What are the results? 1. Students will incorporate the theories and principles of Historical Administration into their written work, oral presentations, and major projects, thus demonstrating their ability to think critically and analytically about these concepts, as well as demonstrating effective oral and written communication skills. Four major capstone experiences: 1.) installed class exhibit 2.) written comprehensive exam 3.) revised seminar paper (chosen by the student) for defense at oral comprehensive exam 4.) oral comprehensive exam (which covers the gamut of their experience as a student and an intern, as well as the scholarship demonstrated in the seminar paper). Rubric for assessment of oral and written comprehensive exams is attached. 1. Class Exhibit Exhibit plan and design will demonstrate ability to deliver complex messages in simple manner Exhibit plan and design will respond to, address, or demonstrate an awareness of multiple perspectives or arguments Exhibit plan and design will demonstrate an awareness of multiple audiences Exhibit will respond to front-end analysis conducted by students 1. Class Exhibit, 2013-14 Students researched, designed, fabricated, and installed “Text and Textiles: Crafting the Lives of Guy and Irene Buzzard” Central to the exhibit design process is translation of complex topics into a presentation that is readily accessible to visitors with no knowledge of the topic, while remaining interesting to those with more familiarity. The students successfully ‘wove’ personal, institutional, and historical contextual stories into a coherent narrative through careful choice of artifacts for display, concise label copy, and gallery design. They also used a variety of digital platforms to present ideas of text and textile to the public. Students successfully presented multiple historical perspectives, as is evident from the materials they included on the exhibit’s website: http://www.eiu.edu/ha/exhibits/2014/home.h Committee/ person responsible? How are results shared? 1. Instructor for exhibit design course. Results shared through the exhibit itself; through discussion among the HA faculty, guest exhibit curator, and exhibit site client; and through discussions between faculty and students. While the exhibit is public, and therefore the results are technically shared with anyone who visits, the analysis of the outcome remains confined to those who participated in the process. 2. Two to three members of the HA faculty read each written comprehensive exam and Students will critique the exhibit through summative evaluations or final reports 2. Written Comprehensive Exam Students will respond in a clear and organized fashion to a total of four questions Students will employ relevant theories and/or principles to support their answers Students will incorporate their own projects, research, and perceptions gained from the fall and spring study trips into their answers 3. Revised Seminar Paper Paper clearly states and defends a thesis Paper incorporates and explains relevant theories and/or principles Paper demonstrates ability to use primary and secondary sources critically 4. Oral Comprehensive Exam Students will be able to succinctly state seminar paper thesis Looking back on their coursework, students will analyze tml Students reached multiple audiences beyond the physical exhibit in the Tarble through the use of several social media sites, as well as through the creation of the exhibit website, itself. (Access this through the website url above.) Students have also conducted gallery talks and demonstrations. Students conducted multiple front-end analyses prior to determining the precise direction and narrative of the exhibit consisting of interviews, focus groups (on and off campus), and questionnaires Students critiqued the exhibit through summative evaluation based on questionnaires and site visits, and in an ingallery intensive critique 2. Written Comprehensive Exam All members of the class of 2012-2013 passed their written exams. As always, each exam was read by two-three faculty members (including the coordinator) who provided feedback to the coordinator on whether or not the students had achieved the previously stated expectations. Students continue to emphasize the applied aspects of the program in their written comprehensives, often citing field-wide standards. As was true last year, few chose to incorporate literature from the field that helps to contextualize (providing underlying philosophy or historiography) those professional standards and practices. 3. Revised Seminar Paper Students are required to revise one seminar paper in accordance with one faculty member’s suggestions. Seminar papers are share impressions of those exams prior to the oral comprehensive exam. Comments are either written on the exam or shared with the coordinator. Students are also informed of the assessment of their written exams at their oral exams (assuming there were no serious problems that would have required retaking the exam—which has never happened to my knowledge), and are sometimes asked to delve more deeply into a topic they addressed in their written comprehensive exam. 3. One to two members of the HA faculty read each seminar paper submitted for the student’s permanent file. The student then responds to suggestions for revision. The revised paper is read by twothree members of the HA faculty, who share their critiques prior to and during the oral comprehensive exam. All faculty are informed of the student’s performance after the oral exam. 4. Two to four members the ways in which relevant theories and principles were incorporated into their various projects and papers, as well as into their written comprehensive exam answers Students will discuss the theory and principles of Historical Administration in the context of their internships read by two-three faculty members prior to oral comprehensive exams. Eight of the 10 full-time members of the class of 2012-13 have submitted their papers and passed their oral exams. The part-time student also submitted his paper and completed his oral exam. All of those students who have completed their oral comps have successfully fulfilled all of the expectations for their seminar paper, which becomes a permanent part of their file. We had several strong papers, but not due a change in curriculum, as this class completed their coursework under the old curriculum. We continue to strive to have students make original contributions to the literature through the use of primary sources and analysis of the literature in the field, but some are more successful than others. 4. Oral Comprehensive Exam Eight of the ten full-time members of the class of 2013-14 have taken and passed the oral comprehensive exam. Our most recent part-time student has taken and passed his oral exam. Another oral exam is scheduled for late May 2014, and the last one is yet to be scheduled. All were able to clearly and concisely restate their seminar paper’s thesis, and were able to discuss/ defend their paper topic. All reflected on their coursework and affiliated projects, as well as their internships, in the process considering the principles and theories that underlay that work. of the HA faculty participate in each oral comprehensive exam. Results of the exam are discussed immediately following administration, prior to deciding whether the student has passed, or achieved a pass with distinction. The student is informed of the decision immediately. The HA Program Committee meets at least once a year to discuss overall results of written and oral comprehensive exams, as well as papers and projects that have been or will be incorporated into class work. The HA faculty continue to evaluate and discuss student performance and stress levels throughout the year. In this way we are able to detect and address particular strengths and weaknesses of the group and of individuals. Finally, the HA coordinator, at multiple points throughout the year reviews with the students (as a group) the expectations for the written and oral exams, the internship reports, 2. Students will apply the theories and principles of Historical Administration in accordance with professional standards. Successful applications will demonstrate the ability to think critically and analytically about the application of these concepts. In order to achieve Objective 1, students need the opportunity to apply the theory and principles they are attempting to master. They have multiple opportunities throughout the academic year and during their 6-month internship to apply these theories and principles. Specific projects vary from year to year but include such things as writing grants for area historical and cultural agencies, creating public programming for local historical and cultural organizations, designing, fabricating and installing an exhibit in a local venue. These projects are completed as part of the course requirements for HIS 5010 Administration of Historical Organizations, HIS 5030 Archival Methods, HIS 5060, HIS 5090 Care and Management of Historic Artifacts, HIS 5110 and 5111 History Museum Exhibits. The assessment measure for this objective is simply the attempt to incorporate theory and principle into practice Applied projects will explore the relationship between theory or principles and practice. Student exhibit will demonstrate familiarity with a variety of media and presentation methods. Internship special project report will explain both the practical and theoretical principles involved in the project. Internship supervisor will express satisfaction with the knowledge and skills the student brought to the internship. Written exams will incorporate students’ own projects, research, and perceptions gained from the fall and spring study trips into their answers, and will explain the role of theory and principles in their projects and/or in the projects and practices of the institutions visited. During the oral exam, students will explain the application of theory and principles in their own APPLIED PROJECTS Multiple courses throughout the academic year require students to complete projects in which they must translate readings and theoretical knowledge into practice. Projects conducted by the class of 2013-14 included: writing grants for local and regional cultural institutions; researching, designing, fabricating, and installing the annual HA exhibit, “Text and Textiles: Crafting the Lives of Guy and Irene Buzzard”; developing feasibility plans for open storage of collections for the College of Education and Professional Studies and for the Tarble Art Center; cataloguing the Tarble’s Buzzard textile collection; completing archival projects for the College of Ed’s Lab School holdings and for the University archives; digitizing the Tarble’s folk music collection; and creating public programs for the Charleston Historic Preservation Commission. All of these projects required that the students grapple with the application of professional standards and principles while coping with real-world limitations and constraints. STUDENT EXHIBIT The students’ annual exhibit incorporated a wide variety of media and presentation methods. The extent of their involvement with those media and presentation methods can be seen at http://www.eiu.edu/ha/exhibits/2014/home.h tml. They created a carefully thought-out web presence, as well as a well-designed exhibit. An excerpt from an article written for the HAPA newsletter, gives their sense and their revised seminar papers. The students are reminded to employ their HA Student Handbooks as reference guides. APPLIED PROJECTS Instructors for respective courses. Depending on project, results shared publicly, or within in HA Grad Committee, or with the partner institution. STUDENT EXHIBIT Course instructor. Results shared publically, with partner institution, and with HA faculty through formal and informal meetings as well as through the exhibit opening. INTERNSHIP/SPECIAL PROJECT REPORTS HA Coordinator. All HA faculty participating in the student’s oral comprehensive exam review these reports. COMPREHENSIVE EXAMS HA Coordinator schedules. At least two HA faculty, usually 3 and sometimes 4, participate in oral exams. All HA faculty read students’ responses to the questions the faculty member posed on the written comps. Students through these discrete projects. Students’ overall success in applying the theories and principles of the multiple disciplines incorporated in the field of Historical Administration is assessed through the applied projects completed in the course of the academic year, the written and oral comprehensive exams (the written exam administered at the end of the academic year, the oral exam administered at the completion of the internship) and through the required internship report, internship special project report, seminar paper submitted for permanent file, and internship supervisors’ letters of evaluation. applied projects, or as observed on study trips to various institutions. Students will receive a pass or pass with distinction on written and oral comprehensive exams. A pass will constitute a demonstration of knowledge commensurate with an emerging professional: 1. knowledge of professional standards 2. some experience applying those professional standards 3. knowledge of principles and theories that underlay museum and cultural institution practice 4. some experience applying those principles and theories of accomplishment in their own words: “During exhibit installation week, the H.A. students worked long and hard to create the right atmosphere and background for their exhibition. After months of planning and design, led by the Exhibit Team (Amanda Hursch, Brittany Contratto, Caitlin Smith, and Jim Willaert), the objects came to life for the students; labels were created, panels were mounted, and lighting was set. The Marketing/Publicity Team (Stevy Hernandez, Desiree Ramirez, and Sarah DePuy) documented the exhibit progress throughout the week (as well as all of spring semester), posting photos on the Text & Textiles social media sites. “The Text & Textiles exhibit opening on April 11, 2014, had a great turnout, with 114 people in attendance. At the reception, President Perry gave words of welcome to the audience members at the Tarble Arts Center. Special guests, such as Charles “Buzz” Buzzard and family members, shared memories of Guy and Irene Buzzard’s crafts. “On May 1, the Education Team (Cate LiaBraaten, Stephanie Templin, and Erin Richards) led a gallery talk for an insightful group. They interpreted the exhibit for the audience members through Cate’s galley talk as well as Erin’s weaving and Stephanie’s huck embroidery. These demonstrations allowed visitors to realize how much time and dedication it took for Irene and Guy Buzzard to create their textile crafts. “The H.A class learned many crafts and skills from community members, classmates, and personal research, allowing them to produce an exhibit that made connections to the visitors’ collective memories.” INTERNSHIP AND SPECIAL PROJECT REPORTS receive feed-back on their written comps when they return for their oral comps, and receive a decision on their oral comprehensive performance immediately. Nine of the ten members of the class of 2012-13 have completed their internships. The internship and special project reports were all thoughtful critiques of their experiences in general and of their special projects in particular. All students referenced professional standards, and considered where and why their projects met or fell short of those standards. All internship supervisors were pleased with the level of expertise and professionalism that the students brought to their internships. WRITTEN COMPREHENSIVE EXAMS In the course of answering the questions posed for their written comps, all students from the class of 2012-13 demonstrated an ability to address professional standards in the context of projects and study trips. They also grappled with the difficulties inherent in applying best practices based on field-wide principles in the face of ever-present constraints. (The May 2013 Written Comprehensive Exam is attached.) The Written Comps for the class of 2013-14 will be assessed in the Fall when faculty return to campus. In keeping with the revision of the curriculum, the questions have been grouped in a different manner so as to encourage more critical thinking about each area of the curriculum. (May 2014 Written Comp Exam is attached.) ORAL COMPREHENSIVE EXAMS All students completing their oral comps over the course of the last academic year were challenged to consider why their internship institutions functioned the way they did, why their internship projects were structured as they were, what problems they encountered with their projects during the academic year, and so on. Few could cite specific literature in the field that grappled with the problems they encountered (a common problem, perhaps due to the lag time between completion of the academic courses and taking the oral exam), but all understood the issues and could think critically about the application of principles and standards in the field. 3. Students will demonstrate advanced scholarship through achieving a competency in the historiography of the disciplines integral to historical administration. Competency requires familiarity with the literature of the respective disciplines and the incorporation of that literature into their research papers and projects. Students’ grasp of relevant historiography is assessed through their use and discussion of that literature in their research papers and major projects in HIS 5010 Administration of Historical Organizations, HIS 5020 Historical Interpretation for Public Audiences, HIS 5040 Research Methods in Local History, HIS 5050 History of American Architecture, and HIS 5330 Material Life in America. Again, gaining a competency in this literature requires repeated exposure and practice in its use. Students’ overall success in achieving a competency in the relevant historiography is demonstrated through use of and discussion of that In the aggregate, a student’s research papers and comprehensive exams will demonstrate awareness of multiple historiographies associated with the various disciplines within historical administration. Papers and comprehensive exams will demonstrate familiarity with specific authors’ viewpoints and arguments. Papers and comprehensive exams will demonstrate an awareness of the ways in which politics, society, and culture shape perceptions of the past and museum policy. COMPREHENSIVE PASS Nine of the ten members of the class of 2012-13 have passed their oral exams (the last will return in Fall ’14 to take that exam). All demonstrated knowledge commensurate with being an emerging professional (as laid out in the previous column). Members of the class of 2012-13 demonstrated a competency in understanding the literature and historiography of the field through their successful completion of the course requirements under the old curriculum. Members of the class of 2013-14 demonstrated a competency in understanding that literature and historiography through successful completion of the courses listed in the second column. All but one member of the class of 2012-13 have completed both sets of comprehensive exams, and have turned in the seminar paper for their permanent file. Some of those papers deal only superficially with historiography, some of the comprehensive exams demonstrate a difficulty in translating knowledge about historiography garnered in class into their own thinking about those comprehensive questions. Students often incorporate a single author into their work, but are hard pressed to put that author’s work into a larger historical context. We hope, through our revised curriculum, to have more success in this area, but won’t know until the class of 2013-14 completes Course instructors responsible for assuring competency in their respective courses. Results shared informally in HA Grad Committee. All HA faculty responsible for reading their segments of the written comprehensive exam. Those faculty attending the oral comp are responsible for assuring student competency there. Results of comprehensive exams shared informally in HA Grad Committee, and with students as noted above. literature in written and oral comprehensive exams, and in the revised seminar paper turned in for their permanent file. 4. Students will demonstrate a competency in formulating, conducting, and presenting research. Competency in presenting research includes the ability to articulate, in writing and orally, such things as key concepts that drive their research, research questions , thesis statements, and supporting arguments. Assessed throughout the academic year in papers and projects completed for HIS 5010 Administration of Historical Organizations, HIS 5020 Historical Interpretation for Public Audiences (formerly Historical Interpretation and Research Methods in Local History), HIS 5040 Research Methods in Local History (new curriculum), HIS 5050 U.S. Architecture, HIS 5330 Material Life in America. As with the previous learning objectives, this is a skill that requires practice. The assessment measure for this objective is the attempt, in various formats, to articulate key concepts, research questions, thesis statements, and supporting arguments. Students’ overall success in achieving this objective is assessed through the all degree requirements following their internships. Formulating: Students’ papers or projects will be based on a research question or thesis derived from course readings and discussions or from curiosity about a particular aspect of the material studied in that course; seminar papers will reflect wide reading in both primary and secondary sources in the subject area. Conducting: Students will seek out resources in a variety of formats and from a variety of sources, both secondary and primary; student research will ensure that the student has a strong grasp of the scholarship/ historiography in the field of research. Presenting: Seminar papers offer a clear thesis; supporting argument is clearly laid out; argument is based on accurate and valid evidence; paper Course work (papers, projects, class discussion) suggests that students understand that politics, society, and culture shape perceptions of the past and shape museum policy, but incorporating that knowledge into comprehensive exams seems to remain a challenge. Members of the class of 2013-14 have produced a variety of papers and projects throughout the academic year. In the process they have articulated key concepts, devised research questions and thesis statements, and marshaled evidence to support their arguments or procedures. They have presented this research publicly in the form of an exhibit, in gallery talks, and through the creation of websites, and less publicly through informal class presentations. All but one member of the class of 2012-13 have successfully completed their oral comprehensive exams, during which they defended their seminar papers. All submitted seminar papers demonstrated the ability to formulate a clear thesis, reflected research in primary and secondary sources, demonstrated knowledge of the scholarship in their respective areas of research, and were clearly argued. Many of these papers also demonstrated a reluctance to integrate the particular topic into the larger questions posed by scholars in that arena. All students were pressed to consider ways to expand on their research questions, sources, and connections to the larger field during their oral exams. Most could do this. Instructor of record responsible for assessing work on individual papers and projects throughout the academic year. Members of the oral comprehensive committee assess student’s seminar paper and student’s ability to present and defend his/her argument during the oral exam. seminar paper submitted for the oral comprehensive exam and for their permanent file. Students’ overall success is also assessed during their oral comprehensive exam, where the examiners are looking particularly for the ability to verbally articulate and present their research provides synthesis and analysis of arguments from secondary sources and demonstrates that student has interpreted primary sources critically; oral presentations or public presentations of any sort (exhibit, public programming, paper presentation) demonstrate command of subject and ability to engage and inform audience. (Continue objectives as needed. Cells will expand to accommodate your text.) PART TWO Describe your program’s assessment accomplishments since your last report was submitted. Discuss ways in which you have responded to the CASA Director’s comments on last year’s report or simply describe what assessment work was initiated, continued, or completed. The HA faculty were honored that the Program was awarded the Provost’s Assessment Award this past year. We continue to respond to the needs of the museum field, and believe that our new curriculum directly addresses the recommendations made in a recent study by Dr. William Bomar, “Skills Most Valued for Entry-Level Professional Museum Positions,” AASLH Technical Leaflet #261. The first two recommendations directly pertain to museum studies programs: 1. “Students preparing for a museum or public history career should work to strengthen their skills in leadership, oral and written communication, interpersonal relationships, community engagement, and technology. Students should also get as much experience working in museums or heritage sites as possible.” Our new curriculum directly addresses all of these issues—two courses on Leadership in Museums provide students the opportunity to interview leaders in the field, to present the results of those interviews to HA faculty, and to create e-portfolios. All courses provide students the opportunity to hone written and oral communication skills. All of the applied courses build interpersonal relationships and community engagement. And all students are required to complete a 6month internship, giving them additional experience in the field, outside of the direct supervision of faculty members. 2. “Graduate museum students and public history programs should . . . ensure that they adequately emphasize museum administration and management, the public dimension of museums, and technology, and that they provide adequate opportunities for field experiences and closely-supervised funded internships.” Our two courses in Museum Leadership are focused specifically on museum administration and management, and include material on the public dimension of museums as well as technology. Other courses (including HIS 5060, 5110, and 5111) incorporate the history and nature of the public dimension of museums; and 5110 and 5111 specifically provide students with direct experience in engaging a broad public in their exhibit. One thing that has not changed in our curriculum is the multiple opportunities students have for field experience—not only do they devise their own projects in conjunction with external partners (exhibit, preservation project, archival project, grant writing, and more) they also all participate in the Fall and Spring Study Trips to sites well beyond Charleston, and participate in an annual trip to the U. of IL museums and library collections storage, and to Taylor Exhibit Studios. While we do not require that our students’ internships be funded, they must be closely supervised by a museum professional, and meet all the standards laid out in our internship guidelines. This year, we were able to offer three students scholarships to offset some of the expenses of their unfunded internships, thanks again to program supporters. All students from the class of 2012-13 received top marks from their internship supervisors, suggesting to us that we are achieving our program goals. When assessed against these recommendations, the Historical Administration Program continues to deserve its national reputation as one of the top programs for producing museum professionals. PART THREE Summarize changes and improvements in curriculum, instruction, and learning that have resulted from the implementation of your assessment program. How have you used the data? What have you learned? In light of what you have learned through your assessment efforts this year and in past years, what are your plans for the future? In 2013-14 we implemented the new HA curriculum, which was devised after a long process of program analysis (described in last year’s report). That new curriculum includes new opportunities for the HA students to share their work, and thus refine their presentation skills. The students presented their new e-portfolios to the HA program applicants during March interview weekend, and the HA faculty were all impressed with the professionalism of their presentations. Those e-portfolios may be accessed through the Internship web page by clicking on the student locations on the interactive map: http://www.eiu.edu/ha/internships.php. In addition, students introduced the annual exhibit at the exhibit opening, conducted demonstrations at the exhibit opening, and conducted gallery talks at several points throughout the spring semester. Students also presented the programs they had created for the Charleston Historic Preservation Commission to the chair of the Commission, who was impressed with the quality of the projects as well as of the presentations. One of our goals with the new curriculum was to enable and encourage the students to thoughtfully integrate professional and historical literature from the museum field into all of their research and applied projects. The jury is still out on how successful we were in this first year. We should have a better idea of how they have fared in this area when they turn in the required seminar research paper prior to their oral comprehensive exams, and as they return to complete their oral exams. Although our learning objectives have not changed for this round of assessment (as we transition from the students still operating under the old curriculum to all students working under the new curriculum), all members of the HA class of 2013-14 received a copy of the Program Competencies and the rubrics for Oral and Written Comprehensive Exams. (attached) Early in the Fall 2014 semester the incoming HA coordinator (spring 2015) and the outgoing HA coordinator will be examining these learning goals, assessment measures, and expectations to ensure alignment, and will then share their suggestions with the rest of the HA faculty, revising as needed.