STUDENT LEARNING ASSESSMENT PROGRAM SUMMARY FORM AY 2013-2014 Degree and Program Name: B.A. in History Submitted By: Anita Shelton and Bonnie Laughlin-Schultz Please use size 10 font or larger. Please complete a separate worksheet for each academic program (major, minor) at each level (undergraduate, graduate) in your department. Worksheets are due to CASA this year by June 13, 2014. Worksheets should be sent electronically to kjsanders@eiu.edu and should also be submitted to your college dean. For information about assessment or help with your assessment plans, visit the Assessment webpage at http://www.eiu.edu/~assess/ or contact Karla Sanders in CASA at 581-6056. PART ONE What are the learning objectives? How, where, and when are they assessed? What are the expectations? What are the results? Committee/ person responsible? How are results shared? 1. History majors will be able to analyze a source document using the historical method. This includes close reading, sourcing (questioning author credentials, motivation/bias, and audience), corroborating, and contextualizing in order to analyze what the document tells us about the past. ULG: Critical Thinking; Writing and Critical Reading; Transferable Skills 1, 2 2. History majors will read critically, identifying and critiquing assumptions, methods, and theories historians use to create coherent interpretations of the past and understanding that history is an evolving narrative. ULG: Critical Thinking; Writing and Every History major submits one research paper from HIS 2010 (a course that is required of all majors). The History Curriculum Committee evaluates the papers holistically using a departmental rubric for source analysis on an annual basis. The evaluation rubric (distributed to all History faculty) is included in Appendix A. Every History major submits one book review of a historical monograph from HIS 3555 (required of all majors). The History Curriculum Committee evaluates the papers holistically using a departmental rubric for book reviews on an annual basis. 20% will be rated highly competent 50% will be rated competent 20% will be rated minimally competent 10% will be rated not competent Of the selections analyzed for 2013-2014 (n=34): 20.5% (n=7) were rated as highly competent 50% (n=17) were rated competent 23.5% (n=8) were rated minimally competent 5.9% (n=2) was rated not competent We did meet all expectations. The Curriculum and Assessment committees work with faculty to gather items for assessment and to carry out scoring. The Curriculum Committee and the chair discuss the data and results and share information with the rest of the faculty. 20% will be rated highly competent 50% will be rated competent 20% will be rated minimally competent 10% will be rated not competent Of the selections analyzed for 2013-2014 (n=19): 10.5% (n=2) were rated as highly competent 73.6% (n=14) were rated competent 10.5% (n=2) were rated minimally competent 5.26% (n=1) was rated not competent The Curriculum and Assessment committees work with faculty to gather items for assessment and to carry out scoring. The Curriculum Committee and the chair discuss the data and results and share information with Critical Reading; Transferable Skills 1, 2 The evaluation rubric (distributed to all History faculty) is included in Appendix A. 3. History majors will be able to carry independent research projects through from inception to completion. In doing so they will frame useful and appropriate questions about the past, undertake primary research and compile evidence, and integrate evidence from sources into a reasoned and well-organized argument based on documented primary and secondary sources. ULG: Critical Thinking; Writing and Critical Reading; Transferable Skills: 1, 2, 6 Every History major submits one research paper from HIS 2500 (required of all majors). The History Curriculum Committee evaluates the papers holistically using a departmental rubric for research papers on an annual basis. The evaluation rubric (distributed to all History faculty) is included in Appendix A. In the future, we will add a second assessment of the research paper produced in the new capstone course. We will use the same rubric to assess growth throughout the program. A focus group of recent graduates. 20% will be rated highly competent 50% will be rated competent 20% will be rated minimally competent 10% will be rated not competent No focus group met this year because of on-going curricular changes. N/A The Chair will convene the focus group after consulting with faculty to develop questions. The Chair will share results from the focus group with the curriculum committee and the faculty as a whole. In AY13 we began to offer 60% will meet None yet. The Curriculum and 4. History majors will develop a sense of historical perspective, intellectual curiosity, and transferable skills that will be useful to them after graduation, whether in graduate school or directly in the workplace. ULG: Critical Thinking; Writing and Critical Reading; Speaking & Listening; Transferable Skills 1-7. 5. History majors will be able We met all expectations except having 20% as highly competent (though nearly 85% of the total were competent, well above our goal). Of the selections analyzed for 2013-2014 (n=7): 14.3% (n=1) were rated as highly competent 71.4% (n=5) were rated competent 14.3% (n=1) were rated minimally competent 0% (n=0) was rated not competent the rest of the faculty. The Curriculum and Assessment committees work with faculty to gather items for assessment and to carry out scoring. The Curriculum Committee and the chair discuss the data and results and share information with the rest of the faculty. In all instances, we met or exceeded expectations. to demonstrate knowledge about global history and its themes of connection and interrelatedness. This understanding is central to responsible global citizenship and participation in informed discussion in civic life. ULG: Responsible Citizenship; Transferable Skills: TBD our revised global history/ general education courses at the 1000-level and are currently working on a revision of the entire curriculum for majors that will integrate global citizenship into required courses for majors; we will incorporate an assessment instrument for responsible citizenship into the new assessment plan we develop for this revised curriculum. Suggestions for potential assessments would be very helpful as we move forward. expectations 30% will exceed expectations 10% will not meet expectations Assessment committees work with faculty to gather items for assessment and to carry out scoring. The Curriculum Committee and the chair discuss the data and results and share information with the rest of the faculty. PART TWO Describe your program’s assessment accomplishments since your last report was submitted. Discuss ways in which you have responded to the CASA Director’s comments on last year’s report or simply describe what assessment work was initiated, continued, or completed. We were pleased with the recent Watson-Glaser spring 2014 data that shows that history majors are above the university average in critical thinking at 27.35 and have improved over last year’s History result of 26.39. This shows us that our continuing emphasis on Critical Thinking as well as Writing and Critical Reading are yielding benefits. They continue to be emphasized in the revised curriculum and in our transferable skills, so we expect gradual improvement to continue. In 2013-2014 the entire department continued to engage in a curricular review and revision. As a department, we discussed a capstone course proposal for the non-teacher licensure majors (for whom SOS 3400 and student teaching is a capstone); finalized a list of transferable skills that would be stressed across the curriculum (the skills are included as Appendix B); met in small and large groups to uncover which courses will address which of these skills; and discussed the new assessment measures that will emerge from the revised curriculum. Last year the department agreed on the substance of a major revision of HIS 2500 (Historical Research and Writing) and the format for a new capstone course. A proposal has been drawn up for the capstone, and both it and HIS 2500 have been integrated into our broad assessment plan in learning objective three. A long-term assessment plan to compare performance on the HIS 2500 research paper and the capstone paper have been put into place. We have made significant progress with the larger curriculum revision, making fundamental changes based on previous assessment data and attempting to incorporate the new University Learning Goals into our learning objectives and transferable skills foundation. We revised and refined all of our learning goals. In particular, we worked on defining the historical method, the research process, and historical thinking/critical thinking in learning goals 1-3. We used the following sources to help us define and write new learning goals: Fred Drake and Lynne Nelson, Engagement in Teaching History, 2nd edition (Pearson, 2008); National Council on History Education’s Historical Habits of Mind, http://www.nche.net/habitsofmind; Common Core Standards (History/Social Studies ELA Strands and Writing), http://www.corestandards.org/ELA-Literacy/RH/introduction/; and College, Career, & Civic Life C3 Framework for Social Studies State Standards (National Council for the Social Studies, 2013), http://www.socialstudies.org/system/files/c3/C3-Framework-for-Social-Studies.pdf. We continue to work on the revised assessments to accompany our new curriculum, and have made strides this year toward the creation of a major portfolio developed in D2L. Faculty attended training meetings and we have begun work to create the D2L portfolios that will ultimately strengthen our assessment program. Submissions for learning objectives 1-3 will ultimately come from this D2L online portfolio. We also rewrote learning goal 5 to better address the University goal of responsible citizenship and are now trying to determine the best way to assess this goal. In 2012-2013 we revised our World Civilization sequence (HIS 1500G and 1521-1526G). These together with HIS 2010G (U.S. to 1877), HIS 2020G (U.S. since 1877), and HIS 3600G (U.S. Constitution and Nation) are the foundation of our contribution to responsible citizenship, which we have defined in learning objective 5 as demonstrating “knowledge about global history and its themes of connection and interrelatedness, understanding central to responsible global citizenship and participation in informed discussion in civic life.” Many of our electives also fit into this goal. Now, we are working to address this objective in our assessment plan. PART THREE Summarize changes and improvements in curriculum, instruction, and learning that have resulted from the implementation of your assessment program. How have you used the data? What have you learned? In light of what you have learned through your assessment efforts this year and in past years, what are your plans for the future? We have begun to learn from data associated with our revised assessment plan. Of particular use in this assessment cycle is data for lesson objectives 1-3, which shows that our majors are developing many of the attributes that we desire. One thing that could be done in the future is to move from a holistic assessment to gathering data from each category in the rubrics to see where particular areas of strength and weakness are. For objective 1, we noted that students seemed weaker at analyzing point of view and bias but stronger at understanding the historical characteristics of the document they were analyzing. This information will be shared with the faculty so that we can address it. In 2014-2015, we will continue to overhaul our curriculum. First, we will address where the remaining transferable skills—speaking/listening, quantitative reasoning, and visual literacy—fit within our learning goals and assessment measures. Doing this will also us to incorporate the revised University Learning goals of Speaking/Listening and Quantitative Reasoning into our curriculum. As discussed above and in part 1, we continue to develop measures to assess learning objectives, particularly 4 and 5. We will devise questions and convene our first focus group of recent graduates in 2014-2015. Additionally, we will propose a new one-hour orientations course for the History major that will introduce students to the discipline and to its transferable skills much earlier. Professors of the orientations course will lay the foundation for work done in HIS 2500, our other courses, and ultimately, the capstone. Once the orientations and capstone courses are live, we will have three points for collecting data: entry into program (orientation course); early/midway (HIS 2500); and exit (capstone). Data from this will help us continue to improve our program and curriculum. Finally, in 2014-2015 we will revise the rubrics currently being used to assess learning objectives 1-3. We will move from a four-point to a fivepoint rubric, and we plan to revise the categories and descriptions to better align with our new curriculum. Appendix C contains a tentative sketch of a proposed revision for the source analysis rubric done by members of the Curriculum Committee, and full revisions of all rubrics will be undertaken next year. Appendix A. History Department Evaluation Rubrics Source Analysis Historical Characteristics of the Document Point of View and Bias Organization Development Style Mechanics 4-Highly Competent Understands the historical characteristics of the document in relation to the topic of study Identifies the point of view of the author and bias in relation to the intended audience and the author’s motive The paper is well structured and persuasive in the statement of its thesis, with an introduction and conclusion Demonstrates depth of understanding of the topic, presents supporting arguments clearly and analytically, and provides excellent documentation Written in an appropriately formal tone without slang. Language is precise and sentence structure varied. Correct use of citations (footnotes or endnotes), quotes, and bibliography The mechanics of the paper are correct. It is well written with no grammar or punctuation errors, and little or no use of the passive voice 3-Competent Shows a basic understanding of the characteristics of the document Shows a basic understanding 2-Minimally Competent Shows some understanding 1-Not Competent Demonstrates no understanding Shows some understanding Demonstrates no understanding Logically organized Organization lacks coherence Poorly organized Demonstrates knowledge of the topic and provides supporting evidence and adequate detail Effective Use of Language Citations and bibliography included but not proper format Few errors Presents undeveloped ideas Lacks idea development and includes irrelevant information Simplistic sentence structure and imprecise use of language No citations or bibliography Errors are present that interfere with the presentation of ideas and arguments Incorrect sentence structure and use of slang, jargon, or inappropriate language Excessive errors in grammar and punctuation. Book Review Thesis Organization Development Style Mechanics 4-Highly Competent Clearly identifies and critically evaluates the author’s central argument, purpose, and approach to the subject The review is exceptionally well organized with an introduction, body, and conclusion and follows the thesis throughout A general analysis of the salient features of the book, as opposed to a general summary, and identifies the development of the author’s thesis throughout the book Sophisticated sentence structure and paragraph development The mechanics of the paper are correct. It is well written with no grammar or punctuation errors, and little or no use of the passive voice Research Paper Research Paper 4-Highly Competent or Project Thesis/Argument The thesis is clearly conceptualized, defined, and supported by concrete examples throughout the paper The paper is well structured and Organization persuasive in the statement of its thesis, with an introduction and conclusion Demonstrates depth of understanding Development 3-Competent Identifies but not does not critically evaluate the author’s central argument, purpose, and approach to the subject The review is reasonably well organized 2-Minimally Competent Identified a few main ideas but not the thesis Demonstrates knowledge of the topic and provides supporting evidence and adequate detail Minimal analysis, is repetitious, or and lacks development of salient features of the book being reviewed Effective Use of Language Simplistic sentence structure and imprecise use of language Errors are present that interfere with the presentation of ideas and arguments Few errors 3-Competent Clear focus and consistent statement of argument The view has a semblance of structure but its coherence is minimized by poor organization 2-Minimally Competent Thesis is evident but confused Logically organized Organization lacks coherence Demonstrates knowledge Presents 1-Not Competent The paper lacks a coherent thesis or is poorly developed Poorly organized Lacks idea Style Mechanics of the topic, presents supporting arguments clearly and analytically, and provides excellent documentation Written in an appropriately formal tone without slang. Language is precise and sentence structure varied of the topic and provides supporting evidence and adequate detail Effective Use of Language The mechanics of the paper are correct. It is well written with no grammar or punctuation errors, and little or no use of the passive voice Few errors undeveloped ideas development and includes irrelevant information Simplistic sentence Incorrect sentence structure and structure and use of imprecise use of slang, jargon, or language inappropriate language Errors are present Excessive errors in that interfere with the grammar and presentation of ideas punctuation. and arguments Appendix B. Transferable Skills These are the seven transferable skills that the History department is working into its revised curriculum. They are listed below and are linked to the new university learning goals. They are also identified at the end of each learning objective in part one. 1. Critical thinking skills and Language analysis (Analyzing and documenting written primary sources; analyzing historical point of view in book reviews) [ULG Critical Thinking 1-6; Writing and Critical Reading 1-2,6] 2. Writing skills (formulating research into a coherent, clear argument—skills that include mastery of grammar, structure, clarity, style, organization). [ULG Writing and Critical Reading 1-7] 3. Visual literacy (Analyzing and documenting paintings, architecture, material culture, political cartoons, film) [ULG Critical Thinking 1-5; Writing and Critical Reading 5-6] 4. Quantitative Reasoning aka Numeracy (Analyzing and documenting statistics, interpreting charts and graphs, review of quantitative analysis and production of quantitative analysis) [ULG Quantitative Reasoning 1-6] 5. Digital Database Use (search strategies, data mining written/printed primary sources) [ULG Quantitative Reasoning 6] 6. Research and Citation Management (not simply writing a citation, but saving and re-deploying research in one way for one course or paper, in another for the next; teaching students to build their own reference collection through Zotero or otherwise) [ULG Critical Thinking 2; Writing and Critical Reading 7] 7. Presentation/public speaking through oral presentation at mini-conference, in-class presentation or video (i) communicates ideas orally in an effective manner; (ii) Logical organization to research presentation; (iii) demonstrates understanding of presentation technology (e.g., Mastery of Power Point or Prezi) and uses it effectively [ULG Speaking and Listening 1-7] Appendix C. Proposed Revision, Source Analysis Rubric Close Reading (ability to accurately summarize; understanding document contents) Sourcing: Characteristics of the Document & Author’s Point of View Contextualizing & Historical Knowledge Analysis Organization/ Development Style/ Mechanics Exceeds Expectations Very-well written summary, covering all key points of document(s) and demonstrating excellent understanding of large themes and important details Excellent understanding of the source (author, purpose, audience, type of document) and limitations of the source Often Exceeds Accurate summary of document; demonstrates understanding of some but not all large themes Meets Expectations Competent summary of document(s) contents; acknowledgment of most themes and details Minimally Competent Constructs some meaning from text Does Not Meet Demonstrates no understanding of document’s contents Understands the historical characteristics of the document in relation to the topic of study Shows a basic understanding of the characteristics of the source Illustrates an inconsistent understanding of the source Demonstrates no understanding of the source Analysis and summary is informed by strong content knowledge; specific dates and facts astutely used as vehicles for analysis and as evidence Recognizes and accurately summarizes historical changes; accurately uses dates and facts to support analysis Some attempt to include specific historical content; some facts/dates may be wrong and does not substantially affect the analysis Demonstrates some historical knowledge; text contains factual errors that undermine analysis Sophisticated, informed, and thorough analysis of what a historian learns from this document; evidence from document serves to support analysis The paper is very well structured and persuasive in the statement of its thesis, with a compelling introduction and conclusion. Demonstrates depth of understanding of the topic, presents supporting arguments clearly and analytically, and provides excellent documentation Paper is well written and flows nicely; proper and professional format; free from mechanical errors; Language is precise and sentence structure varied. Accurate but not sophisticated analysis of document Demonstrates basic understanding of document without errors that undermine analysis Demonstrates a minimal understanding of the document Little or no attempt to use specifics historical knowledge to assist in analysis; major errors in content knowledge may be evident; facts may be listed with no application No real claims and/ or use of evidence. Demonstrates depth of understanding of the topic, presents supporting arguments clearly and analytically, and provides excellent documentation Demonstrates knowledge of the topic and provides supporting evidence and adequate detail Presents undeveloped ideas Lacks idea development and includes irrelevant information Well written in a formal and objective tone, and contains few mechanical errors Paper is written in an appropriate and formal, objective tone with few errors Errors interfere with the presentation of ideas and arguments; simplistic sentence structure Excessive errors in grammar and punctuation; use of slang or inappropriate language