Report of General Education Assessment 2014 - 2015 General Education Committee 2014 – 2015 Joey Anderson, Chair - Mathematics Debbie Bouton – Learning Unit Allan DiDonato - Collaborative Learning English, Reading & Humanities Catherine Felton – Behavioral and Social Sciences Lisa Foley – Collaborative Learning English, Reading & Humanities Richard Helms – Behavioral and Social Sciences Carolyn Jacobs – Arts and Communication Mary Ann Bradham - Mathematics Jorge Koochoi – Foreign Language Holly Maurer – Arts and Communication Erin Payton – Library Services Theresa Russo – Arts and Communication Lisa Spring - Science Eric Taylor – Business and Accounting Yanessa Page – English, Reading, Humanities Kathryn Wells – Behavioral and Social Sciences Linda White - Arts and Communication Elizabeth West – Collaborative Learning English, Reading & Humanities Keith Powell – Construction Technology Carl Arrington – Information Technology Roschella Stephens – Health Sciences Terri Manning - Institutional Research Denise Wells – Institutional Effectiveness Krys Swartz – Institutional Effectiveness 1 Contents General Education Goals and Courses Used for Assessment during the 2014-2015 Academic Year .......................................................................................................................................4 General Education Assessment Procedure ..............................................................................................5 2014-2015 General Education Assessment – Overall Results Summary ..............................................6 General Education Goal One: Communication .....................................................................................9 A. Oral Communication Assessment: ................................................................................... 9 B. Written Communication Assessment ............................................................................. 12 General Education Goal Two: Mathematics ........................................................................................14 General Education Goal Three: Critical Thinking and Problem Solving .........................................16 A. BIO 110, CHM 131, CHM 151, PHY 110 –Critical Thinking ...................................... 16 B. ENG 112 – Argument-Based Research ......................................................................... 20 C. PSY 150 – General Psychology ..................................................................................... 21 D. ECO 251 & ECO 252 – Economics ............................................................................... 23 E. CIS 110 – Introduction to Computers ............................................................................ 24 General Education Goal Four: Cultural Awareness ...........................................................................25 A. COM 110 – Introduction to Communication ................................................................. 25 B. COM 231 – Public Speaking ......................................................................................... 26 C. COM 120 – Interpersonal Communication.................................................................... 27 D. SPA 112 – Elementary Spanish II ................................................................................. 28 General Education Goal Five: Social and Behavioral Social Sciences...............................................31 A. HIS 111 – World Civilizations I .................................................................................... 31 B. SOC 210 – Introduction to Sociology............................................................................ 32 General Education Goal Six: Natural Sciences ....................................................................................34 General Education Goal Seven: Humanities/Fine Arts ........................................................................38 A. ART 111 – Art Appreciation ......................................................................................... 38 B. MUS 110 – Music Appreciation .................................................................................... 40 C. HUM 130 - Myth in Human Culture ............................................................................. 41 General Education Goal Eight: Information Literacy ........................................................................44 APPENDIX ................................................................................................... Error! Bookmark not defined. Goal One: Communication ........................................................................ Error! Bookmark not defined. A. Oral Communication .......................................................Error! Bookmark not defined. 2 B. Written Communication..................................................Error! Bookmark not defined. Goal Two: Mathematics ............................................................................. Error! Bookmark not defined. Goal Three: Critical Thinking and Problem Solving ............................. Error! Bookmark not defined. A. BIO 110, CHM 131, CHM 151, PHY 110 –Critical Thinking ..... Error! Bookmark not defined. B. ENG 112 – Argument Based Research ...........................Error! Bookmark not defined. C. PSY 150 – General Psychology ......................................Error! Bookmark not defined. D. ECO 251 & ECO 252 – Economics ................................Error! Bookmark not defined. E. CIS 110 - .........................................................................Error! Bookmark not defined. Goal Four: Cultural Awareness ............................................................... Error! Bookmark not defined. A. COM 110 – Introduction to Communications ................Error! Bookmark not defined. B. COM 231 – Public Speaking ..........................................Error! Bookmark not defined. C. COM 120 – Interpersonal Communication.....................Error! Bookmark not defined. D. SPA 112 – Intermediate Spanish ....................................Error! Bookmark not defined. Goal Five: Social and Behavioral Science ................................................. Error! Bookmark not defined. A. HIS 111 – World Civilizations I .....................................Error! Bookmark not defined. B. SOC 210 – Introduction to Sociology.............................Error! Bookmark not defined. Goal Six: Natural Sciences ......................................................................... Error! Bookmark not defined. Goal Seven: Humanities and Fine Arts .................................................... Error! Bookmark not defined. A. ART 111 – Art Appreciation ..........................................Error! Bookmark not defined. B. MUS 110 – Music Appreciation .....................................Error! Bookmark not defined. C. HUM 130 – Myth in Human Culture ..............................Error! Bookmark not defined. Goal Eight: Information Literacy ............................................................ Error! Bookmark not defined. 3 General Education Goals and Courses Used for Assessment during the 2014-2015 Academic Year Courses General Ed Goal assessed Communication – Students will effectively communicate both orally and COM 110 in writing. Students will demonstrate the ability to locate, critically COM 231 evaluate, and present information. COM 120 ENG 111 Mathematics – Students will apply mathematical concepts and skills to MAT 115 analyze, manipulate, and interpret quantitative data. MAT 161 Critical Thinking / Problem solving – Students will demonstrate the BIO 110 ability to identify, analyze, question, and evaluate content as a guide to CHM 131 understanding and action. CHM 151 PHY 110 ENG 112 PSY 150 ECO 251 ECO 252 CIS 110 Cultural Awareness – Students will demonstrate knowledge of cultural COM 110 differences. SPA 112 COM 231 COM 120 Social / Behavioral Sciences – Students will demonstrate an HIS111 understanding of social institutions and of the diversity of human SOC210 experiences within a framework of historical and cultural contexts. Natural Sciences – Students will demonstrate comprehension of the BIO110 major steps of the scientific method. CHM 131 CHM 151 PHY 110 Humanities / Fine Arts – Students will demonstrate knowledge of the MUS 111 humanities and critical skills in assessing cultural/artistic merit and HUM 130 significance. Information Literacy - Students will effectively use research techniques to identify, select, use, document and evaluate information sources appropriate to a particular need. 4 ENG 112 General Education Assessment Procedure Every fall, the General Education Committee begins the process of creating a general education portfolio for Central Piedmont Community College. The process is as follows: 1. In early fall, sections of the appropriate courses by goal area are randomly selected by Planning and Research for assessment. 2. The randomly selected sections are distributed to committee members representing academic areas reflected in the general education portfolio and the appropriate division directors and deans. 3. Assessment data are collected by the faculty members assigned to those randomly selected sections during the fall term. 4. Grading is completed in fall for some courses and in spring for others. 5. Faculty review, discuss results and decide what change, if any, they should make. 6. Results are examined by the General Education Committee in the spring. 7. Reports of results are made to the division directors of each unit. 8. Committee members prepare a written report of assessment results, analysis and strategies for improvement. Reports are reviewed by the committee and submitted to Planning and Research. 9. Planning and Research compiles the written reports, assessment materials and student samples into a portfolio. 10. The committee edits the final report. 11. The report is taken to the Learning Council and the Cabinet. 12. A response is received from the deans in regard to action items, recommendations, budget issues, needs, etc. by September 25th of the following year. 5 2014-2015 General Education Assessment – Overall Results Summary Result 70% score 4 on all parts of rubric ENG 111 64% of students met minimum qualification 67% of seated students scored 3 or better 53% of online students scored 3 or better 70% score 3 of 5 each of 3 goal areas MAT 115 and MAT 161 70% of students met each of three goal areas 68% of seated students met each of three goal areas 79% of online students met each of three goal areas BIO 110, CHM 131, CHM 151, PHY 110 - 70% score 70% or above BIO 110, CHM 131, CHM 151, PHY 110 – 80% of students score 70 or better 77% of seated students scored 70 or better 90% of online students scored 70 or better ENG 112 – 70% score 2 of 2 on rubric ENG 112 –70% of students scored 2 of 2 69% of seated students scored 2 of 2 77% of online students scored 2 of 2 PSY 150 - 70% score 12 or better PSY150 –74% of students scored 12 or better 81% of seated students scored 12 or better 70% of online students scored 12 or better ECO 251 & 25270% of students score 6 or better ECO – 74% of students scored 6 or better 69% of seated students scored 6 or better 84% of online students scored 6 or better CIS 110 – 70% score 7 of 10 or better CIS 110 – 73% of students scored 7 or better 74% of seated students scored 7 or better 79% of online students scored 7 or better Overall –75% of students met the benchmark 74% of seated students met the benchmark 79% of online students met the benchmark 6 Met NOT MET MET 88% of students scored 3 or better 86% of seated students scored 3 or better 92% of online students scored 3 or better MET Objective 70% score 3 of 5 on rubric COM 110,COM 120 and COM 231 MET General Ed Goal Area 1. Oral Communication Students will effectively communicate orally by demonstrating the ability to locate, critically evaluate, and present information. 2. Written Communication – Students will effectively communicate in writing by demonstrating the ability to locate, critically evaluate, and present information. 3. Mathematics – Students will apply mathematical concepts and skills to analyze, manipulate, and interpret quantitative data. 4. Critical Thinking / Problem solving – Students will demonstrate the ability to identify, analyze, question, and evaluate content as a guide to understanding and action. COM 110 –84% of students scored 7 or better 84% of seated students scored 7 or better 82% of online students scored 7 or better SPA 112 - 70% score 70 or better SPA 112 - 83% of students scored 70 or better 76% of seated students scored 70 or better 96% of online students scored 70 or better 6. Social / Behavioral Sciences – Students will demonstrate an understanding of social institutions and of the diversity of human experiences within a framework of historical and cultural contexts. 70% meet objective HIS 111 - 12 of 20 on rubric Overall –85% of students met the benchmark 84% of seated students met the benchmark 86% of online students met the benchmark HIS 111 –91% of students scored 12 or better 92% of seated students scored 12 or better 89% of online students scored 12 or better 7. Natural Sciences – Students will demonstrate comprehension of the major steps of the scientific method. 8. Humanities / Fine Arts – Students will demonstrate knowledge of the humanities and critical skills in assessing cultural/artistic merit and significance. 70% score 70% or above SOC 210 - 2 of 3 on rubric SOC 210 –87% of students scored 2 or higher 91% of seated students scored 2 or better 87% of online students scored 2 or better Overall –89% of students met the benchmark 92% of seated students met the benchmark 87% of online students met the benchmark 91% of students scored 70% or higher 89% of seated students scored 70 or better 97% of online students scored 70 or better ART 111- 70% score 70 or above ART 111 –97% of students scored 70 or above 96% of seated students scored 70 or above 98% of online students scored 70 or above MUS 110 - 70% score 70 or above MUS 110 –63% of students scored 70 or above 61% of seated students scored 70 or above 79% of online students scored 70 or above HUM 130 – 70% score 3 of 5 on rubric HUM 130 - 85% of students scored 3 or higher 75% of seated students scored 3 or better 87% of online students scored 3 or better Overall –79% of students met the benchmark 71% of seated students met the benchmark 89% of online students met the benchmark 7 MET COM 120 - 79% of students scored 70 or better 87% of seated students scored 70 or better 71% of online students scored 70 or better MET COM 231 - 94% of students scored 70 or better 93% of seated students scored 70 or better 100% of online students scored 70 or better MET COM 110 COM 231 COM 120 70% score 7 of 10 points MET 5. Cultural Awareness – Students will demonstrate knowledge of cultural differences. Pilot results: 70% score 70 or above 8 ENG 112 –81% of students scored 7 or more of 10 points 83% of seated students met the benchmark 78% of online students met the benchmark MET 9. Information Literacy Students will effectively use research techniques to identify, select, use, document and evaluate information sources appropriate to a particular need. General Education Goal One: Communication Students will effectively communicate both orally and in writing. Students will demonstrate the ability to locate, critically evaluate, and present information. (Note: Students are assessed in both Communication and English classes for oral and written communication skills.) A. Oral Communication Assessment: Objective: 70% of students will meet minimal objective for effective oral presentation. Assessment Benchmark: 70% of student speeches evaluated will receive at least a score of 3 or better on a 5-point evaluation rubric. Three communication courses were selected for the assessment with the following enrollments in the Fall 2014: Term Fall 2014 Fall 2014 Fall 2014 Number of Sections 56 4 50 Number Enrolled 1,355 97 1,227 Course & Number COM 110 COM 120 COM 231 Reporting for General Education Results Reporting Year 2014-2015 White, Linda Last Name: Communication Department: Oral Communication Goal Measured: COM 110, COM 120, COM 231 Course(s) in which assessment took place: 70 % Benchmark percentage: 3 Benchmark minimum: 5 Benchmark maximum: 201 Assessed: 176 Met benchmark: 88 % Percentage Met Benchmark: 152 Seated assessed: 131 Seated met benchmark: 86 % Percentage seated met benchmark: 49 Online assessed: 45 Online met benchmark: Percentage of online met benchmark: 92 % Method: Twenty-three out of 114 sections of COM 110, COM 120 and COM 231 were randomly selected by Planning and Research for CPCC’s General Education Oral Communication assessment. The selected sections include classes taught by full-time and part-time Communication 9 Tool: faculty, traditional and online sections offered at various campuses. A standard assignment for all students in these courses is to prepare and deliver speeches (Informative speeches in COM 110; persuasive speeches in COM 231; Informative research Oral Presentation in COM 120). The instructors of the sections selected for GEN ED assessment are given standardized directions for the recording of student speeches and for the return of the recordings to the designated person. The management of the COM 110, COM 231 and COM 120 GEN ED Oral Communication Assessment process is the responsibility of the full-time Communication faculty member serving on the College General Education Committee. Spring 2015 - The faculty member responsible for GEN ED Oral Communication Assessment randomly distributes the recorded student speeches to full-time Communication faculty to review. These are blind reviews and are completed using a standard oral communication rubric (see attachment) developed and tested by Communication faculty. Sections were identified as COM 110, COM 231 or COM 120 but were assigned a different section number. Each section included specifics about the assignment provided by the instructor such as time limits, notes allowed, source citations, visual aid requirements. (Note: one section of COM 231 was not included for assessment due to theft from vehicle) Student speeches were assessed using the Oral Communication Rubric. Faculty the rubric developed and revised by COM in spring 2014. The rubric focuses on three criteria (Organization, Content and Delivery). Each criteria was detailed in the rubric. Scores were averaged to determine the overall score. A summary of the benchmark items of the rubric include: Students demonstrate adequate oral communication skills by including all or almost all of the following according to the oral communication rubric:(see attached) Organization: An attention-getting introduction that orients the audience to the subject and motivates the audience to listen Well-developed points with transitions and internal summaries; content should reflect excellent research and appropriate citation of sources A conclusion that restates the central idea; summarizes main points; and uses an effective concluding statement which motivates the audience. Content: Focused, logical and coherent development of information; use of vivid, accurate language; good use of repetition to reinforce key ideas, establish speaker credibility Content which develops main ideas using appropriate supporting material - examples, statistics, personal experience Appropriate citation of sources to support content Delivery: Use of vocal variety in rate, pitch and volume in order to maintain and heighten audience interest; effective pronunciation and articulation; lacks inarticulates Confident physical stance; eye contact addresses the entire audience; complementary gestures that demonstrate enthusiasm 10 Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: Use of extemporaneous style, effective use of notes; well-polished delivery No distinction was made between seated and online classes when reviewers assessed presentations. They were identified by course. What tool did you use? (Please include any differences between seated, online, or hybrid course or state that there were no differences.) Faculty noted no differences related to assessment tool for seated or online classes. Faculty commented that they were more confident of assessment when using the revised rubric. It was agreed that after two years with this rubric, faculty are satisfied and view this as an improvement. We will continue to suggest that COM classes have designated speaking areas with built-in recording equipment. It was noted that in the years since the assessment process began, faculty agree that overall improvement can be seen in student speeches. Strengths noted this year were better delivery skills which include more extemporaneous/conversational delivery, better citation of research sources, improved research sources At the same time, it was also noted that students do still struggle with the following: developing introductions with strong attention getters transitions inconsistency in citation of sources topic selection in COM 110 classes excessive use of video clips delivery with PowerPoint development of good PowerPoint We noted inconsistencies in student presentations, while speeches have improved, not all students are excelling but are more adequate. Students who are engaged with their topic tend to do better work. Scores were a bit higher but there are inconsistent results between sections so we do not find the difference due to improvement? 1. Distribute Assessment data and report to all COM faculty (full and part time) explaining the process, results and importance of General Education assessment process. 2. Chairs will continue to reinforce expectations for consistent assignments when reviewing syllabi, when communicating with part time faculty and as part of regular classroom observations. 3. Continue to pursue online site to house resources for part time faculty. 4. Explore feasibility of conducting Gen Ed workshop to be planned by full time faculty for part time faculty. The Oral Communication Goal Was Met. 11 B. Written Communication Assessment The requirement of the English 111 course for students is designed to assure that each student meets a minimal level of competence in writing. For this reason, faculty set the following objective: Objective: 70% of students will be able to communicate effectively in writing. Means of assessment: 70% of students will complete the writing exam with a passing grade. One English course was selected for the assessment and the enrollments in Fall 2014 was as follows: Term Fall 2014 Number of Sections 150 Number Enrolled 3,719 Course & Number ENG 111 Reporting for General Education Results Reporting Year 2014-2015 Page Last Name: English, Reading & Humanities Department: Written Communication Goal Measured: ENG 111 Course(s) in which assessment took place: 70 % Benchmark percentage: 4 Benchmark minimum: 4 Benchmark maximum: 151 Assessed: 97 Met benchmark: 64 % Percentage Met Benchmark: 121 Seated assessed: 81 Seated met benchmark: 67 % Percentage seated met benchmark: 30 Online assessed: 16 Online met benchmark: Percentage of online met benchmark: 53 % In order to measure objectives and student outcomes, students were Method: asked to choose one of the topics below and write one complete paragraph. Students were expected to include prewriting, drafting, and revising of the paragraph. The paragraph was to be reflective of their level of writing and include a topic sentence, supporting details, an appropriate closing, and meet standards of correctness. Students chose one of the following topics: Describe a risk that paid off. Tool: What do you believe is the main purpose of obtaining a college degree? Explain or tell about a career that suits you best. The following Grading Rubric was used to evaluate each paragraph: Yes No __ ___ The paragraph has an appropriate topic sentence ___ ___ The paragraph stays on one topic that is stated in the topic sentence. ___ ___ The paragraph has supporting sentences that gave reasons/details/facts ___ ___ The 12 Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: Other: paragraph meets standards of correctness. Would this paragraph have scored a C or better? ___yes ___no Seated students would have been limited to time in class; online students may not have had the same time constraints Some faculty believe that this particular method of assessment for ENG111 is not sufficient for measuring the extent to which students should be capable of communicating in writing by the time they complete ENG111. Others believe that it does a good job assessing basic paragraph writing. Some feel that it is a good measure of students' ability to develop basic paragraphs, which is key to writing good essays. In addition, instructors do not "grade" their own students for this assessment, reducing the possibility of bias in the assessment. There are still concerns as to whether the tool effectively measures grammar and mechanics. The subjective nature of grading writing was also concern in terms of assessing student writing. The assessment is not a “regular” part of instructors’ syllabus or planned activities and some instructors feel rushed or that they have to interrupt their planned activities for the assessment. The assessment reveals a range in skills in students' writing. There was not enough evidence to suggest that online scores were lower than the scores of seated students simply as a result of the courses being online. Given that students online were not time-bound as were students in seated classes, the expectation was that online students might score higher. It might be beneficial to assess more online courses in the future. Some faculty said that there has been a noticeable decline in students' skills over the last year or two, so the results of the assessment were not surprising. This decline is reflected in the assessments over the last three years. Other instructors were concerned that the assessment still gives only a snapshot of an aspect of ENG111 that may not be significant and that we would need to assess substantive student work (like an essay) to get a better picture of what students are capable of. Recommend that all ENG111 instructors begin assessing an entire essay and that all full-time English faculty be involved on a rotating schedule as “readers” for the assessment. The tool for the assessment should be changed to reflect the move towards using an essay (the research essay that all ENG 111 instructors currently use for the core competency assignment) and the following rubric: Yes No ____ ___ The essay has a clear thesis statement. ____ ___ The essay contains multiple paragraphs that support the main idea with reasons/details/facts. ____ ___ The essay stays on topic. ____ ___ The essay effectively incorporates and cites relevant sources. ____ ___ The essay meets standards of correctness. Would this essay have scored a C or better? ____yes ____no The benchmark was not met for this assessment. The Written Communication Goal Was NOT Met. 13 General Education Goal Two: Mathematics Students will apply mathematical concepts and skills to analyze, manipulate, and interpret quantitative data. Math faculty determined that the skills necessary to meet the above goal are: 1. The ability to analyze quantitative data 2. The ability to manipulate quantitative data 3. The ability the interpret quantitative data Therefore, the following objective was set for the purpose of general education assessment: Objective: 70% of those taking the final exam will show mastery of each of the three goals. Means of Assessment: 70% of those taking the final exam will correctly answer three of five questions on each of the three goal areas. Two math courses were selected for the assessment and their enrollments for Fall 2014 were as follows: Term Fall 2014 Fall 2014 Number of Sections 12 31 Number Enrolled 281 982 Course & Number MAT 115 MAT 161 Reporting for General Education Results Reporting Year 2014-2015 Bradham Last Name: Mathematics Department: Mathematics Goal Measured: MAT 115, MAT 161 Course(s) in which assessment took place: 70 % Benchmark percentage: 60 Benchmark minimum: 100 Benchmark maximum: 408 Assessed: 284 Met benchmark: 70 % Percentage Met Benchmark: 345 Seated assessed: 234 Seated met benchmark: 68 % Percentage seated met benchmark: 63 Online assessed: 50 Online met benchmark: 79 % Percentage of online met benchmark: 15 Multiple Choice Questions on Final Exam. All classes had to be tested Method: in a proctored environment. Therefore there were no differences. Mastery on each of 3 goal requirements. Students must have 60% of the Tool: questions correct on each of the 3 parts. The mathematics full and part time faculty believe the assessment is a Faculty analysis: 14 Strengths: Weaknesses: Learn: Compare: Strategies: Other: strong tool for measuring the three goals accurately. Students who show mastery of the 3 goals on the assessment also are shown to have a high pass rate for the course. The first goal is the ability of a student to analyze quantitative data. This goal has students to view data in different formats such as tables or formulas or to know the definitions of terms. This is the strongest goal for the MAT 161 students who passed at 88%. The second goal requires students to have the ability to manipulate quantitative data. Both courses had successful results with this goal. However, this was the strength for the MAT 115 class with a pass rate of 90 percent. The third and often times the most difficult goal for students is the ability to interpret quantitative data. In this goal students are expected to use more applied real world problems. The students are also expected to be able to understand the meaning of the resulting answers. Once again both classes were successful with this goal as well. MAT 115 had a success rate of 89% which increased from the previous year. MAT 161 online students seem to perform stronger on all three goals over the face to face students in this course. Whereas the MAT 115 online vs. face to face students seem to perform at a much closer success rate. MAT 161 seated sections had the most difficulty with the second and third goals with having the pass rates at about 79%. Again this is above our benchmark. Therefore even though these were the most difficult goals, the students still showed mastery in both. Even though the MAT 115 students had a high success rate, it still shows that problems dealing with financial math and Venn diagrams can be difficult for students. Mathematics education is moving towards more applied word problems involving real life scenarios. As math education continues to evolve in this way, these are still the areas that students need continued practice. Students are being required to use higher level, critical thinking skills to solve problems in both Mathematical Models and College Algebra. The previous year, the College Algebra scores were higher for all three goals in the seated and online classes. One of the factors we believe may have contributed is the change in required math courses this past year. Many students are no longer needing the MAT 161 and therefore are taking other requirements. Many of the students in this course are either students who should be towards the end of their program or students who may have needed to repeat this course before the course is to be phased out. Both MAT 115 and MAT 161 will no longer be offered in Fall 2015. Both of these courses are evolving into new courses that are trying to have students work on more real life math concepts through discovery and lab/group work. The new courses will continue to have students look into more applied problems and push them to use more critical thinking skills in the future. Even though the MAT 161 and 115 courses will no longer be available to students, I still believe the data that we have received from the Gen Ed assessment will continue to help us improve the new courses that have been developed. The three goals that are assessed in these two courses are still goals that need to continue to be met in the math courses along with the emphasis on real world math. The College Goal for Mathematics Was Met. 15 General Education Goal Three: Critical Thinking and Problem Solving Students will demonstrate the ability to identify, analyze, question, and evaluate content as a guide to understanding and action. Efforts this year toward assessments of Critical Thinking and Problem Solving have been completed in multiple courses: Critical thinking assessments were conducted in BIO 110, CHM 131, CHM 151, PHY 110, ENG 112, PSY 150, ECO 251, and ECO 252. During the Fall 2014 term, course enrollments for BIO 110, ENG 112, PSY 150 ECO 251 were as follows: Term Fall 2014 Fall 2014 Fall 2014 Fall 2014 Fall 2014 Fall 2014 Fall 2014 Fall 2014 Fall 2014 Objective: Number of Sections 22 3 20 5 54 65 22 12 39 Number Enrolled 513 70 475 119 1,268 1,825 575 374 1,439 Course & Number BIO 110 CHM 131 CHM 151 PHY 110 ENG 112 PSY 150 ECO 251 ECO 252 CIS 110 70% of students will meet minimal standard set for Critical Thinking. A. BIO 110, CHM 131, CHM 151, PHY 110 –Critical Thinking To measure the goal, the following objective was set: Objective: 70% of students will meet minimal standards for Critical Thinking as to design an experiment using the scientific method. In using a story problem scenario students are encouraged to ‘role play’ to help them answer questions 5-14 which were based on critical thinking. Reporting for General Education Results Reporting Year 2014-2015 Spring Last Name: Natural Science Department: Critical Thinking/Problem Solving Goal Measured: BIO 110 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 16 Assessed: Met benchmark: Percentage Met Benchmark: Seated assessed: Seated met benchmark: Percentage seated met benchmark: Online assessed: Online met benchmark: Percentage of online met benchmark: Method: Tool: Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: 170 148 87 % 82 69 84 % 88 79 90 % A series of 10 multiple choice questions were developed that continued from the Scientific Method assessment. These questions allowed students to move from the scenario of developing their own scientific experiment to applying their ability to think through discipline based questions. A 10 multiple choice question assessment was given to all students (seated and online). ** 3 of the 4 online sections were assessed in the lab classroom Pros and Cons to the tool: Students were already in a 'scientific thinking’ mode as they continued through this assessment. Students had different scenarios in each question to consider. Faculty continue to see the trend of online scores higher than face to face students. There was much improved overall scoring from last year: from 77.7% to 87.1% This increase may have several factors involved: different student population, different faculty assessing the sections, varied student science backgrounds and less variation in dates when students were assessed. Although online students scored higher than seated students, 3 of the 4 sections were assessed the same as the seated students. Changing wording and questions for this assessment helped student understanding and ability to show what they had learned. Although there will always be the faculty and student variability, placing the assessment closer to the end of the semester for all students reduced the variability in scores due to practice skills and accomplished learning. Online students continued to score better than seated students. This year, however, faculty saw a slight reduction in scores for the online students (93.7% in 2013 compared to 89.77% in 2014). The gap between online students and seated students was less this year. There was a 11.9% difference in 2013 compared to a 5.62% difference in 2014. Faculty will continue to work with modeling and teaching scientific thinking and will plan on giving the assessment at the end of the semester. Reporting for General Education Results Reporting Year 2014-2015 Spring Last Name: Natural Science Department: Critical Thinking/Problem Solving Goal Measured: CHM 131, CHM 151 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 242 Assessed: 17 Met benchmark: Percentage Met Benchmark: Seated assessed: Seated met benchmark: Percentage seated met benchmark: Online assessed: Online met benchmark: Percentage of online met benchmark: Method: Tool: Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: 187 77 % 193 143 74 % 49 44 90 % A series of 10 multiple choice questions were developed that continued from the Scientific Method assessment. These questions allowed students to move from the scenario of developing their own scientific experiment to applying their ability to think through discipline based questions. A 10 multiple choice question assessment was given. It was the same assessment given to both seated and online students. Online students scored higher than seated students, but this percentage difference could be due to sample sizes. Pros and Cons to tool: Students were already in a 'scientific thinking' mode as they continued through this assessment. Students had different scenarios to consider. Students met faculty expectations. The number of questions were changed for this year's assessment. Online students continue to score better than the seated students. Students overall scored better than last year with a continued trend of better online student scoring over seated student scores. The increase in scores over last year may be due to less variation in dates when students were assessed. All students were assessed in late October to early November. Some difference in learning could be due to varied student science backgrounds. Those students who have taken other science courses may have more ability to think critically. The number of problematic questions reduced from 5 to 4 questions this year. There was a smaller increase in assessment scores for the seated students: 72% passing in 2013 compared to 74.09% passing in 2014. There was a bigger increase in assessment scores for the online students: 85.6% in 2013 compared to 89.8% in 2014. Faculty will continue to work with modeling and teaching scientific thinking. Faculty will be encouraged to plan on giving the assessment at the end of the semester. Reporting for General Education Results Reporting Year 2014-2015 Spring Last Name: Natural Science Department: Critical Thinking/Problem Solving Goal Measured: PHY 110 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 86 Assessed: 18 Met benchmark: Percentage Met Benchmark: Seated assessed: Seated met benchmark: Percentage seated met benchmark: Method: Tool: Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: 65 76 % 86 65 76 % A series of 10 multiple choice questions were developed that continued from the Scientific Method assessment. These questions allowed students to move from the scenario of developing their own scientific experiment to applying their ability to think through discipline based questions. A 10 multiple choice question assessment was given to all students (no online students were assessed). Pros and Cons to tool: Students were already in a 'scientific thinking' mode as they continued through this assessment. Students had different scenarios to consider. Faculty were partially pleased with student results. Although questions were reworded and changed completely, students still struggled with 4 of the questions on the assessment. The score met the benchmark but was lower than last year's scores. Some of the possible reasons for lack of improvement: different student population varied student science backgrounds "fear" of physics There were no online students assessed. A good number of students were able to apply their knowledge. Although wording and questions changed, the core ability to apply knowledge and thinking skills is still a challenge. (no online students assessed) Rewording of questions and changing of questions was hoped to eliminate confusion for students. At this point, the results are not showing that faculty have identified where the confusion or lack of skill is specifically. Last year 86% of students (all seated students) passed the assessment with 4 questions that were common challenges. This year 75.58% of students (all seated students) passed the assessment with 4 questions that were common challenges. Faculty will continue to work with modeling and teaching scientific thinking. They will continue to search for better or different wording to questions that may help students understand what is being asked of them in their application of knowledge. Faculty will plan on giving the assessment at the end of the semester. BIO 110, CHM 131, CHM 151, & PHY 110 Totals Overall 80% of students met the Critical Thinking benchmark 77% of seated students met the Critical Thinking benchmark 90% of online students met the Critical Thinking benchmark 19 B. ENG 112 – Argument-Based Research Objective: 70% of students will meet minimal standards for Critical Thinking using student essays as the basis for assessment. Reporting for General Education Results Reporting Year 2014-2015 West Last Name: English, Reading & Humanities Department: Critical Thinking/Problem Solving Goal Measured: ENG 112 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 203 Assessed: 143 Met benchmark: 70 % Percentage Met Benchmark: 168 Seated assessed: 116 Seated met benchmark: 69 % Percentage seated met benchmark: 35 Online assessed: 27 Online met benchmark: Percentage of online met benchmark: 77 % We reviewed argument/research essays to evaluate skills. Method: We used a critical thinking rubric to analyze these essays. Tool: Assessment tool needs to change because the course is changing for Fall Faculty analysis: 2015. We have a new rubric that accurately reflects outcomes and will apply it next year. Online students were strong this year due to some Blackboard initiatives Strengths: (rubrics, in-line grading) also utilized Online Tutoring. Weaknesses: Learn: Compare: Strategies: Face-to-Face students were sharing more complex topics. Challenging course for online students to grasp. Seeing a difference in skill set for ENG 112 students. Again, difference in skill set. Students are not willing to utilize resources such as the Academic Learning Center. We are interested in seeing how our new course is going to provide more guidance and opportunities for faculty to pursue own interests. This year's assessment numbers were better, but we feel that the numbers will be even better next year due to faculty excitement for the new course. Our rubric is changing and we are going to do more training for FT and PT instructors to align outcomes and curriculum goals. 20 C. PSY 150 – General Psychology Objective: 70% of students taking the Critical Thinking test will answer correctly 7 of 10 questions. Reporting for General Education Results Reporting Year 2012-2013 Helms Last Name: Social/Behavioral Sciences Department: Critical Thinking/Problem Solving Goal Measured: PSY 150 Course(s) in which assessment took place: 70 % Benchmark percentage: 60 Benchmark minimum: 100 Benchmark maximum: 161 Assessed: 119 Met benchmark: 74 % Percentage Met Benchmark: 53 Seated assessed: 43 Seated met benchmark: 81 % Percentage seated met benchmark: 108 Online assessed: 76 Online met benchmark: 70 % Percentage of online met benchmark: Ten Question Tool based on a fictitious experiment investigating the Method: efficacy of providing early reading instruction to kindergarten students to prepare them for first grade. One group received reading instruction, the other did not. Dependent variable was first grade reading scores. Assessment consisted of ten questions: First four questions focus on general scientific method, and count 5 points each The next two questions focus on analysis of data, and count 10 points each Final four questions involve evaluative critical thinking, and count 15 points each Total possible points: 100. Benchmark is a grade of 60 or higher. This grading system was implemented this year to reduce confusion over the meaning of grades on the assessment. No differences between seated and online courses. Ten Question Tool based on a fictitious experiment investigating the Tool: efficacy of providing early reading instruction to kindergarten students to prepare them for first grade. One group received reading instruction, the other did not. Dependent variable was first grade reading scores. No difference between seated and online courses. Faculty were generally satisfied with the assessment tool, which was Faculty analysis: altered from previous years in two ways: 1: Two questions were added that tasked students on higher levesl of Bloom's Taxonomy. 2: Grading was changed to reflect a 100 point range, rather than the previous grading system. This 100 point system helped prevent student confusion regarding the outcome of the test. As in previous years, students demonstrated significant strengths in Strengths: 21 Weaknesses: Learn: Compare: Strategies: analysis/interpretation components of the assessment process, indicating critical thinking skills consistent with higher levels of Bloom's Taxonomy. No significant differences between seated and online courses. As in previous years, students demonstrated some difficulty with terminology and research processes. There was significant variation from section to section, but overall students had difficulty labeling different components of research (independent versus dependent variables, for instance). No real differences between seated and online course sections. Most faculty learned that students do appear to have developed sufficient analytical and interpretive skills to determine what the outcomes of research imply, while they still have some difficulty identifying the mechanical components of that research. No real difference between seated and online courses. Results from this year were somewhat improved over results from 20132014 academic year. This appears to be largely attributable to the changes made in the assessment tool this year that emphasized critical thinking and analytical/interpretive skills (the actual focus of the assessment) rather than on memorization of research terminology. This appears to apply to both seated and online course sections. Given the diversity of the psychology teaching faculty, and the fact that a Gen Ed assessment will not be conducted in the 2015-2016 academic year, it is unlikely that the results of this assessment will produce a systemic change in the overall department-wide approach to teaching psychology, to the extent that such an approach exists (or doesn't exist) already. Each individual instructor will take from these results either endorsement of their approach to the course material, or an indication of a need for change in the emphasis on specific topics in future course sections. Traditionally, results of the Gen Ed assessments have not produced significant department-wide changes in teaching methods. 22 D. ECO 251 & ECO 252 – Economics Objective: 70% of students taking the Critical Thinking test will answer correctly 6 of 10 questions. Reporting for General Education Results Reporting Year 2014-2015 Taylor Last Name: Economics Department: Critical Thinking/Problem Solving Goal Measured: ECO 251, ECO 252 Course(s) in which assessment took place: 70 % Benchmark percentage: 6 Benchmark minimum: 10 Benchmark maximum: 465 Assessed: 344 Met benchmark: 74 % Percentage Met Benchmark: 320 Seated assessed: 222 Seated met benchmark: 69 % Percentage seated met benchmark: 145 Online assessed: 122 Online met benchmark: 84 % Percentage of online met benchmark: All sections were assessed with a 10 question assessment. The majority of Method: the questions are scenario based questions the students must think through to come to a proper conclusion. The assessment was identical for all classes assessed. Same as above Tool: The faculty analysis and discussions focused on the persistent gap Faculty analysis: between face-to-face and online classes. In previous semesters, faculty have speculated on the differences and attempted to test the differences. These small tests were met with little success. Failure to substantially explain the performance gap between the delivery methods had led to new conclusions. Newer faculty (i.e. new eyes on the problem) suggest it may be a selection-bias problem. Online students know they must be selfmotivated and are able to search and find answers more efficiently. This may lead to higher test results. The online students performed well. As discussed above, the faculty Strengths: perceive a work ethic gap in the online sections. This leads to better results as they are able to use the book without being hand-held through the process. The face-to-face students usually require a bit more hand-holding through Weaknesses: the materials. Even when presented with an open-book/open-note assessment, the face-to-face students performed worse in what would otherwise be identical circumstances. The face-to-face students may require a bit more attention to close the Learn: performance gap relative to the online sections. Overall results were about the same and was not statistically different. Compare: The online vs face-to-face gap has existed for a few semesters. The current strategy has been placed on hold until we see what the new Strategies: 23 assessment will be in the future as the goals are going through a transformation. E. CIS 110 – Introduction to Computers Objective: 70% of students enrolling in CIS 110 will score a 70% or higher on the combined average of unit performance from Word, PowerPoint, Excel, and Access course units. Reporting for General Education Results Reporting Year 2012-2013 Arrington Last Name: Information Technology Department: Computer Skills/Critical Thinking Goal Measured: CIS 110 Course(s) in which assessment took place: 70 % Benchmark percentage: 7 Benchmark minimum: 10 Benchmark maximum: 203 Assessed: 149 Met benchmark: 73 % Percentage Met Benchmark: 72 Seated assessed: 59 Seated met benchmark: 82 % Percentage seated met benchmark: 131 Online assessed: 90 Online met benchmark: 69 % Percentage of online met benchmark: Used An Average of Unit Performance from Word, PowerPoint, Excel, Method: and Access course units SimNet Tool: Faculty Approved of Results Faculty analysis: Easy to administer, doesn’t disrupt normal course activity Strengths: There could be more summative assessment Weaknesses: SimNet has produced better course performance than previous tool, SAM Learn: Compare: Strategies: Overall 75% of students met the Critical Thinking benchmark 74% of seated students met the Critical Thinking benchmark 79% of online students met the Critical Thinking benchmark The College Goal for Critical Thinking Was Met. 24 General Education Goal Four: Cultural Awareness Students will demonstrate knowledge of cultural similarities and differences. Because cultural awareness is not the domain of one discipline but is viewed by the College as being incorporated across the curriculum, assessment for cultural awareness should be done in a number of General Education courses. This assessment has been conducted in COM 110 and SPA 112. Enrollments for Fall 2014 are as follows: Term Fall 2014 Fall 2014 Fall 2014 Fall 2014 Objective: Number of Sections 56 50 4 9 Number Enrolled 1,355 1,227 97 210 Course & Number COM 110 COM 231 COM 120 SPA 112 70% of students taking COM 110 Cultural Awareness test will answer correctly 7 of 10 questions; 70% of SPA 112 students will score 80% or higher on the assessment. A. COM 110 – Introduction to Communication Reporting for General Education Results Reporting Year 2014-2015 Russo Last Name: Communication Department: Cultural Awareness Goal Measured: COM 110 Course(s) in which assessment took place: 70 % Benchmark percentage: 7 Benchmark minimum: 10 Benchmark maximum: 819 Assessed: 686 Met benchmark: 84 % Percentage Met Benchmark: 747 Seated assessed: 627 Seated met benchmark: 84 % Percentage seated met benchmark: 72 Online assessed: 59 Online met benchmark: 82 % Percentage of online met benchmark: Students completed a 10 multiple choice question test assessing Method: knowledge of cultural similarities and differences relevant to communication course content. Given in 48 total COM110 sections (42 traditional, 6 distance). First short session sections were not assessed and 4 other sections due to faculty error or non-participation. Questions were related to communication/culture and language, nonverbal, gender, and 25 Tool: Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: Other: perception. Students were given an incentive in 25% of sections. (It is becoming increasingly difficult to capture this information). Incentive ranged from extra credit points, to stand alone quiz, extra points on exam. Assessment was administered after 10/20 and before the close of the semester. Assessments administered online were timed. A 10 question multiple choice assessment Overall results were lower overall vs. yag. Overall 83.8 (2014)vs. 86.7% (2013), Since 2012 results have exceeded 83% meeting benchmark. Faculty recognized that students may lack strength in recognizing the possibility of more than one correct answer (Q8-answer is both A&C) Generally students are open about recognizing culture as an integral component of effective communication. Results overall were lower vs. yag. Overall 83.8 (2014)vs. 86.7% (2013). Online student scored slightly lower than traditional seated classes (81.9% vs. 83.9%). Online student scored 7% lower than yag. This may be due to an increase in sections assessed (4 sections 2013 vs. 6 sections in 2014) Change Question #8 Faculty and students value the assessment process for this goal B. COM 231 – Public Speaking Reporting for General Education Results Reporting Year 2012-2013 Russo Last Name: Communication Department: Cultural Awareness Goal Measured: COM 231 Course(s) in which assessment took place: 70 % Benchmark percentage: 7 Benchmark minimum: 10 Benchmark maximum: 155 Assessed: 146 Met benchmark: 94 % Percentage Met Benchmark: 136 Seated assessed: 127 Seated met benchmark: 93 % Percentage seated met benchmark: 19 Online assessed: 19 Online met benchmark: 100 % Percentage of online met benchmark: Students completed a 10-question multiple choice test assessing student Method: knowledge of culture as an aspect of Public Speaking course content. The assessment was given in 10 randomly selected (8 f to f, 2 OL-combined). Questions related to communication/culture, language, and demographic audience analysis. Assessments for traditional classes were administered in the classroom; online they were administered online. A 10-question multiple choice assessment Tool: Faculty was satisfied with results (overall +10% vs. yag). 94.2% met Faculty analysis: benchmark vs. 84.9% met benchmark yag. No changes were made to the 26 Strengths: Weaknesses: Learn: Compare: assessment BUT the benchmark was "7" in 2014 vs. 8 in 2013. This was due to an error on the assessment. This change in benchmark still yielded improved results. 100% of students completing the assessment online met the benchmark. Overall results are satisfying and improved since yag due to the nature of this course where culture is not as overt as in other COM courses. Assessment results show that students are aware of culture as an aspect of all types of communication including COM231. Students continue to have difficulty with vocabulary. Faculty is not always communicating if there is an incentive to complete the assessment successfully--so capturing this information has been challenging. Generally students are open to recognizing culture as an integral component of effective Public Speaking, despite not being a major focus of the course. Situational or scenario-based questions continue to show improvement in scores because it is modeled how they are learning the concept of culture in the course from a practical standpoint. Scores improved vs. yag +9.3% vs. YAG (Overall) +10.6% vs. YAG (Face to face) Strategies: +11.1% vs. YAG (online) Faculty will revise questions if necessary--Gen Ed assessment is on hold for next year--no assessment will be given. C. COM 120 – Interpersonal Communication Reporting for General Education Results Reporting Year 2014-2015 Russo Last Name: Communication Department: Cultural Awareness Goal Measured: COM 120 Course(s) in which assessment took place: 70 % Benchmark percentage: 7 Benchmark minimum: 10 Benchmark maximum: 62 Assessed: 49 Met benchmark: 79 % Percentage Met Benchmark: 31 Seated assessed: 27 Seated met benchmark: 87 % Percentage seated met benchmark: 31 Online assessed: 22 Online met benchmark: Percentage of online met benchmark: 71 % Students completed a 10 multiple choice question test assessing student Method: knowledge of culture in the context of Interpersonal Communication. The assessment was given in all sections (4 total-2 traditional seated and 2 online sections). a 10 question multiple choice assessment Tool: 27 D. SPA 112 – Elementary Spanish II Reporting for General Education Results Reporting Year 2014-2015 Koochoi Last Name: Foreign Languages Department: Cultural Awareness Goal Measured: SPA 112 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 155 Assessed: 128 Met benchmark: 83 % Percentage Met Benchmark: 104 Seated assessed: 79 Seated met benchmark: 76 % Percentage seated met benchmark: 51 Online assessed: 49 Online met benchmark: Percentage of online met benchmark: 96 % Students need to demonstrate knowledge and understanding through a Method: written examination (cultural section). Administration in class with instructor scoring was determined to be a better method for the scoring process and student’s participation. For the online classes, students need to demonstrate knowledge and understanding through a written examination (cultural section) as well. For face to face classes: Written examination with 10 short answers (fill Tool: in the blank) based on authentic cultural readings. Word bank is provided (some words will not be used).The total points for the assessment tool is 10 points. Each answer weights one point. The assessment tool is from the test bank of the Spanish textbook students are using for a traditional class. For online classes: Written examination with 10 short answers (multiplechoice) based on authentic cultural readings. The total points for the assessment tool is 10 points. Each answer weights one point. Our faculty was not so surprised by the results of face-to -face classes Faculty analysis: because we changed some questions. We believe that students needed a little more challenge. But for 2014-2015 we met the Cultural Awareness goal. Face-to-face: Students demonstrated knowledge of cultural points by Strengths: answering the questions correctly content wise. Online: Students demonstrated knowledge of cultural points by answering the questions correctly content wise. Overall: Students demonstrated knowledge of cultural points by answering the questions correctly content wise. We believe in both method of instruction, students are gaining more cultural awareness based on the integration of more cultural activities. Face-to-face: Students did not score higher on the examination because Weaknesses: they did not demonstrate enough reading comprehension and understanding skills. This means that the percentage of students who knew enough cultural points to meet the benchmark was higher than the reported 70%. 28 Learn: Compare: Strategies: Online: Students did not score higher on the examination because they did not demonstrate enough reading comprehension and understanding skills. This means that the percentage of students who knew enough cultural points to meet the benchmark was higher than the reported 70%. Overall: Students did not score higher on the examination because they did not demonstrate enough reading comprehension and understanding skills. This means that the percentage of students who knew enough cultural points to meet the benchmark was higher than the reported 70%. Face-to-face: We were pleased to note the Cultural Awareness Goal was met. We learned that our students did not show their improvement of gaining awareness of other cultures compared to last year in term of percentage. But in terms of knowledge we learned that our students are gaining more awareness of other cultures. Therefore, we can safely assume that the cultural component of our courses is adequate to achieve desired student learning outcomes. It was also noted that online classes selected were taught by full time faculty. Online: We learned that our students are showing the same progress on gaining awareness of other cultures. Therefore, we can safely assume that the cultural component of our courses is adequate to achieve desired student learning outcomes. Overall: We learned that our students are gaining significant awareness of other cultures. It was noted online students are performing somewhat better than students seated in class. Therefore, we can safely assume that the cultural component of our courses is adequate to achieve desired student learning outcomes. Face-to-face: Integrating the cultural section on each test. Changing some questions with a little more difficulty the percentage of students who met the goal has decreased somewhat from last year. Online: Students continued to examine cultural/linguistic to be based on online tasks and which are interactive, meaning that students should aim at capturing the contextual and culturally embedded mediated nature of target language. The percentage of students who met the goal has slightly improved from last year. Overall: Integrating more cultural activities and the cultural section on each test. The percentage of students who met the goal has decreased somewhat from last year due to the changing of questions with a little more difficulty in face-to-face classes. For distance classes the percentage of students who met the goal has slightly improved from last year. Face-to-face: The faculty members will be asked to provide more activities on areas of culture which were less known by students participating in this assessment. We will integrate more visual material. To emphasize on pre reading and post reading activities, Training of new faculty members will emphasize these cultural aspects, as well. Online: The faculty members will be asked to assign more online activities on areas of culture which were less known by students participating in this assessment. We will integrate more visual material. Training of new faculty members will emphasize these cultural aspects, as well. Overall: The faculty members will be asked to reinforce pre and post reading activities and to provide more activities on areas of culture which were less known by students participating in this assessment tool. The primary advantage of using either (or both) strategies is that they actively involve students in what they are reading and studying, enhancing both comprehension and appreciation of what is being read. 29 Training of new faculty members will emphasize these cultural aspects, as well. We believe that in both methods of instruction, students are gaining more cultural awareness based on integration of more cultural activities offered in our courses. Overall 85% of students met the Cultural Awareness benchmark 84% of seated students met the Cultural Awareness benchmark 86% of online students met the Cultural Awareness benchmark The College Goal for Cultural Awareness Was Met. 30 General Education Goal Five: Social and Behavioral Social Sciences Students will demonstrate an understanding of the influence of the individual on group behavior and, conversely, the influence of the group on the individual. Objective: Seventy percent (70%) of students will score: 12 or better on the history essay and 2 or better on the sociology review question. The Behavioral and Social Sciences goal is offered in a large number of history, political science, sociology, psychology, geography, anthropology and economics courses. Students may choose from an array of courses in each area. For transfer requirements, students must choose a history class (HIS 131, 132, 111, or 112). Students then choose 3 electives from discipline areas. Through an analysis of enrollment trends, it was found that the majority of students select HIS 131 (American History I) and SOC 210 (Introduction to Sociology) to fulfill their social science course requirements. PSY 150 (General Psychology) also captures a large number of students; see the Critical Thinking section of this report for the assessment in PSY 150. Enrollment in Behavioral and Social Science courses is substantial. Enrollment in Fall 2014 was as follows: For HIS 131 and SOC 210 Term Fall 2014 Fall 2014 Number of Sections 35 50 Number Enrolled 889 1,386 A. HIS 111 – World Civilizations I Reporting for General Education Results Reporting Year 2014-2015 Wells Last Name: Social/Behavioral Sciences Department: Social/Behavioral Sciences Goal Measured: HIS 111 Course(s) in which assessment took place: 75 % Benchmark percentage: 12 Benchmark minimum: 20 Benchmark maximum: 194 Assessed: 177 Met benchmark: 91 % Percentage Met Benchmark: 159 Seated assessed: 146 Seated met benchmark: 92 % Percentage seated met benchmark: 35 Online assessed: 31 Online met benchmark: 89 % Percentage of online met 31 Course & Number HIS 111 SOC 210 benchmark: Method: Tool: Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: Other: (HIS 111) - Students were given an essay exam in a testing situation. Instructors noted the testing situation for each section reviewed. Instructors were allowed to choose from (10) pre-approved prompts that reflected the goals of Behavioral & Social Sciences. TBA - Faculty has not met to discuss. TBA - Faculty has not met to discuss. TBA - Faculty has not met to discuss. TBA - Faculty has not met to discuss. TBA - Faculty has not met to discuss. TBA - Faculty has not met to discuss. Since our Assessment results came in late, we are still meeting about the outcomes and discussing the results. B. SOC 210 – Introduction to Sociology Reporting for General Education Results Reporting Year 2014-2015 Felton Last Name: Social/Behavioral Sciences Department: Social/Behavioral Sciences Goal Measured: SOC 210 Course(s) in which assessment took place: 70 % Benchmark percentage: 2 Benchmark minimum: 5 Benchmark maximum: 164 Assessed: 143 Met benchmark: 87 % Percentage Met Benchmark: 22 Seated assessed: 20 Seated met benchmark: 91 % Percentage seated met benchmark: 142 Online assessed: 123 Online met benchmark: Percentage of online met benchmark: 87 % Rubric Question Method: Rubric with three points. Students are to respond to questions in a short Tool: answer essay format Seated class scores were higher than non-seated classes. The majority of Faculty analysis: classes selected were on line classes, In the on line classes the assessment was given as an assignment. The class that had the highest scores was the result of the instructor assigning the assessment as a paper assignment worth 20% of their final grade. One instructor included the assessment as a test question. Student scores were lower than those who completed it as a separate assignment. The students who scored two and above had a better understanding of Strengths: what was being asked and contained reflection of their own personal experiences. Some good examples were provided and related to sociological terms, concepts and definitions. Online student responses were better than last year more in depth. For both on line and the seated classes the writing has improved. Most of Weaknesses: the responses were submitted in an essay format. Another observation: 32 Learn: Compare: Strategies: students focused more on how society impacts the individual rather than giving examples of the relationship between the individual and society. Faculty also expressed in their teaching of this information that more time should be spent on showing students how the individual relates to society. When assessments are required as part of an exam results are better. Faculty need to emphasis through examples as to how the individual impacts society. By including the assessment as a part of the syllabus the expectations are laid out and students know what to expect. Instructors are now aware of the importance of focusing on the assessment throughout the semester. They have also agreed that when given as an assignment. It allows students the time to prepare more thoughtful responses to the assessment. Faculty will continue to access the best way of administering the assessment to students. Having it completed as an assignment has shown an increase in student scores Overall 89% of students met the Behavioral & Social Sciences benchmark 92% of seated students met the Behavioral & Social Sciences benchmark 87% of online students met the Behavioral & Social Sciences benchmark The College Goal for Behavioral & Social Sciences Was Met. 33 General Education Goal Six: Natural Sciences Goal: Students will demonstrate knowledge of the scientific method, the central tool for all scientific endeavors. This goal was measured in BIO 110, the science class with the largest enrollment. Enrollment for Fall 2014 was as follows: Term Fall 2014 Fall 2014 Fall 2014 Fall 2014 Number of Sections 22 3 20 5 Number Enrolled 513 70 475 119 Course & Number BIO 110 CHM 131 CHM 151 PHY 110 Reporting for General Education Results Reporting Year 2014-2015 Spring Last Name: Natural Science Department: Natural Sciences Goal Measured: BIO 110 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 170 Assessed: 160 Met benchmark: 94 % Percentage Met Benchmark: 82 Seated assessed: 75 Seated met benchmark: 91 % Percentage seated met benchmark: 88 Online assessed: 85 Online met benchmark: 97 % Percentage of online met benchmark: A scenario was given to students that required them to design an Method: experiment using the Scientific Method. In using a "story problem" scenario, students are encouraged to 'role play' to help them answer the questions of this 10 question multiple choice test. The seated sections were given a paper/pencil test. 3 of the 4 online sections were given the same paper/pencil test during their lab classroom time. One online section was given the same test to complete online. A 10 question multiple choice test was given to all students using the Tool: same 'story problem' scenario. Students were required to only recognize the steps of the Scientific Faculty analysis: Method. This does not allow students to defend their thought process as they designed their experiment. Multiple choice does not allow for partial credit of knowledge or understanding. However, requiring students to write out an answer may cause more stress for students. The tendency for some students would be to give up before they even started. Biology faculty were pleased with student results. Students were well prepared to identify steps of the Scientific Method and Strengths: 34 did quite well in designing their experiment. There was little difference between the seated students (91.5% passing) and the online students (96.6%). Weaknesses: Learn: Compare: Strategies: Other: Faculty had adjusted the vocabulary on this new version of the assessment and found students did better. Faculty are not all convinced that the administering of the assessment gives true and accurate results. Some of the increase in scores could be due to varied student science backgrounds and less variation in dates when students were assessed. The adjustment in vocabulary has helped in this year's assessment. The overall passing increased significantly (from 79.7% in 2013 to 94% in 2014). There was more increase in scoring in the seated classes. The increase seen for seated students: 75.7% in 2013 to 91.46% in 2014. Online students continued to have better scores. Their increase was less significant: 91.8% in 2013 to 96.59% in 2014. The number of problematic questions reduced from 3 to 2. Discussion will continue in administering the assessment so that all students have equal time and access to materials. Faculty will continue to work with modeling and teaching scientific thinking. Faculty will be encouraged to plan on giving the assessment at the end of the semester. Another possibility for the much improved scores this year may be due to a different student population this year, different faculty assessing the sections and the amount of attention placed on practicing scientific thinking. Reporting for General Education Results Reporting Year 2014-2015 Spring Last Name: Natural Science Department: Natural Sciences Goal Measured: CHM 131, CHM 151 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 242 Assessed: 215 Met benchmark: 89 % Percentage Met Benchmark: 193 Seated assessed: 167 Seated met benchmark: 87 % Percentage seated met benchmark: 49 Online assessed: 48 Online met benchmark: 98 % Percentage of online met benchmark: A scenario was given to students that required them to design an Method: experiment using the Scientific Method. In using a 'story problem' scenario, students are encouraged to 'role play' to help them answer the questions of this 10 question multiple choice test. A 10 question multiple choice test was given in a 'story problem' scenario Tool: to all students (online and seated). 35 Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: Students had only to recognize the steps of the Scientific Method. This does not allow students to defend their thought process as they designed their experiment. Multiple choice does not allow for partial credit of knowledge or understanding. However, requiring students to write out an answer may cause more stress for students. The tendency for some students would be to give up before they even started. There were a couple of questions that gave students some problem, but faculty felt that they were valid questions. Faculty were pleased with all student results. Seated students did well (86.53%). Online students scored even higher (97.96%). This variance may be due to online delivery allowing students to "Google" answers. The questions that were problematic for students were considered valid and good questions. It is hoped that students will build their skills as time progresses. Students did well on the assessment. They are learning to think scientifically and be able to identify the Scientific Method. The same questions were problematic in both years but faculty feel that these are valid questions and are not ready to make adjustments to the questions. The scores from last year to this year are almost identical. Discussion will continue to devise a way to deliver assessments for all students (seated and online) in the same online manner and/or find a way to prohibit online students from being able to "Google" answers. Reporting for General Education Results Reporting Year 2014-2015 Spring Last Name: Natural Science Department: Natural Sciences Goal Measured: PHY 110 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 86 Assessed: 80 Met benchmark: 93 % Percentage Met Benchmark: 86 Seated assessed: 80 Seated met benchmark: 93 % Percentage seated met benchmark: A scenario was given to students that required them to design an Method: experiment using the Scientific Method. In using a "story problem" scenario, students are encouraged to "role play" to help them answer the questions of this 10 question multiple choice test. A 10 question multiple choice test was used. This assessment contained a Tool: scenario that required students to design an experiment using the Scientific Method. There were no online students assessed this semester. Faculty were pleased with student results. Several questions were Faculty analysis: rewritten for this year's assessment. Students scored better. Students were able to identify the parts of the Scientific Method. (no Strengths: online students assessed) There were 2 questions that students found problematic that were basic to Weaknesses: Scientific Method. Some of the possible reasons for seeing these better scores and student Learn: 36 Compare: Strategies: understanding could be: different student population, more attention was placed on Scientific thinking, and changing of wording and questions. ( no online students assessed) Faculty had rewritten several questions for this year's assessment. The number of problematic questions reduced from 4 to 2. There was significant increase from last year's scores: 85% in 2013 compared to 93% in 2014(no online students assessed). Faculty will continue to work with students on the scientific thinking process. Overall 91% of students met the Natural Sciences benchmark 89% of seated students met the Natural Sciences benchmark 97% of online students met the Natural Sciences benchmark The College Goal for Natural Sciences Was Met. 37 General Education Goal Seven: Humanities/Fine Arts Goal: Students will demonstrate knowledge of the humanities and critical skills in assessing cultural/artistic merit and significance. Students may select from a range of courses for Humanities/Fine Arts requirements including Art, Music, Drama, Literature and Humanities. Through an analysis of enrollment trends, it was found that the majority of students select ART 111 (Art Appreciation), MUS 110 (Music Appreciation) and HUM 130 (Myth in Human Culture). Objective: At least 70% of students will score a 70% or higher on ART111; 70% will score 70% or higher on MUS 110; 70% will score 3 of 5 on HUM 130 assessment. Enrollments for the Fall 2014 were as follows: Term Fall 2014 Fall 2014 Fall 2014 Number of Sections 28 43 8 Number Enrolled 714 1,235 288 Course & Number ART 111 MUS 110 HUM 130 A. ART 111 – Art Appreciation Reporting for General Education Results Reporting Year 2014-2015 Jacobs Last Name: Art Department: Humanities/Fine Arts Goal Measured: ART 111 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 107 Assessed: 104 Met benchmark: 97 % Percentage Met Benchmark: 57 Seated assessed: 55 Seated met benchmark: 96 % Percentage seated met benchmark: 50 Online assessed: 49 Online met benchmark: 98 % Percentage of online met benchmark: Students completed a Virtual Exhibit via PowerPoint or Word, including Method: a thematic statement + 10 images. There were no differences between seated and online. Students were provided the following rubric: 1. Identify a thoughtful Tool: theme for your exhibit. The ideas for themes of the exhibit are limitless. Use your own interests and creativity to find a possible theme for your show. 1. Research and identify at least 10 works of art for your exhibit. You 38 Faculty analysis: Strengths: Weaknesses: Learn: Compare: Strategies: Other: must include artwork by at least 2 different artists and 2 different media in your exhibit. Use the internet, museum websites, your book, books in the library, or galleries. The 10 artworks do not have to be artworks that you have seen in person. For example, if you want to include the “Mona Lisa” in an exhibit about portraits, the Louvre will graciously lend you this priceless work of art. 1. For each of the 10 artworks, provide the following in label format: (40 points total). These can be provided with the images or in a separate numbered list. Title of artwork (1 point) Artist (1 point) Date of artwork (1 point) Media (1 point) 1. Write a catalogue essay that explains your choice in theme and why you picked these 10 artworks to go together. Essay should be at least 500 words, nicely organized, and in complete and correct sentences. You must investigate the WHY of putting together these images. Is there a connection between a certain formal element? Do they all use light in a certain way, use the same color, have the same style, etc.? How do your images explore your theme? What can the visitor to your exhibit learn or take away from seeing the artworks? (60 points total) For essay scoring purposes: - Introduction paragraph (5 points) - Three to four body paragraphs making the argument for your theme including information on your choices (15 points) - Referring specifically to at least 4 artworks in your exhibit with supporting details and explanations of how these works relate to your theme (20 points) - Using correct grammar, punctuation, sentence structure, no misspellings, etc. (5 points) - Bibliography of at least 2 sources other than your book. (5 points) - Creativity of overall theme and presentation (10 points) This assessment is a valuable tool in perceiving how much cultural awareness students can convey. Performance is consistent in both online and face-to-face classes. When students can relate cultural studies to their own lives, they are far more engaged in the project. Students have improved regarding citation, though this continues to be an issue in some areas. Reinforces the idea that when students can relate cultural studies to their own lives, they are far more engaged in the project. N/A TBD We are concerned that the majority of classes are taught by part-time instructors, and the data has been very difficult to collect at times. Two courses did not report at all, and one course gave incomplete information. A fourth course from the list was cancelled. We need strategies to encourage instructors to comply.... 39 B. MUS 110 – Music Appreciation Reporting for General Education Results Reporting Year 2014-2015 Maurer Last Name: Music Department: Humanities/Fine Arts Goal Measured: MUS 110 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 178 Assessed: 113 Met benchmark: 63 % Percentage Met Benchmark: 154 Seated assessed: 94 Seated met benchmark: 61 % Percentage seated met benchmark: 24 Online assessed: 19 Online met benchmark: 79 % Percentage of online met benchmark: 10 question multiple choice test, with questions chosen by faculty to show Method: basic knowledge of musical terms, styles and historical significance of music. Assessment administered at the end of the semester in both faceto-face and on-line settings. 10 question multiple choice test. Tool: The assessment is seen as appropriate for both on-line and traditional Faculty analysis: classroom sections as well as appropriate for multiple instructors with different teaching styles and content emphasis. In both the on-line and traditional classes students scored best on material Strengths: from the latter half of the course; including material about more wellknown composers. Faculty believe that this material is freshest and contains at least some material that is in the general knowledge of some of our students. Again this year, on-line classes scored better with 79.2% meeting the goal in contrast to 53.24% of face-to-face classes. This is the first year the goal has not been met that I know of. Many students still had more difficulty with material presented at the Weaknesses: beginning of the semester and earlier periods in music history. Faculty discussed the possibility that since the assessment is cumulative, students have forgotten earlier material. Faculty believe that the setting in which the student takes the assessment can influence scores as on-line students have access to the text. Retention of material continues to be an issue. Learn: Faculty discussed the fact that the goal was not met this year and Compare: suggested several possible reasons. We have had an increase in the number of part-time instructors who have never administered the assessment and perhaps did not stress the definitions used in the assessment. Also, it could be a fluke since this is the lowest score in many years. Faculty were encouraged to continue referencing musical definitions Strategies: throughout the semester. We were able to get results from all but one instructor this year, a marked Other: improvement from the five missing sections last year. 40 C. HUM 130 - Myth in Human Culture Reporting for General Education Results Reporting Year 2014-2015 Di Donato Last Name: English, Reading & Humanities Department: Humanities/Fine Arts Goal Measured: HUM 130 Course(s) in which assessment took place: 70 % Benchmark percentage: 3 Benchmark minimum: 5 Benchmark maximum: 168 Assessed: 143 Met benchmark: 85 % Percentage Met Benchmark: 24 Seated assessed: 18 Seated met benchmark: 75 % Percentage seated met benchmark: 144 Online assessed: 125 Online met benchmark: Percentage of online met benchmark: 87 % Essay Question. No differences between traditional and online. Method: In order to measure objectives and student outcomes, students were Tool: asked to respond to the following question: The term “myth” comes from the Greek word mythos, which means “story.” We commonly perceive myths to be “untrue”; however, a myth performs many vital functions in a society that believes it, and for that society the myth contains “truth.” During this semester we have discussed various theories of how a myth functions in a society: including the natural, etiological, cosmological, psychological, sociological, linguistic, mystical, and pedagogical. We have discussed most of these functions, but not all of them. First, pick a story that you really enjoyed this semester. Analyze it as to how it probably functioned in the society that believed it was true. Discuss this function, and show how this myth contains this function. Second, analyze your myth in terms of the values it contains for the society that believed it. Discuss at least one value at length. For Example: Let’s say I enjoyed the myth of Demeter and Persephone and how it relates to the natural world. I can then discuss how this myth functions. Obviously, the best function is the Nature-Myth, also called the natural function, which explains some aspect of the natural world. I will discuss how the DemeterPersephone myth explains the changing of the seasons. As for the values the story contains, I can discuss what it tells the society about death, and I can explain how the story reflects the marriage customs of the ancient Greeks, where the father selects the husband for his daughter. You will write a thoughtful paper in which you explore both the function and values of the story. Do not simply retell the story, but do use examples from the story to support your point. You can include research in your paper, but you are not required to have it. This paper is designed to show me what you have learned in class by analyzing a myth. 41 Faculty analysis: Strengths: Weaknesses: Grading Rubric: 5 – the student’s response clearly describes a function of a myth in a particular culture. The response further provides an example of a specific myth that shows a clear understanding of a “truth” for a culture. The response is clearly organized and well written. 4 – the student’s response describes a function of a myth in a particular culture. The response further provides an example of a specific myth that shows some understanding of a “truth” for a culture. The response is clearly organized and well written 3 – the student’s response describes a function of a myth in a particular culture. The response further provides an example of a specific myth that shows a minimal understanding of a “truth” for a culture. The response may be poorly organized and poorly written 2 – the student’s response inadequately describes a function of a myth in a particular culture. The response may provide an example but shows a poor understanding of the “truth” of a particular culture. The response is poorly organized and poorly written. 1 – the student’s response does not describe a function of a myth in a particular culture. The example, if provided, does not show an understanding of the “truth” for a culture. The response is poorly organized and poorly written. The main drawback is the amount of work it takes to grade the essays, many of which end up being 2 to 3 page papers. On the positive side it allows us to see how the student thinks through the process of analyzing the functions of myth. The Humanities Faculty continues to view the current assessment question favorably with respect to communicating department expectations of the students. Feedback has indicated strong approval for the tool’s design, and that the faculty is pleased with its ability to address the essential relationships between myth and culture. The faculty believes the assessment tool is equally well suited for both traditional and online testing. Negatively, the assessment tool is not as precise as could be, which is why the department uses a grading procedure that requires multiple grades for each essay. As a result, the grading procedure is rather time intensive. We have discussed the possibility of adjusting this. Online: This year the online student performance was stronger than every preceding year, and surpassed the student results in the traditional class assessed. Overall: Most students showed a strong understanding of the cultural issues addressed by the myths discussed over the course of the semester. Even students who failed to communicate a clear understanding of a function of myth were able to identify some culturally significant elements in the traditional stories, particularly a general understanding of sociological and cosmological function. This year, as was the case last year, we observed that more students were grasping the pedagogical function of myth. Online: Online performance surpassed the results of past years. Overall: As in the past, the most prominent weakness evident in the assessments was student inability to organize and communicate their thoughts. Though a fair understanding of the issue in question could be gathered 42 Learn: Compare: Strategies: from many essays, it was often difficult to identify in a single reading. This problem can be the result of a lack of clarity in thinking through the material, but is more likely due to student ability in the area of composition. Students seem able to think though the essay with the help of the worksheet we developed to aid them in preparation. This was implemented last year, and the overall results have been consistent. The improvement we saw last year with the implementation of the worksheet seems to have been duplicated this year. This may speak to the effectiveness of the new tool. We are continuing the use of the worksheet tool which helps walk students through the essay writing process specific to our course goals. Overall 79% of students met the Fine Arts benchmark 71% of seated students met the Fine Arts benchmark 89% of online students met the Fine Arts benchmark The College Goal for Fine Arts Was Met. 43 General Education Goal Eight: Information Literacy Goal: Students will effectively use research techniques to identify, select, use, document and evaluate information sources appropriate to a particular need. Students were selected from ENG 112 courses in order to assess the Information Literacy goal. Objective: At least 70% of students will score 70% or higher on the assignment where students have to evaluate two of their sources for their research paper. Enrollments for fall 2014 were as follows: Term Number of Sections Fall 2014 54 Number Enrolled 1,268 Course & Number ENG 112 Reporting for General Education Results Reporting Year 2014-2015 Payton Last Name: Library Department: Information Literacy Goal Measured: ENG 112 Course(s) in which assessment took place: 70 % Benchmark percentage: 70 Benchmark minimum: 100 Benchmark maximum: 135 Assessed: 110 Met benchmark: 81 % Percentage Met Benchmark: 99 Seated assessed: 82 Seated met benchmark: 83 % Percentage seated met benchmark: 36 Online assessed: 28 Online met benchmark: Percentage of online met benchmark: 78 % Evaluation worksheet of two sources supposedly used in a research Method: paper and those worksheets were graded with a rubric. Students were given two worksheets to fill out that to evaluate two Tool: sources that were attached to a writing assignment (research paper). One was an evaluation checklist, and one was a further evaluation of one of their sources asking to cite the source in MLA, give their thesis statement, speak to the author's credentials, and evaluate why they chose that source. Librarians believed the tool to be an ineffective measure of a student's Faculty analysis: information literacy skills. Every instructor has a different research assignment, making consistency challenging, if not impossible. This assignment is an "add on" assignment, so the perception is it's less important to the students than their other assignments. It's facilitated and graded by librarians, who are not in the classroom. And the worksheet itself simplifies the information literacy/research process (students check off boxes) and does not accurately indicate learning on the subject. Nothing that could be stated across the board. Strengths were all over the Strengths: map. 44 Weaknesses: Learn: Compare: Strategies: Other: Evaluation (students often didn't think much about their evaluation criteria) and creating effective thesis statements. That an add-on assignment not truly PART of a writing assignment is not an effective assessment. If the assignments they're based on are not consistent, then the assessment isn't consistent and therefore has less meaning. ENG 112 is still a natural fit for information literacy skills, but without more consistency in writing assignments (which happened last year as well), it's challenging to get good data. Last year's having students use the CRAAP method actually seemed like a better assessment tool--more consistency. Work with the ENG 112 faculty to try to scaffold research into a master writing assignment in Fall 2015 for the new Writing Across the Disciplines course. It might be time to move to evaluating IL either a) not at all, or b) in COM 110/231 as they all have the same assignment and the research portion is integrated into that assignment (and has been for years). The College Goal for Information Literacy Was Met. 45 Data for each assessment is available and documented in the Planning and Research Department. 46