2013-2014 Annual Program Assessment Report Please submit report to your department chair or program coordinator, the Associate Dean of your College and the assessment office by Tuesday, September 30, 2014. You may submit a separate report for each program, which conducted assessment activities. College: MCAMC Department: Art Program: B.A. in Art Assessment liaison: Magdy Rizk 1. Overview of Annual Assessment Project(s). Provide a brief overview of this year’s assessment plan and process. Assessment for the year 2013-14 was planned and implemented by the department liaison, assessment committee and a professor who coordinates the gateway sections to implement our pilot assessment with gateway classes. For 2013-2014, the Art Department closed the loop on Project 1, re-scored and re-analyzed results for Project 2 (an ongoing assessment started in 2012-13; see detailed descriptions of both projects below). Our SLOs 1 (Basic skills) and 3 (Critical thinking) contain several measurable components within each SLO, therefore the following projects focused on portions of each. a. Project 1 (a gateway-capstone test using the first half of our SLOs 1 & 3) was re-implemented to capstone students in Spring 2013. The results of this second implementation provided valid results, which were then utilized in closing the loop on the project. Closing the loop consisted of meeting with foundation-level course instructors and presenting several possible solutions to the deficits found within the results of the above assessment. Detailed Description of Project 1 The final scores and analysis from this assessment resulted in a closing the loop session with Art 140 instructors during the Spring 2014 semester. The Assessment committee presented some sample solutions including writing assignments analyzing for selected elements of art and principles of design, and connecting those to constructing meaning in an artwork. Both of those 1 items relate to basic skills and specific aspects of critical thinking including identification of concepts and problem solving. In addition, such a writing assignment would address the need for improvement discovered through the writing component of the assessment test. An online repository has also been created for all Art 140 instructors for the purpose of sharing course syllabi and assignments that pertain to these topics. Finally, the members of the assessment committee and the liaison will be meeting with Foundation faculty annually to talk about updates and results. b. Project 2 (Gateway students, Surreal assignment) was a pilot of a new assessment project using an existing assignment in the gateway courses, focusing on the second half of our SLOs 1 & 3.) We collected student work at the end of Spring 2014, normed, scored and analyzed twice during the Spring of 2014, revised the rubric and scoring criteria, then re scored and analyzed for a final set of valid results. Detailed Description of Project 2 The Surreal assignment was designed to assess basic art skills and critical thinking through art production, specifically identification of concepts and problem solving. The assignment was planned with the committee and participating faculty in the Fall of 2013. The final version of the assignment was distributed among all sections of Art 200, and data was collected during Spring of 2014. As mentioned above, the committee normed and re-normed the rubric, however wide disparities in scoring showed a lack of consensus on various aspects of the rubric criteria. As a result, the rubric was revised and a new norming session was held. The scores were closer in range, but still appeared too variable for analysis. The committee decided that the criteria for scoring was too complex and the concepts and skills covered were too broad. The committee therefore narrowed down the rubric criteria to focus on specific aspects of composition, characteristics of surrealism, and simpler writing criteria. The norming and scoring were done as a group. A consensus was easier to attain and data was easily captured and processed. 2. Assessment Buy-In. Describe how your chair and faculty were involved in assessment related activities. Did department meetings include discussion of student learning assessment in a manner that included the department faculty as a whole? 2 The chair and faculty were kept current on the committee’s activities through monthly department meetings. The liaison was in periodic contact with all faculty members on the assessment committee, participating assessment project faculty and the general full and part-time faculty group, to keep everyone apprised and on target with the assessment schedule. The liaison presented the results of PROJECT 1 to all faculty at a monthly department meeting. Discussion on the results and possible solutions followed. The faculty appeared engaged and interested, especially with the presence of hard data showing the need for improvement. Re PROJECT 2, the chair and associate chair were kept informed about results of closing the loop activities with Foundation faculty. Participants felt that the meeting with the instructors was both crucial and effective in setting a precedent for ongoing conversations about instructional practices and student learning outcomes for the department. Again, per positive feedback, assessment committee members will be conducting ongoing meetings with faculty on either an annual or semester basis. 3. Student Learning Outcome Assessment Project. Answer items a-f for each SLO assessed this year. If you assessed an additional SLO, copy and paste items a-f below, BEFORE you answer them here, to provide additional reporting space. 3a. Which Student Learning Outcome was measured this year? We assessed portions of the following SLOs (see bolded blue text below for specific portions): SLO 1 - PROJECT 1 was writing; PROJECT 2 was writing and art making SLO 3 – PROJECT 1 looked at analyzing and interpreting; PROJECT 2 looked at analyzing, interpreting and problem solving. PROJECT 1 SLO 1: BASIC SKILLS Developing a foundation of art knowledge, theories, skills, craftsmanship and technologies, where ideas and concepts are communicated in writing, speaking and art making. 3 SLO 3: CRITICAL THINKING Analyzing, interpreting, and questioning traditional methodologies and pre-conceived notions of art and art making through the process of generating and solving problems. PROJECT 2 SLO 1: BASIC SKILLS Developing a foundation of art knowledge, theories, skills, craftsmanship and technologies, where ideas and concepts are communicated in writing, speaking and art making. SLO 3: CRITICAL THINKING Analyzing, interpreting, and questioning traditional methodologies and pre-conceived notions of art and art making through the process of generating and solving problems. 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply) Critical Thinking Written Communication PROJECT 1 Critical Thinking: Students were asked to do a formal analysis of an artwork by using terms related to the elements of art and principles of design, followed by a written explanation of how they are used in artwork. Students were also asked how these principles and elements are used to communicate a message, concept or idea conveyed within the artwork. Written Communication: Students were asked to write a clear explanation of each of the above items using applicable art vocabulary. PROJECT 2 Critical Thinking: Students were asked to create an artwork using digital software that focused on concepts and visual characteristics found in the genre of Surrealism. Students then were instructed to explain how those concepts and characteristics 4 were formally used to compose their own artwork. Students were also asked to verbally describe how they used these concepts and characteristics to communicate the message or idea conveyed within their artwork. Written Communication: Students were asked to write a clear explanation of each of the above items using applicable art vocabulary. 3c. What direct and/or indirect instrument(s) were used to measure this SLO? The following is a description of all components of Project 2: Gateway Students, Surreal Assignment, including rubric, results and analysis of scores. _______________________________________________________________________________________________________ PROJECT 2: GATEWAY STUDENTS, SURREAL ASSIGNMENT Image Assignment: Using rights free photos (from morguefile.com or your own photos, etc.), create a 1050 x 750 pixel image that use the following strategies: 1. Dramatic aberrations in scale. 2. An ordinary item or items combined and juxtaposed in unusual new ways to convey a fragmented reality. 3. The idea of an “inner world” by incorporating interior spaces, frames, boxes, etc. 4. A floating image or images to suggest that in the inner realms, physics no longer applies. 5. An image that is a transposition of one quality to another—for example, Magritte's bird made of clouds. Written Assignment: Using the language and concepts of the above strategies identified in numbers 1-5 above, write a paragraph that explains how your image falls within the surrealist genre and describe the overall concept or meaning of your art piece. 5 SAMPLE STUDENT WORK: Artwork and Written Explanation 6 RUBRIC - PROJECT 2: GATEWAY STUDENTS, SURREAL ASSIGNMENT Scoring Scale (1-5) Composition: Unity, Balance, Positive and Negative Shapes/Space and Focal Point. Concept: Visual depiction of a surrealist idea: 1. Scale 2. Juxtaposition 3. Inner world/Interior 4. Floating Images 5. Transposition of Qualities Written Analysis: Describe and explanation of surreal assignment strategies EXCEPTIONAL – 5 Completed work is perfectly unified and balanced with exceptionally strong use of positive and negative shapes/space and an exceptionally clear focal point. Completed work displays 5 out of the 5 concepts in an exceptionally clear manner. Completed writing describes and explains 5 out of the 5 concepts in an exceptionally clear analysis. SKILLED – 4 Completed work is well unified and balanced with strong use of positive and negative shapes/space and a clear focal point. Completed work displays 4 out of the 5 concepts in a clear manner. Completed writing describes and explains 4 out of the 5 concepts in a clear analysis. PROFICIENT - 3 Completed work is mostly unified and balanced with a sufficient use of positive and negative shapes/space and a focal point is present. Completed work displays 3 out of the 5 concepts in sufficiently clear manner. Completed writing describes and explains 3 out of the 5 concepts in a sufficiently clear analysis DEVELOPING - 2 Completed work is somewhat unified balanced with some use of positive and negative shapes/space and an unclear focal point. Completed work displays 2 out of the 5 concepts in a somewhat clear manner. Completed writing describes and explains 2 out of the 5 concepts in a somewhat clear analysis INADEQUATE - 1 Completed work is not unified or balanced with a no use of positive and negative shapes/space and no focal point is present. Completed work displays 1or no concepts out of the 5 concepts. Completed writing describes and/or explains 1 or none out of the 5 concepts. 7 3d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. PROJECT 2– PROJECT 2: GATEWAY STUDENTS, SURREAL ASSIGNMENT This assessment was a cross-sectional comparison. Students from all sections of the gateway and capstone were to be tested using a similar instrument. The instrument, an art production assignment, was already-existing assignment in the gateway Art 200, and was modified slightly for assessment purposes by the coordinating instructor, in collaboration with the assessment committee. The assessment committee assessed a sample set utilizing a rubric that was normed twice, however the committee could not come to consensus on benchmarks for the criteria. The rubric was twice revised, with the final version reflecting a narrower scope in the criteria. The latest rubric proved to be effective for norming, scoring, and valid data acquisition. 3e. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. SCORES– PROJECT 2: GATEWAY STUDENTS, SURREAL ASSIGNMENT Note that the committee is made up of 4 members, however one member did not participate in final scoring. Results: Averages (based on the rubric shown above) Rubric Scores (Out of 4) Scores: Com Mmbr 1 Com Mmbr 2 Com Mmbr 3 Composition Concept Written Analysis 3.09 2.84 1.27 3.07 2.80 1.32 3.14 2.78 1.29 __________________________________ 3.10 2.81 1.29 TOTALS/AVGS. 8 Assessment Committee Analysis DISCUSSION - PROJECT 2: GATEWAY STUDENTS, SURREAL ASSIGNMENT Student work for this pilot assessment project were collected by the liaison from each section that was given the assignment (4 sections of approximately 30 students each). A sample of 12 projects was disseminated to all assessment committee members for the norming process. After several attempts and two norming sessions, the scores still varied greatly. As mentioned above the committee decided that the findings and scoring were flawed due to the following: variables in instructors’ implementation; knowledge and skills about digital mediums amongst committee members; criteria in the rubric was too broad; and finally, ambiguous language in the rubric. With changes to the rubric, the final copy produced more stable assessment and viable scores. Students reached 78% meeting the SLO for Composition; 70% on the Concept portion; 32% on the written analysis portion. Students appeared to barely meet Basic Skills in terms of Composition and Concept, while Written Analysis fell far below average. Recommendation: This assessment provides a benchmark for beginning junior-level students that shows a clear need to teach components of critical thinking skills throughout the BA, at least s in Foundation Art 140 and mid-program such as Art 307 (the new course replacing Art 200 as gateway). The scores may indicate the need to reinforce this material at mid-program and upper division levels, but variables in implementation and student buy-in also need to be taken into account. The liaison met with Foundation Art 140 instructors in Spring 2014 to ways of implementing this instruction. Using a CSUN Box to collect and store syllabi and course materials from all sections of Art 140, Foundation instructors will now have access to current and appropriate content that will initiate effective instruction of critical thinking and analysis. 3f. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) 9 CLOSING THE LOOP - PROJECT 1: GATEWAY-CAPSTONE TEST As noted above in the overview section, the final scores and analysis from this assessment, conducted the prior year, resulted in a closing-the-loop session with Art 140 instructors during the Spring 2014 semester. The Assessment committee presented some sample solutions including writing assignments analyzing for selected elements of art and principles of design, and connecting those to constructing meaning in an artwork. Both of those items relate to basic skills and specific aspects of critical thinking including identification of concepts and problem solving. In addition, such a writing assignment would address the need for improvement discovered through the writing component of the assessment test. An online repository has also been created for all Art 140 instructors for the purpose of sharing course syllabi and assignments that pertain to these topics. Finally, the members of the assessment committee and the liaison will be meeting with Foundation faculty annually to talk about updates and results. PROJECT 2: GATEWAY STUDENTS, SURREAL ASSIGNMENT Results of this project will be presented to faculty this semester, with emphasis once again on the need to include aspects of critical thinking periodically throughout the BA. On the surface, this assessment confirms the results and curricular changes related to the previous assessment, PROJECT 1. Other ideas based on art production and critical thinking will be solicited during discussion with faculty in a department meeting; specific closing-the-loop activities will be decided upon following faculty discussion. 4. Assessment of Previous Changes: Present documentation that demonstrates how the previous changes in the program resulted in improved student learning. Project 1, Closing the Loop curricular changes: The assessment committee will be collecting student work from Foundation Art 140 assignments that address critical thinking and analyze them for improvement. Project 2: Gateway Students, Surreal Assignment: As mentioned above, curricular changes have not yet been made as a result of this assessment. This assessment yielded results that showed a need to improve students’ ability to analyze and write about basic precepts of art. Specifically, their ability to deconstruct their own artwork formally and conceptually is weak at gateway level. Although it is to be expected that students would need improvement at this level, faculty will be considering additional strategies to raise the bar in both viewing and art production, 10 through foundation courses so that students entering gateway courses may begin upper division with a stronger base of knowledge and skills. 5. Changes to SLOs? Please attach an updated course alignment matrix if any changes were made. (Refer to the Curriculum Alignment Matrix Template, http://www.csun.edu/assessment/forms_guides.html.) No Changes have been made to the department SLOs as of this date. 6. Assessment Plan: Evaluate the effectiveness of your 5 year assessment plan. How well did it inform and guide your assessment work this academic year? What process is used to develop/update the 5 year assessment plan? Please attach an updated 5 year assessment plan for 2013-2018. (Refer to Five Year Planning Template, plan B or C, http://www.csun.edu/assessment/forms_guides.html.) The Department of Art assessment committee will be making changes to the current five-year plan. We determined last year not to proceed to new SLOs as the plan outlined, because our SLOs contain multiple components and we wanted to address both art viewing and art production, two different activities, regarding basic skills and critical thinking. It was speculated that there would be improvement if students were analyzing their own artwork vs. the work of others, but there was not much difference. We will be moving forward to new SLOs this year as we examine the new gateway and a new capstone that is being discussed in the department. 7. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your program? Please provide citation or discuss. No Art Department faculty has published a manuscript about assessment activities. 8. Other information, assessment or reflective activities or processes not captured above. 11 No other activities have been implemented. 12