2013-2014 Annual Program Assessment Report Please submit report to your department chair or program coordinator, the Associate Dean of your College, and to james.solomon@csun.edu, director of assessment and program review, by Tuesday, September 30, 2014. You may submit a separate report for each program which conducted assessment activities. College: Humanities Department: Liberal Studies Program: BA Multiple Subject/SPED Teacher Preparation Assessment liaison: Tineke Scholten 1. Overview of Annual Assessment Project(s). Provide a brief overview of this year’s assessment plan and process. In accordance with a 5-year plan for assessment of its BA Multiple Subject/SPED Teacher Preparation program, the Interdisciplinary Advisory Committee for Liberal Studies (IDC) decided to assess in 2013-14 the extent to which students in this program meet the following two program SLOs: SLO5: Students will be able to successfully adapt their reading and writing to a range of disciplines, genres, media and purposes SLO7: Students will be able to access, evaluate and make use of a range of informational resources (electronic and otherwise). Relevant data in the form of graded student work were collected in two sections of LRS 300 (Gateway Experience for Pre-Credential Students) in the Fall of 2013 and Spring of 2014 and subsequently evaluated. 2. Assessment Buy-In. Describe how your chair and faculty were involved in assessment related activities. Did department meetings include discussion of student learning assessment in a manner that included the department faculty as a whole? Assessment is a regular item on the agenda of the Interdisciplinary Committee. The committee members wish to ensure that Program SLOs are assessed in the most effective manner and committee members as well as teaching faculty have also traditionally been willing to help with data collection and analysis. 1 3. Student Learning Outcome Assessment Project. Answer items a-f for each SLO assessed this year. If you assessed an additional SLO, copy and paste items a-f below, BEFORE you answer them here, to provide additional reporting space. 3a. Which Student Learning Outcome was measured this year? SLO5: Students will be able to successfully adapt their reading and writing to a range of disciplines, genres, media and purposes 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? Reaching these SLOs requires the student to have competencies in the following areas: Critical Thinking Written Communication Quantitative Literacy Information Literacy 3c. What direct and/or indirect instrument(s) were used to measure these SLOs? Data consisted of responses to three embedded assignments that required engagement with different genres of text: An essay in which students were to describe and provide relevant context of an arts event, a rebuttal to a forum posting by a peer who had commented on a work of fiction, and a PowerPoint slide sequence describing background, procedure and results of a science experiment. 3d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Data were collected in one section of LRS 300 in Spring of 2014. The assessment liaison and the course instructor each independently rated students’ work relative to a rubric that defined the subskills of this SLO. To ensure inter-examiner reliability, a norming session preceded the rating. 3e. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. The following table quantifies the combined results of all three writing assignments: 2 Combined Results (n=28-34) Choice of material: The student’s choice of information and the amount and type of detail included reveal an excellent understanding of the purpose of the assignment and the needs of the intended audience. Development and organization of ideas: The student develops his/her ideas in a clear and cohesive fashion. The reader can easily follow the writer’s line of thinking, understanding what the main ideas or claims are and the supporting arguments. Use of organizing elements of the genre: The student makes excellent use of organizational elements that are useful for the genre of writing (for instance, paragraph structure, use of subheadings, numbering, fonts, use of bullet points or other visual aids to effectively state the relationship between various info on PPT slides, etc. Style: The style of writing (register: level of formality) is appropriate for the rhetorical context and audience. Writing Mechanics: The student’s writing demonstrates proficiency in the use of Standard Written English as required by the genre in terms of sentence structure, vocabulary, spelling and punctuation. Scale Avg 0-5 4.03 0-5 4.40 0-5 4.44 0-5 0-5 4.98 4.15 Overall, students’ writing was fairly effective and matched the purpose of each assignment. The current emphasis on a diversity of writing assignments appears to be an appropriate and meaningful component of the LRS 300 curriculum. The largest range in quality occurred in the students’ choice of material relative to the assignment at hand. 3f. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) The results were discussed during the September 2014 IDC meeting where the topic was broadened to include the overall efficacy of LRS 300 in its current form. The course instructor involved in this year’s data collection and analysis attended the meeting and reported on her initiatives towards updating LRS 300. 3a. Which Student Learning Outcome was measured this year? SLO7: Students will be able to access, evaluate and make use of a range of informational resources (electronic and otherwise). 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? 3 Reaching these SLOs requires the student to have competencies in the following areas: Critical Thinking Written Communication Quantitative Literacy Information Literacy 3c. What direct and/or indirect instrument(s) were used to measure these SLOs? To evaluate this SLO, annotated bibliographies corresponding to a preapproved thesis statement were rated. 3d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Data were collected in one section of LRS 300 in Fall of 2014. The assessment liaison and the course instructor each independently rated students’ work relative to a rubric that defined the subskills of this SLO. To ensure inter-examiner reliability, a norming session preceded the rating. 3e. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. The table below summarizes the average ratings relative to the rubric items: Scale Avg. Annotated Bibliography (n=22) Reliability of the sources: The student selected credible sources (relative to their intended purpose). Usefulness of the sources: The student selected sources that address the topic of their thesis statement in a meaningful way. Understanding of content: The student provides a thorough and accurate summary of the sources. Evaluation of content: The student explains clearly and insightfully why the selected sources are relevant to the thesis. 4 0-5 0-5 4.23 3.95 0-5 0-5 3.91 3.64 The students exhibited a range of abilities for this assignment. Weaknesses in reading comprehension were revealed in the summaries and evaluation of the relevance of the sources used. Some students also exhibited considerable difficulty in determining the reliability of source material. 3f. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) These results were also discussed during the September 2014 IDC meeting where the topic was broadened to include the overall efficacy of LRS 300 in its current form. The course instructor involved in this year’s data collection and analysis attended the meeting and reported on her initiatives towards updating LRS 300. Specifically in regards to this SLO, she reported now taking her students to the library to receive direct instruction in accessing and evaluating content. 4. Assessment of Previous Changes: Present documentation that demonstrates how the previous changes in the program resulted in improved student learning. Last year’s evaluation of student performance in GEOL 406 has led to the following changes: The instructors of GEOL 406 have redesigned the instructional activities related to vocabulary in science and are making vocabulary issues a more systematic part of the regular course feedback. The misdirected overemphasis on vocabulary in 406 should also be tackled earlier in the LRS program. Instructors are updating the LRS 300 Gateway course and specific attention will be paid to combating the vocabulary overemphasis in the science unit of that course. 5. Changes to SLOs? Please attach an updated course alignment matrix if any changes were made. (Refer to the Curriculum Alignment Matrix Template, http://www.csun.edu/assessment/forms_guides.html.) No changes in SLOs or alignment occurred in 2013-14. 5 6. Assessment Plan: Evaluate the effectiveness of your 5 year assessment plan. How well did it inform and guide your assessment work this academic year? What process is used to develop/update the 5 year assessment plan? Please attach an updated 5 year assessment plan for 2013-2018. (Refer to Five Year Planning Template, plan B or C, http://www.csun.edu/assessment/forms_guides.html.) The Liberal Studies Program’s 5-year assessment plan so far provides a useful template to ensure that the department assesses essential competencies within a 5-year period. Future modifications should be expected as the faculty develops concerns or questions about strengths and weaknesses of the program offering. 7. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your program? Please provide citation or discuss. No 8. Other information, assessment or reflective activities or processes not captured above. 6