Annual Assessment Report to the College 2010-11
College: Humanities
Department:
Program: Liberal Studies
Note: Please submit your report to (1) the director of academic assessment, (2) your department chair or program coordinator and (3) the
Associate Dean of your College by September 30, 2011. You may submit a separate report for each program which conducted assessment
activities.
Liaison: Tineke Scholten
1. Overview of Annual Assessment Project(s)
1a. Assessment Process Overview: Provide a brief overview of the intended plan to assess the program this year. Is assessment under the
oversight of one person or a committee?
LRS is participating in a pilot study run by the “Simplifying Assessment Committee” which intends to streamline assessment work on campus by
providing a central electronic data storage system with added functionality to simplify the rating and longitudinal or cross sectional analysis of
the data. To meet the criteria of the committee, the annual assessment procedure for LRS was to have the following features:
1. Data collection should come from a “Gateway” and “Capstone” course so that there is the opportunity of pre- post- comparison.
2. Preferably identical, but if not, then comparable “signature assignments” were to be created or identified in each course that can
measure the extent to which (some of) the Program/Dept SLOs are met.
3. The signature assignments were to be evaluated using the same rubric.
For the academic year 2010-2011 the program decided to evaluate the extent to which students meet two of the current program SLOs: SLOs 5
and 6. LRS 300 was designated as the Gateway course and ENG 428 (Children’s Literature) as capstone. Student performance was measured
through evaluation of a student generated paper outline with introductory and final paragraph in LRS 300 and a term paper in ENG 428.
1b. Implementation and Modifications: Did the actual assessment process deviate from what was intended? If so, please describe any
modification to your assessment process and why it occurred.
July 18, 2011, Bonnie Paller
No
2. Student Learning Outcome Assessment Project:
2a. Which Student Learning Outcome was assessed this year?
SLO5: Students will be able to think critically and creatively.
SLO6: Students will be able to write and speak clearly, coherently, and thoughtfully.
2b. What assessment instrument(s) were used to gather evidence about this SLO?
The following rubric was used to assess the data from both courses.
1. Effectiveness of thesis: the essay should contain a thoughtful, insightful thesis that responds to the assignment and is sustained
throughout the paper. The thesis should demonstrate critical thinking and awareness of the complexity of the issue under discussion.
a. Outstanding: The thesis effectively and insightfully develops a point of view on an issue and demonstrates outstanding critical
thinking and clear awareness of the complexity of the issue under discussion. (4 points)
b. Effective: The thesis effectively develops a point of view and generally demonstrates critical thinking and awareness of the
complexity of the issue under discussion. (3 points)
c. Competent: The thesis develops a point of view on an issue and demonstrates some critical thinking. May lack full awareness of
the complexity of the issue under discussion. (2 points)
d. Weak: The thesis develops a point of view on an issue that is vague or seriously limited. May demonstrate weak critical thinking
and lack awareness of the complexity of the issue under discussion. (1 point)
2. Support: The essay/outline should provide support for main points with reasons, explanations and examples making insightful use of
relevant sources.
a. Outstanding: The essay/outline is fully and convincingly developed, supported with good reasons and examples that fully
explain the thesis. (4 points)
b. Effective: The essay/outline is adequately developed, supported with reasons and examples that mostly explain the thesis. (3
points)
c. Competent: Most points in the essay/outline are somewhat developed, supported with some reasons, explanations and
examples. (2 points)
d. Weak: The essay/outline is inadequately developed; examples are not supported with discussion that explains the thesis. (1
point)
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type of
students, which courses, how decisions were made to include certain participants.
July 18, 2011, Bonnie Paller
All students in one section of LRS 300 and in one section of ENG 428 uploaded their outlines/term papers through a link provided by Institutional
Research.
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was
a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
The assessment compared student performance in LRS 300 and ENG 428.
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from the
collected evidence.
Three faculty members accessed this material electronically and applied the rubric to the data after a brief norming session. Each paper/outline
was rated by two faculty members. In case of more than a one point difference in rating, a third faculty member added her rating.
Ratings were averaged and the results were as follows:
Effectiveness of Thesis
Support
LRS 300 Gateway (n=22)
2.00
2.08
ENG 428 Capstone (n=19)
2.00
2.26
Interexaminer reliability turned out to be high.
•
•
•
•
The data indicate that LRS students on average can create an adequate thesis statement that demonstrates some insight in the subject
matter and support the thesis with some relevant arguments.
Comparing capstone and gateway performance should – when all goes well – provide the program with evidence of progress, or lack
thereof. While this assessment did take the form of a capstone-gateway comparison, it is unlikely that these data can be used to gauge
whether students are making substantial progress while in the program: Students do not always take LRS 300 upon entering the
program and end with ENG 428. There is no clear trajectory and significant lapse of time between LRS 300 and ENG 428.
In order to compare these results, the assignments also had to be sufficiently similar. While one of the assignments involved an actual
paper and the other an outline with introductory and final paragraph, the examiners felt that the rubric could nevertheless be effectively
applied to both groups of students. That being said, the nature of the subject matter does have an effect on how challenging it is to
develop an insightful thesis and argument. It is therefore doubtful whether the assignments were truly compatible in that sense.
Other factors also make it difficult to fully assess the level of performance: Students receive different degrees of assistance from their
July 18, 2011, Bonnie Paller
•
instructors when formulating a thesis with proper argumentation. This should also affect the results.
The link between this assessment procedure and LRS’ current SLOs is by necessity rather weak as the current SLOs do not really lend
themselves to behavioral assessment. This year’s assessment procedure confirms the need for new SLOs for the program and its
specializations, a process that is currently underway as part of a broader overhaul of the entire program.
2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic
quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program,
student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc.
Please provide a clear and detailed description of how the assessment results were or will be used.
The IDC discussed the results during their September 2011 meeting and reflected on how both the assessment process as well as the overall
performance of the students could be improved upon:
• While students demonstrated that they have a basic understanding of how to develop a thesis statement and support it with relevant
arguments, the results indicate that there is room for improvement. The development of critical thinking and reading and writing skills
should remain an important focus as LRS overhauls its curriculum.
• The program revision that is in progress should provide new opportunities to engage in valuable assessment in the form of
Gateway/Capstone comparison of specialization specific student learning outcomes.
3. How do this year’s assessment activities connect with your program’s strategic plan and/or 5-yr assessment plan?
The program aims to assess each of its SLOs over a period of 5 years. In doing so, it hopes to prompt an evaluation of the efficacy of various
components of the curriculum at reasonable intervals.
4. Overall, if this year’s program assessment evidence indicates that new resources are needed in order to improve and support student
learning, please discuss here.
5. Other information, assessment or reflective activities not captured above.
July 18, 2011, Bonnie Paller
As mentioned above, LRS is in the process of revising its curriculum. New program and specialization-specific learning outcomes are being
developed and aligned with the proposed curriculum.
6. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your
program? Please provide citation or discuss.
No
July 18, 2011, Bonnie Paller