Assessment of Anthropology Major.
Spring 2010
Determination: Assess SLO #3
Student Learning Objective 3. Knowledge of the significant findings of archaeology,
cultural anthropology and physical anthropology, and familiarity of the important issues
in each sub-discipline:
Operationalized Learning Objective B:
Ability to synthesize information for different areas of
anthropology.
Venue: Anth 162 Inca, Aztec Maya
Assessment instrument: Written synthesis assignment. Students selected a topic covered
during the semester and write about the issues in a synthetic way, drawing on multiple
sources to assess environmental, social, and historical integration.
Assessment based on a student’s comprehension of content, ability to frame the content
in a wider arena integrating historical, social and environmental interactivity. Student
also assessed on developing a central thesis and following the theme consistently through
the assigned paper using examples and relevant arguments. The papers were graded
holistically. This approach allows for maximum flexibility in topic selection while
subjecting all student papers to the same criteria of assessment.
Assessment process. After all papers from students were graded a representative 20%
random sample was pulled for intensive analysis of three specific criteria associated with
the Learning Objective: Are students effectively synthesizing information? Are students
presenting historical, archaeological, or anthropological data accurately? Are students
developing a thesis that integrates course content with various source materials to
generate an original synthesis of ideas? The assignment was issued twice. Scores were
compared between the first and second assignment to holistically measure growth, but
also to determine whether students’ performance was also improved through familiarity
with the assignment.
Rationale. Synthesis is an important skill within the discipline and developing this
capacity is a critical theme incorporated throughout the many Learning Objectives of the
Major. It takes many forms across the curriculum and regardless of assessment tool or
format, synthesis can be made manifest in a variety of ways owing to the multiple
intelligences and experiences of our students. Nevertheless, there are certain features of
synthesis and the process of integrating knowledge to achieve new understanding or in
formulating original ideas that are recognizable regardless of format. Some of these are
grounded in cognitive development and some in training. The question that deserves
examining is not simply assessing whether or not students are synthesizing information
but whether the structure of the curriculum is facilitating the training of synthesis as a
way of learning and whether the curriculum and course design spur cognitive
development. The model for nudging cognitive development forward has been referred
to as N + 1 in educational literature, wherein instructors assess a student’s cognitive skills
through various instruments (although in practice this usually is informal assessment
gained through interaction) and then provides content and integrated learning techniques
that are strategically just beyond the students’ ability. The challenge to students is
obvious as they must grow their skills and intellect incrementally through the duration of
a course. The challenge for the instructor is just as difficult, ensuring that the increments
are neither too great nor too complex. Furthermore, instructors face a greater dilemma in
achieving N + 1 when students are not staring with the same value of N.
In the case of ANTH162, the underlying assumption was that the majority of the students
would have a limited background in the course content or a personal interest in some
select aspect of the material. For instance, the majority of students in the class were
Latino/Latina heritage and expressed an interest in one specific culture or another that
were the subject of the course, however, few admitted any deep background knowledge.
Only three students had any in-depth knowledge of any of the primary cultures to be
investigated in the course and none of the more obscure predecessors of the Inca, Maya
or Aztecs. Therefore, it was deemed generally safe to assess the class as somewhat
homogeneous in terms of N (content) was concerned. As for N (cognitive ability) was
concerned, this would need to be assessed through short synthesis questions on the Unit
Exams.
However, this assessment was not about individual student performance but whether
students had developed or were being exposed to adequate training to engage in
meaningful synthesis in anthropology. Therefore, the comparison of Synthesis 1 and
Synthesis 2 during the semester was used only to investigate performance consistency.
Specific examples of student work following Synthesis 2 were examined for assessing the
ability to synthesize.
If students were found to synthesize in accordance with course objectives there would be
reason to believe that the SLO was being addressed in a substantive way. However, if
students were failing to synthesize or producing naïve synthesis, there would be reason to
make adjustments to the way the course was being delivered, or perhaps the skill would
need to be addressed more often, or more vigorously, or in more courses across the
curriculum. This would be a subject for discussion among faculty.
Results
The Synthesis papers were worth 50 points. The average score of 38 papers for Synthesis
1 was 45.36 and for 36 papers Synthesis 2 was calculated at 47.69. The slight increase of
from 90% to 96% likely indicates familiarity with the assignment protocols more so than
cognitive gains. The high initial scores and consistency from one assignment to the next
is an indicator of relatively high capacity to begin with suggesting that N (cognitive) was
consistent, but that students were not as challenged as would be desired. This reflects
more on the instructor’s initial assessment than on the implementation of lessons or
instructional strategies for achieving the SLO. The results indicate that skills at synthesis
and higher order cognitive development are receiving attention in the major and students
in the course were able to demonstrate, on average, a reasonable integration of the skills
as described in the SLO. In terms of student majors, 34% (13) of the class were not from
the Anthropology/Behavioral Science program of the 66%, (25) the ratio was 3:1
Anthropology to Behavioral Science.
The analysis of individual papers from the random 20% provides another data set for
discussion. Eight papers were randomly drawn from the Synthesis 2 assignment. One
caveat was, non-majors were eliminated from the sample to control for the possibility that
students were getting training in disciplines other than Anthropology that might impact
the assessment invisibly.
The range in scores for these was a low of 47 and a high of 50. Because the lowest
scores were not represented by the sample these were checked and found not to be
Anthropology majors. It was included anyway to assess the reason for the low score of
42. The problem was found to be language skills. The student is not a native English
speaker and much of the low score was based on writing mechanics rather than synthesis.
The basic thesis of the student was in fact of significance but language difficulties
interfered with communication. However, the student exhibited a 30% increase in score
from Synthesis 1 to Synthesis 2, demonstrating that something was working. Students
receive exposure to synthesis writing in several courses in the major and it is evident that
repeated opportunities pay dividends.
In the case of the eight papers, all students demonstrated the ability to frame a relevant
and generally original thesis. Each displayed critical thinking through development of
arguments supporting their thesis and providing supporting evidence. Thematic strands
were consistent in the narratives and students were able to match evidence to conclusions.
Some were more comprehensive than others in terms of details. The weaker papers
tended to summarize more than synthesize, yet still managed to make synthetic
statements in their conclusions. All but two were technically sound as far as writing was
concerned.
Action to be taken: We have determined that the curriculum for this course is well suited
to developing students’ skills in critical thinking and synthesis. However, increasing the
rigor is probably warranted. While the content is substantial, increased emphasis on
synthesis and integration of content could be managed.
The SLO is being met in this course and very likely across the many courses of the major.
Changes in the specific curriculum are unwarranted. Synthesis is what Anthropologists
and Behavioral Scientists do and we believe we are being effective in meeting this SLO.
However, the results of this analysis suggest that students may require more challenges to
spark improvement in higher order cognitive skills and to push the envelope on their
potential. Strategies to achieve this aim were discussed by faculty at the end of year
department meeting.