Annual Assessment Report to the College 2010-11
College: _Social and Behavioral Sciences
Department: Geography
Program: ______________________________
Note: Please submit your report to (1) the director of academic assessment, (2) your department chair or program coordinator and (3) the
Associate Dean of your College by September 30, 2011. You may submit a separate report for each program which conducted assessment
activities.
Liaison: Ronald A. Davidson
Overview of Annual Assessment Project(s)
1a. Assessment Process Overview: Provide a brief overview of the intended plan to assess the program this year. Is assessment under the
oversight of one person or a committee?
The assessment committee consists of Steve Graves, Shawna Dark, Julie and Ron Davidson.
1b. Implementation and Modifications: Did the actual assessment process deviate from what was intended? If so, please describe any
modification to your assessment process and why it occurred.
We assessed what we had planned to assess, but our efforts evolved beyond this in building additional assessment tools for three of our 100level courses (101, 107 and 150). We are creating question banks for embedded assessment in these courses. We also decided to update our
department/course SLO alignment matrix. Faculty are currently correcting a draft of this matrix, which will allow us to ensure that all of our
July 18, 2011, Bonnie Paller
department SLOs are being met at appropriate levels of the program.
2. Student Learning Outcome Assessment Project: Answer questions according to the individual SLOs assessed this year. If you assessed more
than one SLO, please duplicate this chart for each one as needed.
2a. Which Student Learning Outcome was assessed this year?
SLO 2.2: Student demonstrates ability to construct a literature review.
2b. What assessment instrument(s) were used to gather evidence about this SLO?
Literature reviews in senior theses (written in Geography 490, the capstone course) were assessed.
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type of
students, which courses, how decisions were made to include certain participants.
Nearly all of the Geography 490 student papers were read. (Unfortunately, we lost track of three papers and those did not get assessed.) N=19.
This represents virtually all of the graduating seniors from our department for the year.
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was
a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
Students are introduced to literature reviews in Geography 300 and their abilities are assessed by Dr. Graves in that course; they were found to
be of consistently low quality as this is the first attempt for most students to write a literature review. Literature reviews are assessed again in
Geography 490. Thus assessment of this SLO occurs at two points in the program.
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from the
collected evidence.
The SLO was assessed with a rubric according to which literature reviews either exceed department expectations (“Student makes appropriate
use of quality research material, using it effectively to support or engage argument. Ideal selection of materials, reviewed resources”); meet
department expectations (“Moderately effective use of prior research, underutilization of material, over-reliance on non-refereed materials”);
or do not meet department expectations (“Student frequently misuses research material, uses material sparingly, chooses poor materials”).
July 18, 2011, Bonnie Paller
The results for Geography 490 were:
Exceeds expectations: 8
Meets expectations: 8
Does not meet expectations: 3
It has long been clear to faculty that our students struggle to write high-quality literature reviews. We assessed literature reviews in 2007-08
and found the average score to be 3.33 (out of 5); in 2008-09 we assessed literature reviews again and the average dipped to 3.0. This year we
were pleasantly surprised to find the average significantly up, to 3.5. This result likely reflects changes that various faculty have made to their
courses in an effort to improve student performance on literature reviews. (These changes are described in previous assessment reports.)
Given the small number of majors in our department (i.e. the small N), it is not possible to interpret the results with high confidence.
2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic
quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program,
student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc.
Please provide a clear and detailed description of how the assessment results were or will be used.
The assessment result was perceived as encouraging by the department. Faculty believe the relatively high quality of literature reviews in
Geography 490 papers most likely reflect past course modifications by faculty to produce better writers of literature reviews.
Nonetheless, this year’s positive results have invited some skepticism. Dr. Laity, one of the 490 instructors, suggests that high student
performance on literature reviews may misrepresent actual student ability levels. The final product assessed reflects, to some extent, the
direction, editing and writing advice of faculty over multiple drafts. Thus, student skills may be significantly weaker than final products indicate.
Future assessments of literature reviews will be modified to produce more accurate data on student skill levels. This may occur by assessing
drafts and by interviewing students in addition to assessing their work. The strategy will have to be worked out by the assessment committee
this year.
Because of our uncertainty in the reliability of the assessment data, the department remains committed to emphasizing literature-review skills.
Several instructors have recently made or plan this semester to implement changes to courses designed to improve student performance on
literature reviews.
What faculty are doing:
Dr. Jackiewicz, one of the 490 instructors, has introduced a WRAD-funded activity into his Geography 351 (Urban Geography) course designed to
teach students how to critically respond to and synthesize batches of scholarly articles on a given topic. This activity hence directly addresses
skills required in the writing of literature reviews.
Dr. Laity requires her Geography 366 (Geography of Environmental Hazards) students to read the scientific literature and use it in their written
July 18, 2011, Bonnie Paller
papers. Compliance has been weak (resulting in poor grades), so she is implementing a stricter policy. She also added a new assignment to her
Geography 467 (Arid Lands Geomorphology) class requiring students to read several journal articles and submit a written and oral report on
them.
Dr. Graves devotes significant time in Geography 300 to teaching literature review skills. His approach is to creatively respond to deficiencies
that are exposed by the students’ work, with specific teaching methods being updated continually. Familiarizing students with literature reviews
and teaching students how to write literature reviews are central goals of Geography 300.
2a. Which Student Learning Outcome was assessed this year?
2.4: Student demonstrates ability to collect useful data or information from field observation.
2b. What assessment instrument(s) were used to gather evidence about this SLO?
Senior theses written in Geography 490, our capstone course, were assessed.
2c. Describe the participants sampled to assess this SLO: discuss sample/participant and population size for this SLO. For example, what type of
students, which courses, how decisions were made to include certain participants.
Nearly all of the 490 student papers were read. (Unfortunately, we lost track of three papers and those did not get assessed.) N=19. This
represents virtually all of the graduating seniors from our department for the year
2d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was
a cross-sectional comparison used (comparing freshmen with seniors)? If so, describe the assessment points used.
Only the capstone papers were assessed.
2e. Assessment Results & Analysis of this SLO: Provide a summary of how the evidence was analyzed and highlight important findings from the
July 18, 2011, Bonnie Paller
collected evidence.
Exceeds expectations:
5
Meets expectations:
1
Does not meet expectations:
1
No applicable
12
These results drew two immediate and sharp responses from faculty, who see them as importantly revealing. The data indicate that few
students actually collect field data or make field observations for their 490 projects. Although we have not assessed this SLO in the past, Dr.
Laity believes that if we had we would have found things different 15-20 years ago. She believes that there has been a long-term decline in the
number of students who focus on field-based, physical geography. While students are interested in this topic, she believes that they “lack basic
skills that would enable them to feel comfortable moving forward.” In Dr. Laity’s view, students lack the skills to do mathematical or statistical
analysis of data. “Even when students do collect field data,” she observes, “they don't know how to analyze it. Many students, who have
otherwise done excellent work, reach a brick wall at this point.”
A second opinion generated by the data was that a 15-week time-frame is too short for students to engage in meaningful, field-based research.
Under the circumstances, there is a temptation to bypass field work and collect data from online sources for use in, for example, GIS projects.
2f. Use of Assessment Results of this SLO: Think about all the different ways the resulting evidence was or will be used to improve academic
quality. For example, to recommend changes to course content/topics covered, course sequence, addition/deletion of courses in program,
student support services, revisions to program SLO’s, assessment instruments, academic programmatic changes, assessment plan changes, etc.
Please provide a clear and detailed description of how the assessment results were or will be used.
The finding suggests the need for more field-based physical geography classes and exercises in field data collection and observation. Dr. Laity
has made a change to her Geography 467 course, requiring students to complete specific data gathering tasks in the field to help them
understand the nature of team-based research and learn to acquire their own data.
In general, however, modifying courses or the program to address the need indicated by this assessment report presents special challenges. For
several years, the faculty have expressed great interest in expanding the number of field courses offered, but have been frustrated by economic
July 18, 2011, Bonnie Paller
constraints and burdensome administrative requirements. What to do under the circumstances will discussed at faculty meetings this year.
3. How do this year’s assessment activities connect with your program’s strategic plan and/or 5-yr assessment plan?
(Sorry: this wouldn’t go in the box below-) This year’s assessment falls within the parameters of the previous five-year plan. Our new
plan goes into effect this year (2011-12). The new plan follows the same basic pattern of assessment, however, with the committee
choosing which SLOs to assess.
4. Overall, if this year’s program assessment evidence indicates that new resources are needed in order to improve and support student
learning, please discuss here.
The clearest signal from the data generated this year points to the need for additional field-based courses and field activities for students. These
require financial resources above the level that the department currently has at its disposal. At the same time, the administrative burden on
faculty is prohibitive. Additional funding for field activities and courses would be the most important resource for us. A way to streamline or
simplify the administrative requirements for taking students into the field would also be helpful.
5. Other information, assessment or reflective activities not captured above.
As noted above, we are building banks of test questions for embedded assessment of three 100-level courses. This year we hope to complete
these banks and introduce exams on this material in Geography 300 and 490 as a way of documenting knowledge gained (and retained)
July 18, 2011, Bonnie Paller
throughout the program.
6. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your
program? Please provide citation or discuss.
July 18, 2011, Bonnie Paller