Program Assessment Report
PROGRAM INFORMATION
Degree Program(s):
Department Chair:
Department:
B.A.
Political Science
James Brent
Phone:
4-5550
Report Prepared by:
Ken Nuger
Phone:
4-5346
Next Self-Study due:
2017
E-mail:
ken.nuger@sjsu.edu
Note: Schedule is posted at: http://www.sjsu.edu/ugs/programplanning/
ARCHIVAL INFORMATION
Location:
Clark Hall #471
Person to Contact:
(Bldg/Room #)
James Brent or Ken Nuger
4.5550 4.5346
(Name)
(Phone)
Does the information (e.g., Mission, Goals, and/or Learning Outcomes) posted on the web (see,
http://www.sjsu.edu/ugs/assessment/programs/ ) for this program need to be updated?
No
If yes, please submit changes to jacqueline.snell@sjsu.edu
SCHEDULE OF ASSESSMENT ACTIVITIES*
Please complete the schedule of assessment activities below by listing all program Student Learning
Outcomes (SLOs) by number down the left column.
.
When will data be collected? (must be
Semester the SLO is focus of efforts (no
at least once each program planning
PP cycle, each semester put this
cycle)
semester by the appropriate SLO)
SLOs
1
Breadth, ongoing
Spring, 2011
2
Application, ongoing
3
Disciplinary Methods, ongoing
4
Communication Skills, ongoing
Spring, 2010
5
Citizenship, ongoing
Page 1 of 10
Program Assessment Report
Learning Objective 4, Communication Skills: Written
Communication, Spring and Fall, 2009
BE SURE to enter the semester. This will become a progressive document to which you add information
each semester.
Initial Evidence of Student Learning:
The current political science department five-year program review is being finalized for
submission in the spring, 2010 and therefore, will begin a new program review period. We are mindful of
the recent suggestions and pleased with this new, simplified form provided by Undergraduate Studies to
construct our annual program assessment summary. Based on the assessment guidelines for Spring
2009 to Spring 2010, a memorandum to the Chairs and Assessment Coordinators from Jackie Snell
dated February 24, 2009 as well as a College of Social Sciences Assessment meeting conducted by Bob
Cooper, in his capacity as director of assessment for COSS and Ron Rogers, in his capacity as director of
Assessment for Undergraduate Studies, our department thought it prudent to accept WASC's
recommendation to assess written communication skills, which is our department's fourth learning
objective, for our Spring 2010 report. We last visited this learning objective in 2006. As we are all aware,
both from department, college and university level discussion, as well communications from WASC,
having our students write well remains a serious challenge. Since 2006, our department has had ongoing
discussions about how to try and improve our students' writing skills. The following represents a
summary of our efforts.
In our department's annual program assessment report dated October 1, 2006, our focus was
written communication. Using the rubric we developed at that time, we evaluated the written
communication skills of our seniors using the final research papers they submitted in their senior seminar,
a culminating and rigorous requirement for all of our graduating seniors. The rubric is attached at the end
of this report. At that time, our department's representatives on our curriculum committee read all 14
senior seminar papers and using our rubric, averaged the composite evaluation of the papers assessed
by the committee. On a five point scale, five representing competence well beyond normal expectations
for baccalaureate students in political science, to one, where a student does not meet normal
expectations, the first writing assessment of our graduating seniors seminar papers were averaged as
follows:
1. Student's work is free of technical errors: 2.47
2. Student clearly identifies and explains concepts related to politics, public administration, public policy
and law: 2.62
3. Student logically and clearly develops the thesis, arguments and/or theme of the written assignment:
2.38
4. Student correctly uses an acceptable attribution style, including footnotes, or parenthetical references,
as well as a complete bibliography or works cited section: 2.64
5. Composite assessment: 2.53
To summarize our first attempt to assess our students' writing abilities in 2006, our senior seminar
students fell between 3, student fully meets expectations for baccalaureate students in political science
and 2, student generally meets but in some respects, falls short of expectations for baccalaureate
students in political science.
Page 2 of 10
Program Assessment Report
Change(s) to Curriculum or Pedagogy:
Our initial efforts led us to conclude that most of our students' written communication skills were,
for the most part, barely adequate for a graduating senior in political science. We had hoped for evidence
of stronger written communication skills and since that initial assessment, our department has had
ongoing discussions about what we could reasonably do to help students improve their writing skills.
While not mandating changes with formal departmental policy, we started to be more vigilant in advising
students to take 100w in their junior year and we attempted to more vigorously monitor and supervise
student writing and encourage students to utilize the various writing support services offered at San Jose
State. As of this spring, 2010 semester, our department implemented a modest policy requiring a
statement in all syllabi about the importance of correct writing. While we are not requiring that everyone
utilize the same wording, we did create a model statement that would serve as a guideline for faculty to
construct their own commitment to correct writing statements for their syllabi. The model statement is
attached in the appendix.
Similarly, the department created a one-page handout, identifying the Writing Center, the Peer
Mentor Center and the Learning Assistance Resource Center as valuable on site resources students can
utilize to improve their writing. In addition, the handout identifies Owl, the Purdue writing Lab, A Research
Guide and the University of Wisconsin Writing Guide as three online resources we believe will help
students better understand what is expected of university level scholarly writing and offer opportunities
students can utilize to improve their writing skills.
While these changes are too new to currently assess, it is our hope that by having the department
and faculty continue to more openly discuss the importance of correct writing, we will make students more
aware early in their academic careers that writing skills are important to have and important to practice.
We hoped that our focus on student writing would help similarly make students focus more attention to
writing well. Assessing our current students’ writing abilities affords us an opportunity, as limited as it may
be, to make a judgment about whether our informal efforts are possibly helping our students’ writing skills.
Evidence of Student Learning after Change.
Our department continues to use our senior seminar final research papers as a means to assess
our department's five learning objectives. Per WASC and the university's recommendation, and using the
same writing rubric we used in 2006, we revisited our fourth learning objective, written communication
skills by assessing senior seminar papers from the spring and fall, 2009 semesters. The following
summarizes the curriculum committee’s efforts and findings. This year, the committee was composed of
six faculty members including the department chair. The six members were collapsed into three groups
of two, the goal being that each group of two faculty would independently read the same eight senior
seminar papers which we hoped would address data reliability issues. We hoped to evaluate 24 papers
from two semesters of senior seminars, eight papers per group and then take the average scores from
each group for each of the four writing criteria in our rubric. We fell short of that number, but ultimately
were able to use 13 senior seminar papers that were independently evaluated by two of the three groups.
One group evaluated seven papers, the eighth being an essay exam and not a paper; another group
evaluated six papers, with some confusion about which papers that group had to read that reduced the
number of papers evaluated by that group to six. Unfortunately the third group, consisting of the chair
and another committee member was supposed to also independently read a group of eight papers but
from the chair's perspective, there was overwhelming evidence to suggest his group mate did not
carefully read the papers, if at all, so at the discretion of the chair, the average from that group was
omitted from our initial evaluation. The following represents the average scores of 13 of 19 senior
seminar papers from the fall, 2009 semester.
1. Student work is free of technical errors: 3.57
2. Student clearly identifies and explains concepts related to politics, public administration, public policy
and law: 4.11
Page 3 of 10
Program Assessment Report
3. Student logically and clearly develops the thesis, arguments and/or theme of the written assignment:
3.8
4. Student correctly uses an acceptable attribution style, including footnotes, or parenthetical references,
as well as a complete bibliography or works cited section: 3.65
5. Composite assessment average: 3.78
If we include the scores of two papers read by only one committee member in one group and the scores
from eight papers evaluated by the chair from the department’s spring, 2009 senior seminar, the
averages for our four written communication goals are adjusted as follows:
1. Student work is free of technical errors: 3.5
2. Student clearly identifies and explains concepts related to politics, public administration, public policy
and law: 3.88
3. Student logically and clearly develops the thesis, arguments and/or theme of the written assignment:
3.52
4. Student correctly uses an acceptable attribution style, including footnotes, or parenthetical references,
as well as a complete bibliography or works cited section: 3.47
5. Composite assessment average: 3.59
Our committee was encouraged with these averages as they represent a significant increase
from the aggregate written communication scores from senior seminar papers evaluated in 2006. Fully
recognizing the methodological weaknesses raised by not using the first group of seniors evaluated, and
instead, making a judgment about our efforts to improve our students’ written communication skills from a
different group of seniors, possibly with different writing abilities, our committee was still encouraged with
the new data. At the very least, we are able to say that our sample of senior seminar papers generally
meet and in some areas exceed our department’s expectations for a baccalaureate degree in political
science. We believe that our faculty’s modest efforts to identify students who need help with their writing,
coupled with our recent efforts, stated above, to stress the need for polished writing has benefitted our
students and will inspire them to continually work at improving their writing skills.
As our annual reviews proceed and our focus turns to other department program objectives, the
department will continue to spot monitor our students' writing skills. Of course we shall continue to
carefully monitor our students writing skills as they progress through our program but recent evidence
suggests that our students are literate and possess effective writing skills.
Page 4 of 10
Program Assessment Report
Appendices
Appendix 1, Writing Rubric
Rubric for assessing written communication skills of our graduating majors
The following criteria shall be used to assess each identified written communication skill identified below.
5 = Student demonstrates competence well beyond normal expectations for baccalaureate students in
political science
4 = Student fully meets and in some regards exceeds expectations for baccalaureate students in political
science
3 = Student fully meets expectations for baccalaureate students in political science.
2 = Student generally meets but in some respects falls short of expectations for baccalaureate students in
political science.
1 = Student does not meet normal expectations for baccalaureate students in political science.
Written communication skills
1. The student's written work is free of technical writing errors.
2. The student clearly identifies and explains concepts related to politics, public administration, public
policy or law.
3. The student logically and clearly develops the thesis, argument and/or theme of the written
assignment
4. The student correctly uses an acceptable attribution style, including endnotes, footnotes or
parenthetical references, as well as a complete bibliography or works cited section.
Page 5 of 10
Program Assessment Report
Appendix Two, Writing handout
Department of Political Science
Writing assistance resources
Do you need assistance with your written assignments? The university provides excellent resources to
help you write high quality papers. You can also find some excellent resources online. We hope you will
find this list helpful.
At San Jose State:
1. The Peer Mentor Center, first floor Clark Hall
The peer mentor center is staffed with expertly trained undergraduate students, whose job is to assist
students in many different capacities, including writing assistance. You can drop in and arrange to meet
with one of the many peer mentors on duty. You can find out at: http://www.sjsu.edu/muse/peermentor/.
2. Writing Center, Clark Hall, Suite #126
The writing center will work with students in a one on one setting to help students in all phases of writing.
You can find out more at: http://www.sjsu.edu/writingcenter/.
3. Learning Assistance Resource Center, Student Services Center #600
The staff here will be happy to work with you and help you with your assignments. You can find out more
at: http://www.sjsu.edu/larc/resources/.
In addition to the excellent resources on campus, there are several academic websites that have
developed excellent online writing assistance programs. Three of the many well regarded of these online
cites are:
1. Owl, The Purdue University writing lab: http://owl.english.purdue.edu/.
2. A research guide for students: http://www.aresearchguide.com/index.html.
3. University of Wisconsin writing guide: http://writing.wisc.edu/Handbook/index.html.
In addition, reading a high quality newspaper on a regular basis will expose you to good, if not always
great writing and is highly recommended, especially for a political science major. The Wall Street Journal,
New York Times, Washington Post and realistically, any major sized city newspaper read on a regular
basis will, in the long run, help your writing skills.
Page 6 of 10
Program Assessment Report
Appendix Three, model writing statement for course syllabi:
Students of political science should develop the ability to write in clear, grammatical English.
Spelling and grammar count! The following guidelines and resources will help students avoid common
mistakes in spelling and grammar and provide useful information to help you craft university level
scholarship. Students must ensure that appropriate citations are used. Direct quotations must be so
indicated with quotation marks and a specific reference to the page in the source from which it was taken.
Ideas from others must also be referenced although quotations marks are inappropriate. Failure to cite
your sources constitutes academic misconduct which carries with it serious sanctions. A tutorial on
citations is available at the library website at
http://www.sjlibrary.org/services/literacy/info_comp/citing.htm.
For further writing assistance:
The SJSU Writing Center is staffed by professional instructors and upper-division or graduatelevel writing specialists from each of the several SJSU colleges. Writing specialists have met a rigorous
GPA requirement, and they are well trained to assist all students at all levels within all disciplines to
become better writers. It is your responsibility to arrange a time to meet with the instructors. Consult with
your professor in advance to determine whether you need assistance with your writing.
Arguably the best way to improve your own writing is to read quality material on a regular basis. To that
end, students are encouraged to read articles from scholarly sources but also newspapers such as The
Wall Street Journal, The Financial Times, The New York Times, and The Economist. These publications
also contain articles of direct relevance to their political science classes.
Page 7 of 10
Program Assessment Report
Learning Objective One, Breadth, Spring and Fall, 2010, for March,
2011 annual report
Initial Evidence of Student Learning:
For Spring and Fall, 2010, our department focused on assessing our breadth learning objective,
which essentially attempts to determine how well our graduating seniors learned the substance of our
major. Additionally, we spot checked 12 senior seminar papers to compare our newest graduating
seniors' writing skills for the spring and fall, 2010 semesters with those students submitting their papers in
2009.
To assess our students' mastery of our first learning objective, breadth, our department has
formulated a senior seminar exit exam that utilizes 60 multiple choice questions divided into our four subdisciplines, American Politics, Comparative Politics, International Relations and Political Theory. While
we are troubled with how best to accurately assess how much our students learned, we believe our
senior seminar exit exam is a good first step toward uniformly judging our students' overall mastery of our
material. Our concern primarily is how we can reliably gauge how well our students demonstrate mastery
of our broad curriculum when many of the courses are often taught by different professors. Of course,
there is much material that should be covered by any professor teaching a particular subject and we
attempted to draft exam questions that would reflect the core curricula in each of our four subfields. It is
naïve, however, to assume that students learn the same material regardless of the professor teaching the
seminar. Also, while most of our focus is on upper division coursework, it is similarly naïve to assume
that transfer students will get the same background training as our students who enter our program as
frosh. Finally, it is difficult to determine if better or worse exam scores are a result of our efforts as
faculty, or the natural variations in talent from different, relatively small groups of seniors from different
semesters. Those caveats stated, we have diligently worked on devising a senior seminar exit exam that
would assess how well our students appear to master the core substance of political science. The exam
has been used since 2008 to see if our students' mastery is either similar from one semester to the next
and preferably, to see if our students do better one year to the next.
Our department offered two senior seminars in the spring, 2010 and one in the fall, 2010.
Regrettably, we are unable to supply any data from Professor Harris’ senior seminar offered last spring
because he was unable to properly administer our 75 question multiple choice exam. In his seminar,
students provided most of the demographic data in the exam's first 15 questions, but left unanswered
almost all of the remaining 60 questions covering four sub disciplines. That left our department with data
from Professor Quill's senior seminar for the spring, 2010 semester and Professor Haas' senior seminar
for the fall, 2010 semester.
The exam itself explores student responses categorized multiple ways. We attempt to discern
our students' broad knowledge of our discipline from several perspectives. We compare transfers with
non transfer students, as well as compare scores based on a variety of student characteristics including
gender, race, age, part and full time student, average hours of work per week, where core political
science courses were completed, sub discipline specialty, to name some but not all of the variables
examined.
This year, we compared our senior seminar exit exam results with those of 2008. For the
purpose of this annual assessment report, one measure, our students' mean score on the exam, is
summarized. There were 78 students examined in 2008 and 41 students examined in 2010, not including
the 20 plus students from Professor Harris' seminar whose scores were unusable.
Page 8 of 10
Program Assessment Report
The overall mean score for 2008 was 52% and 56% in 2010. While these mean scores seem low
to us, justifying a discussion of what to make of these scores, we do note an improvement from 2008.
The median score was 55% in 2008 and 57% in 2010. Within our four sub disciplines, students had a
mean score of 61% in American Politics, 50% in Comparative Politics, 65% in International Relations and
50% in Political Theory. The corresponding mean scores in these sub disciplines in 2008 were, 61%,
44%, 59% and 47% respectively.
Our committee does not yet know what to make of what seems to us to be relatively low scores
across all sub disciplines. Should we be alarmed? We simply do not know yet. We were cautiously
optimistic with the improvement from two years ago and will monitor future administrations of this exam
for a hoped for upward trend. Also, while we are chagrined at the low score, we are confident that our
senior seminar exam is a good instrument to use to assess breadth of political science knowledge. The
exam was carefully written by our faculty to include the core curriculum within each of our four sub
disciplines. Using this test every year, at least for a few years, will enable us to compare each year's
group of graduating seniors using the same criteria. After a few administrations, we believe we will be in
a better position to understand what modifications to our curriculum and/or our pedagogy, if any, may be
warranted.
In sum, using this senior seminar exit exam to assess our breadth program learning objective has
piqued our curiosity and has started serious department discussions about how well our students broadly
know the discipline of political science. In time, we hope the data will bring some clarity to our
understanding of this learning objective and that we may be in a better position to know what to do
differently to increase our majors' broad knowledge of the discipline.
As a follow up to last year's report, our curriculum committee spot read 12 senior seminar papers
from the Professor's Quill's, Harris' and Haas' seminars. Contrary to university expectations, Professor
Harris chooses not to use his university assigned email and therefore is unable to receive electronic
copies of his students' culminating papers. In attempt to minimize the damage to our assessment efforts
brought on from Professor Harris' defiant choice to not appropriately conduct his university
responsibilities, our committee solicited his students to send us their papers. Through their benevolence,
we were able to collect over half of the papers in Professor Harris' seminar. Both Professor Quill and
Haas collected all of their students’ papers in their respective seminars. The committee read four papers
randomly from each of the three senior seminar sections. Of the dozen spot checked, the results are as
follows:
1. The student's written work is free of technical writing errors. 3.58
2. The student clearly identifies and explains concepts related to politics, public administration, public
policy or law. 3.16
3. The student logically and clearly develops the thesis, argument and/or theme of the written
assignment. 3.416
4. The student correctly uses an acceptable attribution style, including endnotes, footnotes or
parenthetical references, as well as a complete bibliography or works cited section. 3.25
5. Composite average, 3.375
With the exception of question #2, which was significantly lower this year than last, the scores are
very similar to our average scores last year and all at above basic competence for graduating seniors.
Our department feels confident that our students, at least as graduating seniors, are meeting our
demands for basic writing competency in political science.
Change(s) to Curriculum or Pedagogy:
Page 9 of 10
Program Assessment Report
With regard to written communication, it is increasingly evident that standards for seminar papers
vary greatly from professor to professor. Of particular concern is the variation our students have in
question #2 in which we assess how well our students demonstrate how well they can identify and explain
concepts related to politics, public administration, public policy or law. Perhaps this is in part due to how
different the demands are of our students by their professor for writing a culminating senior seminar
paper. We will discuss this issue and attempt to resolve it. However, it is not our department's mission to
micro manage how faculty teach their courses. Still, for the foreseeable future, the department shall
initiate discussions on how we may be able to minimize the variation of expectations for research papers
from different faculty teaching senior seminar. Within the next few weeks, our goal is to draft specific
policy guidelines for culminating senior seminar papers and also develop standards for the prospectus
assignment in our 195 methods class that we will use next year to evaluate next year's learning objective,
application skills.
Evidence of Student Learning after Change
As explained earlier, it is our department's collective view that significant changes are in order
only after we are clearly convinced that any change is warranted and meaningful. Therefore, we are
consciously attempting to create continuity in our efforts for this program assessment cycle with the
ultimate goal of considering changes after we are more confident where changes might be needed. To
merely react to small variations from year to year in our assessment efforts seems ill advised. For the
moment, however, one issue we are becoming more aware of is the different standards imposed by
different professors for their students culminating research papers. We will note any modifications we
may make to try to ensure that regardless of which professor a student takes for senior seminar, the
standards for their culminating papers shall be roughly equivalent.
Final observation for this 2011 report
The committee wishes to note that in the last year or two, our department's assessment efforts,
which is overseen by a five person curriculum committee who actively involve other faculty who, in turn,
are becoming increasingly aware of and used to our assessment efforts, appear to be slowly building a
department wide culture that is at the very least, amenable to assessment. At the assessment workshop
on February 11, one of the topics addressed was how to get the broad faculty to "buy into assessment."
By spreading the various assessment responsibilities to many faculty, and then funneling the data from
their efforts to the curriculum committee so it may analyze their efforts and prepare these reports, most
faculty in our department are indeed buying into the concept. Most of the tenured/tenure track faculty are
actively involved in some aspect of our overall assessment efforts and increasingly, our adjunct faculty
are becoming more aware of and participating in our assessment responsibilities. While there are
certainly challenges still to come, our department is confident that we are building a long range plan that
will allow us to reflect on how well we are teaching our students what we want them to learn and over the
course of a few cycles, make adjustments to our curricula and pedagogy.
Page 10 of 10