Undergradute Curriculum
Assessment/Evaluation Report
Ted Baker
CS Faculty Workshop
18 August 2009
Overview
•
•
•
•
Training for instructors
Review of student exit survey responses
Accreditation status summary
Things we need to do
Not the most logical order, but the most critical items first.
Training for Instructors
• Go to www.cs.fsu.edu/ugcc for
– approved course objectives; include them.
– SMALC outcomes that you must assess (if any)
• Rubric you must use
• Action plan from prior year
– end-of term reports from prior instructors
• Go to DOF website for most recent
– Attendance statement
– ADA statement
– Honor code statement
• Send syllabus to DUGS at start of semester
– Preferably, in time for review before start of classes
• We will soon (hopefully) have a web-based system for uploading
the syllabi.
Other Beginning-of-Term Matters
• Confer with instructor from prior term, if possible. Get
advice, materials.
• Find out who is teaching the course for distance/PC
campus, and communicate with them.
• If you want to change textbooks, propose the change
to the UGCC (but try for stability).
• Beware of university and departmental requirement
for final examination in all UG courses, given during
final exam week.
• If you have a TA, be sure to supervise, observe, mentor,
and evaluate.
Issues from Exit Survey
• See http://www.cs.fsu.edu/ugcc/smalcs09/surveys.html
• Today, see spreadsheet.
Issues raised by student advisory panel
• See http://www.cs.fsu.edu/ugcc/minutes/ugsac_02_10_09.html
• Discrete math sequence is not working
– Some math instructors are not teaching the courses well
– Students are not motivated, because they don’t see the applications
– We are not using the material enough in our courses soon enough, so
student forget what they did learn
– Significant repetition between and within Discrete Math courses,
COP4530, and COP4531
– Many students seem to be discourage dfrom major by the DM sequence
• Have CS faculty teach discrete math
• Integrate DM II into CS courses where it is used
More
• COP4531 tries to rush through too many algorithms, without adequately
developing the underlying design techniques
• Courses that are logically connected, as prerequisites, need to be offered
in contiguous semesters, so that students don’t forget what is needed
from the preceding course
• Overlapping course pairs not well coordinated and not consistent:
sometimes duplication, and sometimes gaps. Need more instructor
coordination, more consistency.
•
•
•
COP3014-COP3330
COP3330-COP4530
COP4530-COP4531
More
• Problems with instructors who are new to a course.
Suggest less turnover of instructors in a course.
• Excessive use of Powerpoint makes lectures move to fast,
too hard to follow. Suggest more hand-writing to engage
students.
• Need more exposure to event-driven programming
• Electives are a problem. Department should poll students
to see what electives they want.
• Need more room for electives in the curriculum (less
required courses).
• Department seems to put more priority on graduate
education than undergraduate.
• Too much overlap between CDA3101 and CDA3100.
More
• COP4710 requires students to use Java. This ends up being
duplicative of the Java course.
• Intro to Unix and Unix Tools courses sometimes overlap a
lot, and vary a lot in content from term to term.
• We are not making good use of recitation meetings. They
should not just be another lecture, perhaps by the TA. They
should be used to interact with students, with a focused
activity.
• COT4420 notes need updating.
• More use of IDEs (along with command line) in courses
beyond the introductory level.
• COP3014 might attract more students if there were more
interesting (e.g., GUI) assignments in it.
Accreditation Status
• Interim report submitted in June, now being
reviewed by ABET/CAC representative.
– You can read the report if you go to
http://www.cs.fsu.edu/ugcc/interim.pdf.
– Username/password = smalcs/smalcs
• If this report is approved (in July 2010), we will
be approved for another 4 years.
• Otherwise, we would probably be asked to do
another interim report in June 2011.
Issues in the Report
• Coverage of social and ethical issues related to
computing.
• Adequacy of elective offerings.
• Sufficiency of university resources.
• Outcomes and assessment process.
Ethical & Social Issues
• Added material to DB and SE courses
• Created new course, CIS4250
• This addresses the ABET complaint, but
–
–
–
–
–
DB and SE course treatment is now duplicative
It can’t be phased out until grandfathered students graduate,
and we find a way to handle distance students.
Distance students can’t do oral debates.
Oral debates can’t be made optional for CIS4250, since it is an “oral
competency” course for SACS accreditation purposes.
• One idea:
– Create another ethics course, related to computing (maybe only 2 hrs,
maybe only DL, maybe only taught by adjuncts)
– Students who take that course must also take SPC2600 to cover oral
competency requirement.
Elective Offerings
• This is partly a student perception problem.
• Report shows that we actually are offering more electives than the
institution of the chair of the team that made the criticism.
• Remaining issues:
– Many are not practically available to most students, due to tough
prerequisites (dual-listed graduate courses, other electives as
prerequisite, senior-level courses as pre-requisite)
– Distance students have fewer options
• What we are doing:
– Offer popular ones, without daunting pre-requisites, more often.
• Things we might do:
– Allow students to take some electives out of CS?
– Work with other Florida universities to share distance course
offerings?
Adequacy of Resources
• ABET reviewers probably thought they were helping
the program, by putting pressure on the FSU
administration to replace lost faculty lines.
• Our interim report explains effects of current economic
distress on the University, and that
• we are lucky to have suffered no layoffs, and to have
been allowed to replace Ken Baldauf and Chrissy.
• I have explained this also, in person, to the ABET
reviewer.
• We can hope that ABET will appreciate putting further
pressure on FSU will not help.
Assessment
• We have expanded our assessment process.
• Tried to combine ABET requirements with SACS/SMALC
requirements.
• Doubled number of SMALC outcomes
– to cover min. set of ABET/CAC-required outcomes
• Added end-of-term instructor course reports
– to provide more useful information for program improvements
• Added focus-group meeting with undergraduate students
• Added more formal review and reporting process
– including this report, to the workshop
Remaining Problems
• Faculty education.
– Everyone needs to understand the process and
the goal.
– Therefore, this talk and more to come.
• Increased workload (“death by assessment”)
– Therefore, proposals to come, to reduce load.
• Fundamental fallacy in ABET model
– We can still make a good faith effort.
“Death by Assessment”
• Gathering the assessment data is a lot of work.
• This is distracting us from other essential duties, including the evaluation
and planning that are supposed to be driven by the assessment process.
• Faculty members still do not all understand what is expected.
• The DUGS is overwhelmed by the workload.
– The DUGS has already spent more than a man-month at this, dunning for data,
meeting with people to explain what is needed, extracting files from e-mails
and Web pages, editing, renaming to a standard system, copying into
directories, reformatting into .pdf, renaming and reformatting them, creating
web links to the files, etc. Moreover, thhis manual work is very error-prone.
– Much of the supporting data files still need to be populated on our web site,
to which the interim report refers.
– The DUGS has been meeting with the webmaster since late fall 2008 to
develop an on-line tool that would automate most of this, but the process
seems to have stayed on the back burner.
Assessement Processes
• State Mandated Academic Learning Compacts
– We specify some specific minimal achievements
(“student educational outcomes”, and “program
outcome”) for our graduating students, which we
must assess and report every year.
• ABET
– Starts similarly to SMALC, but goes further.
We must demonstrate that we use the data from
assessments as the basis for program
improvements.
ABET Model
• Engineering method should be applied to the
process of education.
• We define a set of desired outcomes for our
educational program.
• We gather data on how well those outcomes are
being achieved.
• We review and evaluate the data, which drives
our plans to improve our program.
– Not just the instructional process, but also the metaprocesses (outcome statements, assessment methods,
evaluation process, etc.)
Fallacy in the Model
• Assumes we have a repeatable process.
– So that we can vary one parameter, and assess how
this has improved/hurt the program.
– This seems to be called “regression-discontinuity
design” by experimental design theorists.
– We have different people teaching courses,
sometimes in the same term.
– Students coming into courses also have had different
experiences in preceding courses.
– There are so many differences in courses and students
that we can’t isolate the effects of any specific change
with reliability.
What can we do?
• Try to make the process more repeatable?
– Not rotate instructors so frequently.
– Communicate more between successive instructors,
reusing text, slides, etc.
• Take a more holistic view of the program
assessment, recognizing that regressiondiscontinuity experimental method is not feasible
in this environment.
• Turn in syllabus before the term, and have it
reviewed for consistency.
How to reduce workload
• Increase priority of on-line data collection.
• Collect SMALC data only once a year.
• Reduce set of courses in which SMALC data is collected, to
a few capstone courses.
• Collect instructor reports in every course, and rely more on
them for directing program improvements.
• Extend survey to gather more feedback from alumni.
• Continue
• Do thorough review of assessment data less frequently,
rotating through courses and outcomes on a multi-year
cycle.
– Which ones to start with? The math/theory sequence?
Remaining Instructor Responsibilities
• SMALC data collection, in some courses.
• End-of-term report, in every course.
• Reviewing SMALC and other data, and making
specific recommendations to DUGS and/or
UGCC.
Other UG Curriculum Issues
• New BA degree requirements
• Relaxing prerequisites, to make courses more
accessible
New BA Requirements
• Tougher than BS; requires 12 more courses than BS, in
humanities.
• 2 students graduated with BA in 2007, none in 2008 &
2009.
• ABET considers this a separate degree; charges double fees,
sends extra accreditation visitor; wants examples of
student transcripts to prove program is vital.
• We can’t provide example transcripts.
• We decide to drop ABET accreditation of the BA degree.
• This opens up possibilities
– To attract more students to CS
– By eliminating for the BA some requirements imposed solely to
satisfy ABET/CAC
Relaxing Prerequisites
• CS curriculum discourages cross-disciplinary
studies, due to very deep prerequisite
structure.
• This also causes scheduling problems for
students, and limits their choices of electives.
• It is not clear that we actually depend on all
these prerequisites.
• It is time to review all prerequisites, to see if
they are essential.
Some changes, already
• Java for non-majors
– dropped C++ prerequisite
• C++ for non-majors
– Reduced math pre-requisite
• COP4610
– Reduced CDA3101 to co-requisite or instructor
permission
• Security courses
– COP4530 replaced by CDA3100
• Can we go further?