Executive Summary of Assessment of Student Learning and Development
Number 4
2000-2001
Introduction
Assessment in Higher Education Today
When we think of “assessment,” we typically think of “tests,” “grades,” “evaluations,” and so on. In higher
education today, the term “assessment” has come to mean much more. It goes beyond testing and
grading of students by professors. It is more than evaluation. In the words of one researcher,
assessment, as it is understood by most in higher education today, is “the measurement of the
educational impact of an institution on its students” (Terenzini, 1989).
The goal of the assessment process is the improvement of all institutional practices that affect student
learning and development, from the classroom to the athletic field. It begins with setting clear objectives
for student learning and development that are consonant with the University’s mission statement. It
continues with the gathering of meaningful information to measure the accomplishment of institutional,
departmental, and classroom objectives. The assessment “loop” is complete only when the information
gained from this process informs planning and decision-making in each and every department and results
in constructive change.
The assessment movement in higher education began in the 1980s with public insistence on educational
accountability. After two decades, this movement shows no signs of going away. Effective assessment
programs are required by all six of the higher education accrediting associations. The North Central
Association, which accredits Concordia University, St. Paul, requires institutions it accredits to provide
evidence of the effectiveness of their educational programs (North Central Association of Colleges and
Schools Commission on Institutions of Higher Education, 1996). Performance-based or outcomes-based
budgeting is part of spending formulas for public colleges and universities in at least half the states
(Lopez, 1999). Lopez, of the North Central Association, predicts that the “public demand for educational
accountability will continue to require colleges, universities and systems of higher education to document
and improve student learning” (1999, p. 6).
Accountability mandates aside, the essential goal of assessment is to improve the quality of student
learning by improving the practices that impact learning. Improved institutional practices result in
institutional excellence. And institutional excellence benefits all stakeholders: students and their families,
faculty and staff, the church and the community (Wolff & Harris, 1994).
Relationship of Assessment Activities to Other Measures of Institutional Effectiveness
The assessment of student learning and development, which is covered in this document, is only one
aspect of the evaluation of institutional effectiveness (Lopez, March 1996). The evaluation of other
aspects of institutional effectiveness is carried out by department chairs, deans, program directors, and
vice-presidents as they complete program evaluations, annual reports, and other activities to evaluate the
achievement of Concordia University’s strategic goals. The following sections summarize the ongoing
departmental and institutional activities that assess student learning and development and the activities
that impact them.
Executive Assessment Summary, 2000-2001
Summary
The 2000-2001 reports submitted by faculty assessing student learning in their academic majors continue
to demonstrate that most faculty at Concordia University are fully engaged in reflecting on their practice
as teachers and on improving the curriculum and student achievement. Most departments were able to
document faculty collaboration and discussion of assessment activities, outcomes, and changes made as
a result. It is evident that departments that have experienced success with their assessment strategies
have also been very intentional about discussing teaching, learning, and assessment as a department.
Academic majors at Concordia University include a wide range of assessment activities, many of which
are themselves learning activities, such as capstone courses, internships, portfolios, performances,
exhibitions, case studies, business plans, comprehensive exams, research projects, and conference
presentations.
For a significant number of faculty, assessment activities are imbedded in the curriculum and the learning
process, and provide the means to inform improvement of instructional practices and curricular offerings,
and to highlight areas that would benefit from broader institutional efforts (e.g., writing skills). However,
there continue to be some faculty that do not fully engage in the assessment process and do not yet see
how assessment can increase the quality and focus of student learning. For these faculty, assessment
has remained something to be added on, an “administrative headache,” or an intrusion into the act of
teaching. For others, assessment is seen not as learning-focused, but as a series of “hoops” students
must go through. Others have not separated the documenting activity of report writing from the learningfocused activity of assessment, and therefore approach it in a superficial and minimal way.
The reasons why some faculty do not take advantage of assessment activities is unclear from the
materials presented here. Some may simply feel they do not have the resources (time, know-how,
personnel) to plan and carry out effective assessment practices. In order to provide the support faculty
need to be effective as reflective practitioners, the assessment committee would do well to tease out the
reasons for underdeveloped assessments. Programs that have established effective means of assessing
learning throughout their curriculum should be encouraged and rewarded for sharing successful learning
practices across courses and departments and among professors, and for sharing their assessment
strategies in professional settings at the local or national level.
.
It is evident that systematic assessment of general education outcomes is not yet part of the institutional
assessment process. The general education curriculum has proved to be a controversial and continually
evolving entity, and the faculty charged with overseeing it have been consumed by the ongoing demands
of clarifying and maintaining it. A general review of the curriculum currently in place and its relationship to
the current general education philosophy has begun, and will initiate an assessment process that will
eventually address the learning outcomes of general education.
Reports of assessment activities from areas outside of academic departments were unavailable. In order
to get the complete picture of how the university impacts student learning and development, other
university units could be encouraged to engage in assessment activities.
Overall, faculty in academic majors reported that students are satisfactorily meeting the goals and
objectives set for academic programs, despite a wide degree of variability in students’ performance. The
institution will continue to be challenged to provide sufficient time and resources to allow faculty and staff
to develop and implement assessments that will target learning areas to be improved and provide the
means to improve them.
M. Luebke, Office of Assessment
2
Executive Assessment Summary, 2000-2001
Learning and Development in General Education
During the 2000-2001 academic year, members of the general education committee continued to focus
on defining the general education curriculum goals and began to explore the implications of those goals
for all the colleges of the University. The committee developed a set of criteria to define a University
general education course, and will undertake to assess the fit of current general education courses using
these criteria. This activity will be the first step in an assessment process to determine the impact of the
curriculum on student learning. It will provide a “snapshot” of what the curriculum looks like right now and
perhaps some insights into how the curriculum has evolved over the 5 years since its inception.
During the summer of 2000, the University assessment committee met with the general education
committee to develop a plan assessing general education outcomes using the revised goal areas. The
committees proposed a two-tiered plan. The “first tier” is at the course level. Faculty teaching courses
within goal areas (e.g., history and political science) developed outcomes for students taking any course
in that goal area, and determined course-level assessments that will demonstrate student learning in that
area. This task was begun by academic areas during the August 2000 faculty retreat. Plans for
documenting these assessment activities have not yet been developed.
The second level of assessment involves measuring skills or outcomes that are common across general
education offerings, for example, writing, critical thinking, or quantitative reasoning skills. The
assessment and general education committees determined to tackle one of these broader areas at a
time, with at least one year being devoted to studying outcomes in one area before attempting the next
one. Based on evidence from faculty surveys and assessment reports, it was determined that writing is
an area of immediate concern. This “second tier” project was not initiated during the 2000-2001
academic year, but is proposed for the general education committee agenda for 2001-2002.
During the 2001-2002 academic year, faculty members who have received funds from the Bush
Foundation faculty development grant will focus on the creation or modification of general education
courses to enhance students’ critical thinking skills. Initial assessment of these activities and their impact
will be completed at the end of the 2001-2002 academic year.
This most recent Bush Foundation faculty development grant used Bloom’s taxonomy of mental activities
(memorization, analysis, synthesis, evaluation, and application) to focus faculty on course activities that
would foster the higher level thinking skills, such as synthesis, evaluation, and application. Relevant to
these efforts are some results from the National Survey of Student Engagement. (Please see the section
below titled Learning and Development Beyond the Classroom.) Significantly fewer seniors (in a group
that included students in both the traditional and degree-completion curricula) than all comparison groups
reported an emphasis on memorizing facts, ideas, or methods. First-year students (traditional curriculum)
and seniors both indicated that their courses require them to analyze, synthesize, evaluate, and apply
concepts or theories to practical problems or new situations at about the same rate as students in the
comparison groups.
Learning and Development in the Academic Major
Academic departments document their assessment activities and use of assessment results for
improvement in an annual assessment report for each major or graduate program. These reports consist
of four sections (see Appendix I). The first section outlines the expected learner outcomes, or goals and
objectives, for that program. The second section indicates which assessment activities were conducted
that year for students in that program. The third section reports the actual results of those assessment
activities, that is, how students as a group actually performed when measured against the goals and
objective for their program. Finally, the fourth section reports how those results were used by the
department to improve or strengthen curriculum and instruction in the program.
M. Luebke, Office of Assessment
3
Executive Assessment Summary, 2000-2001
Twenty-seven assessment reports for 2000-2001 were completed for undergraduate academic majors,
and reports were written for six graduate programs. The process for reviewing these annual reports was
revised this year, and now involves small groups of faculty peers meeting together to discuss their reports
and their findings. The new process is intended to facilitate broader sharing of assessment strategies
across disciplines and departments and to make it a more collegial activity focused on sharing what
works to improve student learning. Each small group session is facilitated by a member of the
assessment committee, and participants discuss their struggles and triumphs with assessment and
student learning in their majors and programs.
Brief summaries of 2000-2001 assessment activities for majors and programs, as reported in the annual
assessment reports from academic departments, follow.
College of Arts and Sciences
Department of Art
The department of art offers three majors, studio art, community arts, and art education for K-12. An
assessment report was completed for the major in studio art. The primary comprehensive assessment
procedure for student learning in this major is the senior capstone portfolio review and exhibition that is
evaluated by peers and professors. Students are evaluated on the quality of their work and its
professional presentation, and also on their level of understanding of it and its context and historical
influences. Faculty reported that their expectations regarding students’ exit portfolio exhibitions and
expression of understanding were met or exceeded. Faculty indicated that their discussion of these
outcomes and student feedback led to improvements in course syllabi and the senior capstone exhibition.
Department of Business
The department of business offers three majors: accounting, marketing, and finance. The department
has been granted candidacy status by the Association of College Business Schools and Programs
(ACBSP) and will be pursuing full status. In all three majors, activities at the course level were used to
assess learning in the major, and a standardized exam in business (Major Field Test in Business) was
administered.
In the finance major, various projects requiring the application of finance principles and practices were
used to assess students’ learning, including a financial analysis project and site analysis. Faculty
indicated excellent results on the site analysis and above average results on the financial analysis, with
some weaknesses noted in statistical and cost accounting skills, analytical thinking skills, and writing
skills. On the standardized exam in business, students scored higher on the finance portion than the
previous year, but no analysis of the significance of this was included. In response to all these results,
faculty decided to create a business writing model and include it in all business courses taught by the
department, to add an international finance course, and to adjust course grading standards to more
accurately reflect the demands of the standardized exam.
In the marketing major, assessment strategies also emphasized performance assessments as an ongoing part of coursework. (Performance assessment requires students to demonstrate their ability to
carry out the skill or technique, or apply the concept.) These included case analysis, secondary data
analysis of internet resources, analysis of datasets, marketing research techniques, observation studies,
and marketing experiments. Student performance on these assessments gave faculty direct evidence of
students’ learning of the principles involved and their ability to apply those principles to real-life marketing
practices. Students indicated that these activities helped them to “put theoretical knowledge into practice.”
Faculty reported that students satisfactorily demonstrated achievement of the objectives for the major,
and indicated that their use of performance assessments and frequent student input would be continued,
with an increase in the use of the internet.
The assessment report for the accounting major indicated the use of weekly quizzes, homework
problems, and four semester exams to assess student achievement of the learning outcomes for the
M. Luebke, Office of Assessment
4
Executive Assessment Summary, 2000-2001
major. Results provided in the report did not indicate in which particular areas student performance was
weak or strong and did not indicate how the curriculum or instruction could be revised to improve student
learning in weak areas. No results of the standardized exam were included in the report, and its
relationship to the learning outcomes for the major was not elucidated.
Department of Communication Studies
Faculty refined the learning outcomes for their communication major into six goals, three that
demonstrate knowledge and understanding and three that reflect ability. These goals are measured by
the success of student conference presentations, student course evaluations, reaction papers to a film,
production of the local access television show, “Detail,” and ongoing faculty discussions. At the
completion of their program, senior communication majors in the required capstone course submitted
papers to the annual Undergraduate Communication Research Conference at St. Thomas University (St.
Paul); 100% of them were accepted and presented in April 2001. One faculty panelist (from another
institution), in response to one of the Concordia University’s group of communication presenters, stated,
“The best piece of undergraduate research I have had the pleasure to read.” Another successful learning
and assessment strategy is the use of a television reality show as a “laboratory” for small group analysis.
In order to address the need to further delineate departmental objectives as they relate to required
courses, faculty are developing a grid that will outline the departmental objectives and in what
classes/assignments those objectives are to be met. This will be incorporated into a portfolio
requirement. The faculty will continue to explore ways to assess intrapersonal communication, and hope
to accomplish this at their first assessment meeting in the fall of 2001.
The frequency and detail of this department’s discussion of curriculum and pedagogy in the process of
assessment seems to be their strongest means of improving their program.
Department of English and Modern Languages
The department of English and modern languages offers two majors, English and secondary English
teaching. Seniors in both majors completed three outcomes assessment activities in spring 2000: the
senior capstone course, ENG499: Framing the Literary Tradition; a written comprehensive exam; and an
oral comprehensive exam. This year faculty noticed significant improvement in student learning
compared to other years, particularly with regard to analysis, interpretation, and synthesis of literary texts,
and the ability to question independently. Faculty speculated that this improvement might be due to
several factors. One factor is their experience in teaching and discussing and improving the capstone
course as a team for several years. Another factor is the on-going refinement of courses in the major to
better reflect the program objectives and to lead to greater ability in students at the capstone level. A
reordered reading list for the capstone also seemed to contribute to students’ improved synthesis of texts.
Faculty noted that performance on the oral exam was still below that of performances on the written exam
and in classroom discussion. In response to this weakness, the Literature Seminar (offered before the
capstone) will incorporate an oral exam as a way to help prepare students for the comprehensive oral
exam in the capstone. Faculty noted that students continue to vary widely in their ability to write clearly,
support arguments, raise questions, and apply learning from one course to another.
This department’s periodic and focused discussion of curriculum, the process of assessing student
learning, and their connections to program outcomes seems to be a strong component in their process of
improving their programs.
Department of History
The department of history reported that outcomes for the major are assessed primarily at the course level
using essay exams, essay assignments, research papers, structured readings, class discussion,
presentations using primary and secondary sources, and, for some students, internship experiences.
Results were similar to last year’s, and indicated that student skills across all program objectives continue
to be mixed, with wide ranging differences in student effort and ability. Some students demonstrate
strengths in skills of inquiry and critical thinking that are critical to the discipline, particularly in
M. Luebke, Office of Assessment
5
Executive Assessment Summary, 2000-2001
assignments that focus specifically on these skills. Faculty again noticed improvement in students’ ability
to conduct historical research based on primary source analysis, and will continue to emphasize these
skills in the coursework. Results also continued to indicate a weakness in writing skills, which limit
students’ ability to carry out the expected activities of the discipline. Students’ skills in computer
technology also varied widely, with strengths in basic applications, but weaknesses in areas specific to
the discipline of history. Faculty continue to value and encourage the benefits of internships, although
their outcomes have been mixed. A notable application of the goal of “practice history in real-life
situations” this past year was student participation in the oral history project, where students interviewed
people about their experiences during WWII. Faculty noted that this learning and assessment activity
demonstrated student achievement of this goal, as well as generated an enthusiastic response from
students.
Department of Music
Academic majors in the music department include music, church music, and K-12 music (vocal and/or
instrumental) teaching. Majors in this department require several of the same assessments, including
semester juries, performance opportunities, a senior project in conjunction with senior seminars (which
serve as the capstone experience), and a standardized outcome exam in music (Major Field Test in
Music). In general, scores for music terminology and identification were lower than for other areas, and
the department plans to explore these measures further. The department is discussing the development
of pre/post tests in theory and history to measure students’ improvement within the program.
Faculty in the music major implemented an audition process for admission to the major and a keyboard
proficiency exam during the 2000-2001 academic year. The audition requirement was developed in
response to assessments of past years, which indicated a need for “gates” earlier in the program so that
students with weak skills who were not likely to be successful could be encouraged to pursue other
majors. The importance of keyboard proficiency was also seen in previous assessments, and the new
requirement provides a common measure of this skill for all music majors. Based on their assessment of
senior recitals and other performances, faculty plan to require students to perform in a “Monday Noon
Recital” during the junior year, to better prepare for the senior recital. Music faculty also saw a need to
develop a clear and consistent assessment tool for measuring success in the semester juries, especially
since these end-of-semester performances are juried by several different faculty members. Faculty plan
to develop a rubric for evaluating performance juries, which will make faculty expectations clear to
students and provide for consistent evaluations across juries.
In the church music major, the same assessments are used in common with the other two majors as
indicated above, with the addition of a field experience. The majority of students showed very satisfactory
skills in musical ability and in their ability to work with congregation members and staff. The difficulty in
finding the right match between student and congregation makes the use of this learning and assessment
activity a continuing challenge for faculty. Church music faculty noted that students seem to focus their
energies on performance, while showing less enthusiasm for music theory or music history, which seems
to be reflected in their mediocre outcomes on the standardized exam. In response to this phenomenon,
faculty proposed greater collaboration between “classroom profs” and “applied music profs” so that
students could demonstrate theoretical and historical knowledge in their performances.
A summary of the assessment report for the K-12 music (vocal and/or instrumental) teaching major
follows in the College of Education section.
Department of Natural Science and Mathematics
The majors offered by this department are biology; natural science, which is comprised of two minors
within the science and math disciplines; environmental science; life science teaching major; mathematics
teaching major; and chemistry teaching major. Because this department has experienced significant
changes in faculty over the last two years and as a result is in the process of undertaking significant
changes in curriculum, assessment reports for majors in this department were not completed. Instead, an
M. Luebke, Office of Assessment
6
Executive Assessment Summary, 2000-2001
assessment plan is being developed by the department, in consultation with the associate dean for
assessment.
Department of Theology
The department of theology offers majors in theology and Christian outreach and a minor in confessional
Lutheranism. To assess student learning in the theology major and confessional Lutheranism minor,
faculty used two methods: a series of pre- and post- essay exams and a study of student scores for four
years on old testament and new testament exams used for seminary admission. Results showed
improvement from the pre- to the post-exams in all categories, and the averages for the seminary
admission exams were above the “passing” level, and showed an upward trend. The faculty noted that
despite improvement in the scores on the post-tests, the degree of variability was the same, suggesting
that the weaker responders did not “catch up” to the higher scorers after they had further coursework.
Faculty also concluded that students do well with Biblical content, but are less skilled in the interpretation
and application of Biblical materials. Faculty who scored the essay exams noticed that the questions on
the exam did not necessarily reflect the outcomes for the major/minor. In response to these observations,
faculty plan the following changes: 1) revise the pre- and post-essay exams to better reflect the stated
program outcomes; 2) revisit administrative procedures for the exams to provide more consistent results;
3) consider restructuring the 8 credits of Bible courses into a 3-3-2 credit sequence; 4) reconsider several
of the learning outcomes in terms of their applicability and assessibility.
Due to a change in personnel, no assessment report for the major in Christian outreach was available.
Department of Social and Behavioral Sciences
The department of social and behavioral sciences offers three majors: psychology, sociology, and
criminal justice.
For the psychology major, assessment of learning in the program includes a standardized outcomes
exam (Major Field Test in Psychology), an internship, and graduating senior evaluation of program. The
results of the Major Field Test in Psychology indicated that students were achieving, in terms of content
knowledge, at a level that is fairly comparable to students in other psychology programs across the
country, although the actual mean scores fell below the national means. In order to foster stronger
performance on this exit exam, the faculty plan several changes: 1) requiring one of the courses on the
physiological aspects of psychology, 2) getting scores to students early by means of SASEs, 3)
encouraging study groups and using introductory texts to prepare for the exam, 4) helping students
connect high performance with reputation for quality on the part of the department and university, 5)
making administration times consistent and available twice a year, 6) using scores to achieve honors in
psychology, 7) not requiring students with a minor to take the exam. Regarding internships, faculty noted
that students continue to be rated high in their ability to connect psychological theory to practice by
supervisors. Faculty plan modifications in internship requirements relating to students with double majors
and those using a work site as an internship experience. Results on the “Graduating Senior Evaluation of
Program” were “almost identical” to the previous two years, with students highly satisfied with their
experiences in the program, particularly the faculty, the curriculum, small class sizes, student involvement
in classes, and the variety of course offerings, but with a continued desire for more psychology offerings
each semester. The faculty also plan to develop a psychology honors program with requirements for
GPA, psychology credits completed, and membership in the national psychology honors association.
In the sociology major, assessments of student learning included a standardized outcomes exam (Major
Field Test in Sociology), SOC453 Social Theory (including major research papers connecting social
issues to theory and research), a senior portfolio, and internship and field supervisor evaluations.
Performance on the standardized exam for the four sociology graduates for whom a score was available
was well above the national mean. A program strength, measured by this exam, portfolios, and course
projects, continues to be research methods and statistics. Faculty plan to encourage students to
enhance research projects begun in SOC454 by enrolling in a one-credit seminar, to further develop their
M. Luebke, Office of Assessment
7
Executive Assessment Summary, 2000-2001
research skills. Internship evaluations by field supervisors continue to demonstrate that students in this
major were able to apply sociological tools to community issues.
The criminal justice major is only a couple of years old, but faculty found the learning assessed by the
internship journals and summary papers and internship supervisor reports to be satisfactory. Faculty
reported that field supervisors were “extremely pleased” with interns, and students’ summary papers
effectively demonstrated the ability to connect theory and research to applied settings. Scores on the
standardized exam (Major Field Test in Criminal Justice) were not yet available at the time of the report,
and the senior portfolio assessment will not be initiated until the 2001-2002 academic year.
Department of Theatre and Dance
Assessment activities in the theatre major primarily take place as a part of required coursework or
participation in theatre productions. Student achievement in course activities and productions is tracked
by means of individualized checklists for each student, which are kept and compiled by faculty advisors.
Faculty also complete course assessments for courses they taught, outlining which of the goals for the
major were addressed in the course and the instructor’s conclusions on the degree to which students
achieved that goal. Each area of specialization (e.g., direction, design, etc.) requires a capstone
experience, and the outcomes of that experience are evaluated and documented by the student’s
capstone adviser. In addition, post-production reviews are held for faculty and students to assess
strengths and weakness in all areas of the production.
Results indicated that faculty continued to see overall improvement in acting skills, production skills,
ability to work effectively with directors and other actors, professionalism in student staged productions,
and analysis of texts, with overall weaknesses noted in writing skills and the oral presentation of ideas.
Faculty discussion of these outcomes led to the following plans for future action: 1) instructors will
continue to review course content with particular attention to improving writing skills; 2) students
participating in productions will continue to be encouraged to register for credit and have their experience
formally evaluated in order to more adequately track participation and learning; 3) computer skills will
continue to be emphasized, especially in design; 4) closer mentoring of students will be attempted in
order to identify weaknesses and provide appropriate skill-building opportunities; 5) an emphasis in
design and increased involvement by students in design work will continue to be developed; and 6)
additional performance opportunities for dance students will be supported.
College of Education
In response to new laws regarding teacher licensure, new mandates from the Minnesota Board of
Teaching, feedback from a national accrediting organization (NCATE), and ongoing assessment, the
College of Education (COE) has undergone significant changes over the last year. These changes
include a revised mission statement and conceptual framework, a reorganized administrative structure, a
fully articulated strategic plan, and redesigned licensure programs (including all the academic majors
affected by licensure), including revised assessment plans. The new assessment plan is in the process
of being implemented and will include the following:
At entrance to professional program:
1) Admission to Program portfolio;
2) content examination (pre-test) – multiple choice;
3) pedagogy examination (pre-test) – essay;
4) admission interview;
5) Pre-Professional Skills Test in Reading, Writing, and
Mathematics (PPST).
At completion of program:
1) Professional portfolio;
2) content examination (post-test) – multiple choice;
3) pedagogy examination (post-test) – essay;
4) presentation of action-research project;
5) Principles of Learning and Teaching test (PLT);
6) standardized content area examination (different for
different fields);
7) Student Teaching evaluations by University
Supervisor and Cooperating Teacher.
M. Luebke, Office of Assessment
8
Executive Assessment Summary, 2000-2001
Results from the assessment activities in place last year were summarized for elementary education,
early childhood education, secondary education, and parent education majors. Student test scores
indicated that the majority of students passed the state-required Prep-Professional Skills Test (PPST).
Scores collected for the Principles of Learning and Teaching indicate that 96% of Concordia students met
or exceeded the Minnesota “temporary” cutoff score. Faculty reported that participating teacher education
students met the requirements for the capstone research project. Students were found capable of
identifying classroom situations to study, integrating the conceptual framework into such study, and
reporting findings. All but 3 students completed and presented projects that were assessed as “passing.”
No changes are planned for this assessment activity.
Majors in K-12 Art Education, K-12 Music Education, and K-12 Physical Education
K-12 Art Education major: No assessment report was available.
K-12 Music Education major: Assessment procedures include student teaching evaluations, admission
portfolio, minimum GPA for the major, coursework and performance assessments throughout their
program, standardized test scores required for all teacher education majors (PPST, PLT), and the music
department standardized exam (Major Field Test in Music). A portfolio requirement was just initiated with
first- and second-year students, so information is not yet available on how that is working. One student
took the newly developed keyboard proficiency exam. Faculty suggested that procedures for both of
these new assessments be made more clear to students. In general, faculty reported variability in
student achievement in both coursework and performance, based on ability, work ethic, and instruction
received. Since there were no students teachers last year, student teacher evaluations by the
supervising and cooperating teachers were not available.
K-12 Physical Education: Assessments included student teaching evaluations, portfolio, and minimum
CGPA and standardized test scores required for all teacher education majors, plus exams, presentations,
papers, and other activities in the methods block courses. All students reportedly complete the student
teaching component with “quality” reports. Faculty noted that students have struggled with the “why and
how” of teaching, and, in response to this observation, faculty plan to place greater emphasis on the
application of content knowledge. Faculty also saw in their assessments that students’ presentation of
current theory is average, and creativity in lesson planning is weak. As a results, some course
assignments are being changed to generate greater application of current theories and more creativity in
lesson planning. Two courses were revamped to focus on more on current research methodology, and to
target the importance for professionals of regular review of current research. The department is also
working to bring the health minor up to a major so that students can be licensed in both areas, bringing
greater marketability to those graduates.
Department of Kinesiology and Health Science
The kinesiology major uses assessments such as course labs, exams, presentations, papers, and
internships. Faculty noted that, on average, assessment results showed that students were able to
adequately describe current theories in the field. As indicated in previous reports, however, they were not
all able to demonstrate higher levels of comprehension, as might be reflected in adept use of the
terminology for the field. Faculty also concluded that students continue to struggle with the application of
current theory to practice. In response to these ongoing results, faculty continue to emphasize the
importance of theories (“how’s and why’s”) to the applied practices of kinesiology. Technology usage
continued to be an area of program strength, and faculty indicated that students have demonstrated
competence with technology appropriate to that in the field at large. Faculty stated that the new research
course being developed will address the needs for greater research skills among students.
Department of Parish Education and Administration
Assessment activities for majors in parish education and administration/director of Christian education
(DCE)/Lutheran church teacher take place throughout the coursework and internship and at the end of
M. Luebke, Office of Assessment
9
Executive Assessment Summary, 2000-2001
the program. Student achievement of the three major program goals is assessed by means of internship
assessment meetings in the congregation, student interviews with faculty, student assessments of
spiritual retreats, Scripture exams, internship supervisor assessments, alumni leadership assessment,
portfolios, and certification interviews and presentations. Faculty noted positive development on the part
of students in the spiritual formation goal, as seen in congregational assessments, spiritual retreat
assessments, student reports of prayer and devotional life, and in the increased scores (70% or higher)
on the exam covering Scriptural knowledge. Internship assessments reflected positive leadership
development in the majority of interns from last year.
Faculty noted that interns who were less successful seemed to struggle with self-management and time
management issues. In response, faculty will be asking students to complete a more comprehensive
self-assessment on health and self-management issues, to write an essay on the educator as decisionmaker, and to present a wellness assessment and plan prior to internship. In addition, the department
has moved to a no-late-papers policy. Professional portfolios, certification interviews, and certification
presentations continue to demonstrate students’ competence in applying and synthesizing material from
the curriculum. In order to match the COE’s changes in mission and model, faculty will modify the
portfolio structure and interview questions and will change the presentation to the “research to action”
project, which will demonstrate professional capabilities. The philosophy of ministry portion will be written
and included in the portfolio rather than be part of the presentation.
College of Graduate and Continuing Studies
Concordia School of Accelerated Learning (CSAL)
Organizational Management
Student learning in the organizational management major is assessed by multiple assignments within the
courses and by the final research project. Faculty noted that feedback from students continues to reflect
their achievement of program goals, such as improvement in problem-solving strategies, values
clarification, personal leadership skills, and the ability to apply this learning to their workplace. Students
reported learning outcomes from the final research project that include project planning and time
management, moving from problem formulation to problem analysis, application of research and research
methods to a problem, and solution development. One assessment goal suggested in the last report is
the development of a course evaluation tool that is more reflective of what students have learned in the
course; this is suggested again in the current report. It was suggested again that the curriculum be
reviewed with faculty from other majors in CSAL to consider revisions in light of changing needs in the
business world.
Innovation and Marketing Management
Students in the innovation and marketing management major integrated learning from coursework into
several papers, including a marketing plan, e-commerce plan, promotional plan, or product launch plan,
and a full-fledged business plan, which serve as assessments of the breadth and depth of their
achievement of the program goals. In addition, students complete evaluations of courses and instructors,
and participate in an exit interview. Faculty noted that papers and business plans over the last year
continue to exhibit the skills, synthesis, and theory applications implicit in these activities. Faculty
observed that speaking and writing skills improved markedly over the 18 months of the program, through
extensive feedback from instructors and peers. Case studies continue to be used to stimulate higherlevel problem-solving and decision-making skills and to assess students’ development of those skills.
Faculty again noted that most students were able to meet the challenge these cases present, but that
faculty need to continually update cases to be responsive to students’ desire to tackle current marketing
problems. In response to student feedback and demands of the field, the curriculum now includes a goal
area for integrating e-commerce techniques, where students develop an e-marketing plan. Assignments
and final projects demonstrate student students’ synthesis of marketing and e-commerce strategies.
Exit interviews with students and course evaluations continue to show that students value the highly
applicable nature of the learning and assessment activities in the innovation and marketing management
major and the improvement they see in their abilities as a result of their experience in the program.
M. Luebke, Office of Assessment
10
Executive Assessment Summary, 2000-2001
Students also consistently report that they enjoy the learning process in this program more than they ever
have, making learning more positive for them, with more students at the end of the program interested in
pursuing graduate school.
Information Technology in Management
The major in information technology in management integrates technology and business management.
Course-level assessments, including papers, hands-on projects, and formal presentations, required
students to demonstrate their command of the technology and to synthesize it within the business
context. Students demonstrate learning in a final applied research project, where they were required to
synthesize the materials from the entire program, demonstrate their ability to analyze problems and
opportunities, apply a structured systems analysis, evaluate hardware/software design capabilities,
address human factors, provide a cost analysis and formal documentation, and present the project with
effective oral and written skills. Faculty noted that the 1999-2000 revisions to the applied research project
guide seem to have provided the clarity, structure, and focus students needed to complete solid,
impressive projects. Based on these course assessments and the demands of the field, the project
management component was expanded and moved up in the curriculum to allow for more training in the
software and better application to the final project.
Problem-solving and ethical decision-making are also goal areas that were assessed in projects or case
studies. On course evaluations, students continued to remark on their increased confidence regarding
decision-making and conflict resolution, and indicated a heightened awareness of the ethical implications
to their decisions. Faculty again observed improvement in students’ group problem-solving and decisionmaking skills throughout the program. Faculty also noted that speaking and writing skills improved
significantly throughout the program, although a few students needing writing help did not take advantage
of available resources.
Human Resource Management
The major in human resource management is brand new, so no assessment data is yet available. Plans
for assessing program outcomes include writing assignments, presentations, problem-solving exercises,
and final project, which is a major synthesis and problem analysis paper with a particular human resource
management focus. The department chair will conduct exit interviews with each cohort prior to
graduation.
Concordia School of Human Services (CSHS)
This unit offers three undergraduate majors using distance learning technology: school-age care, youth
development, and human service.
School-Age Care
The major in school-age care assesses learning using course synthesis reflections (CSR), professional
portfolios, course projects and assignments, and exit interviews. Faculty reported that they were satisfied
that student portfolios and course synthesis reflections demonstrated student achievement of program
goals, but noted that some modifications have been made in the curriculum in response to some
unsatisfactory outcomes. Faculty noted the continued program strength in the area of teaching social
skills, likely the result of course improvements and additions added last year, which have had an
observable impact on student learning in this area.
Students also develop school-age care program guides, which require students to synthesize and apply
the learning from material across the program. Faculty asserted that the resulting manuals regularly
demonstrate that program graduates are clearly leaders in the field of school-age care. One program
objective has been revised and numerous course assignments have been redesigned in response to
ongoing assessments. To facilitate the use of all this assessment data, the department chair sends a
letter to the regular faculty teaching in the program highlighting strengths and areas that need to be
changed as seen in the exit interview with each cohort, and a review of the major assessments.
The department chair also meets with faculty of individual courses to ensure that course components
match program objectives and are revised in accordance with assessment information.
M. Luebke, Office of Assessment
11
Executive Assessment Summary, 2000-2001
Youth Development
Course assignments and final portfolio and synthesis are used to assess student achievement of program
outcomes for the youth development major. Faculty reported that student bulletin boards, e-mail
reflections, assignments, and papers reflected students’ ability to understand and apply coursework to
their work with youth, but no particular improvements or modifications were noted.
Human Service
The major in human service is new, and the initial cohort has not yet finished the program, so no
assessment data has been compiled. Portfolios will be a primary synthesis, application, and assessment
feature of this major. Course evaluations, instructor feedback, and student feedback have already
resulted in two courses being replaced by two new courses. The department chair will conduct exit
interviews with the cohort and the feedback will be integrated into programmatic change.
Graduate Studies
Master of Arts in Organizational Management
The Master of Arts in organizational management program continues to use instructor evaluation of
student performance, a thesis activity, a portfolio, and student self-evaluations to assess students’
achievement of the program outcomes. Faculty noted again that committees who evaluate and advise
students on their thesis activity continue to find that overall theses demonstrated students’ ability to
undertake a self-directed, synthesizing activity and to engage in in-depth, higher order thinking. Faculty
again reported that instructors in this program meet informally to discuss students’ performance and
factors in their success, but the report did not indicate any particular improvements made in response to
assessment results. Faculty reported that evaluation of students and students’ self-reports indicated
satisfactory achievement of program goals.
Master of Arts in Education, with an Emphasis in Early Childhood Education
The Master’s degree program in early childhood education assesses outcomes by means of the written
capstone research project and critical dialogs, two synthesizing seminars, and ongoing coursework.
Faculty described numerous revisions in the curriculum and instruction made in response to these
assessments. Based on results from on-going assessment in the research core, faculty continue to
modify course activities in order to help students grasp the challenging content of research methodology.
New texts have been adopted and others have been written by program faculty, and courses have been
restructured to assist students in building on their work for the capstone project.
Written responses to case studies, responses in on-line discussion forums, and applied course
assignments continue to demonstrate students’ thorough understanding and appropriate application of
theories, and competent skills in decision-making, observation, and assessment tool development.
Faculty continue to note that the program goal of collaboration with other professionals was clearly seen
in assignments requiring interaction with classroom teachers and others in early-childhood environments
and in on-line discussions with classmates during and following the program. Feedback from students
and instructors continues to confirm these outcomes.
Master of Arts in Education, with an Emphasis in School-Age Care
Significant changes were made in this program last year, and the first cohort to begin after these changes
were implemented has not completed the program. However, faculty report that analysis of the
coursework of this group so far confirms that the changes made to the curriculum over the last nine
months have made a significant impact on students’ ability to make effective decisions in their profession.
A case study approach to problem solving and decision making was initiated and the same case study
was revisited in a later course. Faculty observed that this method resulted in thorough comprehension
and application of theory to a case problem and effective decision making for solutions. Visioning papers
were integrated throughout the curriculum, culminating in a leadership course where students use all their
vision papers to develop a strategic plan for program development. In addition to these changes, new
M. Luebke, Office of Assessment
12
Executive Assessment Summary, 2000-2001
faculty have been hired to coordinate research and teach core courses, and several courses have been
revised to focus on areas where gaps had been found. Faculty have found that students’ research skills
are increasing with the research component, and students in the program will be leading a seminar on
their current research projects at the National School-Age Care Conference 2002.
Master of Arts in Education, with an Emphasis in Youth Development
The assessment report for this program was incomplete.
Master of Arts in Education with an Emphasis in Family Studies
This program is less than two years old, and the first cohort has not yet completed the program at the
time of the assessment report. However, the assessment report submitted outlined a clear assessment
plan and demonstrated ongoing assessment and evaluation of courses, student performance, and
curriculum. As part of program assessment, faculty will analyze composites of student goals, course
synthesis reflections, course evaluations, and ongoing evaluations to identify areas of program strength
and needs for further program development. Data analysis of recurrent themes in the feedback will
inform program expectations and modification. Ongoing development of effective strategies for meeting
course objectives will be a primary goal of the assessment review process for this program. The
department chair has also developed a matrix that clearly articulates where program objectives are met
within individual courses, which has already been used as a model by other departments.
Master of Arts in Criminal Justice
The criminal justice master’s is new, and the initial cohort has not yet finished the program, so no
assessment data has been compiled. Portfolios will be a primary synthesis, application, and assessment
feature of this program. Course evaluations, instructor feedback, and student feedback are being used to
revise the curriculum on an ongoing basis. The department chair will conduct exit interviews with the
cohort and the feedback will be integrated into programmatic change.
Learning and Development Beyond the Classroom
One of the components of the University assessment program is the annual administration of a student
survey as an indirect measure of students’ experiences with Concordia University. Last year, Concordia
participated in the National Survey of Student Engagement, a project administered by the Indiana
University Center for Postsecondary Research and Planning and supported by a grant from The Pew
Charitable Trusts and cosponsored by The Carnegie Foundation for the Advancement of Teaching and
The Pew Forum on Undergraduate Learning. The survey is intended to focus on learning-centered
indicators of quality by asking undergraduates about their college experiences, e.g., how they spend their
time, what they get out of their classes, the nature of their interactions with fellow students and faculty,
and so on. Research has shown that good educational practices and quality interactions with peers and
faculty in and outside the classroom, which these kinds of indicators can reflect, are directly related to
successful student outcomes.
The results overall showed that Concordia University performed at or above the national mean in these
indicators of quality educational experiences. In fact, Concordia was also identified by project analysts as
“a strong performer” on the Active and Collaborative Learning benchmark.
National Survey of Student Engagement 2001
During spring 2001, 225 Concordia University, St. Paul seniors and first-year students participated in the
National Survey of Student Engagement (NSSE). Over 70,000 students from 321 colleges and
universities, including five other Concordia University System institutions, completed the survey.
Questions examined the frequency and types of experiences in which students engaged that are
considered valuable for their academic and personal growth and indicative of institutional quality. Authors
M. Luebke, Office of Assessment
13
Executive Assessment Summary, 2000-2001
of the NSSE 2001 Overview noted that institutions fostering high levels of student engagement
emphasize diversity, have supportive faculty members, and have good advising. The Concordia, St. Paul
community values these characteristics, as seen by its strategic priorities, and many of the results
indicate that Concordia is successful in living out many of these values.
Survey results are generally positive for Concordia, St. Paul. When viewing results through a comparison
of mean responses, CSP means were comparatively better on a vast majority of the 62 national items.
The difference in means was statistically significant and favorable to CSP compared to the aggregate
sample of the other Concordia institutions and to the national sample on a number of items, and
significant and unfavorable to CSP on only a couple of items.
Results show that Concordia, St. Paul first-year students engaged in an above average amount of
interaction with each other and with faculty members both inside and outside of classes. The CSP
climate for first-year students actively promotes communication at all levels. CSP seniors read more
assigned books and similar readings and wrote more papers and reports than seniors at comparison
institutions. Differences in means were the greatest, for both seniors and first-year students, on
questions related to use of technology. It appears that the laptop university initiative has engaged
students in the educational uses of technology at a level above the national norm.
Responses related to diversity reveal a campus community that is generally more diverse and
emphasizes diversity issues more than the other Concordia institutions participating in the survey. For
example, significantly more CSP students indicated that they had had serious conversations with
students of a different race or ethnicity. These results were about the same as or slightly lower than the
national average for these characteristics. CSP students engaged in community-based class projects
with about the same frequency as the comparison groups, but results suggest there is an interest on the
part of CSP students in engaging in more community service and service learning.
Twenty questions were directed toward participating Concordia institutions only, most of them dealing
with religious/spiritual issues and issues related to life styles and values. The only significant difference
found was that CSP first-year students attended Sunday worship less frequently than their counterparts at
the other Concordia institutions. This may be partially due to the fact that the other Concordia institutions
had a higher percentage of church work respondents. Responses were generally similar on other
questions in this section, revealing similar campus climates in these areas.
Despite generally positive results on individual items, responses to two questions related to overall quality
and student satisfaction were mixed. For example, 86% of CSP’s first-year students described their
educational experience at Concordia as good or excellent, yet only 75% said they would likely start over
at CSP if they had the opportunity to do so. (The actual attrition rate for this group was 39%.) This
response contributes to Concordia’s retention concerns. Eighty-five percent of CSP’s senior respondents
rated their education at Concordia as good or excellent, and 82% indicated that would start over at CSP if
they could do so. These percentages for seniors are similar across respondent groups.
(This summary of NSSE results was compiled by David Stueber, director of institutional research. See
Appendix II for his more detailed summary of Concordia, St. Paul survey findings.)
No other reports were available from non-academic units regarding assessment activities of student
development in non-academic areas.
Review of the Assessment Process at Concordia University
The University assessment committee is comprised of one faculty member from the college of arts and
sciences, one from the college of education, one from the college of graduate and continuing studies, one
professional staff member from student affairs, the director of institutional research, and the associate
M. Luebke, Office of Assessment
14
Executive Assessment Summary, 2000-2001
dean for assessment. Since the University’s assessment plan includes some form of external review of
institutional assessment processes every three years, a consultant was enlisted last year to review our
assessment policies and procedures.
The consultant, Michael Miller, Ph.D., associate dean for assessment and program development from
Gustavus Adolphus College, reviewed documents outlining the assessment plans and procedures in
place at Concordia University. He then spent one morning with the assessment committee discussing
our activities and their relationship to other university processes like accreditation and the professional
development of department chairs. His statements to the committee indicated that he found Concordia
University’s assessment policies and procedures to be appropriate, in line with the expectations of most
accrediting bodies, and in many cases, far ahead of other similar institutions. He stated the importance of
documenting assessment activities, and noted that Concordia University’s process of departmental
reports is an excellent way of providing that documentation. He suggested that assessment of student
learning become an intentional part of the professional development of and expectations for academic
department chairs. Subsequently, the university assessment committee proposed that an inservice or
workshop be created for academic department chairs that would further develop their ability to manage
and utilize assessment for improved quality in curriculum, instruction, and student learning. Such an
activity is still under consideration.
The primary task of the assessment committee for the last three years has been to individually review
each of the 30 or more reports of assessment activities in the majors and programs. Needless to say, this
task was extremely time-consuming, and made it difficult to attend to other matters that came under the
purview of the committee. In response to this difficulty and in order to expand the input departments
received about their reports, the process for reviewing these annual reports was revised this past year,
and now involves small groups of faculty peers meeting together to discuss their reports and their
findings. As stated earlier in this report, the new process is intended to facilitate broader sharing of
assessment strategies across disciplines and departments and to make it a more collegial activity
focused on sharing what works to improve student learning. Members of the assessment committee
each facilitated the small group sessions, and participants discussed their struggles and triumphs with
assessment and student learning in their majors and programs.
In the previous executive summary of assessment, for 1999-2000, the assessment committee included
the following recommendations:
1. Reward faculty who integrate assessment into their academic work and document assessment
practices in courses and programs.
2. Provide additional training for faculty and staff, especially those in leadership positions, in the
principles and best practices of assessment in higher education, e.g., through attendance at
national assessment conferences.
3. Encourage leadership that is actively involved in assessment activities and promotes the timely
documentation of those activities.
4. Provide for an institutional research position, where assessment and research projects such as
surveys could be coordinated, analyzed, and interpreted.
In response to the ongoing need for institutional data and research, the fourth recommendation was
carried out. In July 2001, a new position, director of institutional research, who reports to the associate
dean for assessment, was created in the academic affairs unit. The individual in this position will be
responsible for coordinating external demands for institutional data, will act as a clearinghouse for
university information, and will assist with institutional needs for data gathering and analysis. This
individual will also assist and inform the university assessment committee’s efforts to gather and manage
data regarding student learning and development.
The other recommendations that were made by the committee were not formally acted upon, but will
return to the agenda this year for further development.
M. Luebke, Office of Assessment
15
Executive Assessment Summary, 2000-2001
References
American Association for Higher Education. (1992). Principles of good practice for assessing student
learning. Washington, D.C.: Author.
Indiana University Center for Postsecondary Research & Planning (2001). National survey of student
engagement: Overview. Indianapolis, IN: Author.
Lopez, C.L. (1999). Assessing student learning: Why we need to succeed. Assessment and
Accountability Forum, Summer1999, 5-7/18.
Lopez, C.L. (March 1996). Opportunities for improvement: Advice from consultant-evaluators on
programs to assess student learning. Chicago: North Central Association of Colleges and Schools
Commission on Institutions of Higher Education.
North Central Association of Colleges and Schools Commission on Institutions of Higher Education.
(1996). Commission statement on assessment of student academic achievement. Chicago:
Author.
Terenzini, P.T. (1989). Assessment with open eyes: Pitfalls in studying student outcomes. The Journal of
Higher Education, 60, 644-664.
Wolff, R.A. & Harris, O.D. (1994). Using assessment to develop a culture of evidence. In D. F. Halpern &
Associates (Eds.), Changing college classrooms (pp. 271-288). San Francisco: Jossey-Bass
Publishers.
M. Luebke, Office of Assessment
16