Assessment Report Standard Format
July 1, 2006 - June 30, 2007
PROGRAM(S) ASSESSED __M.S. in Social and Applied Economics____
ASSESSMENT COORDINATOR __Leonard J. Kloft_________________
YEAR _____3____ of a _____3____YEAR CYCLE
1. ASSESSMENT MEASURES EMPLOYED
1. Internship Report Review: A subcommittee of the Graduate Studies
Committee (Department of Economics) reviewed the completed
internships for the academic year 2006-2007. Each subcommittee
member evaluated each internship based on the following criteria:
a) Statistical Analysis Capabilities
LO-2, 3
b) Writing and Other Communications
LO-3
c) Creativity and Originality
LO-2, 4
d) Analytical (Modeling) Skills
LO-3, 4
e) Knowledge of Economic Theory
LO-1
f) Knowledge of Economic Institutions
LO-1, 2
2. Core Course Key Skill Assessment: Each faculty member, who
teaches a core course in the M.S. in Social and Applied Economics
Program (faculty who teach elective courses can but are not required
to submit key skill assessment forms) for the academic year, 20062007, completed a key skill evaluation of each student based on the
following criteria:
a) Statistical Analysis Capabilities
LO-2, 3
b) Writing and Other Communication Skills
LO-3
c) Creativity and Originality
LO-2, 4
d) Knowledge of Economic Theory
LO-1
e) Knowledge of Economic Institutions
LO-1, 2
f) Other Key Skills (Identified on the form)
g) Comments
3. Major Papers Portfolio: A subcommittee of the Graduate Studies
Committee (GSC - Department of Economics) reviewed each student’s
major papers and projects completed during the student’s progress
through the program for the academic year 2006-2007. Each
subcommittee member evaluated each portfolio based on the following
criteria:
a) Statistical Analysis Capabilities
LO-2, 3
b) Writing and Other Communications
LO-3
c) Creativity and Originality
LO-2, 4
d) Analytical (Modeling) Skills
LO-3, 4
e) Knowledge of Economic Theory
e) Knowledge of Economic Institutions
f) Improvement
LO-1
LO-1. 2
2. ASSESSMENT FINDINGS
OUTCOMES:
1. Students will develop an understanding of economic theory. (LO-1)
2. Students will integrate and apply their knowledge to public policy,
business problems, and social issues. (LO-2)
3. Students will be proficient in analyzing and communicating
quantitative and qualitative information. (LO-3)
4. Students will demonstrate the ability to apply economics to solving
problems. (LO-4)
Internship Report Review
The Subcommittee’s purpose in reviewing the internships for 2006-2007
is to determine the degree to which the internship program is meetings its
critical role in fulfilling the M.S. in Social and Applied Economics
program’s mission: to prepare students with a professional skill set so
they can be successful as applied economists upon entering the work
force. The assessment of the internships evaluated 6 skill categories on a
scale of 1 (low) to 5 (high).
Table 1: Internship Reports Assessed 2006-2007
Mean
Standard
Deviation
Statistical
Analysis
Capabilities
Writing and
Other
Communication
Skills
Creativity
and
Originality
Analytical
(Modeling)
Skills
Knowledge
of
Economic
Theory
Knowledge
of
Economic
Institutions
3.92
3.92
3.67
3.67
3.83
3.75
0.665
0.801
0.983
1.033
0.516
0.418
The preceding table reports the GSC subcommittee’s review of the 20062007 Internships. The average scores range between 3.5 (high average)
and 4.0 (strong) for the various skill sets. The average score for all skill
sets combined is 3.79 (0.721). Statistical and Analysis Capabilities and
Writing and Other Communication Skills represent the strongest skill sets.
In general, the results suggest that the internship component of the M.S.
program is meeting its objective of preparing students with a professional
skill set they will need when they enter the job market.
Table 2: Internship Reports Assessed 2004-2005 and 2005-2006
Statistical
Writing and
Creativity Analytical Knowledge Knowledge
Year
Analysis
Capabilities
2005-06
2004-05
4.375
3.92
Other
Communication
Skills
3.875
3.67
and
Originality
3.875
3.54
(Modeling)
Skills
of
Economic
Theory
3.75
3.54
of
Economic
Institutions
3.40
3.67
When the Internship Report results for 2006-2007 are compared to 20052006, three of the skill sets have a higher score, two skill sets declined,
and one skill set was added to the review. None of the skill sets were
below the 2004-2005 levels. The 2004-2005 report identified two areas of
improvement for the program, Writing and Other Communication Skills
and the Internship. Based on the increase in Writing and other
Communication skills average score, the increased writing and
presentation activities added to courses in the M.S. in Social and Applied
Economics appear to have had some impact. In addition, the increase in 4
of 5 category skills from the 2004-2005 report suggests that the Internship
component of the program continues to improve.
Course Key Skills Assessment
The M.S. in Social and Applied Economics program requires a minimum
of 48 credit hours, which include eight core courses, three elective
courses, and the internship. Each faculty member who teaches a core
course in the M.S. in Social and Applied Economics program is provided a
key skills assessment form for each student. Faculty members who teach
elective courses are provided with the key skill assessment forms but are
not required to submit the forms. Forms were returned for students in the
core courses:
EC 724 Development of Economic Thought (Fall 2006),
EC 709 Applied Econometrics (Winter 2007),
EC 725 Economic, Sociological, and Ecological Economic
Systems (Winter 2007),
EC 712 Forecasting Economic Activity (Spring 2007), and
EC 726 Contemporary Political Economy (Spring 2007).
Key Skills assessment forms also were submitted for the elective courses:
EC 740 Cost Benefit Analysis (Fall 2006), and
EC 783 Internship Prep (Spring 2007).
Table 3: Key Skills Course Assessment 2006-2007
Mean
Standard
Deviation
Statistical
Analysis
Capabilities
Writing and
Other
Communication
Skills
Creativity
and
Originality
Knowledge
of
Economic
Theory
Knowledge
of
Economic
Institutions
Other
Key
Skills
4.35
4.19
4.28
4.26
4.30
5.00
0.812
0.921
1.007
1.059
0.954
-
Table 3 presents the scores for the core course key skills, which range
from 1 to 5, where 5 is the highest. Statistical Analysis Capabilities
represents the highest rated skill set, 4.35, although the other skills
categories have very similar scores. The Other Key Skills is an open
category for which a professor can report a skill which is not part of the
reviewed skill set. One professor chose to report Other Key Skills
(Presentation, Critical Thinking, etc.).
Since the M.S. in Social and Applied Economics Program enrolls both
full-time and part-time students, the information from Table 3 is separated
according to full-time status (Table 4) and part-time status (Table 5).
Table 4: Key Skills Course Assessment 2006-2007: Full-time Students
Mean
Standard
Deviation
Statistical
Analysis
Capabilities
Writing and
Other
Communication
Skills
Creativity
and
Originality
Knowledge
of
Economic
Theory
Knowledge
of
Economic
Institutions
Other
Key
Skills
4.45
4.21
4.35
4.35
4.34
5.00
0.783
0.965
1.044
1.089
0.998
-
Table 5: Key Skills Course Assessment 2006-2007: Part-time Students
Mean
Standard
Deviation
Statistical
Analysis
Capabilities
Writing and
Other
Communication
Skills
Creativity
and
Originality
Knowledge
of
Economic
Theory
Knowledge
of
Economic
Institutions
Other
Key
Skills
3.80
4.10
4.00
3.90
4.11
5.00
0.837
0.738
0.816
0.876
0.782
-
From Tables 4 and 5, full-time students receive higher evaluations on
average than the part-time students. Table 6 and Table 7 report the Key
Skills Course Assessment information for 2005-2006 and 2004-2005
respectively. Comparing these tables to Table 3, 2006-2007 has higher
scores than previous years. These findings do not imply necessarily that
the program has engendered substantial improvement in Key Skills since
there are number of core courses for which Key Skills assessment form
were not received.
Table 6: Key Skills Course Assessment 2005-2006
Mean
Standard
Deviation
Statistical
Analysis
Capabilities
Writing and
Other
Communication
Skills
Creativity
and
Originality
Knowledge
of
Economic
Theory
Knowledge
of
Economic
Institutions
Other
Key
Skills
3.74
3.77
3.59
3.62
3.67
4.12
0.876
1.137
1.012
0.844
0.953
1.107
Table 7: Key Skills Course Assessment 2004-2005
Mean
Standard
Deviation
Statistical
Analysis
Capabilities
Writing and
Other
Communication
Skills
Creativity
and
Originality
Knowledge
of
Economic
Theory
Knowledge
of
Economic
Institutions
Other
Key
Skills
3.88
3.25
3.35
3.29
3.08
3.06
1.038
1.051
1.114
1.071
0.989
1.459
Given the information gathered for 2004-2005, 2005-2006, and 2006-2007
the M.S.in Social and Applied Economics Program provides a good
knowledge of economic theory (LO-1), a good knowledge of public
policy, business problems, and social issues (LO-2), a very good ability to
analyze information (LO-3) and good ability to communicate information
(LO-3), and a good ability to solve problems (LO-4).
In summary, the M.S. in Social and Applied Economics Program, as
reflected in the 2006-2007 course key skills review, not only is achieving
its specified learning outcomes but is improving the quality of its
delivered course key skills. Based on the course key skills review, the
M.S. in Social and Applied Economics Program equips its students with a
portfolio of skills, which they will need to have successful careers as
economists.
Major Papers Portfolio:
Whereas the Internship Review examines the program’s final product and
the Core Course Key Skills reviews skills development course by course,
the Major Papers Portfolio assesses longitudinally core skills
development. Papers and projects were collected from the following
courses:
EC 724 Development of Economic Thought,
EC 712 Applied Forecasting, and
EC 726 Contemporary Political Economy.
A subcommittee of the Graduate Studies Committee reviewed each
student’s papers/projects portfolio. Each portfolio is assessed on the same
key skills components as the Core Course Key Skills except that the Other
Key Skills category was replaced with Improvement. The Improvement
category strives to capture the reviewers overall assessment of
improvement from the start of the program to the completion of the
program. Student portfolios were considered only if the student had items
from each quarter evaluated.
Table 8: Major Papers Portfolio Assessed 2006-2007
Mean
Standard
Deviation
Statistical
Analysis
Capabilities
Writing and
Other
Communication
Skills
Creativity
and
Originality
Analytical
(Modeling)
Skills
Knowledge
of
Economic
Theory
Knowledge
of
Economic
Institutions
3.75
3.54
3.58
3.50
3.63
3.73
0.500
0.689
0.417
0.603
0.433
0.410
The number of student portfolios reviewed is small given that only a few
courses are represented and a significant number of students left the
program during both Fall and Winter quarters while a significant number
of students entered the program during both Winter and Spring quarters.
A cohort of students for Fall, Winter, and Spring quarters to review yields
a relatively small number of students and therefore can lead to unreliable
results. The improvement rating from Fall to Spring quarters was a 3.75
average across the portfolios.
Table 9: Major Papers Portfolio Assessed 2004-2005
Mean
Standard
Deviation
Statistical
Analysis
Capabilities
Writing and
Other
Communication
Skills
Creativity
and
Originality
4.00
3.37
0.938
1.027
Analytical
(Modeling)
Skills
Knowledge
of
Economic
Theory
Knowledge
of
Economic
Institutions
3.63
3.56
3.45
1.104
0.929
0.923
The data set for 2005-2006 was too small to use as a basis for reliable
comparison. However, comparing the 2006-2007 results to the 2004-2005
results does show improvement in three categories and decline in two
categories. Once again, the small number of cohort student portfolio
papers available for 2006-2007 does reduce the ability to garner useful
insights from the Major Papers Portfolio.
3. PROGRAM IMPROVEMENTS
After reviewing the various assessment findings, the Internship Review
and the Core Course Key Skills Review suggest unambiguously that the
M.S. in Social and Applied Economics program during academic year
2006-2007 did a good job at meeting the learning outcomes specified.
The M.S. in Social and Applied Economics program is succeeding in its
mission of equipping its students with a portfolio of skills, which they will
need to have successful careers as economists. Although, the Major
Papers Portfolio Review does not include a large number of portfolios, the
results for 2006-2007 do not differ substantially from 2004-2005 and
depict an upward trend in the average skill scores.
In pursuit of continuous improvement, the M.S. in Social and Applied
Economics program desires to improve in the following areas for the
2007-2008 academic year:
1. Upgrade the quality of communication skills through increased
practice of writing and presentation skills (more project
oriented assignments with presentation requirements.
2. Improve the Internship process:
a. Identify student areas of interest sooner.
b. Provide a more balanced distribution of internship
supervision responsibilities among the faculty.
c. Expand the practitioner focus of the internship
(incorporate Executive Summary).
3. Create better compliance:
a. Submission of Core Course Key Skill forms
b. Submission of Student Papers for the Portfolio
4. Review the Assessment Plan for its efficacy in determining the
success of the M.S. in Social and Applied Economics program.
4. ASSESSMENT PLAN COMPLIANCE
The assessment plan as developed in 2004 continues to be undertaken as
written. Internship Report assessment occurred as originally planned.
Given the increase in students for the program in 2007-2008, more faculty
participation in the review must occur. Core Course Key Skills
assessment occurred as originally planned. Better compliance by faculty
in submitting their individual student assessment forms is required. Three
of the eight core courses did not provide assessment forms for the students
enrolled in the courses. The Paper Portfolio assessment occurred, but is
the weakest link in the assessment process. Only three courses provided
papers for the review. Given that the Paper Portfolio assesses the
improvement in the student’s papers longitudinally, the student must be
enrolled in courses during each of the academic year quarters. The winter
quarter did not have any papers added to the portfolio.
5. NEW ASSESSMENT DEVELOPMENTS
The Graduate Studies Committee plans to review the assessment plan
during the 2007-2008 academic year to determine the efficacy of the
current plan.