A model of Assessing Competencies of
Undergraduate Business
Students at Grant MacEwan College
Lyle Benson and Davar Rezania
2005-2008 Processes
1.
2.
3.
4.
5.
6.
7.
8.
9.
Definition of learning outcome
Competency profile
Professional skills
Performance benchmarks
Curriculum mapping of benchmark to courses
Faculty team analysis
Revisiting professional skills
Data collection
Preliminary results
Definition of learning outcome
2005: The literature reviews from peer-reviewed
journals on the following databases:
• ERIC
• Expanded Academic ASAP
• Academic Search Premier
• Business Source Premier
• MasterFILE Premier
The search targeted material dated from 2000
onwards.
Definition of learning outcome
•
There is overwhelming agreement in the literature that
the skills that will help business graduates succeed in
the workplace go far beyond the content area
knowledge traditionally associated with schools of
business. The professional skills that are most
frequently mentioned are the ability to think abstractly,
analytically and critically; writing and presentation
skills; the ability to work in a team; interpersonal skills
including the ability to handle conflict or criticism;
technology; and information literacy (Taylor, 2003;
Vice and Carnes, 2001, Harpe, Radloff & Wyber, 2000;
Candy, 2000). In fact, employers are quoted as valuing
these “soft” or professional skills more highly than
specific functional knowledge (Taylor, 2003).
Definition of learning outcome
•
•
“soft” skills will tend to be lost if they are
not made explicit in the curriculum
Montano, Anes, Hassall, & Joyce, 2001
and Milne and McConnell (2001).
Writers note a perception among
students and faculty that relational and
behavioural themes are less legitimate
than content themes (McAlister; Medline,
Graves, & McGowan, 2003).
Definition of learning outcomes
•
•
these generic skills should be attained in an
integrated way (Bath et al., 2004; Arquero,
Anes, Hassall & Joyce, J., 2001; Harpe et al.,
2000).
learning occurs most effectively in context so
that professional skills need to permeate the
curriculum. Discipline knowledge and
professional skills are interdependent (Robley,
Whittle & Murdoch-Eaton, 2005; Bath et al,
2004; Harpe et al, 2000; Johnston & Webber,
2000).
Competency profile
1. What differentiates a high performing
undergraduate student from an average
undergraduate student?
Professional skills (2005)
1. Ethical Practice
2. Presentation Skills
3. Writing Skills
4. Research Skills
5. Group work Skills
6. Case Study Analysis Skills (critical thinking)
7. Technology Skills
* Note also ASCSB requirements for
undergraduate degrees.
Performance benchmarks
1. Benchmarks were developed for each
professional skill for years 2,3, and 4
(see handouts)
Curriculum mapping of benchmark
to courses
1. Ethics example
page 10 Another way students learn about ethical behaviours
is through the ethical culture they observe in their respective
business schools. Students cannot be expected to internalize
the importance of ethics and
values unless business schools demonstrate such
commitment within their own organizations. This means that
business school deans need to think of themselves as ethical
leaders who communicate regularly about ethics and values;
who model ethical conduct; and who hold community
members—faculty, staff, and students—accountable for their
actions. Academic policies and systems should clearly be an
integral, living part of the school’s culture, and not simply a
stack of documents in the file drawer.
70 (s
6.1)
page 17 FROM THE PAST AACSB STANDARDS
(Revised February 14, 2001) – Page 17
54 (s.
4.5),
2. :7. Ethical conduct – The
upholds integrity and ethica
demonstrated by the releva
by which it conducts its bus
ethical policies in place gov
recruitment of students an
evaluating and awarding ac
Faculty team analysis (2007-2008)
1. Review professional skills in 2nd/3year
core courses
2. Show assessment methods with marking
rubrics for each professional skill
3. In each major review curriculum maps for
all professional skills and create plan for
addressing identified gaps
Example of Marking rubrics
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
TEAM PROCESS EVALUATION (15%)
______________________________________________________________________
Section A - Course Content
/ 20 marks - Tuckman’s Analysis of your group.
- remember to do research on Tuckman and in text citations and references.
- identified two examples of the group’s behaviour within each stage.
- identified two examples of your behaviour which you did in each stage, which helped your group’s process be more effective. You cannot simply
be an observer or do work.
- 20- excellent insight and analysis, 17 - very good analysis, 15 - good analysis, 13 - basic analysis
/20 marks
Section B - Professional Skills
1. Group Work
/ 5 marks - In-class 360 Degree Feedback – Keep Doing, Start Doing, and Stop Doing
- this is based on the 360 Degree face –to-face feedback you received from your Team Member Evaluation and the insights you gained about your
behaviour in the group about what do you have to Keep Doing, Start Doing, and Stop Doing.
- 5- excellent insight and analysis, 3 - good analysis, 2 - basic analysis
/ 4 marks – Chairing Two Meetings
- prepare and chair two team meetings and report on them.
- 4- excellent insight and analysis, 3 - good analysis, 2 - basic analysis
/ 4 marks - Final 360 Degree Feedback
- this is based on the final out-of-class 360 Degree face-to-face feedback you received from your Team Member Evaluation.
- determine the contribution mark of each group member for the written research paper.
-what was the feedback given to you about how you participated as a team member? What have you learned about yourself and group processes that
you can apply to other MacEwan course projects, work situations, etc.?
- 4- excellent insight and analysis, 3 - good analysis, 2 - basic analysis
/ 3 marks - Time Sheet Activity
- list each week, record activities for each day of the week, and record hours worked on each activity.
3- excellent activity report, 2 - good activity report, 1 - basic activity report
/16 marks
Example of Marking rubrics
•
•
•
•
•
•
2. Writing Skills
Centered title page with course name, #, and section.
Student’s name and instructor’s name.
Table of contents
Uses headings for sequences
Introductory report statement
.
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Clear, concise, and readable.
Grammar, punctuation, sentence structure, and tenses.
Marks Available
1
Received
_____
1
1
_____
_____
2
_____
_____
/10 marks
______________________________________________________________________________
3. Research - APA Format
Alphabetical references.
1
_______
Proper APA reference citations for Tuckman .
4
_______
( Not simply database urls.)
In text citation of primary sources
5
_______
(Author(s), year, and page number(s).)
In text citation of secondary sources.
(Author “as cited in”.)
/ 10 marks
Tuckman’s Analysis of your group
/20 marks
Group Work
/16 marks
Writing Skills
/10 marks
Research - APA Format
2
3
/10 marks
__________/ 56 Total Marks
__________/ 15%
_____
Revisiting professional skills
(2008)
1. From skills to competency
Example: Group Work
Definition: -varies personal behaviour, appropriately acting as both team
leader and team member, to make and support business and industry
decisions.
• 1. Defines the task and the team’s process and develops plans on how to
proceed with the task and the team’s process.
• 2. States personal ideas and feelings and asks team members for their
ideas and feelings about the task and the team’s process.
• 3. Summarizes and joins together major points discussed and figures out
sources of difficulties in accomplishing the task and difficulties in the team’s
process.
• 4. Compares personal contributions and the team’s accomplishments with
the team’s stated task goals and the team’s stated processes.
• 5. Encourages team member participation through acknowledging and
praising team member contributions.
• 6. Makes sure team members understand what each other is saying in
order to find solutions to team member problems.
Example: Case Study Analysis
Definition: demonstrates logical critical thinking skills to make
and support business and industry decisions.
• 1. Follows a variety of prescribed logical critical thinking skills to
analyze cases, such as identifying problems, symptoms, and
causes, etc.
• 2. Identifies a chain of events to analyze root causes of case issues
or capitalize on case opportunities.
• 3. Identifies pros and cons or advantages and disadvantages of
alternatives in the case based on criteria, such as costs, time,
urgency, importance, personnel, expertise, etc.
• 4. Develops a realistic action plan with a timeline for implementation.
• 5. Evaluates large amounts of ambiguous or disorganized
information and data, and selects the most relevant information to
apply to the case.
Example: Presentations
Definition: effectively creates and delivers individual and team
presentations to make and support business and industry
decisions.
• 1. Has a structured presentation with an introduction, body, and
conclusion.
• 2. Has logical and emotional key points with supporting data and
examples.
• 3. Uses an appropriate medium to highlight key points in the
presentation, such as, visuals, props, handouts, music, etc.
• 4. Uses a logical and emotional chain of linkage between key
points.
• 5. Uses discussions, questions, and activities.
• 6. Uses gestures appropriately.
• 7. Varies voice volume, rate, and tone.
Questionnaire
Data collection
Pilot courses
ORGA 201
ORGA 316
ORGA 310 (critical incidents)
Factor Analysis
Component Matrixa
R1
R2
R3
R4
R5
R6
R7
R8
R9
R10
R11
R12
R14
R15
R16
R17
R18
R19
R20
R21
R22
R23
R24
R25
R26
R27
R28
R29
R30
R31
R33
R34
R35
R36
1
.340
.258
.246
.469
.362
.425
.450
.487
.573
.269
.416
.336
.499
.436
.526
.475
.401
.558
.468
.597
.497
.416
.515
.483
.477
.492
.285
.229
.358
.400
.498
.583
.584
.427
2
3
.441
-.422
.448
.181
-.101
-.264
.127
-.374
-.268
-.230
-.299
.326
.369
.178
.336
.164
-.251
.242
.288
-.223
-.210
.112
.194
.126
.214
-.179
Component
6
.409
.317
.146
.390
.155
.156
.213
-.452
.181
-.462
-.291
-.170
-.385
-.443
.163
-.415
.162
-.615
5
-.371
.278
.161
.284
.329
-.210
.299
-.299
-.167
-.366
-.370
-.282
.300
.420
.302
.304
.270
.233
4
.228
Extraction Method: Principal Component Analys is.
a. 11 components extracted.
-.108
.204
.379
.227
.242
.383
.602
.378
-.395
-.278
-.448
-.332
.217
.387
.331
.153
-.289
.122
.203
-.188
-.118
-.396
.109
.220
.253
.439
.556
-.216
-.124
.110
.235
.134
-.230
-.311
-.101
-.214
-.103
-.416
-.526
-.488
-.406
.489
.490
.329
.298
-.217
.203
7
.176
.310
8
-.136
.372
.195
.269
.315
-.109
.161
.231
.407
-.164
-.306
-.134
-.280
-.255
-.304
-.204
-.289
.116
.122
-.212
.130
.328
-.145
.419
.241
-.158
-.197
10
-.324
.117
-.114
-.243
.113
.425
.277
.394
-.242
-.303
-.168
-.155
.170
.111
.197
-.201
-.187
-.280
-.234
.124
.132
-.156
-.229
-.147
-.332
.225
-.353
-.104
.175
.189
-.117
-.243
-.245
.178
-.104
-.305
.202
-.155
.187
.295
.128
-.161
11
.113
-.314
.406
-.175
-.218
.166
.138
.159
.144
9
.349
.136
-.165
.268
-.111
.211
-.144
-.223
-.130
.146
.122
.106
-.212
.281
.193
-.131
.112
-.139
-.137
-.191
-.178
-.316
-.164
.242
Discussion
•
•
•
•
•
Next steps
Validation
Measurement of Progression
Collaboration
…