The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012)
‫المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي‬
OUTCOMES ASSESSMENT AND QUALITY ASSURANCE PROJECT AT
ELECTRICAL ENGINEERING DEPARTMENT OF UMM AL-QURA
UNIVERSITY
DR. MUHAMMAD ARSHAD MALIK
Professor, Electrical Engineering Department,
Accreditation Consultant, College of Engineering and Islamic Architecture,
Umm Al-Qura University, Saudi Arabia
ibnehsanh1@yahoo.com, mamalik@uqu.edu.sa
DR. MOHAMMED TALAL SIMSIM
Dean, College of Engineering
Ex-Chair, Electrical Engineering Department
Umm Al-Qura University, Saudi Arabia
msimsim@yahoo.com, mtsimsim@uqu.edu.sa
DR. ABDEL MONEM ABDEL RAHMAN ABBAS
Program Coordinator, Electrical Engineering Department,
College of Engineering and Islamic Architecture,
Umm Al-Qura University, Saudi Arabia
a_r_abbas@yahoo.com , aamahmoud@uqu.edu.sa
DR. WAHEED AHMED YOUNIS
Assistant Professor, Electrical Engineering Department
College of Engineering and Islamic Architecture,
Umm Al-Qura University, Saudi Arabia
waheed_younis@hotmail.com
Abstract: The Electrical Engineering (EE) department, at Umm Al-Qura University (UQU), prepares graduates
equipped with the knowledge and practical aspects of electrical engineering, religious studies, and understanding of
socio-economic problems to function professionally and ethically toward solving the problems of society and humanity.
In doing so the department strives to enhance educational level of its students by developing strategies that help in
establishing quality assurance and meet the national, and international academic accreditation standards. Based on
this vision and consistent with the Mission and vision of Umm Al-Qura University EE Program Educational Objectives
(PEOs) have been developed. The two most important student outcomes indicating the achievement of PEOs are,
namely; the ability to function on teams effectively, and the ability to communicate effectively. The Outcome Assessment
and Quality Assurance Project of UQU EE department implements the assessment of these two student outcomes across
the EE curriculum to ensure that the students/graduates of EE program have acquired the most important professional
skills required by the employers. We will present a novel approach adopted by the team responsible for this project to
achieve its goals. This approach may be used by any discipline to enhance the two professional and essential Student
Outcomes.
Keywords: Quality Assurance, Team-Skills, Effective-Communication, PBL
UQU administration, which plays an important
role in developing curriculum for the essential
outcomes that are highly desired by the industry
and other employers as well as being one of the
important factors in acquiring accreditation for the
Electrical Engineering (EE) Program. These
outcomes are also linked to other curriculum
outcomes of the Electrical Engineering Program.
The success of this Outcomes Assessment and
Quality Assurance (OAAQA) Project will lead to
the development of a comprehensive assessment
program for Student Outcomes of the EE
curriculum that will enhance the academic
Introduction:
Unrestrained support of the Custodian of the two
Holy Mosques (Khadim Al-Haramain AlShareefain), King Abdullah bin Abdulaziz, and
his vision toward higher education in the kingdom
of Saudi Arabia has brought energy and
enthusiasm among higher education faculty to
work toward improvement of departments and
enhancement of academic programs to meet the
international accreditation standards.
The Student Outcomes Assessment and Quality
Assurance project is driven by incentives from the
Ministry of Higher Education and the support of
1085
Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University
standards and educational level of the graduates.
This will also be one of the important factors in
acquiring International EAC of ABET
accreditation for the EE Program.
The ABET (Accreditation Board of Engineering
and Technology) Engineering Criteria 2000 (EC
2000) [1] has become international academic
accreditation standard for engineering and
technology programs. There are eleven (a, .., k)
Student Outcomes specified by Criterion 3 of EC
2000 [1]. But both the industry and educational
community stresses the need to inculcate
performance type (soft) skills such that
engineering students attain the following
outcomes from the set of (a… k) Student
Outcomes by the time of graduation to be
successful in their engineering profession [2].
(d) an ability to function on multi-disciplinary
teams, (f) an understanding of professional and
ethical responsibility, (g) an ability to
communicate effectively, (i) a recognition of the
need for, and an ability to engage in, life-long
learning,
(j) a knowledge of contemporary issues, The
importance of these soft skills, in general, and
effective team and communication skills, in
particular, is emphasized in engineering education
literature over and over again. In the December
1998 issue of the American Society for
Engineering Education’s (ASEE) magazine,
PRISM, society president Ernest Smerdon writes
about how not just domestic educators, but the
entire international engineering education
community is faced with the challenge of teaching
performance-type skills [3]: From Utah to the
Ukraine and from Milwaukee to Manila, industry
is demanding that our graduates have better
teamwork skills, communication abilities, and an
understanding of the socio-economic context in
which engineering is practiced [4]. In a recent
survey of recruiters from companies with more
than 50,000 employees, communication skills
were cited as the single most important decisive
factor in choosing managers. This survey,
conducted by the University of Pittsburgh's Katz
Business School, points out that communication
skills, including written and oral presentations, as
well as an ability to work with others, are the
main factors contributing to job success [5].
The aforementioned performance type (soft) skills
(d), (f), (g), (i), and (j) can be achieved by using a
problem-based Learning (PBL) approach [6].
OAAQA Project applies a similar approach
through mini-projects in the laboratory courses
and via research projects in the theory courses.
The project focuses on Effective team work and
Effective Communication skills but the
assessment of other soft skills is embedded in the
assessment process. This approach also provides
opportunity to the students to practice critical
thinking, engineering design, and professional and
ethical conduct.
Therefore the proposed project targets to assess
the two outcomes with the following needs and
expectations of the students and the employers of
our graduates.
• The students are able to function effectively on
a team to implement a common task, e.g., a
technical research report, a mini-project in a
lab course, or a senior design project. In this
way they are prepared to function
professionally on teams to accomplish the
project assigned by the employer.
• The students understand the impact of the
socio-economic and ethical factors on a team
project. The students grasp these concepts in
two sequences of courses on Quran Kareem
and Islamic Culture. The application of these
concepts will be assessed in the team skills
through project assessment. This will allow the
employers to be confident and comfortable in
hiring our graduates who are well rounded
engineers.
• The students are able to communicate
effectively, both orally and in writing. They
are prepared to present their task in front of an
evaluating committee with confidence and
competence.
• The students are able to use technology to
present their ideas and work in an organized,
logical, and professional manner.
The project was proposed by three faculty
members and is being implemented by the help of
other UQU EE faculty members for the
assessment and evaluation of two professional
student outcomes, namely, An ability to work
effectively on Teams, and An ability to
communicate effectively in writing and orally.
Project Planning: Strategies, Assessment
Tools and Time-Line:
The project will be implemented in three years
starting at the beginning of Term-1, 1431/1432
(Fall-2010) after all the appropriate financial and
human resources are available. The project team
consists of a Principle Investigator, two coinvestigators, and a project coordinator. The
Outcomes Assessment process is implemented
during the two main semesters (Fall and Spring)
and the summer semester is utilized to perform
analysis
and
evaluation,
develop
recommendations, and to suggest actions to
improve the outcomes as well as the assessment
process.
1086
The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012)
The assessment strategies involve selection of the
courses in which the assessment data will be
collected and evaluated. The main strategies are to
select the courses across the curriculum such that
the two aforementioned Communication and
Team professional skills are emphasized at
sophomore, junior, and senior levels of students’
studies. We call this a Long Term Plan. The
students in this will be graduating from UQU by
the end of the assessment and quality assurance
project. The courses selected were a mix of lab
and lecture courses generic purposes so that later
our assessment process can be used for any
discipline.
In order to benefit students who are graduating
within two years, we have devised a Short Term
Plan and assessment data will be collected only at
the junior and senior levels. An exit-interview
survey will be conducted for the Short Term Plan
to measure the performance of the outcomes
assessment and quality assurance project. The
graduates of the Short Term Plan will have
worked for one year in the field by the end of the
outcomes assessment and quality assurance
project. This will allow the project team to
conduct employer and alumni surveys to evaluate
and validate the efficacy of the project.
The Long Term Plan is shown in the following
Figure 1.
STRATEGIES
ASSESSMENT
TOOLS
TIME-LINE
ELECTRONIC
CIRCUITS (802312)
PEER/ SELF
AND FACULTY
EVALUATION
Term-2,
1431/1432AH
(Spring 2010)
INTRODUCTION TO
COMMUNICATION
(802322)
PEER/SELF AND
FACULTY
EVALUATION
Term-1,
1432/1433AH
(Fall 2011)
SEMINARS IN EE
(802304)
PEER/SELF AND
FACULTY
EVALUATION
Term-2,
1432/1433AH
(Spring 2012)
COMMUNICATION
SYSTEMS
(802421)
PEER/SELF AND
FACULTY
EVALUATION
Term-1,
1433/1434AH
(Fall 2012)
‫المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي‬
graduation of these cohorts and by the outcomes
of External advisory board.
The Short Term Plan is shown in the following
Figure 2.
STRATEGIES
ASSESSMENT
TOOLS
ELECTROMECHANICAL
ENERGY CONVERSION I
(802345)
PEER /SELF AND
FACULTY
EVALUATION
Term-2,
1431/1432AH
(Spring 2010)
COMMUNICATION
SYSTEMS
(802421)
PEER/SELF AND
FACULTY
EVALUATION
Term-1,
1432/1433AH
(Fall 2011)
TIME-LINE
EXIT-INTERVIEW
AT GRADUATION
GRADUATE
SURVEY
Term-2,
1432/1433AH
(Spring 2012)
EXTEERNAL
ADVISORY BOARD
MEEETING
ALUMNI AND
EMPLOYER
SURVEYS
Term-2,
1433/1434AH
(SPRING2013)
Figure 2:
Short Term Plan, Strategies,
Assessment Tools, and Time of Data Collection
The first column in this figure shows a junior
level
course,
Electromechanical
Energy
Conversion I and a senior level course,
Communication Systems. Both of these courses
have a laboratory component and we applied our
assessment process only on the lab component.
The students in this plan will be graduating at the
end of Term-2, 1432/1433 AH (Spring 2012).
The assessment tools being used in our project are
peer evaluation of team skills, self evaluation of
both communication and team skills, and faculty
evaluation of both outcomes. The other methods
of assessment to be used are Exit-Interview
surveys, Alumni surveys, and Employers surveys.
An exit Interview Survey was conducted for these
graduates to measure the performance of the
project as well as their communication and team
skills. Alumni and Employers survey will be
conducted after one year of graduation to validate
the performance of the project and the results of
these surveys will be discussed with the External
Advisory Board members to get their feedback for
future actions.
The comparison will be done between the
evaluations (of assessment data collected) of Short
Term Plan and Long Term Plan This will reflect
the outcome of applying the educational strategies
early on in the curriculum.
After the planning phase, the implementation of
the outcomes assessment process involves
adopting an effective approach to inculcate and
emphasize professional communication and team
skills in the students and developing performance
criteria and rubrics for assessment. The novel
approach adopted by the team is derived from the
collaborative project-based learning (CPBL) and
Problem-Based Learning (PBL) approaches.
Figure 1: Long Term Plan, Strategies, Assessment
Tools, and Time of Data Collection
The first column in this figure shows a sophomore
level course, Electronic Circuits, junior level
courses, Introduction to Communication &
Seminars in Electrical Engineering, and a senior
level course, Communication Systems. These
courses are a mix of a laboratory and theory. The
outcomes assessment process is implemented on
both laboratory and theory courses to measure its
performance so that it not restrained by discipline.
The students in this plan will be graduating at the
end of the project. Therefore, the evaluation
results will be validated by conducting Alumni
and Employer surveys after 1-2 years of
1087
Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University
the learner acquisition of critical knowledge,
problem solving proficiency, self-directed
learning strategies, and team participation skills.
The process replicates the commonly used
systemic approach to resolving problems or
meeting challenges that are encountered in life
and career [10].
The two approaches have shared attributes as well
as unique attributes as defined in the literature
[11]. The shared attributes are as follows:
Collaborative Project-Based Learning
(Cpbl) And Problem- Base Learning (Pbl):
The transformation in higher education from
content based assessment to outcomes based
assessment has motivated the educators to focus
on how the students learn instead of how we teach
them. This paradigm shift stresses the need to use
instructional strategies or pedagogical approaches
that focus on student centered learning. The goal
of these approaches for engineering education is
to close the gap between student’s learning and
industry needs. The advances in technology and
changes in the organisational infrastructure put an
increased emphasis on team work within the
workforce. Workers need to be able to think
creatively, solve problems, and make decisions as
a team [8]. Therefore, in order to develop the
important
professional
skills,
such
as
communication and teamwork, students need to
work in groups or teams. The students can only be
well versed in a skill if they practice it and a
group learning environment provides opportunity
to students to learn how to work on teams,
exchange ideas, listen and communicate with
others, convince and negotiate based on logical
reasoning in order to resolve conflicts. These are
important skills for students to develop as
research indicates that employers worldwide want
graduates
who
have
well
developed
communication, teamwork and problem solving
skills [10].
The group learning environment is provided by
two well known pedagogical approaches, namely,
Collaborative Project-Based Learning (CPBL) and
Problem-Based Learning (PBL). The former
approach is well matured for primary and
secondary education, but well established
principles can be applied in engineering
education. The later approach has its origin in
medical education.
Project-Based Learning is an individual or group
activity that goes on over a period of time,
resulting in a product, presentation, or
performance. It typically has a time line and
milestones, and other aspects of formative
evaluation as the project proceeds [10].
Collaborative learning has been hailed as giving
students an opportunity to engage in discussion,
take responsibility for their own learning, and thus
become critical thinkers [8][9].
Problem-Based Learning is a teaching method in
which students learn through solving a problem
[7]. PBL, as defined by Dr. Howard Barrows and
Ann Kelson of Southern Illinois University
School of Medicine, is both a curriculum and a
process. The curriculum consists of carefully
selected and designed problems that demand from
1. both approaches engage students in authentic,
real world tasks
2. open-ended project or problems have more than
one approach or answer
3. projects or problems are intended to simulate
professional situations
4. student-centered, teacher-facilitated
5. students work in for extended period of time
6. students encouraged to seek out multiple
sources of information
7. emphasis on authentic performance-based
assessment
8. ideally both approaches provide adequate time
for student reflection and self evaluation.
The unique attributes of the two approaches are as
follows [11]:
Problem-Based
Learning
begins with problem
for a student to solve
or learn about
problems can be
framed in a scenario
or case study format
problems are
somewhat ambiguous
to mirror the
complexity of real
life
uses an inquiry
model
students present
conclusion of
problem-solving
process but do not
necessarily create a
product
defined problem is
driving force
Project Based Learning
begins with an end product
or “artifact” in mind
production of artifact raises
one or more problems for
student to solve
uses a production model
and mirrors real world
production activities
content knowledge and
skills acquired during the
production process are
important to success
student use or present the
product they have created
end product is driving force
In practice, it is likely that the line between projectand problem-based learning is frequently blurred and
that the two are used in combination and play
complementary roles [10]. Keeping in mind the
1088
The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012)
‫المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي‬
sustainability of the assessment process in terms of
faculty engagement and students’ active participation
we have used the complementary roles of the two
strategies. We have used the Problem-based approach
in a laboratory course in which students are required to
design a mini-project and in a theory course the
students are required to do a research project to solve a
problem.
Prolem-Based Mini-Project Design In A
Laboartory Course:
A variant of Project-Based Collaborative Learning
(PBCL) approach is applied in the laboratory courses
starting with sophomore level for the Long Term Plan.
The students in Electronic Circuits laboratory perform
several experiments for the first 10 weeks and then
they are required to design two mini-projects based on
their prior theoretical knowledge in the course and the
prior laboratory experience, during the last four weeks
of the term. They are required to use Multisim for
simulation and then implement the hardware. The
teams of 3-4 students are built by the instructor at the
beginning of the term and are kept stable till the end of
the term. The team skills are assessed throughout the
term via faculty evaluation as well as peer and self
evaluations. The teams are required to demonstrate
their project and present their results in the form of
power point presentations in front of an assessment
panel at the end of the term. During these presentations
individual
students
are
assessed
for
oral
communication skills. The teams are also assessed for
written communication skills based on their final
technical report. The final demonstration and
presentation assessment is considered as a laboratory
final exam for grading purposes. The assessment data
is used to evaluate the performance of students for the
two professional outcomes, namely, ability to work
effectively on teams, and ability to communicate
effectively orally, in written, and presentation form.
The students are required to design these projects with
the devices available in the laboratory. Therefore, they
need to think of feasible solutions for their projects.
Similar approach is applied for the Short Term Plan
starting with the junior level course of
Electromechanical Energy Conversion I.
The examples of these design projects are depicted in
the following figures 3, 4, 5, & 6:
Figure 3: Mini-Project Design in the Electronic
Circuits Laboratory
Electromechanical Energy Conversion I: MiniProject I
Using only three phase AC voltage source, devise a
suitable configuration to control the speed of a DC
motor from zero to full speed in both directions. Plot
the torque speed characteristics for different values of
the driving voltage
Proposed block diagrams:
The two machines
are coupled on the
same shaft
3-phase
Supply with
a Reversal
Switch
3-phase
Induction
Motor
The two machines
are coupled on the
same shaft
Self excited
DC
Compound
Generator
Field
Control
Variable
DC
Voltage
Self
Excited
DC Motor
Mechanica
l Load
Torque
Torque and
Speed
Measurement
s
Figure 4: Mini-Project Design in the Electromechanical
Energy Conversion I Laboratory
ELECTRONIC CIRCUITS:
Mini- Project II
Using the available electric machines in the lab,
construct a suitable configuration to generate a variable
frequency AC supply. Determine the range of variation
of frequency and illustrate how to use this
configuration to control the speed of a three phase
induction motor.
Two solutions are available
aAdding a synchronous generator on the same shaft
of the variable speed dc motor of experiment 7. The
resultant output is a variable frequency AC supply
which can be used to drive a three phase induction
motor at different speeds. The overall efficiency of this
configuration is poor since it involves four machines to
1089
Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University
generate the variable frequency AC supply with the
advantage of building upon exp.7
bUsing a variable DC supply available in the lab to
drive a DC motor on which the synchronous generator
will be coupled. The circuit has a higher efficiency
than that proposed at (a)
The students have to think, discuss in teams, and
choose the best solution which reflects their ability of
critical thinking and design capabilities. Both miniprojects require students to demonstrate effective team
work and good communication skills to present their
work.
Two Alternate Solutions:
3-phase
Induction
Motor
PROBLEM-BASED
RESEARCH
PROJECT IN A THEORY COURSE:
A variant of Problem-Based learning approach
is applied in the junior level theory course, Introduction
to Communication. This is a theory course in which
students learn various modulation techniques for
electromagnetic transmission of message signals. In the
beginning of the term, the instructor builds stable teams
of three students each and assigns a research project to
the teams to be completed by the end of the term. The
research project is a real situation open ended problem
and requires teams to have prior knowledge of the
discipline to comprehend and come up with their own
problem statement. The following learning model,
similar to that of Problem-Based Learning (PBL)
approach [11], is applied in the research project:
1. Identify the Problem: Read and analyze the
situation described in the problem. Discuss your
understanding of the problem as a group to know the
important issues to be dealt with. Based on that,
your team will need to actively search for
information to solve the problem.
2. List the known factors: List the given items in the
problem. Your team needs to create a list containing
the known items in the problem, what do you know
about the problem scenario, and what knowledge
individual members of the team have about the
problem.
3. Write the Problem Statement in Your own
Words: Based on the knowledge your team has,
write a brief (2-3 sentences) problem statement in
your own words describing the situation and what
your team is trying to solve, find, or produce. You
may need to revise the problem statement when new
information is revealed and brought to affect the
problem scenario.
4. List Unknown Factors and What is to be
Determined: List all the questions you consider to
be answered before solving the problem. List the
items to be determined to solve the problem. What is
required to be done? Ask your instructor the
questions you may have.
5. Write Alternate Solutions to the Problem: Based
on your research you may come up with more than
one solution to the problem. Write all possible
solutions to the problem and compare and contrast
various solutions.
6. Present Your Findings: Prepare a report in which
you make your recommendations, draw a
conclusion, or present alternate solutions to the
problem. You are required to present and defend
your results in the form of a power point
presentation and a technical written report.
An example of the Problem assigned to teams in
Introduction to Communication course is described as
follows:
The two machines are
coupled on the same
shaft
The two machines are
coupled on the same
shaft
3-Phase
Supply with a
Reversal
Switch
This approach needs teams to interact more with
instructor and that is possible only in a laboratory
course. In case of theory course, that level of
interaction is not possible. Therefore, we have used a
variant of Problem-Based Learning approach in the
theory courses.
Self excited
DC
compound
Generator
Variable
Frequency
AC Voltage
Variable
DC
Voltage
Self
Excited
DC Motor
Synchronous
Generator
Field
Control
Torque and
speed
Measurements
Mechanical
Load
3-phase
Induction
Motor
Figure 5: Mini-Project Design in the Electromechanical
Energy Conversion I Laboratory Configuration (a)
The two machines
are coupled on the
same shaft
Variable
DC
Voltage
The two machines
are coupled on the
same shaft
Variable
Self
Excited
DC Motor
Synchronous
Generator
Frequency
AC
Voltage
3-phase
Induction
Motor
Mechanical
Load
Torque and
Speed
Measurements
Figure 6: Mini-Project Design in the Electromechanical
Energy Conversion I Laboratory Configuration (b)
These projects also expose students to design
experience, critical thinking, and problem solving skills
other than the main two professional outcomes of
effective team work and effective communication. The
Project-Based Collaborative Learning approach is used
in these mini-projects but it is complemented by
inquiry model of Problem-Based learning. The design
of the project start with an open-ended problem and
students need to utilize their prior knowledge to
understand the problem and come up with their own
problem statement and then utilize multiple sources to
get information to solve that problem. This approach is
different than the conventional Project-Based Learning
strategy because it is not using structured projects and
the instructor’s role is not that of a tutor or facilitator.
The instructor work as an advisor with expertise in the
discipline and student’s dependency on the instructor is
very low regarding design and implementation of the
project.
1090
The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012)
‫المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي‬
Table 1: Faculty Evaluation of Student Outcome1: An
ability to work effectively on teams
Introduction To Communication: Research-Project
Problem Statement
You are required to design an AM modulator to
generate a modulated signal
(as shown in the figure below).
An ability to work effectively in teams
Faculty Evaluation
4: Exemplary,
3: Satisfactory,
2: Progressing,
1: Unsatisfactory
SCALES
PERFORMANCE
INDICATOR
4
1. Demonstrate understanding
of team-roles when assigned.
2. Demonstrates good listening
skills.
Figure 7: Modulated Signal Generator
Message signal:
is baseband signal
The message signal
with bandwidth equals 5 kHz. The dynamic range of
the signal is from 3 mV to -2 mV.
D
I
M
E
N
T
I
O
N
S
3. Cooperates with teammates.
4 Makes fair decisions.
5. Be punctual.
6 The tasks are described, those
tasks are divided among team
members according to
individuals’ abilities and
interests.
7. Develop ways to resolve
conflicts and reach
agreement.
8. Respond positively to
teammates’ ideas.
Figure 8: Baseband Message Signal
Modulated Signal:
Local government regulation restricts that the
modulated signal should not exist outside of 800 kHz
to 950 kHz band.
Carrier Frequency:
The carrier frequency generator can only generate
between 260 kHz to 300 kHz. It can generate the
following types of waveforms with zero off-set:
i. Full rectified wave,
ii. Saw tooth wave,
iii. Square wave, iv. Triangular wave v. Cosine wave
Modulation Index: Should not be less than 90%.
You are asked to do the following:
1. Draw the modulator diagram.
2. Decide about the settings of the carrier frequency
generator (the frequency and waveform). Show your
reasoning why you selected a particular waveform
and frequency. [Hint: you will have to analyze the
carrier waveforms given above using Fourier series].
3. Decide about the filter (filter type, center frequency
and bandwidth).
4. Draw the spectrum (as exact as you can) at the
points 'a', 'b' and 'c' in the figure above.
As discussed earlier, the main goal of the Outcomes
Assessment and Quality Assurance project is to
measure the performance of students for the two
professional outcomes. Therefore, the project team
has developed different assessment methods as
described in the next section.
D
E
S
C
R
I
P
T
O
R
S
Performs all duties
assigned and actively
assist others.
Consistently listens
and responds to others
appropriately.
3
2
1
Performs duties
that are assigned.
Inconsistently performs
duties that are assigned.
Does not perform any
duties of assigned team
role.
Listens most of the
time.
Usually doing most of
the talking-rarely allows
others to speak.
Is always talking and
never allows anyone
else to speak.
Sometimes argues.
Usually argues with
teammates.
Never argues with
teammates.
Rarely argues.
Always helps team to
reach a fair decision.
Usually considers
all views.
Hands in all
assignments on time.
Hands in most
assignments on
time.
Understands the
tasks completely
and often shows
interest.
Understands all the
tasks completely,
always shows interest,
and motivates others
to work on tasks
according to their
ability.
Always come up with a
method to resolve
conflicts and reach an
agreement.
Always responds
positively to the
teammates’ ideas.
Always tries to
resolve conflicts
and helps team
reaching an
agreement
Usually responds
positively to the
teammates’ ideas.
Often sides with friends
instead of considering
all.
Hands in most
assignments late.
Usually wants to have
things their way.
Does not hand in
assignments.
Does understand the
tasks partially and
sometimes show interest.
Does not understand
the tasks And does not
show interest.
Sometimes tries to
resolve conflicts.
Always argues with
teammates and never
try to resolve the
conflicts.
Often responds
positively to the ideas
presented by the
teammates.
Always criticize the
ideas presented by a
teammate.
The assessment process also includes the team self
assessment and individual self assessment. The forms
for
these
assessments
are
adopted
form
www.performancexpress.org. For the Short Term
Plan exit interview surveys are conducted to measure
the performance of the graduating student outcomes
based on the self assessment. The graduating students
are also asked in the exit interview survey to indicate
their level of achievement of Communication and
Team skills after participating in the outcomes
assessment and quality assurance project. The
following assessment data was analyzed and the results
were very encouraging in the sense that students
performed very well for most of the performance
indicators of the two outcomes.
1. Faculty Evaluations, 2. Team Self Assessment,
3.Individual Self Assessment Test,
4. Exit Interview Surveys.
The students’ responses on team self assessment
and individual self assessment as well as on the exitinterview survey were very good. The results of
evaluation are discussed in the next section.
RESULTS AND DISCUSSION:
•
OUTCOMES ASSESSMENT PROCESS:
The project team researched and developed the
performance indicators for the assessment of the two
professional outcomes, and the rubrics in the form of
descriptors and dimensions [12]. One of the faculty
evaluation forms, containing Performance Indicators,
Dimensions, and Descriptors, used for the assessment
of the student outcome is shown as an example in the
following table:
1091
The performance of students for the two student
outcomes is evaluated by faculty. The levels of
performance are defined as follows:
Exemplary = 4.0, Satisfactory = 3.0,
Progressing = 2.0, & Unsatisfactory = 1.0
The performance targets were set for all the
performance indicators (PIs) as follows:
70% or more students perform at exemplary or
satisfactory level.
The results of evaluation of Electronic Circuits course
for two groups for the two outcomes in the form of bar
graphs are shown in the following figures 9 and 10.
In the Introduction to Communication (ITC) course
during Term-1, 1432/1433 (Fall 2011), most of the
teams responded that they were at the norming or
performing stages for all these characteristics except
characteristics #2 and # 3 but even for those
characteristics 60% of the teams indicated that they
were at the norming or performing stage. The results
are shown in the form of a bar graph in the following
figure.
Outcome 1: An ability to work effectively on teams
100
90
80
70
60
50
40
30
20
Good Team Characteristics: The team is at the
Norming Stage or Performing Stage
10
0
1
2
3
4
5
6
Performance Indicators (PIs)
7
8
Percentage of Teams at Norm ing and Perform ing Levels
Percentage of Students who Performed Examplary or Satisfactory
Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University
Figure 9: Bar Graph of Student Performance for Team
Skills Performance Indicators
The results in Figures 9 and 10 are for the sophomore
level students and indicate that more than 70% of
students performed exemplary or satisfactorily in all
the PIs for the Team skills outcome. This indicates that
most of the students participating in the project have
achieved the professional outcome of effective team
skills.
More than 70% of students performed exemplary or
satisfactorily in all the PIs for the Communication
skills outcome except #1. This indicates that most of
the students participating in the project have achieved
the professional outcome of effective Communication
skills except for the PI #1. This was the first term for
the students to participate in the project. They will
perform well in the next term to identify audience and
organize their presentation to meet the needs of
audience. The students need to organize their
presentation better.
Percentage of Students who performed
Exeplary or Satisfactory
90
80
70
60
50
40
30
20
10
2
3
4
5
6
Performance Indicators
7
80
70
60
50
40
Faculty 1
Faculty 2
30
20
10
0
1
2
3
4
Team Characteristics #
5
6
The assessment performed for the self assessment of
teams' skills by individual students asked students 18
questions related to team skills to respond as yes or no
to indicate achievement of the particular skill. Students
from two different groups of ITC course taught by two
different faculty members participated in this
assessment. The results depicted in Figure 12 indicate
that most of the students responded affirmatively that
they achieved most of the team skills.
The faculty evaluation of the two outcomes for the
Introduction to Communication (ITC) course were
performed by the faculty during Term-1, 1432/1433
(Fall 2011) and at the end of the term. The team skills
evaluation was not promising because, firstly, ITC
course was a theoretical course and the teams of
student were working outside the classroom and
secondly the team-faculty interaction time was not
enough for a good assessment to be done
100
1
TARGET
90
Figure 11: Self Assessment of Teams (Percentage of Teams
with good Team Characteristics)
Outcome 2: An ability to communicate effectively in
oral, written, and presentation forms
0
ITC Research Project:Team Characteristics Assessment by Teams, Fall 2011
100
8
Figure 10: Bar Graph of Student Performance for
Communication Skills Performance Indicators
The team self assessment was meant to see if the teams
of students were at a norming or performing stage and
the teams were asked to indicate at what stage is their
team for 6 different team characteristics as follows:
1. Communication between members, 2. Members’
Commitment, 3. Flexible Purpose,
4. Constant Members Involvement,
5. Normal expectation of Trust, 6. Natural and
smooth process
1092
The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012)
Exit interviews surveys were conducted at the end of
Spring 2012. Only 9 students have participated as of
yet. The results of evaluation of these surveys are very
good but we will be able to draw conclusions only after
getting an appropriate size of data. The results of exitinterview surveys are shown in Figure 15 and 16.
ITC Research Project:Team Skills Achievement Self Assessment, Fall 2011
COMM. SKILLS SELF ASSESSMENT TEST
60
TARGET
55
Faculty 1
Faculty 2
50
100
CUMULATED WEIGHTED AVERAGE
10
20
30
40
50
60
70
80
90
Percentage of Students Achieved the Particulat Team Skill
45
40
35
30
25
20
15
10
Figure 12: team Skills Self Assessment by Individual
Students (Percentage of Students who achieved the particular
skill)
5
0
1
2
3
4
STUDENT #
70
60
50
CUMULATED WEIGHTED AVERAGE
Percentage of Students w ith Exam plary or Satisfacory Performance
80
Faculty 2, Group 2
Faculty 2, Group 1
20
10
0
1
2
3
4
5
6
Performance Indicator #
7
8
Figure 13: Faculty Evaluation of Students’ Communication
Skills (Percentage of Students with Exemplary and
Satisfactory Performance)
Percentage of Students with Examplary or Satisfacory Performance
Faculty 1
Faculty 2, Group 2
Faculty 2, Group1
TARGET
70
60
50
40
30
20
10
0
1
2
3
4
5
6
Performance Indicator #
7
STUDENT #
PROJECT ASSESSMENT FOR TEAM SKILLS
60
58
TARGET
56
54
52
50
48
46
44
42
40
38
36
34
32
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
0
1
2
3
4
5
6
7
8
(EXIT INTERVIEW SURVEY FALL 2011)
STUDENT #
Figure 16: Cumulative Team Self Assessment Test &
Cumulative Project Assessment for TEAM SKILLS
These outcomes are also linked to other curriculum
outcomes of the Electrical Engineering Program. The
success of this Outcomes Assessment and Quality
Assurance (OAAQA) Project will lead to the
development of a comprehensive assessment program
for Student Outcomes of the EE curriculum that will
enhance the academic standards and educational level
of the graduates. We employed a novel strategy to let
the students learn how to become better collaborators
and effective communicators. Our approach utilizes the
best practices of the two well know instructional
strategies; Collaborative Project-Based Learning
(CPBL) and Problem-Based learning (PBL). Keeping
in mind the sustainability of the assessment process in
terms of faculty engagement and students’ active
participation we have used the complementary roles of
the two strategies. We have used the Problem-based
Faculty Evaluation of ITC Research Project:Team Skills Fall 2011
80
9
TEAM SKILLS SELF ASSESSMENT TEST
60
58
TARGET
56
54
52
50
48
46
44
42
40
38
36
34
32
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
0
1
2
3
4
5
6
7
8
STUDENT #
100
90
8
The Student Outcomes assessment and Quality
Assurance project at UQU plays an important role in
developing curriculum for the essential outcomes,
namely; Ability to work effectively on Teams & Ability
to Communicate Effectively, that are highly desired by
the industry and other employers as well as being one
of the important factors in acquiring accreditation for
the Electrical Engineering (EE) Program.
90
30
7
(EXIT iNTERVIEW SURVEY FALL 2011)
Conclusion:
Faculty Evaluation of ITC Research Project:Communication Skills Fall 2011
100
TARGET
Faculty 1
6
Figure 15: Cumulative Comm. Skills Self Assessment Test &
Cumulative Project Assessment for COMM. SKILLS
The communication skills performance was done much
better than that of the team skills because students were
required to present their research project in front of an
assessment panel. The results of the two evaluations
are shown in the form of bar graphs in Figure 13, and
14.
40
5
PROJECT ASSESSMENT FOR COMM. SKILLS
60
58
TARGET
56
54
52
50
48
46
44
42
40
38
36
34
32
30
28
26
24
22
20
18
16
14
12
10
8
6
4
2
0
1 2 3 4 5 6 7 8 9
CUMULATED WEIGHTED AVERAGE
0
CUMULATED WEIGHTED AVERAGE
Q uestion #
TARGET
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
‫المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي‬
8
Figure 14: Faculty Evaluation of Students’ Team Skills
(Percentage of Students with Exemplary and Satisfactory
Performance)
1093
Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University
Project approach in a laboratory course in which
students are required to design a mini-project and in a
theory course the students are required to perform a
research project to solve a problem. The feedback from
students and faculty indicates that our approach keeps
the students interested in learning and the faculty is not
burnt out in terms of conventional facilitator or tutor.
The faculty role is that of an advisor and students’
dependency on the instructor is very low. The self
assessment and peer assessment in our project is not
biased because it does not count toward students
grading. The team assessment in theory courses needs
some adjustments so that students’ performance
assessment in regards to Team skills can be reliable
and consistent.
[9].
[10].
[11].
[12].
Acknowledgement:
We would like to acknowledge the support of Ministry
of Higher Education, Kingdom of Saudi Arabia for
partially funding this project. We are also grateful to
Umm Al-Qura University for support toward this
project in terms of equipment and human resources.
The moral support of Dr. Esam Ahdal, Vice Rector of
Development and Community Services and Dr. Abul
Razaq Sultan, the supervisor of the project is worth
mentioning. We are very grateful to both of them for
supporting this very important project toward the
development of Electrical Engineering Department at
Umm Al-Qura University.
[13].
References:
[1].
[2].
[3].
[4].
[5].
[6].
[7].
[8].
ABET (Accreditation Board for Engineering and
Technology)
Criteria
for
accrediting
engineering programs; Effective for Evaluations
during the 2010-2011 Accreditation Cycle.
<http://www.abet.org/images/Criteria/201112EACCriteria.pdf>, accessed September 2010.
Muhammad A. Malik, Mohammed T. Simsim,
Abdel Monem A. Abbas, Assessment of Student
Professional Outcomes for Quality Assurance
and Continuous Improvement, First International
Arab Conference on Quality Assurance in
Higher Education, May10, 2011, Amman,
Jordan.
Briedes D., (2002) Developing Effective
Assessment of Student Professional
Outcomes, International Journal Engineering
Education, vol.18, No. 2, pp. 208-216.
Smerdon, E. T., (1998) President’s letter: Global
Engineering, PRISM. vol. 8, no. 4, The
American Society for Engineering Education,
Washington, DC, p 37.
Introduction
to
Communication,
Why
Communications Skills are so Important,
http://www.mindtools.com/CommSkll/Communi
cationIntro.htm
Theory
Behind
PBL,
http://ldt.stanford.edu/~jeepark/jeepark
+portfolio/PBL/howtodopbl.htm
Patrick Walsh, Problem Based Learning in
Engineering, International Symposium for
1094
Engineering Education, 2007, Dublin City
University, Ireland.
Gokhale, A.A., Collaborative Learning Enhances
Critical Thinking, J. of Technology Educ., 7, 1
(1995)
Totten, S., Sills, T., Digby, A. and Russ, P.,
Cooperative Learning: a Guide to Research. New
York: Garland (1991).
Roisin Donnelly and Marian Fitzmaurice,
Collaborative Project-Based Learning and
Problem-Based Learning in Higher Education: A
Consideration of Tutor and Student Roles in
Learner-Focused Strategies, retrieved from
www.aishe.org on January 20, 2012.
PBL, Problem Based Learning, Marico Center
for
Learning
and
Instruction,
Comparing/Contrasting Problem and ProjectBased Learning Approaches.
Gloria Rogers, (2010) Bottom Line: Assessment,
IDEAL, The Institute for the Development of
Excellence in Assessment Leadership, August 2
– August 6, 2010.