The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012) المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي OUTCOMES ASSESSMENT AND QUALITY ASSURANCE PROJECT AT ELECTRICAL ENGINEERING DEPARTMENT OF UMM AL-QURA UNIVERSITY DR. MUHAMMAD ARSHAD MALIK Professor, Electrical Engineering Department, Accreditation Consultant, College of Engineering and Islamic Architecture, Umm Al-Qura University, Saudi Arabia ibnehsanh1@yahoo.com, mamalik@uqu.edu.sa DR. MOHAMMED TALAL SIMSIM Dean, College of Engineering Ex-Chair, Electrical Engineering Department Umm Al-Qura University, Saudi Arabia msimsim@yahoo.com, mtsimsim@uqu.edu.sa DR. ABDEL MONEM ABDEL RAHMAN ABBAS Program Coordinator, Electrical Engineering Department, College of Engineering and Islamic Architecture, Umm Al-Qura University, Saudi Arabia a_r_abbas@yahoo.com , aamahmoud@uqu.edu.sa DR. WAHEED AHMED YOUNIS Assistant Professor, Electrical Engineering Department College of Engineering and Islamic Architecture, Umm Al-Qura University, Saudi Arabia waheed_younis@hotmail.com Abstract: The Electrical Engineering (EE) department, at Umm Al-Qura University (UQU), prepares graduates equipped with the knowledge and practical aspects of electrical engineering, religious studies, and understanding of socio-economic problems to function professionally and ethically toward solving the problems of society and humanity. In doing so the department strives to enhance educational level of its students by developing strategies that help in establishing quality assurance and meet the national, and international academic accreditation standards. Based on this vision and consistent with the Mission and vision of Umm Al-Qura University EE Program Educational Objectives (PEOs) have been developed. The two most important student outcomes indicating the achievement of PEOs are, namely; the ability to function on teams effectively, and the ability to communicate effectively. The Outcome Assessment and Quality Assurance Project of UQU EE department implements the assessment of these two student outcomes across the EE curriculum to ensure that the students/graduates of EE program have acquired the most important professional skills required by the employers. We will present a novel approach adopted by the team responsible for this project to achieve its goals. This approach may be used by any discipline to enhance the two professional and essential Student Outcomes. Keywords: Quality Assurance, Team-Skills, Effective-Communication, PBL UQU administration, which plays an important role in developing curriculum for the essential outcomes that are highly desired by the industry and other employers as well as being one of the important factors in acquiring accreditation for the Electrical Engineering (EE) Program. These outcomes are also linked to other curriculum outcomes of the Electrical Engineering Program. The success of this Outcomes Assessment and Quality Assurance (OAAQA) Project will lead to the development of a comprehensive assessment program for Student Outcomes of the EE curriculum that will enhance the academic Introduction: Unrestrained support of the Custodian of the two Holy Mosques (Khadim Al-Haramain AlShareefain), King Abdullah bin Abdulaziz, and his vision toward higher education in the kingdom of Saudi Arabia has brought energy and enthusiasm among higher education faculty to work toward improvement of departments and enhancement of academic programs to meet the international accreditation standards. The Student Outcomes Assessment and Quality Assurance project is driven by incentives from the Ministry of Higher Education and the support of 1085 Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University standards and educational level of the graduates. This will also be one of the important factors in acquiring International EAC of ABET accreditation for the EE Program. The ABET (Accreditation Board of Engineering and Technology) Engineering Criteria 2000 (EC 2000) [1] has become international academic accreditation standard for engineering and technology programs. There are eleven (a, .., k) Student Outcomes specified by Criterion 3 of EC 2000 [1]. But both the industry and educational community stresses the need to inculcate performance type (soft) skills such that engineering students attain the following outcomes from the set of (a… k) Student Outcomes by the time of graduation to be successful in their engineering profession [2]. (d) an ability to function on multi-disciplinary teams, (f) an understanding of professional and ethical responsibility, (g) an ability to communicate effectively, (i) a recognition of the need for, and an ability to engage in, life-long learning, (j) a knowledge of contemporary issues, The importance of these soft skills, in general, and effective team and communication skills, in particular, is emphasized in engineering education literature over and over again. In the December 1998 issue of the American Society for Engineering Education’s (ASEE) magazine, PRISM, society president Ernest Smerdon writes about how not just domestic educators, but the entire international engineering education community is faced with the challenge of teaching performance-type skills [3]: From Utah to the Ukraine and from Milwaukee to Manila, industry is demanding that our graduates have better teamwork skills, communication abilities, and an understanding of the socio-economic context in which engineering is practiced [4]. In a recent survey of recruiters from companies with more than 50,000 employees, communication skills were cited as the single most important decisive factor in choosing managers. This survey, conducted by the University of Pittsburgh's Katz Business School, points out that communication skills, including written and oral presentations, as well as an ability to work with others, are the main factors contributing to job success [5]. The aforementioned performance type (soft) skills (d), (f), (g), (i), and (j) can be achieved by using a problem-based Learning (PBL) approach [6]. OAAQA Project applies a similar approach through mini-projects in the laboratory courses and via research projects in the theory courses. The project focuses on Effective team work and Effective Communication skills but the assessment of other soft skills is embedded in the assessment process. This approach also provides opportunity to the students to practice critical thinking, engineering design, and professional and ethical conduct. Therefore the proposed project targets to assess the two outcomes with the following needs and expectations of the students and the employers of our graduates. • The students are able to function effectively on a team to implement a common task, e.g., a technical research report, a mini-project in a lab course, or a senior design project. In this way they are prepared to function professionally on teams to accomplish the project assigned by the employer. • The students understand the impact of the socio-economic and ethical factors on a team project. The students grasp these concepts in two sequences of courses on Quran Kareem and Islamic Culture. The application of these concepts will be assessed in the team skills through project assessment. This will allow the employers to be confident and comfortable in hiring our graduates who are well rounded engineers. • The students are able to communicate effectively, both orally and in writing. They are prepared to present their task in front of an evaluating committee with confidence and competence. • The students are able to use technology to present their ideas and work in an organized, logical, and professional manner. The project was proposed by three faculty members and is being implemented by the help of other UQU EE faculty members for the assessment and evaluation of two professional student outcomes, namely, An ability to work effectively on Teams, and An ability to communicate effectively in writing and orally. Project Planning: Strategies, Assessment Tools and Time-Line: The project will be implemented in three years starting at the beginning of Term-1, 1431/1432 (Fall-2010) after all the appropriate financial and human resources are available. The project team consists of a Principle Investigator, two coinvestigators, and a project coordinator. The Outcomes Assessment process is implemented during the two main semesters (Fall and Spring) and the summer semester is utilized to perform analysis and evaluation, develop recommendations, and to suggest actions to improve the outcomes as well as the assessment process. 1086 The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012) The assessment strategies involve selection of the courses in which the assessment data will be collected and evaluated. The main strategies are to select the courses across the curriculum such that the two aforementioned Communication and Team professional skills are emphasized at sophomore, junior, and senior levels of students’ studies. We call this a Long Term Plan. The students in this will be graduating from UQU by the end of the assessment and quality assurance project. The courses selected were a mix of lab and lecture courses generic purposes so that later our assessment process can be used for any discipline. In order to benefit students who are graduating within two years, we have devised a Short Term Plan and assessment data will be collected only at the junior and senior levels. An exit-interview survey will be conducted for the Short Term Plan to measure the performance of the outcomes assessment and quality assurance project. The graduates of the Short Term Plan will have worked for one year in the field by the end of the outcomes assessment and quality assurance project. This will allow the project team to conduct employer and alumni surveys to evaluate and validate the efficacy of the project. The Long Term Plan is shown in the following Figure 1. STRATEGIES ASSESSMENT TOOLS TIME-LINE ELECTRONIC CIRCUITS (802312) PEER/ SELF AND FACULTY EVALUATION Term-2, 1431/1432AH (Spring 2010) INTRODUCTION TO COMMUNICATION (802322) PEER/SELF AND FACULTY EVALUATION Term-1, 1432/1433AH (Fall 2011) SEMINARS IN EE (802304) PEER/SELF AND FACULTY EVALUATION Term-2, 1432/1433AH (Spring 2012) COMMUNICATION SYSTEMS (802421) PEER/SELF AND FACULTY EVALUATION Term-1, 1433/1434AH (Fall 2012) المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي graduation of these cohorts and by the outcomes of External advisory board. The Short Term Plan is shown in the following Figure 2. STRATEGIES ASSESSMENT TOOLS ELECTROMECHANICAL ENERGY CONVERSION I (802345) PEER /SELF AND FACULTY EVALUATION Term-2, 1431/1432AH (Spring 2010) COMMUNICATION SYSTEMS (802421) PEER/SELF AND FACULTY EVALUATION Term-1, 1432/1433AH (Fall 2011) TIME-LINE EXIT-INTERVIEW AT GRADUATION GRADUATE SURVEY Term-2, 1432/1433AH (Spring 2012) EXTEERNAL ADVISORY BOARD MEEETING ALUMNI AND EMPLOYER SURVEYS Term-2, 1433/1434AH (SPRING2013) Figure 2: Short Term Plan, Strategies, Assessment Tools, and Time of Data Collection The first column in this figure shows a junior level course, Electromechanical Energy Conversion I and a senior level course, Communication Systems. Both of these courses have a laboratory component and we applied our assessment process only on the lab component. The students in this plan will be graduating at the end of Term-2, 1432/1433 AH (Spring 2012). The assessment tools being used in our project are peer evaluation of team skills, self evaluation of both communication and team skills, and faculty evaluation of both outcomes. The other methods of assessment to be used are Exit-Interview surveys, Alumni surveys, and Employers surveys. An exit Interview Survey was conducted for these graduates to measure the performance of the project as well as their communication and team skills. Alumni and Employers survey will be conducted after one year of graduation to validate the performance of the project and the results of these surveys will be discussed with the External Advisory Board members to get their feedback for future actions. The comparison will be done between the evaluations (of assessment data collected) of Short Term Plan and Long Term Plan This will reflect the outcome of applying the educational strategies early on in the curriculum. After the planning phase, the implementation of the outcomes assessment process involves adopting an effective approach to inculcate and emphasize professional communication and team skills in the students and developing performance criteria and rubrics for assessment. The novel approach adopted by the team is derived from the collaborative project-based learning (CPBL) and Problem-Based Learning (PBL) approaches. Figure 1: Long Term Plan, Strategies, Assessment Tools, and Time of Data Collection The first column in this figure shows a sophomore level course, Electronic Circuits, junior level courses, Introduction to Communication & Seminars in Electrical Engineering, and a senior level course, Communication Systems. These courses are a mix of a laboratory and theory. The outcomes assessment process is implemented on both laboratory and theory courses to measure its performance so that it not restrained by discipline. The students in this plan will be graduating at the end of the project. Therefore, the evaluation results will be validated by conducting Alumni and Employer surveys after 1-2 years of 1087 Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University the learner acquisition of critical knowledge, problem solving proficiency, self-directed learning strategies, and team participation skills. The process replicates the commonly used systemic approach to resolving problems or meeting challenges that are encountered in life and career [10]. The two approaches have shared attributes as well as unique attributes as defined in the literature [11]. The shared attributes are as follows: Collaborative Project-Based Learning (Cpbl) And Problem- Base Learning (Pbl): The transformation in higher education from content based assessment to outcomes based assessment has motivated the educators to focus on how the students learn instead of how we teach them. This paradigm shift stresses the need to use instructional strategies or pedagogical approaches that focus on student centered learning. The goal of these approaches for engineering education is to close the gap between student’s learning and industry needs. The advances in technology and changes in the organisational infrastructure put an increased emphasis on team work within the workforce. Workers need to be able to think creatively, solve problems, and make decisions as a team [8]. Therefore, in order to develop the important professional skills, such as communication and teamwork, students need to work in groups or teams. The students can only be well versed in a skill if they practice it and a group learning environment provides opportunity to students to learn how to work on teams, exchange ideas, listen and communicate with others, convince and negotiate based on logical reasoning in order to resolve conflicts. These are important skills for students to develop as research indicates that employers worldwide want graduates who have well developed communication, teamwork and problem solving skills [10]. The group learning environment is provided by two well known pedagogical approaches, namely, Collaborative Project-Based Learning (CPBL) and Problem-Based Learning (PBL). The former approach is well matured for primary and secondary education, but well established principles can be applied in engineering education. The later approach has its origin in medical education. Project-Based Learning is an individual or group activity that goes on over a period of time, resulting in a product, presentation, or performance. It typically has a time line and milestones, and other aspects of formative evaluation as the project proceeds [10]. Collaborative learning has been hailed as giving students an opportunity to engage in discussion, take responsibility for their own learning, and thus become critical thinkers [8][9]. Problem-Based Learning is a teaching method in which students learn through solving a problem [7]. PBL, as defined by Dr. Howard Barrows and Ann Kelson of Southern Illinois University School of Medicine, is both a curriculum and a process. The curriculum consists of carefully selected and designed problems that demand from 1. both approaches engage students in authentic, real world tasks 2. open-ended project or problems have more than one approach or answer 3. projects or problems are intended to simulate professional situations 4. student-centered, teacher-facilitated 5. students work in for extended period of time 6. students encouraged to seek out multiple sources of information 7. emphasis on authentic performance-based assessment 8. ideally both approaches provide adequate time for student reflection and self evaluation. The unique attributes of the two approaches are as follows [11]: Problem-Based Learning begins with problem for a student to solve or learn about problems can be framed in a scenario or case study format problems are somewhat ambiguous to mirror the complexity of real life uses an inquiry model students present conclusion of problem-solving process but do not necessarily create a product defined problem is driving force Project Based Learning begins with an end product or “artifact” in mind production of artifact raises one or more problems for student to solve uses a production model and mirrors real world production activities content knowledge and skills acquired during the production process are important to success student use or present the product they have created end product is driving force In practice, it is likely that the line between projectand problem-based learning is frequently blurred and that the two are used in combination and play complementary roles [10]. Keeping in mind the 1088 The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012) المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي sustainability of the assessment process in terms of faculty engagement and students’ active participation we have used the complementary roles of the two strategies. We have used the Problem-based approach in a laboratory course in which students are required to design a mini-project and in a theory course the students are required to do a research project to solve a problem. Prolem-Based Mini-Project Design In A Laboartory Course: A variant of Project-Based Collaborative Learning (PBCL) approach is applied in the laboratory courses starting with sophomore level for the Long Term Plan. The students in Electronic Circuits laboratory perform several experiments for the first 10 weeks and then they are required to design two mini-projects based on their prior theoretical knowledge in the course and the prior laboratory experience, during the last four weeks of the term. They are required to use Multisim for simulation and then implement the hardware. The teams of 3-4 students are built by the instructor at the beginning of the term and are kept stable till the end of the term. The team skills are assessed throughout the term via faculty evaluation as well as peer and self evaluations. The teams are required to demonstrate their project and present their results in the form of power point presentations in front of an assessment panel at the end of the term. During these presentations individual students are assessed for oral communication skills. The teams are also assessed for written communication skills based on their final technical report. The final demonstration and presentation assessment is considered as a laboratory final exam for grading purposes. The assessment data is used to evaluate the performance of students for the two professional outcomes, namely, ability to work effectively on teams, and ability to communicate effectively orally, in written, and presentation form. The students are required to design these projects with the devices available in the laboratory. Therefore, they need to think of feasible solutions for their projects. Similar approach is applied for the Short Term Plan starting with the junior level course of Electromechanical Energy Conversion I. The examples of these design projects are depicted in the following figures 3, 4, 5, & 6: Figure 3: Mini-Project Design in the Electronic Circuits Laboratory Electromechanical Energy Conversion I: MiniProject I Using only three phase AC voltage source, devise a suitable configuration to control the speed of a DC motor from zero to full speed in both directions. Plot the torque speed characteristics for different values of the driving voltage Proposed block diagrams: The two machines are coupled on the same shaft 3-phase Supply with a Reversal Switch 3-phase Induction Motor The two machines are coupled on the same shaft Self excited DC Compound Generator Field Control Variable DC Voltage Self Excited DC Motor Mechanica l Load Torque Torque and Speed Measurement s Figure 4: Mini-Project Design in the Electromechanical Energy Conversion I Laboratory ELECTRONIC CIRCUITS: Mini- Project II Using the available electric machines in the lab, construct a suitable configuration to generate a variable frequency AC supply. Determine the range of variation of frequency and illustrate how to use this configuration to control the speed of a three phase induction motor. Two solutions are available aAdding a synchronous generator on the same shaft of the variable speed dc motor of experiment 7. The resultant output is a variable frequency AC supply which can be used to drive a three phase induction motor at different speeds. The overall efficiency of this configuration is poor since it involves four machines to 1089 Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University generate the variable frequency AC supply with the advantage of building upon exp.7 bUsing a variable DC supply available in the lab to drive a DC motor on which the synchronous generator will be coupled. The circuit has a higher efficiency than that proposed at (a) The students have to think, discuss in teams, and choose the best solution which reflects their ability of critical thinking and design capabilities. Both miniprojects require students to demonstrate effective team work and good communication skills to present their work. Two Alternate Solutions: 3-phase Induction Motor PROBLEM-BASED RESEARCH PROJECT IN A THEORY COURSE: A variant of Problem-Based learning approach is applied in the junior level theory course, Introduction to Communication. This is a theory course in which students learn various modulation techniques for electromagnetic transmission of message signals. In the beginning of the term, the instructor builds stable teams of three students each and assigns a research project to the teams to be completed by the end of the term. The research project is a real situation open ended problem and requires teams to have prior knowledge of the discipline to comprehend and come up with their own problem statement. The following learning model, similar to that of Problem-Based Learning (PBL) approach [11], is applied in the research project: 1. Identify the Problem: Read and analyze the situation described in the problem. Discuss your understanding of the problem as a group to know the important issues to be dealt with. Based on that, your team will need to actively search for information to solve the problem. 2. List the known factors: List the given items in the problem. Your team needs to create a list containing the known items in the problem, what do you know about the problem scenario, and what knowledge individual members of the team have about the problem. 3. Write the Problem Statement in Your own Words: Based on the knowledge your team has, write a brief (2-3 sentences) problem statement in your own words describing the situation and what your team is trying to solve, find, or produce. You may need to revise the problem statement when new information is revealed and brought to affect the problem scenario. 4. List Unknown Factors and What is to be Determined: List all the questions you consider to be answered before solving the problem. List the items to be determined to solve the problem. What is required to be done? Ask your instructor the questions you may have. 5. Write Alternate Solutions to the Problem: Based on your research you may come up with more than one solution to the problem. Write all possible solutions to the problem and compare and contrast various solutions. 6. Present Your Findings: Prepare a report in which you make your recommendations, draw a conclusion, or present alternate solutions to the problem. You are required to present and defend your results in the form of a power point presentation and a technical written report. An example of the Problem assigned to teams in Introduction to Communication course is described as follows: The two machines are coupled on the same shaft The two machines are coupled on the same shaft 3-Phase Supply with a Reversal Switch This approach needs teams to interact more with instructor and that is possible only in a laboratory course. In case of theory course, that level of interaction is not possible. Therefore, we have used a variant of Problem-Based Learning approach in the theory courses. Self excited DC compound Generator Variable Frequency AC Voltage Variable DC Voltage Self Excited DC Motor Synchronous Generator Field Control Torque and speed Measurements Mechanical Load 3-phase Induction Motor Figure 5: Mini-Project Design in the Electromechanical Energy Conversion I Laboratory Configuration (a) The two machines are coupled on the same shaft Variable DC Voltage The two machines are coupled on the same shaft Variable Self Excited DC Motor Synchronous Generator Frequency AC Voltage 3-phase Induction Motor Mechanical Load Torque and Speed Measurements Figure 6: Mini-Project Design in the Electromechanical Energy Conversion I Laboratory Configuration (b) These projects also expose students to design experience, critical thinking, and problem solving skills other than the main two professional outcomes of effective team work and effective communication. The Project-Based Collaborative Learning approach is used in these mini-projects but it is complemented by inquiry model of Problem-Based learning. The design of the project start with an open-ended problem and students need to utilize their prior knowledge to understand the problem and come up with their own problem statement and then utilize multiple sources to get information to solve that problem. This approach is different than the conventional Project-Based Learning strategy because it is not using structured projects and the instructor’s role is not that of a tutor or facilitator. The instructor work as an advisor with expertise in the discipline and student’s dependency on the instructor is very low regarding design and implementation of the project. 1090 The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012) المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي Table 1: Faculty Evaluation of Student Outcome1: An ability to work effectively on teams Introduction To Communication: Research-Project Problem Statement You are required to design an AM modulator to generate a modulated signal (as shown in the figure below). An ability to work effectively in teams Faculty Evaluation 4: Exemplary, 3: Satisfactory, 2: Progressing, 1: Unsatisfactory SCALES PERFORMANCE INDICATOR 4 1. Demonstrate understanding of team-roles when assigned. 2. Demonstrates good listening skills. Figure 7: Modulated Signal Generator Message signal: is baseband signal The message signal with bandwidth equals 5 kHz. The dynamic range of the signal is from 3 mV to -2 mV. D I M E N T I O N S 3. Cooperates with teammates. 4 Makes fair decisions. 5. Be punctual. 6 The tasks are described, those tasks are divided among team members according to individuals’ abilities and interests. 7. Develop ways to resolve conflicts and reach agreement. 8. Respond positively to teammates’ ideas. Figure 8: Baseband Message Signal Modulated Signal: Local government regulation restricts that the modulated signal should not exist outside of 800 kHz to 950 kHz band. Carrier Frequency: The carrier frequency generator can only generate between 260 kHz to 300 kHz. It can generate the following types of waveforms with zero off-set: i. Full rectified wave, ii. Saw tooth wave, iii. Square wave, iv. Triangular wave v. Cosine wave Modulation Index: Should not be less than 90%. You are asked to do the following: 1. Draw the modulator diagram. 2. Decide about the settings of the carrier frequency generator (the frequency and waveform). Show your reasoning why you selected a particular waveform and frequency. [Hint: you will have to analyze the carrier waveforms given above using Fourier series]. 3. Decide about the filter (filter type, center frequency and bandwidth). 4. Draw the spectrum (as exact as you can) at the points 'a', 'b' and 'c' in the figure above. As discussed earlier, the main goal of the Outcomes Assessment and Quality Assurance project is to measure the performance of students for the two professional outcomes. Therefore, the project team has developed different assessment methods as described in the next section. D E S C R I P T O R S Performs all duties assigned and actively assist others. Consistently listens and responds to others appropriately. 3 2 1 Performs duties that are assigned. Inconsistently performs duties that are assigned. Does not perform any duties of assigned team role. Listens most of the time. Usually doing most of the talking-rarely allows others to speak. Is always talking and never allows anyone else to speak. Sometimes argues. Usually argues with teammates. Never argues with teammates. Rarely argues. Always helps team to reach a fair decision. Usually considers all views. Hands in all assignments on time. Hands in most assignments on time. Understands the tasks completely and often shows interest. Understands all the tasks completely, always shows interest, and motivates others to work on tasks according to their ability. Always come up with a method to resolve conflicts and reach an agreement. Always responds positively to the teammates’ ideas. Always tries to resolve conflicts and helps team reaching an agreement Usually responds positively to the teammates’ ideas. Often sides with friends instead of considering all. Hands in most assignments late. Usually wants to have things their way. Does not hand in assignments. Does understand the tasks partially and sometimes show interest. Does not understand the tasks And does not show interest. Sometimes tries to resolve conflicts. Always argues with teammates and never try to resolve the conflicts. Often responds positively to the ideas presented by the teammates. Always criticize the ideas presented by a teammate. The assessment process also includes the team self assessment and individual self assessment. The forms for these assessments are adopted form www.performancexpress.org. For the Short Term Plan exit interview surveys are conducted to measure the performance of the graduating student outcomes based on the self assessment. The graduating students are also asked in the exit interview survey to indicate their level of achievement of Communication and Team skills after participating in the outcomes assessment and quality assurance project. The following assessment data was analyzed and the results were very encouraging in the sense that students performed very well for most of the performance indicators of the two outcomes. 1. Faculty Evaluations, 2. Team Self Assessment, 3.Individual Self Assessment Test, 4. Exit Interview Surveys. The students’ responses on team self assessment and individual self assessment as well as on the exitinterview survey were very good. The results of evaluation are discussed in the next section. RESULTS AND DISCUSSION: • OUTCOMES ASSESSMENT PROCESS: The project team researched and developed the performance indicators for the assessment of the two professional outcomes, and the rubrics in the form of descriptors and dimensions [12]. One of the faculty evaluation forms, containing Performance Indicators, Dimensions, and Descriptors, used for the assessment of the student outcome is shown as an example in the following table: 1091 The performance of students for the two student outcomes is evaluated by faculty. The levels of performance are defined as follows: Exemplary = 4.0, Satisfactory = 3.0, Progressing = 2.0, & Unsatisfactory = 1.0 The performance targets were set for all the performance indicators (PIs) as follows: 70% or more students perform at exemplary or satisfactory level. The results of evaluation of Electronic Circuits course for two groups for the two outcomes in the form of bar graphs are shown in the following figures 9 and 10. In the Introduction to Communication (ITC) course during Term-1, 1432/1433 (Fall 2011), most of the teams responded that they were at the norming or performing stages for all these characteristics except characteristics #2 and # 3 but even for those characteristics 60% of the teams indicated that they were at the norming or performing stage. The results are shown in the form of a bar graph in the following figure. Outcome 1: An ability to work effectively on teams 100 90 80 70 60 50 40 30 20 Good Team Characteristics: The team is at the Norming Stage or Performing Stage 10 0 1 2 3 4 5 6 Performance Indicators (PIs) 7 8 Percentage of Teams at Norm ing and Perform ing Levels Percentage of Students who Performed Examplary or Satisfactory Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University Figure 9: Bar Graph of Student Performance for Team Skills Performance Indicators The results in Figures 9 and 10 are for the sophomore level students and indicate that more than 70% of students performed exemplary or satisfactorily in all the PIs for the Team skills outcome. This indicates that most of the students participating in the project have achieved the professional outcome of effective team skills. More than 70% of students performed exemplary or satisfactorily in all the PIs for the Communication skills outcome except #1. This indicates that most of the students participating in the project have achieved the professional outcome of effective Communication skills except for the PI #1. This was the first term for the students to participate in the project. They will perform well in the next term to identify audience and organize their presentation to meet the needs of audience. The students need to organize their presentation better. Percentage of Students who performed Exeplary or Satisfactory 90 80 70 60 50 40 30 20 10 2 3 4 5 6 Performance Indicators 7 80 70 60 50 40 Faculty 1 Faculty 2 30 20 10 0 1 2 3 4 Team Characteristics # 5 6 The assessment performed for the self assessment of teams' skills by individual students asked students 18 questions related to team skills to respond as yes or no to indicate achievement of the particular skill. Students from two different groups of ITC course taught by two different faculty members participated in this assessment. The results depicted in Figure 12 indicate that most of the students responded affirmatively that they achieved most of the team skills. The faculty evaluation of the two outcomes for the Introduction to Communication (ITC) course were performed by the faculty during Term-1, 1432/1433 (Fall 2011) and at the end of the term. The team skills evaluation was not promising because, firstly, ITC course was a theoretical course and the teams of student were working outside the classroom and secondly the team-faculty interaction time was not enough for a good assessment to be done 100 1 TARGET 90 Figure 11: Self Assessment of Teams (Percentage of Teams with good Team Characteristics) Outcome 2: An ability to communicate effectively in oral, written, and presentation forms 0 ITC Research Project:Team Characteristics Assessment by Teams, Fall 2011 100 8 Figure 10: Bar Graph of Student Performance for Communication Skills Performance Indicators The team self assessment was meant to see if the teams of students were at a norming or performing stage and the teams were asked to indicate at what stage is their team for 6 different team characteristics as follows: 1. Communication between members, 2. Members’ Commitment, 3. Flexible Purpose, 4. Constant Members Involvement, 5. Normal expectation of Trust, 6. Natural and smooth process 1092 The Second International Arab Conference on Quality Assurance in Higher Education (IACQA'2012) Exit interviews surveys were conducted at the end of Spring 2012. Only 9 students have participated as of yet. The results of evaluation of these surveys are very good but we will be able to draw conclusions only after getting an appropriate size of data. The results of exitinterview surveys are shown in Figure 15 and 16. ITC Research Project:Team Skills Achievement Self Assessment, Fall 2011 COMM. SKILLS SELF ASSESSMENT TEST 60 TARGET 55 Faculty 1 Faculty 2 50 100 CUMULATED WEIGHTED AVERAGE 10 20 30 40 50 60 70 80 90 Percentage of Students Achieved the Particulat Team Skill 45 40 35 30 25 20 15 10 Figure 12: team Skills Self Assessment by Individual Students (Percentage of Students who achieved the particular skill) 5 0 1 2 3 4 STUDENT # 70 60 50 CUMULATED WEIGHTED AVERAGE Percentage of Students w ith Exam plary or Satisfacory Performance 80 Faculty 2, Group 2 Faculty 2, Group 1 20 10 0 1 2 3 4 5 6 Performance Indicator # 7 8 Figure 13: Faculty Evaluation of Students’ Communication Skills (Percentage of Students with Exemplary and Satisfactory Performance) Percentage of Students with Examplary or Satisfacory Performance Faculty 1 Faculty 2, Group 2 Faculty 2, Group1 TARGET 70 60 50 40 30 20 10 0 1 2 3 4 5 6 Performance Indicator # 7 STUDENT # PROJECT ASSESSMENT FOR TEAM SKILLS 60 58 TARGET 56 54 52 50 48 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 (EXIT INTERVIEW SURVEY FALL 2011) STUDENT # Figure 16: Cumulative Team Self Assessment Test & Cumulative Project Assessment for TEAM SKILLS These outcomes are also linked to other curriculum outcomes of the Electrical Engineering Program. The success of this Outcomes Assessment and Quality Assurance (OAAQA) Project will lead to the development of a comprehensive assessment program for Student Outcomes of the EE curriculum that will enhance the academic standards and educational level of the graduates. We employed a novel strategy to let the students learn how to become better collaborators and effective communicators. Our approach utilizes the best practices of the two well know instructional strategies; Collaborative Project-Based Learning (CPBL) and Problem-Based learning (PBL). Keeping in mind the sustainability of the assessment process in terms of faculty engagement and students’ active participation we have used the complementary roles of the two strategies. We have used the Problem-based Faculty Evaluation of ITC Research Project:Team Skills Fall 2011 80 9 TEAM SKILLS SELF ASSESSMENT TEST 60 58 TARGET 56 54 52 50 48 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 STUDENT # 100 90 8 The Student Outcomes assessment and Quality Assurance project at UQU plays an important role in developing curriculum for the essential outcomes, namely; Ability to work effectively on Teams & Ability to Communicate Effectively, that are highly desired by the industry and other employers as well as being one of the important factors in acquiring accreditation for the Electrical Engineering (EE) Program. 90 30 7 (EXIT iNTERVIEW SURVEY FALL 2011) Conclusion: Faculty Evaluation of ITC Research Project:Communication Skills Fall 2011 100 TARGET Faculty 1 6 Figure 15: Cumulative Comm. Skills Self Assessment Test & Cumulative Project Assessment for COMM. SKILLS The communication skills performance was done much better than that of the team skills because students were required to present their research project in front of an assessment panel. The results of the two evaluations are shown in the form of bar graphs in Figure 13, and 14. 40 5 PROJECT ASSESSMENT FOR COMM. SKILLS 60 58 TARGET 56 54 52 50 48 46 44 42 40 38 36 34 32 30 28 26 24 22 20 18 16 14 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 CUMULATED WEIGHTED AVERAGE 0 CUMULATED WEIGHTED AVERAGE Q uestion # TARGET 18 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 المؤتمر العربي الدولي الثاني لضمان جودة التعليم العالي 8 Figure 14: Faculty Evaluation of Students’ Team Skills (Percentage of Students with Exemplary and Satisfactory Performance) 1093 Outcomes Assessment and Quality Assurance Project at Electrical Engineering Department of Umm Al-Qura University Project approach in a laboratory course in which students are required to design a mini-project and in a theory course the students are required to perform a research project to solve a problem. The feedback from students and faculty indicates that our approach keeps the students interested in learning and the faculty is not burnt out in terms of conventional facilitator or tutor. The faculty role is that of an advisor and students’ dependency on the instructor is very low. The self assessment and peer assessment in our project is not biased because it does not count toward students grading. The team assessment in theory courses needs some adjustments so that students’ performance assessment in regards to Team skills can be reliable and consistent. [9]. [10]. [11]. [12]. Acknowledgement: We would like to acknowledge the support of Ministry of Higher Education, Kingdom of Saudi Arabia for partially funding this project. We are also grateful to Umm Al-Qura University for support toward this project in terms of equipment and human resources. The moral support of Dr. Esam Ahdal, Vice Rector of Development and Community Services and Dr. Abul Razaq Sultan, the supervisor of the project is worth mentioning. We are very grateful to both of them for supporting this very important project toward the development of Electrical Engineering Department at Umm Al-Qura University. [13]. References: [1]. [2]. [3]. [4]. [5]. [6]. [7]. [8]. ABET (Accreditation Board for Engineering and Technology) Criteria for accrediting engineering programs; Effective for Evaluations during the 2010-2011 Accreditation Cycle. <http://www.abet.org/images/Criteria/201112EACCriteria.pdf>, accessed September 2010. Muhammad A. Malik, Mohammed T. Simsim, Abdel Monem A. Abbas, Assessment of Student Professional Outcomes for Quality Assurance and Continuous Improvement, First International Arab Conference on Quality Assurance in Higher Education, May10, 2011, Amman, Jordan. Briedes D., (2002) Developing Effective Assessment of Student Professional Outcomes, International Journal Engineering Education, vol.18, No. 2, pp. 208-216. Smerdon, E. T., (1998) President’s letter: Global Engineering, PRISM. vol. 8, no. 4, The American Society for Engineering Education, Washington, DC, p 37. Introduction to Communication, Why Communications Skills are so Important, http://www.mindtools.com/CommSkll/Communi cationIntro.htm Theory Behind PBL, http://ldt.stanford.edu/~jeepark/jeepark +portfolio/PBL/howtodopbl.htm Patrick Walsh, Problem Based Learning in Engineering, International Symposium for 1094 Engineering Education, 2007, Dublin City University, Ireland. Gokhale, A.A., Collaborative Learning Enhances Critical Thinking, J. of Technology Educ., 7, 1 (1995) Totten, S., Sills, T., Digby, A. and Russ, P., Cooperative Learning: a Guide to Research. New York: Garland (1991). Roisin Donnelly and Marian Fitzmaurice, Collaborative Project-Based Learning and Problem-Based Learning in Higher Education: A Consideration of Tutor and Student Roles in Learner-Focused Strategies, retrieved from www.aishe.org on January 20, 2012. PBL, Problem Based Learning, Marico Center for Learning and Instruction, Comparing/Contrasting Problem and ProjectBased Learning Approaches. Gloria Rogers, (2010) Bottom Line: Assessment, IDEAL, The Institute for the Development of Excellence in Assessment Leadership, August 2 – August 6, 2010.