2012-2013 Annual Program Assessment Report Please submit report to your department chair or program coordinator, the Associate Dean of your College and the assessment office by Monday, September 30, 2013. You may submit a separate report for each program which conducted assessment activities. College: COBAE Department: GBUS Program: MBA Assessment liaison: Deborah Cours/Glen Whitman 1. Overview of Annual Assessment Project(s). Provide a brief overview of this year’s assessment plan and process. To be consistent with the College’s assessment program, the MBA program has adopted a bi-annual assurance of learning plan. This year, we assessed all program SLO’s in bookend classes: GBUS 600 (required first semester) and GBUS 697/698 (culminating experience). MBA also uses the EBI Exit Survey every other year, it was used in 2012-13. We also compile each semester aggregate evaluations of student ratings of courses and how well they help achieve the program learning goals. 2. Assessment Buy-In. Describe how your chair and faculty were involved in assessment related activities. Did department meetings include discussion of student learning assessment in a manner that included the department faculty as a whole? Assessment data including external surveys of students and alumni are shared and discussed thoroughly with the MBA faculty through the Graduate Faculty Retreat held each semester and with the Graduate Committee. 3. Student Learning Outcome Assessment Project. Answer items a-f for each SLO assessed this year. If you assessed an additional SLO, copy and paste items a-f below, BEFORE you answer them here, to provide additional reporting space. 1 SUMMARY OF CSUN MBA LEARNING OUTCOME GOALS: Integrate Theory and Practice Understand interdisciplinary relationships in the firm and its environment Demonstrate senior-level management skills: analytic thinking Demonstrate senior-level management skills: Clear Communication Demonstrate senior-level management skills: Effective Teamwork Demonstrate senior-level management skills: Global Perspective Demonstrate senior-level management skills: Ethical Practices 2 3a. Which Student Learning Outcome was measured this year? Integrate Theory and Practice 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply) Critical Thinking 3c. Does this learning outcome align with University’s commitment to supporting diversity through the cultivation and exchange of a wide variety of ideas and points of view? In what ways did the assessed SLO incorporate diverse perspectives related to race, ethnic/cultural identity/cultural orientations, religion, sexual orientation, gender/gender identity, disability, socio-economic status, veteran status, national origin, age, language, and employment rank? Integrating theory and practice requires an understanding of the diverse, real world context of organizations. 3d. What direct and/or indirect instrument(s) were used to measure this SLO? DIRECT MEASURES: GBUS 600: Management paper (Rubric) GBUS 698: MBA Consulting Project (rubric) / GBUS 697: MBA Comprehensive Exam (rubric) INDIRECT MEASURES: EBI Exit Survey & Student Evaluations of Courses on Achieving Program Learning Goals 3e. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Evaluation of student performance against a benchmark of 80% in points or 3.0/4.0 on GPA scale. We expect 90% of students to achieve this benchmark. GBUS 600: Management paper (Rubric) n = 25 GBUS 698A: MBA Consulting Project, completing a real world consulting project. n = 44 GBUS 697G: MBA Comprehensive Exam, case analysis, n = 11 3 3f. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. GBUS 600: Management paper (Rubric) The average score on the assignment was 3.2; however, 15% of the students scored less than a B (3.0). As this is the first class in the MBA program, we hope to see growth in their ability to apply theories to real world contexts. GBUS 698A: 100% of the students enrolled successfully completed the MBA Consulting Project, completing a real world consulting project. The average rating on the scale was 3.5, and all students performed above a 3.0/4.0 standard. GBUS 697G: 91% (10 of 11 students) satisfactorily demonstrated an ability to apply and integrate theory to a real world (Harvard case) problem. Students were better at applying analytical tools in external and internal analysis than strategic analysis. OTHER: The indirect measures show wide variation of student agreement of individual courses and their ability to integrate theory and learning. 3g. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) Based on previously years’ results, faculty continue to develop more hands-on experiences, simulations and opportunities to apply theory to real world experiences. Although students are generally performing well at this learning outcome, the student exit surveys and course evaluations reflect a continued need for faculty to focus on application of theory to practice. This was discussed at the Faculty Retreat. Curriculum review planned for the MBA program during 13-14 may address this issue as well through recommended curriculum changes and pedagogy. Reinforces need for “strategy” introduction earlier in the curriculum so students have more time to practice and develop. 4 3 CONTINUED 3a. Which Student Learning Outcome was measured this year? Understand interdisciplinary relationships in the firm and its environment 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply) Critical Thinking Information Literacy 3c. Does this learning outcome align with University’s commitment to supporting diversity through the cultivation and exchange of a wide variety of ideas and points of view? In what ways did the assessed SLO incorporate diverse perspectives related to race, ethnic/cultural identity/cultural orientations, religion, sexual orientation, gender/gender identity, disability, socio-economic status, veteran status, national origin, age, language, and employment rank? 3d. What direct and/or indirect instrument(s) were used to measure this SLO? DIRECT MEASURES: GBUS 600: Industry Paper (rubric) GBUS 698 /GBUS 697 Rubric INDIRECT MEASURES: EBI Exit Survey & Student Evaluations of Courses on Achieving Program Learning Goals 3e. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Evaluation of student performance against a benchmark of 80% in points or 3.0/4.0 on GPA scale. GBUS 600: Management paper (Rubric) n = 25 5 GBUS 698A: MBA Consulting Project, completing a real world consulting project. n = 44 GBUS 697G: MBA Comprehensive Exam, case analysis, n = 11 3f. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. GBUS 600: Company Paper (rubric): Students scored an average of 3.2 on a 4.0 scale, with 80% (20/25) scoring at a 3.0 or higher level on the items that demonstrate understanding of interdisciplinary relationships in the firm. GBUS 698: MBA Consulting Project: 93% (42/45) of the students demonstrated a satisfactory understanding of interdisciplinary relationships in the firm and its environment. Students struggle most with implementation, but because the culminating projects are all interdisciplinary in nature, the students demonstrate a good ability to integrate the disciplines and environmental analysis. GBUS 697 Rubric: 91% (10/11) demonstrated a satisfactory ability to integrate the functional areas. The students are very strong at internal and external analysis, and satisfactory at strategic analysis and recommendations. 3g. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) The curriculum is designed to focus throughout the entire program on integrating the disciplines. Porter Five Forces and SWOT are introduced in the first class in the curriculum and students demonstrate an ability to perform these well at the end of program. Implementation remains the weakest area, so curriculum innovations should force students to think through application. Hands on projects that require implementation earlier in the program will help build this skill. 6 3 CONTINUED 3a. Which Student Learning Outcome was measured this year? Demonstrate senior-level management skills: analytic thinking 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply) Critical Thinking Written Communication 3c. Does this learning outcome align with University’s commitment to supporting diversity through the cultivation and exchange of a wide variety of ideas and points of view? In what ways did the assessed SLO incorporate diverse perspectives related to race, ethnic/cultural identity/cultural orientations, religion, sexual orientation, gender/gender identity, disability, socio-economic status, veteran status, national origin, age, language, and employment rank? 3d. What direct and/or indirect instrument(s) were used to measure this SLO? DIRECT: GBUS 600 Economics Paper (rubric) GBUS 697G/GBUS 698A: Rubrics 3e. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Evaluation of student performance against a benchmark of 80% in points or 3.0/4.0 on GPA scale. GBUS 600: Economics paper (Rubric) n = 25 GBUS 698A: MBA Consulting Project, completing a real world consulting project. n = 44 GBUS 697G: MBA Comprehensive Exam, case analysis, n = 11 7 3f. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. GBUS 600 Economics Paper (rubric): This assignment requires difficult economic analysis of a firm’s decision about in or outsourcing activities. The analysis required involves tradeoff analysis, cost-benefit analysis and comprehensive consideration of the firm’s activities and resources and opportunity costs. In this assignment 8/25 (32%) failed to demonstrate at least a B (3.0/4.0) performance. GBUS 697G MBA Comprehensive Exam: The comprehensive exam requires analytical thinking from problem identification through alternative consideration and strategic recommendations. 10/11 of the students (91%) satisfactorily demonstrated strong analytical thinking in their ability to correctly frame a problem, consider alternatives and recommend a solution. GBUS 698A: MBA Consulting Project: All students (100%) demonstrated satisfactorily an ability, at time of presenting the final project, to think analytically and defend their analysis and recommendations. 3g. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) The results of this assessment, especially when we compare performance in the first class (GBUS 600) to the last requirement (GBUS 697/GBUS 698) demonstrates a growing ability of analytical thinking through the program. Next year, the GMAT will add a new section on Integrated Reasoning and we might compare these scores to the performance of our students in assignments that require analytical thinking to consider how we might use that new GMAT component in admissions and mentoring. 8 3 CONTINUED 3a. Which Student Learning Outcome was measured this year? Demonstrate senior-level management skills: Clear Communication 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply) Oral Communication Written Communication 3c. Does this learning outcome align with University’s commitment to supporting diversity through the cultivation and exchange of a wide variety of ideas and points of view? In what ways did the assessed SLO incorporate diverse perspectives related to race, ethnic/cultural identity/cultural orientations, religion, sexual orientation, gender/gender identity, disability, socio-economic status, veteran status, national origin, age, language, and employment rank? 3d. What direct and/or indirect instrument(s) were used to measure this SLO? DIRECT MEASURES: GBUS 600: Industry Paper Presentation/Company Paper report (rubric) GBUS 698 /GBUS 697 Rubric 3e. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Evaluation of student performance against a benchmark of 80% in points or 3.0/4.0 on GPA scale. GBUS 600: Company paper (written), industry presentation (oral) (Rubric) n = 25 GBUS 698A: MBA Consulting Project, completing a real world consulting project. n = 44 GBUS 697G: MBA Comprehensive Exam, case analysis, n = 11 9 3f. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. Written Communication: GBUS 600 Company Paper (rubric): Writing is assessed and coached throughout the class and culminates in the individual company report. 80% of the students demonstrate satisfactory graduate business writing skills in this paper. International students struggle the most. GBUS 697G MBA Comprehensive Exam: Although writing is not directly a component of the grade given the time constraints of this exam, the effectiveness of the communication is evaluated. 100% were evaluated as satisfactorily written. GBUS 698A: MBA Consulting Project: 95% of the students demonstrated individual ability to write well, with opportunities to edit and rewrite. International students struggle the most, but seek out resources such as peer editing, to produce a final document that is satisfactory. Oral communication: GBUS 600 Industry project presentation: MBA students are generally very good presenters. International students struggle the most, but having instruction on elements of a good presentation (vs. a speech) and a defined, prepared role in this presentation seem to help them be more successful. 88% of the students earned at least a 3.0/4.0 on effective oral presentation skills, slightly below our goal of 90%. GBUS 698A MBA Consulting Project presentation: 100% of the students demonstrated an effective formal business presentation. All students presented to clients and faculty advisors in a stressful situation and were prepared, clear communicators and able to answer questions from clients and faculty. 3g. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes 10 in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) The results, especially in the oral presentation of the MBA Consulting Project, reflect increased coaching and emphasis throughout the program on business presentation, and specifically coaching to the students before they began their project. However, we need to continue to consider how to help international students be comfortable and prepared – especially if we increase recruitment of international students. 11 3 CONTINUED 3a. Which Student Learning Outcome was measured this year? Demonstrate senior-level management skills: Effective Teamwork 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply) 3c. Does this learning outcome align with University’s commitment to supporting diversity through the cultivation and exchange of a wide variety of ideas and points of view? In what ways did the assessed SLO incorporate diverse perspectives related to race, ethnic/cultural identity/cultural orientations, religion, sexual orientation, gender/gender identity, disability, socio-economic status, veteran status, national origin, age, language, and employment rank? Effective teamwork requires diversity sensitivity and understanding 3d. What direct and/or indirect instrument(s) were used to measure this SLO? DIRECT MEASURES: GBUS 600: Industry Project GBUS 698 MBA Consulting Project 3e. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Evaluation of student performance against a benchmark of 80% in points or 3.0/4.0 on GPA scale. GBUS 600: Industry Project = 25; self & peer evaluations, professor observations 12 GBUS 698A: MBA Consulting Project, completing a real world consulting project. n = 44 self & peer evaluations, faculty and client observations 3f. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. GBUS 600: Most of the students are effective at working with others. 42/44 students (95%) were evaluated as effective at teamwork including timeliness, preparation and contributions and respect. Two students received slightly lower scores and were provided feedback about their poor communication and failure to meet expectations to the team. GBUS 698A: 100% of the team members were rated as effective at teamwork, in communication, preparation, contribution and delivery. 3g. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) Increased use of peer evaluations at midterm or midpoints of assignments seems to influence students and make them more aware of how they are seen by their team members. Engaging in team building and bonding exercises seems to improve effective teamwork. 13 3 CONTINUED 3a. Which Student Learning Outcome was measured this year? Demonstrate senior-level management skills: Ethical Practices 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply) 3c. Does this learning outcome align with University’s commitment to supporting diversity through the cultivation and exchange of a wide variety of ideas and points of view? In what ways did the assessed SLO incorporate diverse perspectives related to race, ethnic/cultural identity/cultural orientations, religion, sexual orientation, gender/gender identity, disability, socio-economic status, veteran status, national origin, age, language, and employment rank? Ethical problems in business often involve cultural and personal differences. 3d. What direct and/or indirect instrument(s) were used to measure this SLO? DIRECT MEASURES: GBUS 600: Ethical Analysis assignment and quiz GBUS 698 /GBUS 697 Rubric 3e. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Evaluation of student performance against a benchmark of 80% in points or 3.0/4.0 on GPA scale. GBUS 600: Ethics assignment and quiz, n = 25 GBUS 698A: MBA Consulting Project, completing a real world consulting project. n = 44 GBUS 697G: MBA Comprehensive Exam, case analysis, n = 11 3f. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. 14 GBUS 600: Students are presented with a lecture on ethical analysis, and asked to prepare and discuss scenarios. Following this assignment, they complete a short written exercise and quiz. `40/44 (91%) earned at least 80% on this assignment. GBUS 697G: The case selected had a potential ethical problem; 10 of 11 (91%) of the students identified and addressed it in their analysis. GBUS 698A: Students seem good at identifying potential ethical conflicts, problems and challenges. 100% of students were identified as satisfactorily identifying and addressing ethical challenges in their project. 3g. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) Although our performance is good here, we need better end of program measures. Faculty discussion at the retreat focused on how we can better expose students to ethical challenges they may face and examine their reactions. MGT 693 should be assessed in next assessment cycle because that course includes significant discussion about ethical business behavior. 15 3 CONTINUED 3a. Which Student Learning Outcome was measured this year? Demonstrate senior-level management skills: Global Perspective 3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply) 3c. Does this learning outcome align with University’s commitment to supporting diversity through the cultivation and exchange of a wide variety of ideas and points of view? In what ways did the assessed SLO incorporate diverse perspectives related to race, ethnic/cultural identity/cultural orientations, religion, sexual orientation, gender/gender identity, disability, socio-economic status, veteran status, national origin, age, language, and employment rank? Global perspective inherently includes cultural sensitivity. 3d. What direct and/or indirect instrument(s) were used to measure this SLO? GBUS 698 /GBUS 697 Rubric 3e. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used. Evaluation of student performance against a benchmark of 80% in points or 3.0/4.0 on GPA scale. GBUS 698A: MBA Consulting Project, completing a real world consulting project. n = 44 GBUS 697G: MBA Comprehensive Exam, case analysis, n = 11 3f. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the collected evidence. GBUS 697G: the case selected featured a global firm considering expansion. 10/11 students successfully analyzed the global environment, including analysis of multiple countries. 16 GBUS 698A: All students had to consider international competition and the role of their client’s firm in the global environment. All performed satisfactorily on this requirement. 3g. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised assessment instruments, other academic programmatic changes, and changes to the assessment plan.) Quite frankly, we need a better assessment of global perspective. AACSB increasing means international experiences, and while an increased number of our students are taking advantage of study trips and exchange opportunities, we need to develop a better way to assess this learning goal. 17 4. Assessment of Previous Changes: Present documentation that demonstrates how the previous changes in the program resulted in improved student learning. See above. 5. Changes to SLOs? Please attach an updated course alignment matrix if any changes were made. (Refer to the Curriculum Alignment Matrix Template, http://www.csun.edu/assessment/forms_guides.html.) No changes made. 6. Assessment Plan: Evaluate the effectiveness of your 5 year assessment plan. How well did it inform and guide your assessment work this academic year? What process is used to develop/update the 5 year assessment plan? Please attach an updated 5 year assessment plan for 2013-2018. (Refer to Five Year Planning Template, plan B or C, http://www.csun.edu/assessment/forms_guides.html.) (SEE ATTACHED) 7. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your program? Please provide citation or discuss. NOT TO MY KNOWLEDGE. 8. Other information, assessment or reflective activities or processes not captured above. It is hard to capture the amount of discussion, collaboration, mentoring and exchange of ideas that occur at our semesterly graduate faculty retreats. We present data from external and internal sources. 18