STUDENT LEARNING ASSESSMENT PROGRAM SUMMARY FORM AY 2010-2011 Degree and Program Name: Bachelor of Science in Business – Business Core Submitted By: Christie Roszkowski Associate Chair, School of Business Please complete a separate worksheet for each academic program (major, minor) at each level (undergraduate, graduate) in your department. Worksheets are due to CASA this year by June 15, 2011. Worksheets should be sent electronically to kjsanders@eiu.edu and should also be submitted to your college dean. For information about assessment or help with your assessment plans, visit the Assessment webpage at http://www.eiu.edu/~assess/ or contact Karla Sanders in CASA at 581-6056. PART ONE What are the learning objectives? 1. How, where, and when are they assessed? What are the expectations? What are the results? Committee/ person responsible? How are results shared? Associate Chair coordinates administration of ETS Major Field Test and organizes and reports results to Business Curriculum Committee Associate Chair coordinates administration of ETS Major Field Test and organizes and reports results to Business Curriculum Committee Associate Chair coordinates administration of ETS Major Field Test and organizes and reports results to Business Curriculum Committee Associate Chair coordinates administration of ETS Major Field Test and organizes and reports results to Business Curriculum Committee Demonstrate basic knowledge of functional areas of business. 1.1. Demonstrate knowledge of basic financial and managerial accounting terminology, theory, and principles. ETS Major Field Test in Business administered each semester in BUS 4360 Students’ mean score will be at or above the national mean EIU Mean FA 10: 47% SP 11: 43% National Mean: 45.1% 1.2. Demonstrate knowledge of basic finance terminology, theory, and principles. ETS Major Field Test in Business administered each semester in BUS 4360 Students’ mean score will be at or above the national mean EIU Mean FA 10: 57% SP 11: 45% National Mean: 43.9% 1.3. Demonstrate knowledge of basic management terminology, theory, and principles. ETS Major Field Test in Business administered each semester in BUS 4360 Students’ mean score will be at or above the national mean EIU Mean FA 10: 54% SP 11: 56% National Mean: 58.4% 1.4. Demonstrate knowledge of basic marketing terminology, theory, and principles. ETS Major Field Test in Business administered each semester in BUS 4360 Students’ mean score will be at or above the national mean EIU Mean FA 10: 51% SP 11: 54% National Mean: 56.3% 2. Recognize and analyze ethical and legal issues in the business decision-making process. 2.1. Articulate relevant ethical principles and values from the perspectives of various business stakeholders and apply those theories in making and assessing business decisions; 2.2. Compare and contrast the characteristics of business structures/legal entities; 2.3. Apply fundamental principles of tort, contract, agency, intellectual property, and employment law in analyzing business decisions; Embedded assignment in BUS 4360 FA10: Written assignment piloted in SU10 was used in all sections of BUS 4360 SP11: Oral debates on ethical issues in one section of BUS 4360 Embedded objective test questions administered each semester in BUS 2750 Embedded objective test questions administered each semester in BUS 2750 At least 70% of the students will score proficient or better based on ethics rubric 70% average scores on questions across all sections 70% average scores on questions across all sections SU10: 60.6% of the students scored proficient or better FA10: 70.3% of the students scored proficient or better SP11: Average scores were3.6 (on a 6 point scale). Due to scale used on the rubric, proficiency was not clearly defined Average scores across all sections: FA10: 73.5% SP11: 74.9% Average scores across all sections: FA10 Torts: 77.7% Contracts: 87.6% IP: 71.6% Employment: 84.5% Agency: 65.8% SP11 Torts: 78.3% Contracts: 87.5% IP: 68.5% Employment: 83.1% Agency: 60.3% Assignment developed or selected by BUS 4360 faculty (submitted to Business Curriculum Committee for review); results reported to Business Curriculum Committee. Faculty teaching BUS 2750 develop questions (submitted to Business Curriculum Committee for review) and administer each semester; Dr. Denise Smith organizes and reports results to Business Curriculum Committee Faculty teaching BUS 2750 develop questions (submitted to Business Curriculum Committee for review) and administer each semester; Dr. Denise Smith organizes and reports results to Business Curriculum Committee 3. Understand the role of technology in organizations and use technology effectively. 3.1. Demonstrate competency in business productivity software (e.g., Microsoft Office); 3.2. Effectively use technology tools to analyze business information (e.g., spreadsheets, databases, financial calculators); 3.3. Analyze, design, and develop a small relational database using a current development methodology. Excel and Access hands-on tests administered in BUS 1950 in FA10 and SP11 70% average scores on both exams across all sections Average scores across all sections FA 10 Excel: 81.9% Access: 85.3% SP 11 Excel: 74.6% Access: 85.9% Embedded capital budgeting final exam problem in BUS 3710 in FA10 (4 sections) and SP11 (2 sections) Embedded case analysis in BUS 3950 70% average scores on all questions across all sections BUS3710 problem average scores (across all sections) FA10: 74.2% SP11: 68.0% BUS3950 case: data collected but organization and analysis not complete Database design and development project in all sections of BUS 3500 in FA10 and SP11 70% of students will score proficient or better based on Database Rubric Results of all database projects FA10: 87.9% of students scored proficient or better SP11: 90.8% of students score proficient or better Faculty teaching BUS 1950 administer tests; BUS 1950 coordinator organizes and reports results to Business Curriculum Committee Faculty teaching BUS 3710 develop questions (submitted to Business Curriculum Committee for review) and administer; Dr. Patrick Lach organizes and reports results to Business Curriculum Committee Faculty teaching BUS 3500 develop project (submitted to Business Curriculum Committee for review) and assess using Database Rubric. MIS Assistant Chair organizes and reports results to Business Curriculum Committeee. 4. Communicate effectively. 4.1 Write effective business communications appropriate for the audience; 4.2. Prepare and give a formal oral presentation appropriate for the audience; Embedded writing in assignment in BUS3950 in FA10 (2 sections) and SP11 (4 sections) Sampling of business student submissions to the Electronic Writing Portfolio (samples taken from submissions during calendar year 2010) 70% of students will score proficient or better based on School of Business Writing Rubric BUS3950 writing assignment Based on a random sample of 39 students, 83.9% scored proficient or better EWP writing samples Based on a sample of 40 students, 50% scored proficient or better Embedded oral presentation in selected sections of BUS 3200 and BUS 4360 in FA10 and SP11 [See Part III for discussion of pilot of pre/post assessment in BUS 3200] 70% of students will score proficient or better based on School of Business Oral Presentation Rubric Based on a random sample of 54 oral presentations, 58.6% were assessed at proficient or better Faculty teaching BUS 3950 assess assignments and forward copies of assignments and results to Chair of the Business Curriculum Committee. Faculty committee evaluates sample of assignments and reports results to Business Curriculum Committee Associate Chair arranges taping of presentations Faculty committee evaluates sample of taped presentations and reports results to Business Curriculum Committee PART TWO Describe what your program’s assessment accomplishments since your last report was submitted. Discuss ways in which you have responded to the CASA Director’s comments on last year’s report or simply describe what assessment work was initiated, continued, or completed. The School of Business was successful in maintaining AACSB accreditation. Because much of the review focused on assessment of the core curriculum, the School considers accreditation to be an affirmation of its accreditation efforts and processes. This academic year was one of transition for the School of Business Curriculum Committee which is responsible for the business core curriculum. A new Committee chair was elected and two faculty members (of five faculty members) were newly elected to the Committee. Because Dr. Richard Flight, the new Committee chair, provided strong leadership in the assessment area, the Committee was able to build on past progress. The fact that the Committee was able to continue effective assessment activities notwithstanding the personnel change in the Committee is evidence that assessment has become an integral part of the School. The Curriculum Committee succeeded in better managing and organizing assessment data during the last year: The School of Business was able to purchase a server for assessment data with private funds donated for that purpose. The server provides a centralized site for assessment data and has sufficient capacity to allow easy storage and retrieval of video data. The School of Business has devoted a central file for storage of non-digital data. A clerical staff member has been working with the Curriculum Committee Chair and School of Business Associate Chair to organize those data including cross-referencing to the digital data. During the past academic year, faculty in courses designated for assessment data collection provided data to the Curriculum Committee for both fall and spring semesters. In the past, data generally were provided for only one semester. The Curriculum Committee began using submissions to the Electronic Writing Portfolio as a source of information for assessing written communication. These changes have enabled the Curriculum Committee to focus on analyzing data rather than on the logistics of collection. School of Business faculty, working with the Curriculum Committee, successfully developed several new assessment activities since the last report: As noted in last year’s report, assessment of ethics was piloted in BUS4360 Strategy and Policy (the capstone course for the core) in SU10. One of the BUS 4360 faculty members added a written assignment requiring ethical analysis of a business issue that was evaluated using a rubric adapted from one developed by the Graduate Committee. Data drawn from the assessment of the assignment in FA10 are incorporated in this report. The rubric further was used at the SP11 assessment retreat to assess oral debates on ethical issues that were taped in other sections of BUS 4360 in SP11. These data also are included in this report. As part of last year’s assessment, faculty in BUS3950 Operations Management assessed students’ use of computer software (spreadsheet software) in conjunction with a homework assignment. While the results showed students were able to use the spreadsheet technology, they also showed weaknesses in students’ quantitative analysis and reasoning. In SP11, a BUS3950 faculty member began development of a quantitative analysis rubric and piloted its use on an assignment similar to that used in the previous year. Based on recommendations following last year’s assessment of oral communication, a faculty member piloted preliminary critiquing of oral presentations in BUS 3200. After taping practice sessions of student presentations, the faculty member watched the tapes with the students and critiqued and provided recommendations to the students. As part of the assessment retreat, a faculty committee reviewed tapes of both the preliminary and final presentations. Results are shown on the next page. Pre/Post Oral Communication Presentations Assessment Criteria Organization Language Content Nonverbal Delivery Verbal Delivery Total Average Preliminary Presentation (Pre-critiquing and coaching) 4 point scale 2.09 2.43 1.96 1.89 1.89 2.05 Final Presentation (Post-critiquing and coaching) 4 point scale 2.82 2.89 2.86 2.70 2.52 2.76 Percent Change 25.9% 16.0% 31.3% 29.8% 24.8% 25.5% Response to CASA comments from last year The School continued to use the ETS Field Test in Business focusing on the Assessment Indicators that provide scores by subject area. Last year, the Committee recommended that that the School purchase supplemental data from ETS to provide more in-depth information. In FA10, however, ETS substantially revised the exam. It was decided to postpone obtaining the supplemental information until data from the revised test were available. In SU11, the School will request the supplemental data for the test administration in FA10, SP11, and SU11. Depending on the value of that information—taking into consideration the cost—the Committee will determine whether the test provides data that are sufficiently useful to justify continued use of the test. Indirect data. The School has collected indirect assessment data through surveys conducted in BUS4360. Because of AACSB’s intense focus on direct assessment data, the School has not been using those data for the last two years. These data will be used as part of next year’s reporting. Adopting more specific expectations. The Business Curriculum Committee will consider adopting more specific expectations in FA11. PART THREE Summarize changes and improvements in curriculum, instruction, and learning that have resulted from the implementation of your assessment program. How have you used the data? What have you learned? In light of what you have learned through your assessment efforts this year and in past years, what are your plans for the future? Based on data collected for Learning Objective 2.1, the Committee plans to develop a more systematic approach to ethics in the core curriculum. The Holley Ethics Professor has agreed to assist the Committee in revising the ethics rubric. The revised ethics rubric will be shared with all faculty. The faculty teaching BUS2750 Legal and Social Environment of Business have agreed to reorganize and revise the material used for the introductory module in ethics in FA11. The Curriculum Committee will review ethics coverage in the core curriculum (data were collected in SP11 by the Holley Ethics Professor) and make changes if needed, including changes in assessment. Based on assessment data collected for Learning Objective 3.1 – which generally show that students in BUS1950 are proficient in using Excel and Access – the faculty teaching BUS1950 have added new applications and skills in these areas for the upcoming year. The Business Curriculum Committee will share the data collected for Learning Objective 4.2 (requiring a preliminary presentation involving critiquing and coaching) with faculty, , and will encourage faculty to use critiquing and coaching to improve presentation skills particularly in mid-level courses. The Curriculum Committee will review the pilot data collected in BUS3950 concerning quantitative analysis and reasoning. The Committee plans to add a new learning objective focusing on analysis and/or critical thinking and will determine appropriate assessment of the objective. The Committee plans to revise and improve the quantitative reasoning rubric used in BUS3950. The Committee will share the information gathered, as well as the revised rubric, with all faculty and will encourage faculty to emphasizes the skills throughout the core curriculum. The Curriculum Committee will review its rubric and expectations concerning written communication. Assessment results from the assessment retreat are significantly lower than results from business faculty. It appears that some anchoring may be needed as well as analysis of appropriate expectations for students. Attachments: Ethical Analysis Rubric Quantitative analysis rubric (first draft) Ethical Analysis Rubric Emerging 1 --------------------------2 1. Identification of Ethical Issues 2. Consideration of Stakeholders 3. Analysis of Issue from Multiple Ethical Theories or Perspectives Provides little or no discussion of ethical dilemma or relevant facts Does not identify the ethical issues or inaccurately identifies the ethical issue Does not identify stakeholders Inaccurately identifies stakeholders Provides cursory analysis of ethical issue and/or analyzes issue from one theory/perspective Fails to identify and describe appropriate ethical theories/perspectives Applies ethical theory/perspective in simplistic or mechanical way 4. Discussion of Options 5. Proposal of Decision Fails to identify options for resolving dilemma or identifies only one option for resolving dilemma Limited or no incorporation of ethical theories/perspectives and consideration of stakeholders Fails to propose decision for resolution or proposes decision for resolution with little or no support Fails to relate proposed decision to analysis of issues and options Developing 3 --------------------------4 Discusses the ethical dilemma including some relevant facts Identifies ethical issue but may be unclear or incomplete Identifies stakeholders but may ignore some stakeholders Identifies or implies potential impact on stakeholders Adequately analyzes issues applying multiple ethical theories/perspectives Describes appropriate ethical theories/perspectives but description may be unclear or inaccurate Applies ethical theories/perspectives of theories but fails to fully clarify issues or provide basis for decision-making Describes at least two options for resolving dilemma Discusses some potential ramifications of options Incorporates ethical theories/perspectives and consideration of stakeholders but discussion may be unclear, disjointed or superficial Proposes decision for resolution supported by reasoning Relates proposed decision to analysis of issues and options May provide some suggestions for implementation Mastering 5 --------------------------6 Fully discusses the ethical dilemma including all relevant facts Clearly states the primary ethical issues Identifies secondary and implicit issues Clearly identifies all stakeholders Fully discusses potential impact on stakeholders Thoroughly analyzes issue applying multiple appropriate ethical theories Clearly and accurately describes appropriate ethical theories/perspectives Fully applies ethical theories/perspectives to facts clarifying issues and providing basis for decision-making Clearly identifies and describes various options for resolving dilemma Clearly discusses potential ramifications of various options Effectively integrates various ethical theories/perspectives and consideration of stakeholders in discussion of options Clearly proposes decision for resolution supported by logical and thorough reasoning Proposed decision fully integrates analysis of issues and options Formulates clear implementation plan Quantitative Reasoning Rubric Objectives Emerging 1------------------------2 Infrequently selects the correct mathematical procedures to analyze data. Developing 3------------------------4 Generally selects the correct mathematical procedures to analyze data. Mastering 5------------------------6 Consistently selects the correct mathematical procedures to analyze data. Analysis Infrequently performs the analysis correctly. Generally performs the analysis correctly. Consistently performs the analysis correctly. Interpretation Infrequently interprets quantitative measures accurately. Generally interprets quantitative measures accurately. Consistently interprets quantitative measures accurately. Application Infrequently utilizes quantitative measures to make informed decisions in a variety of contexts. Generally utilizes quantitative measures to make informed decisions in a variety of contexts. Consistently utilizes quantitative measures to make informed decisions in a variety of contexts. Selection Comments: Total: [Adapted from rubric developed at St. John’s University] Score