STUDENT LEARNING ASSESSMENT PROGRAM SUMMARY FORM AY 2013-2014 Degree and Program Name: Submitted By: Bachelor of Science in Business – Business Core Patrick Lach Assurance of Learning Coordinator, School of Business Please use size 10 font or larger. Please complete a separate worksheet for each academic program (major, minor) at each level (undergraduate, graduate) in your department. Worksheets are due to CASA this year by June 13, 2014. Worksheets should be sent electronically to kjsanders@eiu.edu and should also be submitted to your college dean. For information about assessment or help with your assessment plans, visit the Assessment webpage at http://www.eiu.edu/~assess/ or contact Karla Sanders in CASA at 581-6056. PART ONE What are the learning objectives? 1. How, where, and when are they assessed? What are the expectations? What are the results? Committee/ person responsible? How are results shared? Demonstrate basic knowledge of functional areas of business. 1.1. Demonstrate knowledge of basic financial and managerial accounting terminology, theory, and principles. ETS Major Field Test in Business administered each semester in BUS 4360 Students’ mean score will be at or above the national mean EIU Mean: 41.0% National Mean: 43.8% 1.2. Demonstrate knowledge of basic finance terminology, theory, and principles. ETS Major Field Test in Business administered each semester in BUS 4360 Students’ mean score will be at or above the national mean EIU Mean: 42.9% National Mean: 42.6% 1.3. Demonstrate knowledge of basic management terminology, theory, and principles. ETS Major Field Test in Business administered each semester in BUS 4360 Students’ mean score will be at or above the national mean EIU Mean: 54.6% National Mean: 57.0% Associate Chair coordinates administration of ETS Major Field Test and organizes and presents results to Assurance of Learning Coordinator who reports results to the Curriculum Committee Associate Chair coordinates administration of ETS Major Field Test and organizes and presents results to Assurance of Learning Coordinator who reports results to the Curriculum Committee Associate Chair coordinates administration of ETS Major Field Test and organizes and presents results to Assurance of Learning Coordinator who reports results to the Curriculum Committee 1.4. Demonstrate knowledge of basic marketing terminology, theory, and principles. ETS Major Field Test in Business administered each semester in BUS 4360 Students’ mean score will be at or above the national mean Recognize and analyze ethical and legal issues in the business decision-making process. At least 70% of the students 2.1. Articulate relevant ethical will score four ('acceptable) principles and values from or higher on a six point scale the perspectives of various based on graduate Written case assignment business stakeholders and used in BUS 2750 (undergraduate) ethics apply those theories in rubric. Graduate ethics rubric Written case assignment making and assessing is used for BUS 4360 cases were used in BUS 4360 business decisions; and the undergraduate ethics rubric is used for BUS 2750 cases. EIU Mean: 55.9% National Mean: 55.0% Associate Chair coordinates administration of ETS Major Field Test and organizes and presents results to Assurance of Learning Coordinator who reports results to the Curriculum Committee 2. 2.2. Compare and contrast the characteristics of business structures/legal entities; 2.3. Apply fundamental principles of tort, contract, agency, intellectual property, and employment law in analyzing business decisions; Embedded objective test questions administered each semester in BUS 2750 Embedded objective test questions administered each semester in BUS 2750 70% average scores on questions across all sections 70% average scores on questions across all sections 61.2% of students scored four or higher on the BUS 4360 case. 88.0% of students scored 'acceptable' or higher on the BUS 2750 case. Average scores across all sections: 66.3% Average scores across all sections: Torts: 83.0% Contracts: 67.5% Employment: 75.8% IP: 86.3% Agency: 52.9% Assignment developed or selected by BUS 2750 and BUS 4360 faculty; results reported to Assurance of Learning Coordinator who reports results to the Curriculum Committee Faculty teaching BUS 2750 develop questions and administer each semester; Dr. Denise Smith organizes and presents results to Assurance of Learning Coordinator who reports results to the Curriculum Committee Faculty teaching BUS 2750 develop questions and administer each semester; Dr. Denise Smith organizes and presents results to Assurance of Learning Coordinator who reports results to the Curriculum Committee 3. Understand the role of technology in organizations and use technology effectively. 3.1. Demonstrate competency in business productivity software (e.g., Microsoft Office); 3.2. Analyze, design, and develop a small relational database using a current development methodology. Excel and Access hands-on tests administered in BUS 1950 each semester Database design and development project in all sections of BUS 3500 each semester 80% average scores on both exams across all sections 80% of students will score ‘proficient’ or higher based on Database Rubric Average scores across all sections Excel: 77.5% Access: 75.8% Results of all database projects 90.3% of students scored proficient or better Faculty teaching BUS 1950 administer tests; Paul Brown organizes and presents results to Assurance of Learning Coordinator who reports results to Business Curriculum Committee Faculty teaching BUS 3500 develop project and assess using Database Rubric. MIS Assistant Chair organizes and presents results to Assurance of Learning Coordinator who reports results to Business Curriculum Committee 4. Communicate effectively. 4.1 Write effective business communications appropriate for the audience; 4.2. Prepare and give a formal oral presentation appropriate for the audience; Writing samples collected from case assignments in BUS 3950 Embedded oral presentation in selected sections of BUS 4360 in FA13 and SP14 70% of students will score ‘proficient’ or higher based on School of Business Writing Rubric Based on a sample of 58 students, 79.3% scored proficient or better 70% of students will score ‘proficient’ or higher based on School of Business Oral Presentation Rubric Based on a sample of 44 students, 59.1% scored proficient or better BUS 3950 instructor assesses case assignments and submits to Assurance of Learning Coordinator who organizes and reports results to Business Curriculum Committee Assurance of Learning Coordinator arranges taping of presentations Members of the Curriculum Committee are asked to evaluate sample of assignments. Assurance of Learning Coordinator organizes and reports results to Business Curriculum Committee 5. Apply critical thinking skills to reach sound business decisions. 5.1. Clearly identify and articulate issues in structured or unstructured business situations; 5.2. Effectively apply analytic tools for business decisionmaking; 5.3. Effectively use decisionmaking processes to reach appropriate business decisions Cases administered each semester in BUS 3950 Embedded objective test questions administered each semester in BUS 3710 in addition to cases administered in BUS 3950. Cases administered each semester in BUS 3950 At least 70% of the students will score ‘acceptable’ or higher based on the critical thinking rubric 70% average scores on embedded exam questions across all sections At least 70% of the students will score ‘acceptable’ or higher based on the critical thinking rubric At least 70% of the students will score ‘acceptable’ or higher based on the critical thinking rubric Average score across all sections assessed: 88.5% Average score for BUS 3710 embedded questions: 76.28% Average score across all sections assessed (BUS 3950): 77.6% Average score across all sections assessed: 87.9% Assignment developed or selected by BUS 3950 faculty; results reported to Assurance of Learning Coordinator who reports to Business Curriculum Committee. Faculty teaching BUS 3710 and BUS 3950 develop questions and administer each semester; results reported to Assurance of Learning Coordinator who reports to Business Curriculum Committee. Faculty teaching BUS 3950 develop questions and administer each semester; results reported to Assurance of Learning Coordinator who reports to Business Curriculum Committee. PART TWO Describe your program’s assessment accomplishments since your last report was submitted. Discuss ways in which you have responded to the CASA Director’s comments on last year’s report or simply describe what assessment work was initiated, continued, or completed. The Assurance of Learning Coordinator summarized the results of the data collected during the previous academic year and presented the results to the faculty council and then to all faculty members at the Fall faculty meeting. The Assurance of Learning Coordinator attended an AACSB Seminar on critical thinking in the spring. The information presented at this seminar inspired the topic of the spring Brown Bag session: Using Peer Review to Improve Student Performance The School of Business continued to demonstrate buy-in several ways. First, there were two Brown Bag sessions held last year which were well-attended by the faculty. The topic of the first Brown Bag session was how to improve our students' oral presentation skills, since this was the area where our students struggled most during AY13. The second session discussed using peer review to improve student learning. Faculty members also demonstrated buy-in by sharing data from their courses to the Assurance of Learning Coordinator, as well as the School of Business. In addition, the faculty demonstrated buy-in by volunteering to assist by assessing the video samples which were collected over the semester. The undergraduate ethics rubric was reevaluated due to the absence of a category regarding multiple ethical perspectives. After speaking with ethics instructors, it was determined that using multiple ethical perspectives is a skill that students beginning the business program may not have. Therefore, it was decided the undergraduate rubric would be more appropriate for students beginning the business program, and the graduate ethics rubric, which has a category for multiple perspectives, would be more appropriate for upperlevel undergraduate students. A critical thinking rubric was developed by the Curriculum Committee and was piloted in the fall. Last year, the Curriculum Committee identified courses where data for the new critical thinking learning goal will be collected, as well as acceptable performance levels for each assignment. After the successful pilot of the critical thinking rubric in the fall, the Curriculum Committee began to collect data in the spring. PART THREE Summarize changes and improvements in curriculum, instruction, and learning that have resulted from the implementation of your assessment program. How have you used the data? What have you learned? In light of what you have learned through your assessment efforts this year and in past years, what are your plans for the future? The following summarize the changes and improvements in curriculum, instruction, and learning for each learning goal: o Learning Goal 1: During AY13, the Curriculum Committee decided to raise the minimum acceptable performance level from the national average, to ¼ of a standard deviation above the national average. On the Fall 2013 exam, our students met the higher standard. However, on the Spring 2014 exam, our students were below the national average in two categories. This underperformance made the Curriculum Committee realize the difficulty in ‘closing the loop’ when the members are unable to see the contents of the ETS exam. As a result, the members of the Curriculum Committee decided to coordinate with the instructors of each course in the business core to create our own “in-house” capstone exam. The instructors of the core courses will develop 5 questions per course, and the questions will be compiled to create our custom capstone exam. Using this custom exam will allow the faculty to pinpoint the exact areas where business students struggle. The Assurance of Learning Coordinator will begin to work with the instructors during the summer, and the questions will be developed by October 1, so they can be piloted in the fall. o Learning Goal 2: After reviewing the undergraduate ethics rubric during the academic year and discussing it with the BUS 2750 and BUS 4360, it appeared that faculty believed it was acceptable to not have a category for ‘identification of multiple ethical perspectives’ on the undergraduate rubric since lower-level students may not have studied ethical theories. Therefore, the undergraduate rubric will not have the category for ‘identification of multiple ethical perspectives’ Instead, the graduate rubric will be used for upper-level undergraduate students, since this rubric contains a category for multiple ethical perspectives. During AY13, the Curriculum Committee decided to raise our minimum acceptable performance level for learning goal 2.1. During AY14, it appears that the students fell short of the new, higher goal. In particular, students struggled with multiple ethical perspectives. During the May Curriculum Committee retreat, Dr. James Sysko noted that he has had success by providing students with a list of ethical theories and assigning students to write a brief summary of each. This strategy will be shared with instructors of BUS 4360 in the fall. While our students are slightly below the minimum performance level for LG 2.2, they have performed better than in AY13, and are now slightly below the minimum performance threshold. For LG 2.3, the students have exceeded the minimum performance level overall, however, only 52.9% correctly answered embedded exam questions relating to agency law. Dr. James Sysko, who teaches BUS 2750, mentioned that this topic comes late in the course and that he personally only spends one lecture covering this topic. He said that he will begin to spend two lectures covering this topic in the fall. o Learning Goal 3: In AY13, the Curriculum Committee noted that our students have consistently exceeded our minimum acceptable performance level for all measures with this learning goal. The Committee evaluated the rubric used to measure student performance and noted that the rubric measures very basic tasks, such as knowing how to properly save a document in Access or Excel. The Committee decided to lump all of the 'basic' tasks into one category in order to isolate areas where students need improvement. The Assurance of Learning Coordinator met with Paul Brown, who helped to develop the rubric. He agreed that it would be best to combine all of the basic tasks into one category and he agreed to change the rubric beginning in the Fall semester. In addition to changing the rubric, in AY13 the Committee also decided to raise the minimum acceptable performance level from 70% to 80%. During AY14, our students are below the minimum performance level. This is likely due to increasing the minimum performance level, and due to the lumping of all basic tasks into one category. Furthermore, more difficult questions were added by the BUS 1950 faculty during AY14. This will help the BUS 1950 faculty identify the tasks where students are weak so that they can spend more time on those skills in class during AY15. During the end-of-year Curriculum Committee retreat, Dr. Gurkan Akalin, who teaches BUS 3950, noted that upper-level students’ Excel skills are weak, in spite high performance level of the students in BUS 1950. The Assurance of Learning Coordinator will work with BUS 3950 faculty, and BUS 1950 faculty to determine if the questions asked in BUS 1950 are adequately preparing students for upper-level business courses. In addition, the Assurance of Learning Coordinator will speak instructors of 2000-level courses to determine if the Excel and Access skills are being reinforced after BUS 1950. o Learning Goal 4: This learning goal continues to be the biggest area where business students need to improve. One of the Brown Bag sessions held during the year focused on finding ways to improve students’ oral presentation skills, while the other Brown Bag session discussed using peer review to improve student learning, particularly with writing and oral communication. After the Brown Bag session on using peer review, two faculty members decided to experiment with peer review in their non-core courses and reported success. The results of using peer review will be discussed at the Fall faculty meeting in August, and faculty members will be encouraged to experiment with peer review for written and oral assignments. Faculty members often complain that when students give an oral presentation, it is clear that many students do not seem to rehearse the presentation before presenting in front of the class. Likewise, faculty members have complained that many writing assignments submitted for a grade appear to be first drafts. As part of the “closing the loop” activities during AY13, Mr. Scott Stevens, who teaches BUS 1000 has started to more heavily emphasize the Writing Center to his BUS 1000 students. However, since these students are early in their business career, and since writing is not assessed until students are in their upper-level business courses, it may take another year or two to see the impact of this change. o Also, during the AY13 retreat, faculty discussed the possibility of requiring students in BUS 1000 to purchase a 'cheat sheet' which lists common grammatical mistakes. This action was not taken in AY14, but the possibility of making this change in Fall 2014 is being considered. Lastly, the Curriculum Committee discussed using data collected in the senior seminar courses. Learning Goal 5: This learning goal is still in the development phase. In the fall, the critical thinking rubric was piloted, and data was collected for the first time in the spring semester. However, the case chosen for the critical thinking rubric did not seem to be the most effective way to measure critical thinking. The Assurance of Learning Coordinator will work with the BUS 3950 instructors to help identify a more appropriate case.