Curriculum Management Report Summer 2014–Spring 2015 Table of Contents I. Overview .................................................................................................................................................................................................. 3 II. Curriculum Development and Review ................................................................................................................................................... 3 III. Assurance of Learning Process ........................................................................................................................................................... 26 IV. Assurance of Learning in the BBA (AACSB)MBA (AACSB) ............................................................................................................... 39 V. Assurance of Learning in the MAcc (AACSB/HLC) ............................................................................................................................. 50 VI. Assurance of Learning in the MBA (AACSB/HLC)......................................................................... Error! Bookmark not defined. VII. Assurance of Learning in the BBA (HLC #8) ................................................................................ Error! Bookmark not defined. VIII. Assurance of Learning in the BS/BA in Economics (HLC) .............................................................................................................. 60 IX. Assurance of Learning in the UCA Core (HLC) .................................................................................................................................. 66 Appendix. VIII. Curriculum for proposed BS in Information Systems .................................................................................................... 60 2 II. Assurance of Learning in the BBA (AACSB) I. Overview The academic year 2014-2015 included many substantial changes in Curriculum Management (CM). A number of significant curricular changes occurred, including the COB approval of a new program of study (Bachelor of Science in Information Systems), new concentrations, new courses and a formalized process for reviewing curricula, along with the first formal college-level curricular review. The addition of Curriculum Management dictated a revision to the previous AOL Handbook (of August 2014). The handbook was renamed Curriculum Management and approved in April 2015. In addition, this annual report is now labeled Curriculum Management Report (instead of Assessment Report). Both the handbook and report now include both curriculum development/review and AOL. The Assessment of Learning (AOL) program is also undergoing significant changes. The major proposed changes for AOL include using improvement teams to make recommendations to improve student learning for all learning objectives, a complete revision of the scheduling process for assessing learning objectives, and the inclusion of assessment item (rubric items and test questions) averages and not just student aggregate scores in analyzing assessment data. These changes were pilot tested this past year (Spring 2015), in which an improvement team analyzed the oral communication learning objective and made recommendations for improvement. Section IIIa below covers in more detail the new process. The changes to the AOL process are to be incorporated into a new revision of the Curriculum Management Handbook. This handbook is expected to be approved in Fall 2015. This annual report covers both components of CM, curriculum development and review and then AOL. II. Curriculum Development and Review Curriculum development and review has long existed as a function of each department as well as the Curriculum and Assessment Committees (CACs) and college. This process has not really changed; what has changed, however is including the management of curricula formally into the latest version of our handbook (April 2015). CM was not formally addressed in previous editions of the AOL Handbook (August 2014 and before). While it has always been under the purview of Curriculum and Assessment Committees, its inclusion into the latest handbook signifies its importance within the College of Business (COB). The process of developing curricula includes designing or changing courses, changing 3 II. Assurance of Learning in the BBA (AACSB) program requirements, defining new programs, and is carried out as directed by UCA guidance. In addition to curriculum development, however, the new handbook includes formal guidance on curriculum reviews. Prior to this, reviewing curricula was carried out solely by departments. The revised process is summarized below and is detailed in the latest handbook. Curricula review occurs frequently in departments and within majors. The addition of a formal process does not change this nor does it suddenly mandate that departments/majors review their curriculum. Rather this process is more about providing a forum so that other departments and CAC members may learn or be updated on the curriculum of a major not their own. It provides the opportunity for sharing ideas and best practices. Programs of study, in particular the BBA and MBA, consist of courses from every department, and seeing how these fit together to produce a graduate, in any major, is important. The review process will also include programmatic areas, including the Business Foundation and Business Core. In Spring 2015, the first major, the BBA-MIS was presented to interested faculty to some interesting discussion. The presentation covered current curriculum, curricula at other institutions, current pedagogies (online, hybrid, face-to-face), the new proposed major, BS in Information Systems, relationships between MIS and external stakeholders, and the future Major Curricula Highlights (2014-2015) Conducted the First Curricula Review BBA-MIS (Spring 2015) Major Curricula Changes Past Year 4 New proposed program: BS in Information Systems New MBA concentration: Information Management New couses: six new courses, three in the MBA and three in MIS Electronic Delivery: five courses approved for online delivery II. Assurance of Learning in the BBA (AACSB) of MIS. A copy of this presentation is available on the UCA COB Pool Drive (P:\COB\Committees\CAC\CurriculumReviews). In addition to the first college-wide curriculum review, there were a number of curricula development changes in the past academic year (2014-2015). Below are highlights: 1. New Program: Bachelor of Science in Information Systems. This was approved at the COB CAC in February 2015. It was approved by the Undergraduate Council and Council of Deans in March/April 2015. It is now at the Arkansas Department of Higher Education for final approval. This is expected in the summer of 2015; the projected start date for the program is Spring 2016. The program is an interdisciplinary in nature, with a required minor in Computer Science as well as a large selection of both business courses and MIS courses. The CS required courses include the first three in the major, CS I, CS II, and Data Structures (CSCI 1470/1480/2320), plus an additional seven hours in CS. The business courses include several courses in the Business Foundation, four courses in the Business Core, and 30 additional hours of courses in either CS or MIS. The Appendix provides the curriculum for the new major. This new major was conceived and developed almost completely as a result of indirect assessment processes. Formal meetings with a total of fifteen different employers (who hire MIS and CS graduates) were held in a yearlong period. Students majoring in CS, MIS and other business majors were also contacted and surveyed about majors. The consensus was clear—the majority of employers in Arkansas need a more technical MIS major or a more business-savvy CS major. As one IT manager (from ArcBest in Ft. Smith) put it, “Our ideal candidate is a combination MIS and CS major”. Students also overwhelmingly supported such a major. The intent is for this new major, a true STEM major, to be accredited by both AACSB and ABET. 2. New Concentration in MBA. A new concentration in Information Management was approved for the MBA program, to go along with two previously approved concentrations in Finance and Health Care Administration. Students wishing to earn the Information Management Concentration must complete three of the following elective courses: MBA 5351 Quantitative Analytical Methods, MBA 5355 Project Management, MBA 5361 Information Security, and/or MBA 5380 Business Intelligence. 5 II. Assurance of Learning in the BBA (AACSB) 3. New Courses. Six new courses were approved in the COB. Three of these belong to the MBA program, and consist of courses cross-listed as MIS senior-level courses. These include MBA 5351 Quantitative Analytical Methods, MBA 5355 Project Management, and MBA 5380 Business Intelligence. Three other courses were also approved, all in MIS. These included MIS 3335 Programming in Scripting Languages, MIS 4339 Programming in Java II, and MIS 4364 Computer and Network Security. 4. Electronic Delivery. Five courses were approved for conversion to electronic delivery (online teaching). This includes FINA 2330 Personal Finance, MKTG 4354 Consumer Behavior, MKTG 4362 Services Marketing, and MGMT 4305/MBA 5345 Governance of the Sustainable Organization. 5. MBA Admission Requirements. Admission requirements were changed to waive the GMAT/GRE for some applicants. 6. Other Changes. There were several other more minor curricula changes, including modifying prerequisites and changing course titles. Overall, the curriculum development and review process has been busy the past year, with new programs, concentrations, courses and the first formal curriculum review. III. Assurance of Learning Process This section (III) starts the AOL part of the Annual Report. The past academic year saw a number of substantial changes to the AOL process and plan, as well as many assessments. The year marked the end of the previous cycle of how and when assessment is done for each learning objective. It therefore closes out the previous schedule, while introducing a new methodology which includes a renewed emphasis on using AOL data to improve student learning. This section briefly describes both the revised process for AOL (starting in Fall 2015) as well as the current process used, which is important since most of the assessments done the past year used the current process. The following sections, IV6 II. Assurance of Learning in the BBA (AACSB) IX, consist of the assessment results from this past year. It is divided into sections, based on accrediting body (AACSB or HLC). However, first we look at the changes in store for the COB starting in Fall 2015. IIIa. AOL Changes for Fall 2015 The last five years, the COB has assessed learning objectives in each of its three programs (BBA, MAcc and MBA). The approved process called for selected objectives to be assessed each semester the appropriate course (or courses) were offered; the objective then had to pass the benchmark in three of four assessed semesters. If an objective passed 3 of 4 (or 4 of 4), then the objective was taken off cycle and a new objective was selected to replace it. Four objectives were assessed at one time (out of 11 for the BBA, 13 for the MAcc, and 7 for the MBA). While in most cases this system yielded positive results, based on our collective experience the past five years, it could clearly be improved. On the positive side, the data collected suggested that for the most part, students were learning as expected. This was especially true in the BBA. Most of the assessed objectives passed the benchmark. In the MAcc and MBA, the assessments yielded mixed results; some objectives passed but some did not meet the benchmark, and at times failed for multiple semesters. This led to several occasions in which improvements were made based on the data, in all three programs (past Annual Reports list these efforts, as does this report). Some of these improvements or interventions worked; some did not. But while the process and results were mainly positive, it became apparent that there were two major areas that needed improvement: 1. Not all objectives were assessed in the five year period. The rotation that required each objective to pass 3 of 4 semesters and only doing four at a time meant that some were not assessed in a reasonable time frame (five years for us). In all, three were not assessed in BBA, four in MAcc, and two in MBA. Waiting too long to assess an objective is not acceptable. For the MAcc (and even the MBA), the problem is compounded in that each course is only offered once per year. So in addition to difficulty getting each objective assessed, the sample size was also small. 2. Assessment did not focus enough attention on improving student learning, particularly for objectives that passed the benchmark. The committees (COB and departmental CACs) focused mostly on objectives which did not pass. While passing the benchmark is a positive, there were lost opportunities in which student learning could be improved in specific weak areas, even though overall the benchmark was met. In addition, data were not examined except by aggregate. This meant that improvements could have been made for particular skills or knowledge areas within a particular objective had the data been available. 7 II. Assurance of Learning in the BBA (AACSB) In the spirit of continuous improvement, the COB initiated and has proposed a new process, one that hopefully better accomplishes the intent of AOL. This process changes the scheduling of assessing learning objectives and incorporates a specific improvement process. It is outlined briefly below: Change the process of scheduling the assessment of learning objectives. In general, the process will consist of assessing an objective in year one (using the approved rubric/test items, etc.), formally examining the data and recommending improvements in year two, and then reassessing in year three. In year four, the new data will be analyzed; those improvements that worked will be institutionalized (closing the loop). The new data will then be used to make more improvement recommendations. Instead of only examining aggregated data (i.e., how many students scored above the benchmark), the improvement process will examine each rubric or test item to determine weak areas within each objective. Thus (over time) the benchmark must change to not only include the aggregate average, but also include a minimum average for each rubric or test item. The schedule for assessing learning objectives will be staggered, so that approximately half will be assessed in year one and the other half in the next year, with improvement years to follow. In this way each learning objective will be assessed, improved (hopefully), and reassessed in each five year period. The improvement process will be carried out by an improvement team, made up of experts within each learning objective field or area, and typically one from each department. These changes will ensure that every learning objective will be assessed (and reassessed) each five year period, and it emphasizes improving student learning. Each objective will examined by a team of knowledgeable faculty members and after every assessment, weak areas will be identified and improvements recommended. These changes will be incorporated into a new revision of the Curriculum Management Handbook this fall. IIIb. Current AOL Process While this is the last year to examine assessed learning objectives under the current process, it is described below since the assessment results that follow are based on this process. The development of learning goals and objectives for the various accrediting bodies is described in detail in the COB Curriculum Management Handbook (latest revision is April 2015). In addition, the Handbook outlines the process for reviewing and revising the assessment program, the assessment calendar, and the responsibilities of faculty and administrators in the assessment process. The Handbook, the UCA College of Business website and pool drive contains rubrics and data submission forms for all assessment objectives. 8 II. Assurance of Learning in the BBA (AACSB) The College of Business performs assurance of learning for many different programs which is reported to two accrediting bodies. While the process is about the same, the results (Sections IV-IX) are divided into different sections based on the accrediting body and the program assessed. This makes it easier to cull the appropriate information when needed. It should be pointed out that the goal of the COB is to ultimately have one AOL plan, which incorporates all learning objectives. These assurance of learning programs are summarized below and then described in more detail. Accrediting Body AACSB Programs Assessed Bachelor of Business Administration (single degree) Master of Accountancy Master of Business Administration HLC (Specific Degree Programs) Bachelor of Business Administration with distinct majors Master of Accountancy Master of Business Administration Bachelor of Science and Bachelor of Arts in Economics HLC General Education (UCA) Core Courses 1. Association to Advance Collegiate Schools of Business (AACSB) The current learning goals and objectives were developed during the spring and summer of 2011. While assurance of learning has a long and well-developed history at UCA, the desire for improvement suggested that our goals and objectives should be reconsidered at that time. In response to this, a concerted effort by faculty, students, administrators, advisory board members and other stakeholders developed learning goals and objectives for the three distinct degree programs. These learning goals and objectives flow directly from the COB’s Vision, Mission, and Values statement that was formally adopted on July 15, 2011, and amended since. The three distinct programs assessed in the College of Business are the Bachelor of Business Administration (BBA), the Master of Accountancy (MAcc), and the Master of Business Administration (MBA). 2. Higher Learning Commission (HLC-Specific Degrees Programs) The current HLC learning goals and objectives for the College of Business Programs include the same goals and objectives as reported to AACSB with the following differences: 9 II. Assurance of Learning in the BBA (AACSB) An additional learning goal is added to the BBA program for discipline specific assurance of learning. Each of the eight majors has one discipline-specific objective. The additional goals and objectives were formalized in Spring 2013 and are labeled HLC #8 (it is the BBA’s 8th goal). The BS/BA in Economics has its own set of learning objectives as it is not assessed under the AACSB program. These goals and objectives were formalized in Spring 2013. The Master of Accountancy assessment for AACSB and HLC are identical. The Master of Business Administration assessment for AACSB and HLC are identical. 3. Higher Learning Commission (HLC-General Education Core Courses) The UCA Core, the general education program at UCA, is outlined at www.uca.edu/core. It is a two tiered program with 11 objectives at both the lower-division and upper-division level. The assessment of general education is still in development, though formal assessment is scheduled to start in Fall 2015. What is being assessed, however, is still under some debate. This annual report does not include assessment results for this general education AOL, given that it has not yet been started. IV. Assurance of Learning in the BBA--AACSB IVa. AOL for the BBA Program For the BBA program, there are seven goals and eleven learning objectives. The goals/objectives are summarized in the table below, along with the measure, course(s) used, and benchmark. 10 II. Assurance of Learning in the BBA (AACSB) Bachelor of Business Administration (BBA) Assurance of Learning Plan Objective: Students will be able to Measure: Students will Course Goal 1. Our graduates shall possess critical thinking and analytical thinking skills 2. Our graduates shall possess awareness of the global business environment 1a. Use appropriate analytical techniques to identify and frame problems, generate and compare alternatives, and use knowledge and reasoning skills to optimize organizational performance. Demonstrate learning and mastery of subject matter through their performance on the Capstone © Business Simulation. MGMT 4347 1b. Accurately apply the appropriate quantitative skills to solve specific problems in various business disciplines. Demonstrate sufficient quantitative skills through their performance on targeted pre- and post-testing in designated courses QMTH 2330 ACCT 4315, FINA 3330, MGMT 3344, Take a series of Blackboard-hosted quizzes related to global business. ECON 2310 Take a pre-test and a post-test of questions regarding global perspectives. MKTG 3350 ACCT 3315 MGMT 3344 2a. Identify cultural/global perspectives among stakeholders. 3a. Be aware of ethical issues inherent in business decisions and articulate the manner in which they arrived at an ethical decision. 3. Our graduates shall possess ethical reasoning abilities 3b. Be aware of legal issues inherent in business decisions. 4. Our graduates shall possess effective communication abilities 5. Our graduates shall possess effective collaborative skills 6. Our graduates shall possess effective information management skills 7. Our graduates shall possess an understanding of a broad range of business disciplines 11 Complete an assignment (business case or other writing assignment) dealing with ethical decision making or the social responsibility of business. Demonstrate knowledge of government regulation, employment law, property law, and contract law by their performance on objective questions embedded in an exam. 4a. Produce professional quality written documents. Prepare a report, paper, or case analysis on a disciplinespecific topic in the writing intensive course designated in their major. 4b. Deliver professional quality oral presentations. Give an oral presentation on a business topic. MGMT 3340 ACCT 2321 MGMT 4348, ECON 4380, INSU 4320, ACCT 4317, MGMT 4376, MIS 3328, MKTG 4355, FINA 4336 MKTG 2376, MGMT 2301 Benchmark The College’s mean score will be in the 50th percentile or above. Mean score of students assessed will improve; post-test mean score will be > 70%. The mean number of points for all students assessed will be ≥ 70% of the total possible points. There will be a statistically significant improvement in the mean score between the pre-test and the post-test. Mean rubric score will be four (4) points, or greater, out the available six (6) points. Mean score of students assessed will be ≥ 70%. Mean rubric score of students assessed will be nine (9) points or greater, of the fourteen (14) available points. Mean rubric score will be eleven (11) out of sixteen (16) points. 6a. Effectively apply business-oriented software applications to manage data in support of business operations. Complete group projects as part of the Capstone© Business Simulation. Using Capstone's internal teamwork assessment process, each student will be assessed by peers several times during the semester. Be assessed using a test question set that requires a demonstration of Microsoft Excel and Access application skills. 6b. Understand the role of information systems in support of organizational activities. Be assessed by their performance on objective questions embedded in an exam. ACCT 3320 MIS 3321 Mean score of all students assessed will be ≥ 70% of the points possible. 7a. Demonstrate comprehension of key concepts and theories in various functional areas of business, and demonstrate the ability to draw on knowledge and insights from a variety of disciplines when analyzing and formulating solutions to problems and opportunities. Demonstrate learning and mastery of subject matter through their performance on the Capstone © Comp-XM examination. MGMT 4347 Mean score of all students completing the exam will be ≥ 50%. 5a. Work in teams to solve business problems. II. Assurance of Learning in the BBA (AACSB) MGMT 4347 The cumulative mean score will be ≥ 70% on Capstone’s teamwork assessment. MIS 2343 Mean score of all students assessed will be ≥ 70% of the points possible. For the past four years, objectives were assessed four at a time. To be cycled off the schedule, an objective must pass three of four semesters. The diagram below displays the rotation of these goals since the start of this version of AOL in Fall 2011. Objective 1a Fall 2011 Spring 2012 Fall 2012 Spring 2013 Fall 2013 Spring 2014 Fall 2014 Spring 2015 Spring 2016 Assessed; passed 3 of 4 semesters 1b Assessed; passed 4 of 4 semesters 2a Assessed; passed 4 of 4 semesters 3a Assessed; passed 4 of 4 semesters 3b 4a Fall 2015 Assess Assessed; passed 4 of 4 semesters 4b Assessed; passed 4 of 4 semesters 5a Assessed; passed 4 of 4 semesters 6a Assess 6b 7a Assess Assessed; passed 4 of 4 semesters The current process calls for three new objectives to rotate in the schedule in academic year 2015 (3b, 6a, and 6b). Two will be assessed in the fall, the third in the spring. IVb. AOL Results for the BBA Program The below lists results for each learning objective, summarized over the previous four years. Note that some objectives were not in rotation this year (only four per year are assessed). Those four objectives that were assessed in this past year (the year of this report) are covered first. These include 1b, 4b, 5a and 7a. Those that were not on rotation are summarized in order after the first four. 12 II. Assurance of Learning in the BBA (AACSB) BBA Objective 1b Goal: Our graduates shall possess critical thinking and analytical thinking skills Objective: Students will be able to accurately apply the appropriate quantitative skills to solve specific problems in various business disciplines. Course(s): QMTH 2330; FINA 3330; MGMT 3344 OR ACCT 4315 Measure: Students will demonstrate sufficient quantitative skills through their performance on targeted pre- and posttesting in designated courses Benchmark: Mean score of students assessed will improve; post-test mean score of students assessed will be 70% or above Program Placement: Data Collection: Quantitative skills are assessed in three classes throughout the BBA curriculum. The first assessment in QMTH 2330 is in a business statistics course required of all majors. The second assessment will be in FINA 3330, of which QMTH 2330 is a prerequisite, and also required by all business majors. The third assessment will take place in MGMT 3344 or, for accounting majors, ACCT 4315 (both similar courses). Students in each section of the assessment courses were given pre-tests and post-tests on quantitative skills specific to the particular course. This data is collected for at least four semesters. Data Summary & Closing the Loop: 100.0 Last year, this objective was measured for the first time in the new process; both 78.3 76.65 semesters saw significant improvement from pre-test to post-test; however, the 72.17 75.0 post-test in Fall (2013) failed to meet the benchmark of 70%. This was a 57.2 continuation of two failures under the old process, in 2010-2011 (old Learning 50.0 Goal 4, Objective 1). Based on the two failures (2010-2011), a committee was 36.8 35.3 30.7 formed to examine the issue; it consisted of one faculty member from Finance, 28.3 Economics and MIS. The group met in 2012 and examined the assessment results 25.0 line for line. This primary issue was determined to be a lack of preparation in algebra. This led the group to formally examine the pre-requisite course to 0.0 Business Statistics, MATH 1395 Business Calculus. After a series of meetings and Fall 2013 Spring 2014 Fall 2014 Spring 2015 discussions with the Math Department’s lead faculty for this course, in Spring 2013 the math course was changed substantially to add more algebra and less Pre Post calculus. The course still included the derivative (i.e. some calculus) and coverage in several important functions, but added significant content in algebra. The changes took place in Fall 2013, and coincided with assessing the learning objective for the first time under the new assessment plan. During that semester, the quantitative skills objective passed part of the benchmark (significant improvement between the pre- and post-tests), but did not pass another part which stipulated the post-test score would be greater than 70% (the score was actually 57.2). Further discussions with the 13 II. Assurance of Learning in the BBA (AACSB) Math Department noted that some instructors had not yet made the change, and was to be corrected. The second assessment in Spring 2014, passed, suggesting the intervention is starting to work. This year, there was continued improvement, with significant growth and the post-test scoring above 70%. Our students met both benchmarks in three of four semesters. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. This objective and related measure and benchmark will be examined by a task force prior to reevaluation in future semesters. 14 II. Assurance of Learning in the BBA (AACSB) BBA Objective 4b Goal: Our graduates shall possess effective communication abilities. Objective: Students will be able to deliver professional quality oral presentations Course(s): MGMT 2301 or MKTG 2376 Measure: Students will give an oral presentation on a business topic. A grading rubric will be used to assess levels of skills. Benchmark: Mean rubric score of students assessed with the rubric will be 11 points, or greater, out of the 16 available points. Program Placement: Data Collection: MGMT 2301 or, for Innovation and Entrepreneurship majors, MKTG 2376, is a business communications course dedicated to the study and practice of a variety of types of business communications including an oral presentation. While students may take this course at any time, it is usually taken in the sophomore or junior year. Faculty in each assessment course required an oral presentation by the students. The student presentations are evaluated using the oral communications rubric for the university. Faculty report the total points earned by each student, and the results are aggregated across all sections and all majors for a given semester. Data Summary/Closing the Loop: Like last year, our students again met the benchmark. This made four passes in four semesters. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. 16.00 12.28 12.99 11.45 12.00 12.44 8.00 This learning objective, however, was selected to be a sort of pilot as the COB changes the process of AOL. Covered much more extensively in the Curriculum Management Handbook, essentially each objective will be assessed, examined/analyzed for ways to improve (including examining each rubric or test item), and then re-assessed to determine if the improvements worked. The examination or analysis of data will be accomplished by an improvement task force consisting of knowledgeable faculty members for that particular objective. 4.00 0.00 Fall 2013 Spring 2014 Fall 2014 Spring 2015 A task force was formed to consider Fall 2014 assessment results. It consisted of representatives from each department, plus one from the COB CAC. Several recommendations were made by the committee. System improvements recommended: change rubric to include additional items on the actual presentation, including visual aids; change benchmark to state: Students shall attain a score of at least 85% on each individual item in the rubric; this equates to a minimum score of 4.25 for each item (anchors are from 0 – 5). Learning improvements recommended: provide example(s) of good presentations to students prior to theirs; could be done by videotaping a presentation by some outside group (such as the Target or Windstream competitions), or having a group do a mock presentation. These recommendations will be sent out to each department for comment in Fall 2015, followed by routing through COB CAC and the Executive Committee. 15 II. Assurance of Learning in the BBA (AACSB) BBA Objective 5a Goal: Our graduates shall possess effective collaboration abilities. Objective: Students will be able to work in teams to solve business problems Course(s): MGMT 4347 Measure: Benchmark: Program Placement: Data Collection: Students will complete group projects as part of the Capstone© Business Simulation. Using Capstone’s internal teamwork assessment process, each student will be assessed by peers several times during a semester. The cumulative mean score of all students will be 70% or above. As the BBA Capstone course, students typically take MGMT 4347 in their last semester at UCA. Because all five Business Core courses must be taken before this one, it is a rare student who does not take the course in their last year of studies (e.g., double majors). This course is therefore summative in nature, and provides a picture of what students have learned as a result of our curriculum. Students in each section of the assessment course participated in the Capstone © Business Simulation requiring among other things, repeated evaluation of team members. Capstone © reports generated at the end of each semester indicates a teamwork score for each student. Data is collected for four semesters, and the results are aggregated across all sections and all majors for a given semester. Data Summary & Closing the Loop: Our students met the benchmark in four consecutive semesters. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. This objective and related measure and benchmark will be examined by a task force prior to reevaluation in future semesters. 91.80 100.00 82.30 86.99 86.35 Fall 2014 Spring 2015 75.00 50.00 25.00 0.00 Fall 2013 16 II. Assurance of Learning in the BBA (AACSB) Spring 2014 BBA Objective 7a Goal: Our graduates shall possess an understanding of a broad range of business disciplines. Objective: Students will be able to demonstrate understanding of key concepts and theories in various functional areas of business and the ability to draw on knowledge and insights from a variety of disciplines when analyzing and formulating solutions to problems and opportunities. Course(s): MGMT 4347 Measure: Benchmark: Students will be able to demonstrate understanding of key concepts and theories in various functional areas of business and the ability to draw on knowledge and insights from a variety of disciplines when analyzing and formulating solutions to problems and opportunities Mean score of all students completing the exam shall be equal to 50% in each of the subject areas. Program Placement: As the BBA Capstone course, students typically take MGMT 4347 in their last semester at UCA. Because all five Business Core courses must be taken before this one, it is a rare student who does not take the course in their last year of studies (e.g., double majors). This course is therefore summative in nature, and provides a picture of what students have learned as a result of our curriculum. Data Collection: Students in each section of the assessment course participated in the Capstone © Business Simulation requiring among other things, answering several rounds of multiple-choice questions about the different business disciplines. Capstone © reports generated at the end of each semester indicates an examination score for each student. Data is collected for four semesters, and the results are aggregated across all sections and all majors for a given semester. Data Summary& Closing the Loop: Our students met the benchmark in four semesters. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. This objective and related measure and benchmark will be examined by a task force prior to reevaluation in future semesters. 100.00 75.00 54.40 54.50 53.75 55.71 Fall 2013 Spring 2014 Fall 2014 Spring 2015 50.00 25.00 0.00 17 II. Assurance of Learning in the BBA (AACSB) The below tables summarizes assessment results from previous semesters, using the last semester information for each objective. BBA Objective 1A Goal: Objective: Students will be able to use appropriate analytical techniques to identify and frame problems, generate and compare alternatives, and use knowledge and reasoning skills to optimize organizational performance. Course(s): MGMT 4347 Measure: Benchmark: Program Placement: Data Collection: 18 Our graduates shall possess critical thinking and analytical thinking skills Students will demonstrate learning and mastery of subject matter through their performance on the Capstone © Business Simulation. The College’s mean score will be in the 50th percentile or above. As the BBA Capstone course, students typically take MGMT 4347 in their last semester at UCA. Because all five Business Core courses must be taken before this one, it is a rare student who does not take the course in their last year of studies (e.g., double majors). This course is therefore summative in nature, and provides a picture of what students have learned as a result of our curriculum. Students in each section of the assessment course participated in the Capstone © Business Simulation requiring among other things, answering questions requiring critical and/or analytical thinking. Capstone © reports generated at the end of each semester indicates the percentile of UCA students compared to all other students (undergraduate and graduate, internationally). This data was collected for four semesters and is summarized in the table below. II. Assurance of Learning in the BBA (AACSB) Data Summary & Closing the Loop: 100.0 From Annual Report 2012-2013: After our first semester of collecting assessment data, we failed to meet our 69.3 75.0 62.9 benchmark. Four changes were undertaken as a result this failure: 1. Examine 52.7 CapSim Simulation, from which the data are collected; 2. Admission requirements to 47.0 50.0 enter COB were changed so that a 2.0 is required for the average of five critical quantitative type courses, including ACCT 2301/2302 (Accounting I and II), ECON 2320/2321 (Macro- and Micro-economics), and QMTH 2330 (Statistics); 3. Better 25.0 prepare those who teach online (MGMT 4347), to include better monitoring of student progression and an improved syllabus; 4. Start a dialogue with Math Department concerning MATH 1395 (Business Calculus). To better understand 0.0 CapSim (#1), this year faculty who teach MGMT 4347, members of the Curriculum Fall 2011 Spring 2012 Fall 2012 Spring 2013 and Assessment Committee, and College of Business administrators took a webinar provided by CapSim to better understand the process of questioning, scoring, and reporting data in the Capstone simulation. The webinar took place in Spring 2012, and the new knowledge was disseminated to all members of the faculty at the opening year meeting for Fall 2012. The newly-learned information allowed faculty teaching the course to modify course content so that students are better prepared for the types of questions they will be asked. The Curriculum and Assessment Committee also brainstormed ideas if scores did not improve. Only #3 was begun in Spring 2012, but that semester, this objective passed (barely), with 52.7%. With the new admittance standards in force, and a better understanding of the CapSim Simulation, continued improvement was noted, this year with percentages of 62.9% and 69.3%. Our students met the benchmark in three of four semesters. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. It will be evaluated again after the new schedule is in place with the new process. 19 II. Assurance of Learning in the BBA (AACSB) BBA Objective 2A (Measurement 1) Goal: Our students shall possess awareness of the global business environment. Objective: Students will be able to identify cultural/global perspectives among stakeholders Course(s): ECON 2310 Measures: Students will take a series of Blackboard-hosted quizzes related to global business. Benchmarks: Program Placement: Data Collection: The mean number of points will be equal to or greater than 70% of the total points that could possibly be earned on the Blackboard quizzes. ECON 2310 is specifically dedicated to the study of the global environment of business. Most college of business majors take this course in their sophomore or junior year; however, it may be taken at any time. For each section of ECON 2310, faculty gave a series of topical quizzes. The results from each section were aggregated for the semester to determine the average score for students taking the course in that semester. (Scores of zero for those students who failed to take a quiz, were not included in the average.) Data Summary & Closing the Loop: From Annual Report 2012-2013: Both semesters, for both measurements, students are meeting the benchmark. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. It will be evaluated again after the new schedule is in place with the new process. 100.0 75.0 78.8 77.2 75.0 Spring 2012 Fall 2012 Spring 2013 70.7 50.0 25.0 0.0 Fall 2011 20 II. Assurance of Learning in the BBA (AACSB) BBA Objective 2A (Measurement 2) Goal: Our students shall possess awareness of the global business environment. Objective: Students will be able to identify cultural/global perspectives among stakeholders Course(s): MKTG 3350 and MGMT 3344 or ACCT 3315 Measures: Students will take a pre-test and a post-test of questions regarding global perspectives. Benchmarks: There will be a statistically significant improvement in the mean score of all students assessed between the pretest and the post-test. Program Placement: The courses selected for this assessment are all related to production and sales of a product, and each course includes a topical discussion of conducting business in multiple countries or cultures. Students in the BBA program will take these courses in their junior or senior year, and will be assessed twice: once in MKTG 3350 (a marketing course) and once in MGMT 3344 or, for accounting majors, ACCT 3315 (both production courses). Data Collection: For each section of designated courses, faculty gave a pretest and posttest over global environment topics as they relate to the specific course. Due to academic freedom and the instructor’s designing their own courses, assessment questions may not be the same in all sections of the same course, and are certainly not the same across courses. The results from each section were aggregated for the semester to determine the average score for students taking the course in that semester; Results were not aggregated across courses. (Scores of zero for those students who failed to take a quiz, were not included in the average.) Data Summary & Closing the Loop: From Annual Report 2012-2013: 100.0 75.0 Both semesters, for both measurements, students are meeting the benchmark. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. It will be evaluated again after the new schedule is in place with the new process. 84.7 61.0 82.7 62.8 II. Assurance of Learning in the BBA (AACSB) 84.9 62.1 51.6 50.0 25.0 0.0 Fall 2011 Spring 2012 Pre 21 80.1 Fall 2012 Post Spring 2013 BBA Objective 3A Goal: Our graduates shall possess ethical reasoning abilities. Objective: Our students will be aware of ethical issues inherent in business decisions and articulate the manner in which they arrived at an ethical decision Course(s): MGMT 3340 Measure: Benchmark: Program Placement: Data Collection: Students will complete an assignment (business case or other writing assignment) dealing with ethical decision making or the social responsibility of business. Mean score of students assessed with the rubric will be four (4) points, or greater, out of the available six (6) points. Students in the BBA program take MGMT 3340 in their junior and senior year after completion of the required courses needed for entry into the College of Business. Students in each section of the assessment course completed a written case in which they had to resolve an ethical decision or evaluate the social responsibility of business. Course instructors evaluated each student’s response using the rubric described above and submitted mean score and number of students assessed. Data for all sections were combined for a mean score per semester. This data was collected for four semesters and is summarized in the table below. Data Summary & Closing the Loop: From Annual Report 2012-2013: 6.00 5.00 Like last year, both semesters students scored above the benchmark. Our students are meeting the ethical reasoning goals that we have set for them. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective It will be evaluated again after the new schedule is in place with the new process. 22 II. Assurance of Learning in the BBA (AACSB) 4.96 4.76 4.89 Spring 2012 Fall 2012 Spring 2013 4.16 4.00 3.00 2.00 1.00 0.00 Fall 2011 BBA Objective 3B Goal: Our graduates shall possess ethical reasoning abilities. Objective: Students will be aware of legal issues inherent in business decisions. Course(s): ACCT 2321 Measure: Benchmark: Program Placement: Data Collection: Students will demonstrate knowledge of government regulation, employment law, property law, and contract law. Students will be assessed by their performance on objective questions embedded in an exam. Mean score of students will be 70% or above. ACCT 2321, is specifically dedicated to the study of the legal environment of business. Most college of business majors take this course in their sophomore or junior year; however, it may be taken at any time. This objective is scheduled to be assessed for the first time in Fall 2015. Data Summary & Closing the Loop: This objective is scheduled to be evaluated for the first time in Fall 2015. 23 II. Assurance of Learning in the BBA (AACSB) BBA Objective 4A Goal: Our graduates shall possess effective communication abilities. Objective: Students will be able to produce professional quality business documents. Course(s): ACCT 4317, ECON 4380, FINA 4336, INSU 4320, MGMT 4348, MGMT 4376, MIS 3328, or MKTG 4355 Measure: Benchmark: Students will prepare a report, paper, or case analysis on a discipline-specific topic in the writing-intensive course designated in their majors. A grading rubric will be used to assess writing in each of the writing-intensive courses. Mean score of students assessed with the rubric will be nine (9) points, or greater, out of fourteen (14) available points. Program Placement: In the BBA, students take a business core and eight courses specific to their major; the eight courses includes some required and some s/electives. In each major, a course has been designated as writing-intensive based on its placement in the curriculum and ability to effectively assess a student’s ability to communicate in writing about their chosen field. Most students take these courses in their senior year, but may take it in their junior year. The writing-intensive course is offered in either the fall or spring for some majors, but depending on demand and number of majors, it may be offered in both fall and spring. Data Collection: Faculty in each writing-intensive course develops a discipline-specific writing assignment, which may include a research paper or a business report of some kind. The student submissions are evaluated using the writing rubric for the college. Faculty report the total points earned by each student, and the results are aggregated across all sections and all majors for a given semester. Data Summary & Closing the Loop: From Annual Report 2012-2013: Like last year, both semesters students scored above the benchmark. Our students are meeting the written communications goals that we have set for them. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. It will be evaluated again after the new schedule is in place with the new process. 24 II. Assurance of Learning in the BBA (AACSB) 14.00 12.00 11.51 11.49 11.54 11.12 Fall 2011 Spring 2012 Fall 2012 Spring 2013 10.00 8.00 6.00 4.00 2.00 0.00 BBA Objective 6A Goal: Our graduates shall possess effective management skills. Objective: Students will be able to effectively apply business-oriented software applications to manage data in support of business operations Course(s): MIS 2343 Measure: Students will be assessed using a test question set that requires a demonstration of Microsoft Excel & Access application skills. The questions are embedded in an end of course module provided by Cengage. Benchmark: The mean score of all students assessed will be equal to or greater than 70% of the points possible on the exam questions. Program Placement: MIS 2343 is dedicated to the use of technology to accomplish business tasks. While it can be taken at any time during the curriculum, most students take this course in their sophomore or junior year. Data Collection: To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed for the first time in Fall 2015. BBA Objective 6B Goal: Our graduates shall possess effective management skills. Objective: Students will be able to understand the role of information systems in support of organizational activities. Course(s): Acct 3320, MIS 3321 Measure: Benchmark: Program Placement: Data Collection: Students will students will be assessed by their performance on objective questions embedded in an exam. Mean score of all students assessed will be ≥ 70% of the points possible. ACCT 3320 and MIS 3321 are the courses designed to introduce students to information systems. Most students take this class in their junior year, some senior. It must be taken after MIS 2343. To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed for the first time in Spring 2016. 25 II. Assurance of Learning in the BBA (AACSB) V. Assurance of Learning in the MAcc (AACSB/HLC) Master of Accountancy (MAcc) Assessment Plan Goal Objective: Students will be able to 1a. Correctly apply financial accounting standards 1. Our graduates shall be competent in the theory and practice of accounting. 1b. Correctly apply auditing standards 1c. Correctly apply the tax law 1d. Solve unstructured problems 2a. Communicate effectively in writing 2. Our graduates shall be able to communicate effectively. 3. Our graduates shall be able to effectively manage information. 4. Our graduates shall be able to demonstrate effective leadership. 5. Our graduates shall be aware of the global perspective and the effects of the global marketplace. 6. Our graduates shall be able to recognize ethical dilemmas and respond ethically. 7. Our graduates shall be able to think dynamically. 26 Measure: Students will complete test questions requiring the application of financial accounting concepts complete an exam question requiring the application of auditing standards prepare a solution to a tax research project write an individual solution to a business case problem write a research paper on a financial accounting issue Course ACCT 6310 ACCT 6317 ACCT 6309 ACCT 6340 ACCT 6310 2b. Communicate effectively in an oral presentation make a presentation on an auditing standard ACCT 6317 2c. Work effectively in teams Student groups will solve a business problem ACCT 6340 prepare a solution to a tax research project ACCT 6309 3a. Locate the appropriate quality and quantity of information needed to solve a problem 3b. Understand the development and structure of a database. 4a. Demonstrate the qualities necessary for effective leadership 5a. Interpret and apply international financial accounting standards generate a database that generates an income statement based on sales and acquisitions demonstrate whether they possess leadership qualities in classroom interactions complete test questions requiring the application of international financial accounting concepts write an individual solution to an international business problem case ACCT 6320 ACCT 6350 Benchmark 80% of students will score > 15 out of 20 on rubric). 80% of students will score > 4 out of 6 on rubric. 80% of students will score > 6 out of 8 on rubric. 80% of students will score > 38 out of 55 on rubric. 80% of students will score > 11 out of 14 on rubric; 80% will receive no zeros. 80% of students will score > 38 out of 55 on rubric. 80% of students will score > 14 out of 20 on rubric. 80% of students will score > 6 out of 8 on rubric; 80% will receive no zeros. 80% of students will score > 42 out of 60 on rubric. 80% of students will score > 98 out of 140 on rubric. ACCT 6310 80% of students will score > 4 out of 8 on rubric. ACCT 6340 80% of students will score > 38 out of 55 on rubric. 6a. Identify ethical dilemmas and choose an appropriate course of action complete an exam question related to an ethical issue ACCT 6317 80% of students will score > 4 out of 6 on rubric. 7a. Speak on unprepared topics give presentations on topics for which they have not prepared in advance. ACCT 6350 80% of students will score > 17 out of 25 on rubric. III. Assurance of Learning in the MAcc (AACSB) The COB presumes that participants in a master’s level degree program possess the base of general knowledge and skills appropriate to an undergraduate degree. Learning in master’s level program develops in a more integrative, interdisciplinary fashion than in undergraduate education. Students must demonstrate knowledge of theories, models, and tools relevant to the field and shall be able to apply appropriate specialized theories, models, and tools to solve concrete business and managerial problems. B. Rotation of Goals During any given semester, we are assessing four of our learning objectives. A Learning Objective shall be assessed for four semester (four years, as courses in the MAcc program are offered once per year). If a Learning Objective is satisfied (Assessment threshold is met or exceeded) for three out of the four assessment periods, then that Learning Objective will drop out of the active assessment cycle and be replaced by the next objective in the queue. The diagram below indicates the rotation of these goals since the start of this assurance of learning program in Fall 2011. Objective Fall 2011 1A Fail Spr/Su 2012 Fall 2012 Spr/Su 2013 Fail Fall 2013 Fail Spr/Su 2014 Fall 2014 Spr/Su 2015 Pass Fall 2015 Assess 1B 1C Assess Fail Pass Pass Pass 1D 2A Assess Fail Fail Fail Pass 2B Pass 2C 3A Pass Pass Pass Pass Fail Pass 3B Assess 4A Assess 5A Assess 6A 7A 27 Spr/Su 2016 III. Assurance of Learning in the MAcc (AACSB) Pass Assess Assess C. Assessment Results The following pages contain details of each objective, benchmark, assessment method along with assessment data and actions taken to close the loop. The objectives assessed in the past year (1a, 1c, 2a, 2b, 3a, and 6a) are covered first. MAcc Objective 1a Goal: Our graduate shall be competent in the theory and practice of accounting. Objective: Students will be able to correctly apply financial accounting standards. Course(s): ACCT 6310 Measure: Benchmark: Program Placement: Data Collection: Students will complete test questions requiring the application of financial accounting concepts 80% of students will score > 3 out of 4 on rubric (15 out of 20 on new rubric). The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Students in ACCT 6310 (offered in Fall only) complete an embedded test question which is graded for assessment purposes using the Financial Accounting Rubric. Data Summary/Closing the Loop: After three failures, this year this objective passed. See the previous three Annual Reports for more detailed information, but faculty believe that the failures were due more because of poor writing than a lack of financial accounting standards knowledge. Students were not expressing what they knew in an appropriate manner. Here are the improvements made after each assessment: 2011: Stress correct/appropriate writing techniques to students prior to the exam. 2012: Changed the admission standards for the MAcc, both overall GMAT/GRE scores (450+), as well verbal/quantitative individual minimum scores. It was expected that this improvement may take time to realize. 2013: Changed the rubric used. The rubric changed from two items worth a maximum of four points to four items worth 20 points. This would provide additional granularity to assess students. 100.00 11.1 21.10 80.00 50.00 45.50 60.00 40.00 88.90 78.90 20.00 50.00 54.50 Fall 2012 Fall 2013 0.00 Fall 2011 % Over Fall 2014 % Under The combination of interventions apparently made a difference, in that this year the assessment passed. It will be evaluated again after the new schedule is in place with the new process. 28 III. Assurance of Learning in the MAcc (AACSB) MAcc Objective 1c Goal: Our graduate shall be competent in the theory and practice of accounting. Objective: Students will be able to correctly apply the tax law. Course(s): ACCT6309 Measure: Benchmark: Program Placement: Data Collection: Students will prepare a solution to a tax research project. 80% of students will score > 6 out of 8 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Students in ACCT 6309 (offered in Fall only) complete a tax research project, which is graded for assessment purposes using the Tax Rubric. On this rubric 4 traits assess the ability to correctly apply the tax law. Data Summary & Closing the Loop: This objective passed for the third time in four periods. As discussed in previous Annual Reports, how the rubric was applied was changed, by allowing partial points for its items. Students met the benchmark in the following three semesters. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. This objective and related measure and benchmark will be examined by a task force prior to reevaluation in future semesters. 100.00 80.00 23.80 10.00 60.00 40.00 76.20 90.00 Fall 2013 Fall 2014 0.00 Fall 2012 % Over III. Assurance of Learning in the MAcc (AACSB) 100.00 20.00 Fall 2011 29 100.00 % Under MAcc Objective 2a Goal: Our graduates shall be able to communicate effectively. Objective: Students will be able to communicate effectively in writing. Course(s): ACCT 6310 Measure: Benchmark: Program Placement: Data Collection: Students will write a research paper on a financial accounting issues. 80% of students will score > 11 out of 14 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Students in ACCT 6310 (offered in Fall only) complete an embedded test question which is graded for assessment purposes using the MAcc Writing Rubric. Data Summary/Closing the Loop: After consecutive failures, this objective passed for the first time. Here is what has been done after previous failures (see 2013-2014 Annual Report): 2011: Stress correct/appropriate writing techniques to students prior to the paper. 2012: Changed the admission standards for the MAcc, both overall GMAT/GRE scores (450+), as well verbal/quantitative individual minimum scores. It was expected that this improvement may take time to realize. 2013: Accounting faculty determined not to wait for admissions standards to make an impact, but to try (in addition) another approach. It will start requiring students to take a final draft of their paper to the UCA Writing Center. 100.00 80.00 11.1 31.60 40.00 45.50 60.00 40.00 88.90 68.40 60.00 54.50 Fall 2011 Fall 2012 Fall 2013 20.00 0.00 Fall 2014 % Over % Under This year the objective benchmark was met. It could be a combination of new admissions standards as well as papers being taken to the Writing Center. But using the Writing Center will be institutionalized. This objective and its results will be evaluated by a writing task force before it is reevaluated. 30 III. Assurance of Learning in the MAcc (AACSB) MAcc Objective 2b Goal: Our graduates shall be able to communicate effectively. Objective: Students will be able to communicate effectively in an oral presentation. Course(s): ACCT 6317 Measure: Benchmark: Program Placement: Data Collection: Students will make a presentation on an auditing standard. 80% of students will score > 80% on the rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Students in ACCT 6317 (offered in Spring only) make a presentation on an auditing standard. The presentation is evaluated using an oral communications rubric. Data Summary & Closing the Loop: The benchmark for this objective was met the first time it was evaluated. This objective will be evaluated in the future before any interventions or actions are considered. 100.00 80.00 60.00 100.00 40.00 20.00 0.00 Spring 2015 Spring 2016 Spring 2017 Spring 2018 100.00 31 III. Assurance of Learning in the MAcc (AACSB) % Under MAcc Objective 3a Goal: Our graduates shall be able to effectively manage information. Objective: Students will be able to locate the appropriate quality and quantity of information needed to solve a problem. Course(s): ACCT 6309 Measure: Benchmark: Students will prepare a solution to a tax research project 80% of students will score > 6 out of 8 on rubric. Program Placement: The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Data Collection: Students in ACCT 6309 (offered in Fall only) complete a tax research project, which is graded for assessment purposes using the Tax Rubric. On this rubric 4 traits assess the ability to locate information. Data Summary & Closing the Loop: Our students met the benchmark four semesters. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. This objective and related measure and benchmark will be examined by a task force prior to reevaluation in future semesters. 100.00 13.30 10.00 15.40 86.70 90.00 84.60 Fall 2011 Fall 2012 Fall 2013 80.00 60.00 40.00 100.00 20.00 0.00 % Over 32 III. Assurance of Learning in the MAcc (AACSB) % Under Fall 2014 MAcc Objective 6a Goal: Our graduates shall be able to recognize ethical dilemmas and respond ethically. Objective: Students will be able to identify ethical dilemmas and choose an appropriate course of action. Course(s): ACCT 6317 Measure: Benchmark: Program Placement: Data Collection: Students will complete an exam question related to an ethical issue. 80% of students will score > 4 out of 6 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Students in ACCT 6317 (offered in Spring only) complete an embedded test question which is graded for assessment purposes using an ethical issue rubric. Data Summary & Closing the Loop: The benchmark was met. It will be evaluated again after the new schedule is in place with the new process. 100.00 18.18 80.00 60.00 40.00 81.82 20.00 0.00 Spring 2015 Spring 2016 Spring 2017 Spring 2018 % Over 33 III. Assurance of Learning in the MAcc (AACSB) % Under The following objectives either been rotated off cycle or have not yet been assessed. MAcc Objective 1b Goal: Our graduate shall be competent in the theory and practice of accounting. Objective: Students will be able to correctly apply auditing standards. Course(s): ACCT 6317 Measure: Benchmark: Program Placement: Data Collection: Students will complete an exam question requiring the application of auditing standards. 80% of students will score > 4 out of 6 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed in Spring 2016 for the first time. MAcc Objective 1d Goal: Our graduate shall be competent in the theory and practice of accounting. Objective: Students will be able to solve unstructured problems. Course(s): ACCT 6340 Measure: Benchmark: Program Placement: Data Collection: Students will write an individual solution to a business case problem 80% of students will score > 6 out of 8 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Students in ACCT 6340 (offered in Summer only) complete a case analysis which is graded for assessment purposes using the Case Analysis Rubric. Data Summary & Closing the Loop: This objective will be assessed for the first time in Summer 2015. 34 III. Assurance of Learning in the MAcc (AACSB) MAcc Objective 2c Goal: Our graduates shall be able to communicate effectively. Objective: Students will be able to work effectively in teams. Course(s): ACCT 6340 Measure: Benchmark: Program Placement: Data Collection: Students will solve a business problem in groups. 80% of students will score > 16 out of 20 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Students in ACCT 6340 (offered in Summer only) work in teams to reevaluate cases they have analyzed individually. After each group interaction, each group member evaluates their group members using a group-work rubric. The scores by their peers are averaged over the semester to determine their teamwork score. Data Summary & Closing the Loop: The benchmark was met for this objective in both semesters (Summer 2012 and 2013). The objective was not assessed this year (Summer 2014). It will be assessed again in Summer 2015. 100.00 16.67 80.00 60.00 100.00 40.00 83.33 20.00 0.00 Su 2012 Su 2013 % Over 35 III. Assurance of Learning in the MAcc (AACSB) % Under MAcc Objective 3b Goal: Our graduates shall be able to effectively manage information. Objective: Students will be able to understand the development and structure of a database. Course(s): ACCT 6320 Measure: Benchmark: Program Placement: Data Collection: Students will generate a database that generates an income statement based on sales and acquisitions 80% of students will score > 16 out of 20 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed for the first time in Spring 2016. MAcc Objective 4a Goal: Our graduates shall be able to demonstrate effective leadership. Objective: Students will be able to demonstrate the qualities necessary for effective leadership. Course(s): ACCT 6350 Measure: Benchmark: Program Placement: Data Collection: Students will demonstrate whether they possess leadership qualities in classroom interactions. 80% of students will score > 100 out of 140 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed for the first time in Fall 2015. 36 III. Assurance of Learning in the MAcc (AACSB) MAcc Objective 5a (Measure 1) Goal: Our graduates will be aware of the global perspective and the effects of the global marketplace. Objective: Students will be able to interpret and apply international financial accounting standards. Course(s): ACCT 6310 Measure: Benchmark: Program Placement: Data Collection: Students will complete test questions requiring the application of international financial accounting concepts. 80% of students will score > 80% on the rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed for the first time in Fall 2015. MAcc Objective 5a Goal: Our graduates will be aware of the global perspective and the effects of the global marketplace. Objective: Students will be able to interpret and apply international financial accounting standards. Course(s): ACCT 6340 Measure: Benchmark: Program Placement: Data Collection: 37 (Measure 2) Students will write an individual solution to an international business problem case. 80% of students will score > 48 out of 60 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. Students in ACCT 6340 (offered in Summer only) complete a case analysis which is graded for assessment purposes using the Case Analysis Rubric. III. Assurance of Learning in the MAcc (AACSB) Data Summary & Closing the Loop: The benchmark for this objective was not met in Summer 2012, but it was close and there was a small sample size. Evaluation of the results indicated that the weaknesses were in determining a complete list of alternatives and writing the conclusion. The weakness was unrelated to the international nature of the assignment. A reassessment of the rubric and measure will be conducted before this goal is reassessed. 100.00 80.00 23.08 60.00 40.00 76.92 20.00 0.00 Summer 2012 % Over % Under MAcc Objective 7a Goal: Our graduates shall be able to think dynamically. Objective: Students will be able to speak on unprepared topics. Course(s): ACCT 6350 Measure: Benchmark: Program Placement: Data Collection: Students will give presentations on topics for which they have not prepared. 80% of students will score > 20 out of 25 on rubric. The MAcc program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MAcc students will be assessed during completion of the program. To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed for the first time in Fall 2015. 38 III. Assurance of Learning in the MAcc (AACSB) VI. Assurance of Learning in the MBA (AACSB/HLC) The College of Business presumes that participants in a master’s level degree program possess the base of general knowledge and skills appropriate to an undergraduate degree. Learning in master’s level program develops in a more integrative, interdisciplinary fashion than in undergraduate education. Master of Business Administration (MBA) Assessment Plan Goals (Our graduates shall possess) Learning Objectives (Students will be able to) 1a. Provide organizations with effective executive 1. Dynamic thinking skills in management through decision making in uncertain, a global environment dynamic, global business situations 2a. Provide organizations with effective executive management by choosing ethical courses of action 2. Ethical reasoning abilities and by articulating the manner by which they arrived at their decision 3a. Provide organizations with effective executive management through their professional written 3. Effective communication communication skills abilities 3b. Provide organizations with effective executive management through their professional oral communication skills 4. Effective information management abilities 5. Effective leadership/followership abilities 39 4a. Comprehend the strategic role of information systems in support of business activities 5a. Provide organizations with effective executive leadership under conditions of uncertainty requiring adaptation and innovation 5b. Provide organizations with effective executive management by acting as team members and collaborators Measure (Students will) Compete in global organizations, under conditions of uncertainty requiring adaptation and innovation, in the GLO-BUS Strategy Simulation—a strategy simulation designed around a global industry setting in which student teams from many nations compete head-tohead Demonstrate ethical leadership skills in global organizations, under conditions of uncertainty requiring adaptation and innovation, measured by the Corporate Social Responsibility percentile score from GLO-BUS Simulations Course MBA 6320 The institutional score for “all companies” shall be equal to or greater than the GLO-BUS “all companies” score for the appropriate time frame MBA 6320 The mean score of all students will exceed the 50th percentile. Prepare a report, paper, or case analysis on a business topic MBA 6308 Prepare a report, paper, or case analysis on a business topic MBA 6308 Demonstrate sound, and well-informed decision making, on a written assignment (business case or other writing assignment) that reflects the ability to effectively collect and appropriately use information to make business decisions Demonstrate leadership skills in global organizations, under conditions of uncertainty requiring adaptation and innovation. The basis for assessment will be the leadership percentile score from GLO-BUS Simulations Demonstrate collaboration skills in global organizations, under conditions of uncertainty requiring adaptation and innovation. The basis for assessment will be the collaboration and teamwork percentile score from GLOBUS Simulations IV. Assurance of Learning in the MBA (AACSB) Benchmark Seventy-five percent of students assessed with the rubric will score 70% or above. Seventy-five percent of students assessed with the rubric will score 70% or above MBA 6301 Seventy percent of students will earn at least 70% or more of the possible points on the assignment MBA 6320 The mean score of all students will exceed the 50th percentile. MBA 6320 The mean score of all students will exceed the 50th percentile. B. Rotation of Goals During any given semester, we are assessing four of our learning objectives. A learning objective shall be assessed for four semesters (duration depends on whether the course is offered every semester or only once a year). If a learning objective is satisfied (Assessment threshold is met or exceeded) for three out of the four assessment periods, then that learning objective will drop out of the active assessment cycle and be replaced by the next objective in the queue. The table below indicates the rotation of these goals since the start of this assurance of learning program in Fall 2011. The schedule for next year is being Objective Fall 2011 Spring 2012 Fall 2012 Spring 2013 Fall 2013 Spring 2014 Fall 2014 Spring 2015 1A Fail Fail Fail Fail 2A Fail Fail Fail Fail 3A Pass 3B Pass Pass Fall 2015 Spring 2016 Pass Fail Fail Fail 4A 5A 5B C. Assessment Results Assess Pass Pass Assess The following pages contain details of each objective, benchmark, assessment method along with assessment data and actions taken to close the loop. The objectives assessed in the past year are covered first (1a, 2a, 3b, 5a). 40 IV. Assurance of Learning in the MBA (AACSB) MBA Objective 1a Goal: Our graduates shall have dynamic thinking skills in a global environment. Objective: Students will be able to provide organizations with effective executive management through decision making in uncertain, dynamic, global business situations Course(s): MBA 6320 Measure: Students will compete in global organizations, under conditions of uncertainty requiring adaptation and innovation, in the GLO-BUS Strategy Simulation—a strategy simulation designed around a global industry setting in which student teams from many nations compete head-to-head Benchmark: The institutional score for “all companies” shall be equal to or greater than the GLO-BUS “all companies” score for the appropriate time frame. Program Placement: The MBA program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MBA students will be assessed during completion of the program. Data Collection: Each Spring, students in MBA 6320 participate in the GLO-BUS Strategy Simulation as a required portion of their course grade. The GLO-BUS website reports individual simulation and course average simulation scores for the semester along with the national average. Data Summary/Closing the Loop: This objective has now failed four consecutive semesters. The average crept up some from last year, but is still well below the benchmark. Here is a synopsis of strategies used to improve student learning/performance for global decisionmaking. The first three years come from previous Annual Reports: 2012: Do some additional instruction on decision-making in a global environment in class. All instructors and CAC members attended video-training on GLO-BUS and how it calculates averages. It is hoped that by being able to better inform students of the process, it may help them perform better and learn more. 2013: Improvement was noted in the second assessment; a rise from 69-76%. While still not a passing score, it suggests that perhaps improvement efforts are working. In examining the data more closely, it showed that one area of weakness was accurately reading financial statements. The decision was made to add an assignment early in the class for all students to analyze a financial statement. 41 IV. Assurance of Learning in the MBA (AACSB) 100.0 75.0 84.0 69.0 85.0 84.0 76.0 61.0 86.0 66.0 50.0 25.0 0.0 Spring 2012 Spring 2013 UCA Spring 2014 National Spring 2015 2014: In the third assessment, student performance worsened, disappointingly so. The progress noted last year vanished. While the decision was made to keep the examination of financial statements in the curriculum (see below), a meeting was held with the MBA Director and the entire CAC, which occurred in Fall 2014. A variety of alternatives were discussed. Some of the results: For overall team performance (we use teams of 2-3), upon which this assessment is based, nine areas are graded, which when averaged becomes the overall score. Our benchmark is that our average for all teams will be greater than the average across the globe for that semester. In Spring 2014, for example, that included 5090 students, 1801 companies (i.e., teams of students), from 97 schools in 15 countries. For this assessment, we had twelve teams, consisting of 34 students. Teams overall from UCA scored between 32 and 111, with the average at 61 (overall the average globally, and therefore our benchmark, was 85). We had three teams (of 12) better the benchmark. By far the lowest of the nine areas graded was financial management, with an average of 22; interestingly, the highest average was financial analysis, with an average of 64. It appears that extra instruction on financial analysis works. Financial management, with the lowest score, is the ability of teams to apply financial management principles, such as ROE, credit rating, and stop price performance. The group discussed at length the ability of GLO-BUS to realistically assess student learning. On the one hand, performance in this simulation is indeed based on “global” decision-making, and how each team aligns its resources in nine different simulated rounds. But there are conflicting factors which hamper success of teams (and thus their overall performance). One poor round, especially early, can lead to teams spiraling down because they take increased chances or risks, which may not accurately reflect their overall learning. In this class, we have stressed ethical decision making (see 2a below), given our failures in that objective. It is assessed using the Corporate Social Responsibility area in GLO-BUS (one of the nine areas). But improving scores in this one area (objective 2a below) frequently has a detrimental effect on overall performance, which is how this assessment (1a) is measured. In addition, other teams (the other 1789 for this semester) are an unknown quantity against which our teams compete. We do not know how many team members other schools use (perhaps larger teams might score better, given more expertise), nor do we have any control whatsoever in their performance or curriculum or its placement in the program (perhaps a later placement of this simulation leads to better results). We do know the overall average for all teams over several years is consistent, but have no way of knowing whether we really should be in the upper 50%. What should our benchmark really be? Using GLO-BUS to assess global decision making (and even the other four objectives which use it) is problematic. Its ability to accurately reflect student learning is questionable; there may be too much measurement error. That said, it does have some positive features. The group decided that the use of GLO-BUS for assessing this objective should be continued at least one more round. Further, after the next assessment cycle, if it reveals nothing new, then another task force should be assembled to determine what should change. Should the benchmark change? Should a different measurement be used? A new rubric? 2015: Based on the above actions, and in concert with our new process for assessment, we will refer the GLO-BUS discussion to an improvement task force, to examine and recommend the best way forward, in accordance with a schedule to be determined. This group will examine if GLO-BUS is an appropriate way to assess global decision-making learning and if so, what the benchmark should be. It will look at alternative ways to measure this objective. 42 IV. Assurance of Learning in the MBA (AACSB) MBA Objective 2a Goal: Our graduates shall have ethical reasoning abilities Objective: Students will be able to provide organizations with effective executive management by choosing ethical courses of action and by articulating the manner by which they arrived at their decision Course(s): MBA 6320 Measure: Students will demonstrate ethical leadership skills in global organizations, under conditions of uncertainty requiring adaptation and innovation, measured by the Corporate Social Responsibility percentile score from GLO-BUS Simulations Benchmark: The mean score of all students will exceed the 50th percentile. Program Placement: The MBA program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MBA students will be assessed during completion of the program. Data Collection: Each Spring, students in MBA 6320 participate in the GLO-BUS Strategy Simulation as a required portion of their course grade. The GLO-BUS website reports individual social responsibility and course average social responsibility scores for the semester both in terms of global percentile. Data Summary/Closing the Loop: 100.0 After three failures, this objective passed for the first time this year. Here is what has happened previously, drawn from last year’s Annual Report: 75.0 2012: Do some additional instruction on social responsibility in the context of using 52.0 this simulation. All instructors and CAC members attended video-training on GLO46.0 43.0 50.0 BUS and how it calculates averages. It is hoped that by being able to better inform 34.0 students of the process, it may help them perform better and learn more. 2013: Improvement was noted in the second assessment; a rise from 34-43%, a 25.0 relatively significant rise of 9 percentage points. While still not a passing score, it suggests that perhaps improvement efforts are working. A discussion was held by 0.0 faculty and CAC members on how to improve social responsibility by the teams taking Spring 2012 Spring 2013 Spring 2014 Spring 2015 part in the simulations. Given the rise from last year, continued emphasis in class on this topic was stressed as important. 2014: Another relatively good rise was noted in the recent assessment. While still not at the benchmark of 50, it is an interesting improvement from last year. However, given the three failures using GLO-BUS for objectives 1a and 2a (global decision-making and ethical reasoning), a discussion was held between the MBA Director and CAC. The gist of the discussion is provided in the discussion in 1a above. But for this 43 IV. Assurance of Learning in the MBA (AACSB) objective, ethical reasoning, there are some differences. Unlike 1a (global decision-making), the average scores have improved. The emphasis placed in class on social responsibility have reaped some dividends, and it is possible it may reach the benchmark at some point. However, much of the discussion concerned the impact that higher scores in social responsibility have on overall performance. A team that puts too much emphasis on social responsibility (thus increasing the average for this objective), frequently does so at the expense of overall performance. In other words, there are tradeoffs, which make this type of measuring tool problematic in assessing learning. The decision was made to assess this using GLO-BUS one more time and analyze the results. If nothing changes, then another formal improvement team will meet to discuss/analyze if and how GLO-BUS should be appropriately used to assess learning. 2015: This year this objective finally passed after three consecutive failures. The continued emphasis on social responsibility has finally paid dividends in terms of meeting the benchmark. That said, there are some significant problems noted in the use of GLO-BUS to measure this objective. These concerns have been noted before, and are delineated at some length in the discussion noted above for objective 1a. In essence, there may be (or at least can be) a negative correlation between overall performance (used as the measure for 1a) and social responsibility (used as the measure for this objective). A team that focuses too many resources on social responsibility may score more poorly overall. While such a tradeoff may be present in some or even many industries and/or organizations, using the social responsibility percentile of GLO-BUS probably does not accurately assess ethical reasoning abilities. The COB CAC and MBA Director have decided that a change to how this objective is measured is in order. It will take place by an improvement team in concert with the future discussions noted above concerning GLO-BUS. 44 IV. Assurance of Learning in the MBA (AACSB) MBA Objective 3b Goal: Our graduates shall have effective communication abilities. Objective: Students will be able to provide organizations with effective executive management through their professional oral communication skills Course(s): MBA 6308 Measure: Students will prepare a report, paper, or case analysis on a business topic. Benchmark: Seventy-five percent of students assessed with the rubric will score 70% or above Program Placement: The MBA program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MBA students will be assessed during completion of the program. Data Collection: Students in 6308 give an oral presentation as part of their course grade. Each submission was graded using the MBA Presentation Rubric and un-aggregated data was reported to indicate how many students exceeded the benchmark of 39 rubric points. Data Summary & Closing the Loop: In Fall 2011, the benchmark was met. In Fall 2012 and 2013, the benchmark was not met, though in both cases it was fairly close. In Fall 2014, the benchmark was not met again, this time by a larger margin. 100.00 80.00 28.60 28.60 34.8 71.40 71.40 65.20 Fall 2012 Fall 2013 Fall 2014 60.00 One of the problems noted is a lack of understanding of why these failed. While we know that 75% of students did not score 39 rubric points or higher, what is lacking in this and in previous data collection efforts is what students did not do well at in oral presentations. We do not have sufficient granularity, and unfortunately no longer have the data except in aggregate form. In our new process, we will keep and use raw data. Still, this objective is scheduled for improvement in AY 2015-2016. 100.00 40.00 20.00 0.00 Fall 2011 % Over 45 IV. Assurance of Learning in the MBA (AACSB) % Under MBA Objective 5a Goal: Our graduates shall have effective leadership/followership abilities. Objective: Students will be able to provide organizations with effective executive leadership under conditions of uncertainty requiring adaptation and innovation Course(s): MBA 6320 Measure: Students will demonstrate leadership skills in global organizations, under conditions of uncertainty requiring adaptation and innovation. The basis for assessment will be the leadership percentile score from GLO-BUS Simulations Benchmark: The mean score of all students will exceed the 50th percentile. Program Placement: The MBA program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MBA students will be assessed during completion of the program. Data Collection: Each spring, students in MBA 6320 participate in the GLO-BUS Strategy Simulation as a required portion of their course grade. The GLO-BUS website reports individual leadership and course average leadership scores for the semester both in terms of global percentile. Data Summary & Closing the Loop: The benchmark has been met in two consecutive semesters. This measure comes from the Leadership Skills area of GLO-BUS; this is an individual score based on “comanagers answers for items 4-7 and 10 on the peer evaluation exercise”. Like the other GLO-BUS measures, this is not perfect, but seems to be at this early stage an appropriate means to assess leadership and followership ability. That said, using GLO-BUS will be examined by improvement teams for both 1a (global decision making) and 2a (ethical reasoning ability); this examination may impact this objective. It is scheduled to be assessed next year (in accordance with the new schedule). 100.0 75.0 53.0 59.0 50.0 25.0 0.0 Spring 2014 Spring 2015 Spring 2016 Spring 2017 46 IV. Assurance of Learning in the MBA (AACSB) The objectives below were not assessed in the past year. They were assessed previously or have not yet been assessed. MBA Objective 3a Goal: Our graduates shall have effective communication abilities Objective: Students will be able to provide organizations with effective executive management through their professional written communication skills Course(s): MBA 6308 Measure: Students will prepare a report, paper, or case analysis on a business topic Benchmark: Seventy-five percent of students assessed with the rubric will score 70% or above. Program Placement: The MBA program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MBA students will be assessed during completion of the program. Data Collection: Student in 6308 completed a written report, paper, or case analysis as part of their course grade. Each submission were graded using the MBA Writing Rubric and unaggregated data was reported to indicate how many student exceeded the benchmark of 9 rubric points. Data Summary & Closing the Loop: Our students met the benchmark in three semesters. Per our assurance of learning plan, the results were discussed by the Curriculum and Assessment Committee, reported to the departments by the committee members, and discussed in Executive Committee. No further action was recommended, and per our assurance of learning plan, this objective was dropped from the rotation after four semesters and replaced by another objective. This objective and related measure and benchmark will be examined by a task force prior to reevaluation in future semesters. 100.00 15.40 80.00 60.00 40.00 100.00 100.00 Fall 2011 Summer 2012 84.60 20.00 0.00 % Over 47 IV. Assurance of Learning in the MBA (AACSB) Fall 2012 % Under MBA Objective 4a Goal: Our graduates shall have effective Information management abilities. Objective: Students will be able to comprehend the strategic role of information systems in support of business activities. Course(s): MBA 6301 Measure: Students will demonstrate sound, and well-informed decision making, on a written assignment (business case or other writing assignment) that reflects the ability to effectively collect and appropriately use information to make business decisions. Benchmark: Seventy percent of students will earn at least 70% or more of the possible points on the assignment. Program Placement: The MBA program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MBA students will be assessed during completion of the program. Data Collection: To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed for the first time in Spring 2016. 48 IV. Assurance of Learning in the MBA (AACSB) MBA Objective 5b Goal: Our graduates shall have effective leadership/followership abilities. Objective: Students will be able to provide organizations with effective executive management by acting as team members and collaborators. Course(s): MBA 6320 Benchmark: The mean score of all students will exceed the 50th percentile. Measure: Students will demonstrate collaboration skills in global organizations, under conditions of uncertainty requiring adaptation and innovation. The basis for assessment will be the collaboration and teamwork percentile score from GLO-BUS Simulations Program Placement: The MBA program may be started and completed in any semester, so there is no course sequencing. However, this is a required course, so all MBA students will be assessed during completion of the program. Data Collection: To date, no data has been collected for this objective (not in rotation). Data Summary & Closing the Loop: This objective is scheduled to be assessed for the first time in Spring 2016. 49 IV. Assurance of Learning in the MBA (AACSB) VII. Assurance of Learning in the BBA (HLC #8) A. Program Goals, Objectives, Measure, and Benchmarks AACSB assesses by program (BBA, MBA, MAcc). HLC assesses by major. The learning goals and objectives for both are exactly the same, except for one discipline-specific goal (i.e., major). This goal, associated with each of the majors, we label HLC #8. It examines the depth of one’s knowledge and is assessed summatively (at the end of one’s program). The list below includes all BBA goals, including the discipline-specific one, #8: Our graduates shall possess: 1. Critical Thinking & Analytical Thinking Skills; 2. Awareness of the Global Business Environment; 3. Ethical Reasoning Abilities; 4. Effective Communication Abilities; 5. Effective Collaborative Skills; 6. Effective Information Management Skills; 7. Understanding of a Broad Range of Business Disciplines. 8. Knowledge Appropriate to the Practice of their Major Discipline. For each Learning Goal, the COB has defined at least one corresponding Learning Objective. Goal 8 is summarized on the following page. (Summaries of goals 1 through 7 are found in Section IV.) B. Rotation of Goals Goals 1 through 7 are rotated as described in Chapter II. Goal 8 has not been rotated. Data is collected in every fall and spring semester that the course is offered. However it will be rotated with our new AOL scheduling, to be implemented this coming fall. C. Assessment Results The table below lists each discipline’s objective associated with Learning Goal #8, including the objective, benchmark, assessment method, followed by assessment data. 50 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Bachelor of Business Administration (BBA) Assessment Plan: Discipline Specific Goal 8 Goals Learning Objectives 8a. Accounting graduates will demonstrate an understanding of accounting topics in the areas of financial, cost, tax, government/nonprofit, accounting information systems, and auditing. 8. Our graduates shall possess knowledge appropriate to the practice of their major discipline. 8a. Finance graduates will be able to value financial assets, analyze and manage financial risks in a business environment, and apply financial analysis techniques to diverse business situations, demonstrating a thorough knowledge of financial decision-making skills and financial markets. 8a. Insurance & Risk Management graduates will possess the skills and knowledge base necessary to effectively counsel clients about insurance and risk management strategies and products. 8a. Management graduates will demonstrate the decision-making, organizing, and interaction knowledge and skills needed in the global business environment. 8a. Marketing graduates will demonstrate the promotion, retailing, sales, and marketing management knowledge and skill sets necessary to meet the requirements of the global business environment. 8a. Innovation & Entrepreneurship graduates will possess the interdisciplinary skills needed to start and operate a small business. 8a. Economics graduates will possess the analytical skills to formulate an appropriate, discipline-specific question, research possible solutions, and provide accurate analysis. 8a. Management Information Systems graduates will understand the key competencies that comprise the planning, analysis, design, and implementation of highquality organization information systems. 51 Measure Course Each student must pass a comprehensive accounting exam comprised of questions written by the accounting faculty. The exam covers the 6 major topics in accounting. ACCT 4312 Each student must pass a comprehensive finance exam comprised of questions written by the finance faculty. The exam covers the major topics in valuation, financial modeling, and financial decision-making aptitudes. FINA 4336 70% of all students who have completed all other Finance classes will score seventy percent or above. INSU 3315 UCA’s student median grade will equal the national median grade. MGMT 4348 Mean score of students assessed will be 8.4 out of 12 or above on the MGMT 4348 Rubric. Students will prepare a research project on a marketing topic, which will embellish their knowledge and skills in a large number of skill areas. MKTG 4355 Mean score of students assessed will be 8.4 out of 12 or above on the MKTG 4355 Rubric. Students will create and present a thorough convincing business plan. MGMT 4376 Economics students will prepare a research paper demonstrating the analytical skills to formulate an appropriate, discipline-specific question, research possible solutions, and provide accurate analysis. ECON 4380 Each student must pass appropriate questions in exams comprised of questions written by the MIS faculty. Students will demonstrate an understanding of the systems development life cycle and be able to apply the most appropriate methodologies and tools to resolve organization information systems issues and be able to effectively communicate appropriate solutions. MIS 3328 Students will take the UACIC Life and Health Exam as a final in INSU 3315. This exam is the first of two parts of obtaining the nationally recognized UAcIC (Undergraduate Associate Certified Insurance Counselor) designation. Management majors will prepare a research project on a management topic, which will embellish their knowledge and skills in a large number of skill areas. VIII. Assurance of Learning in the BS/BA in Economics (HLC) Benchmark 70% of all students who have completed or are concurrently enrolled in all other major required Accounting classes will score seventy percent or above. The mean score of students assessed will be 140 out of 200 or above on the MGMT 4376 Rubric. The mean of students’ points will be 7 (of 12) or greater on the ECON 4380 Discipline Specific Assessment Rubric. The mean score of students assessed will be 70 percent or higher on the assessment exam questions. BBA Objective 8 Accounting Goal: Our graduates shall possess knowledge appropriate to the practice of their major discipline. Objective: Accounting graduates will demonstrate an understanding of accounting topics in the areas of financial, cost, tax, government/nonprofit, accounting information systems, and auditing. Course(s): ACCT 4312 Benchmark: Each student must pass a comprehensive accounting exam comprised of questions written by the accounting faculty. The exam covers the 6 major topics in accounting. Measure: Seventy percent of all students who have completed or are concurrently enrolled in all other major required Accounting classes will score seventy percent or above. Program Placement: Student may take ACCT 4312 as soon as they have finished ACCT 2310, 2311, 3311, and 3312 (the latter two with a C or better). For most accounting majors, ACCT 4312 is taken in their last or next to last semester in their undergraduate studies. Data Collection: A comprehensive accounting exam is given near the end of the semesters when ACCT 4312 is offered. The exam consists of 40 questions (4 from each of the 10 upper-division accounting courses). Students are instructed to skip question sets of 4 for classes that haven’t completed or are not currently taking. Each student’s average is based only on the questions they should have answered. Data Summary & Closing the Loop: In both Fall 2013 and Spring 2014, the benchmark was not met by large margin. This has been problematic in every assessment. While there may be multiple issues, it is clear that this objective needs close examination. This objective is set to be examined by an improvement team next year (AY 2015-2016). 100.00 80.00 60.00 92.90 95.30 7.10 4.70 Fall 2013 Spring 2014 84.20 88 15.80 12.00 Fall 2014 Spring 2015 40.00 20.00 0.00 % Over 52 VIII. Assurance of Learning in the BS/BA in Economics (HLC) % Under BBA Objective 8 Economics Goal: Our graduates shall possess knowledge appropriate to the practice of their major discipline. Objective: Economics graduates will possess the analytical skills to formulate an appropriate, discipline-specific question, research possible solutions, and provide accurate analysis. Benchmark: Economics students will prepare a research paper demonstrating the analytical skills to formulate an appropriate, discipline-specific question, research possible solutions, and provide accurate analysis. Measure: The mean of students’ points will be 7 (of 12) or greater on the ECON 4380 Discipline Specific Assessment Rubric. Course(s): ECON 4380 Program Placement: Generally taken in final semester; offered Spring only. Data Collection: Data Summary & Closing the Loop: The benchmark was met in Spring 2014. In Spring 2015, the instructor for the course resigned in mid-semester, and was replaced. The scheduled assessment did not get completed. This will be assessed again in Spring 2016 as scheduled. 12.00 11.92 9.00 6.00 3.00 0.00 Spring 2014 Spring 2015 Spring 2016 Spring 2017 53 VIII. Assurance of Learning in the BS/BA in Economics (HLC) BBA Objective 8 Finance Goal: Our graduates shall possess knowledge appropriate to the practice of their major discipline. Objective: Finance graduates will be able to value financial assets, analyze and manage financial risks in a business environment, and apply financial analysis techniques to diverse business situations, demonstrating a thorough know Benchmark: Each student must pass a comprehensive finance exam comprised of questions written by the finance faculty. The exam covers the major topics in valuation, financial modeling, and financial decisionmaking aptitudes. Measure: Seventy percent of all students who have completed all other Finance classes will score seventy percent or above. Course(s): FINA 4336 Program Placement: Generally taken in final semester; offered Spring only. Data Collection: Data Summary & Closing the Loop: The benchmark has was not met in Spring 2014, though it was relatively close (65.3 vs. 70). The decision was made to continue assessing. In Spring 2015, the benchmark was met. This will be assessed next spring (2016), and combined with this last to provide the benchmark for an improvement team. 100.00 80.00 34.70 28.57 65.30 71.43 60.00 40.00 20.00 0.00 Spring 2014 Spring 2015 Spring 2016 Spring 2017 % Over 54 VIII. Assurance of Learning in the BS/BA in Economics (HLC) % Under BBA Objective 8 Management Goal: Our graduates shall possess knowledge appropriate to the practice of their major discipline. Objective: Management graduates will demonstrate the decision-making, organizing, and interaction knowledge and skills needed in the global business environment. Benchmark: Management majors will prepare a research project on a management topic, which will embellish their knowledge and skills in a large number of skill areas. Measure: Mean score of students assessed will be 8.4 out of 12 or above on the MGMT 4348 Rubric. Course(s): MGMT 4348 Program Placement: Data Collection: Data Summary & Closing the Loop: The benchmark was met in Fall 2013, Spring 2014, Fall 2014 and Spring 2015. In accordance with the schedule, this will be examined by an improvement team in 2016. 14.00 11.50 10.78 9.60 10.32 7.00 0.00 Fall 2013 55 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Spring 2014 Fall 2014 Spring 2015 BBA Objective 8 Marketing Goal: Our graduates shall possess knowledge appropriate to the practice of their major discipline. Objective: Marketing graduates will demonstrate the promotion, retailing, sales, and marketing management knowledge and skill sets necessary to meet the requirements of the global business environment. Benchmark: Students will prepare a research project on a marketing topic, which will embellish their knowledge and skills in a large number of skill areas. Measure: Mean score of students assessed will be 8.4 out of 12 or above on the MKTG 4355 Rubric. Course(s): MKTG 4355 Program Placement: Data Collection: Data Summary & Closing the Loop: 14.00 12.73 10.40 The first two assessments passed; the last two did not. Using the available data, this objective will be examined by an improvement team in the next year. It is possible that that in the last two assessments there were measurement error issues. 6.62 7.00 4.95 0.00 Fall 2013 56 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Spring 2014 Fall 2014 Spring 2015 BBA Objective 8 Insurance and Risk Management Goal: Our graduates shall possess knowledge appropriate to the practice of their major discipline. Objective: Insurance & Risk Management graduates will possess the skills and knowledge base necessary to effectively counsel clients about insurance and risk management strategies and products. Benchmark: Students will take the UACIC Life and Health Exam as a final in INSU 3315. This exam is the first of two parts of obtaining the nationally recognized UAcIC (Undergraduate Associate Certified Insurance Counselor) designation. Measure: UCA’s student median grade will equal the national median grade. Course(s): INSU 3315 Program Placement: Data Collection: Data Summary & Closing the Loop: This was assessed for the first time in Fall 2014. It did not pass the benchmark although it was relatively close. In accordance with our new schedule, this will be assessed in Fall 2015, and the results combined to provide data for an improvement team. 100.0 75.0 70.5 61.1 50.0 25.0 0.0 Fall 2014 UCA 57 VIII. Assurance of Learning in the BS/BA in Economics (HLC) National BBA Objective 8 Innovation and Entrepreneurship Goal: Our graduates shall possess knowledge appropriate to the practice of their major discipline. Objective: Innovation & Entrepreneurship graduates will possess the interdisciplinary skills needed to start and operate a small business. Benchmark: Students will create and present a thorough convincing business plan. Measure: The mean score of students assessed will be 140 out of 200 or above on the MGMT 4376 Rubric. Course(s): MGMT 4376 Program Placement: Data Collection: Data Summary & Closing the Loop: This was assessed for the first time in Fall 2014. It passed the benchmark. In accordance with our new schedule, this will be assessed in Fall 2015, and the results combined to provide data for an improvement team. 200.00 168.50 150.00 100.00 50.00 0.00 Fall 2014 58 VIII. Assurance of Learning in the BS/BA in Economics (HLC) BBA Objective 8 Management Information Systems Goal: Our graduates shall possess knowledge appropriate to the practice of their major discipline. Objective: Management Information Systems graduates will understand the key competencies that comprise the planning, analysis, design, and implementation of high-quality organization information systems. Benchmark: Each student must pass appropriate questions in exams comprised of questions written by the MIS faculty. Students will demonstrate an understanding of the systems development life cycle and be able to apply the most appropriate methodologies and tools to Measure: The mean score of students assessed will be 70 percent or higher on the assessment exam questions. Course(s): MIS 3328 Program Placement: Most students take this late in their program, as a capstone course. However it may be taken earlier. Data Collection: Data collected as part of a regularly scheduled exam. Data Summary & Closing the Loop: The benchmark has been met in all four semesters. It will be improved by an improvement team scheduled in 2016-2017. 100 75 89.1 70.6 72.4 Fall 2013 Spring 2014 78.4 50 25 0 59 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Fall 2014 Spring 2015 VIII. Assurance of Learning in the BS/BA in Economics (HLC) A. Program Goals, Objectives, Measure, and Benchmarks There are three goals in the BS/BA in Economics Assurance of Learning program. Our graduates shall possess: 1. A knowledge base that helps students ask more informed questions and analyze complex situations 2. Effective communication abilities 3. Ethical reasoning abilities For each Learning Goal, the COB has defined at least one corresponding Learning Objective. The result is 4 objectives assessed in 1 class. The program is summarized below. Bachelor of Arts/Bachelor of Science in Economics (BA/BS) Assurance of Learning Plan Goal: Our graduates shall possess 1. Knowledge base that helps students ask more informed questions and analyze complex situations Objectives: Students will be able to 1a.Students will be able to formulate an appropriate, discipline-specific question, research possible solutions, and provide accurate analysis Course Benchmark Students will prepare a research paper or other assignment on a discipline-specific topic ECON 4380 The mean of students’ points will be 8 (of 12) or greater on the UCA Critical Inquiry Rubric. 2a. Students will be able to produce professional quality research documents. Students will prepare a report, paper, or case analysis on a discipline-specific topic in the writing-intensive course designated in their majors ECON 4380 2b. Students will be able to produce professional quality research presentations. Students will present in class an analysis of a major research publication assigned by the instructor. ECON 4380 3a. Students will be aware of ethical issues inherent in decisions and articulate the manner in which they arrived at an ethical decision Students will complete an assignment (business case or other writing assignment) dealing with ethical decision making or the social responsibility of business. ECON 4380 2. Effective communication abilities. 3. Ethical reasoning abilities. 60 Measure: Students Will VIII. Assurance of Learning in the BS/BA in Economics (HLC) Mean score of students assessed with the rubric will be eleven (11) points, or greater, out of sixteen (16) available on the UCA Communication-Written Rubric. Mean score of students assessed with the rubric will be eleven (11) points, or greater, out of sixteen (16) available points on the UCA Communication-Oral Rubric. Mean score of students assessed with the rubric will be six (6) points, or greater, out of the available eight (8) points on the UCA Responsible Living Rubric (Goal 1 only). B. Rotation of Goals Goals in the BS/BA in Economics have not been rotated. Data is collected in every fall and spring semester that the course is offered. This will change as we move into our new scheduling in Fall 2015. C. Assessment Results The following pages contain details of each objective, benchmark, assessment method along with assessment data and actions taken to close the loop. This assessment program was created in Summer 2013, data is reported for goal 8 beginning in Fall 2013. 61 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Objective 1A Goal: Our graduates shall possess knowledge base that helps students ask more informed questions and analyze complex situations Objective: Students will be able to formulate an appropriate, discipline-specific question, research possible solutions, and provide accurate analysis Benchmark: Students will prepare a research paper or other assignment on a discipline-specific topic Measure: ECON 4380 Course(s): The mean of students’ points will be 8 (of 12) or greater on the UCA Critical Inquiry Rubric. Program Placement: Generally taken in final semester; offered Spring only. Data Collection: Data Summary & Closing the Loop: The benchmark was exceeded in Spring 2014. In Spring 2015, there was no assessment because the instructor for the course resigned in mid-semester. This will be assessed again in Spring 2016 in accordance with our schedule. 12.00 9.56 9.00 6.00 3.00 0.00 Spring 2014 Spring 2015 Spring 2016 Spring 2017 62 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Objective 2A Goal: Our graduates shall possess effective communication abilities. Objective: Students will be able to produce professional quality research documents. Benchmark: Students will prepare a report, paper, or case analysis on a discipline-specific topic in the writingintensive course designated in their majors Measure: ECON 4380 Course(s): Mean score of students assessed with the rubric will be eleven (11) points, or greater, out of sixteen (16) available on the UCA Communication-Written Rubric. Program Placement: Generally taken in final semester; offered Spring only. Data Collection: Data Summary & Closing the Loop: The benchmark was exceeded in Spring 2014. In Spring 2015, there was no assessment because the instructor for the course resigned in mid-semester. This will be assessed again in Spring 2016 in accordance with our schedule. 16.00 12.00 11.92 8.00 4.00 0.00 Spring 2014 Spring 2015 Spring 2016 Spring 2017 63 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Objective 2B Goal: Our graduates shall possess effective communication abilities. Objective: Students will be able to produce professional quality research presentations. Benchmark: Students will present in class an analysis of a major research publication assigned by the instructor. Measure: ECON 4380 Course(s): Mean score of students assessed with the rubric will be eleven (11) points, or greater, out of sixteen (16) available points on the UCA Communication-Oral Rubric. Program Placement: Generally taken in final semester; offered Spring only. Data Collection: Data Summary & Closing the Loop: The benchmark was exceeded in Spring 2014. In Spring 2015, there was no assessment because the instructor for the course resigned in mid-semester. This will be assessed again in Spring 2016 in accordance with our schedule. 16.00 12.10 12.00 8.00 4.00 0.00 Spring 2014 Spring 2015 Spring 2016 Spring 2017 64 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Objective 3A Goal: Our graduates shall possess ethical reasoning abilities. Objective: Students will be aware of ethical issues inherent in decisions and articulate the manner in which they arrived at an ethical decision Benchmark: Students will complete an assignment (business case or other writing assignment) dealing with ethical decision making or the social responsibility of business. Measure: ECON 4380 Course(s): Mean score of students assessed with the rubric will be six (6) points, or greater, out of the available eight (8) points on the UCA Responsible Living Rubric (Goal 1 only). Program Placement: Generally taken in final semester; offered Spring only. Data Collection: Data Summary & Closing the Loop: The benchmark was exceeded in Spring 2014. In Spring 2015, there was no assessment because the instructor for the course resigned in mid-semester. This will be assessed again in Spring 2016 in accordance with our schedule. 8.00 6.72 6.00 4.00 2.00 0.00 Spring 2014 65 VIII. Assurance of Learning in the BS/BA in Economics (HLC) Spring 2015 Spring 2016 Spring 2017 IX. Assurance of Learning in the UCA Core (HLC) Under construction during pilot study. No goals or data to report at this time. Appendix. BS in Information Systems - Curriculum General Education: 38 hours Includes the following courses: MATH 1395 Business Calculus or MATH 1491 Applied Calculus ECON 2320 Macroeconomics MGMT 2301 Business Communications ECON 2310 Global Environment of Business Business Requirements: 24 hours ECON 2321 Microeconomics ACCT 2310 Accounting I MATH 2311 or QMTH 2330 Statistics ACCT 2321 Legal Environment of Business MIS 3321 Managing Systems and Technology FINA 3330 Managing Finance & Capital MGMT 3340 Managing People and Work MKTG 3350 Principles of Marketing Minor in Computer Science (required): 18 hours CSCI 1470 CS I CSCI 1480 CS II 66 IX. Assurance of Learning in the UCA Core (HLC) CSCI 2320 Data Structures CSCI 3190 Social Implications of Technology Six hours of upper-level CS course credit (6 hours) Information Systems Requirement: 30 hours CSCI 1340 or MIS 3335 Programming in Python MIS 3365 or CSCI 3360 Databases MIS 3363 or CSCI 3335 Networks CSCI 3381 or MIS 4339 Java (MIS Java II course) MIS 3328 Systems Analysis and Design CSCI 3375 or MIS 3382 Internship (or one advanced CS/MIS course approved by chair); a maximum of six hours of internship may count for the degree MIS 4360 or CSCI 4315 Information Security MIS 4355 Project Management CSCI 4305 Linux/Unix or CSCI 4365 Web Technology or CSCI 4370 Data Mining or MIS 4329 Database Management Systems or MIS 4366 Advanced Web Design or MIS 4370 Advanced Web Design with Databases 3 hours advanced course electives in CSCI/MIS/QMTH (could include MATH 3311 Stats II or WRTG 3310 Tech. Writing or a second internship) Electives: 10 hours (9 if MATH 1491 taken) Total: 120 hours 67 IX. Assurance of Learning in the UCA Core (HLC)