West Texas A&M University College of Business Assessment Plan 2008-2009 Academic Year TABLE OF CONTENTS 1.0 COLLEGE OF BUSINESS ASSESSMENT PROCESS 1.1 Mission 1.2 Assessment Management Structure 1.3 Curriculum Management Structure 1.4 Explicit Curriculum Assessment Methods 1.5 Program Review Process 1.6 Assurance of Learning Task by Role 1.7 Assessment Plan Enhancement 2.0 COLLEGE OF BUSINESS INTERNAL ASSESSMENT FOR THE BBA 2.1 Learning Goals for the BBA Program 2.2 Learning Objectives for the BBA Program 2.3 Core Courses Aligned with Learning Goals for the BBA Program 2.4 Methods and Instruments of Measurement for the BBA Program 2.5 Traits, Standards, and/or Rubrics for the BBA Program 2.6 Dissemination and Review Process 2.7 Closing the Loop for the BBA Program 2.8 Assessment Action Report for the BBA Program 3.0 COLLEGE OF BUSINESS INTERNAL ASSESSMENT FOR THE MBA 3.1 Learning Goals for the MBA Program 3.2 Learning Objectives for the MBA Program 3.3 Core Courses Aligned with Learning Goals for the MBA Program 3.4 Methods and Instruments of Measurement for the MBA Program 3.5 Traits, Standards, and/or Rubrics for the MBA Program 3.6 Dissemination and Review Process 3.7 Closing the Loop for the MBA Program 3.8 Assessment Action Report for the MBA Program 4.0 COLLEGE OF BUSINESS INTERNAL ASSESSMENT FOR THE MPA 4.1 Learning Goals for the MPA 4.2 Learning Objectives for the MPA Program 4.3 Core Courses Aligned with Learning Goals for the MPA Program 4.4 Methods and Instruments of Measurement for the MPA Program 4.5 Traits, Standards, and/or Rubrics for the MPA Program 4.6 Dissemination and Review Process 4.7 Closing the Loop for the MPA Program 1 4.8 Assessment Action Report for the MPA Program 5.0 COLLEGE OF BUSINESS INTERNAL ASSESSMENT FOR THE MSFE 5.1 Learning Goals for the MSFE Program 5.2 Learning Objectives for the MSFE Program 5.3 Core Courses Aligned with Learning Goals for the MSFE Program 5.4 Methods and Instruments of Measurement for the MSFE Program 5.5 Traits, Standards, and/or Rubrics for the MSFE Program 5.6 Dissemination and Review Process 5.7 Closing the Loop for the MSFE Program 5.8 Assessment Action Report for the MSFE Program 2 1 COLLEGE OF BUSINESS ASSESSMENT PROCESS 1.1 Mission Statement The mission of the College of Business is to provide high quality undergraduate and graduate business education with a global perspective and ethical awareness. Measurement of specific program-level learning goals and objectives occurs in multiple collegelevel undergraduate and graduate courses through both direct and indirect assessment methods. The linkage between the mission and assessment is represented by Figure 1. Figure 1: Linkage between Mission and Assessment 3 1.2 Assessment Management Structure The College of Business at West Texas A&M University is dedicated to delivering quality business programs and the fulfillment of our mission. To this end, a number of processes to assess its mission and programs have been implemented. The assurance of learning process is faculty driven, systematic, and continuous with a variety of feedback loops. This document describes the College’s major processes and its program assessment measures. To provide a framework for continuous improvement, the College has adopted a simple model that considers basic resources and a set of critical processes that lead to improvement and a set of assessments to monitor and evaluate these processes. The College of Business at West Texas A&M University consists of five unique academic programs. They are the BBA, MBA, MPA, MSFE, and the BA/BS in Economics. A faculty responsibility is assurance of learning outcomes to insure that students learn critical skills and acquire knowledge deemed important and appropriate by college faculty members. Representing the collective college faculty, the College Curriculum Committee has the responsibility for reviewing and revising specific program-level learning goals and periodically reviewing these goals to insure currency and relevance to disciplines within the business field and organizations that recruit and hire College of Business graduates. Committee membership includes a committee chairperson, director of continuous improvement, one faculty member from the Accounting, Economics, and Finance Department, one faculty member from the Management, Marketing, and General Business Department, one faculty member from the Information & Decision Management Department, and a maximum of four (minimum of two) additional members. Administrators are accountable for program effectiveness and need to know whether the program is delivering promised learning outcomes. Representing both administration and faculty, the College Strategic Planning & Assessment Committee reviews and provides oversight of assurance of learning information for all degree programs in the College. Committee membership includes a committee chairperson, dean, associate dean, director of continuous improvement, one member representing department heads, one faculty member from the Accounting, Economics, and Finance Department, one faculty member from the Management, Marketing, and General Business Department, one faculty member from the Information & Decision Management Department, and a maximum of two (minimum of zero) additional members. 4 1.3 Curriculum Management Structure The curriculum exists within the framework defined by the mission, goals, and values of West Texas A&M University and the College of Business (COB). The curriculum is informed by the external opportunities and threats faced by the COB and its internal resources and needs. Accordingly, in the design of the curriculum the COB strives to create degrees, concentrations, and courses that respond to its mission. In addition, the COB receives and uses feedback from the assurance of learning process to make recommendations to revise, improve, or terminate existing courses, programs, or concentrations. The stewardship of the curriculum lies with the faculty. Accordingly, the primary vehicle of curriculum changes is faculty members. When a faculty member or group has an idea or proposal, a document or description for departmental discussion and College Curriculum Committee review is prepared. The process encourages the initiator of the proposed change to provide a justification of how the course or program changes fit within the mission and strategy of the COB, is financially viable based on an analysis of the market, and meets the needs of the targeted student group. The Strategic Planning & Assessment Committee facilitates this process through the periodic assignment of task forces related to specific issues or opportunities. Alumni, students, employers, and other stakeholders engage in surveys and discussions that provide external feedback and influence program development and changes. The legislative and juridical body within the curriculum management process is the College Curriculum Committee. The Curriculum Committee receives proposals from the various possible sources (e.g., faculty, task forces, Strategic Planning & Assessment Committee, or other stakeholder groups). The Curriculum Committee also reviews assessment outcomes and generates original curriculum change proposals. The College Curriculum Committee periodically reviews the content of the business courses for compliance with learning goals, mission and accreditation standards. For minor and non-substantive curriculum changes, it provides an immediate determination through vote of committee members. The process for changing the curriculum of a degree program, major, minor, or concentration begins with the faculty of the relevant department or program who gather input from a variety of sources such as students, faculty, alumni, other business programs, employers, alumni, and advisory groups. If the faculty members of the program propose changes, the proposal will be submitted to the College of Business Curriculum Committee for review. Proposals approved by the College Curriculum Committee must be approved internally by the department head, faculty of the College of Business by vote, and dean before submission to the University’s governance system (including University Curriculum Committee, Provost/Vice President for Academic Affairs, University President, and the Texas Higher Education Coordinating Board). Implementation of a curriculum proposal is assigned to the department head charged with leadership of the program in which the curricular element is housed. All curriculum changes are documented in the University catalog and in appropriate supporting fliers, booklets, degree plan checklist, brochures, web pages, and other materials. There are three primary modes of monitoring and evaluation used by the College of Business. 5 First, the faculty, Curriculum Committee, and Strategic Planning & Assessment Committee manage the process of measuring learning goal attainment. Each degree program area has specified goals and a process to monitor and measure goal attainment on a scheduled basis. Likewise, there is a schedule for evaluating, summarizing, and feeding back this information to the College Curriculum Committee. Second, the department heads administer course evaluations in all sections of courses during all semesters. They review the quantitative and qualitative responses, and make staffing changes and curricular recommendations when necessary. Third, the College of Business administration and staff regularly review enrollment in sections of courses for indications of student demand or problems. In addition to these primary sources of information, the College of Business engages in stakeholder surveys, advisory boards, assignments/projects, course embedded assessment, comprehensive examinations, and other activities to gather specific and general information and reactions to the curricula. 1.4 Explicit Curriculum Assessment Methods The College of Business Curriculum Committee, in coordination with the College of Business faculty and administration, are responsible for the curriculum review of all business degree programs. All business degree programs are monitored for effectiveness and outcomes assessment. The College of Business conducts ongoing assessment processes for benchmarking and facilitating continuous improvement. The College uses various methods of assessment to provide feedback on the extent to which the elements covered in the curriculum are being met. Explicit methods include, but are not limited to, the following: Direct Methods: Assignment/Projects with Rubric Application Capstone Business Simulation Comprehensive Examination Course Embedded Assessment Educational Testing Services (ETS) Major Field Achievement Test in Business Learning Portfolio Indirect Methods: Advisory Board Survey Alumni Survey Employer Survey Faculty Survey Graduating Student Survey Stand Alone Methods: CPA Examination Results Student-Managed Investment Fund Performance Students in Free Enterprise Team Production Program Enrollment Trends 6 1.5 Curriculum and Program Review Process The College of Business internal assessment process of all business programs follows a circular flow illustrated below by figure 2. Figure 2: Internal Assessment Process Although all business programs are reviewed on an annual basis and subject to curriculum revision, each program area is extensively reviewed at least once within a three-year period. The focus of the fall semester is a review of assessment results from the previous year, identifying areas of weakness and proposals for improvement, a review of learning goals, a review of rubrics and other instruments, and the identification of courses for spring testing implementation. The focus of the spring semester is primarily on testing for assurance of learning in courses that have been identified by the faculty members. The focus of the summer session is analyzing and 7 reporting assessment results. The summer results are forwarded to the next fall semester and a new assessment cycle begins. Tentative Schedule for an extensive curriculum and assessment review by program and major (three-year rotation) is as follows: BBA Program: Summer 2009 and 2012 MBA Program: Summer 2010 and 2013 MPA and MSFE Programs: Summer 2011 and 2014 1.6 Assurance of Learning Task by Role College of Business Faculty at Large (both undergraduate and graduate faculty) Undertake assessment efforts as an integral portion of teaching activity Participate in meetings to discuss assessment data and offer recommendations for program improvement Develop and implement program improvements Department Heads Meet with the appropriate program faculty to collect and review the assessment data With faculty input, determine what the data mean in light of program/option improvement With faculty input, develop an action plan proposal based on the meaning of the data to the business degree program under review With faculty input, determine appropriate re-assessment follow-up for goals and plans Document the process and decisions made on current goals and send report to dean and director of continuous improvement Review course syllabi for inclusion of appropriate learning goals/objectives on ongoing basis, looking at each syllabi at least once a year Update assessment course alignment matrix as needed The College of Business Strategic Planning & Assessment Committee Review program level assessment data Determine what the data mean in light of the program goals and objectives Recommend program interventions or changes to the Curriculum Committee, dean, and other stakeholders Document recommendations for action plans The College of Business Curriculum Committee Review program level assessment data Evaluate recommendations put forth by the Strategic Planning & Assessment Committee Determine what the data mean in light of the program goals and objectives Recommend curriculum interventions or changes Document recommendations for action plans 8 Director of Continuous Improvement Meet with appropriate stakeholders as needed Record decisions made by Strategic Planning & Assessment Committee, Curriculum Committee, and other stakeholders via meeting minutes and assessment action reports Oversee development of assessment plans and activities in total as well as for each learning goal Collect, analyze, and report data for business degree programs Attend appropriate assessment related conferences and seminars Maintain the knowledge base for all assessment related data, analyses, and resulting reports Dean and Associate Dean: Provide assessment leadership and accountability for all assessment-related activity in the College of Business Schedule and host a minimum of two all faculty meetings (October and February) to discuss assessment data analysis and recommendations for changes to programs offered by the College of Business Provide sufficient financial support for development and implementation of assessment plan Ensure faculty assessment efforts are appropriately reflected in annual teaching and workload agreements Evaluate director of continuous improvement and department heads regarding assessment activity and results Report assessment results to campus and accreditation entities 1.7 Assessment Plan Enhancement The faculty, after reviewing the collected data and the processes used to collect it, may decide to alter the assessment plan. Modifications can be applied to any component of the plan, including the goals, objectives, methods, assessment instruments, or any other aspect of the plan. Substantive changes are to be approved by the faculty of the applicable business degree program. 9 2 COLLEGE OF BUSINESS INTERNAL ASSESSMENT FOR THE BBA 2.1 Learning Goals for the BBA Program The College of Business at West Texas A&M University seeks to prepare students in the Bachelor of Business Administration (BBA) degree program for careers in business and to foster their professional growth and advancement. The learning goals for the program are: Goal 1: Communication: Graduates of the BBA program will be effective communicators. Goal 2: Critical Thinking: Graduates of the BBA program will be critical thinkers. Goal 3: Business Environment: Graduates of the BBA program will be knowledgeable of ethical, global, and social environmental factors and how they relate to business decisions. Goal 4: Functional Business Analyses and Applications: Graduates of the BBA program will be knowledgeable in the functional areas of business and their integration. 10 2.2 Learning Objectives for the BBA Program Goal 1: Communication: Objectives: 1.1 Written Communication: Demonstrate effective writing for business. 1.2 Oral Communication: Demonstrate effective oral communications in business situations. 1.3 Electronic Communication: Demonstrate effective use of technology and multimedia to support effective presentations. Goal 2: Critical Thinking: Objectives: 2.1 Critical Thinking: Demonstrate critical thinking skills by analyzing complex problems and recommending feasible solutions. Goal 3: Business Environment: Objectives: 3.1 Ethical Decisions: Analyze ethical issues when making decisions to achieve individual and organizational responsibilities in business context. 3.2 Global and Domestic Economy: Analyze the implications of global and domestic economic circumstances for effective problem solving and making business decisions. 3.3 Diversity: Analyze the implications and value of ethnic and cultural diversity for effective problem solving and making business decisions. Goal 4: Functional Business Analyses and Applications: Objectives: 4.1 Students will demonstrate a basic understanding of the common body of knowledge in the following business disciplines: a. Accounting b. Economics c. Finance d. Quantitative Business Analysis e. Computer Information Systems f. Business Law g. Management h. Marketing 11 2.3 Core Courses Aligned with Learning Goals for the BBA Program GOAL POSSIBLE CORE COURSES PRIMARY DIRECT ASSESSMENT INSTRUMENT(S) FREQUENCY/START DATE Goal 1: Communication: BUSI 3312 IDM 2342, 3330 MGT 4315 Writing component course Assignments/Projects with evaluation by faculty using standardized rubrics. Review of at least one objective in a rotation every year beginning 2008-2009. FIN 3320 MGT 3330, 4315 Upper-level Econ Course Assignments/Projects with evaluation by faculty using standardized rubrics. Every year beginning 2008-2009. BBA is focus review program for summer of 2009 and 2012. BUSI 3312 CIS 3330 ECON 2301, 2302 FIN 3320 MGT 3330, 4315 MKT 3340 Upper-level Econ Course Assignments/Projects with evaluation by faculty using standardized rubrics. Review of at least one objective in a rotation every year beginning 2008-2009. ACCT 2301, 2302 BUSI 3312 ECON 2301, 2302 FIN 3320 IDM 2342, 3330 MGT 3330, 4315 MKT 3340 (1) Business application in capstone (1) Every year beginning course. 2006-2007. Objectives: 1.1 Written Communication 1.2 Oral Communication 1.3 Electronic Communication Goal 2: Critical Thinking: Objective: 2.1 Critical Thinking Goal 3: Business Environment: Objectives: 3.1 Ethical Decisions 3.2 Global and Domestic Economy 3.3 Diversity Goal 4: Functional Business Analyses and Applications: Objective: 4.1 Common Body of Knowledge (2) Course embedded final exam. (2) Every other year starting 2008-2009. First cycle in 2007-2008 with second cycle 2008-2009. 12 2.4 Methods and Instruments of Measurement for the BBA Program Direct Methods Assignments/Projects—Rubrics created by College of Business faculty are employed in courses as means of assessing learning objectives from course assignments and projects. Frequency—Every year beginning 2008-2009. Actions—First cycle in 2008-2009. Business Application in Capstone Course—Business simulation Capsim, business case competition, and/or Educational Testing Services (ETS) Major Field Achievement Test. Frequency—One or more of the business application tools are administered every semester in the undergraduate business capstone course (MGT 4315). Actions—Initiated in the 2006-2007 Academic Year with a focus on the ETS exam for 2006-2008. The college is experimenting with Capsim and a case competition in 2009. Course Embedded Assessment—The College of Business employs a formal course embedded assessment process for the BBA degree core business knowledge. The assessment instrument and implementation is conducted by the faculty members in each discipline. Frequency—Every other year. Actions—First cycle completed in the 2007-2008 Academic Year with a second cycle initiated for the 2008-2009 Academic Year. The third iteration is scheduled for 20102011. Indirect Methods Graduating Student Survey— Graduating students are surveyed to determine student opinions concerning the appropriateness of the curricula, perceptions of the amount of coverage of important topics, and overall satisfaction with educational experience. Frequency—Annual with EBI surveys employed for odd years (e.g., 2007-2008 Academic Year) and University surveys employed in even years (e.g., 2008-2009). Actions—Rotation of student surveys established in 2006-2007 and remains active. Employer Survey—Employers of College of Business graduates are surveyed to ask their opinions concerning the preparedness of our graduates with respect to important characteristics such as communication skills and other key aspects of the business program. EBI and internally created surveys are used. Frequency—Every three years. Actions—First cycle of EBI employer survey administered April of 2009. Faculty Survey—College of Business faculty are surveyed to determine their perceptions of the business curricula and other key aspects of the business program. Frequency—Every three years as an EBI survey for benchmarking. Actions—2008-2009 EBI survey has been administered with a second survey to be administered in 2011-2012. Ad Hoc Tools—The College of Business employs numerous ad hoc assessment tools, including but not limited to, Student-Managed Investment Fund performance, Students in Free Enterprise annual report, program enrollment trends, and many other tools. Frequency—Every year. Actions—Ad hoc tools are actively employed as part of the assessment program. 13 2.5 Traits, Standards, and/or Rubric for the BBA Program Goal and Objective 1.1: Written Communication Assessment Form (BBA) Course:______________________ Semester:______________ Year: ____________ Student:______________________ Major:_________________ Content Poor Acceptable Excellent Topic is poorly developed. Supporting details are absent or vague. Trite ideas and/or unclear wording reflect lack of understanding of topic and audience. Topic is evident with some supporting details; generally meets requirements of assignment. Topic is well developed, effectively supported and appropriate for the assignment. Effective thinking is clearly and creatively expressed. 1 2 3 Poor Acceptable Excellent Writing is rambling and unfocused, with main theme and supporting details presented in a disorganized, unrelated way. Writing demonstrates some grasp of organization, with a discernible theme and supporting details. Writing is clearly organized around a central theme. Each paragraph is clear and relates to the others in a well-planned framework. 1 2 3 Poor Acceptable Excellent Writing lacks sentence variety. Significant deficiencies in wording, spelling, grammar, punctuation, or presentation. Sources, if consulted, poorly cited. Some sentence variety; adequate usage of wording, grammar, and punctuation. Some cited sources used. Wide variety of sentence structures. Excellent word usage, spelling, grammar and punctuation. Multiple sources correctly cited. Effective integration of information. 1 2 3 Comments: Organization Comments: Language Comments: Total = 14 Goal and Objective 1.2: Oral Communication Assessment Form (BBA) Course:______________________ Semester:______________ Year: ____________ Student:______________________ Major:_________________ Content Poor Acceptable Excellent Topic is poorly developed. Supporting details are absent or vague. Trite ideas and/or unclear wording reflect lack of understanding of topic and audience. Topic is evident with some supporting details; generally meets requirements of assignment. Topic is well developed, effectively supported and appropriate for the assignment. Effective thinking is clearly and creatively expressed. 1 2 3 Poor Acceptable Excellent Speech is rambling and unfocused, with main theme and supporting details presented in a disorganized, unrelated way. Speech demonstrates some grasp of organization, with a discernible theme and supporting details. Speech is clearly organized with effective introduction and conclusion. Each segment relates to the others according to a carefully planned framework. 1 2 3 Poor Acceptable Excellent Speakers appear unpracticed. Use of unnecessary pauses or filler words. Problems with voice control, eye contact, or posture. Incorrect or inappropriate language. Speaker appears proficient with language, vocal, and physical expression. Notes and visuals used as needed. Speaker uses grammatically correct and appropriate language. Delivery is smooth and effective. Good voice control, eye contact, and physical demeanor. Notes and visuals used to enhance the presentation. 1 2 3 Comments: Organization Comments: Delivery Comments: Total = 15 Goal and Objective 1.3: Electronic Communication Assessment Form (BBA) Course:______________________ Semester:______________ Year: ____________ Student:______________________ Major:_________________ Information Processing Poor Acceptable Excellent Fails to demonstrate competency in basic word processing, spreadsheet, and presentation applications. Demonstrates clear evidence of competency in basic word processing, spreadsheet, and presentation applications. Demonstrates thorough competency in use of word processing, spreadsheet, and presentation applications. 1 2 3 Poor Acceptable Excellent Does not know or understand the different components of a database. Knows and understands the different components of a database as well as the logical structures. Knows and understands components of a database, terms associated with data warehouses, and data mining tools. 1 2 3 Poor Acceptable Excellent The student exhibits limited Web search engine skills and requires remediation to perform basic tasks. The student performs basic Web search engine tasks overall and demonstrates adequate application skills. The student demonstrates ability to explore Internet resources, conduct a detailed search for information on the assigned topic, and download necessary files. 1 2 3 Comments: Database Comments: Internet Use Comments: Total = 16 Goal and Objective 2.1: Critical Thinking Assessment Form (BBA) Course:______________________ Semester:______________ Year: ____________ Student:______________________ Major:_________________ Identification Poor Acceptable Excellent Fails to identify and summarize the problem accurately. Summary of issues is mostly accurate but nuances and critical details are glossed over. Clearly identifies and summarizes main problem or issue. Identifies secondary or implicit issues. 1 2 3 Poor Acceptable Excellent Does not distinguish among fact, opinion, and value judgements. Discerns fact from opinion and may recognize bias in evidence. Routine exploration of the issue. Thoroughly examines the evidence and questions its accuracy and completeness. Clear evidence of search, selection, and source evaluation skills. 1 2 3 Poor Acceptable Excellent Quantitative or qualitative analysis conducted is inappropriate, inaccurate, or superficial. Quantitative or qualitative analysis conducted is appropriate and accurate but rather superficial. Quantitative or qualitative analysis conducted is appropriate, accurate, and thorough. 1 2 3 Poor Acceptable Excellent Fails to identify conclusions and implications of the issue or the key relationships between the other elements, such as other perspectives, assumptions, or data and evidence. Conclusions begin to reflect influence of other perspectives, assumptions, and evidence that leads to consequences that extend beyond the borders of a discipline or single issue. Identifies conclusions, implications, and consequences considering assumptions, data, and evidence. Recognizes limitations of correlations and qualifies implications of assertions accordingly. 1 2 3 Comments: Evaluation Comments: Analysis Comments: Decision Comments: Total = 17 Goal and Objective 3.1: Ethical Decision Assessment Form (BBA) Course:______________________ Semester:______________ Year: ____________ Student:______________________ Major:_________________ Identification Poor Acceptable Excellent Fails to identify all major ethical, factual, and conceptual issues/options. Identifies most of the relevant practical issues/options but overlooks some related conceptual problems and issues. Identifies all major issues and stakeholders. Correctly identifies all of the relevant practical options and the related conceptual issues. 1 2 3 Poor Acceptable Excellent Fails to apply moral standards and ethical principles correctly. Applied some moral standards and principles correctly but overlooks or misapplies some ethical principles. Correctly applies the moral standards and ethical principles that are relevant to the case. 1 2 3 Poor Acceptable Excellent Fails to provide weightings of various ethical arguments and reasons. Provides some evaluation of reasons and arguments but overlooks important facts or puts forth minor errors with respect to logical reasoning. Provides appropriate and defensible evaluation for all relevant arguments, notes facts and avoids errors in logical reasoning. 1 2 3 Poor Acceptable Excellent Decision is not justified by the arguments and reasons put forth. Decision put forth is justified by ethical arguments, but failed to identify and respond to reasonable counterarguments. Decision put forth is justified by ethical arguments. Care is taken in responding to reasonable counterarguments. 1 2 3 Comments: Construction Comments: Evaluation Comments: Decision Comments: Total = 18 Goal and Objective 3.2: Global and Domestic Economy Assessment Form (BBA) Course:______________________ Semester:______________ Year: ____________ Student:______________________ Major:_________________ Connectivity Poor Acceptable Excellent Fails to understand that economies of nations impact one another. Understands that economic conditions of one nation can impact those of other nations but is not aware of political/social environmental issues raised by economic interdependence. Understands that economic conditions of one nation impacts those of other nations and is aware of political/social environmental issues raised by economic interdependence. 1 2 3 Poor Acceptable Excellent Fails to understand the impacts of decisions made by national and international organizations. Understands how some specific decisions made by national and international organizations impact the individual, society, and the future. Thoroughly understands how specific decisions made by national and international organizations impact the individual, society, and the future. 1 2 3 Poor Acceptable Excellent Fails to understand the role, impact, and opportunities of the individual in a global society. Understands the role, impact, and opportunities of an individual in a global society. Understands the role, impact, and opportunities of an individual in a global society and is prepared to take personal action. 1 2 3 Poor Acceptable Excellent Unaware of differences in societies’ access to technology and information. Understands some of the ideological and economic issues that influence access to technology and information. Well-developed knowledge of the ideological and economic issues that influence access to technology and information. 1 2 3 Comments: Organizations Comments: Individual Comments: Technology Comments: Total = 19 Goal and Objective 3.3: Diversity Assessment Form (BBA) Course:______________________ Semester:______________ Year: ____________ Student:______________________ Major:_________________ Awareness Poor Acceptable Excellent Student is largely unaware of specific value systems and stereotyped beliefs that contribute to the way he/she and others behave. Student possesses some knowledge of the specific beliefs, values, and sensibilities that contribute to the way he/she and others behave. Student is highly knowledgeable about specific cultural beliefs, values, and sensibilities that might affect the way that he/she and others behave. 1 2 3 Poor Acceptable Excellent Student fails to recognize similarities between his/her own culture and other cultures; student judges cultural differences negatively. Student is cognizant of similarities between his/her own culture and other cultures; student appreciates and accepts cultural differences. Student recognizes that individuals from diverse cultures share some beliefs; student accepts cultural differences and seeks opportunities to learn about different cultures. 1 2 3 Poor Acceptable Excellent Student interacts, communicates, or works poorly with others. Student usually interacts, communicates, or works positively with others. Student interacts, communicates, or works positively with others; student actively seeks opportunities to learn from diverse perspectives. 1 2 3 Poor Acceptable Excellent Student avoids using technology to gain access to individuals or resources from other cultures. Student often uses technology to gain access to individuals or resources from other cultures. Student regularly uses technology to gain access to individuals or resources from other cultures. 1 2 3 Comments: Tolerance Comments: Interaction Comments: Technology Comments: Total = 20 Goal 4: Functional Business Analyses and Applications (BBA): Objectives: Students will demonstrate a basic understanding of the body of knowledge common to the following business disciplines: a. Accounting b. Economics c. Finance d. Quantitative Business Analysis e. Computer Information Systems f. Business Law g. Management h. Marketing Accounting: Students will demonstrate an understanding of the following accounting skills: Financial statements. Basic concepts underlying generally accepted accounting principles. Principles and processes necessary for the implementation of product/service costing systems. Principles and processes necessary for effective planning and control. Analytical tools to facilitate decision-making. Measure: Performance will be assessed by grading a common course embedded multiple-choice and/or essay component on the final exam. Performance Goal: Seventy percent of the students in the course demonstrate characteristics of an acceptable competence level. Acceptable competence is defined as correctly answering seventy percent of the embedded questions on the discipline core component of the course final. 21 Economics: Students will demonstrate an understanding of the following economic skills: Concepts of scarcity and opportunity cost as they relate to economics and business; Causes of economic growth, business cycles, and inflation. Impact and potential limitations of monetary and fiscal policy. Determinants of supply and demand and apply the supply and demand model to illustrate changes in prices and output. Price elasticity of demand and supply. Behavior of the firm as it relates to production, price, cost, and profit with respect to alternative market structures. Welfare effects of alternative market structures. Concepts of comparative advantage and specialization leading to gains from trade. Measure: Performance will be assessed by grading a common course embedded multiple-choice and/or essay component on the final exam. Performance Goal: Seventy percent of the students in the course demonstrate characteristics of an acceptable competence level. Acceptable competence is defined as correctly answering seventy percent of the embedded questions on the discipline core component of the course final. Corporate Finance: Students will demonstrate an understanding of the following finance skills: Role of the finance function of the firm. Financial markets and efficiency. Time value of money. Capital budgeting and the cost of capital. Risk, return, and diversification. Financial statement analysis. Measure: Performance will be assessed by grading a common course embedded multiple-choice and/or essay component on the final exam. Performance Goal: Seventy percent of the students in the course demonstrate characteristics of an acceptable competence level. Acceptable competence is defined as correctly answering seventy percent of the embedded questions on the discipline core component of the course final. 22 Quantitative Business: Students will demonstrate an understanding of the following quantitative business skills: Descriptive statistics, statistical measures, binomial and normal probabilities, spreadsheet construction of graphs, and frequency distributions. Interval estimates and hypothesis testing for one or more population means and proportions. Appropriate hypotheses, perform the test, and interpret the results for analyzing analysis of variance, regression, or time series applications. Spreadsheets in constructing graphs and tables for applications in descriptive statistics. Spreadsheets to derive statistical models that are used in analysis of variance, regression, and time series applications. Measure: Performance will be assessed by grading a common course embedded assignment and/or final exam component. Performance Goal: Seventy percent of the students in the course demonstrate characteristics of an acceptable competence level. Acceptable competence is defined as correctly answering seventy percent of the embedded questions on the discipline core component of the course final. Computer Information Systems: Students will demonstrate an understanding of the following computer information skills: Challenges facing MIS managers. Organizational information systems architectures, information management, and knowledge management frameworks. Business analysis skills using personal productivity and business software tools. Emerging trends and standards as they apply to information systems and technologies. Measure: Performance will be assessed by grading a common course embedded assignment and/or final exam component. Performance Goal: Seventy percent of the students in the course demonstrate characteristics of an acceptable competence level. Acceptable competence is defined as correctly answering seventy percent of the embedded questions on the discipline core component of the course final. 23 Business Law: Students will demonstrate an understanding of the following business law skills: Ethical and legal issues relating to business. Principles and concepts of various laws that affect the many aspects of business. Consequences of unethical and illegal business behavior. Measure: Performance will be assessed by grading a common course embedded multiple-choice and/or essay component on the final exam. Performance Goal: Seventy percent of the students in the course demonstrate characteristics of an acceptable competence level. Acceptable competence is defined as correctly answering seventy percent of the embedded questions on the discipline core component of the course final. Management: Students will demonstrate an understanding of the following management skills: Concepts and principles of management including the basic roles, skills, and functions of management. Historical developments, theoretical aspects, and practice applications of the managerial process. Human issues in the workplace and the impact on organizational effectiveness. Major dimensions used to classify national cultures and how the U.S. compares to selected other cultures. Measure: Performance will be assessed by grading a common course embedded multiple-choice and/or essay component on the final exam. Performance Goal: Seventy percent of the students in the course demonstrate characteristics of an acceptable competence level. Acceptable competence is defined as correctly answering seventy percent of the embedded questions on the discipline core component of the course final. 24 Marketing: Students will demonstrate an understanding of the following marketing skills: Product, prices, place, and promotion within domestic and global marketing activities. The stages of the new product development process. How emerging technologies have changed marketing practices. Marketing communications and their role in marketing strategy. Ethical responsibilities of marketing professionals. Measure: Performance will be assessed by grading a common course embedded multiple-choice and/or essay component on the final exam. Performance Goal: Seventy percent of the students in the course demonstrate characteristics of an acceptable competence level. Acceptable competence is defined as correctly answering seventy percent of the embedded questions on the discipline core component of the course final. 25 2.6 Dissemination and Review Process College of Business faculty members are responsible for implementing and organizing direct assessment efforts. Faculty members are required to report information to the department head by using the standard Course Learning Report Form or equivalent. Direct methods of assessment, including course embedded assessment final examinations and rubric applications, should include multiple faculty members with expertise in the business discipline. The director of continuous improvement, associate dean, and dean are responsible for implementing and organizing indirect methods of assessment. The direct and indirect assessment data is used to ensure compliance with University and College of Business mission statements and goals and to revise courses and curriculum as needed. The following is a timetable for assessment activities during an assessment cycle: Program faculty and/or course instructors collect raw data throughout the academic year (September through June of year one). The department heads and director of continuous improvement will organize and report summary assessment results to the dean and various stakeholders, as appropriate, on each academic program (June of year one). The director of continuous improvement and dean organize the aggregated assessment results into an annual assessment report with supporting data (July of year one). A hard copy of the assessment report is distributed to each department, the Curriculum Committee, the Strategic Planning & Assessment Committee, director of continuous improvement, dean, and provost. In addition, the information is published in an electronic format available to all College of Business stakeholders (August of year one). Strategic Planning & Assessment Committee reviews previous year assessment results and recommends actions consistent with College mission and continuous improvement commitment (September and January of year two). Curriculum Committee reviews previous year assessment results and formulates potential curriculum changes for business degree programs consistent with College mission and continuous improvement commitment (September, October, November, January, February, and April of year two). Undergraduate faculty at large review assessment results and vote with respect to proposed program and curriculum changes (October and February of year two). Graduate faculty at large review assessment results and vote with respect to proposed program and curriculum changes (October and February of year two). Program faculty and/or course instructors collect data in an effort to see if implemented changes actually made a difference (September through June of years two and/or three) Department heads and director of continuous improvement review effectiveness of previous program and curriculum changes by analyzing updated assessment continuation results (June of years two and/or three). Curriculum Committee reviews effectiveness of previous program and curriculum changes by analyzing updated assessment continuation results (June of years two and/or three). 26 Course Learning Report (Sample Form) Department: Course Name and Number: Submission Date: Faculty participants: Course Learning Goal and Objective Evaluated Assessment Instrument Analysis of Findings (include number of students and percent that exceed, meet, and do not meet expectations) Suggestions for Improvement _______________________________________________________ Department Head Signature 27 COLLEGE OF BUSINESS West Texas A&M University ANNUAL REPORT ON THE ASSESSMENT OF STUDENT LEARNING FOR ACADEMIC YEAR (PLACE ACADEMIC YEAR HERE) Academic Program: PLACE ACADEMIC PROGRAM HERE Department: PLACE DEPARTMENT DOING REPORT HERE Submitted by/Date: _______________________________________________________ Please submit this report to the Director of Continuous Improvement PLACE DATE REPORT IS DUE HERE PART I. LEARNING GOALS, OBJECTIVES, STANDARDS, MEASUREMENTS: LEARNING GOALS: LIST LEARNING GOALS HERE LEARNING OBJECTIVES: LIST LEARNING OBJECTIVES FOR EACH LEARNING GOAL HERE STANDARDS: (Example, course embedded questions should have an average pass rate of 70% or higher). LIST MEASUREMENTS: List measurements (embedded questions, metrics, case studies, etc.) for each Learning goal and objective by course used and academic semester administered. Attach this information in an appendix to this report. PART II. DATA COLLECTION: Present the data resulting from the assessment activities using the following formats as an appendix to this report for each learning goal and objective (if assessment was done in more than one course aggregate the results.) (Examples of the tables are attached). Please do not include any faculty or student identification in results to protect confidentiality. Table 1 Learning Objective # ___ Q#1 Q#2 Continue for each Question Avg. # Correct Avg. % Correct Avg. # Correct Answers Avg. % Correct by Question 28 Table 2 Learning Objective # ___ Total # Students # students % correct with correct answers answers Above 25% Above 50% Above 75% Average PART III. ANALYSIS: Please list observations/conclusions for each Learning Goal based on data, e.g., where did the student perform well, poorly, etc. For each observation/conclusion, list your supporting reasons. PART IV. RECOMMENDATION FOR IMPROVEMENT: List changes you plan to make to improve student learning for each Learning Goal, e.g. changes in curriculum, instruction, individual course content, etc. PART V. DOCUMENTATION: The process used to assess, analyze, and implement change must be documented. Attach to this report the pertinent part of minutes of department or curriculum committee meetings detailing the activities in this process and the steps taken for continuous improvement for each Learning Goal. EXAMPLE OF REPORTING TABLES Total Number of Students = 8 Above 25% Above 50% Above 75% Average Table 1 Learning Objective # -# Students with Correct Answers 8 7 3 % Correct Answers 100% 87% 37% 71% 29 SUMMARY REPORT OF ASSURANCE OF LEARNING RESULTS BACHELOR OF BUSINESS ADMINISTRATION A summary of the assessment process for each learning goal is provided based upon reports, which have been prepared as noted. The full report and supporting documents are available in separate volumes entitled College of Business Assessment Report. Learning Goal #1: Graduates of the BBA program will be effective communicators. Learning Objective 1.1: Demonstrate effective writing for business. When Where Assessment By Whom Results Method Dr. Syed 83% of students Spring Anwar’s Standard Dr. Pulliam scored 2009 MKT Grading Dr. Ramos “acceptable or 4348 Rubric Dr. Xiao better” Class Spring 2009 Spring 2009 Dr. Syed Anwar’s MKT 4348 Class Standard Grading Rubric Dr. Furner Dr. Jarmon Dr. Kilic 62% of students scored “acceptable or better” Ms. Jean Walker’s FIN 3350 Class Standard Grading Rubric Dr. Macy Dr. Pjesky Dr. Terry 79% of students scored “acceptable or better” Changes Implemented By whom Reviewed and revised the explicit courses offered in the College of Business writing component for the BBA Curriculum Committee Identified weakness in the overall writing skills of undergraduate business students. Reviewed and revised the explicit courses offered in the College of Business writing component for the BBA. Reviewed and revised the explicit courses offered in the College of Business writing component for the BBA Curriculum Committee Curriculum Committee 30 Learning Goal #1: Graduates of the BBA program will be effective communicators. Learning Objective 1.2: Demonstrate effective oral communication in business situations. When Where Assessment By Whom Results Changes Implemented Method Dr. Syed Given variation in scores in 100% of students Anwar’s Standard Dr. Jafar MKT 4340 versus BUSI scored Spring MKT Grading Dr. Pjesky 3312, the decision was made “acceptable or 2009 4340 Rubric Ms. Upshaw to gain additional data in better” Class spring 2010 assessment cycle. Dr. Gary Given variation in scores in 55% of students Spring Rider’s Standard Dr. Jafar MKT 4340 versus BUSI scored 2009 BUSI Grading Dr. Pjesky 3312, the decision was made “acceptable or 3312 Rubric Ms. Upshaw to gain additional data in better” Class spring 2010 assessment cycle. By whom Director of Continuous Improvment Director of Continuous Improvment 31 Learning Goal #1: Graduates of the BBA program will be effective communicators. Learning Objective 1.3: Demonstrate effective use of technology and multimedia to support effective presentations. When Where Assessment By Whom Results Changes Implemented By whom Method Dr. Rex Initial results positive but AEF faculty area 82% of students Spring Pjesky’s Standard Ms. Cooley need additional data and of responsibility scored 2009 ECON Grading Dr. Macy closer alignment of for AY 2009-2010. “acceptable or 4342 Rubric Dr. Murphy assignments with rubric. better” Class Initial results positive but AEF faculty area 100% of students need additional data and of responsibility All faculty Faculty All faculty scored acceptable Spring closer alignment of for AY 2009-2010 Teaching Graded Teaching IDM or better 2009 assignments with rubric. IDM 2342 Homework 2342 Spring 2009 Spring 2009 Spring 2009 All faculty Teaching IDM 2342 All faculty Teaching IDM 2342 All faculty Teaching IDM 2342 Faculty Graded Homework Faculty Graded Homework Faculty Graded Homework All faculty Teaching IDM 2342 All faculty Teaching IDM 2342 All faculty Teaching IDM 2342 82% of students scored acceptable or better Initial results positive but need additional data and closer alignment of assignments with rubric. AEF faculty area of responsibility for AY 2009-2010 86% of students scored acceptable or better Initial results positive but need additional data and closer alignment of assignments with rubric. AEF faculty area of responsibility for AY 2009-2010 69% of students scored acceptable or better Initial results positive but need additional data and closer alignment of assignments with rubric. AEF faculty area of responsibility for AY 2009-2010 32 Learning Goal #2: Graduates of the BBA program will be critical thinkers. Learning Objective 2.1: Demonstrate critical thinking skills by analyzing complex problems and recommending feasible solutions. When Where Assessment By Whom Results Changes Implemented By whom Method Dr. Neil Given variation in scores in CIS faculty area if 56% of students Spring Terry’s Standard Dr. Anwar ECON 4351 versus IDM responsibility for scored 2009 ECON Grading Dr. Li 2342, the decision was made AY 2009-2010. “acceptable or 4351 Rubric Dr. Racherla to gain additional data in better” Class spring 2010 assessment cycle. Given variation in scores in CIS faculty area if 79% of students ECON 4351 versus IDM responsibility for All faculty Faculty All faculty Spring scored acceptable 2342, the decision was made AY 2009-2010 Teaching Graded Teaching IDM 2009 or better to gain additional data in IDM 2342 Homework 2342 spring 2010 assessment cycle. 33 Learning Goal #3: Graduates of the BBA program will be knowledgeable of ethical, global, and social environmental factors and how they relate to business decisions. Learning Objective 3.1: Analyze ethical issues when making decisions to achieve individual and organizational responsibilities in business context. When Where Assessment By Whom Results Changes Implemented By whom Method Identified a lack of explicit Curriculum coverage of ethical issues for Committee Dr. Neil 76% of students all BBA majors. As a result, Spring Terry’s Standard Dr. Kreissl scored all upper-level economics 2009 ECON Grading Dr. Pulliam, “acceptable or courses will require an 4351 Rubric Dr. Wang better” explicit ethics assignment Class application starting spring 2010. Identified a lack of explicit Curriculum coverage of ethical issues for Committee all BBA majors. As a result, 78% of students all upper-level economics All faculty Faculty All faculty Spring scored acceptable courses will require an Teaching Graded Teaching IDM 2009 or better explicit ethics assignment IDM 3330 Homework 3330 application starting spring 2010. Spring 2009 All faculty Teaching MGT 3330 Faculty Graded Homework All faculty Teaching MGT 3330 67% of students scored acceptable or better Identified a lack of explicit coverage of ethical issues for all BBA majors. As a result, all upper-level economics courses will require an explicit ethics assignment application starting spring 2010. Curriculum Committee 34 Spring 2009 Spring 2009 All faculty Teaching BUSI 3312 All faculty Teaching MKT 3340 Faculty Graded Homework Faculty Graded Homework All faculty Teaching BUSI 3312 All faculty Teaching MKT 3340 64% of students scored acceptable or better 78% of students scored acceptable or better Identified a lack of explicit coverage of ethical issues for all BBA majors. As a result, all upper-level economics courses will require an explicit ethics assignment application starting spring 2010. Identified a lack of explicit coverage of ethical issues for all BBA majors. As a result, all upper-level economics courses will require an explicit ethics assignment application starting spring 2010. Curriculum Committee Curriculum Committee 35 Learning Goal #3: Graduates of the BBA program will be knowledgeable of ethical, global, and social environmental factors and how they relate to business decisions. Learning Objective 3.2: Analyze the implications of global and domestic economic circumstances for effective problem solving and making business decisions. When Where Assessment By Whom Results Changes Implemented By whom Method Identified a lack of explicit Curriculum coverage of global issues for Committee Dr. Rex all BBA majors. As a result, 64% of students Spring Pjesky’s Standard Dr. De’Armond all IDM/CIS 3330 courses scored 2009 ECON Grading Dr. Friske will require an explicit global “acceptable or 4342 Rubric Dr. Shao environment assignment better” Class application starting spring 2010. Spring 2009 Spring 2009 All faculty Teaching ECON 2301 All faculty Teaching ECON 2302 Faculty Graded Homework Faculty Graded Homework All faculty Teaching ECON 2301 All faculty Teaching ECON 2302 64% of students scored acceptable or better 85% of students scored acceptable or better Identified a lack of explicit coverage of global issues for all BBA majors. As a result, all IDM/CIS 3330 courses will require an explicit global environment assignment application starting spring 2010. Identified a lack of explicit coverage of global issues for all BBA majors. As a result, all IDM/CIS 3330 courses will require an explicit global environment assignment application starting spring 2010. Curriculum Committee Curriculum Committee 36 Spring 2009 Spring 2009 Sum. 2009 All faculty Teaching MGT 3330 All faculty Teaching FIN 3320 MGT 4315 Faculty Graded Homework Faculty Graded Homework ETS Exam in International All faculty Teaching MGT 3330 All faculty Teaching FIN 3320 ETS Curriculum Committee 85% of students scored acceptable or better Identified a lack of explicit coverage of global issues for all BBA majors. As a result, all IDM/CIS 3330 courses will require an explicit global environment assignment application starting spring 2010. Curriculum Committee 66% of students scored acceptable or better Identified a lack of explicit coverage of global issues for all BBA majors. As a result, all IDM/CIS 3330 courses will require an explicit global environment assignment application starting spring 2010. Curriculum Committee 65th Percentile Identified a lack of explicit coverage of global issues for all BBA majors. As a result, all IDM/CIS 3330 courses will require an explicit global environment assignment application starting spring 2010. 37 Learning Goal #3: Graduates of the BBA program will be knowledgeable of ethical, global, and social environmental factors and how they relate to business decisions. Learning Objective 3.3: Analyze the implications and value of ethnic and cultural diversity for effective problem solving and making business decisions. When Where Assessment Method By Whom Results Changes Implemented By whom 38 Learning Goal #4: Graduates of the BBA program will be knowledgeable in the functional areas of business and their integration. Learning Objective 4.1: Students will demonstrate a basic understanding of the common body of knowledge in the following business disciplines: Accounting, Economics, Finance, Quantitative Business Analysis, Computer Information Systems, Business Law, Management, and Marketing. When Where Assessment By Whom Results Changes Implemented By whom Method Identified inconsistencies and Accounting faculty Common content problems with course Component, embedded testing instruments multiple for all BBA foundation All choice exam 81% of the courses. Major revisions in Spring Sections administered Accounting questions were accounting. 2009 of ACCT across Faculty answered 2301 sections, correctly embedded in the final Spring 2009 All Sections of ACCT 2302 Common Component, multiple choice exam administered across sections, embedded in the final Accounting Faculty 83% of the questions were answered correctly Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Major revisions in accounting. Accounting faculty 39 Spring 2009 Spring 2009 Spring 2009 All Sections of ECON 2301 All Sections of ECON 2302 All Sections of FIN 3320 Common Component, multiple choice exam administered across sections, embedded in the final Common Component, multiple choice exam administered across sections, embedded in the final Common Component, multiple choice exam administered across sections, embedded in the final Economics Faculty Economics Faculty Finance Faculty 68% of the questions were answered correctly 67% of the questions were answered correctly 68% of the questions were answered correctly Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Minor revisions in economics. Economics faculty Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Minor revisions in economics. Economics faculty Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Minor revisions in finance. Finance faculty 40 Spring 2009 Spring 2009 Spring 2009 All Sections of IDM 2342 All Sections of IDM 3330 All Sections of BUSI 3312 Common Component, multiple choice exam administered across sections, embedded in the final Common Component, multiple choice exam administered across sections, embedded in the final Common Component, multiple choice exam administered across sections, embedded in the final Business statistics faculty members IDM Faculty Faculty members teaching Business Law 83% of the questions were answered correctly 85% of the questions were answered correctly 59% of the questions were answered correctly Identified inconsistencies and CIS faculty content problems with course embedded testing instruments for all BBA foundation courses. Minor revisions in computer information systems (statistics). Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Minor revisions in computer information systems. CIS faculty Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Major revisions in business law. Business law faculty 41 Spring 2009 Spring 2009 Sum. 2009 All Sections of MGT 3330 All Sections of MKT 3340 MGT 4315 Common Component, multiple choice exam administered across sections, embedded in the final Common Component, multiple choice exam administered across sections, embedded in the final ETS Exam in Accounting MGT Faculty MKT Faculty ETS 74% of the questions were answered correctly 82% of the questions were answered correctly 65th Percentile Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Minor revisions in management. Management faculty Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Minor revisions in marketing. Marketing faculty Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Department Heads and Director of Continuous Improvement. 42 Sum. 2009 Sum. 2009 Sum. 2009 Sum. 2009 MGT 4315 MGT 4315 MGT 4315 MGT 4315 ETS Exam in Economics ETS Exam in Finance ETS Exam in Business Stat ETS Exam in Information Systems ETS ETS ETS ETS 85th Percentile 35th Percentile 75th Percentile 25th Percentile Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Department Heads and Director of Continuous Improvement. Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Department Heads and Director of Continuous Improvement. Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Department Heads and Director of Continuous Improvement. Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Department Heads and Director of Continuous Improvement. 43 Sum. 2009 Sum. 2009 Sum. 2009 Sum. 2009 MGT 4315 MGT 4315 MGT 4315 MGT 4315 ETS Exam in Management ETS Exam in Marketing ETS Exam Overall Mean ETS Exam in Overall Median ETS ETS ETS ETS 45th Percentile 65th Percentile 55th Percentile 65th Percentile Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Organize and prepare an internal version of the ETS exam for summer 2010 deployment in MGT 4315 in an effort to obtain applicable results consistent with program learning objectives. Department Heads and Director of Continuous Improvement. Department Heads and Director of Continuous Improvement. Department Heads and Director of Continuous Improvement. Department Heads and Director of Continuous Improvement. 44 2.7 Closing the Loop for the BBA Program The Strategic Planning & Assessment Committee and Curriculum Committee are responsible for reviewing the assessment results, discussing results with various stakeholders, and recommending actions to faculty at large and college administrators. Actions can be anything from concluding that student performance with respect to a learning goal meets expectations to major curriculum change. Other actions may include increasing admission requirements, remediation, adding prerequisites, improvements to the administration and preparation for the ETS or Capsim examinations, increasing or changing specific assignments in existing courses, and providing support structures such as tutoring or help sessions. Another action could be to reevaluate whether the learning goal or expectations for performance on that goal are appropriate. The director of continuous improvement is responsible for completing and updating an assessment action report for the College of Business. The assessment action report includes year of assessment, description of assessment tools administered, areas for improvement identified based on assessment data, action implemented to address weakness, and continuation actions to see if the implemented change actually made a difference. Section 2.8 presents a summary action report for assessment of the BBA program. 45 2.8 Year Assessment Actions Report for BBA Program Areas for Improvement Identified Based on Assessment Data ETS examination in MGT Student scores on the ETS 4315 (capstone course) averaged below the 50th percentile in multiple business fields ETS examination and Reviewed the inclusion of graduating student surveys upper-level economics course in the business core Actions Implemented to Address Weaknesses Continuation Actions (Year and Results) Eliminated the option of BBA degree completion with D’s in the core classes 2008 Course embedded final examinations in business core courses Weak performance by students completing courses with part-time instructors, especially in marketing and management Reduction of part-time instructors employed by the College and proposal to convert part-time instructor funds into one full-time instructor position. 2008 and 2009 EBI Surveys and course embedded examinations of core courses Corporate finance course tested poorly with low student survey results. 2008: Finance faculty modified curriculum to focus key learning concepts across all course sections. 2009: Student scores on the ETS average moved above the 55th percentile in multiple business fields. 2009: ETS score in economics at the 85th percentile. Decision to keep upper-level economics in BBA core and use the discipline as a vehicle for applied assessment in the area of ethics. 2009: Significant improvement on course embedded examinations with reduction of part-time instructors in management, marketing, and computer information systems disciplines. 2009: Corporate finance remains a poor performing discipline on the ETS exam. 2007 2007 Assessment Tools Administered Economics was found to be an area of strength with respect to both student performance and satisfaction 2009: Second review of course materials by finance faculty. 46 2009 2009 2009 Identified inconsistencies and content problems with course embedded testing instruments for all BBA foundation courses. Major revisions in business law, business statistics, and accounting. Minor revisions in economics, finance, marketing, management, and computer information systems. Identified weakness in the overall writing skills of undergraduate business students. Reviewed and revised the explicit courses offered in the College of Business writing component for the BBA. Identified a lack of explicit coverage of ethical issues for all BBA majors. As a result, all upper-level economics courses will require an explicit ethics assignment application starting spring 2010. Discipline specific faculty will create more efficient courses embedded assessment instruments in order to improve quality, efficiency, and usability of assessment results. Improvement of writing objective of the communication goal by ensuring writing component courses in COB core are following explicit standards. Improve student learning of the ethics objectives of goal three. 47 2009 Identified a lack of Improve student learning of explicit coverage of global the global issues objectives issues for all BBA majors. of goal three. As a result, all IDM/CIS 3330 courses will require an explicit global environment assignment application starting spring 2010. 48 3 COLLEGE OF BUSINESS INTERNAL ASSESSMENT FOR THE MBA 3.1 Learning Goals for the MBA Program The College of Business at West Texas A&M University seeks to prepare students in the Master of Business Administration (MBA) degree program for careers in business and to foster their professional growth and advancement via the key learning goals. The key learning goals for the MBA are as follows: Goal 1: Professional Communication: Graduates of the MBA program will communicate effectively as managers. Goal 2: Leadership: Graduates of the MBA program will be leaders. Goal 3: Business Environment: Graduates of the MBA program will be knowledgeable of ethical, global, and social trends affecting business decisions. Goal 4: Business Integration: Graduates of the MBA program will be able to integrate quantitative and qualitative techniques from business functional areas. 49 3.2 Learning Objectives for the MBA Program Goal 1: Professional Communication: Objectives: 1.1 Professional Writing: Demonstrate capacity to employ written presentation channels to effectively communicate with different levels of personnel and meet organizational goals. 1.2 Oral Communication: Demonstrate capacity to employ oral presentation channels to effectively communicate with different levels of personnel and meet organizational goals. Goal 2: Leadership: Objectives: 2.1 Capacity to Lead: Demonstrate understanding of the leadership skills necessary to foster team effectiveness for making decisions. 2.2 Goal Setting: Demonstrate capacity to establish and evaluate organizational goals. Goal 3: Business Environment: Objectives: 3.1 Business Ethics: Demonstrate capacity to recognize and evaluate ethical dimensions of business decisions and the effects on stakeholders. 3.2 Global Business Environment: Demonstrate knowledge of the issues involved in conducting business in a diverse, global environment. Goal 4: Business Integration: Objectives: 4.1 Decision Making: Demonstrate capacity to integrate quantitative and qualitative techniques from functional business areas to analyze business alternatives. 4.2 Strategic Planning: Demonstrate capacity to formulate strategies that are feasible, understandable, and foster long-term sustainability within the context of achieving organizational goals and social responsibilities. 50 3.3 Core Courses Aligned with Learning Goals for the MBA Program GOAL CORE COURSES PRIMARY DIRECT ASSESSMENT INSTRUMENT(S) FREQUENCY/START DATE Goal 1: Professional Communication: ACCT 6305 IDM 6310 MKT 6340 MGT 6334 MGT 6318 (1) Assignments/Projects with evaluation by faculty using standardized rubrics. (1) Every other year beginning 2009-2010. IDM 6310 MKT 6340 MGT 6334 MGT 6318 (1) Assignments/Projects with evaluation by faculty using standardized rubrics. Objectives: 1.1 Professional Writing 1.2 Oral Communication Goal 2: Leadership Objectives: 2.1 Capacity to Lead 2.2 Goal Setting Goal 3: Business Environment: Objectives: 3.1 Business Ethics 3.2 Global Business Environment Goal 4: Business Integration: Objectives: 4.1 Decision Making 4.2 Strategic Planning (2) MBA Learning Portfolio. (2) MBA Learning Portfolio. ACCT 6305 ECON 6305 or 6306 FIN 6320 IDM 6310 MKT 6340 MGT 6318, 6334 International Course (1) Assignments/Projects with evaluation by faculty using standardized rubrics. ACCT 6305 ECON 6305 or 6306 FIN 6320 IDM 6305, 6310 MKT 6340 MGT 6318, 6334 (1) Assignments/Projects with evaluation by faculty using standardized rubrics. (2) MBA Learning Portfolio. (2) MBA Learning Portfolio. (2) Every year beginning fall 2009. (1) Every other year beginning 2008-2009. (2) Every year beginning fall 2009. (1) Every other year beginning 2008-2009. MBA is focus review program for summer of 2010. (2) Every year beginning fall 2009. (1) Every other year beginning 2009-2010. (2) Every year beginning fall 2009. 51 3.4 Methods and Instruments of Measurement for the MBA Program Direct Methods Assignments/Projects—Rubrics created by College of Business faculty are employed in multiple MBA courses as means of assessing professional communication, leadership, business environment, and business integration skills. Assignment and projects are used in multiple courses throughout different stages of the program in order to assess student development. Frequency—Every year with a rotation of learning objectives. The director of continuous improvement, department heads, and faculty members identify the specific objectives and rotation for evaluation at the start of each academic-year. Actions—First cycle of grading rubrics employed in 2009. Business Application in Capstone Course—Business simulation Capsim. Frequency—Applied to one or more classes each academic year in the graduate capstone course. Actions—The College is experimenting with Capsim capstone simulation in 2009. Learning Portfolio—The College of Business employs a learning portfolio for the MBA degree as a capstone event as a means of evaluating student applications of program learning objectives. Frequency—Every year. Actions—Approved in 2008 curriculum-cycle and formally initiated in 2009. Indirect Methods Graduating Student Survey— Graduating students are surveyed to determine student opinions concerning the appropriateness of the curricula, perceptions of the amount of coverage of important topics, and overall satisfaction with various aspects of their educational experience. Frequency—Annual with EBI surveys employed for odd years (e.g., 2007-2008 Academic Year) and University surveys employed in even years (e.g., 2008-2009). Actions—Rotation of student surveys established in 2006-2007 and remains active. Employer Survey—Employers of College of Business graduates are surveyed to ask their opinions concerning the preparedness of our graduates with respect to important characteristics such as communication skills and other key aspects of the business program. EBI and internally created surveys are used. Frequency—Every three years. Actions—2008-2009 EBI survey will be employed with a second survey administered in 2011-2012. Faculty Survey—College of Business faculty are surveyed to determine their perceptions of the business curricula and other key aspects of the business program. EBI surveys are used in order to provide benchmarking. Frequency—Every three years. Actions—2008-2009 EBI survey deployed with a second survey to be administered in 2011-2012. 52 3.5 Traits, Standards, and/or Rubric for the MBA Program Goal and Objective 1.1: Professional Writing Assessment Form (MBA) Course:____________________ TRAIT Executive Summary Introduction Background Analysis of Issue Recommendations Conclusion References Style Semester:____________________ UNACCEPTABLE Topic, organizational issues, and recommendations not presented in a clear, concise, and informative manner. Fails to engage and/or provide the reader with a clear map of the organizational structure of the manuscript. Provides limited overview and background information of the relevant literature. Analysis is incomplete and/or does not apply appropriate theories/concepts/methods. Recommendations encompass less than four of the following five characteristics: relevant, feasible, high quality, considers limitations, and based on solid theoretical foundation. Fails to summarize results and clearly convince reader recommendations are valid. Incomplete with multiple errors. Structure is not easy to follow and significant errors with respect to writing fundamentals. Year:____________________ ACCEPTABLE Topic, organizational issues, and recommendations presented but with minor deficiencies in clarity, conciseness, or content. Moderately engages the reader and provides a clear map to the organizational structure of the manuscript. Provides adequate overview and background information of relevant literature. Provides analysis of chosen issue using most of the appropriate theories/concepts/methods. Recommendations encompass four of the following five characteristics: relevant, feasible, high quality, considers limitations, and based on solid theoretical foundation. Summarizes results and convinces reader recommendations are valid. Complete with minor errors. Structure is easy to follow with minor spelling, typographical, and/or grammatical errors. Student:____________________ EXEMPLARY Topic, organizational issues, and recommendations presented clearly, concisely, and in an informative manner. Engages the reader and provides a clear map to the organizational structure of the manuscript. Provides extensive overview and background information of relevant literature. Thorough analysis of chosen issue using appropriate theories/concepts/methods. Recommendations are relevant, feasible, high quality, considers limitations, and based on solid theoretical foundation. Summarizes results and reinforces validity of recommendations. Complete and accurate. Structure is easy to follow with no spelling, typographical, and/or grammatical errors. 53 Goal and Objective 1.2: Oral Communication Assessment Form (MBA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Organization UNACCEPTABLE Opening statement not relevant. Poor organization and delivery that is lacking focus. ACCEPTABLE Opening statement relevant to topic and provides outline of speech. Mostly organized and loses focus only once or twice. EXEMPLARY Clear opening statement that engages audience and provides outline of speech. Maintains focus and organization. Content Presentation is not current, complete, and/or objective. Appropriate sources are not employed. Presentation is current, complete, objective, and employs appropriate sources. Presentation distinguished with respect to being current, complete, objective, and employing appropriate sources. Summary The summary of key concepts and issues are incomplete or not directly related to the topic. Most of the key concepts and issues in the summary are useful and related to the topic. The summary of key concepts and issues are useful and related to the topic. Delivery Demonstrates one or more of the following: mumbling, improper English, or inappropriate voice modulation. Demonstrates distracting mannerisms, eye contact, and/or poor posture. Most of the presentation is interesting, confident, creative, and clear. Correctly modulates voice. Displays appropriate posture and eye contact without distracting mannerisms. Presentation is interesting, confident, creative, and clear. Correctly modulates voice. Technology, slides, props, and/or handouts do not clearly enhance the presentation. Technology, slides, props, and/or handouts are appropriately employed to enhance presentation Technology, slides, props, and/or handouts are appropriately employed and clearly augment presentation quality and viewer engagement. Mannerisms Use of Media Uses body language effectively to engage audience. 54 Goal and Objective 2.1: Capacity to Lead Assessment Form (MBA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Relating to Group UNACCEPTABLE Works independently but struggles to work with others. ACCEPTABLE Works well with others. EXEMPLARY Encourages, motivates, and works well with others. Ability to Listen Asks for ideas or suggestions but neglects to consider or integrate. Ability to Manage Has an incomplete or inappropriate agenda for the group. Struggles to prevent the group from rambling or straying from agenda. Unable to achieve balance between completing assignments versus the need to focus on interpersonal relations. Considers and respects suggestions of team members when preparing decision resolution. Has a clear agenda for the group. Fosters discussion without excessively losing agenda focus. Respects opinions of team members and makes an effort to integrate multiple perspectives in decision resolution. Circulates a clear agenda to the group in advance. Fosters discussion but facilitates agenda discussion in a timely and efficient fashion. Effectively and efficiently assigns projects to team members. Provides positive reinforcement. Balance Balances the need for assignment completion with the needs for individuals in the group. Decision Decision is not consistent with theoretical foundation and/or does not include multiple perspectives. Makes a decision based on theoretical foundation and multiple perspectives. Decision consistent with best practice, strong theoretical foundation, and reflective of multiple stakeholder input. Understanding Change Has limited understanding and difficulty analyzing change within and outside organizational system. With some limitation, is able to analyze and articulate impact of change within and outside organizational system. Able to analyze and articulate impact of change within and outside organizational system. 55 Goal and Objective 2.2: Goal Setting Assessment Form (MBA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Goal Recommendations UNACCEPTABLE Goals and recommendations lack relevance, clarity, feasibility, or quality. ACCEPTABLE Provides relevant, clear, feasible, and high quality goals and recommendations. EXEMPLARY Provides relevant, clear, feasible, and high quality goals and recommendations. Demonstrates consideration of alternative goals and recommendations as part of the decision process. Measuring Outcomes The plans for measuring outcomes are not detailed and realistic. With some limitation, the plans for measuring outcomes are detailed and realistic. The plans for measuring outcomes are detailed and realistic. Conclusion Goals do not provide tangible value added to the organization. With some limitation, goals provide tangible value added to the organization. Goals provide tangible value added to the organization. Presentation Structure is difficult to follow and there are numerous errors. Structure is easy to follow and errors are minimal. Structure is easy to follow and there are no errors. 56 Goal and Objective 3.1: Business Ethics Assessment Form (MBA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Detection UNACCEPTABLE Little or no recognition of the ethical issue and unable to ascertain what must be decided. ACCEPTABLE Demonstrates ability to recognize the ethical issue and ascertains what must be decided. EXEMPLARY Demonstrates ability to describe the ethical issue in detail and ascertains exactly what must be decided. Considers Stakeholders Inability to describe the legitimate rights and responsibilities of multiple stakeholders within business activities. Demonstrates ability to use one or more ethical frameworks in describing the legitimate rights and responsibilities of multiple stakeholders within business activities. Demonstrates ability to compare and contrast multiple ethical frameworks in describing the legitimate rights and responsibilities of multiple stakeholders within business activities. Chooses an Action Has difficulty identifying appropriate course of action among alternatives. No integration of actions across multiple stakeholders. Formulates an appropriate plan of action that partially integrates multiple stakeholder considerations in the analysis. Formulates an appropriate plan of action reflecting the benefits and risks of action and fully integrates multiple stakeholder considerations in the analysis. 57 Goal and Objective 3.2: Global Business Environment Assessment Form (MBA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Identification of Global Factors UNACCEPTABLE No or incomplete identification of some of the following relevant global business factors: cultural, economic, environmental, historical, legal, political, or social. ACCEPTABLE Identification of most of the relevant global business factors. EXEMPLARY Clear and some detailed identification of relevant global business factors. International Trade Theories Demonstrates rudimentary understanding of international trade theories and applications. Demonstrates understanding of international trade theories and applications. Demonstrates thorough understanding of international trade theories and applications. Analysis of Global Factors Rudimentary analysis of impact of global factors. Some analysis of impact of global business factors with minor inaccuracies in the analysis. Clear, accurate, and detailed analysis of global business factors. Application of Analysis to Management Situation Application of analysis to specific management situations incorrect, unclear, or significantly incomplete. With some limitation, appropriate application of analysis to specific management situations with valid conclusions and recommendations. Clear application of analysis to specific management situations with valid conclusions and recommendations. 58 Goal and Objective 4.1: Decision Making Assessment Form (MBA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Identifies Alternatives UNACCEPTABLE Identifies some relevant alternatives but understanding of the situation is limited. ACCEPTABLE Identifies relevant alternatives consistent with a basic understanding of the situation. EXEMPLARY Identifies relevant alternatives consistent with an in-depth understanding of the situation. Appropriateness Does not employ appropriate models, techniques, theory, and evaluation criteria. Integrates appropriate models, techniques, theories, and evaluation criteria. Integrates the most appropriate models, techniques, theories, and evaluation criteria. Interpretation Misinterprets multiple managerial issues. Demonstrates limited ability to estimate outcomes from alternative decision options. Correctly interprets most managerial issues. Demonstrates ability to estimate outcomes from alternative decision options. Correctly interprets all managerial issues. Demonstrates strong ability to estimate outcomes from alternative decision options. 59 Goal and Objective 4.2: Strategic Planning Assessment Form (MBA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Theoretical Foundation UNACCEPTABLE Does not employ a suitable theoretical framework. ACCEPTABLE Employs an appropriate theoretical framework. EXEMPLARY Employs the most suitable theoretical framework. Critical Elements and Feasibility Incorporates some critical elements for strategy formulation. Considers little or no aspects of feasibility. Incorporates most critical elements for strategy formulation. Considers most major aspects of feasibility. Incorporates all critical elements for strategy formulation. Considers all major aspects of feasibility. Evidence of Strategy List actions or behaviors of Discusses relevance of actions or Discusses nuances of the strategy but does not discuss the behaviors associated with strategic actions in detail. relevance of these actions or strategy. behaviors. Implementation Demonstrates limited awareness of the potential problems associated with the implementation process. Not consistent with organizational goals. Aware of some issues in implementation of change. Consistent with organizational goals. Aware of issues in implementation of change, such as time, budget constraints, delegation of responsibility, rumor control, flexibility to modify, and response from competition. Consistent with organizational goals. Conclusions, Implications, and Consequences Fails to identify conclusions, implications, and consequences of strategy. Identifies and discusses conclusions, implications, and consequences of strategy. Objectively reflects on longterm sustainability of strategy within the context of achieving organizational goals and social responsibilities. 60 3.6 Dissemination and Review Process College of Business faculty members are responsible for implementing and organizing direct assessment efforts. Faculty members are required to report information to the department head by using the standard Course Learning Report Form or equivalent. Direct methods of assessment, including course embedded assessment final examinations and rubric applications, should include multiple faculty members with expertise in the business discipline. The director of continuous improvement, associate dean, and dean are responsible for implementing and organizing indirect methods of assessment. The direct and indirect assessment data is used to ensure compliance with University and College of Business mission statements and goals and to revise courses and curriculum as needed. The following is a timetable for assessment activities during an assessment cycle: Program faculty and/or course instructors collect raw data throughout the academic year (September through June of year one). The department heads and director of continuous improvement will organize and report summary assessment results to the dean and various stakeholders, as appropriate, on each academic program (June of year one). The director of continuous improvement and dean organize the aggregated assessment results into an annual assessment report with supporting data (July of year one). A hard copy of the assessment report is distributed to each department, the Curriculum Committee, the Strategic Planning & Assessment Committee, director of continuous improvement, dean, and provost. In addition, the information is published in an electronic format available to all College of Business stakeholders (August of year one). Strategic Planning & Assessment Committee reviews previous year assessment results and recommends actions consistent with College mission and continuous improvement commitment (September and January of year two). Curriculum Committee reviews previous year assessment results and formulates potential curriculum changes for business degree programs consistent with College mission and continuous improvement commitment (September, October, November, January, February, and April of year two). Undergraduate faculty at large review assessment results and vote with respect to proposed program and curriculum changes (October and February of year two). Graduate faculty at large review assessment results and vote with respect to proposed program and curriculum changes (October and February of year two). Program faculty and/or course instructors collect data in an effort to see if implemented changes actually made a difference (September through June of years two and/or three) Department heads and director of continuous improvement review effectiveness of previous program and curriculum changes by analyzing updated assessment continuation results (June of years two and/or three). Curriculum Committee reviews effectiveness of previous program and curriculum changes by analyzing updated assessment continuation results (June of years two and/or three). 61 Course Learning Report (Sample Form) Department: Course Name and Number: Submission Date: Faculty participants: Course Learning Goal and Objective Evaluated Assessment Instrument Analysis of Findings (include number of students and percent that exceed, meet, and do not meet expectations) Suggestions for Improvement _______________________________________________________ Department Head Signature 62 COLLEGE OF BUSINESS West Texas A&M University ANNUAL REPORT ON THE ASSESSMENT OF STUDENT LEARNING FOR ACADEMIC YEAR (PLACE ACADEMIC YEAR HERE) Academic Program: PLACE ACADEMIC PROGRAM HERE Department: PLACE DEPARTMENT DOING REPORT HERE Submitted by/Date: _______________________________________________________ Please submit this report to the Director of Continuous Improvement PLACE DATE REPORT IS DUE HERE PART I. LEARNING GOALS, OBJECTIVES, STANDARDS, MEASUREMENTS: LEARNING GOALS: LIST LEARNING GOALS HERE LEARNING OBJECTIVES: LIST LEARNING OBJECTIVES FOR EACH LEARNING GOAL HERE STANDARDS: (Example, 80% of the students should have a minimum score of 80% correct answers). LIST MEASUREMENTS: List measurements (embedded questions, metrics, case studies, etc.) for each Learning goal and objective by course used and academic semester administered. Attach this information in an appendix to this report. PART II. DATA COLLECTION: Present the data resulting from the assessment activities using the following formats as an appendix to this report for each learning goal and objective (if assessment was done in more than one course aggregate the results.) (Examples of the tables are attached). Please do not include any faculty or student identification in results to protect confidentiality. Table 1 Learning Objective # ___ Q#1 Q#2 Continue for each Question Avg. # Correct Avg. % Correct Avg. # Correct Answers Avg. % Correct by Question 63 Table 2 Learning Objective # ___ Total # Students # students % correct with correct answers answers Above 25% Above 50% Above 75% Average PART III. ANALYSIS: Please list observations/conclusions for each Learning Goal based on data, e.g., where did the student perform well, poorly, etc. For each observation/conclusion, list your supporting reasons. PART IV. RECOMMENDATION FOR IMPROVEMENT: List changes you plan to make to improve student learning for each Learning Goal, e.g. changes in curriculum, instruction, individual course content, etc. PART V. DOCUMENTATION: The process used to assess, analyze, and implement change must be documented. Attach to this report the pertinent part of minutes of department or curriculum committee meetings detailing the activities in this process and the steps taken for continuous improvement for each Learning Goal. EXAMPLE OF REPORTING TABLES Total Number of Students = 8 Above 25% Above 50% Above 75% Average Table 1 Learning Objective # -# Students with Correct Answers 8 7 3 % Correct Answers 100% 87% 37% 71% 64 SUMMARY REPORT OF ASSURANCE OF LEARNING RESULTS MASTER OF BUSINESS ADMINISTRATION A summary of the assessment process for each learning goal is provided based upon reports, which have been prepared as noted. The full report and supporting documents are available in separate volumes entitled College of Business Assessment Report. Learning Goal #1: Graduates of the MBA program will communicate effectively as managers. Learning Objective 1.1: Demonstrate capacity to employ written presentation channels to effectively communicate with different levels of personnel and meet organizational goals. When Where Assessment By Whom Results Changes Implemented By whom Method Dr. Neil Revised scoring scale, Director of 62% of students Sum. Terry’s Standard Dr. Macy continue assessment of MBA Continuous scored 2009 ECON Grading Dr. Pjesky in 2010, and deploy the MBA Improvement, “acceptable or 6341 Rubric Dr. Terry portfolio course faculty, and MBA better” Class Coordinator Learning Goal #1: Graduates of the MBA program will communicate effectively as managers. Learning Objective 1.2: Demonstrate capacity to employ oral presentation channels to effectively communicate with different levels of personnel and meet organizational goals. When Where Assessment By Whom Results Changes Implemented By whom Method 65 Learning Goal #2: Graduates of the MBA program will be leaders. Learning Objective 2.1: Demonstrate understanding of the leadership skills necessary to foster team effectiveness for making decisions. When Where Assessment By Whom Results Changes Implemented By whom Method Learning Goal #2: Graduates of the MBA program will be leaders. Learning Objective 2.2: Demonstrate capacity to establish and evaluate organizational goals. When Where Assessment By Whom Results Changes Implemented Method By whom 66 Learning Goal #3: Graduates of the MBA program will be knowledgeable of ethical, global, and social trends affecting business decisions. Learning Objective 3.1: Demonstrate capacity to recognize and evaluate ethical dimensions of business decisions and the effects on stakeholders. When Where Assessment By Whom Results Changes Implemented By whom Method Dr. Rex Revised scoring scale, Director of 71% of students Sum. Pjesky’s Standard Dr. Macy continue assessment of MBA Continuous scored 2009 ECON Grading Dr. Pjesky in 2010, and deploy the MBA Improvement, “acceptable or 6341 Rubric Dr. Terry portfolio course faculty, and MBA better” Class Coordinator Learning Goal #3: Graduates of the MBA program will be knowledgeable of ethical, global, and social trends affecting business decisions. Learning Objective 3.2: Demonstrate knowledge of the issue involved in conducting business in a diverse, global environment. When Where Assessment By Whom Results Changes Implemented By whom Method Dr. Neil Revised scoring scale, Director of 87% of students Sum. Terry’s Standard Dr. Macy continue assessment of MBA Continuous scored 2009 ECON Grading Dr. Pjesky in 2010, and deploy the MBA Improvement, “acceptable or 6341 Rubric Dr. Terry portfolio course faculty, and MBA better” Class Coordinator 67 Learning Goal #4: Graduates of the MBA program will be able to integrate quantitative and qualitative techniques from business functional areas. Learning Objective 4.1: Demonstrate capacity to integrate quantitative and qualitative techniques from functional business areas to analyze business alternatives. When Where Assessment By Whom Results Changes Implemented By whom Method Learning Goal #4: Graduates of the MBA program will be able to integrate quantitative and qualitative techniques from business functional areas. Learning Objective 4.2: Demonstrate capacity to formulate strategies that are feasible, understandable, and foster long-term sustainability within the context of achieving organizational goals and social responsibilities. When Where Assessment By Whom Results Changes Implemented By whom Method 68 3.7 Closing the Loop for the MBA Program The Strategic Planning & Assessment Committee and Curriculum Committee are responsible for reviewing the assessment results, discussing results with various stakeholders, and recommending actions to faculty at large and college administrators. Actions can be anything from concluding that student performance with respect to a learning goal meets expectations to major curriculum change. Other actions may include increasing admission requirements, remediation, adding prerequisites, revising the MBA learning portfolio, increasing or changing specific assignments in existing courses, and providing support structures such as tutoring or help sessions. Another action could be to reevaluate whether the learning goal or expectations for performance on that goal are appropriate. The director of continuous improvement is responsible for completing and updating an assessment action report for the College of Business. The assessment action report includes year of assessment, description of assessment tools administered, areas for improvement identified based on assessment data, action implemented to address weakness, and continuation actions to see if the implemented change actually made a difference. Section 3.8 presents a summary action report for assessment of the MBA program. 69 3.8 Year Assessment Actions Report for MBA Program Areas for Improvement Identified Based on Assessment Data Capsim business Limited understanding of the simulation results and EBI global business environment survey of graduates Actions Implemented to Address Weaknesses 2008 Course embedded rubric Unsatisfactory organization of research paper Raise program admission standards. Creation of an MBA portfolio course to capture and assess core skills of students, including multiple writing samples. 2008 EBI survey of graduates Survey results in key categories (e.g., faculty instruction) lower than anticipated Multiple actions: (1) Reduce average class size; (2) Create explicit learning rubrics for the MBA program in an effort to focus course content; (3) Require learning portfolio to reinforce learning objectives. 2008 Assessment Tools Administered Continuation Actions (Year and Results) Addition of an international requirement to the MBA core curriculum. 2009: First set of portfolios collected fall of 2009. 70 2009 Basic application of rubrics in a few classes for a few objectives in order to gain insight on the assessment process and potential problems for MBA. Identified negative bias with the scoring scale employed in applied assignments, which utilized three faculty evaluators. In addition, the importance of the leaning portfolio was realized. As a response, the College created a MBA Coordinator position to help facilitate the portfolio and assessment process. 71 4 COLLEGE OF BUSINESS INTERNAL ASSESSMENT FOR THE MPA 4.1 Learning Goals for the MPA Program The College of Business at West Texas A&M University seeks to prepare students in the Master of Professional Accounting (MPA) degree program for careers in business and to foster their professional growth and advancement via the key learning goals. The key learning goals for the MPA are as follows: Goal 1: Professional Communication: Graduates of the MPA program will be effective communicators. Goal 2: Critical Thinking: Graduates of the MPA program will be analytical thinkers and problem solvers. Goal 3: Business Environment: Graduates of the MPA program will be aware of ethical, global, and social ramifications of business and accounting decisions. Goal 4: Accounting Integration: Graduates of the MPA program will be knowledgeable in accounting and its functional support of organizations. 72 4.2 Learning Objectives for the MPA Program Goal 1: Professional Communication: Objectives: 1.1 Communication: Demonstrate capacity to convey accounting information in both technical and non-technical terms. 1.2 Technology: Demonstrate proficiency in the use of accounting information software packages to process information and computers and/or networks to effectively complete tasks. Goal 2: Critical Thinking: Objectives: 2.1 Critical Thinking: Demonstrate capacity to use analytical tools to analyze complex accounting problems and recommend feasible solutions. Goal 3: Business Environment: Objectives: 3.1 Business Ethics: Demonstrate capacity to recognize and evaluate ethical dimensions of accounting decisions and effects on stakeholders. 3.2 Global Business Environment: Demonstrate knowledge of the issues involved in conducting business in a diverse, global environment. Goal 4: Accounting Integration: Objectives: 4.1 Core Knowledge: Demonstrate knowledge of accounting principles generally accepted in the United States of America (GAAP) for business enterprises, not-for-profit organizations, and governmental entities, and the skills needed to apply that knowledge. 4.2 Professional Standards: Demonstrate understanding of auditing procedures, standards, and rules of professional conduct. 4.3 Application: Demonstrate understanding of tax laws applied to personal and business decisions. 73 4.3 Core Courses Aligned with Learning Goals for the MPA Program GOAL CORE COURSES PRIMARY DIRECT ASSESSMENT INSTRUMENT(S) FREQUENCY/START DATE Goal 1: Professional Communication: ACCT 6306 ACCT 6307 ACCT 6309 ACCT 6310 Assignments/Projects with evaluation by faculty using standardized rubrics. Every other year beginning 2009-2010. MPA is focus review program for summer of 2011 and 2014. ACCT 6306 ACCT 6309 Assignments/Projects with evaluation by faculty using standardized rubrics. Every other year beginning 2008-2009. MPA is focus review program for summer of 2011 and 2014. ACCT 6310 ACCT 6320 ACCT 6321 (1) Assignments/Projects with evaluation by faculty using standardized rubrics. (1) Every other year beginning 2008-2009. Objectives: 1.1 Communication 1.2 Technology Goal 2: Critical Thinking: Objective: 2.1 Critical Thinking Goal 3: Business Environment: Objectives: 3.1 Business Ethics 3.2 Global Business Environment Goal 4: Accounting Integration: Objectives: 4.1 Core Knowledge 4.2 Professional Standards 4.3 Application (2) Comprehensive examination focusing on ethics and global business environment. ACCT 6306 ACCT 6309 ACCT 6310 ACCT 6320 Comprehensive examination focusing on the three accounting integration learning objectives. (2) Comprehensive exam offered every year. Comprehensive exam offered every year. MPA is focus review program for summer of 2011 and 2014. 74 4.4 Methods and Instruments of Measurement for the MPA Program Direct Methods Comprehensive Examinations—The College of Business employs comprehensive examinations for the MPA degree program as the primary assessment tool. Frequency—Every year. Actions—Comprehensive examinations are employed every semester. Assignments/Projects—Rubrics created by College of Business faculty are employed in multiple MPA courses as means of assessing professional communication, critical thinking, business environment, and business integration skills. Assignment and projects are used in multiple courses throughout different stages of the program in order to assess student development. Frequency—Every year with a rotation of learning objectives. The director of continuous improvement, director of the MPA program, department heads, and faculty members identify the specific objectives and rotation for evaluation at the start of each academic-year. Actions—First cycle of grading rubrics employed in 2009. CPA Examination—The College of Business tracks and evaluates CPA pass rates provided by the Texas State Board of Public Accountancy. Frequency—Every year. Actions—The College actively obtains information relevant to exam content and reviews curriculum to determine adequacy of course coverage. Indirect Methods Employer Survey—Employers of College of Business graduates are surveyed to ask their opinions concerning the preparedness of our graduates with respect to important characteristics such as communication skills and other key aspects of the business program. EBI and internally created surveys are used. Frequency—Every three years. Actions—2008-2009 EBI survey will be employed with a second survey administered in 2011-2012. Faculty Survey—College of Business faculty are surveyed to determine their perceptions of the business curricula and other key aspects of the business program. EBI surveys are used in order to provide benchmarking. Frequency—Every three years. Actions—2008-2009 EBI survey deployed with a second survey to be administered in 2011-2012. 75 4.5 Traits, Standards, and/or Rubric for the MPA Program Goal and Objective 1.1: Communication Form (MPA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Deliver an organized and persuasive oral presentation that demonstrates focus, effective visuals, and time management UNACCEPTABLE Message is without clear purpose. Fails to use coherent organizational structure, follow logical progression within ideas, or use appropriate visuals. ACCEPTABLE Message has a clear and appropriate purpose. Uses a clear and coherent organizational structure, follows logical progression within ideas, and employs appropriate visuals. EXEMPLARY Message has an exceptionally clear and appropriate purpose. Uses an exceptionally clear and coherent organizational structure, follows logical progression within ideas, and employs appropriate visuals. Create a clear and organized research paper that employs appropriate tools Composition flawed in the clarity of the report, its organization, or its development. Writing style detracts from clarity. Accomplishes the goals of the assignment. Demonstrates good composition skills, although writing style may suffer minor flaws. Clearly accomplishes the goals of the assignment. Demonstrates excellent composition skills, effective organization, appropriate supporting materials, and effective writing style. Collaborate effectively in a team environment on a course project Collaborates in a one-sided manner. Does not effectively or productively contribute to task completion of the team. Collaborates effectively, making valuable contributions to a group effort and group analysis, but with imbalanced contributions among team members. Collaborates exceptionally well, improving the ability of the group to effectively and efficiently meet goals with in an environment of clear communication and direction. 76 Goal and Objective 1.2: Technology Assessment Form (MPA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Proficient in the use of business and accounting application software packages UNACCEPTABLE May enter data and get output, but only weakly understands the logic of the program and its capabilities. ACCEPTABLE Understands the data needs and basic programming abilities of applications and can use it to generate useful output. EXEMPLARY Thoroughly understands the capabilities and limitations of computer applications and can use it to generate useful output. Ability to select appropriate technology and information processing techniques to solve problems relating to accounting applications Requires significant assistance with the selection of appropriate technology resources to solve problems. Often does not understand appropriate technology techniques to solve problems. With minimal assistance, selects appropriate technology resources to solve problems. Understands appropriate technology techniques to solve problems. Independently and effectively develops strategies to complete task while using appropriate technology resources. Clearly understands appropriate technology techniques to solve problems. Ability to utilize computers and/or networks to effectively analyze and solve problem Analysis is inappropriate with too many errors and/or inconsistencies. Analysis is appropriate with minor errors and/or inconsistencies. Analysis is appropriate with no significant errors and/or inconsistencies. 77 Goal and Objective 2.1: Critical Thinking Assessment Form (MPA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Ability to formulate an effective strategy for solving problems UNACCEPTABLE Develops a strategy, but includes misconceptions about an important tool or framework need to solve the problem. ACCEPTABLE Develops a satisfactory strategy, but one or more key strategy elements related to the problem are missing. EXEMPLARY Develops a complete strategy. Strategy includes all needed key elements to solve the problem. Ability to implement an effective strategy for solving problems Prepares problem analysis that includes fundamental misunderstanding of a tool or framework needed to solve the satisfactorily solve the problem. Prepares analysis needed to solve the problem, but conclusions and/or recommendations lack depth. Synthesizes an effective strategy to determine desired results and devise a coherent plan for solving the problem. Ability to defend and support an appropriate decision Data selection, analysis, and/or conclusions are not objective or logical. Decision is appropriate and defensible. Data selection is appropriate and the analysis is sound but not creative. Decision is appropriate and defensible. Data selection is thorough and thoughtful. Analysis extends beyond replication of textbook examples. 78 Goal and Objective 3.1: Business Ethics Assessment Form (MPA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Detection UNACCEPTABLE Little or no recognition of the ethical issue and unable to ascertain what must be decided. ACCEPTABLE Demonstrates ability to recognize the ethical issue and ascertains what must be decided. EXEMPLARY Demonstrates ability to describe the ethical issue in detail and ascertains exactly what must be decided. Considers Stakeholders Inability to describe the legitimate rights and responsibilities of multiple stakeholders within business activities. Demonstrates ability to use one or more ethical frameworks in describing the legitimate rights and responsibilities of multiple stakeholders within business activities. Chooses an Action Has difficulty identifying appropriate course of action among alternatives. No integration of actions across multiple stakeholders. Formulates an appropriate plan of action that partially integrates multiple stakeholder considerations in the analysis. Demonstrates ability to compare and contrast multiple ethical frameworks in describing the legitimate rights and responsibilities of multiple stakeholders within business activities. Formulates an appropriate plan of action reflecting the benefits and risks of action and fully integrates multiple stakeholder considerations in the analysis. 79 Goal and Objective 3.2: Global Business Environment Assessment Form (MPA) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Identification of Global Factors UNACCEPTABLE No or incomplete identification of some of the following relevant global business factors: cultural, economic, environmental, historical, legal, political, or social. ACCEPTABLE Identification of most of the relevant global business factors. EXEMPLARY Clear and some detailed identification of relevant global business factors. International Trade Theories Demonstrates rudimentary understanding of international trade theories and applications. Demonstrates understanding of international trade theories and applications. Demonstrates thorough understanding of international trade theories and applications. Analysis of Global Factors Rudimentary analysis of impact of global factors. Some analysis of impact of global business factors with minor inaccuracies in the analysis. Clear, accurate, and detailed analysis of global business factors. Application of Analysis to Accounting Application of analysis to specific accounting issues incorrect, unclear, or significantly incomplete. With some limitation, appropriate application of analysis to specific accounting issues with valid conclusions and recommendations. Clear application of analysis to specific accounting issues with valid conclusions and recommendations. 80 MPA Comprehensive Examination Standards for Goals 3 and 4: Measure: The program employs a comprehensive examination covering business ethics, global business environment, core knowledge, professional standards, and application. Performance Goal: Exam sections are evaluated on a scale of unacceptable (no pass), acceptable (pass), and exemplary (high pass). Students must earn an overall score of acceptable or exemplary to complete degree requirements. 81 4.6 Dissemination and Review Process College of Business faculty members are responsible for implementing and organizing direct assessment efforts. Faculty members are required to report information to the department head by using the standard Course Learning Report Form or equivalent. Direct methods of assessment, including course embedded assessment final examinations and rubric applications, should include multiple faculty members with expertise in the business discipline. The director of continuous improvement, associate dean, and dean are responsible for implementing and organizing indirect methods of assessment. The direct and indirect assessment data is used to ensure compliance with University and College of Business mission statements and goals and to revise courses and curriculum as needed. The following is a timetable for assessment activities during an assessment cycle: Program faculty and/or course instructors collect raw data throughout the academic year (September through June of year one). The department heads and director of continuous improvement will organize and report summary assessment results to the dean and various stakeholders, as appropriate, on each academic program (June of year one). The director of continuous improvement and dean organize the aggregated assessment results into an annual assessment report with supporting data (July of year one). A hard copy of the assessment report is distributed to each department, the Curriculum Committee, the Strategic Planning & Assessment Committee, director of continuous improvement, dean, and provost. In addition, the information is published in an electronic format available to all College of Business stakeholders (August of year one). Strategic Planning & Assessment Committee reviews previous year assessment results and recommends actions consistent with College mission and continuous improvement commitment (September and January of year two). Curriculum Committee reviews previous year assessment results and formulates potential curriculum changes for business degree programs consistent with College mission and continuous improvement commitment (September, October, November, January, February, and April of year two). Undergraduate faculty at large review assessment results and vote with respect to proposed program and curriculum changes (October and February of year two). Graduate faculty at large review assessment results and vote with respect to proposed program and curriculum changes (October and February of year two). Program faculty and/or course instructors collect data in an effort to see if implemented changes actually made a difference (September through June of years two and/or three) Department heads and director of continuous improvement review effectiveness of previous program and curriculum changes by analyzing updated assessment continuation results (June of years two and/or three). Curriculum Committee reviews effectiveness of previous program and curriculum changes by analyzing updated assessment continuation results (June of years two and/or three). 82 Course Learning Report (Sample Form) Department: Course Name and Number: Submission Date: Faculty participants: Course Learning Goal and Objective Evaluated Assessment Instrument Analysis of Findings (include number of students and percent that exceed, meet, and do not meet expectations) Suggestions for Improvement _______________________________________________________ Department Head Signature 83 COLLEGE OF BUSINESS West Texas A&M University ANNUAL REPORT ON THE ASSESSMENT OF STUDENT LEARNING FOR ACADEMIC YEAR (PLACE ACADEMIC YEAR HERE) Academic Program: PLACE ACADEMIC PROGRAM HERE Department: PLACE DEPARTMENT DOING REPORT HERE Submitted by/Date: _______________________________________________________ Please submit this report to the Director of Continuous Improvement PLACE DATE REPORT IS DUE HERE PART I. LEARNING GOALS, OBJECTIVES, STANDARDS, MEASUREMENTS: LEARNING GOALS: LIST LEARNING GOALS HERE LEARNING OBJECTIVES: LIST LEARNING OBJECTIVES FOR EACH LEARNING GOAL HERE STANDARDS: (Example, 80% of the students should have a minimum score of 80% correct answers). LIST MEASUREMENTS: List measurements (embedded questions, metrics, case studies, etc.) for each Learning goal and objective by course used and academic semester administered. Attach this information in an appendix to this report. PART II. DATA COLLECTION: Present the data resulting from the assessment activities using the following formats as an appendix to this report for each learning goal and objective (if assessment was done in more than one course aggregate the results.) (Examples of the tables are attached). Please do not include any faculty or student identification in results to protect confidentiality. Table 1 Learning Objective # ___ Q#1 Q#2 Continue for each Question Avg. # Correct Avg. % Correct Avg. # Correct Answers Avg. % Correct by Question 84 Table 2 Learning Objective # ___ Total # Students # students % correct with correct answers answers Above 25% Above 50% Above 75% Average PART III. ANALYSIS: Please list observations/conclusions for each Learning Goal based on data, e.g., where did the student perform well, poorly, etc. For each observation/conclusion, list your supporting reasons. PART IV. RECOMMENDATION FOR IMPROVEMENT: List changes you plan to make to improve student learning for each Learning Goal, e.g. changes in curriculum, instruction, individual course content, etc. PART V. DOCUMENTATION: The process used to assess, analyze, and implement change must be documented. Attach to this report the pertinent part of minutes of department or curriculum committee meetings detailing the activities in this process and the steps taken for continuous improvement for each Learning Goal. EXAMPLE OF REPORTING TABLES Total Number of Students = 8 Above 25% Above 50% Above 75% Average Table 1 Learning Objective # -# Students with Correct Answers 8 7 3 % Correct Answers 100% 87% 37% 71% 85 SUMMARY REPORT OF ASSURANCE OF LEARNING RESULTS MASTER OF PROFESSIONAL ACCOUNTING A summary of the assessment process for each learning goal is provided based upon reports, which have been prepared as noted. The full report and supporting documents are available in separate volumes entitled College of Business Assessment Report. Learning Goal #1: Graduates of the MPA program will be effective communicators. Learning Objective 1.1: Demonstrate capacity to convey accounting information in both technical and non-technical terms. When Where Assessment By Whom Results Changes Implemented By whom Method Learning Goal #1: Graduates of the MPA program will be effective communicators. Learning Objective 1.2: Demonstrate proficiency in the use of accounting information software packages to process information and computers and/or networks to effectively complete tasks. When Where Assessment By Whom Results Changes Implemented By whom Method 86 Learning Goal #2: Graduates of the MPA program will be analytical thinkers and problem solvers. Learning Objective 2.1: Demonstrate capacity to use analytical tools to analyze complex accounting problems and recommend feasible solutions. When Where Assessment By Whom Results Changes Implemented By whom Method Learning Goal #3: Graduates of the MPA program will be aware of ethical, global, and social ramifications of business and accounting decisions. Learning Objective 3.1: Demonstrate capacity to recognize and evaluate ethical dimensions of accounting decisions and effects on stakeholders. When Where Assessment By Whom Results Changes Implemented By whom Method 100% of students MPA Sum. Subjective Accounting scored Accounting faculty Comp. No changes at this time. 2009 Evaluation Faculty “acceptable or Exam better” 87 Learning Goal #3: Graduates of the MPA program will be aware of ethical, global, and social ramifications of business and accounting decisions. Learning Objective 3.2: Demonstrate knowledge of the issues involved in conducting business in a diverse, global environment. When Where Assessment By Whom Results Changes Implemented By whom Method Learning Goal #4: Graduates of the MPA program will be knowledgeable in accounting and its functional support of organizations. Learning Objective 4.1: Demonstrate knowledge of accounting principles generally accepted in the United States of America (GAAP) for business enterprises, not-for-profit organizations, and governmental entities, and the skills needed to apply that knowledge. When Where Assessment By Whom Results Changes Implemented By whom Method Fall 100% of students and MPA Accounting faculty Subjective Accounting scored Spring Comp. No changes at this time Evaluation Faculty “acceptable or 2009 Exam better” 88 Learning Goal #4: Graduates of the MPA program will be knowledgeable in accounting and its functional support of organizations. Learning Objective 4.2: Demonstrate understanding of auditing procedures, standards, and rules of professional conduct. When Where Assessment By Whom Results Changes Implemented By whom Method Fall 100% of students and MPA Accounting faculty Subjective Accounting scored Spring Comp. No changes at this time Evaluation Faculty “acceptable or 2009 Exam better” Learning Goal #4: Graduates of the MPA program will be knowledgeable in accounting and its functional support of organizations. Learning Objective 4.3: Demonstrate understanding of tax laws applied to personal and business decisions. When Where Assessment By Whom Results Changes Implemented Method Fall 100% of students and MPA Subjective Accounting scored Spring Comp. No changes at this time Evaluation Faculty “acceptable or 2009 Exam better” By whom Accounting faculty 89 4.7 Closing the Loop for the MPA Program The Strategic Planning & Assessment Committee and Curriculum Committee are responsible for reviewing the assessment results, discussing results with various stakeholders, and recommending actions to faculty at large and college administrators. Actions can be anything from concluding that student performance with respect to a learning goal meets expectations to major curriculum change. Other actions may include increasing admission requirements, remediation, adding prerequisites, modifying MPA comprehensive examination, increasing or changing specific assignments in existing courses, and providing support structures such as tutoring or help sessions. Another action could be to reevaluate whether the learning goal or expectations for performance on that goal are appropriate. The director of continuous improvement is responsible for completing and updating an assessment action report for the College of Business. The assessment action report includes year of assessment, description of assessment tools administered, areas for improvement identified based on assessment data, action implemented to address weakness, and continuation actions to see if the implemented change actually made a difference. Section 4.8 presents a summary action report for assessment of the MPA program. 90 4.8 Assessment Actions Report for MPA Program Year Assessment Tools Administered Areas for Improvement Identified Based on Assessment Data Pass rate in multiple categories below state average Actions Implemented to Address Weaknesses Continuation Actions (Year and Results) 2007 CPA Pass Rate Review and revision of accounting curriculum and course content in 2007 2008-2009 CPA pass rates moved above the state average for first two quarters but fell below state average the last two quarters. 2008 Program Review for CPA Exam Lack of depth in the coverage of accounting ethics issues Created an ethics course dedicated to accounting issues instead of employing ethics course derived from management A new Ethical Solutions in Professional Accounting (ACCT 6321) course added in 2008 and offered Spring of 2009. 2009 Consider revision of the CPA review course offered in the MPA program Pass rate on the CPA exam 91 5 COLLEGE OF BUSINESS INTERNAL ASSESSMENT FOR THE MSFE 5.1 Learning Goals for the MSFE Program The College of Business at West Texas A&M University seeks to prepare students in the Master of Science in Finance and Economics (MSFE) degree program for careers in business and to foster their professional growth and advancement via the key learning goals. The key learning goals for the MSFE are as follows: Goal 1: Professional Communication: Graduates of the MSFE program will be effective communicators. Goal 2: Critical Thinking: Graduates of the MSFE program will be analytical thinkers and problem solvers. Goal 3: Business Environment: Graduates of the MSFE program will be aware of ethical, global, and social ramifications of business and economic decisions. Goal 4: Finance and Economics Integration: Graduates of the MSFE program will be knowledgeable of advanced theoretical and applied concepts in economics and finance. 92 5.2 Learning Objectives for the MSFE Program Goal 1: Professional Communication: Objectives: 1.1 Writing: Demonstrate capacity to employ written presentation channels to communicate finance and economics research results. 1.2 Oral Communication: Demonstrate capacity to employ oral presentation channels to communicate finance and economics research results. Goal 2: Critical Thinking: Objectives: 2.1 Critical Thinking: Demonstrate capacity to use finance and economic analytical tools to analyze complex problems and recommend feasible solutions. Goal 3: Business Environment: Objectives: 3.1 Business Ethics: Demonstrate capacity to recognize and evaluate ethical dimensions of financial decisions and effects on stakeholders. 3.2 Global Business Environment: Demonstrate knowledge of the issues involved in conducting business in a diverse, global environment. Goal 4: Finance and Economics Integration: Objectives: 4.1 Theoretical Integration: Demonstrate knowledge and understanding of advanced theoretical concepts in microeconomics, macroeconomics, and financial management. 4.2 Quantitative Integration: Demonstrate knowledge and understanding of advanced quantitative and statistical skills to solve finance and economic problems. 93 5.3 Core Courses Aligned with Learning Goals for the MSFE Program GOAL CORE COURSES PRIMARY DIRECT ASSESSMENT INSTRUMENT(S) FREQUENCY/START DATE Goal 1: Professional Communication: FIN 5321 FIN 6322 FIN 6326 Assignments/Projects with evaluation by faculty using standardized rubrics. Every other year beginning 2009-2010. MSFE is focus review program for summer of 2011 and 2014. ECON 6312 FIN 6330 FIN 6350 Assignments/Projects with evaluation by faculty using standardized rubrics. Every other year beginning 2008-2009. MSFE is focus review program for summer of 2011 and 2014. ECON 6305 ECON 6306 ECON 6341 FIN 5311 FIN 5323 FIN 6370 (1) Assignments/Projects with evaluation by faculty using standardized rubrics. (1) Every other year beginning 2008-2009. ECON 6305 ECON 6306 FIN 5321 FIN 6320 FIN 6322 FIN 6370 IDM 6320 Comprehensive examination focusing on theoretical and quantitative integration of finance and economics. Objectives: 1.1 Writing 1.2 Oral Communication Goal 2: Critical Thinking: Objective: 2.1 Critical Thinking Goal 3: Business Environment: Objectives: 3.1 Business Ethics 3.2 Global Business Environment Goal 4: Finance and Economics Integration: Objectives: 4.1 Theoretical Integration 4.2 Quantitative Integration (2) Comprehensive examination focusing on ethics and global business environment. (2) Comprehensive exam offered every year. Comprehensive exam offered every year. MSFE is focus review program for summer of 2011 and 2014. 94 5.4 Methods and Instruments of Measurement for the MSFE Program Direct Methods Comprehensive Examinations—The College of Business employs comprehensive examinations for the MSFE degree program as the primary assessment tool. Frequency—Every year. Actions—Comprehensive examinations have been employed since the creation of the MSFE degree. Assignments/Projects—Rubrics created by College of Business faculty are employed in multiple MSFE courses as means of assessing professional communication, critical thinking, business environment, and business integration skills. Assignments and projects are used in multiple courses throughout different stages of the program in order to assess student development. Frequency—Every year with a rotation of learning objectives. The director of continuous improvement, department heads, and faculty members identify the specific objectives and rotation for evaluation at the start of each academic-year. Actions—First cycle of grading rubrics employed in 2009. Indirect Methods Employer Survey—Employers of College of Business graduates are surveyed to ask their opinions concerning the preparedness of our graduates with respect to important characteristics such as communication skills and other key aspects of the business program. EBI and internally created surveys are used. Frequency—Every three years. Actions—2008-2009 EBI survey will be employed with a second survey administered in 2011-2012. Faculty Survey—College of Business faculty are surveyed to determine their perceptions of the business curricula and other key aspects of the business program. EBI surveys are used in order to provide benchmarking. Frequency—Every three years. Actions—2008-2009 EBI survey deployed with a second survey to be administered in 2011-2012. 95 5.5 Traits, Standards, and/or Rubric for the MSFE Program Goal and Objective 1.1: Writing Assessment Form (MSFE) Course:____________________ TRAIT Executive Summary Introduction Background Analysis of Issue Recommendations Conclusion References Style Semester:____________________ UNACCEPTABLE Topic, organizational issues, and recommendations not presented in a clear, concise, and informative manner. Fails to engage and/or provide the reader with a clear map of the organizational structure of the manuscript. Provides limited overview and background information of the relevant literature. Analysis is incomplete and/or does not apply appropriate theories/concepts/methods. Recommendations encompass less than four of the following five characteristics: relevant, feasible, high quality, considers limitations, and based on solid theoretical foundation. Fails to summarize results and clearly convince reader recommendations are valid. Incomplete with multiple errors. Structure is not easy to follow and significant errors with respect to writing fundamentals. Year:____________________ ACCEPTABLE Topic, organizational issues, and recommendations presented but with minor deficiencies in clarity, conciseness, or content. Moderately engages the reader and provides a clear map to the organizational structure of the manuscript. Provides adequate overview and background information of relevant literature. Provides analysis of chosen issue using most of the appropriate theories/concepts/methods. Recommendations encompass four of the following five characteristics: relevant, feasible, high quality, considers limitations, and based on solid theoretical foundation. Summarizes results and convinces reader recommendations are valid. Complete with minor errors. Structure is easy to follow with minor spelling, typographical, and/or grammatical errors. Student:____________________ EXEMPLARY Topic, organizational issues, and recommendations presented clearly, concisely, and in an informative manner. Engages the reader and provides a clear map to the organizational structure of the manuscript. Provides extensive overview and background information of relevant literature. Thorough analysis of chosen issue using appropriate theories/concepts/methods. Recommendations are relevant, feasible, high quality, considers limitations, and based on solid theoretical foundation. Summarizes results and reinforces validity of recommendations. Complete and accurate. Structure is easy to follow with no spelling, typographical, and/or grammatical errors. 96 Goal and Objective 1.2: Oral Communication Assessment Form (MSFE) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Organization UNACCEPTABLE Opening statement not relevant. Poor organization and delivery that is lacking focus. ACCEPTABLE Opening statement relevant to topic and provides outline of speech. Mostly organized and loses focus only once or twice. EXEMPLARY Clear opening statement that engages audience and provides outline of speech. Maintains focus and organization. Content Presentation is not current, complete, and/or objective. Appropriate sources are not employed. Presentation is current, complete, objective, and employs appropriate sources. Presentation distinguished with respect to being current, complete, objective, and employing appropriate sources. Summary The summary of key concepts and issues are incomplete or not directly related to the topic. Most of the key concepts and issues in the summary are useful and related to the topic. The summary of key concepts and issues are useful and related to the topic. Delivery Demonstrates one or more of the following: mumbling, improper English, or inappropriate voice modulation. Demonstrates distracting mannerisms, eye contact, and/or poor posture. Most of the presentation is interesting, confident, creative, and clear. Correctly modulates voice. Displays appropriate posture and eye contact without distracting mannerisms. Presentation is interesting, confident, creative, and clear. Correctly modulates voice. Technology, slides, props, and/or handouts do not clearly enhance the presentation. Technology, slides, props, and/or handouts are appropriately employed to enhance presentation Technology, slides, props, and/or handouts are appropriately employed and clearly augment presentation quality and viewer engagement. Mannerisms Use of Media Uses body language effectively to engage audience. 97 Goal and Objective 2.1: Critical Thinking Assessment Form (MSFE) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Ability to formulate an effective strategy for solving problems UNACCEPTABLE Develops a strategy, but includes misconceptions about an important tool or framework need to solve the problem. ACCEPTABLE Develops a satisfactory strategy, but one or more key strategy elements related to the problem are missing. EXEMPLARY Develops a complete strategy. Strategy includes all needed key elements to solve the problem. Ability to implement an effective strategy for solving problems Prepares problem analysis that includes fundamental misunderstanding of a tool or framework needed to solve the satisfactorily solve the problem. Prepares analysis needed to solve the problem, but conclusions and/or recommendations lack depth. Synthesizes an effective strategy to determine desired results and devise a coherent plan for solving the problem. Ability to defend and support an appropriate decision Data selection, analysis, and/or conclusions are not objective or logical. Decision is appropriate and defensible. Data selection is appropriate and the analysis is sound but not creative. Decision is appropriate and defensible. Data selection is thorough and thoughtful. Analysis extends beyond replication of textbook examples. 98 Goal and Objective 3.1: Business Ethics Assessment Form (MSFE) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Detection UNACCEPTABLE Little or no recognition of the ethical issue and unable to ascertain what must be decided. ACCEPTABLE Demonstrates ability to recognize the ethical issue and ascertains what must be decided. EXEMPLARY Demonstrates ability to describe the ethical issue in detail and ascertains exactly what must be decided. Considers Stakeholders Inability to describe the legitimate rights and responsibilities of multiple stakeholders within business activities. Demonstrates ability to use one or more ethical frameworks in describing the legitimate rights and responsibilities of multiple stakeholders within business activities. Chooses an Action Has difficulty identifying appropriate course of action among alternatives. No integration of actions across multiple stakeholders. Formulates an appropriate plan of action that partially integrates multiple stakeholder considerations in the analysis. Demonstrates ability to compare and contrast multiple ethical frameworks in describing the legitimate rights and responsibilities of multiple stakeholders within business activities. Formulates an appropriate plan of action reflecting the benefits and risks of action and fully integrates multiple stakeholder considerations in the analysis. 99 Goal and Objective 3.2: Global Business Environment Assessment Form (MSFE) Course:____________________ Semester:____________________ Year:____________________ Student:____________________ TRAIT Identification of Global Factors UNACCEPTABLE No or incomplete identification of some of the following relevant global business factors: cultural, economic, environmental, historical, legal, political, or social. ACCEPTABLE Identification of most of the relevant global business factors. EXEMPLARY Clear and some detailed identification of relevant global business factors. International Trade Theories Demonstrates rudimentary understanding of international trade theories and applications. Demonstrates understanding of international trade theories and applications. Demonstrates thorough understanding of international trade theories and applications. Analysis of Global Factors Rudimentary analysis of impact of global factors. Some analysis of impact of global business factors with minor inaccuracies in the analysis. Clear, accurate, and detailed analysis of global business factors. Application of Analysis to Finance and Economics Application of analysis to specific finance and economic issues incorrect, unclear, or significantly incomplete. With some limitation, appropriate application of analysis to specific finance and economic issues with valid conclusions and recommendations. Clear application of analysis to specific finance and economic issues with valid conclusions and recommendations. 100 MSFE Comprehensive Examination Standards for Goals 3 and 4: Measure: The program employs a comprehensive examination covering business ethics, global business environment, theoretical integration of core concepts, and quantitative integration of core concepts. Performance Goal: Exam sections are evaluated on a scale of unacceptable (no pass), acceptable (pass), and exemplary (high pass). Students must earn an overall score of acceptable or exemplary to complete degree requirements. 101 5.6 Dissemination and Review Process College of Business faculty members are responsible for implementing and organizing direct assessment efforts. Faculty members are required to report information to the department head by using the standard Course Learning Report Form or equivalent. Direct methods of assessment, including course embedded assessment final examinations and rubric applications, should include multiple faculty members with expertise in the business discipline. The director of continuous improvement, associate dean, and dean are responsible for implementing and organizing indirect methods of assessment. The direct and indirect assessment data is used to ensure compliance with University and College of Business mission statements and goals and to revise courses and curriculum as needed. The following is a timetable for assessment activities during an assessment cycle: Program faculty and/or course instructors collect raw data throughout the academic year (September through June of year one). The department heads and director of continuous improvement will organize and report summary assessment results to the dean and various stakeholders, as appropriate, on each academic program (June of year one). The director of continuous improvement and dean organize the aggregated assessment results into an annual assessment report with supporting data (July of year one). A hard copy of the assessment report is distributed to each department, the Curriculum Committee, the Strategic Planning & Assessment Committee, director of continuous improvement, dean, and provost. In addition, the information is published in an electronic format available to all College of Business stakeholders (August of year one). Strategic Planning & Assessment Committee reviews previous year assessment results and recommends actions consistent with College mission and continuous improvement commitment (September and January of year two). Curriculum Committee reviews previous year assessment results and formulates potential curriculum changes for business degree programs consistent with College mission and continuous improvement commitment (September, October, November, January, February, and April of year two). Undergraduate faculty at large review assessment results and vote with respect to proposed program and curriculum changes (October and February of year two). Graduate faculty at large review assessment results and vote with respect to proposed program and curriculum changes (October and February of year two). Program faculty and/or course instructors collect data in an effort to see if implemented changes actually made a difference (September through June of years two and/or three) Department heads and director of continuous improvement review effectiveness of previous program and curriculum changes by analyzing updated assessment continuation results (June of years two and/or three). Curriculum Committee reviews effectiveness of previous program and curriculum changes by analyzing updated assessment continuation results (June of years two and/or three). 102 Course Learning Report (Sample Form) Department: Course Name and Number: Submission Date: Faculty participants: Course Learning Goal and Objective Evaluated Assessment Instrument Analysis of Findings (include number of students and percent that exceed, meet, and do not meet expectations) Suggestions for Improvement _______________________________________________________ Department Head Signature 103 COLLEGE OF BUSINESS West Texas A&M University ANNUAL REPORT ON THE ASSESSMENT OF STUDENT LEARNING FOR ACADEMIC YEAR (PLACE ACADEMIC YEAR HERE) Academic Program: PLACE ACADEMIC PROGRAM HERE Department: PLACE DEPARTMENT DOING REPORT HERE Submitted by/Date: _______________________________________________________ Please submit this report to the Director of Continuous Improvement PLACE DATE REPORT IS DUE HERE PART I. LEARNING GOALS, OBJECTIVES, STANDARDS, MEASUREMENTS: LEARNING GOALS: LIST LEARNING GOALS HERE LEARNING OBJECTIVES: LIST LEARNING OBJECTIVES FOR EACH LEARNING GOAL HERE STANDARDS: (Example, 80% of the students should have a minimum score of 80% correct answers). LIST MEASUREMENTS: List measurements (embedded questions, metrics, case studies, etc.) for each Learning goal and objective by course used and academic semester administered. Attach this information in an appendix to this report. PART II. DATA COLLECTION: Present the data resulting from the assessment activities using the following formats as an appendix to this report for each learning goal and objective (if assessment was done in more than one course aggregate the results.) (Examples of the tables are attached). Please do not include any faculty or student identification in results to protect confidentiality. Table 1 Learning Objective # ___ Q#1 Q#2 Continue for each Question Avg. # Correct Avg. % Correct Avg. # Correct Answers Avg. % Correct by Question 104 Table 2 Learning Objective # ___ Total # Students # students % correct with correct answers answers Above 25% Above 50% Above 75% Average PART III. ANALYSIS: Please list observations/conclusions for each Learning Goal based on data, e.g., where did the student perform well, poorly, etc. For each observation/conclusion, list your supporting reasons. PART IV. RECOMMENDATION FOR IMPROVEMENT: List changes you plan to make to improve student learning for each Learning Goal, e.g. changes in curriculum, instruction, individual course content, etc. PART V. DOCUMENTATION: The process used to assess, analyze, and implement change must be documented. Attach to this report the pertinent part of minutes of department or curriculum committee meetings detailing the activities in this process and the steps taken for continuous improvement for each Learning Goal. EXAMPLE OF REPORTING TABLES Total Number of Students = 8 Above 25% Above 50% Above 75% Average Table 1 Learning Objective # -# Students with Correct Answers 8 7 3 % Correct Answers 100% 87% 37% 71% 105 SUMMARY REPORT OF ASSURANCE OF LEARNING RESULTS MASTER OF SCIENCE IN FINANCE AND ECONOMICS A summary of the assessment process for each learning goal is provided based upon reports, which have been prepared as noted. The full report and supporting documents are available in separate volumes entitled College of Business Assessment Report. Learning Goal #1: Graduates of the MSFE program will be effective communicators. Learning Objective 1.1: Demonstrate capacity to employ written presentation channels to communicate finance and economics research results. When Where Assessment By Whom Results Changes Implemented By whom Method Learning Goal #1: Graduates of the MSFE program will be effective communicators. Learning Objective 1.2: Demonstrate capacity to use finance and economic analytical tools to analyze complex problems and recommend feasible solutions. When Where Assessment By Whom Results Changes Implemented By whom Method 106 Learning Goal #2: Graduates of the MSFE program will be analytical thinkers and problem solvers. Learning Objective 2.1: Demonstrate capacity to use finance and economic analytical tools to analyze complex problems and recommend feasible solutions. When Where Assessment By Whom Results Changes Implemented By whom Method Dr. Neil 100% of students Sum. Terry’s Standard Dr. Macy Economics and scored 2009 ECON Grading Dr. Pjesky No changes at this time finance graduate “acceptable or 6341 Rubric Dr. Terry faculty better” Class Learning Goal #3: Graduates of the MSFE program will be aware of ethical, global, and social ramifications of business and economic decisions. Learning Objective 3.1: Demonstrate capacity to recognize and evaluate ethical dimensions of financial decisions and effects on stakeholders. When Where Assessment By Whom Results Changes Implemented By whom Method Fall 100% of students and MSFE Economics and Subjective Economics and scored Spring Comp. No changes at this time finance graduate Evaluation Finance Faculty “acceptable or 2009 Exam faculty better” 107 Learning Goal #3: Graduates of the MSFE program will be aware of ethical, global, and social ramifications of business and economic decisions. Learning Objective 3.2: Demonstrate knowledge of the issues involved in conducting business in a diverse, global environment. When Where Assessment By Whom Results Changes Implemented By whom Method Reviewed and revised MSFE Economics and examination process, which finance graduate Fall 91% of students included the creation of faculty. and MSFE Subjective Economics and scored explicit examination dates Spring Comp. Evaluation Finance Faculty “acceptable or with questions that are 2009 Exam better” directly tied to program learning goals. 108 Learning Goal #4: Graduates of the MSFE program will be knowledgeable of advanced theoretical and applied concepts in economics and finance. Learning Objective 4.1: Demonstrate knowledge and understanding of advanced theoretical concepts in microeconomics, macroeconomics, and financial management. When Where Assessment By Whom Results Changes Implemented By whom Method Reviewed and revised MSFE Economics and examination process, which finance graduate Fall 82% of students included the creation of faculty. and MSFE Subjective Economics and scored explicit examination dates Spring Comp. Evaluation Finance Faculty “acceptable or with questions that are 2009 Exam better” directly tied to program learning goals. 109 Learning Goal #4: Learning Goal #4: Graduates of the MSFE program will be knowledgeable of advanced theoretical and applied concepts in economics and finance. Learning Objective 4.2: Demonstrate knowledge and understanding of advanced quantitative and statistical skills to solve finance and economic problems. When Where Assessment By Whom Results Changes Implemented By whom Method Reviewed and revised MSFE Economics and examination process, which finance graduate Fall 82% of students included the creation of faculty. and MSFE Subjective Economics and scored explicit examination dates Spring Comp. Evaluation Finance Faculty “acceptable or with questions that are 2009 Exam better” directly tied to program learning goals. 110 5.7 Closing the Loop for the MSFE Program The Strategic Planning & Assessment Committee and Curriculum Committee are responsible for reviewing the assessment results, discussing results with various stakeholders, and recommending actions to faculty at large and college administrators. Actions can be anything from concluding that student performance with respect to a learning goal meets expectations to major curriculum change. Other actions may include increasing admission requirements, remediation, adding prerequisites, modifying MSFE comprehensive examination, increasing or changing specific assignments in existing courses, and providing support structures such as tutoring or help sessions. Another action could be to reevaluate whether the learning goal or expectations for performance on that goal are appropriate. The director of continuous improvement is responsible for completing and updating an assessment action report for the College of Business. The assessment action report includes year of assessment, description of assessment tools administered, areas for improvement identified based on assessment data, action implemented to address weakness, and continuation actions to see if the implemented change actually made a difference. Section 5.8 presents a summary action report for assessment of the MSFE program. 111 5.8 Assessment Actions Report for MSFE Program Year Assessment Tools Administered Areas for Improvement Identified Based on Assessment Data Marginal student performance on international section of exam (scores of five no pass, two pass, and two high pass on first attempt on international section on comprehensive exam) Actions Implemented to Address Weaknesses Continuation Actions (Year and Results) 2007 MSFE Comprehensive Examination Review and revision of MSFE curriculum and course content in 2007 2008 scores improve (two no pass, six pass, and four high pass) 2008 Stakeholder (alumni) Survey Results Lack of depth in the coverage of corporate governance and ethics issues Created a corporate governance course for the MSFE program and hired new faculty member with research area in field The new corporate governance course added in 2008 and offered Fall of 2009 2009 MSFE Comprehensive Examination Reviewed and revised MSFE examination process, which included the creation of explicit examination dates with questions that are directly tied to program learning goals. 112