Accreditation Council for Business Schools and Programs (ACBSP) Quality Assurance Report, 2008-2010 Accreditation Council for Business Schools and Programs (ACBSP) Associate Degree Commission Accredited Institutions Quality Assurance (QA) Report (Rev H – December 2009) Institution Name: Roane State Community College Address: Date December 7, 2010______________ 276 Patton Lane, Harriman, TN 37748 Year Accredited/Reaffirmed: 2004 This Report Covers Years: 2008 – 2010 List All Accredited Programs (as they appear in your catalog): 1. A.A.S. Business Management Technology Program Accounting Option Business Administration Option Business Management Option Computer Science Option E-Commerce Option General Business Option Office Administration Option (Terminated, Effective Fall 2010) 2. A.A.S. Contemporary Management Program Quality Assurance Report, 2008-2010 Review Page 2 List all campuses that a student can earn a business degree from your institution: o Roane County (Main campus), Oak Ridge Branch Campus, Campbell County Center, Cumberland County Center, Fentress County Center, Loudon County Center, Morgan County Center, and Scott County Center . * Though not all classes are offered on-ground at each of Roane State’s campuses/centers, students are able to supplement the on-ground courses with courses taught via distant education. Person completing report: Mrs. Gloria Drake, Chairperson of Business Department Phone: (865) 354-3000 Ext 4332 E-mail address: drakega@roanestate.edu ACBSP Champion name: Dr. Diane Ward, Dean of Social Science, Business, and Education Department ACBSP Co-Champion name: Mrs. Gloria Drake, Chairperson of Business Department A. Faculty Qualifications 1. Complete the following tables for new full-time and part-time faculty members only since last Report: o TABLE VI New Full-time and Part-time Faculty Qualifications. During this QAR reporting period (2008-2010), 99.48% of the full-time equivalent (FTE) faculty were Master's or Doctorate degree qualified. Only .52% of the FTE faculty was considered Master's or Doctorate degree Exception; thus, remaining within the 10% exception rate allowed. B. Curriculum 1. List any existing accredited associate degree programs/curricula that have been substantially revised since your last report and attach a Table VII – Curriculum Summary for each program. o None. Existing programs have not been substantially revised since our last QAR. 2. List any new degree programs that have been developed and attach a Table VII – Curriculum Summary for each new program since your last report. o AAS Business Management Technology – General Business Option -- The General Business option was approved by Roane State’s Academic Council during the Spring 2009 semester and became effective Fall 2009. The curriculum for this new option is shown in Table VII and complies with the 25% standard on the professional component, general education, and business major. Quality Assurance Report, 2008-2010 Review Page 3 3. List any accredited programs that have been terminated since your last report. o AAS Business Management Technology – Office Administration Option – Due to low enrollment, the Office Administration option was approved for termination by Roane State’s Academic Council and the Tennessee Board of Regents during the Spring 2010 semester with a phase-out period ending Spring 2011. Because this termination was approved during the academic years for this QAR (2008-2010), it is noted here. Students who are currently enrolled in this option are being assisted by the Office Administration instructor to plan a schedule for completion. However, any student who may not complete this degree option prior to the end of this phase-out period will be encouraged to follow the new General Business option which will accept up to 30 previous credit hours from the OAD program in addition to the common core classes already in place for both options. This will eliminate the loss of credit hours toward the same Business Management Technology degree. Each student will be assisted by an advisor within the Business Department to help make this transition seamless. In addition, if a student wishes to pursue additional Office Administration courses, he or she may choose to attend one of our local Tennessee Technology Centers. C. Organization 1. List any organizational or administrative personnel changes within the business unit since your last report. o Dr. Diane Ward, Dean of Social Science, Business, and Education Department: Since our last QAR, we have undergone an internal administrative change which we feel will have a positive effect on faculty and staff within the Department as well as on ACBSP. Our previous dean, Dr. Chris Whaley, was promoted internally to Vice President of Student Learning, a key position for faculty, staff, and student support. Dr. Diane Ward, previously a program director and the assistant dean within our current division, was then promoted internally to become our dean. Both Dr. Ward and Dr. Whaley are very supportive of ACBSP and realize the positive impact accreditation has on our programs. They are both true assets to our department and we foresee no negative implications with this internal change in personnel. In addition, the makeup of the business department itself remains unchanged. The Chairperson of the Business Department since January 2001, Mrs. Gloria Drake, continues to guide the business unit. (Appendix A: Organizational Chart, pg 50) 2. List all new sites where students can earn an accredited business degree (off-campus or on campus, online) that have been added since your last report? o None. No new sites have not been added since our last QAR. Quality Assurance Report, 2008-2010 Review Page 4 D. Conditions/Notes/Opportunity for Improvement (OFI) to be Addressed (Either for accreditation or from feedback by commissioners for the Quality Assurance Report) Please explain and provide the necessary documentation/evidence for addressing each condition/note/OFI since your last report. o Based on the last QAR submitted in 2008, there were three comments and/or Opportunities for Improvement. Each is addressed below: 1. Commissioner’s Comment: “Major organizational changes but not effective during this reporting cycle. Business Department dissolved. Now they are called the Social Science, Business, and Education Department. What implications might this have for their future support for ACBSP?” As indicated above, since our last QAR, we have undergone an internal administrative change which we feel will have a positive effect on faculty and staff within the Department as well as on ACBSP. Our previous dean, Dr. Chris Whaley, was promoted internally to Vice President of Student Learning, a key position for faculty, staff, and student support. Dr. Diane Ward, previously a program director and the assistant dean within our current division, was then promoted internally to become our dean. Both Dr. Ward and Dr. Whaley are very supportive of ACBSP and realize the positive impact accreditation has on our programs. They are both true assets to our department and we foresee no negative implications with this internal change in personnel. In addition, the makeup of the business department itself remains unchanged. The Chairperson of the Business Department since January 2001, Mrs. Gloria Drake, continues to guide the business unit. 2. Commissioner’s Comment: “While there are Performance Measures for Business Management Technology as stated on p. 9 of the QA report, the five areas of concentration (Accounting, Business Administration, Business Management, Computer Technology, and Office Administration) lack specific performance measures. While the five stated are perhaps core program outcomes, to determine assessment of student learning within the five concentrations, performance measures would provide valuable data to improve the teaching/learning process.” Since receiving this OFI, we have further developed our overall Business Management Technology program outcomes to include student learning outcomes specific to each option (area of concentration) including Accounting, Business Administration, Business Management, Computer Science, ECommerce, and General Business. The chart on the following pages includes common program outcomes as well as outcomes specific to the individual program options (concentrations). Quality Assurance Report, 2008-2010 Review Page 5 3. Commissioner’s Comment: “Manning and Standridge are listed as exceptions. Does this put Roane State over the 10% allowed for exceptions?” Having both Mrs. Glenna Manning and Mr. Jamie Standridge listed as an exception calculated to 4.6% of the allotted 10% exception rate, thus, not going over the allowed range. However, since the submission of the last QAR, we learned from Dr. Larry Zachrich of ACBSP that an MBA serves as Master’s Qualified for all business-related courses. Following that understanding, we had only a 3.5 percent exception rate. Quality Assurance Report, 2008-2010 Review Page 6 E. Program Outcomes: List program outcomes for each accredited program. The following chart indicates both common program outcomes as well as outcomes specific to individual program options (concentrations). A.A.S. Business Management Technology Program A.A.S. Contemporary Management Program Accounting Option Business Admin Option Business Mgt Option Computer Science Option E-Commerce Option General Business Option Communicate effectively in business situations through physical or virtual presence in writing, speaking, listening, and electronic media Communicate effectively in business situations through physical or virtual presence in writing, speaking, listening, and electronic media Communicate effectively in business situations through physical or virtual presence in writing, speaking, listening, and electronic media Communicate effectively in business situations through physical or virtual presence in writing, speaking, listening, and electronic media Communicate effectively in business situations through physical or virtual presence in writing, speaking, listening, and electronic media Communicate effectively in business situations through physical or virtual presence in writing, speaking, listening, and electronic media Communicate effectively in business situations through physical or virtual presence in writing, speaking, listening, and electronic media. Achieve the core educational knowledge and skills to succeed in the 21st century. Achieve the core educational knowledge and skills to succeed in the 21st century. Achieve the core educational knowledge and skills to succeed in the 21st century. Achieve the core educational knowledge and skills to succeed in the 21st century. Achieve the core educational knowledge and skills to succeed in the 21st century. Achieve the core educational knowledge and skills to succeed in the 21st century. Achieve the core educational knowledge and skills including critical thinking to succeed in the 21st century. Develop skills of critical thinking for identifying problems and selecting appropriate means for problem solutions. Develop skills of critical thinking for identifying problems and selecting appropriate means for problem solutions. Develop skills of critical thinking for identifying problems and selecting appropriate means for problem solutions. Develop skills of critical thinking for identifying problems and selecting appropriate means for problem solutions. Develop skills of critical thinking for identifying problems and selecting appropriate means for problem solutions. Develop skills of critical thinking for identifying problems and selecting appropriate means for problem solutions. Demonstrate the knowledge and skills needed to function as a manager/supervisor in a business environment. Quality Assurance Report, 2008-2010 Review Page 7 Develop specific skills, Develop specific skills, Develop specific skills, Develop specific skills, Develop specific skills, Develop specific skills, Lead by using team competencies, ethical competencies, ethical competencies, ethical competencies, ethical competencies, ethical competencies, ethical building skills and behavior, and points behavior, and points behavior, and points behavior, and points behavior, and points behavior, and points facilitating of view needed for of view needed for of view needed for of view needed for of view needed for of view needed for collaborative employment in employment in employment in employment in employment in employment in behaviors in the marketing, marketing, marketing, marketing, marketing, marketing, accomplishment of management, management, management, management, management, management, group goals and accounting, computer accounting, computer accounting, computer accounting, computer accounting, computer accounting, computer objectives. science, or office science, or office science, or office science, or office science, or office science, or office administration. administration. administration. administration. administration. administration. Use and adapt current Use and adapt current industry standard industry standard technologies that are technologies that are applicable to the applicable to the student's specific area student's specific area of study. of study. Use and adapt current industry standard technologies that are applicable to the student's specific area of study. Use and adapt current industry standard technologies that are applicable to the student's specific area of study. Use and adapt current industry standard technologies that are applicable to the student's specific area of study. Demonstrate a working knowledge of terminology common to the accounting environment. Demonstrate a working knowledge of terminology common to the small business environment. Demonstrate a working knowledge of terminology common to the business management environment. Demonstrate a working knowledge of terminology common to the business technology environment. Demonstrate a working knowledge of terminology common to the e-commerce environment. Accounting students will demonstrate knowledge of the accounting cycle. Demonstrate the knowledge and skills necessary to operate and manage a small business. Demonstrate the knowledge and skills of managerial functions. Demonstrate the ability to apply computer software to create business documents, spreadsheets, databases, and presentations. Demonstrate the skills necessary to build and manage websites for commerce. Demonstrate the ability to apply appropriate new technologies to business problems. Demonstrate the knowledge and skills of the selling process related to ecommerce. Accounting students Demonstrate the Demonstrate the will demonstrate knowledge and skills ability to apply proficiency in use of of the selling process business ethics theory commercial computer to real-world business accounting systems. scenarios. Quality Assurance Report, 2008-2010 Review Use and adapt current industry standard technologies that are applicable to the student's specific area of study. Interpret and analyze accounting information for internal control, planning, performance evaluation, and coordination to continuously improve business processes. Recognize and evaluate potentials of entrepreneurial opportunities for new business ventures Page 8 F. Performance Results Table I Student Learning Results (Required for each accredited program) Performance Indicator 1. Student Learning Results Definition (Required for each accredited program) A student learning outcome is one that measures a specific competency attainment. Examples of a direct assessment (evidence) of student learning attainment that might be used include: capstone performance, third-party examination, faculty-designed examination, professional performance, licensure examination). To help students succeed, community colleges must both assess skills and remediate deficiencies before students take more than 25 percent of the credits in business programs. Add these to the description of the measurement instrument in column two: Formative – An assessment conducted during the student’s education. Summative – An assessment conducted at the end of the student’s education. Internal – An assessment instrument that was developed within the business unit. External – An assessment instrument that was developed outside the business unit. Comparative – Compare results between classes, between online and on ground classes, Between professors, between programs, between campuses, or compare to external results such as results from the U.S. Department of Education Research and Statistics, or results from a vendor providing comparable data. Analysis of Results Performance Measure (Competency) 1. The average score for graduates Description of Measurement Instrument to include Formative, summative, internal, external, or comparative. All graduates of the Business Management Quality Assurance Report, 2008-2010 Review Areas of Success Though the benchmark was met Analysis and Action Taken Each year, the faculty review the results of Results of Action Taken (occurs in the following year) Insert Graph of Resulting Trends for 3-5 Years (please graph all available data up to five years) The faculty will continue to track scores for specific Page 9 taking the Business Graduate Exam (Major Field Test) will be 70 percent or higher. Source: Momentum BGE Technology and the Contemporary Management programs take the Business Graduate Exam (BGE). The 100 question summative examination was developed by business faculty and is based on the core business courses. For BMT majors, the core includes "Introduction to Business" (Bus 101), "Principles of Accounting I" (Bus 221), ―Business Math" (Bus 111), "Management and Supervision" (Bus 281) and "Technology Essentials" (COLL 1020).Comparable classes are taken in the Contemporary Management program. Summative and Internal. Quality Assurance Report, 2008-2010 Review during the Fall 2008 semester, the scores declined slightly over the next two semesters. (Graph 1.01: BGE Class Averages) In comparison to the previous semesters, the scores are consistent. An analysis of the test results for each of the five core business courses are shown in Graphs 1.02, 1.03, 1.04, 1.05, and 1.06. *Note: The data from Fall 2009 became corrupted upon export. Data not available. the BGE, identifying specific areas in which a concerning percentage of students may be struggling. Data is reported for each core area as well as each question within each area so faculty can clearly see the number of students who missed particular questions. Appendix B (pages 51-52) indentifies questions in which 50% or more of the students missed, using Fall 2008 as the baseline for missed questions. Faculty are asked to focus on objectives to improve retention in the identified questions to determine if additional improvements can be made in the wording or in the construction of questions, while still addressing the same subject matter. Since currently there is no incentive or repercussion for students to do well on this exit test, faculty are considering developing a capstone course for all majors in which the BGE will be administered as part of the student's grade. Page 10 areas. Based on the findings, in Fall 2010, the business faculty determined that because Contemporary Management majors are not required to take the Business Math (BUS 111) course, those questions would be removed from the exam, changing less than 20 percent of the overall test. Quality Assurance Report, 2008-2010 Review Page 11 2. Based on results from the ETS Proficiency Profile (College general education exam), business graduates will demonstrate proficiency in the General Education core competencies established by the College and the Tennessee Board of Regents All RSCC graduates are required to complete the Proficiency Profile (previously named MAPP – Measure of Academic Progress and Proficiency) which measures the competencies of the General Education core. Summative, External, Comparative Source: Office of Institutional Research& Effectiveness Quality Assurance Report, 2008-2010 Review For the past four academic years, 100 percent of RSCC graduates surpassed the national norms for the total score on the Proficiency Profile. (Graph 1.07) When comparing scores in specific areas of the Proficiency Profile, the business students scored very close to that of all RSCC students. (Graph 1.08) Because the AAS degrees have fewer general education courses in the curriculums, it would be a normal consequence that the scores would be lower. However, the faculty will continue to monitor the student success rate, particularly in the areas of critical thinking, reading, writing, and math and will take appropriate action as needed. No significant action is required at this time but faculty will continue to monitor. Page 12 3. The average score for the BMT students taking the comprehensive posttest at the end of BUS 101 (Introduction to Business) will be 70% or higher. Faculty teaching Bus 101 established a pre/ post test to assess student learning based on the learning outcomes set for this course. Every student completing the course must complete this test. Source: Faculty and Momentum Formative, Internal Quality Assurance Report, 2008-2010 Review For academic years 20082010, the students exceeded the benchmark with an average overall student score of 80%. (Graph 1.09) During the previous 2006– 2008 years, the average overall score was 84%, indicating a slight decrease in overall scores for 2008-2010. The Bus 101 faculty completed a test item analysis, revealing some areas of item difficulty. These questions were modified to improve quality of the assessment and to improve the index of difficulty. Though the result of the overall scores did not significantly improve, the average was 80%, exceeding the benchmark. The faculty will continue to analyze individual competency areas for improvements in student understanding. In addition, in Fall 2010, the faculty agreed to use a consistent percentage of a student’s overall grade for this posttest. Page 13 4. The average score for the BMT students taking the comprehensive posttest at the end of BUS 111 (Business Math) will be 70% or higher. Source: Faculty and Momentum Faculty teaching Bus 111 established a pre/ post test to assess student learning based on the learning outcomes set for this course. Every student completing the course must complete this test. For the academic years Fall, 2008 through Spring, 2010, the students averaged 74% on the posttest which is lower than the 79% for the previous two years. (Graph 1.11) Formative, Internal Quality Assurance Report, 2008-2010 Review One possible reason for a decrease in posttest scores could be the increase in the number of students who wish to take Business Math as a web course. Though the learning outcomes are the same, many students struggle with this subject. Grade comparisons of on-ground classes to web classes will be monitored and analyzed. The faculty will continue to focus attention on key areas in which students show less than desirable success. In addition, faculty will continue to tweak test questions for improved wordings to enhance students understanding. Page 14 5. The average score for the BMT students taking the comprehensive posttest at the end of BUS 221 (Principles of Accounting) will be 70% or higher Source: Faculty and Momentum Faculty teaching Bus 221 established a pre/ post test to assess student learning based on the learning outcomes set for this course. Every student completing the course must complete this test. The average posttest score for the past two years was 68.25% which is below the 70% benchmark. However, this is up from 67.25% for the previous reporting period. (Graph 1.13) Formative, Internal Quality Assurance Report, 2008-2010 Review The overall averages for 2008-2010 varied only slightly from the previous reporting period, 20062008. The faculty analyzed each question in which a significant number of students missed. These questions were minimally "tweaked" to improve wording and/or student understanding. The faculty will continue to analyze individual questions to determine which areas need more coverage in class to improve student understanding. In addition, in Fall 2010, the faculty agreed to use a consistent percentage of a student’s overall grade for this posttest. Page 15 6. The average score for the BMT students taking the comprehensive posttest at the end of BUS 281 (Management and Supervision) will be 70% or higher Faculty teaching Bus 282 established a pre/ post test to assess student learning based on the learning outcomes set for this course. Every student completing the course must complete this test. Source: Faculty and Momentum Formative, Internal For the past several years, posttest scores have consistently exceeded the benchmark. (Graph 1.15) However, during the 2008-10 reporting period, the average scores decreased slightly from 82.66 to 78.5. Quality Assurance Report, 2008-2010 Review The faculty reviewed each question on the posttest, focusing special attention on the questions in which a significant number of students missed. There were two questions selected for additional study both in the area of improved wording and emphasis in class. Both of these questions related to management theories (Theory Y and Equity Theory). The faculty will continue to analyze individual questions to determine which areas need more coverage in class to improve student understanding. In addition, in Fall 2010, the faculty agreed to use a consistent percentage of a student’s overall grade for this posttest. Page 16 7. The average score for students completing a project on "Investing" in BUS 101 (Introduction to Business) will be 80% or higher. Source: Faculty This project includes investing and the study of financial markets using an imaginary $100,000 to invest in the global stock markets over a 14 week semester. This project grade is not based on the students’ profits; it is based on their activity in investing. The students’ profit range changes each semester based on the economy and investing choices. The average grade for the past two years was 81, which exceed the benchmark set. (Graph 1.18) Faculty will provide additional support to assist students with quality investment choices. Instructors will continue to expand this project to give the students even more insight into global economies. . Formative, Internal Quality Assurance Report, 2008-2010 Review Page 17 8. The average score for students completing a Business Plan will be 80% or higher. Source: Faculty This comprehensive business plan is completed as a team project and covers areas such product, price, place, and promotion in addition to many other elements of business. All students enrolled in BUS 278 (Entrepreneurs hip) and MGT 200 (Foundations of Business) complete this project. For the past two years, the average score was for BUS 278 students was 82.25%, exceeding the benchmark. For MGT 200 students, the average was 85.75, also exceeding the benchmark. When comparing scores in different classes, an anecdotal assumption was the higher overall average in MGT 200 was due in part to the large number of adult students taking that class who often have a broader sense of how a business operates. Faculty will continue to provide students with guidance while working both individually and in teams and continue to track data. Formative, Internal Quality Assurance Report, 2008-2010 Review Page 18 9. The average grade for students completing a Negotiation Plan will be 80% or higher. Source: Faculty Each student enrolled in BUS 236 (Negotiations and Conflict Resolution) completes a negotiation plan and participates in a face-to-face negotiation scenario. The average student grade over the last four semesters was 84.25%, exceeding the benchmark set. Though the average student grades were good, faculty will continue to alter the scenarios as needed for variety and exposure to new situations. The faculty will continue to improve the negotiation cases to be assigned and the student's negotiation techniques. Formative and Internal Quality Assurance Report, 2008-2010 Review Page 19 10. The average grade for students completing a Financial Analysis of a publicly traded company will be 80% or higher. Source: Faculty This comprehensive analysis is completed by all students enrolled in MGT 207 (Managerial Accounting). This project is done as a team. Over the past five semesters, students have demonstrated success with an average grade of 85.4%, exceeding the benchmark. Based on feedback from the students completing this project, faculty will make improvements as needed. The faculty will continue to provide support to students both individually and as a team. Formative, Internal Quality Assurance Report, 2008-2010 Review Page 20 Table II Student and Stakeholder-Focused Results Student- and stakeholder-focused results examine how well your organization satisfies students and stakeholders key needs and expectations. Key indicators may include: satisfaction and dissatisfaction of current and past students and key stakeholders, perceived value, loyalty, persistence, or other aspects of relationship building, end of course surveys, alumni surveys, Internship feedback, etc. 2. Student- and StakeholderFocused Results Each academic unit must demonstrate linkages to business practitioners and organizations, which are current and significant, including an advisory board. Periodic surveys should be made of graduates, transfer institutions, and/or employer s of graduates to obtain data on the success of business programs in preparing students to compete successfully for entry-level positions. Performance Measure (Competency ) Description of Measurement Instrument 1. Current RSCC students will express satisfaction with their overall experience at RSCC with a rating of 80% or higher. The Community College Survey of Student Engagement (CCSSE) is used to conduct a survey of current students every three years. The last survey was conducted in Spring 2009. Source: Office of Institutional Research & Effectiveness . Analysis of Results Areas of Analysis and Success Action Taken The students overall satisfaction rating was 3.2 out of 4 or 81.75% (Graph 2.01). This exceeds the satisfaction rating for the TN Consortium Colleges as well as our National Peers. Quality Assurance Report, 2008-2010 Review Results for specific majors are not available. However, Appendix C (Graphs 2.02 – 2-05, pg. 53) indicates success rates for selected questions that would relate to BMT and CMGT subject areas. Results of Action Taken (occurs in the following year) Insert Graph of Resulting Trends for 3-5 Years (please graph all available data up to five years) Faculty will continue providing the support and assistance needed to students. Page 21 2. At least 90% of Alumni graduates will express satisfaction with their overall academic program. Source: Office of Institutional Research & Effectiveness The Office of Institutional Research & Effectiveness conducts a survey of alumni on a three-year cycle. The last survey was conducted in Spring 2010. Quality Assurance Report, 2008-2010 Review In Spring 2010, No specific 96.2% of all RSCC action is needed alumni were at this time. "satisfied" to "very satisfied" with their Educational Experience (Graph 2.06). Alumni from the BMT and CMGT rated their experience as 100% ―satisfied to very satisfied‖. (Graph 2.07) In comparison to the state, RSCC alumni rated their satisfaction level the same as other TN public colleges (3.52 out of 4.0). (Graph 2.08) Additional success rates are indicated in Appendix D, pages 54-55) Faculty will continue to monitor the results of future surveys. Page 22 3. At least 90% of the employers surveyed will express satisfaction with the work performance of graduates. In addition, a benchmark of 85% of the respondent employers would hire RSCC graduates in the future. The Office of Institutional Research conducts a survey of area employers every five years. The survey was administered in the Spring, 2008 semester. The next survey will be conducted in Spring 2013. RSCC uses a 40% response rate as a benchmark for the Institution and 31.1% responded. Source: Office of Institutional Research & Effectiveness Quality Assurance Report, 2008-2010 Review Of the employers surveyed in Spring 2008, 83.4% expressed an overall satisfaction rating of "satisfied" to "very satisfied" which was below the benchmark of 90%. However, 91.1% of the employers that responded indicated they would hire future Roane State graduates which is higher than the benchmark. The survey did not break responses down by major. The chart to the right shows the top ten selected Competency Areas, the Rank of Importance by the Employer and the Employer’s Ranking as to how well RSCC graduates met the Competency Area. For the complete listing, please see Appendix E, pg 56. Continue to monitor via surveys, advisory board and focus groups. Page 23 4. The placement rate of business graduates will be 85% or higher. Source: Office of Institutional Research & Effectiveness Graduate placement rates are gathered annually through direct contact with the graduates. Only graduates who are employed in work-related field or enrolled in a senior level institution are included. Quality Assurance Report, 2008-2010 Review For the last four academic years, the average placement rate for Contemporary Management graduates was 93%, exceeding the benchmark. The average placement rate for BMT graduates was 85%, meeting the benchmark. Faculty will continue to assist the placement office in collecting accurate data of student placement. The faculty will continue to monitor the placement rates and assist students with employer contacts. Page 24 5. The average student rating of courses and instruction will meet or exceed the institutional and national norms based on the ―Individual Development and Educational Assessment‖ (IDEA). Source: Office of Institutional Research & Effectiveness The ―IDEA‖ Student ratings of instruction provide a measure of instructional effectiveness within the business department and at the College. Two main types of scores are reported: average scores (five-point rating scale) and converted scores. The diagnostic reports provide the following summative scores: 1) an overall summary evaluation score, 2) progress on relevant objectives, 3) overall course, and 4) instructor scores. The average scores have limitations as these do not control for course difficulty or student characteristics. The converted averages (reported right) Quality Assurance Report, 2008-2010 Review Almost every semester, the business and management course evaluation average for the ―Summary‖ items exceeded the institutional and national norms. (Graphs 2.16 – 2.19) The faculty will continue to monitor the student evaluations to make improvements as needed in the individual courses. Workshops are conducted on a semester basis to assist faculty in understanding the detailed feedback analysis given by IDEA instrument. Page 25 compensate for inequality of average scores and provide normative comparisons. Converted normreference scores compare the ratings of all business courses taught here at the institution with all business courses evaluated nationally and with all evaluated at Roane State. Course evaluations are administered in individual classes by the College. The policy states the following as minimum evaluations per semester: 1) First-Year Faculty all classes. 2) Non-Tenured Faculty two classes. 3) Tenured Faculty one class. 4) Adjunct Faculty all classes. Quality Assurance Report, 2008-2010 Review Page 26 6. Roane State will provide cocurricular resources to facilitate student learning. Source: Learning Resource Center The Learning Resource Center was established to assist students with additional help outside the traditional studentteacher relationship. Quality Assurance Report, 2008-2010 Review The Learning Resource Center offers one-on-one tutoring in a variety of general education courses as well as business courses. Students also have free online tutoring from Smarthinking.com. Data supplied by the LRC indicated a high usage rate among students. Though individual participation was not tracked during this QAR period, a large number of writing/research assignments are required in the business and management programs which encourages use of the LRC. Because of the successful benefits to students offered by the Learning Center, the faculty will partner with the center on projects that will encourage even more participation by the students in individual classes. Page 27 7. An Advisory Committee will be utilized to give advice and counsel on programs within the Business Department. Source: Department's Advisory Committee Minutes Committee members were selected for their knowledge and expertise to represent the various areas within our programs. The committee meets two times each academic year with more frequent contact if needed. Quality Assurance Report, 2008-2010 Review Some of the topics the committee discussed during 2008-2010 were the proposal of a new ―Green Leadership‖ course as well as a new ―technology‖ course for all of our business majors. In addition, the committee discussed International Opportunities for students, reviewed the graduation rates of each program, and discussed the overall budget for the College and department Per the recommendation of the committee, a new course in green leadership and sustainability was successfully offered summer 2010. In addition, a new course in technology was approved and developed. The Business faculty will ask the Business Advisory Committee for assistance in recruitment and will continue to ask for their council and advice on opportunities to improve the subject areas in the programs. Faculty will continue to seek advice and counsel from the Advisory Committee members. In addition, faculty will continue to keep members involved in the programs by serving as a guest speaker in our classrooms. Appendix F-- Members of the Advisory Committee (pg 57) Page 28 9. The business faculty will encourage practical experience for students through summer employment and cooperative education opportunities (Co-Op). The faculty of the Business Department will seek internships and co-op opportunities for students in areas relating to business and management. Source: Faculty Quality Assurance Report, 2008-2010 Review Greater emphasis has been placed on assisting students with gaining work experience as well as academic knowledge. The faculty will utilize the Advisory Committees, the Placement Office, and outside contacts to help students secure positions. Some of the projects completed by business students include: building exercises to improve employee productivity and working relationships; researching approval process for state purchase orders; creating a database to include members of the Roane County Employers Association. The faculty will continue to advise students on selected learning projects. Graph not applicable. Page 29 10. The Department will partner with outside organizations to provide assistance in the continuance of business education. Source: FBLA Representative and Advisor The Future Business Leaders of America is a national organization to promote and encourage business education for high school students. The Business Department will host competitions for high school students participating in FBLA. Quality Assurance Report, 2008-2010 Review Over 250 high school students participated in the various competitions. Some of these competitions include: accounting, computer applications, job interview, marketing, entrepreneurship, and ethics. Business faculty assisted in judging competitions in business. RSCC will continue to host the competitions for the Future Business Leaders of America as requested. In addition, faculty will continue to assist in the judging of these competitions as needed. Graph not applicable. Page 30 11. The Business Department will participate in activities organized by Roane State to benefit high school students. Roane State offers an ―Academic Festival‖ for area high school students to compete in a variety of subjects, including business areas. Source: Alumni Relations, Academic Festival Representative Quality Assurance Report, 2008-2010 Review Within the past two years, the average number of students served in these competitions was 2,878 overall. On average, 94 of those students participated in business-related competitions. The business faculty led competitions in keyboarding, word processing, bookkeeping, and web searches. During this QAR period, The Department will continue to be actively involved with various aspects of the Academic Festival activities. Graph not applicable. Page 31 Table III Budgetary, Financial, and Market Results Budgetary, financial, and market performance results examine (1) management and use of financial resources and (2) market challenges and opportunities. 3. Budgetary, Financial, and Market Performance Results Adequate financial resources are vital to ensuring an outstanding faculty and teaching environment. The resources budgeted for and allocated to business units should be adequate to fund the necessary technology and training to allow students to develop the requisite competencies for business environments. Key indicators may include: expenditures per business student, business program expenditures as a percentage of budget, annual business unit budget increases or decreases, enrollment increase or decrease of business students, transfer in or out of business students, student credit hour production, or comparative data. Analysis of Results Areas of Analysis and Success Action Taken Performance Description Measure of (Competency) Measurement Instrument 1. Compare RSCC Business Department with Kansas Cost Study results in area of FullTime Equivalent (FTE) Student per Full-Time Equivalent Faculty.‖ Source: Office of Institutional Research & Effectiveness 2008-2009 Kansas Cost Study results by discipline using the comparison of ―FTE Student per FTE Faculty‖ The Business Department had a lower number of FTE students per FTE faculty than our TN Peers, NonTN Peers and the National Mean. (Graph 3.01) Quality Assurance Report, 2008-2010 Review The lower ratio is partially due to the increase in interactive and web classes. Having classes taught interactively permits sections with low enrollment to ―make.‖ In addition, web classes have a lower-thantraditional class enrollment cap. Results of Action Taken (occurs in the following year) Insert Graph of Resulting Trends for 3-5 Years (please graph all available data up to five years) Continue monitoring the Kansas Cost Study for comparative data. Page 32 2. Compare RSCC Business Department’s Instructional Costs per Student Credit Hour (SCH) with Kansas Cost Study results by discipline. 2008-2009 Kansas Cost Study results by discipline in comparison of ―Instructional Costs per SCH.‖ The Business Department had a higher instructional cost per student credit hour as compared to our TN Peers, Non-TN Peers and the National Mean. (Graph 3.02) This higher comparative ratio is a result of the number of long-time, highly qualified, fulltime tenured faculty in the department. Continue monitoring the Kansas Cost Study for comparative data. Source: Office of Institutional Research & Effectiveness Quality Assurance Report, 2008-2010 Review Page 33 3. Compare RSCC Business Department ―Percentage SCH taught by FT Faculty‖ with Kansas Cost Study results by discipline. Source: Office of Institutional Research & Effectiveness 2008-2009 Kansas Cost Study results by discipline in comparison to the ―Percentage SCH taught by FTE faculty‖ Business Department has seen a drop in the percentage of SCH taught by FTE faculty from 20072008 to 20082009. The percentage is lower than those for the National Mean, TN Peers, and Non-TN Peers. (Graph 3.03) Quality Assurance Report, 2008-2010 Review Because two full time faculty retired but are participating in the Post Retirement program, the number of full time faculty available decreased. Once this postretirement ends, additional fulltime faculty will be hired as budget permits. In addition, an increase in enrollment has led to the necessity of hiring more adjuncts to teach added sections of some courses. Continue monitoring the Kansas Cost Study for comparative data. Page 34 4. Compare business department’s financial budget to previous years. Source: Financial Services Business Department budget from 2008-2010 based on the percent of entire RSCC budget and percent of department budget used for instruction. Business Department has experienced a modest reduction in percent of the overall budget of RSCC which directly affects the percentage provided for instruction. (Graph 3.04) Budget reductions were across the board for all departments due to a reduction in state appropriations. Quality Assurance Report, 2008-2010 Review This reduction is contributed to the economic downturn across the nation. RSCC will continue internal efforts to become more cost effective and continue to track budget. Page 35 Table IV Faculty- and Staff-Focused Results 4. Faculty and Staff Focused Results Performance Measure (Competency) Description of Measureme nt Instrument 1. 100% of the business faculty will participate in two or more professional development activities per year. Faculty and Staff MBO's and evaluations from Deans. Roane State encourages faculty and staff to participate in professional activities of various types. Source: Faculty Faculty and staff-focused results examine how well the organization creates and maintains a positive, productive, learning-centered work environment for business faculty and staff. Key indicators may include: satisfaction or dissatisfaction of faculty and staff, positive, productive, and learning-centered environment, safety, absenteeism turnover, or complaints. Analysis of Results Areas of Analysis and Results of Insert Graph of Resulting Trends for 3-5 Success Action Action Taken Years (please graph all available data up Taken (occurs in the to five years) following year) Roane State offers in-house professional development activities several times each year. The Center for Teaching Arts and Technology (CTAT) offered extensive training in the Desire to Learn (Momentum), program used in all online classes, pre/post testing, as well as to supplement on-ground classes. In addition, workshops were conducted on various topics Quality Assurance Report, 2008-2010 Review A number of faculty have utilized this training as well as individual help with special projects to aid in classroom instruction. Faculty participation has been 100% over the assessment period. Additionally, faculty have been involved in conferences, workshops, and training given by other organizations. Because of budget constraints for the next academic year, the faculty will utilize in-house training more extensively as well as online training and webinars. The State of Tennessee offers free or reduced tuition for employees to take additional coursework. This source will also be encouraged. There will be a professional Appendix G -- Professional Development Activities (pg 58) Page 36 including podcasting, Skype, Camtasia, course design, Web 2.0 and Web 3.0, retention and motivation of students, Google apps, and jazzing up PowerPoints. Quality Assurance Report, 2008-2010 Review development component in the College's upcoming Quality Enhancement Program (QEP) and the Department faculty will be engaged in implementation of this project to enhance student learning through increased us of learning strategies and learning styles. Page 37 2. Employee turnover rate within the business department will reflect a high level of employee job satisfaction and commitment to the college. Source: Department Chair Annual review of employment status of faculty and staff within the Business Department. During the past two years, one instructor has retired and is now serving on a Post Retirement teaching contract through December 2012. No other changes in employment status were made during this period. Quality Assurance Report, 2008-2010 Review A budget request has been submitted to RSCC Executives requesting the hiring of an additional fulltime faculty member to replace the one who retired. Due to budget restraints, hiring a new full-time faculty has been postponed. However, RSCC is fortunate in having quality and qualified adjuncts to fill the void until a full-time person is hired. Graph not applicable. Page 38 3. 100% of the Business Department faculty will earn a minimum of "satisfactory" on their individual performance evaluation each year. Source: Division Dean Annual evaluation from Division Dean. All full-time faculty members within the Business Department received a minimum of a ―satisfactory‖ on the individual performance evaluation conducted by the Dean. Quality Assurance Report, 2008-2010 Review Based on the evaluations, the business department encompasses quality and caring instructors. Any suggested areas of improvement are discussed with the academic dean and/or mentors and improvement plans are created. Strive for continued improvement each year. Graph not applicable. Page 39 Table V Organizational Performance Results Organizational effectiveness results examine attainment of organizational goals. Each business unit must have a systematic reporting mechanism for each business program that charts en rollment patterns, student retention, student academic success, and other characteristics reflecting students' performance. 5. Organizational Effectiveness Results Key indicators may include: improvement in safety, hiring equity, increased use of web-based technologies, use of facilities by community organizations, contributions to the community, or partnerships, graduation and retention rates by program, and what you report to governing boards and administrative units. Performance Measure (Competenc y) Description of Measureme nt Instrument 1. Increase the number of graduates by five percent from the previous year in the A.A.S. Business Management Technology and A.A.S. Contemporary Management programs. Data is collected by the Office of Institutional Research & Effectiveness and reported to the Tennessee Board of Regents using the terms Summer, Fall, Spring to represent a reporting year. Source: Office of Office of Institutional Research & Effectiveness Analysis of Results Areas of Analysis and Success Action Taken In 2009-10, the graduation rate of the BMT program increased 75 percent from the previous year. This increase is partly due to the new General Business option being added to the program. (Graph 5.01) In 2009-10, the graduation rate Quality Assurance Report, 2008-2010 Review Retention efforts at Roane State through student contact, advising and counseling are done on an on-going basis. Results of Action Taken (occurs in the following year) Insert Graph of Resulting Trends for 3-5 Years (please graph all available data up to five years) The faculty will strive to continue their diligence in supporting students from entry into the programs to graduation. Page 40 of the CMGT program increased its graduation rate by 28 percent from the previous year. (Graph 5.01) As indicated in Graph 5.02, the number of graduates of each option under the Business Management Technology program increased, with the exception of the accounting option. Quality Assurance Report, 2008-2010 Review Page 41 2. Increase the number of courses offered using alternative delivery methods and/or varied start times by five percent to better accommodate today’s student needs. Source: Office of Office of Institutional Research & Effectiveness Courses offered via distance education including web, interactive, and hybrid as well as courses offered with various start dates within a semester. When comparing Fall semesters, the business department increased the number of sections offered via interactive delivery as well as web delivery. However, due to altering these offerings, the number of hybrid sections decreased. When comparing spring semesters, each of the delivery method offerings significantly increased. (Graph 5.03) Quality Assurance Report, 2008-2010 Review By increasing our offerings in the interactive format, we are able to reach those students in the outer lying areas of our service district who do not wish to take web courses. In addition, in the Spring 2010, the Contemporary Management program piloted teaching the core classes as a hybrid model. This increase is depicted in (Graph 5.03) The faculty will continue to use the data provided by the Institutional Research & Effectiveness Office as well as other sources to schedule classes that meets student needs. Beginning fall 2010, the business department will offer at least one 10week web course that will begin five weeks after the start of the semester. If proven successful, more will be added in the future. Page 42 3. The Business Department will equal or surpass enrollment in line with the college growth. Source: Office of Institutional Research & Effectiveness Data gathered from the Office of Office of Institutional Research & Effectiveness Enrollment Synopsis. The headcount for BMT majors shows an increase consistent with the overall growth at RSCC for Fall 2009, though that number declined by 10.57 percent the following spring semester. (Graph 5.04 and Graph 5.05). Though this decline is consistent with previous spring semesters, it is not consistent with the overall enrollment increase at RSCC. Quality Assurance Report, 2008-2010 Review The business faculty is offering multiple formats of classes both in the BMT program as well as the CMGT program. Though it is difficult to determine if the alternative delivery options were the sole reason for increased enrollment in the BMT program, the feedback from the students is positive. The Department faculty as well as Roane State’s administration are actively involved in plans to increase the enrollment in both the BMT and the CMGT programs by way of promotional booklets, face-to-face recruiting, and increasing program knowledge of others within the College. Page 43 4. The Business Department to will strive to maintain at least an 80 percent student pass rate and an minimum overall GPA of 2.5. Source: Office of Institutional Research & Effectiveness Data tracked by the Office of Institutional Research. The overall percentage pass rate of students majoring in BMT was in line with RSCC as a whole each semester. The two-year average for RSCC as a whole was 86%, for BMT was 87% and the average for CMGT was 94%. (Graph 5.06) The overall GPA was also consistent with RSCC as a whole. When comparing GPAs, the average for RSCC as a whole was 2.78, for BMT was 2.67, and the average GPA for CMGT was 3.33. Quality Assurance Report, 2008-2010 Review The pass rate and the GPA calculations exceeded the goals set for each program. The consistent high scores from the Contemporary Management program is credited to the high frequency of adult learners. The faculty will continue to maintain vigilance in monitoring the Pass Rates and the GPA of students in their respective majors Page 44 TABLE VI -- 2008-10 Faculty Qualifications During this QAR reporting period (2008-2010), 99.48% of the full-time equivalent (FTE) faculty were Master's or Doctorate degree qualified. Only .52% of the FTE faculty was considered a Master's or Doctorate degree Exception; thus, remaining within the 10% exception rate allowed. TABLE VI NEW FULL-TIME AND PART-TIME FACULTY QUALIFICATIONS NAME MAJOR COURSES LIST ALL DOCUMENT OTHER ACBSP (List TEACHING TAUGHT EARNED PROFESSIONAL QUALIFICATION alphabetically FIELD (List the DEGREES CERTIFICATION 1. Master by Last Name) Courses (State Degree as CRITIERA 2. Doctorate Taught Documented on 3. Professional 18 Graduate Cr. During the Transcript, Must 4. Exception Hrs in Field Reporting Include Major Two Years Work Period, Field) Experience Do Not Teaching Duplicate Excellence Listing) Publications Professional Certifications Branam, Darrell Management Campbell, Karen Business and Management MGT 203 – Human Resource Mgt (1) MGT 207 – Managerial Acctg & Finance (1) BUS 101 – Intro to Business (1) BUS 261—Psy Aspects of Mgt (1) BUS 281 – Mgt & Supervision (1) MGT 200 – Foundations of Business (1) Quality Assurance Report, 2008-2010 Review MS Business Administration BS Mechanical Engineering MBA BS Organizational Mgt Special Achievement Award Member Prof. Education of TN Member Assoc of American Education 4+ yrs Trainer for Employee Assistant Program at Cariten Healthcare Master Master Page 45 Cecil, Justin Management Dials, Bill Business Heys, David Business Kluss, Geoffrey Business Laman, Michael Business MGT 206 – Applied Economics (2) BUS 101 – Intro to Business (2) BUS 273 – Prin of Marketing (1) BUS 215 – Business Ethics (1) BUS 101 – Intro to Business (2) BUS 111 – Bus Math (2) BUS 273 – Prin of Marketing (2) BUS 281 – Mgt & Supervision (3) BUS 111Business Math (1) Quality Assurance Report, 2008-2010 Review MBA BS Organizational Management Juris Doctor MS MBA MA Psychology ML Liberal Arts BA Psychology MBA BA Economics 11+ yrs President & Cofounder of My Union Journal Master Certificate of Training, Tech. Instructor Course MBA BS Business Administration Ph.D Higher Ed Administration (minor Adult Education) MS in Education BS Cardiopulmonary Sciences Master Master Master 24 yrs. Teaching/ Administrative Experience Exception (Does not exceed the 10% allowed exception rate) Page 46 TABLE VII -- Curriculum Summary for new option under the Business Management Technology program Quality Assurance Report, 2008-2010 Review Page 47 Legend for Table VII ―EXHIBIT L‖ P R O F E S S I O N A L C O M P O N E N T Standard III – Curriculum Standard: At least 25 percent of the business curriculum must consist of a professional component including four of the following areas of study: a. b. c. d. e. f. g. h. i. Accounting Computer information applications Quantitative methods of analysis Principles of economics Business in society - the international environment, legal/political environment, and ethical business behavior Marketing Entrepreneurship/free enterprise Finance Management ―EXHIBIT M‖ G E N E R A L E D U C A T I O N R E Q U I R E M E N T Standard III – Curriculum Standard: Excellence in business education requires a foundation in general education. For accreditation, therefore, general education must include no fewer than 25 percent of the credits required for the associate degree and must consist of courses which contribute to the following educational goals: a. b. c. d. e. f. g. h. i. j. Written, interpretive, and oral facility with the English language. An historical perspective. An understanding of the role of the humanities in human experience. A personal ethical foundation. An understanding of social institutions and the obligations of citizenship. Knowledge of science and its applications. An understanding of contemporary technology. An understanding of the principles as well as the investigative strategies of the social sciences. An appreciation of the fine and performing arts. A global perspective. Quality Assurance Report, 2008-2010 Review Page 48 APPENDIX Appendix A – Organizational Chart Appendix B – Business Graduate Exam, Section Analysis Appendix C – CCSSE Results of Business Related Questions Appendix D – Alumni Survey Results Appendix E – Employer Survey Results Appendix F – Business Advisory Committee Members Appendix G – Professional Development Activities Quality Assurance Report, 2008-2010 Review Page 49 Appendix A -- Organizational Chart Social Science, Business, and Education Division Organizational Chart 2009-2010 * OAD Option Terminated, effective Fall 2010 Quality Assurance Report, 2008-2010 Review Page 50 APPENDIX B -- Business Graduate Exam, section analysis (referenced in Table 1.1) The following information identifies questions by section in which 50% or more of the students missed on the Business Graduate Exam. Faculty will review each of these questions for possible edits to the question s’ wording while maintaining the same content. Quality Assurance Report, 2008-2010 Review Page 51 Appendix B continued. Quality Assurance Report, 2008-2010 Review Page 52 APPENDIX C — CCSSE Results of Business Related Questions (referenced in Table 2.1) Quality Assurance Report, 2008-2010 Review Page 53 APPENDIX D – Alumni Survey, Spring 2010 (referenced in Table 2.2) The following graphs indicate the success rates for additional questions on the Alumni Survey that was conducted in Spring, 2010. Quality Assurance Report, 2008-2010 Review Page 54 Appendix D continued Quality Assurance Report, 2008-2010 Review Page 55 APPENDIX E – RSCC Employer Survey, Spring 2008 (referenced in Table 2.3) The following chart from the Employer’s Survey indicates the complete ranking of competencies 1-23 that employers determined as important as well as how well they thought our students have met the competencies. Quality Assurance Report, 2008-2010 Review Page 56 APPENDIX F -- Business Advisory Committee (referenced in Table 2.7) The following lists the current members of the Business Advisory Committee, 2008-2010. Name Place of Employment Below, Quent Kimberly-Clark (retired) Bradshaw, David SunTrust Bank, East Tennessee Chinn, Rick R&R Properties, Oak Ridge Owner/Operator Entrepreneurship Knight, Kathy Economic Development Council, Loudon County Assistant Director Economics Melton, Brenda Career Center, Morgan County Training Manager Business Administration / Training Miget, Ken Wal-Mart, Rockwood General Manager Management/Leadership Nora, Patricia Maid in TN, Clinton Owner/Operator Entrepreneurship Payne, Carol Knoxville Utility Board Team Leader, Learning Center Management / Training Richardson, Angela TTC-Harriman Director, BST program Education / Business Wykle, Jamie Auditor Accounting TVA--Knoxville Quality Assurance Report, 2008-2010 Review Position Financial System, Owner President Anderson/Roane Counties Area of Expertise Accounting Business/Management Page 57 APPENDIX G – Professional Development Activities (referenced in Table 4.1) Conferences: o ACBSP Annual Conference, Los Angeles, CA o ACBSP Team Evaluator o Cengage Technology Conference o National Association for Community College Entrepreneurship (NACCE) o Community Colleges for International Development (presented in 2010) Community Involvement: o Roane County Women’s Association, (current active member and past president) o Anderson County Drug Court o Cumberland County Chamber of Commerce (2009 board member) o Industrial Development Committee (2010 co-chair) o Youth Achievement Foundation, Inc. o Three-Star Program Implementation Team o Mind Your Own Business Entrepreneurship Camp (director) o Cumberland Business Incubator (assistant director) o Creating a Rural Entrepreneurial System in Tennessee (CREST), pilot program o Authoring and conducting a financial training course on Integrated Financial System o TN Technology Center, advisory board member o Presentation to Roane County Employer’s Association on benefits of education o Presentation to Youth Leadership of Roane County on teenage stress Coursework / Training / License: o Basic Economic Development , State of TN o Adobe Photoshop, two courses o Senior tax Advisor, Level V, Tax Institute at H&R Block o Renewal of Certified Public Accountant license o Study Abroad program for CCID and RSCC to Denmark and Sweden, 2009-2010 o Study Abroad program for RSCC to Costa Rica, 2010 o Podcasting workshop o MS Office --Access class Quality Assurance Report, 2008-2010 Review Page 58 Quality Assurance Report, 2008-2010 Review Page 59