ASSESSMENT RESULTS FOR QCC EDUCATIONAL OUTCOME #2 BUSINESS Academy, Spring 2014 GEN ED#__2- ___USE ANALYTICAL REASONING TO IDENTIFY ISSUES OR PROBLEMS AND EVALUATE EVIDENCE IN ORDER TO MAKE AN INFORMED DECISION. QCC Example Outcomes: a. Identify and summarizes the problem/issue b. Develop options by utilizing facts from the assignment. Identify the advantages and disadvantages. c. Evaluate all options by demonstrating support from the assignment. d. Compose a solution with support from evidence in the assignment. GEN ED OUTCOME ADDRESSED IN USE ANALYTICAL REASONING TO IDENTIFY ISSUES OR PROBLEMS AND ASSESSMENT: EVALUATE EVIDENCE IN ORDER TO MAKE AN INFORMED DECISION. EVIDENCE/ MEASURABLE DATA Describe artifacts reviewed No. of artifacts No. of sections RUBRIC We received approximately 10 artifacts from five different courses (one section of each course). This gave us a total of 50 artifacts of student papers from course assignments. (See attached assignments) Courses included are BU520, BU508, BU701, BU102, and BU201 See attached rubric. The rubric consisted of four components of general education objective #2. Each component had a possible score of zero to four points. Maximum score possible for all components would be 16. RESULTS After summarizing and averaging all scores, an average score of 11.62 out of 16 points which was an average score of 73% of total. (SEE BELOW) STATUS – degree to which students have met Gen. Ed. Outcome NOTE THIS IS JUST AN EXAMPLE OF A RATING SCALE CHECK ONE RATIONALE (Briefly explain rating above) DISCUSSION OF THIS PROCESS: What did you learn and what would you recommend? Excellent (4 points) Good (3 points) Minimally Acceptable (2 points) Poor (1 point) Insufficient evidence (0 points) PLEASE SEE ATTACHED SPREADSHEET AND DISCUSSION BELOW. Rationale for the ratings included a rubric that was agreed upon in a norming session on April 9, 2014. Papers were distributed to five faculty. Each paper was scored by two different faculty. If the scores were greater than a 4 point difference, then a third reviewer completed their scoring. All scores for each artifact were averaged together to get the results. The process had many steps and time constraints made it difficult, in getting completed assignments from faculty early enough in the semester to complete the scoring by the end of the semester. The scoring was based on the mutually agreed upon rubric. It was very interesting; in that we were able to see how widely different assignments can achieve the same skill in a variety of ways. I would recommend a debriefing session with the faculty scorers and the faculty developing the assignments. I think if faculty knew the objective and rubric before developing their assignment, they may modify it to further develop all aspects of the objective. REPORT 1. RESULTS Based on the average scores for all 50 artifacts, students achieved a score of 11.62 points out of 16 or 73%. We further broke down the results by course to see if there was a difference between lower and upper level courses. The results were as follows: BU201 – 11.78 points – 74% BU102 – 11.38 points – 71% BU508 – 12.85 points – 80% BU520 – 10.15 points – 63% BU701 – 11.88 points – 74% Based on these results it does not appear to be strong direct relationship between the course level and outcomes. While our goal was 7%, we are encouraged that 73% of our students have achieved this skill. These students are in a variety of classes and their ability to think critically was assessed in many ways. We recommend that rubrics are distributed to all faculty. Rubrics can assist faculty in the development of requirements for their assignments. This may help students achieve a better understanding of the skills required. 2. DISCUSSION OF THIS PROCESS: ASSESSMENT OF GENERAL EDUCATION RUBRIC 1) Fall 2013 – Developed a survey for faculty of the business department to select which general education objective best fits their courses. 2) January 2014 - Discussion with Dr. Falik, chair of the business department, to discuss his views of the general education objectives which are included in most of our business courses. 3) Met with our academic advisors to obtain their views of general education objectives and included their choices in our survey. 4) Summarized the results of the survey and determined that general education objective#2 – Analytical Reasoning was the best fit for most of our classes. 5) We asked six faculty to submit copies of their artifacts. We requested ten papers from each course. The assignments were also given. 6) We developed a rubric that would achieve the learning outcomes for the general education objective. 7) After receiving the assignments from five faculty (one course had not completed their assignment) we held a norming session. 8) Dr. Beckford met with us to discuss our scoring on three different papers. We agreed to a score and were able to find unity in our scoring. 9) Each faculty scored approximately 15 papers using the rubric. 10) We summarized the results of scoring. Those artifacts that had a score differential of 4 points were scored by a third reviewer. 11) All scores were averaged for each artifact. 12) We then analyzed the results for an overall student achievement of the objective and then again by assignment. We also looked at the results of how introductory courses may differ from upper level courses. 13) Please see attached for results.