Annual Progress Report on Assessment of Student Learning Student Life Units (rev. 10/09 and 6/11) Academic year: 2010-2011 Student Life Unit/Program: Academic and Career Information Center Person(s) preparing report: Michelle Haupt Date submitted: July 26, 2011 In one paragraph, summarize your Annual Progress Report Overall, the assessment results indicate that programs and services offered through the Academic and Career Information Center (ACIC) are effective in assisting students in the process of career development. Specifically, students are able to learn more about K-State academic majors as well as create a plan of action as a result of attending our programs and using our services. In addition, students enrolled in EDCEP120 are effectively learning how to communicate during the interview process. Provide the web link to department web site where degree program SLOs, Alignment Matrix, and 2008-2009 APR Summary are posted : www.k-state.edu/acic/about/learningoutcomes.htm 1) List the complete set of Student Learning Outcomes/Objectives for your unit (all SLOs identified for your unit) The student learning outcomes (SLOs) for ACIC were: Outcome 1: As a result of meeting with a career specialist, students will be able to develop a plan of action to choose a major or career. Outcome 2: As a result of attending the Academic Majors Fair, students will be able to identify the different majors available at K-State. Outcome 3: After taking the EDCEP 120 class, students will be able to assess the strengths and weaknesses of their interviewing skills by participating in a mock interview. 2) Identify the Student Learning Outcomes/Objectives (from the full list above) that were assessed for this Annual Progress Report and how they are inclusive of the university undergraduate SLOs (knowledge, communication, critical thinking, diversity, academic and professional integrity). All Student Learning Outcomes (SLOs) identified in question one were assessed in 2010-2011. They are inclusive of the university undergraduate SLOs of knowledge, communication, and critical thinking. As students increase their knowledge of K-State undergraduate programs (outcome 2), they also increase their understanding of the occupational opportunities connected with those programs. In addition, critical thinking skills are developed as students dialogue with career specialists in ACIC (outcome 1) to consider how their interests, abilities and values connect with vocational possibilities. Career assessment inventories allow students to self evaluate and apply information learned to their unique situation. As students experience a mock interview (outcome 3), their communication skills are tested. Successful interviews include not only an ability to interact effectively, but also to demonstrate to an employer sufficient knowledge and skills needed for a position. As students receive feedback, they are able to reflect upon their performance and identify areas for personal development as well as ways to improve their interview responses in the future. 3) For each learning outcome, describe (a) the measures used (at least one direct measure must be used for each student learning outcome) ,(b) the sample of students from whom data were collected, (c) the timetable for the collection, and (d) the forum in which the measures were administered. (Examples of direct measures can be accessed at http://www.k-state.edu/assessment/plans/measures/direct.htm). Assessment details for each outcome are as follows: Outcome 1: As a result of meeting with a career specialist, students will be able to develop a plan of action to choose a major or career. a. Measure - This outcome was measured through a question on the ACIC user survey. (indirect) b. Sample - The survey was administered to all students who used ACIC services in the fall 2010 semester. This includes students who met with a career specialist for an individual appointment, as well as students who stopped in for a brief question and visit. There were 977 students who used ACIC services in the fall of 2010. Of those students, we were able to successfully send a survey via email to 787 students. There were 203 who responded to the survey for a response rate of approximately 26%. c. Timetable - The survey was administered monthly during the fall 2010. d. Forum - The survey was electronic and administered via K-State’s Axio online system. Outcome 2: As a result of attending the Academic Majors Fair, students will be able to identify the different majors available at K-State. a. Measure - This outcome was measured through a question on the Academic Majors Fair survey. (indirect) b. Sample - The survey was administered to all students who attended the Academic Majors Fair. Approximately 850 students attended the fair. Students were asked to swipe their student identification (ID) card at the registration table to sign in, and 806 usable emails were collected to utilize for the survey. There were 205 students who completed the Academic Majors Fair survey for a response rate of approximately 25%. c. Timetable - The survey was administered following the event on October 13, 2010 from October 29 to November 14 d. Forum - The survey was electronic and administered via K-State’s Axio online system. Outcome 3: After taking the EDCEP 120 class, students will be able to assess the strengths and weaknesses of their interviewing skills by participating in a mock interview. a. Measure - This outcome was measured by scoring student mock interview performance using a rubric. (direct) b. Sample - 85 students were enrolled in EDCEP 120 during the spring semester; 34 student feedback sheets were used to collect information. c. Timetable - Mock interviews were conducted in April 2011. Students scheduled individual dates in ACIC for a mock interview. d. Forum - Following a mock interview, a mock interview feedback form was completed by the Career Specialist who conducted the interview. 4) Describe the results of the assessment. (What do they tell you about student learning? What did you learn about strengths and weaknesses of your unit?) If specific results are not available, describe the progress that has been made on the initiatives included in the approved assessment plan. Results learned from Outcome 1 As a result of meeting with a career specialist, students will be able to develop a plan of action to choose a major or career. A question was added to the ACIC user survey to measure outcome 1. The question on the survey read as follows: “As a result of meeting with a Career Specialist, I was able to develop a plan of action to choose my major and career.” There were 203 students who completed the user survey, and the responses to this question were: a. Strongly agree 34 17% b. Agree 96 47% c. Neutral 60 30% d. Disagree 6 3% e. Strong disagree 4 2% f. No response 3 1% Based on the responses, 64% of the students surveyed indicated either “strongly agree” or “agree” that they were able to develop a plan of action. I would have hoped for a higher number, however, a plan of action might be difficult to define depending on where a student happens to be in the career development process. Interestingly, earlier questions in the survey (question 2 and question 4) suggest students were pleased and able to receive the information needed. o Survey question 2 asks: “Was the Career Specialist you worked with able to answer all of your questions?” There were 158 student responses (78%) indicating “all” questions were answered and 32 (16%) indicated “most” were answered. This totals 190 students, or 94% of the students who responded felt as if all or most of their questions were answered. o Survey question 4 asks: “Overall, please rate your satisfaction with your ACIC visit.” There were 153 student responses (76%) indicating they found the visit “very helpful” and 32 responses (16%) found it “helpful”. This totals 185 students, or 92% of the students who responded found their visit helpful or very helpful. Limitations of outcome 1: All students who visited ACIC did not participate in the survey. All students who visited ACIC did not receive the opportunity to complete the survey. Some emails provided by students were invalid. All students who visited ACIC were surveyed, regardless of whether or not they met with career specialist. Although most students did have individual contact with a career specialist, if students dropped by the office for a quick answer or to pick up information, they most likely did not have the opportunity to develop a plan of action. Students using ACIC services enter the office at varying level of career preparedness and background. Some high schools have strong career programs for students while others do not. Students using ACIC services enter the office at varying of career readiness. Some students voluntarily use the service while others are required to use our office as part of a class assignment. At times, those who are required to complete an assignment are not as engaged in their meeting. ACIC user survey and Academic Majors Fair survey are sent from the account acic@kstate.edu. Some students may have thought they received two of the same emails. In the fall semester of any academic year, there are new Career Specialists who are learning how to perform the daily functions associated with the position of career specialist. The skill level of the Career Specialist is directly related to his or her own training and tenure with ACIC. With continued employment, Career Specialists become an increasingly valuable asset to the office. The comfort level of serving in a counseling role, and knowledge of K-State services and programs increases as a student remains employed as a Career Specialist. Results learned from Outcome 2 As a result of attending the Academic Majors Fair, students will be able to identify the different majors available at K-State. A question was added to the Academic Majors Fair follow up survey to measure outcome 2. The question on the survey read as follows: “As a result of attending the Academic Majors Fair, I learned more about majors available at K-State.” There were 205 students who completed the follow up survey, and the responses to this question were: Agree 188 (91.71%) Disagree 17 ( 8.29%) Limitations of outcome 2: All attendees did not participate in the survey All attendees who visited the Academic Majors Fair did not receive the opportunity to complete the survey. Some emails were invalid. ACIC user survey and Academic Majors Fair survey are sent from the account acic@kstate.edu. Some students may have thought they accidentally received two of the same emails. There was a slight delay from the date of the Academic Majors Fair survey to the date the survey was sent to students. Data collected from the student ID card is not immediately available for use. It must first be processed and data must be extracted from the iSIS system. The delay between the event and the sending of the survey could cause a lower response rate among Fair attendees. Results learned from Outcome 3 After taking the EDCEP 120 class, students will be able to assess the strengths and weaknesses of their interviewing skills by participating in a mock interview. Students enrolled in EDCEP 120, Academic and Career Decisions, were assigned a mock interview as a part of their course requirements for the class. In the spring semester of 2011 there were 85 students enrolled in the class. Some sections of the course have a start date that is late in the semester. Students enrolled in these sections were selected to measure this outcome. This was done to give time for career specialists who were administering the interviews to practice and become comfortable evaluating their peers in the interview process. Mock interview feedback forms were collected for 34 students. Students were evaluated in six categories: Body Language Appearance Self-expression/style Enthusiasm and interest Experience Maturity Students were given the opportunity to interview with their company of choice, thus the content of the interview was not evaluated. The categories listed above focus on the affective parts of the interview process. In the above categories, students could be given a rating as follows: Excellent = 4 Good = 3 Satisfactory = 2 Unsatisfactory = 1 Immediately following the interview, career specialists were able to provide initial, verbal feedback about key areas where the student performed well, and key areas a student may need to develop. After the student was dismissed, the career specialist completed a feedback form/rubric with more detailed advice. The completed form was then given to the course instructor. Instructors had individual meetings with each student to finish the semester, and results from the mock interview were one of the discussion topics for the meeting. The high score possible for the mock interview was 24 points. The mean score for the 34 students who were assessed was 19.47. The range of scores was from 24 to 11. The average score of 19.47 indicates that most students performed about average on the mock interview assignment. Students had an opportunity to hear a lecture about proper interview techniques, preview the feedback form, and review possible questions that may be asked during the interview. However, I would have hoped student score averages would have been a bit higher. In addition to the ratings given, career specialists were able to make comments regarding student performance in the mock interviews. Some sample comments are listed below: “Good handshake, try to keep better eye contact, and slow down a little bit” “Very good energy, did a good job of turning negative questions into positive answers” “Remember to use STAR (Situation/Task, Action, Result). This is what should be worked on most!” “I know you have a lot of experience at Michael’s but use other jobs/activities to come up with other examples” “Very calm and composed! Work on sustaining eye-contact a little better” “Excellent job describing your experiences” “You don’t want to come off as impatient, so be careful. Also try to devise strategies that will help you calm down the nerves. (Practice always helps with this.)” The comments from the career specialist demonstrate valuable feedback that was given to the students who interviewed. This assignment is worthwhile in helping students learn strengths and weaknesses in an interview setting and practice a needed skill needed in the career journey. Limitations of outcome 3: Although 85 students were enrolled, only 34 feedback forms were collected. It would have been helpful to have evaluated all students in the class. Some students have had interview experience, and some students have not. Higher scores on the Mock Interview may have come from students who have had more experience. There were multiple Career Specialists who led the interviews and provided feedback to those completing the mock interview. With multiple reviewers, there were likely differences in the way evaluations were considered. The maturity level of the Career Specialist was also a factor in the evaluation process. As indicated previously, the skill level of the Career Specialist is directly related to his or her own tenure with ACIC. Those who have worked in this position longer feel more confident providing feedback to their peers that is sometimes difficult or uncomfortable to discuss. 5) Describe the actions and/or revisions that were implemented in response to the previous year’s assessment results and the effects on student learning observed on this year’s SLOs. This question is not applicable since this is the first assessment year for the Academic and Career Information Center. 6) Describe the process by which staff reviewed the results of this year’s SLOs and the actions and/or revisions that are planned in response to the assessment results. (include changes that may be made to unit SLOs or to the general assessment strategy) Outcome 1: As a result of meeting with a career specialist, students will be able to develop a plan of action to choose a major or career. Results of outcome 1 were reviewed by the coordinator and shared and discussed with ACIC career specialists at a staff meeting. Based on the responses, 64% of the students surveyed indicated either “strongly agree” or “agree” that they were able to develop a plan of action. As mentioned earlier in this report, developing a plan of action takes time. In addition, a plan of action is unique to each student we assist. No revision is needed in the training materials to address this outcome. However, this information is a good reminder for career specialists to consider during the on-going training activities for this position. It is also important to note that 94% of the students who responded to the user survey felt as if most or all of their questions were answered, and 92% of respondents found their visit to ACIC helpful or very helpful. Outcome 2: As a result of attending the Academic Majors Fair, students will be able to identify the different majors available at K-State. Results of outcome 2 were compiled and reviewed by the Academic Majors Fair Planning Committee to consider changes and adjustments to the event for the following year. There were 205 students who completed the follow up survey, and 92% of the students who responded indicated they learned more about majors available at K-State as a result of attending the Academic Majors Fair. The Academic Majors Fair Planning Committee is pleased with those results, and there are no actions or revisions currently planned in response to these results. The Committee will continue to work to improve the event to aid in the retention efforts at K-State. Outcome 3: After taking the EDCEP 120 class, students will be able to assess the strengths and weaknesses of their interviewing skills by participating in a mock interview. Results of outcome 3 were compiled and reviewed by the coordinator to consider changes and improvements to the instruction methods for EDCEP 120. Results will also be shared with the ACIC staff in the 2011-2012 academic year. Adjustment will be made to the mock interview feedback form and teaching materials for the 2011-2012 academic year with regard to student attire. However, the adjustments are not directly related to the SLOs from 2010-2011. The most significant concern noticed during the mock interview process this semester was the inappropriate attire worn by those who interviewed. Although appropriate attire was discussed by course instructors, student interpretations of that discussion were vastly different to the instructor and interviewer expectations. The mock interview materials (both class materials and feedback form) will be modified giving visual representation of expected appropriate interview attire. 7) If needed, revise the department’s Assessment Plan for the next academic year. Attach your assessment plan to this report. There will be no changes to the outcomes for the Academic and Career Information Center for the 2011-2012 academic year. Since 2010-2011 is the first assessment cycle for ACIC, it will be helpful to compare learning outcomes for two academic years in order to establish a baseline for subsequent reports.