Annual Progress Report on Assessment of Student Learning Student Life Units

advertisement
Annual Progress Report on
Assessment of Student Learning
Student Life Units (rev. 10/09 and 6/11)
Academic year: 2010-2011
Student Life Unit/Program: Academic and Career Information Center
Person(s) preparing report: Michelle Haupt
Date submitted: July 26, 2011
In one paragraph, summarize your Annual Progress Report
Overall, the assessment results indicate that programs and services offered through the Academic
and Career Information Center (ACIC) are effective in assisting students in the process of career
development. Specifically, students are able to learn more about K-State academic majors as well
as create a plan of action as a result of attending our programs and using our services. In addition,
students enrolled in EDCEP120 are effectively learning how to communicate during the interview
process.
Provide the web link to department web site where degree program SLOs, Alignment Matrix, and
2008-2009 APR Summary are posted :
www.k-state.edu/acic/about/learningoutcomes.htm
1) List the complete set of Student Learning Outcomes/Objectives for your unit (all SLOs identified for
your unit)
The student learning outcomes (SLOs) for ACIC were:
Outcome 1: As a result of meeting with a career specialist, students will be able to develop a
plan of action to choose a major or career.
Outcome 2: As a result of attending the Academic Majors Fair, students will be able to identify
the different majors available at K-State.
Outcome 3: After taking the EDCEP 120 class, students will be able to assess the strengths and
weaknesses of their interviewing skills by participating in a mock interview.
2) Identify the Student Learning Outcomes/Objectives (from the full list above) that were assessed for this
Annual Progress Report and how they are inclusive of the university undergraduate SLOs (knowledge,
communication, critical thinking, diversity, academic and professional integrity).
All Student Learning Outcomes (SLOs) identified in question one were assessed in 2010-2011.
They are inclusive of the university undergraduate SLOs of knowledge, communication, and
critical thinking.
As students increase their knowledge of K-State undergraduate programs (outcome 2), they also
increase their understanding of the occupational opportunities connected with those programs.
In addition, critical thinking skills are developed as students dialogue with career specialists in
ACIC (outcome 1) to consider how their interests, abilities and values connect with vocational
possibilities. Career assessment inventories allow students to self evaluate and apply
information learned to their unique situation.
As students experience a mock interview (outcome 3), their communication skills are tested.
Successful interviews include not only an ability to interact effectively, but also to demonstrate
to an employer sufficient knowledge and skills needed for a position. As students receive
feedback, they are able to reflect upon their performance and identify areas for personal
development as well as ways to improve their interview responses in the future.
3) For each learning outcome, describe (a) the measures used (at least one direct measure must be used for
each student learning outcome) ,(b) the sample of students from whom data were collected, (c) the
timetable for the collection, and (d) the forum in which the measures were administered. (Examples of
direct measures can be accessed at http://www.k-state.edu/assessment/plans/measures/direct.htm).
Assessment details for each outcome are as follows:
Outcome 1: As a result of meeting with a career specialist, students will be able to develop a
plan of action to choose a major or career.
a. Measure - This outcome was measured through a question on the ACIC user survey.
(indirect)
b. Sample - The survey was administered to all students who used ACIC services in the fall
2010 semester. This includes students who met with a career specialist for an individual
appointment, as well as students who stopped in for a brief question and visit. There were
977 students who used ACIC services in the fall of 2010. Of those students, we were able to
successfully send a survey via email to 787 students. There were 203 who responded to the
survey for a response rate of approximately 26%.
c. Timetable - The survey was administered monthly during the fall 2010.
d. Forum - The survey was electronic and administered via K-State’s Axio online system.
Outcome 2: As a result of attending the Academic Majors Fair, students will be able to
identify the different majors available at K-State.
a. Measure - This outcome was measured through a question on the Academic Majors Fair
survey. (indirect)
b. Sample - The survey was administered to all students who attended the Academic Majors
Fair. Approximately 850 students attended the fair. Students were asked to swipe their
student identification (ID) card at the registration table to sign in, and 806 usable emails
were collected to utilize for the survey. There were 205 students who completed the
Academic Majors Fair survey for a response rate of approximately 25%.
c. Timetable - The survey was administered following the event on October 13, 2010 from
October 29 to November 14
d. Forum - The survey was electronic and administered via K-State’s Axio online system.
Outcome 3: After taking the EDCEP 120 class, students will be able to assess the strengths
and weaknesses of their interviewing skills by participating in a mock interview.
a. Measure - This outcome was measured by scoring student mock interview performance
using a rubric. (direct)
b. Sample - 85 students were enrolled in EDCEP 120 during the spring semester; 34 student
feedback sheets were used to collect information.
c. Timetable - Mock interviews were conducted in April 2011. Students scheduled individual
dates in ACIC for a mock interview.
d. Forum - Following a mock interview, a mock interview feedback form was completed by
the Career Specialist who conducted the interview.
4) Describe the results of the assessment. (What do they tell you about student learning? What did you learn
about strengths and weaknesses of your unit?) If specific results are not available, describe the progress that
has been made on the initiatives included in the approved assessment plan.
Results learned from Outcome 1
As a result of meeting with a career specialist, students will be able to develop a plan of action
to choose a major or career.
A question was added to the ACIC user survey to measure outcome 1. The question on the
survey read as follows: “As a result of meeting with a Career Specialist, I was able to develop
a plan of action to choose my major and career.”
There were 203 students who completed the user survey, and the responses to this question
were:
a. Strongly agree
34
17%
b. Agree
96
47%
c. Neutral
60
30%
d. Disagree
6
3%
e. Strong disagree
4
2%
f. No response
3
1%
Based on the responses, 64% of the students surveyed indicated either “strongly agree” or
“agree” that they were able to develop a plan of action. I would have hoped for a higher
number, however, a plan of action might be difficult to define depending on where a student
happens to be in the career development process. Interestingly, earlier questions in the survey
(question 2 and question 4) suggest students were pleased and able to receive the information
needed.
o Survey question 2 asks: “Was the Career Specialist you worked with able to
answer all of your questions?” There were 158 student responses (78%)
indicating “all” questions were answered and 32 (16%) indicated “most” were
answered. This totals 190 students, or 94% of the students who responded felt as
if all or most of their questions were answered.
o Survey question 4 asks: “Overall, please rate your satisfaction with your ACIC
visit.” There were 153 student responses (76%) indicating they found the visit
“very helpful” and 32 responses (16%) found it “helpful”. This totals 185
students, or 92% of the students who responded found their visit helpful or very
helpful.
Limitations of outcome 1:
 All students who visited ACIC did not participate in the survey.
 All students who visited ACIC did not receive the opportunity to complete the survey.
Some emails provided by students were invalid.
 All students who visited ACIC were surveyed, regardless of whether or not they met
with career specialist. Although most students did have individual contact with a career
specialist, if students dropped by the office for a quick answer or to pick up information,
they most likely did not have the opportunity to develop a plan of action.
 Students using ACIC services enter the office at varying level of career preparedness and
background. Some high schools have strong career programs for students while others
do not.
 Students using ACIC services enter the office at varying of career readiness. Some
students voluntarily use the service while others are required to use our office as part of
a class assignment. At times, those who are required to complete an assignment are not
as engaged in their meeting.
 ACIC user survey and Academic Majors Fair survey are sent from the account acic@kstate.edu. Some students may have thought they received two of the same emails.
 In the fall semester of any academic year, there are new Career Specialists who are
learning how to perform the daily functions associated with the position of career
specialist. The skill level of the Career Specialist is directly related to his or her own
training and tenure with ACIC. With continued employment, Career Specialists become
an increasingly valuable asset to the office. The comfort level of serving in a counseling
role, and knowledge of K-State services and programs increases as a student remains
employed as a Career Specialist.
Results learned from Outcome 2
As a result of attending the Academic Majors Fair, students will be able to identify the
different majors available at K-State.
A question was added to the Academic Majors Fair follow up survey to measure outcome 2.
The question on the survey read as follows: “As a result of attending the Academic Majors Fair,
I learned more about majors available at K-State.”
There were 205 students who completed the follow up survey, and the responses to this question
were:
Agree
188
(91.71%)
Disagree
17
( 8.29%)
Limitations of outcome 2:
 All attendees did not participate in the survey
 All attendees who visited the Academic Majors Fair did not receive the opportunity to
complete the survey. Some emails were invalid.
 ACIC user survey and Academic Majors Fair survey are sent from the account acic@kstate.edu. Some students may have thought they accidentally received two of the same
emails.
 There was a slight delay from the date of the Academic Majors Fair survey to the date
the survey was sent to students. Data collected from the student ID card is not
immediately available for use. It must first be processed and data must be extracted
from the iSIS system. The delay between the event and the sending of the survey could
cause a lower response rate among Fair attendees.
Results learned from Outcome 3
After taking the EDCEP 120 class, students will be able to assess the strengths and
weaknesses of their interviewing skills by participating in a mock interview.
Students enrolled in EDCEP 120, Academic and Career Decisions, were assigned a mock
interview as a part of their course requirements for the class. In the spring semester of 2011
there were 85 students enrolled in the class. Some sections of the course have a start date that is
late in the semester. Students enrolled in these sections were selected to measure this outcome.
This was done to give time for career specialists who were administering the interviews to
practice and become comfortable evaluating their peers in the interview process.
Mock interview feedback forms were collected for 34 students. Students were evaluated in six
categories:
 Body Language
 Appearance
 Self-expression/style
 Enthusiasm and interest
 Experience
 Maturity
Students were given the opportunity to interview with their company of choice, thus the content
of the interview was not evaluated. The categories listed above focus on the affective parts of
the interview process.
In the above categories, students could be given a rating as follows:
 Excellent = 4
 Good = 3
 Satisfactory = 2
 Unsatisfactory = 1
Immediately following the interview, career specialists were able to provide initial, verbal
feedback about key areas where the student performed well, and key areas a student may need to
develop. After the student was dismissed, the career specialist completed a feedback
form/rubric with more detailed advice. The completed form was then given to the course
instructor. Instructors had individual meetings with each student to finish the semester, and
results from the mock interview were one of the discussion topics for the meeting.
The high score possible for the mock interview was 24 points. The mean score for the 34
students who were assessed was 19.47. The range of scores was from 24 to 11.
The average score of 19.47 indicates that most students performed about average on the mock
interview assignment. Students had an opportunity to hear a lecture about proper interview
techniques, preview the feedback form, and review possible questions that may be asked during
the interview. However, I would have hoped student score averages would have been a bit
higher.
In addition to the ratings given, career specialists were able to make comments regarding
student performance in the mock interviews. Some sample comments are listed below:
 “Good handshake, try to keep better eye contact, and slow down a little bit”
 “Very good energy, did a good job of turning negative questions into positive answers”
 “Remember to use STAR (Situation/Task, Action, Result). This is what should be
worked on most!”
 “I know you have a lot of experience at Michael’s but use other jobs/activities to come
up with other examples”
 “Very calm and composed! Work on sustaining eye-contact a little better”
 “Excellent job describing your experiences”
 “You don’t want to come off as impatient, so be careful. Also try to devise strategies
that will help you calm down the nerves. (Practice always helps with this.)”
The comments from the career specialist demonstrate valuable feedback that was given to the
students who interviewed. This assignment is worthwhile in helping students learn strengths
and weaknesses in an interview setting and practice a needed skill needed in the career journey.
Limitations of outcome 3:
 Although 85 students were enrolled, only 34 feedback forms were collected. It would
have been helpful to have evaluated all students in the class.
 Some students have had interview experience, and some students have not. Higher
scores on the Mock Interview may have come from students who have had more
experience.
 There were multiple Career Specialists who led the interviews and provided feedback to
those completing the mock interview. With multiple reviewers, there were likely
differences in the way evaluations were considered.
 The maturity level of the Career Specialist was also a factor in the evaluation process.
As indicated previously, the skill level of the Career Specialist is directly related to his
or her own tenure with ACIC. Those who have worked in this position longer feel more
confident providing feedback to their peers that is sometimes difficult or uncomfortable
to discuss.
5) Describe the actions and/or revisions that were implemented in response to the previous year’s
assessment results and the effects on student learning observed on this year’s SLOs.
This question is not applicable since this is the first assessment year for the Academic and
Career Information Center.
6) Describe the process by which staff reviewed the results of this year’s SLOs and the actions and/or
revisions that are planned in response to the assessment results. (include changes that may be made to unit
SLOs or to the general assessment strategy)
Outcome 1: As a result of meeting with a career specialist, students will be able to develop a
plan of action to choose a major or career.
Results of outcome 1 were reviewed by the coordinator and shared and discussed with ACIC
career specialists at a staff meeting.
Based on the responses, 64% of the students surveyed indicated either “strongly agree” or
“agree” that they were able to develop a plan of action. As mentioned earlier in this report,
developing a plan of action takes time. In addition, a plan of action is unique to each student we
assist. No revision is needed in the training materials to address this outcome. However, this
information is a good reminder for career specialists to consider during the on-going training
activities for this position.
It is also important to note that 94% of the students who responded to the user survey felt as if
most or all of their questions were answered, and 92% of respondents found their visit to ACIC
helpful or very helpful.
Outcome 2: As a result of attending the Academic Majors Fair, students will be able to
identify the different majors available at K-State.
Results of outcome 2 were compiled and reviewed by the Academic Majors Fair Planning
Committee to consider changes and adjustments to the event for the following year.
There were 205 students who completed the follow up survey, and 92% of the students who
responded indicated they learned more about majors available at K-State as a result of attending
the Academic Majors Fair.
The Academic Majors Fair Planning Committee is pleased with those results, and there are no
actions or revisions currently planned in response to these results. The Committee will continue
to work to improve the event to aid in the retention efforts at K-State.
Outcome 3: After taking the EDCEP 120 class, students will be able to assess the strengths
and weaknesses of their interviewing skills by participating in a mock interview.
Results of outcome 3 were compiled and reviewed by the coordinator to consider changes and
improvements to the instruction methods for EDCEP 120. Results will also be shared with the
ACIC staff in the 2011-2012 academic year.
Adjustment will be made to the mock interview feedback form and teaching materials for the
2011-2012 academic year with regard to student attire. However, the adjustments are not
directly related to the SLOs from 2010-2011.
The most significant concern noticed during the mock interview process this semester was the
inappropriate attire worn by those who interviewed. Although appropriate attire was discussed
by course instructors, student interpretations of that discussion were vastly different to the
instructor and interviewer expectations.
The mock interview materials (both class materials and feedback form) will be modified giving
visual representation of expected appropriate interview attire.
7) If needed, revise the department’s Assessment Plan for the next academic year. Attach your
assessment plan to this report.
There will be no changes to the outcomes for the Academic and Career Information Center for
the 2011-2012 academic year. Since 2010-2011 is the first assessment cycle for ACIC, it will
be helpful to compare learning outcomes for two academic years in order to establish a baseline
for subsequent reports.
Download