2013-2014 Annual Program Assessment Report

advertisement
2013-2014 Annual Program Assessment Report
Please submit report to your department chair or program coordinator, the Associate Dean of your College, and to
james.solomon@csun.edu, director of assessment and program review, by Tuesday, September 30, 2014. You may submit a
separate report for each program which conducted assessment activities.
College: Health and Human Development
Department: Kinesiology
Program: Exercise Physiology, Applied Fitness, General Studies, Physical Education, Dance
Assessment liaison: Teri Todd
1. Overview of Annual Assessment Project(s). Provide a brief overview of this year’s assessment plan and process.
The Department of Kinesiology assessment plan was to implement one direct assessment comparing an entry level core course
(KIN 200) to a capstone course (KIN 477). Additionally within this assessment classes taught by tenure-track faculty and lecturers
were compared, as well as a variety of class sizes. This assessment was a qualitative analysis using an integrated question and
assessed with a validated rubric. The purpose of this embedded assessment was to focus on SLO # 1: Apply an integrated
kinesiological approach to encourage the adoption of healthy and physically active lifestyles, across diverse populations.
Students were asked to answer a comprehensive essay question designed to assess content knowledge, writing, and analytical
skills.
The department of Kinesiology participated in the HHD initiative to assess comportment and cultural competence of students in
service learning and internship placements. A survey was developed by the HHD Assessment Committee and sent to faculty in all
options. Faculty members asked precepts to evaluate the comportment of students in service learning and internship
placements by completing the survey. The completed surveys were sent to the HHD Assessment Coordinator for analysis.
1
2. Assessment Buy-In. Describe how your chair and faculty were involved in assessment related activities. Did department
meetings include discussion of student learning assessment in a manner that included the department faculty as a whole?
Assessment was put on the monthly department meeting agendas upon request, time was provided for the Assessment Liaison
to update faculty on assessment projects and information gained through University and College assessment meetings. The
Assessment liaison reported on college assessment initiatives and explained the planned assessment project for the department
for 2013-2014.
Four faculty members serve on the Curriculum/Assessment committee. The faculty members represent different options within
Kinesiology and act as conduits of information regarding assessment and are instrumental in gathering assessment data. The
Assessment liaison is a member of this committee. The committee, as well as other faculty, evaluated student essay responses
for the evaluation of SLO 1.
The majority of faculty who had service learning projects in their classes/programs participated in the comportment and cultural
competence survey for the College of HHD. Faculty distributed surveys to precepts overseeing students in a variety of
placements. Surveys were collected both on paper and electronically depending on the preference of the faculty and precepts.
Department faculty as a whole discussed the impact of part-time instructors in core Kinesiology classes. Of particular concern
was understanding if core content in each course was being covered and the quality of instruction. As our department grows it is
imperative that we utilize non-tenure track instructors so students can move through the program in a timely fashion. A group of
faculty volunteered to meet and discuss this problem in detail. It was proposed that core classes with multiple instructors have
one designated tenure track faculty who will meet with all instructors teaching the course and ensure material is similar across
all sections. This is to be implemented in the 2014-2015 academic year.
3. Student Learning Outcome Assessment Project. Answer items a-f for each SLO assessed this year. If you assessed an additional
SLO, copy and paste items a-f below, BEFORE you answer them here, to provide additional reporting space.
2
3a. Which Student Learning Outcome was measured this year?
Kinesiology has four options and one major (Athletic Training). Each option aligned their required undergraduate courses with
the Program SLOs. This year we measured SLO 1: Apply an integrated kinesiological approach to encourage the adoption of
healthy and physically active lifestyles, across diverse populations.
Program
Student Learning Outcomes (P-SLOs)
1. Apply an integrated kinesiological approach to encourage the adoption of healthy and
physically active lifestyles, across diverse populations.
2. Apply evidence based practices to enhance the study of human movement.
Undergraduate
3. Demonstrate competent problem solving strategies through intentional practices.
4. Demonstrate knowledge of kinesthetic forms, processes and structures as they apply
to the personal expression and culture of human movement.
3b. Does this learning outcome align with one or more of the university’s Big 5 Competencies? (Delete any which do not apply)





Critical Thinking
Oral Communication
Written Communication
Quantitative Literacy
Information Literacy
This SLO is directly aligned with the University’s commitment to supporting diversity through the cultivation of an
understanding of human movement, motivation, and lifestyles across a variety of populations. Health and physical activity are part
of each person’s ecosystem, incorporating their environment, culture, religion, socio-economic status, ethnicity, age, and past
experience into their perception of movement. Students are instructed on how these elements impact lifestyle and are then
challenged to promote interesting, motivating, and appropriate lifestyle options. KIN 200 covers broad content including the
3
definition of kinesiology, historical development, three major types of movement, tenets of a liberal education including scientific
inquiry, and an introduction to the sub-disciplines of kinesiology as well as the subjectivity of human movement. KIN 477 is designed
to enhance students’ knowledge of the development of human movement throughout the lifespan. Specific topics include
theoretical perspectives of human development, physiological maturation, analysis of measures to evaluate motor development,
and principles of motion and stability and how they can be used to assess quality of motor skill performance. Students are given first
hand opportunity through service learning projects to experience how diverse perspectives impact lifestyle and become familiar
with various aspects of diversity including culturally preferred practices, the impact of disability, and socio-economic challenges.
3c. What direct and/or indirect instrument(s) were used to measure this SLO?
This SLO was directly assessed though the evaluation of a short essay written by students in both gateway (KIN 200) and capstone
(KIN 477) Kinesiology classes. An essay prompt was provided. The essays were scored against a rubric designed to evaluate student
knowledge, critical thinking, and written communication.
Essay Prompt: Physical activity positively benefits the health of all people regardless of physical skill or ability. Healthy People 2020
challenges all people to lead a physically active lifestyle and The Centers for Disease Control and Prevention (CDC) has adopted
guidelines for living a physically active lifestyle. Explain what is meant by a physically active lifestyle and describe how you would
help people meet these guidelines from a kinesiological perspective. Keep in mind that the citizenry of the United States is made up
of diverse populations.
3d. Describe the assessment design methodology: For example, was this SLO assessed longitudinally (same students at different
points) or was a cross-sectional comparison used (Comparing freshmen with seniors)? If so, describe the assessment points used.
A cross-sectional approach was used to assess this SLO. One short essay prompt was given to students in two courses, one gateway
course KIN 200 and a capstone course, KIN 477. Data was collected over three semesters with a total of six KIN 200 classes and three
KIN 477 classes providing responses. Five hundred and ten responses were collected, 23% of the responses were scored by five
faculty members.
KIN 200, Foundations of Kinesiology, is taken within the first two semesters of the program and is designed to provide a
comprehensive overview of the interdisciplinary study of human movement and the importance of all sub-disciplines of kinesiology,
4
as well as explore the subjective experience of physical activity. KIN 477, Motor Development is primarily taken by students in their
senior year. This course addresses human movement across the lifespan from an ecological perspective which incorporates a broad
perspective of humanity including diverse populations and environments.
3e. Assessment Results & Analysis of this SLO: Provide a summary of how the results were analyzed and highlight findings from the
collected evidence.
The essays were assessed using a rubric designed to evaluate synthesis of information pertaining to the question, validity of the
information provided, quality of writing, flow within the essay, and completeness of the answer.
The signature assessment was collected over three semesters. During the first semester the essay was given to students as an extra
credit assignment at the time of the final exam in the Spring 2013 semester, one section of 60 students in KIN 200 and two sections
of 45 students in KIN 477 were included in the assessment process. The response rate was fairly low as an extra credit question,
therefore we asked instructors to include the question as part of a routine assignment or exam for the fall of 2013, two KIN 200 and
one KIN 477 classes participated. Responses were collected in the spring 2014 semester from four sections of KIN 200 and two KIN
477. All responses were collected at the end of each semester. The classes were taught by both tenure-track faculty and lecturers,
the responses were coded and the scorers were blind to the class and instructor. Number of students per section was fairly similar
with 60 to 69 students enrolled in each section of KIN 200, and 40 to 42 students in each section of KIN 477.
Each area of assessment was scored on a 3-point scale: unacceptable (1), acceptable (2), or exemplary (3). The percentage of
students in each category is shown below for both the gateway (KIN 200) and capstone (KIN 477) course. Scorers met and reviewed
the rubric. Five Kinesiology professors volunteered to score the responses. Scorers were given the opportunity to practice by
reviewing several responses, this continued until 85% reliability was obtained for all scorers.
The following graphs show the results for each criteria assessed. Results from KIN 200 are represented in blue (left column), KIN 477
in red (right column).
5
100
SLO 1 Criteria 2 - Synthesis of subject matter
Percent of responses
Percent of responses
SLO 1 Criteria 1 - Completeness of answer
80
60
40
20
0
Unacceptable
Acceptable
100
80
60
40
20
0
Exemplary
KIN 200
Unacceptable
SLO1 Criteria 4 - Quality of writing
Percent of responses
Percent of responses
100
80
60
40
20
0
6
Exemplary
KIN 477
SLO 1 Criteria 3 - Validity of facts
Unacceptable
Acceptable
Acceptable
Exemplary
100
80
60
40
20
0
Unacceptable
Acceptable
Exemplary
SLO1 Criteria 5 - Flow of writing
Perent of responses
100
80
60
40
20
0
Unacceptable
KIN 200
Acceptable
Exemplary
KIN 477
The analysis show that as students progress from gateway (KIN 200) to capstone (KIN 477) they gain factual knowledge showing
an increase in the validity of the information within the response (criteria 3) 50% of students in KIN 200 scored in the unacceptable
category compared to 0.08% of KIN 477 students. Technical aspects of answering the questions were difficult for many students,
however improvement was obvious with a trend of more responses scored as acceptable or exemplary in all criteria in KIN 477.
Quality of writing was acceptable for students in both the gateway and capstone course with the majority scoring in the acceptable
and exemplary categories (70% KIN 200 & 84% KIN 477). Flow of writing and synthesis of content were the most difficult criteria for
students. Synthesis of information challenged all students with only 21% of studens in KIN 200 and 43% of KIN 477 students scoring
in the acceptable and exemplary categories. Flow of writing was somewhat better, nevertheless only 60% of students in the gateway
course fell into the acceptable and exemplary categories, leaving 40% of KIN 477 and 51% of KIN 200 students at an unacceptable
level. Only half of KIN 200 students and 62% of KIN 477 completely answered the question.
The results were viewed within KIN 200 to better understand if student performance was affected by the presence of full time
tenure track faculty or lecturers. Responses were collected from six sections of KIN 200, three of the sections were taught be tenuretrack professors and three taught by part-time lecturers. Over the past three semesters 15 sections of KIN 200 have been offered, 4
7
(27%) of the sections were taught by tenure-track faculty and 11 (73 %) by lectures. Distribution of answers in the exemplary,
acceptable, and unacceptable categories was stable across all instructors.
Thus instructor credentials did not change how well the students responded to the essay question on any of the criteria. All sections of KIN 477
were taught by a lecturer, therefore no comparison was available. The table below shows the percentage of scored responses in each category
for each criteria.
Table 1: Percentage of scored responses per category
3f. Use of Assessment Results of this SLO: Describe how assessment results were used to improve student learning. Were
assessment results from previous years or from this year used to make program changes in this reporting year? (Possible changes
include: changes to course content/topics covered, changes to course sequence, additions/deletions of courses in program, changes
in pedagogy, changes to student advisement, changes to student support services, revisions to program SLOs, new or revised
assessment instruments, other academic programmatic changes, and changes to the assessment plan.)
Based on the results of the 2012-2013 assessment initiative a larger sample of responses was collected from more classes. Both
tenure track and lecturers were asked to participate in the project. The larger number of responses provides a representative
sample from which instructional decisions can be made with some confidence. The results of this assessment will be discussed by
the Curriculum/Assessment committee and presented to the faculty in the fall 2014 semester. A faculty discussion will be planned.
The results of the assessment show that writing mechanics are a struggle for many students in both the gateway and capstone
8
courses. The ability to synthesize information and write in a clear, concise manner is a meaningful skill and one that is desirable in
the workplace. Strategies to improve writing skills will be discussed.
4. Assessment of Previous Changes: Present documentation that demonstrates how the previous changes in the program resulted
in improved student learning.
5. Changes to SLOs? Please attach an updated course alignment matrix if any changes were made. (Refer to the Curriculum
Alignment Matrix Template, http://www.csun.edu/assessment/forms_guides.html.)
No changes were made.
6. Assessment Plan: Evaluate the effectiveness of your 5 year assessment plan. How well did it inform and guide your assessment
work this academic year? What process is used to develop/update the 5 year assessment plan? Please attach an updated 5 year
assessment plan for 2013-2018. (Refer to Five Year Planning Template, plan B or C,
http://www.csun.edu/assessment/forms_guides.html.)
Kinesiology used its five year plan (2013 – 2018) to establish a focus for the 2013-14 assessment process. KIN faculty assessed SLO 1
as planned in KIN 200 (gateway) and KIN 477 (capstone). The five year assessment plan informed the direct assessment work of this
past year which was to review KIN 200 to ensure the class addresses specific P-SLOs at the appropriate level and to monitor student
growth by contrasting the assessment to responses in KIN 477. The plan also compared student outcomes in different sections
taught by tenure track faculty and lectures. Consistency in information and presentation of material was of concern, the assessment
results showed that all results were similar across all sections of KIN 200.
9
7. Has someone in your program completed, submitted or published a manuscript which uses or describes assessment activities in your
program? Please provide citation or discuss.
Henige, K. (2012). Use of concept mapping in an undergraduate introductory exercise physiology course. Advances in Physiology Education,
38 (3), 229-234.
8. Other information, assessment or reflective activities or processes not captured above.
The CHHD assessment committee developed a survey to assess comportment and cultural competencies. The surveys were to be
completed for students completing internships in their junior or senior years. Supervising precepts completed the surveys. Ninetyseven surveys were completed for students in Kinesiology. The results were forwarded to the HHD College assessment coordinators.
The surveys included 14 statements on aspects of comportment and 10 for cultural competencies, precepts were asked to rate the
degree to which the student intern met that statement using a 5-point scale ranging from strongly disagree (1) to strongly agree (5).
COMPORTMENT COMPETENCIES:
DRESS & APPEARANCE
1. Complies with workplace dress codes
2. Maintains an appropriate workplace appearance
WORK HABITS
3. Is punctual (arrives and departs on time) and has regular (vs. irregular) attendance
4. Shows initiative and can begin/complete tasks effectively with minimal direction
5. Is dependable and completes tasks and deadlines in a timely manner
6. Adheres to agency/organization policies, rules, and regulations (e.g. works within boundaries set by supervisor and/or
management; adheres to chain of command for problems and decision making)
7. Demonstrates appropriate use of technology (cell phone use, texting, email, etc.)
8. Has ability to adapt to changing demands (copes well with unexpected problems/pressures)
10
PROFESSIONAL DEMEANOR/RAPPORT
9. Has a pleasant, positive demeanor
10. Demonstrates appropriate rapport with clients
11. Demonstrates appropriate rapport with colleagues & coworkers.
12. Is willing and able to work cooperatively and effectively with others.
13. Demonstrates willingness to consider and accept constructive criticism and feedback.
14. Is able to accept and implement instructions and/or suggestions offered by supervisor
CULTURAL COMPETENCIES
1. Does not discriminate against others.
2. Understands the codes of conduct expected at the work site.
3. Treats others with dignity and respect.
4. Recognizes cultural factors that affect health and/or well-being of others.
5. Demonstrates ability to interact effectively with people of different cultures.
6. Engages with community partners to promote a healthy environment and healthy behaviors.
7. Demonstrates ability to assess one’s cultural biases and assumptions.
8. Applies best practices supported by evidence based literature regarding the health and/or well-being of others.
9. Exhibits caring, compassion and empathy.
10. Effectively communicates in a culturally competent manner.
The majority responses were in the strongly agree and agree categories. Precepts rated all items in the comportment section as
strongly agree (5) for thirty-nine percent (39%) of the surveys, an additional 30% of the surveys received a mix of strongly agree (5)
and agree (4) ratings on all statements. However 31% of the precepts indicated that the intern at their facility had some degree of
difficulty with certain comportment items. Precepts reported problems (1-3 on scale) most frequently with punctuality and
dependability followed closely with lack of initiative, inappropriate use of technology, problems complying with dress code and
maintaining an appropriate workplace appearance, and willingness to consider and accept constructive criticism and feedback. Five
precepts noted problems with interns adapting to changing demands and adherence to agency policies and regulations.
11
Precepts rated the interns favorably for cultural competency. Seventy-seven percent of surveys were completed with strongly agree
(5) for all cultural competency items. An additional 18% of surveys were completed with strongly agree and agree (4-5). Only five
percent of the surveys indicated some difficulty in the cultural competence area.
Overall the survey results are favorable. Faculty, specifically the internship coordinator, will be made aware of potential problem
areas which may addressed prior to the start of the internship.
12
Download