2014-2015 Annual Program Assessment Report
Please submit report to your department chair or program coordinator, the Associate Dean of your College, and to james.solomon@csun.edu,
director of assessment and program review, by September 30, 2015. You may, but are not required to, submit a separate report for each program,
including graduate degree programs, which conducted assessment activities, or you may combine programs in a single report. Please identify
your department/program in the file name for your report.
College: Health and Human Development
Department: Physical Therapy
Assessment liaison: Witaya Mathiyakom
1. Please check off whichever is applicable:
A. X Measured student work.
B. X Analyzed results of measurement.
C. X Applied results of analysis to program review/curriculum/review/revision.
2. Overview of Annual Assessment Project(s). On a separate sheet, provide a brief overview of this year’s assessment activities, including:
 an explanation for why your department chose the assessment activities (measurement, analysis, and/or application) that it enacted
 if your department implemented assessment option A, identify which program SLOs were assessed (please identify the SLOs in full), in
which classes and/or contexts, what assessment instruments were used and the methodology employed, the resulting scores, and the
relation between this year’s measure of student work and that of past years: (include as an appendix any and all relevant materials that
you wish to include)
 if your department implemented assessment option B, identify what conclusions were drawn from the analysis of measured results, what
changes to the program were planned in response, and the relation between this year’s analyses and past and future assessment
activities
 if your department implemented option C, identify the program modifications that were adopted, and the relation between program
modifications and past and future assessment activities
 in what way(s) your assessment activities may reflect the university’s commitment to diversity in all its dimensions but especially with
respect to underrepresented groups
 any other assessment-related information you wish to include, including SLO revision (especially to ensure continuing alignment between
program course offerings and both program and university student learning outcomes), and/or the creation and modification of new
assessment instruments
OVERVIEW OF ANNUAL ASSESSMENT PROJECT(S)
For the Academic 2014-2015, the assessment plan focuses on SLO 3: Practice in an independent and interdependent role in providing
physical therapy services. For this purpose, the primary analysis of this SLO focused on the outcome of the PT 797 (Directed Comprehensive
Examination) and PT 798 I (Clinical Internship III) of our third (final) year students. For the Directed Comprehensive Examination course, we
examined the cumulative scores from all four sections of the exam. For the clinical internship, clinical instructors evaluated our students using the
Clinical Performance Instrument (CPI) created by the American Physical Therapy Association. The CPI scores during midterm and final
evaluations of each student were obtained. Additionally, we analyzed the CPI scores during the Clinical Internship I (PT 794I) and Clinical
Internship II (PT 796I) to identify any weaknesses in clinical performance of our students in the beginning and intermediate internships that we
should improve in preparation for their subsequent internships. The results of the secondary goal will be also be used for future assessment
project as well.
Primary Outcomes
1) Scores of directed comprehensive examination:
The distribution of the scores of directed comprehensive examination is shown in Figure 1. The mean and standard deviation of the directed
examination scores of this cohort are 86.77 and 5.76 points, respectively. Based upon the criteria (overall score ≥75% passes), all students
successfully passed the directed comprehensive examination (PT 797). These results suggest that our students have sufficient knowledge to
become entry-level physical therapists. Successful completion of this directed comprehensive examination is considered one of the criteria to
complete the DPT and is essentially considered a pre-requisite for becoming entry-level physical therapists. However, the directed
comprehensive examination was administered in a written format, which may not directly reflect our students’ ability successfully to perform clinical
skills required in the real world situation. Therefore, we analyzed the results of CPI scores provided by clinical instructors as briefly described
below.
Figure 1. Distribution of direction comprehensive examination scores of class
57. Based upon the criteria, all students passed the comprehensive exam.
2
2) CPI scores of final Internship (PT 798I):
2.1) The overall results of CPI scores as well as the results of individual 18 items were used to identify our student’s ability to perform as entrylevel physical therapists at the end of their final internship. The overall results of CPI suggest that at mid-term (the first eight weeks of 16 weeks
final internship), the majority of our students performed at below entry physical therapist level (Figure 2). However, at the end of final internship,
17 of 30 students performed at the entry physical therapy level, 10 of 30 between entry and beyond entry level, 2 of 30 at beyond entry level, and
1 of 30 at below entry level (Figure 2). These results suggest that at the end of the final internship, most students were able to perform as entrylevel physical therapists as identified by their clinical instructors working in the community. However, one student performed at the below entrylevel.
Figure 2. Overall Clinical Performance Instrument (CPI) scores of the final
internship (PT 798 I Clinical Internship III). Although the majority of students
performed at the below entry level (red block, left column) at midterm
evaluation, all but one student successfully reached the entry level (yellow
block, right column and above (green blocks, right column) at the end of final
internship.
3
2.2) Association between the CPI and comprehensive exam scores. The comprehensive examination scores of students at each level of clinical
performance as classified by the CPI scores at final evaluation were examined to see whether or not there were any association between the
knowledge level and clinical skills (Figure 3). The results suggest that students who well during the comprehensive examination also performed
well clinically. Interestingly, the students who performed at the below entry level also had the second lowest score on the comprehensive
examination, while the students who performed at the beyond entry PT level had relatively higher scores (Figure 3). Although this observed
association is inconclusive due to smaller number of students and other factors, these results may provide us with a unique window into how we
could “spot” a student who may not perform well clinically such that we can find ways to improve the student’s knowledge and clinical skills before
s/he gets to the final internship.
Figure 3. Mean (SD) of comprehensive examination scores based upon
overall performance in clinical practice as classified by clinical performance
instrument scores. The student who performed at the below entry level (red
bar) also had the second lowest score on the comprehensive exam, while
student who performed at the entry level (yellow bar) and above (green bars)
had higher scores.
2.3) The results of individual CPI items (Appendix 1-A) of the students who performed at below entry level indicated weakness in the area of
safety, communication, clinical reasoning, examination, evaluation, plan of care, procedures interventions, educational interventions,
documentation, outcomes assessment, and financial resources. The weaknesses of this student were reflected in the qualitative assessment
made by the clinical instructor. As a result, this student has to repeat his internship, and our faculty and clinical instructor continue to mentor his
performance towards the entry-level performance.
2.4) We also performed qualitative analysis of the comments made by clinical instructors for the area for further improvement (Appendix 1-B).
Based on the results of the qualitative analysis, four (4) areas of further improvement: 1) time management; 2) differential diagnosis; 3) manual
skills/therapy; 4) therapeutic exercise. The results of this analysis will be discussed in our faculty meeting to find ways to strengthen these skills.
In fact, we are in the process of internal review of our curriculum, and these results help us set the priority of our meeting.
4
Secondary Outcomes
1) CPI Scores of intermediate internship (PT 796I):
1.1) The overall results of CPI scores as well as the results of individual 18 items were used to identify our student’s ability to perform at the
advanced beginner to advanced intermediate level at the end of their intermediate internship. The overall results of CPI (Figure 4) suggest that all
students performed at the advanced beginner and beyond at midterm evaluation. Additionally, students continued to improve their performance
during their internship. At the end of their intermediate internship, the majority of students were at the beyond advanced intermediate level. These
results suggest that all students met their benchmark of this internship. However, the acceptable range of the CPI scores or benchmark for
intermediate internship is relatively large (2 intervals from advanced beginner to advanced intermediate range) and may not be sufficient to identify
the weaknesses of our students. Therefore, a relatively narrower range of benchmark may be needed.
Figure 4. Overall Clinical Performance Instrument (CPI) scores of the
intermediate internship. In this internship, students were expected to perform
at the advanced beginner to advanced intermediate levels. All students
performed at the advanced beginner and beyond and none of them performed
at the below the benchmark of advanced beginner level.
1.2) Qualitative analysis of the comments made by clinical instructors for the area for further improvement of this cohort (Appendix 2-B) suggested
six areas of further improvement area: 1) Time management; 2) Differential Diagnosis; 3) Evaluation; 4) Screening; 5) Therapeutic Exercise; 6)
Self-confidence. Some of he areas that need further improvement of this group seem to overlap some those recommended for the final internship:
time management, differential diagnosis, evaluation and therapeutic exercise. The results of this qualitative analysis are also consistent with
students’ feedback based on their experiences after their clinical internships.
5
2) CPI scores for beginner internship (Class 59):
2.1) The overall results of CPI scores as well as the results of individual 18 items were used to identify our student’s ability to perform at the
advanced beginner to advanced intermediate level at the end of their intermediate internship. In this internship, students were expected to
perform at the advanced beginner to intermediate level. At midterm, 11 of 29 students performed at the below advanced beginner level while 18
performed at the advanced beginner to beyond. However, at final, all students performed at the advanced beginner and beyond and none of them
performed at the below advanced beginner level (Figure 5).
Figure 5. Overall Clinical Performance Instrument scores of the beginning
internship. Although some students performed at the below advanced beginner
level (red block, left column) at midterm, all students met the benchmark of
advanced beginner level (yellow block, right column) and above (green block,
right column).
2.2) Qualitative analysis of the comments made by clinical instructors for the area for further improvement of this cohort (Appendix 3-B) suggested
six areas of further improvement area: 1) Time management; 2) Differential Diagnosis; 3) Evaluation; 4) therapeutic procedures for acute
care setting, critical care setting, neurological patients; 5) Therapeutic Exercise; 6) Self-confidence.
6
Plan to apply the results of this assessment to improve students’ learning outcomes
1) The results of this assessment will be discussed in our faculty meeting to ensure that all faculty members are aware of the strengths and
weaknesses of our students’ performance in the clinic. In fact, our department is in the process of curriculum review, and the results of this
assessment will help us set the priority of the focus for the review. Informally, some faculty members teaching clinical courses have brainstormed
the ideas for improving or refocusing the contents to emphasize the psychomotor aspect of the courses. The implementation plan for changing or
refocusing of the contents will be formally formalized such that we can assess the outcomes of the modifications.
2) As discussed in the results section, we believe that successful performance of students in the beginning and intermediate internships may also
be attributed in part to the relatively wide range of benchmark set by APTA. As a result, we may not be able to detect the problem in our students’
clinical performance. Therefore, we may need to narrow the ranges of the acceptable benchmarks for these two internships. Currently, we plan to
perform a series of hypothetical analysis of the CPI scores using different ranges of the benchmark to see how they may influence the outcomes
of students’ performance. An example of these hypothetical analyzes is shown figure 6. The results of these analyzes will be interpreted in light
of students’ overall performance in the program before they will be implemented.
Figure 6. An example of hypothetical analyzes of overall CPI scores during
the intermediate internship. In this hypothetical analysis, the benchmark was
elevated to between intermediate and advanced intermediate level as
compared to between advanced beginner to advanced intermediate level
Figure 4. As a result, a few students and one student performed at the level
below the benchmark (red block, right column) at final evaluation of their
internship.
3) We plan to investigate the characteristics of student’s profile at admission (GPA, prerequisite courses, etc.) to determine the factors contributing
to student’s performance in our program (GPA, CPI scores, Comprehensive examination scores, etc.) and learning outcomes such that we can
improve our student advisement, admission criteria, curriculum, teaching and learning strategies, and assessment plans.
7