How Do You Know What's Working:
Accountability
University APE Programs
Barry Lavay - CSU, Long Beach
Rebecca Lytle - CSU, Chico
Hyun-Kyoung Oh - CSU, San Bernardino
National AAHPERD Conference, April 1, 2011
National AHHPERD 2011
Accountability &
Program Effectiveness



Increased National & State attention regarding the
assessment of teacher training of candidate
accountability & program effectiveness
2007 California Commission on Teacher
Credentialing (CCTC) adopted new accreditation
practices for California State University (CSU)
Annual basis all CSU institutions are required to
collect data related to candidate competence &
program effectiveness
National AHHPERD 2011
Mention Program Assessment
Negative Reaction
Can be positive:
 Articulate program to your
students
 Capture systematic plan on paper
 Informed programmatic decision
based on data- is it working?
 Reflection: determine where there
are gaps-needs
 Consistency among faculty in the
program assessment &
implementation
National AHHPERD 2011
Purpose & Outline

Examine 3 CSU APE assessment plans





Share samples of each assessment plan
Share insights such as



Barry Lavay, CSU, Long Beach
Rebecca Lytle, CSU, Chico
Hyun-Kyoung Oh, CSU, San Bernardino
How has it helped to make informed programmatic
decisions?
Challenges that exist
Audience discussion & questions: What are you
doing?
National AHHPERD 2011
Samples of Assessment Plan

Contextual information




Signature assignments examples with



APE mission statement
Program student learning outcomes (SLO) with
signature assignments
Linkage to university, state (CCTC) & national
(NCATE) standards
Grade rubric -data collection & programmatic candidate
decisions
Data collected at key transition points of program
course work
Examples of additional assessment data

e.g., APE candidates’ post exit survey & results
National AHHPERD 2011
CSULB APE Specialist Coursework
Complete the following 27 units of coursework:
 KIN 320
Adapted Physical Education (3)
 KIN 427/527 Physical and Motor Assessment (3)
 KIN 387*
Physical Activity for the Disabled (3)
 KIN 388*
Program Planning and Instruction in APE
 KIN 489A
Fieldwork in APE (6)
 EDP 350
Education of Exceptional Individuals (3)
*Please note: substitute KIN 537 -KIN 387 & KIN 638 for KIN 388
Select two courses from the following with professor permission:
 KIN 526
Applied Behavior Analysis in Physical Education (3)
 EDP 405
Positive Strategies for Classroom Management (3)
 EDSP 577
Instruct Methods & Strategies Mild/Mod Disabilities (3)
 EDSP 578
Instruct Methods & Strategies Mod/Severe Disabilities (3)
 LING 151
Fundamentals of American Sign Language I (3)
National AHHPERD 2011
CSULB APE Mission Statement
Promote school improvement & advocate for all
individuals including those with disabilities by providing a
quality physical education teacher training program
Train APE Specialists to:
 Deliver services in the public schools & community (e.g.,
direct service, consultation, & advocate).
 Design & implement appropriate instructional programs
for individuals with disabilities
 Demonstrate knowledge of the significant role physical
activity can play toward the social responsibility of
individuals with & without disabilities
 Engage in various life long student learning professional
growth opportunities related to APE
National AHHPERD 2011
CSULB APE Assessment Plan
Overview
Next slide captures & identifies:
 6 APE SLOs
 Specific APE course when each SLO is
taught
 When each SLO assignment will be
collected & analyzed
 How each SLO links to conceptual
framework, & state standards
One chart identifies the program
National AHHPERD 2011
CSULB APE Assessment Plan
National AHHPERD 2011
Data Collected Annually

Evolved over time to assessment plan today

Since 2008 systematic & uniform College
(APE) program assessment procedures to
determine effectiveness & student learning
Conducted based on following measures:





6 APE identified SLOs (2004)
Post exit survey (2006)
APE Portfolio Notebook (2000)
Exit interview (2000)
National AHHPERD 2011
Student Learning Outcomes
What is Highly a Qualified APE Teacher?

Outcome 1: Identify & cite current legal mandates including federal &
state legislation pertaining to the PE for individuals/disabilities
including the design of an IEP)

Outcome 2: Create an assessment report in PE appropriate for
individuals/disabilities

Outcome 3: Construct an appropriate PE instructional program for
individuals/disabilities that includes the design of a unit plan

Outcome 4: Construct & implement a positive behavior management
plan in PE appropriate for individuals/disabilities

Outcome 5: Teach, analyze, & reflect PE lessons (e. g., lesson plan,
AIM coding) for individuals/disabilities

Outcome 6: Articulate an APE philosophy that includes the essential
skills & content knowledge to teach PE to individuals/disabilities
National AHHPERD 2011
SLO
Student Learning Outcomes
Course
Signature Assignment
Signature Assignment
Description of the Assignment
SLO 1: Identify & cite current legal mandates including
federal and state legislation pertaining to the physical
education for individuals with disabilities including the design
of an individualized educational program (IEP) for
individuals with disabilities.
Kin 387: IEP
Plan & write a motor IEP based on the test data collected on the
assigned child with a disability & include a present level of
performance statement with annual goals benchmarks, projected
dates, & teaching methods
SLO 2: Create an assessment report in physical education
appropriate for individuals with various disabilities
KIN 489A: Assessment
Report
Plan & write an Assessment Summary Student Report on a child
with a disability based on the test data collected that includes
background information, test data information, and program
activities.
SLO 3: Construct an appropriate physical education
instructional program for individuals with disabilities that
include the design of unit plan
KIN 388: Unit Plan
Plan & write a physical education unit plan designed for
children with disabilities that includes; background information,
safety procedures, rules/routines, facilities/equipment,
bibliography, CA content standards, behavioral objectives, block
plan with specific modifications & methods for evaluation.
SLO 4: Construct & implement a positive behavior
management plan in physical education appropriate for
individuals with disabilities.
KIN 387: Behavior
Management Plan
Plan & implement a behavior management plan based on a
specific child with disabilities that includes: the plan with rules,
routines, consequences; implementation of the plan with a
charting system to collect behavioral data, & reflection of what
was learned implementing the plan.
SLO 5: Teach, analyze, & reflect physical education lessons
(i. e., lesson plan) for individuals with disabilities
KIN 489A: Teaching Plan,
Code, Reflection
Teach, code, analysis & reflect on instruction for children with
disabilities that includes: a lesson plan, an analysis of the
teaching based on the CSULB AIM coding system & a reflection
of their teaching strengths and areas of improvement.
SLO 6: Articulate an APE philosophy that includes the
essential skills and content knowledge to teach physical
education to individuals with disabilities
KIN 388: Philosophy
Formulate & Write an APE philosophy to better guide teaching
that includes; an introduction with needs and justification, APE
content knowledge that includes the major areas & essential
skills of APE, and a summary section
National AHHPERD 2011
College of Education
Assessment Scale Equivalent
Uniformity point system among programs at CSULB
4
Exceeds Expectations
3
Meets Expectations
2
Meets Some Expectations
1
Does Not Meet Expectations
0
Can’t Score
Tie College of ED- 4 point assessment scale with APE Signature
assignments grade rubrics
20 -18
(4) Exceeds Expectations
17-16
(3) Meets Expectations
15-14
(2) Meets Some Expectations
13-12
(1) Does Not Meets Some Expectations
0-11
(0) Can’t Score
National AHHPERD 2011
Behavior Management
Rubric Example Assignment
SLO #4 Construct & implement a positive
behavior management plan in physical
education appropriate for individuals with
disabilities
 Signature Assignment Rubric follows
National AHHPERD 2011
Behavior Management Plan Grade Rubric - SLO 4
Rubric - SLO 4
20 -18 (4)
17-16
(3)
15-14
(2)
13-12
(1)
0-11
(0)
14 minimal
requirement
Categories/
Points
Does not meet
expectation
Inconsistently
meets expectation
Meets some
expectations
Exceeds or
Consistently meets
expectation
Design Plan
0-10 pts
No or little design
that includes; rules,
consequences,
routines, transitions,
methods to increase
& decrease
behaviors
& chart
(0-6)
Inconsistent design
that includes; rules,
consequences,
routines, transitions,
methods to increase
& decease behaviors
& chart
(7-8)
Clear, consistent,
design that includes:
rules, consequences,
routines, transitions,
methods to increase &
decease behaviors &
chart
(9-10)
Implement
Plan
0-5 pts
No or little overall
step-by-step
description of the
implementation of
the
of the plan &
charting
collection of the
behaviors
(0-1)
Inconsistent or little
overall step-by-step
description of the
implementation of
the
of the plan &
charting
collection of the
behaviors
(2-3)
Clear, consistent,
distinct
overall step-by-step
description of the
implementation of the
of the plan & charting
collection of the
behaviors
(4-5)
Reflection
Plan
Writing
Content
0-5 pts
No or little
reflection of plan
effectiveness &
changes/
recommendations
not a well organized
quality professional
plan that lacks clear
grammar &
sentence
structure that is easy
to follow
(0-1)
Inconsistent
reflection of plan
effectiveness &
changes/
recommendations
inconsistent
quality professional
plan with
inconsistent
grammar & sentence
structure that is easy
to follow
(2-3)
Clear consistent
distinct
reflection of plan
effectiveness &
changes/
recommendations
clear quality
professional plan
grammar & sentence
structure that is easy
to follow
(4-5)
Examine The Data

Next slide represents a bar graph of the overall
% comparisons among all students who
completed the 6 APE SLO in AY 08-09
4 point scale
4
3
2
1
0
Exceeds Expectations
Meets Expectations
Meets Some Expectations
Does Not Meet Expectations
Can’t Score
National AHHPERD 2011
AY08-09 SLOs Comparison
Adapted Physical Education
N=45
100.00
90.00
80.00
70.00
3.00 SLO1
60.00
%
3.29 SLO2
2.88 SLO3
50.00
3.00 SLO4
3.50 SLO5
40.00
3.25 SLO6
30.00
20.00
10.00
0.00
4 Points
3 Points
2 Points
National AHHPERD 2011
1 Point
0 Point
Exit Interview Meeting
Candidates complete the following:
Post exit survey
 APE Portfolio notebook - examples of work
 Exit interview

Post exit survey completed on “Survey Monkey”:

Next slide contains responses of candidates
from 2008-2010 regarding questions 10 –
“How effective the APE program was in helping
them to attain competences in specific areas related
to APE teaching”
National AHHPERD 2011
National AHHPERD 2011
CSULB APE Resources

APE program-related assessment
documents located on College of Ed site
http://www.csulb.edu/colleges/ced/accreditation/assessment/docs_by_type.html#plans

For more information on the CSULB
APE Specialist program
www.csulb.edu/APE
National AHHPERD 2011
Chico State
School of Education
Mission School of Education

Mission of the CSU, Chico School of Education … are
called to imaginative citizenship and effective service to
all students, their families and communities to improve
the quality of life through education. These educators are
informed decision-makers who can work collaboratively
to apply the best research-based practices in their fields
to enhance and support the learning, growth and welfare
of their students…

The professional education programs at CSU, Chico
inspire their candidates to accept the challenges to
remain reflective practitioners, serve as agents of change,
exercise responsible leadership, model lifelong learning,
and contribute to our democratic society.
NCATE Conceptual Framework

The conceptual framework was developed to ensure coherence
among curriculum, instruction, field experiences, clinical
practice, and assessment across all programs in the School of
Education.

Course syllabi are aligned to the conceptual framework, and
faculty members use the framework and standards in instruction
to address course outcomes, the content of their courses,
assignments and assessments.

Key assessments are aligned to the conceptual framework to
provide data to monitor each candidate's development,
candidate proficiencies, and professional dispositions.

To determine each program's effectiveness in supporting
candidates in this process.
Mission Statement for APE

The mission of the adapted physical activity
program is to train professionals based on
evidence based research to be effective
practitioners for individuals with disabilities
in a variety of settings across the lifespan.
National AHHPERD 2011
7 Candidate Proficiencies from
Conceptual Framework

Effective Practice





Reflective Practice


Subject Matter Knowledge
Pedagogical/Professional Practice
Diversity
Assessment
Reflection
Engaged


Collaboration
Civic Engagement
Entrance






Candidate has Subject Matter Competency in
Physical Education/Adapted Physical
Education
Minimum GPA in APE content 3.0
Pre-requisites to Credential Program
CBEST
Letters of recommendation/transcripts, etc
Supports During




APE Advisor & Education placement coordinator
Education Advisor
Cooperating Teacher
University site supervisors
Transition Process (Chico)








Mandatory advising with PETE/APE program
Subject Matter Competence, Teacher
Dispositions (upon entrance to student
teaching)
Candidate Field Observations
Candidate competencies reviewed (2+ points)
PACT
Portfolio Completion – Systematic Observation
Exit Interview
Post Survey
National AHHPERD 2011
Data Collected to Ensure
Candidate Competency






Key Assessments throughout coursework
Competency form reviewed at entry,
midpoint, exit
PACT – Performance Assessment for CA
Teachers (student teaching)
Portfolio Review
Exit Interview
Exit Survey
Rubric for Dispositions
Key Assessments
Sample Criteria for Portfolio
Key Assessment:
Adapted Physical Education Portfolio
Semester
Exemplary
Met
Criteria
Still in
progress
Comments
Fall 09
5
0
3
3/3 in progress completed
S10.
Spring 10
6
6
6
2/6 in progress complete. 1 in
F -10, 1 in S-11
Fall 09 – completers
PE/APE 1
APE only 4
Spring 10 – completers
PE/APE 5
APE only 7
Sample: Competency Form
Name of Student______________________________________
Cooperating Teacher___________________________________
University Coordinator_________________________________
APE Coordinator ______________________________________
DEMONSTRATES EFFECTIVE BEHAVIOR MANAGEMENT
TECHNIQUES
E
C
Date:
MC
Implement strategies to maintain and increase appropriate behaviors by using such techniques as
modeling, reinforcement principles, group contingencies, and recent technology.
Implement strategies to decrease or extinguish inappropriate behaviors by using appropriate
techniques of applied behavior analysis.
Identifies and modifies environmental factors that contribute to inappropriate behaviors.
Documents behavioral changes through a variety of data collection techniques.
Utilizes applied behavior analysis research and standards of practice to elicit desired behaviors and to
promote the acquisition of motor skills.
Student has clearly defined classroom rules and management procedures.
Cooperating teacher observes behavior management techniques.
Coding procedures for documenting behavior.
Sample reinforcement plan.
Use of ABC form.
Student can identify the communicative function of a behavior and determine appropriate
intervention strategy.
Sample for one of the student competencies for the APE specialist credential
N
M
N
O
Candidate Competence by Program
Website Resources for CSU, Chico



School of Education
http://www.csuchico.edu/soe/
Chico NCATE Page
http://www.csuchico.edu/ncate/
Kinesiology Department
http://www.csuchico.edu/kine/index.shtml
CSUSB APE Specialist Credential
Mission Statement

To prepare effective teachers and to become a recognized
leader in the field of adapted physical education.

Effective APE Specialists will be able to:




Write IEP goals and benchmarks, design lesson plans, and
implement appropriate instructional programs for individuals with
disabilities
Apply strategies for managing inappropriate behavior caused by
organic and environmental and factors in individuals with
disabilities in physical activity settings
Appreciate health, safety, and liability considerations as it relates to
standards of practices in adapted physical education
Employ measurement and evaluation procedures for assessing
movement performance in individuals with disabilities
National AHHPERD 2011
CSUSB APE Specialist Credential
Program Overview



KINE 311- Introduction to APE (4)
KINE 330 – Movement skill
assessment for individuals with
disabilities (4)
KINE 311 Topics in APE Series (total
8 units from the followings:






KINE 311a- Adapted aquatics (2)
KINE311b- Physical disabilities (2)
KINE 311c- Disabilities in communication
(2)
KINE 311d- Disabilities in learning (2)
KINE311e-Severe & Profound disabilities
(2)





KINE 311- Introduction to APE
(4)
KINE 330 – Movement skill
assessment for individuals with
disabilities (4)
KINE 332- Nature of disabilities
(4)
KINE 333- Applied Behavior
Management (4)
KINE 334- Seminar in APA (4)
ESPE 530- Psychology and education
of exceptional individuals (4)
National AHHPERD 2011
CSUSB APE Specialist Credential Candidate
Assessments/Program Effectiveness Plan




Overall Candidate Assessment Plans
Candidate Learning Outcomes
Comprehensive Exit Exam
Exit Interview
National AHHPERD 2011
Candidate
Learning
Dispositions Pedagogical
Knowledge
Content
Knowledge
Standard
CSUSB APE Specialist Credential Candidate
Assessments/Program Effectiveness Plan
Initial Indicators
Midpoint/Entry to
Clinical Practice
Exit from Clinical
Practice
Program Completion/
Follow-up
Minimum GPA
-Valid CA PE
credential
-CSET
-Course work: exams
-Class assignments
-Exam
-Interview
-Exit exam/Interview
-Self-study report
-Follow-up Survey
-Advisory Board feedback
-Early field
experiences
-Peer teaching
-Student teaching
-Course work: exams
-Class assignments
-Peer teaching
-Early field experiences
-Field Experience
-Livescan
-Disposition Rubric
-Field Experience
-Exit exam/Interview
-Self-study report
-Follow-up Survey
-Advisory Board feedback
-Exit exam/Interview
-Self-study report
-Follow-up Survey
-Advisory Board feedback
N/A
-Course work: exams
-Class assignments
-Peer teaching
-Early field experiences
-Student teaching logs
-Exit Exam/Interview
National AHHPERD 2011
CSUSB APE Specialist Credential Candidate
Learning Outcomes

APE credential candidates will be able to:





Demonstrate professional and pedagogical content knowledge and
skills
Conduct motor assessment, write an assessment report, and
develop IEP goals and objectives for individuals with disabilities
Develop lesson plans and teach lessons for diverse learners
including students with and without disabilities, different ethnic,
racial, gender, language, socio-economic, and religious groups
Engage in professional education experiences through field
experience, clinical practice, volunteering, etc.
Demonstrates an understanding of the legal factors and issues
associated with APE and disability
National AHHPERD 2011
CSUSB APE Specialist Credential Candidate
Learning Outcomes
Candidate Learning Outcomes
Signature
Assignment
Demonstrate professional and
pedagogical content knowledge and skills
KINE 331 B,C,D,E:
Lesson plans
Develop lesson plans for student with hearing impairment/deaf, visual
impairment/blind, physical disabilities (2), multiple disabilities (2), autism,
and inclusive PE
Conduct motor assessment, write an
assessment report, and develop IEP goals
and objectives for individuals with
disabilities
KINE 330: Assessment
Report
Conduct motor assessment and write an assessment report for students
with disabilities. An assessment report including background information,
note on assessment context, assessment instrument, the purpose of
assessment, assessment results, comment related to evaluation,
recommendations, and IEP program
Develop lesson plans and teach lessons
for diverse learners including students
with and without disabilities, different
ethnic, racial, gender, language, socioeconomic, and religious groups.
KINE 331 B,C,D,E:
Develop lesson plans for student with hearing impairment/deaf, visual
Lesson Plans
impairment/blind, physical disabilities (2), multiple disabilities (2), autism,
Teaching Demonstrations and inclusive PE
Do in-class peer teaching demonstrations based on their lesson plans
Engage in professional education
experiences through field experience,
clinical practice, volunteering, etc.
All Classes:
Visitation Reports
Demonstrates an understanding of the
legal factors and issues associated with
APE and disability
KINE 330: Article review Write an article review paper
Paper
Description of the Assignment
Write and submit visitation reports. Visitation reports including site
information, site setting, activities observed or taught, students’
reflections and evaluations from site supervisor/teacher
National AHHPERD 2011
Sample Rubrics


Teaching Demonstration Rubric
Student Visitation Report
National AHHPERD 2011
National AHHPERD 2011
CSUSB APE Specialist Credential Candidate
Comprehensive Exit Exam


Completion of course work and student
teaching
10 Essay questions
National AHHPERD 2011
CSUSB APE Specialist Credential Candidate
Comprehensive Exit Interview



Exit Survey
Exit Interview
Review of the Student file (portfolio)
National AHHPERD 2011
Exit Survey Question 5: Apply the knowledge and skill acquired when teaching
APE to students with disabilities in the public schools (AY 09-10, N=7)
Not
at all
To a
moderate
degree
To a large
degree
Total
2
5
7
1
4
7
1
6
7
Design and implement a unit of instruction
7
7
Design, write, and teach a lesson
7
7
Make accommodations and modifications
7
7
5
7
7
7
Sub-Questions
To a
slight
degree
Conduct formal and informal assessments of
students with disabilities
Write an Individualized Education Plan (IEP)
2
Manage the behavior of a child or class
Include children with disabilities into general
physical education
1
1
Analyze the effectiveness of my teaching
Teach specific disabling conditions
1
6
7
Adapt to unique teaching situations and deal with
problems that arise in APE
1
6
7
National AHHPERD 2011
Audience Input & Questions






Share your Assessment Plan
What has worked for you? What has been the most rewarding?
How has it helped with program
effectiveness?
Has it help students to better learn?
What has been the most challenging aspect?
National AHHPERD 2011