325K: Combined Priority for
Personnel Preparation
Webinar on the
Annual Performance Report
for Continuation Funding
Office of Special Education Programs
U.S. Department of Education
Washington, DC
February 22, 2012
Annual Performance Report (APR)
O S E P
2
An annual report of your performance in
meeting the approved objectives of the
project and responsible use of federal funds
Required for all active grants, including those
in no cost extension (NCE)
OSEP reviews the report to determine if
substantial progress has been made in order
to grant continued funding or a NCE
Overview
O S E P
3
Recognize strong project objectives that can
be associated with high-quality performance
measures
Develop relevant, measurable, outcomeoriented performance measures related to
your objectives that maximize the potential
for meaningful data and positive outcomes
Complete the ED Grant Performance Report
(ED 524B)
Why Is This Important?
O S E P
High-quality objectives and measures …
 Make it easier for you to measure your progress for
the purpose of grant management
 Allow you to report progress easily and
quantitatively
 Establish targets (both short-term/annual and longterm)
 Allow OSEP staff to gather evidence of program
effectiveness
4
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
Goal – Objectives - Measures
O S E P
5
Program Goal
Project Objectives
What your project is doing to support the overall program goal
(Found in your application)
Performance Measures
How you measure your progress toward meeting your objectives
(Program/GPRA, Project)
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
Project Objectives
O S E P
What are you trying to
accomplish?
Objectives should answer this question.
Preferred format for objectives:
Begin the objective with a verb and define
a desired outcome or condition
6
Attributes of
High-Quality Project Objectives
O S E P
7
 Relevance
 How relevant is the project objective to the
overall goal of the program and/or the goal of
your project?
 Applicability
 How applicable is the project objective to the
specific activities that are being conducted
through your particular project?
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
Attributes of
High-Quality Project Objectives
O S E P
8
 Focus
 How focused is the project objective?
 Measurability
 Are there concepts in the project objective
that lend themselves to measurement?
If so, is measurement feasible?
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
Project Objectives -- Examples
O S E P
9
 Recruit and retain high-quality scholars
 Implement a high-quality personnel development program
that is aligned with State and national professional
organization standards
 Prepare high-quality personnel to serve children with
disabilities and their families
 Disseminate information and outcomes of the program
within State and nationally
Performance Measures
O S E P
10
How are you measuring your progress
in meeting your objectives?
Performance measures should answer this question.
Performance Measures are …
O S E P
11
Measurable indicators used to determine
how well objectives are being met.
Important to establish  How progress will be assessed
 How much progress will constitute success
 How will it be known if an objective or part of an objective
has been achieved
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
Performance Measures
O S E P
12
Project Objective
1
Performance Measure
1a
Performance Measure
1b
Performance Measure
1c
Taken from the Center for Evaluation & Educational Policy (CEEP) at Indiana University
2 Types of Performance Measures
O S E P
13
 Program
All grantees funded under the Personnel Development
Program (PDP) must report on the PDP program
performance measures established by OSEP.
 Project
Each grantee reports on the approved project
performance measures established to meet their
project objectives.
Program Performance Measures
O S E P
14
 Program
Measures were established by OSEP for the Personnel
Development Program.
Measures apply to all grants funded under the
Personnel Development Program.
Results on these measures are reported to Congress
under the Government Performance and Results Act
of 1993.
Personnel Development PROGRAM Performance Measures
#1: The percentage of Special Education Personnel Development projects that
incorporate evidence-based practices into their curricula.
#2: The percentage of scholars completing Special Education Personnel Development
funded programs who are knowledgeable and skilled in evidence-based practices for
infants, toddlers, children, and youth with disabilities.
#3: The percentage of Special Education Personnel Development funded scholars
who exit preparation programs prior to completion due to poor academic performance.
#4: The percentage of Special Education Personnel Development funded
degree/certification program recipients who are working in the area(s) in which they
were prepared upon program completion.
#5: The percentage of Special Education Personnel Development funded
degree/certification recipients who are working in the area(s) for which they were
prepared upon program completion, and who are fully qualified under IDEA.
#6: The percentage of Special Education Personnel Development degree/certification
recipients who maintain employment in the area(s) for which they were prepared for 3
or more years, and who are fully qualified under IDEA.
#7: The Federal cost per fully qualified degree/certification recipient.
15
Program Performance Measures
O S E P
16
 OSEP collects data on the program
performance measures each year.
 Grantees should propose a project
performance measure that addresses each
program performance measure reported for the
325K program.
The focus of the project measure must be the
same as and clearly aligned with the program
measure.
Program Performance Measure
O S E P
17
#1. The percentage of Special Education
Personnel Development projects that
incorporate evidence-based practices into
their curricula.
Example of related project performance measure:
By the beginning of Year 3, 100% of syllabi from core
courses will be rated as “evidence-based” by the Project
Advisory Board using a rubric designed by the Project.
Program Performance Measure
O S E P
18
#2. The percentage of scholars completing
Special Education Personnel Development
funded programs who are knowledgeable
and skilled in evidence-based practices for
infants, toddlers, children, and youth with
disabilities.
Example of related project performance measure:
At the end of their program, 100% of scholars will
achieve a minimum of 3 on a 4-point rubric for a
portfolio that provides evidence of their
competencies in providing interventions to children
with disabilities.
Program Performance Measure
O S E P
19
#3. The percentage of Special Education
Personnel Development funded scholars who
exit preparation programs prior to completion
due to poor academic performance.
Example of related project performance measure:
As measured annually, no scholars (0%) will have exited the
program due to poor academic or field-based performances.
Program Performance Measure
O S E P
20
#4. The percentage of Special Education
Personnel Development funded scholars
who are working in the area(s) in which
they were prepared upon program
completion.
Example of related project performance measure:
By the end of Year 5 of the project, 90% of scholars will be
working in the area for which they were trained following
program completion.
Program Performance Measure
O S E P
21
#5. The percentage of Special Education
Personnel Development funded
degree/certification recipients who are working
in the area(s) for which they were prepared
upon program completion, and who are fully
qualified under IDEA.
Example of related project performance measure:
By the end of Year 5 of the project, 90% of program graduates
will be working in the area for which they were trained and fully
qualified under IDEA.
Program Performance Measure
O S E P
22
#6. The percentage of Special Education
Personnel Development degree/certification
recipients who maintain employment in the
area(s) for which they were prepared for 3
or more years, and who are fully qualified
under IDEA.
Example of related project performance measure:
By the end of Year 5 of the project, 90% of eligible program
graduates will have maintained employment in the area for
which they were trained for 3 or more years, and be fully
qualified under IDEA.
Program Performance Measure
O S E P
23
#7. The Federal cost per fully qualified
degree/certification recipient. (Efficiency
Measure)
Example of related project performance measure:
The Federal cost is measured in Year 5 by dividing the total
grant award (e.g., $250,000 x 5) by the number of scholars
counted for Performance Measure #2.
Project Performance Measures
O S E P
24
 Project
Measures that the grantee establishes to meet their
project objectives
 Project performance measures can address both
the process of working towards an objective and the
outcome related to meeting the objective
 Ensure a mix of both process and outcome
measures, but most will be outcome
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
High-Quality Performance
Measures
O S E P
High-quality performance measures show:
25
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
What will change.
How much change you expect.
Who will achieve the change.
When the change will take place.
Project Performance Measure
O S E P
26
Process measure example Project staff (who) will implement 4 (how much)
targeted recruitment activities (what ) during the first
year of the grant (when).
Project Performance Measure
O S E P
27
Outcome measure example –
Inquiries from potential candidates (who) about the
program (what) will increase 20% from previous
recruitment efforts (how much) during the first year of
the grant (when).
Project Performance Measure
O S E P
28
Outcome measure example –
At the end of each semester (when), 100% of scholars
(who) will rate their mentoring support (what) as 5 or
higher on a 7-point scale (how much) using an on-line
survey developed for the project.
Project Performance Measure
O S E P
29
Outcome measure example –
At the end of their program (when), 90% of scholars
(who) will demonstrate advanced critical reflection skills
(what) by achieving a proficiency score of at least 3 on
a 4-point rubric (how much) developed by the project.
Project Performance Measure
O S E P
30
Process measure example –
By the end of the project period (when), the project staff
(who) will present on activities and outcomes (what) of
the project at a minimum of 3 state and national
meetings (how much).
Common Problems
 Activities are NOT performance measures
O S E P
 If the best response is “Yes, we did that,” it
is likely an activity (not a performance
measure)
 Examples of activities:
 Scholars complete required coursework
 Hold an Advisory Board meeting quarterly
 Establish a mentoring program
 Evaluate the project
31
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
Common Problems
O S E P
32
Performance measures need to be
measurable
 Examples with measurement problems
(activities rather than outcomes):
 Maintain meaningful collaborative partnerships
 Provide academic support in areas of concern to scholars
 Establish field placements in low income schools
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
Need additional information on
writing performance measures?
O S E P
33
All grantees are strongly encouraged to seek
training on writing performance measures.
For further information on developing
performance measures and logic models,
see http://www.tadnet.org/model_and_performance
Summary
O S E P
34
 Projects should have a few clear objectives
that explain what the project is doing to
support their overall goal(s)
 Each objective should have a few, specific
performance measures to demonstrate how
progress toward meeting the objective is
being measured
 Both program and project performance
measures are reported
Taken from the Center for Evaluation and Education Policy (CEEP) presentation at www.tadnet.org
Completing the ED 524B
O S E P
The ED 524B is a required annual and final
performance reporting form with specific
instructions.
 The form is used by all ED grants and has been
approved by the Office of Management and Budget
(OMB). Project Directors must follow the directions
listed in the Dear Colleague letter and ED 524B
Instructions provided by OSEP
 Word or PDF versions of the forms are available at
http://www2.ed.gov/fund/grant/apply/appforms/appforms.html
35
X
x
O S E P
36
ANNUAL PERFORMANCE REPORTS
Reporting Period:
For first year grants, the date is the beginning of the project year to February 29th.
For grants in years 2-5, it is the date from the end of the previous reporting period
to February 29th.
02
37
29
2012
-__
ANNUAL PERFORMANCE REPORTS
Budget Expenditures:
Report the expenditures during the “Reporting Period.” Must be data or
information from the business or grants office.
38
ANNUAL PERFORMANCE REPORTS
Performance Measure Status:
This will be checked “No” since OSEP is asking for data for the award reporting
period, not for the budget period.
The date entered here will be the due date for your Final Performance Report;
which is 90 days after the end of the grant.
Signatory must have authority to sign on behalf of the institution since the
grant is from the Department to the institution and not to an individual. The
Authorized Representative signs; not the Project Director.
39
OMB No. 1894-0003
Exp. 02/28/2011
EXECUTIVE SUMMARY SHEET
H325 K - - - - -
*** Provide highlights of the project's activities and the extent to which the expected outcomes and performance
measures were achieved during the reporting period. Do NOT include the project abstract.
40
PROJECT STATUS CHART
H325 K - - - - -
41
PROJECT STATUS CHART
H325 K - - - - -
Enter one of the project’s objectives; on subsequent pages, you will enter
additional project objectives as submitted in your grant application.
42
PROJECT STATUS CHART
Enter the PROGRAM performance measure(s) that align with the
objective immediately followed by any PROJECT measure(s)
developed to address that program measure.
Enter any additional PROJECT performance measures that show you
are measuring progress toward meeting the objective (e.g., 1c., 1d.,).
43
PROJECT STATUS CHART
Here you identify if the performance measure is a PROGRAM measure,
“PRGM,” or a PROJECT measure, “PROJ.”
PRGM
PROJ
Note: Program measure refers to one of OSEP’s 7 performance
measures for the Professional Development Program.
Project measures are unique to your grant.
44
PROJECT STATUS CHART
QUANTITATIVE DATA
Depending on your measure, enter either a raw number, or a ratio and percentage.
Enter the target number identified in the performance measure and then the actual data for this year. If
complete data are not available for the measure, enter “999” (if no baseline) or “NA” in the “Raw Number”
or “%” column, as appropriate. Provide an explanation at the bottom of the page under “Explanation of
Progress.”
8
45
6
Information to Include in the
Explanation of Progress Section
O S E P
46
 Describe the data provided (e.g., what data collection methods
were used, when were the data collected, how was a sample
drawn, are there missing/incomplete data, what was the
response rate, was a reliability measure taken). Your Project
Officer should be able to understand and interpret the numbers
in the chart from your description in this section.
 What changes in the data occurred since last APR (i.e., trend)?
 What activities were undertaken to achieve the targets?
 If targets were not met, what are possible reasons?
 How will activities that failed to meet targets be improved?
PROJECT STATUS CHART
QUALITATIVE DATA
If measure requires the collection of qualitative data, report the performance
measure and type (PROG or PROJ) and then, enter “N/A” under the Raw
Number and Percentage columns.
N/A
N/A N/A
In the “Explanation of Progress” section of the page, referencing the
performance measure by number, report applicable qualitative data along
with other information about how these data were collected, targets and
activities –refer to previous slide for additional content requirements.
47
N/A
O S E P
By Year 3,
20
20
18 20
H325 K - - - - Implement a high-quality personnel development program that is aligned to State and
national professional organization standards
O S E P
Explanation of Progress (Include Qualitative Data and Data Collection Information)
(1b) The program faculty are committed to reviewing the entire curriculum of our personnel development
program and ensuring evidence-based practices are integrated into core courses, including field
experiences. As part of that plan, 100% of the syllabi would be thoroughly reviewed by the Project
Advisory Board using an evidence-based practices (EBP) rubric developed for this Project to decide
whether revision was necessary to meet our new state professional practice standards for licensure
which include knowledge and skills on the use of EPB. By the end of Year 2, a full review of all 20 syllabi
(100%) was completed and it was determined that 18 met the target (90%); 2 require a revision to include
EBP.
We attached the rubric to Section C of this report.
(1c) Since this grant is designed with a focus on universal designs for instruction, a required seminar on
this topic was identified for more in-depth evaluation. The seminar was offered during the Fall 2010
semester. All students in the course completed the rating. Given that the target was not met, students will
be interviewed to see how the seminar might be improved for future scholars, with the goal of meeting the
“usefulness” score target of 4.5 the next time the class is offered and evaluated.
49
Final Page of the Report
H325 K - - - - -
Section B: Refer to the instructions for Section B with the ED 524B
Section C: Refer to the instructions for Section C with the ED 524B—put
attachments here (e.g., recruitment material, evaluation instruments,
journal articles)
50
Section B – Budget Information
O S E P
51
This section is never blank! A table can be helpful!
A.
Provide actual expenditures for this reporting period (through
2/29/2012)
B.
Estimate anticipated expenditures for the rest of this budget
period and balance remaining, if any.
C.
Explain why you did not expend funds at the expected rate.
D.
Indicate how you plan to use the unexpended funds (carryover)
in the next budget period.
Section B – Budget Information
O S E P
E. Describe any significant changes to your budget resulting from
modifications of project activities.
F. Describe any changes to your budget that affect your ability
to achieve your approved project activities and/or project
objectives.
G. Describe any anticipated changes in your budget for the next
budget period that require prior approval from the Department.
Any questions … Talk to your Project Officer
52
Section C – Additional Information
O S E P
53
 Provide a list of current partners on your grant and indicate if:
 Any partners changed during the reporting period. If there were changes,
please describe both the changes and any impact that resulted in your ability to
achieve approved project objectives and/or project activities.
 Any partners are anticipated to change during the next budget period. If so,
please describe both the changes and any impact the change might have on your
ability to achieve approved project objectives and/or project activities.
 Describe any changes that you wish to make in the grant’s
activities for the next budget period that are consistent with the
scope and objectives of your approved application.
Section C – Additional Information
O S E P
54
 If requesting changes to the approved Project Director and/or
other key personnel, please include the person’s name, title, and
contact information. Indicate his/her proposed start date, and
percentage of time working on the grant, and attach a resume or
curriculum vitae to the annual performance report being submitted.
 Do not report on any key personnel changes that were already made
during the current or previous budget period(s).
 Note: Departmental approval must be requested and received prior to
making key personnel changes.
 Provide any other information about your project including
unanticipated outcomes or benefits
Submitting the ED 524B
O S E P
 Submit the ED 524B at http://www.g5.gov/
 Instructions for using G5 are in the continuation packet.
 Signed ED 524B Cover Sheet must be scanned and emailed in PDF format to
your Project Officer.
 Special cases require regular email submission of
the 524B and signed cover sheet in PDF format to your
Project Officer rather than submission through G5 –
 Final Performance Reports or APRs for grants in their last
performance period, or no-cost extension
 Grants that have been front-loaded (forward-funded) last year
55
Due Date: April 27, 2012
(Friday)
Contact your OSEP Project Officer
with any questions!
56