PartMaster
II
Click to edit
title style
The details
1
Core Indicator Data
Click to edit Master title style
• MIS Data
– Data Elements
– Funding
– Accountability
2
Defining the Data
Click to edit Master title style
•
•
•
•
SAM Codes
TOP Codes
Data Elements
Core Indicators
–
–
–
–
Funding
“The Law”
Definitions
Negotiated Performance Targets
Measurement Approaches/Formulas
3
Student Accountability Model (SAM)
Click
to edit of
Master
title(TOP)
style
& Taxonomy
Programs
• Priority “A“ - Apprenticeship
– Must have the of the Division of Apprenticeship Standards
approval
• Priority “B“ – Advanced Vocational
– Used sparingly, no more than two courses in any one
program
– “B” level courses must have a “C” prerequisite in the same
program area
• Priority "C" – Clearly Occupational
– Generally taken in the middle stage of a program, detracts
"drop-ins."
4
Student Accountability Model (SAM)
Click to edit
Master(TOP),
title style
& Taxonomy
of Programs
Continued
• Priority "D" – Possibly Occupational
– Taken by students in the beginning stages of their
occupational programs
– Can be survey course
• Priority “E” = Non-Occupational
Vocational Flag on TOP code
– Designed to identify vocational “Programs” for federal
reporting (*) - see Taxonomy of Programs, Sixth Edition,
July 2007
5
Data Elements
MIS
System title style
Click to edit
Master
• Students, Courses, Degrees, Services
• Student VTEA Data Elements
–
–
–
–
–
–
Economically Disadvantaged
Single Parent
Displaced Homemaker
Cooperative Work Experience Education
Tech Prep
Migrant Worker - Implementation in MIS SU 09
6
Accountability Requirements
Click toSection
edit Master
113(b)title style
5 core indicators of performance:
1.
Student attainment of technical skill proficiencies;
2.
Student attainment of credential, certificate, or degree;
3.
Student retention in postsecondary education or transfer;
4.
Student placement in military, apprenticeship, or employment
5.
Student participation/completion of non-traditional training
State and Local adjusted levels of performance
–
Levels of performance negotiated with USDE / State
Results reported annually
7
Perkins IV (2006)
Indicators
Click to Core
edit Master
title style
1. Technical Skill Attainment
Successful CTE course completion (GPA)
2. Completions
Program completion–Certificate, Degree & Transfer
Prepared
3. Persistence & Transfer
Student persistence in Higher Ed
4. Placement
Placement in apprenticeship, employment, military, fed gov
5. Equity -- Nontraditional Fields
Participation (5a)/Completion (5b) - nontraditional “fields”
8
Cohort Definitions
for Master
Measurement
ClickUsed
to edit
title style
Participant: NT Participation
–
Perkins III -Any enrollment in a CTE course (TOP)
–
Perkins IV – concentrator using assigned major
Concentrator: All other indicators
•
Cohort of students enrolled during the cohort year and
–
Successfully completed at least one course in the middle or end of a
program (SAM A-C) and 12 vocational units within a single
discipline (two digit TOP) or
–
Program completion as indicated by receipt of ANY vocational credit
certificate or degree
Leavers: Not enrolled in year following cohort year
–
2P1 - Completions
–
4P1 - Placement (Employment)
9
Assigning a Program to a Student
Click to edit Master title style
1. Award –TOP code of CTE Certificate or Degree
2. Concentrators
•
Hierarchy based on SAM Priority code
•
Assigned to the TOP where most CTE units occurred
10
Timeline for Outcomes & Outputs
Click to edit Master title style
• Negotiated Performance 2008-09
– Negotiated Spring 2008
– Reports publish in Spring 2009
• Cohort Year (2006-07)
• +1 yr for outcomes (2007-08)
– Transfer
– Persistence
– Employment
• Outcomes have already occurred
– Target low performance now!
11
Timeline for Outcomes & Outputs
Click to edit Master title style
Program
Year
2006-07
Negotiated in
Fall 2007
Measured in
Fall 2006
Spring 2007
2007-08
2008-09
2009-10
Fall 2007
Spring 2008
Fall 2008
Spring 2009
Fall 2009
Spring 2010
Fall 2007
Spring 2008
Fall 2008
Spring 2009
Fall 2009
Spring 2010
Outcome Years
2007-08 cohort w/ 2008-09 outcomes
2006-07 cohort w/ 2007-08 outcomes
2005-06 cohort w/ 2006-07 outcomes
12
Core Indicator 1
Click
to edit Skill
Master
title style
Technical
Attainment
• All Concentrators
• Successful Course Completions
– Technical Skill Proficiencies
• Vocational (CTE) Courses
– SAM A-C
– Vocational TOP
– G.P.A.
– Grade reports (moved to Data Mart)
13
Core Indicator 1: Measurement &
Click Performance
to edit Master
title style
Levels
SAM A - C Courses:
# Student concentrators with GPA > 2.00
÷
# Students concentrators with Grades A – F
Excludes students with only CR/NC or P/NP grades in SAM A-C courses
Plan Year
2005-06
2006-07
2007-08
2008-09
2009-10
TargetActual*
92.46%
92.yy%
* Based on spring 2008 Perkins IV reports.
93.65%
92.58%
92.46%
92.xx%
%
14
CI 1 - 2008 Data
Core One 1P1 - Technical Skill Attainment
100.00%
95.0%
98.00%
96.00%
90.0%
93.98%
93.89% 93.74%
93.65%
93.71% 93.65%
94.00%
92.58% 92.46%
85.0%
92.00%
80.0%
Total
90.00%
88.00%
75.0%
86.00%
70.0%
84.00%
65.0%
82.00%
80.00%
60.0%
1998-99
19992000
2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
Cohort Year
Male
CA Empl Rate
CI 1 - 2008 Data - Student counts
1P1 Totals
250,000
100.0%
95.0%
200,000
90.0%
150,000
85.0%
Grand Total
80.0%
Female
Male
100,000
75.0%
Non-traditional
70.0%
Economically Disadvantaged
65.0%
50,000
60.0%
55.0%
-
50.0%
1998-99 1999- 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
2000
Cohort Year
Limited English Proficiency
CA Empl Rate
CI 1 - 2008 Data
Core One 1P1 - Technical Skill Attainment
100.00%
95.0%
98.00%
96.00%
90.0%
93.98%
93.89%
93.74%
93.65%
93.71%
93.65%
92.58%
94.00%
92.46%
92.00%
85.0%
80.0%
90.00%
Total
Male
88.00%
75.0%
Economically
Disadvantaged
86.00%
70.0%
CA Empl Rate
84.00%
65.0%
82.00%
80.00%
60.0%
1998-99
19992000
2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
Cohort Year
Fake Data
Core One 1P1 - Technical Skill Attainment
Exaggerated
100.00%
1998
2005
98.00%
100.0%
95.0%
96.00%
90.0%
94.00%
93.67%
92.00%
90.00%
85.0%
80.0%
Total
Female
Male
88.00%
75.0%
CA Empl Rate
86.00%
70.0%
84.00%
65.0%
82.00%
80.00%
60.0%
1998-99 1999-2000 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
Cohort Year
Fake Data
Core One 1P1 - Technical Skill Attainment
Exaggerated
100.00%
1998
2005
98.00%
100.0%
95.0%
96.00%
90.0%
94.00%
92.5%
92.00%
90.00%
80.0%
84.00%
Total
Female
Male
88.00%
86.00%
85.0%
75.0%
CA Empl Rate
84.3%
83.2%
70.0%
65.0%
82.00%
80.00%
60.0%
1998-99 1999-2000 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
Cohort Year
Forecasting
Click to edit Master title style
• Wide range of forecasting techniques
– Qualitative Forecasting Methods
• Judgmental Forecasting
– Expert Forecasting
– Consensus Forecasting
– Informal
– Quantitative Forecasting Methods
• Time Series
– Naïve Forecasting
– Averaging
• Causal / Relational Models
– assume cause and effect, and cause can be used to predict outcomes
– if you know one variable, you can forecast the other
• Sophisticated
20
Qualitative Forecasting Methods
Click to edit Master title style
• Judgmental Forecasting
–
–
–
–
Expert Forecasting
Consensus Forecasting
Informal
Work best when background conditions are changing
rapidly
• When economic, political or administrative conditions are in
flux,
• When quantitative methods may not capture important
information about factors that are likely to alter historical
patterns. (e.g., new large apprenticeship program)
21
Qualitative Forecasting
Weaknesses
Click to edit
Master title style
•
anchoring events
–
•
information availability
–
•
giving undue weight to what forecasters and government officials would like to see happen
group think
–
•
ignoring important information that conflicts with the forecaster’s view about causal relationships
wishful thinking
–
•
forecasters using different strategies over time to make their judgments, making them less reliable
selective perceptions
–
•
incorporating information about factors that are assumed to influence outcomes, but do not
inconsistency in methods and judgments
–
•
over-weighting the use of readily available information
false correlation
–
•
allowing recent events to influence perceptions about future events, e.g. the college hosting a recent institute on student
learning
when the dynamics of forming a consensus leads individuals to reinforce each other’s views rather than maintaining
independent judgments
political pressure
–
where forecasters adjust estimates to meet the imperatives of budgetary or other college constraints.
22
Simple
Quantitative
Click to edit Master title style
• Naïve Forecasting
– Random Walk
• Last known
• Random Walk with drift
– Averages
– Seasonal adjustments
used in expert forecasting as the starting
point for estimates that are then adjusted
mentally
23
Random
Walk
Click to edit Master title style
• Last known
24
Random
Walk
Click to edit Master title style
• Last known
25
Random
Walk
Click to edit Master title style
• with drift
26
Random
Walk
Click to edit Master title style
• with drift
27
Averages: CI 1 - 2008 Data
Core One 1P1 - Technical Skill Attainment
2008
100.00%
95.0%
98.00%
96.00%
90.0%
93.98% 93.89% 93.74% 93.65% 93.71% 93.65%
94.00%
Random walk
92.58% 92.46%
85.0%
90.00%
92.46%
92.34%
Random
80.0%
walk w drift
88.00%
75.0%
86.00%
70.0%
92.00%
84.00%
65.0%
82.00%
80.00%
60.0%
1998-99 1999- 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
2000
Cohort Year
Total
Male
CA Empl Rate
Moving Average
Click to edit Master title style
• moving average
– the last N periods of data are used equally
– all prior observations are not used
• Provided in the workbook
3 Yr Ave
4 Yr Ave
5 Yr Ave
93.11%
93.22%
93.12%
93.23%
Average of rates
92.90%
Average adding N's
92.90%
29
Averages: CI 1 - 2008 Data
Core One 1P1 - Technical Skill Attainment
2008
100.00%
95.0%
98.00%
96.00%
90.0%
93.98% 93.89% 93.74% 93.65% 93.71% 93.65%
94.00%
92.58% 92.46%
85.0%
92.6%
92.00%
80.0%
Total
90.00%
88.00%
75.0%
86.00%
70.0%
84.00%
65.0%
82.00%
80.00%
60.0%
1998-99 1999- 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
2000
Cohort Year
Male
CA Empl Rate
What we used on CI 1 - 2008 Data
Core One 1P1 - Technical Skill Attainment
2008
100.00%
95.0%
98.00%
96.00%
90.0%
93.98% 93.89% 93.74% 93.65% 93.71% 93.65%
94.00%
92.58% 92.46%
85.0%
92.46%
92.00%
80.0%
Total
90.00%
88.00%
75.0%
86.00%
70.0%
84.00%
65.0%
82.00%
80.00%
60.0%
1998-99 1999- 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
2000
Cohort Year
Male
CA Empl Rate
Three Basic “Chuck” Rules
Click to edit Master title style
• With no application of local knowledge or
sophisticated projections:
– Declining for three or more years
• random walk, last known
– Increasing for three or more years
• three year average
– Increasing and decreasing
• three year average.
32
Core Indicator 2
ClickProgram
to edit Master
title style
Completions
• Completers (numerator)
– Transfer Prepared
– Award in Current Year
– AA/AS degrees
– Certificates
– Award in subsequent year with no Voc coursework
– or Equivalent
• Leavers & Completers (denominator)
– Left system (college) for one year and/or
– Award in Current Year
– AA/AS degrees
– Certificates
– Transfer Prepared
– Award in subsequent year with no Voc coursework
• Removed Persisters & Life-Long-Learners
33
CI 2-Completions:
Click to edit
title style
Measurement
& Master
Performance
Levels
Certificate/Degree/Transfer Prepared
÷
Concentrators (Leavers & Completers) Not LLL
Year
Target
Actual*
2005-06
64.53%
2006-07
65.70%
2007-08
68.16%
2008-09
3 yr Ave
66.13%
2007-08
2008-09
xx.xx%
88.03%
3 yr Ave
* Based on Perkins IV data.
87.20%
xx.xx%
34
CI 2 2008 data
Core Two 2P1 - Completions
100.00%
90.00%
80.00%
70.00%
68.16%
66.35%
65.70%
65.28%
64.76% 63.59%
63.45% 64.53%
66.13%
60.00%
50.00%
40.00%
1998-99
19992000
2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
Cohort Year
Total
Male
Economically Disadvantaged
Limited English Proficiency
CA Empl Rate
CI 2 2008 data revised
Core Two 2P1 - Completions
100.00%
87.18%
90.00%
87.25%
85.60%
86.68%
86.90%
88.03%
3 yr ave
80.00%
72.21%
71.73%
70.00%
CA hits 6.6% high unemp rate Jan 04
fees increase Sp 2003
from $11 - $18
60.00%
fees increase Fall 2004
from $18 - $26
50.00%
40.00%
1998-99
19992000
2000-01
2001-02
2002-03
2003-04
2004-05
2005-06
Cohort Year
Total
Female
Male
Economically Disadvantaged
Limited English Proficiency
CA Empl Rate
Core Indicator 3
ClickPersistence
to edit Master
title style
& Transfer
Concentrators who were not leavers in the year following the cohort year
or
Transfers to CCC/CSU/UC/Privates (National Student Loan Clearinghouse)
÷
All Concentrators who were not completers with degrees or certificates (unless
transferring)
Year
Target
Actual*
2005-06
93.02%
2006-07
92.80%
2007-08
93.03%
2008-09
3 yr Ave
92.95%
xx.xx%
37
CI 3 – 2008 data
1998
Core Three 3P1 - Persistence & Transfer
2007
98.00%
95.0%
96.00%
90.0%
94.00%
92.00%
85.0%
90.00%
80.0%
88.00%
86.00%
75.0%
84.00%
70.0%
82.00%
80.00%
65.0%
1998-99 1999-2000 2000-01
2001-02
2002-03
2003-04
2004-05
2005-06
Cohort Year
Grand Total
Female
Male
Economically Disadvantaged
Non-traditional
CA Empl Rate
CI 3 – 2008 data
1998
Core Three 3P1 - Persistence & Transfer
100.00%
2007
200
95.0%
95.00%
92.95%
93.02% 92.80%
90.00%
85.00%
90.0%
93.03%
85.0%
82.95%
80.0%
80.00%
75.0%
75.00%
70.0%
70.00%
65.0%
1998-99 1999-2000 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
Cohort Year
Grand Total
CA Empl Rate
Core Indicator 4
Click to edit
Master title style
Placement
• Placement
– Leavers and Completers
• Minus Continuing in Two or Four Year Institutions
– CCC or National Student Loan Clearinghouse
– Employment 1st year out
• UI wage file match
– Employment any quarter in Academic Yr after cohort year
– Apprenticeship, Military, Fed Gov
40
CI 4 Placement :
Click to edit
title style
Measurement
& Master
Performance
Levels
Leavers & Completers in UI covered employment or
Apprenticeship, Military, Fed Gov
÷
All Leavers & Completers
Year
Target
Actual*
2005-06
78.40%
2006-07
80.15%
2007-08
81.04%
2008-09
3 yr Ave
79.86%
xx.xx%
*Based on Spring 2008 Perkins IV data.
41
CI 4 2008 data
4P1 Employment Rates w/ CA Empl Rate
100.00%
1999
2006
95.00%
90.00%
85.56%
Grand Total
85.17%
Female
85.00%
79.92%
80.00%
78.77%
80.15%
78.46%
81.04%
Male
79.86%
78.40%
Economically
Disadvantaged
75.00%
71.87%
CA Empl Rate
70.00%
65.00%
1998-99
1999-2000
2000-01
2001-02
2002-03
Cohort Year
2003-04
2004-05
2005-06
CI 4 2008 data revised
4P1 Employment Rates w/ CA Empl Rate
100.00%
1999
2006
95.00%
Grand Total
90.00%
85.56%
85.13%
Female
85.00%
80.65%
81.16%
79.46%
80.00%
79.20%
82.12%
Male
80.91%
79.46%
Economically
Disadvantaged
CA Empl Rate
75.00%
72.82%
70.00%
1998-99
1999-2000
2000-01
2001-02
2002-03
Cohort Year
2003-04
2004-05
2005-06
Core Indicator 5
Click to edit
Master
Gender
Equitytitle style
Programs leading to Nontraditional Fields
(e.g., Men in Nursing – Women in Auto)
75% / 25% from 2000 census employment data
– NAPE developed Nontraditional CIP table
• Job codes (SOC) mapped to 2000 Census data
• SOC codes mapped to CIP (USDE)
• CIP codes mapped to TOP (CCC)
44
Core Indicator 5
Click to edit
Master
Gender
Equitytitle style
Programs leading to Nontraditional Fields
Nontraditional Gender Students
÷
All Students in NT Program
45
CI 5a: Participation
Click to edit&Master
title style
Measurement
Performance
Levels
Nontraditional participants enrolled in a
Nontraditional TOP Code
÷
All participants enrolled in a
Nontraditional TOP Code
Year
Target
Actual*
2005-06
22.06%
2006-07
21.78%
2007-08
21.47%
2008-09
Random Walk
21.47%
* Based on Spring 2008 Perkins IV data.
xx.xx%
46
CI 5a 2008 data
5P1 Nontraditional Participation Rates
70.00%
100.0%
1998
CA Employment Rates (BLS)
2007
95.0%
60.00%
90.0%
50.00%
85.0%
80.0%
40.00%
31.62% 32.07% 31.78% 30.64%
29.83%
30.00%
19.32%
75.0%
22.06% 21.78% 21.47%
20.00%
70.0%
21.47%
65.0%
60.0%
10.00%
55.0%
0.00%
50.0%
1998-99
19992000
2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
Cohort Year
Total
Female
Male
Econ Disadv
CA Empl Rate
CI 5a 2008 data revised
5P1 Nontraditional Participation Rates
70.00%
1998
CA Employment Rates (BLS)
60.00%
2007 100.0%
95.0%
90.0%
85.0%
50.00%
80.0%
75.0%
70.0%
40.00%
30.00%
20.00%
22.56% 23.02% 23.14% 22.60% 21.88% 21.69% 21.62% 21.63%
21.63%
19.47%
10.00%
65.0%
60.0%
55.0%
50.0%
0.00%
1998-99 1999- 2000-01 2001-02 2002-03 2003-04 2004-05 2005-06
2000
Cohort Year
Total
Female
Male
CA Empl Rate
CI 5b Completion:
Click to edit
title style
Measurement
& Master
Performance
Levels
Nontraditional “completers”
of nontraditional programs
÷
All “completers” of nontraditional programs
Year
Target
Actual*
2005-06
23.85%
2006-07
23.33%
2007-08
23.28%
2008-09
Random Walk
23.28%
* Based on Spring 2008 Perkins IV data.
xx.xx%
49
CI 5b – Spring 2008 data
23.28%
20.95%
CI 5b - 2008 data revised
25.38%
22.84%
CI & Negotiation Reports
Click to edit Master title style
• Negotiation reports
– Developed for negotiations
•
•
•
•
College, District, and State Level
Special Pops detail
8 years history
Final version 10/14/08
• All Core Indicator Reports
– Easier access – Excel or PDF reports
– Easier graphing in Excel
• Progressive scrutiny on use of reports
– Below 90% of Target in any Indicator
52
Access
Click to edit Master title style
• http://www.cccco.edu
• Important Documentation
–
–
–
–
Negotiation Report Instructions (1st Tab)
Quick Reference (not yet available)
System Documentation
System Requirements
• No more Installing plug-in
• No Administrative rights requirements
• Best viewed with MS Internet Explorer
• Email notification when available
53
Report Selections
Report Years
Report Years
Report Years
Indicator 1P1
Indicator 1P1
Indicator 1P1
Indicator 1P1
Indicator 1P1
misweb
misweb
misweb
Report Structures
Click to edit Master title style
• Negotiation Workbooks
– FAUPL negotiation worksheet
• Perkins IC - Local Application
• Forms
• Perkins IC - Local Application
– Targets and Performance
• Perkins IC - Final Report
• Summary Reports
– All five Indicators on one page
• Answer sheet style
• Detail Reports with counts
• Trend Reports
– Percents and counts
– Detailed breakouts for each Indicator component
• Special Population Reports
66
Negotiation Workbooks
Click to edit Master title style
Worksheets:
• Instructions
• Counts
• Percentages
• Charts
– Rates
– Counts
• Data work tables
– Percentages, Success, Totals “only” sheets
– Fees
– Employment
67
Negotiation Tables - Counts
Negotiation Tables - Percentages
Negotiation Charts
Practice using the Data
Click to edit Master title style
Negotiation Exercise:
• Review FAUPL Worksheet
• Walk through common scenarios
• Complete the FAUPL using first indicators
• Negotiate or Accept State Targets
71
Faupl
Local Application CTE-6
Click to edit Master title style
73
2008 FAUPL
Click to edit Master title style
74
FAUPL col 3
Percentages
Click to edit Master title style
76
FAUPL col 456
Indicator 1P1 Rates Default Groups
Core One 1P1 - Technical Skill Attainment
100.00%
1998
2005
95.00%
90.00%
Total
Female
85.00%
Male
Nontrad
80.00%
Econ Disadv
CA Empl Rate
75.00%
70.00%
1998-99
19992000
2000-01
2001-02
2002-03
Cohort Year
2003-04
2004-05
2005-06
Indicator 1P1 2 Rate
Indicator 1P1 Success
Indicator 1P1 Totals
Indicator 1P1
Indicator 1P1 2 Rate
83.17%
74.85%
Columns 4-5-6
Justification
Indicator 2P1
Click to edit Master title style
86
Indicator 2P1 table
Click to edit Master title style
87
Faupl paste
Chart 2p1
Average
Click to edit Master title style
90
Faupl 2p1
Faupl 2p1 complete
Completing the Negotiation
Click to edit Master title style
• Email the completed FAUPL to Monitor
• Monitor will either
– Accept the proposed level & respond by email
– Begin a conversation and negotiation
• District will enter negotiated levels into the
Application.
93
Negotiating Targets
Click to edit Master title style
• State negotiates targets USDE
– Targets for 1 year
– Next 2 years (3rd & 4th)
– Scheduled for April 2009
• Worksheets without state targets are available now
• State Targets will be provided when available
• Locals either:
– Accept state targets
– Negotiate local targets
• Included in Local Plans
– Targets for next year
– Negotiations complete by May 15
94
The 10,000ft. View
Click to edit Master title style
1. District Assesses performance with form I-E-D
a.
b.
c.
Determine improvement status
If necessary, alter planning process
Complete 2009-10 planning
2. Complete the 2009-10 FAUPLE
a.
b.
Analyze overall district performance
Determine proposed targets for the next year
3. Negotiate targets with System Office monitor
4. Complete local application form CTE-6
5. Complete local application form CTE-7
95
Questions?
Click to edit Master title style
Resources:
– Project Monitor
– Nontraditional
• Joint Special Populations Committee (JSPAC.org),
• Institute for Women in Trades, Technology & Science
(IWITTS.com),
• National Alliance for Partnerships in Equity (NAPE.org)
– Forecasting literature
• Fee Impact Study - CCCCO – http://www.cccco.edu/Portals/4/Reports/impact_study_051.pdf
• Enrollment Management papers
– http://www.cap-esp.com/
• Keith Guerin (1999)
Chuck Wiseley - cwiseley@cccco.edu, 916.327.5895
96