ASCCC Noncredit Spring
Accountability
Telling the Noncredit Story
through Accountability Reporting
ASCCC
Ad Hoc Noncredit Committee
2009 - 2010
Janet Fulks, Bakersfield College, Chair
Reynaldo Ortiz, College of the Desert
Vivian Ikeda, City College of San Francisco
Sylvia Ramirez, MiraCosta College
Marsha Elliott, North Orange County CCD – Noncredit
Marne Foster, San Diego Continuing Education
Welcome and Introductions
Noncredit Student Voices
Current Noncredit Accountability Reports
Healthy Metrics and Accountability
CB 21 Recoding for Basic Skills – Review of
the Rubrics – Coding in Teams
 Data collection strategies – Examples from
Colleges
 Group Work
 Student Pathways
 ABE / ESL to ASE to Credit
 Advising Issues – linkages with instruction
and student services




Outcomes
 Participants will:









Understand the main accountability indicators currently
reported
Illustrate the components of good reporting and
accountability
Develop expertise in CB 21 coding of noncredit courses
Evaluate the issues and limitations with indicators
Report issues with current measures
Brainstorm other possible measures of noncredit success
Describe what some other colleges are doing in
accountability
Collect other best practices
Plan to assimilate information into local action
Student Success Stories!
Esperanza
(an ABE/soon-to-be
college student)
The Important Role of Noncredit
ETHNICITY
% Total
Enrollment
Credit
Basic
Skills/ESL
Enrollment
% Total
Credit
Basic
Skills/ESL
Noncredit
Basic
Skills/ESL
% Total
Noncredit
Basic
Skills/ESL
AFRICANAMERICAN
7%
38,265
11.3%
7,900
3.5%
ASIAN
12%
45,880
17%
34,933
15.5%
FILIPINO
3%
10,069
3%
3,012
1.3%
HISPANIC/
LATINO
30%
140,270
41.3%
117,232
52.1%
NATIVE
AMERICAN
1%
3,067
0.9%
694
0.3%
OTHER,
NONWHITE
PAC
ISLANDER
2%
6,471
1.9%
9,688
4.3%
1%
2,912
.9%
688
.3%
WHITE
35%
74,080
21.8%
27,724
12.3%
UNKNOWN
8%
15,931
4.88%
37,511
9.54%
339,278
100%
225,097
100%
TOTAL
CCC General Student Ethnicity 2008-2009 in the General, Credit and Noncredit
Population Compared to California’s Current and Projected Population
Ethnicity
ETHNICITY
% Total
Enrollment
% Total
Credit
Basic
Skills/ESL
% Total
Noncredit
Basic
Skills/ESL
California
Population
2010
California
Population
Ethnicity
Projection
2050
AFRICANAMERICAN
7%
11.3%
3.5%
6%
5%
ASIAN
12%
17%
15.5%
12%
13%
HISPANIC/
LATINO
30%
41.3%
52.1%
37%
52%
1%
0.9%
0.3%
1%
1%
1%
.9%
.3%
0%
1%
35%
21.8%
12.3%
42%
26%
NATIVE
AMERICAN
PAC
ISLANDER
WHITE
Accountability
“Metrics that tell the story…”
 What kind of
ARCC do you
want to build?
 Noncredit is all things to all
people; everyone is on board
 Gathering data is tough
 In some cases we have good
data but can’t seem to get it
on the boat or in the report
 In other cases we cannot get
good data about what is in
the boat or where the boat is
going
Considering Accountability
Healthy accountability should:
 Address higher level learning
outcomes
 Report on authentic student
proficiencies
 Indicate potential interventions and
improvement
 Target improved practice not just
reporting
In Credit Education How Have
We Defined Accountability?
Previously
 Credit attached to units
 Grades
 Degrees, certificates
Now – What are students able to do?
 Student learning outcomes
The Puzzle of Accountability
Current statewide data
 only 2.3 – 5.1% of noncredit students
transition to credit
 All noncredit courses without grades report
zero success.
Is this the noncredit story?
The Puzzle of Accountability
Noncredit needs to:
 Describe noncredit work for funding and accountability
 Explain how and why noncredit is different from credit
 Identify metrics that reflect the work of noncredit
 Go beyond reporting numbers
 Numbers may measure what you want – or may not
 Numbers without context are misleading
 Numbers don’t correct problems
 Qualitative data is essential
 Most noncredit programs have no researcher
Healthy and Responsible
Accountability
 Defines what a student should be able to do
 Identifies a way to assess it
 Collects accurate and relevant data based
on the appropriate assessment
 Analyzes and discusses the data
 Changes practice
 You have always done this!!!
Healthy and Responsible
Accountability
Should acts like vital signs or a compass
informing practice
What could this mean in
noncredit?
 Progress from ABE to ASE
 Completion of GED
 Citizenship
 Completion of ESL Civics
modules
 Bridging to credit
 CDCP certificates
 CASAS
(https://www.casas.org/home/in
dex.cfm)
Reporting requires functional
processes at several levels
Program
completion
reported
Course and
activity data
must be
reported by
faculty
Data gets
into college
MIS
reporting
Correct data must get
reported to CCCCO for final
accountability reporting
Existing Accountability Reporting
in Community Colleges
 Three annual accountability reports
1. Focus on Results: Accountability Reporting
for the California Community Colleges
(ARCC)
2. Career Development and College Preparation
in the State: Supplement to the ARCC
Report
3. Basic Skills Accountability (ARCC
Supplemental)
 “report cards” on a variety of measures
How do we use COMIS* data?
Accountability Reporting
Research Questions
Mandated Reporting
•
•
•
•

•
•
•
•
•
•
Legislative Analyst Office
Department
of Finance
Accountability
Reporting
California
Postsecondary
(ARCC, ARCC
Education
Commission
supplemental,
California Student etc)
Aid
Commission
Career Technical Education
Public
(CTE)Policy Institute
UC/CSU
 Perkins
Indicator
Legislature
– Core
Committees
and
Reports
individual members
Community
 PerkinsCollege
Allocations
Organizations
Justification & Funding
Newspapers
 Matriculation
Labor
Unions
 EOPS
Data Matches
 DSPS
to UC/CSU/NSC
• Transfer
BOGW Administrative
match
• Dept.
of Social Services
Funding
• EDD/UI Match/Wage Study
* Chancellor’s Office MIS Data
Other Reporting
Justification
& Funding
• Matriculation
EOPS
 •Federal
• DSPS
 Integrated
Postsecondary
Career
Technical Education
Education
Data System
Perkins
Core Indicator
Reports
(IPEDS)
Reporting
Perkins
Allocations
BOGW
Administrative
 CCC
Data Mart Funding
Federal Integrated Postsecondary
 Annual Staffing Report
Education Data System
(IPEDS) Reporting
CCC Data Mart
Annual Staffing Report
*CCCCO
17
Management Information Systems
Statewide ARCC Data 20082010
ARCC DATA
Indicator
Student Progress & Achievement
Completed 30 or more units
Fall to Fall Persistence
Voc Ed Course Completion
Basic Skills Course Completion
Basic Skills Course Improvement
ESL Course Improvement
Statewide Rates
2008
2009
51.2%
51.8%
70.4%
71.2%
68.3%
69.2%
78.2%
77.7%
60.5%
60.5%
50.0%
51.2%
44.7%
50.1%
Is this the noncredit story?
2010
52.3%
72.4%
68.7%
77.6%
61.5%
53.8%
50.2%
CDCP– Career Development &
College Preparation
 Certificate = a simple accountability metric
 Noncredit is funded less per FTES than credit
 SB 361 increased noncredit funding from
$2,626 per FTES to $3,092 per FTES
 CDCP includes basic skills, ESL, CTE and
“workforce preparation” courses
 Applies to students enrolled in a sequence of
courses leading to career development or
college preparation (CDCP certificates)
 Problem with Minimum Qualifications
CDCP Progress and
Achievement Rate


Cohort
 Students taking courses for the first time at any CCC
 Did not enroll in any credit courses during the first term
they enrolled in CDCP
 Must have completed 8 or more positive attendance hours
in CDCP courses within their 1st two terms of attendance
Performance indicators – within 3 years
 Completed at least 1 degree-applicable credit course
 Earned a CDCP certificate
 Achieved “transfer-directed” status
 Achieved “transfer-prepared” status
 Earned an AA, AS, and/or credit certificate
 Transferred to a 4-year institution
Persistence Indicators
Is this the noncredit story?
CDCP: Wage Trends
Is this the noncredit story?
CDCP: Wage Trends
CDCP Wage Reporting
Potential Problems with the CDCP
Reporting
 Cohort
 Students taking courses (CDCP or CDCP plus
other noncredit courses) for the first time at
any CCC
 Like ARCC, this excludes students who take
a CDCP course subsequent to a credit course
 Only system-level data reported – noncredit
students across the state (no college-level
data)
Potential Problems with the
New ARCC Supplemental Report
 Reports progress through English, Reading,
Math, ESL levels to transfer
 Needs work on ABE/ASE, VESL
 Reports transition to credit
 Reports degrees or certificates in credit
 All of these are currently zero for noncredit
because there are no grades or way to
track successful progress to outcomes
 See Handout
ARCC Supplemental
Is this the noncredit story?
ARCC Supplemental
Volume and Percentage of First Time Noncredit Students Receiving Matriculation Services
College
Received
Total
Received
Placement
Students Orientation Percent Assessment Percent
2728
Mt San Jacinto
422
Napa
426
North Orange Adult 3942
Mt. San Antonio
396
9
1
509
14.5%
2.1%
0.2%
12.9%
165
36
0
509
6.0%
9.5%
0.0%
12.9%
Is this the noncredit story?
Received
Counseling
40
4
2
157
Percent
Received
Followup
Percent
1.5%
0.9%
0.5%
4.0%
92
7
0
38
3.4%
1.7%
0.0%
1.0%
(CB) Course Basic Data Elements
 Every course is described or defined by 24
course basic data elements (CB)
 Some examples:
 Course title (CB 02)
 TOP code (CB 03)
 Credit status (CB 04)
 Credit – degree applicable
 Credit – not degree applicable
 Noncredit




4/8/2015
Transfer status (CB 05)
Basic skills status (CB 08)
Course Prior to Transfer Level (CB21)
Noncredit Category (CB22)
29
CB 21 Rubrics Created to Describe
Levels Courses Prior to TRANSFER
Discipline Credit
Noncredit
Likely bridge
to credit
Math
Four levels CB 21 A, Six levels CB 21
B, C, D
A, B, C, D, E, F
Levels C & D
English
Four levels CB 21 A, Seven levels CB
B, C, D
21
A, B, C, D, E, F, G
Level B or C
Reading
Four levels CB 21
A, B, C, D
Five levels CB 21
A, B, C, D, E
Level A or B
ESL
6 levels ESL Reading
CB 21
A, B, C, D, E, F
8 levels ESL
Integrated CB 21
A,B,C,D,E, F, G, H
Most noncredit
end 2 levels prior
to English 1 A at
Level B
6 levels ESL Writing
CB 21
A, B, C, D, E, F
Includes
vocational and
Cultural skills
6 levels ESL Speaking
Success Conference 2009
& Listening CBStudent
21
A, B, C, D, E, F
30
TOP code changes
Deleted T.O.P. codes
4930.21 – Writing
4930.70 – Reading Skills Development
4930.71 – Reading Skills, College Level
New T.O.P. Code or Existing Codes
1501.00 – English (writing)
1520.00 – Reading
4930.40 – Career Technical Computational 1701.00 – Mathematics, General
Skills
1702.00 – Mathematics Skills
4930.41 – Pre-Algebra (Basic
Math/Arithmetic)
4930.42 – Elementary Algebra
4930.20 – Communication Skills
1506.00 – Speech Communication
or 4930.33 – Learning Skills, Speech
Impaired
or Other appropriate T.O.P. codes
4930.80 – ESL–Intermediate
4930.84 – ESL Writing
4930.81 – ESL–Advanced
4930.85 – ESL Reading
4930.82 – ESL–Elementary
4930.86 – ESL Speaking/Listening
4930.83 – ESL–Degree-applicable
4930.87 – ESL Integrated
4930.91 – ESL Civics
4/8/2015
4930.87 – ESL Integrated
or 4930.90 – Citizenship
31
CB21 Rubrics
“Design to Implementation”
 Sample ESL course outline
 Breakout groups:





Review your course
How will the rubrics be used?
Who will be involved?
What challenges/barriers do you anticipate?
What strategies will you use to implement?
 Report Out
ESL Course Outline
Write a paragraph of 125 words that has a topic sentence and supporting details
Write a narrative paragraph in chronological order
Write a descriptive paragraph in spatial order
Write a persuasive paragraph with supporting reasons and evidence
Write simple and compound sentences using correct word order
Apply the writing process of brainstorming, drafting, revising, and editing
(including peer reading and instructor feedback) to paragraph writing
Identify subjects and verbs in a sentence.
Edit their own writing for the following:
Correct verb tense (simple present, simple past, future, present continuous, past
continuous, present perfect, present perfect continuous)
Irregular verb forms
Subject verb agreement
Capitalization
Run –on sentences and comma splices
Identify the passive voice and its uses
Identify gerunds, infinitives, and base form verbals
Identify dependent clauses
Demonstrate correct use of coordinating conjunctions
Demonstrate ability to use a dictionary to edit their own writing
Reading Course Outline
Upon completion of Reading 961 the student will:
Condition of Learning: Students will be able to demonstrate the following
outcomes on readings approaching college level.
1. Apply vocabulary-building strategies to improve their analysis of
readings.
2. Demonstrate a literal comprehension of readings, through identification
and analysis of main ideas, supporting details and rhetorical patterns of
organization and development.
3. Critically analyze and evaluate reading material; make inferences;
determine a writer’s purpose and tone; and apply rhetorical reading
strategies.
4. Monitor positive and negative comprehension signals and apply
appropriate strategies to correct incomplete comprehension in a variety
of reading modes.
5. Perceive themselves as achieving college level reading skills.
CB 21
 Coding the CB 21 information
 Problems
 Feedback on rubrics
What is going on with these
data?
Problems
 Definitions are
incomplete
 Metrics are not
valued
 Data tracking is often
not meshed with MIS
 No way to indicate
progress or
completion
Solutions
 Define from the field
 Educate about metrics
– benefit and value
along with negative
consequences
 Describe useful data
tracking - e.g. College
of the Desert
committee, Mira Costa
method, North Orange
DREAM TEAM
 Discuss progress
markers or grades
Potential Additional Metrics
 Citizenship
 ABE/ASE
 Student identified outcomes – help
children with homework, get a job,etc
 CASAS
Examples of Solutions to these
Accountability Problems
Is this REALLY the NONCREDIT Story?
College of the Desert
San Diego Continuing Education
School of Continuing Education
NOCCCD
 MiraCosta
Lunch 12-12:30 and come back for the
solutions and local college work




Is This ReallyOur Story?
 College of the Desert
College of the Desert
BSI Research Project for Academic Improvement
 Mission Statement

The BSI Research Project for Academic Improvement will
close the loop between research and effective action in
all areas requiring basic skills by providing the right
information to the right people at the right time.
 Values Statement : We value a research project that is:



Informative: It delivers data in ways that effectively
inform efforts to improve learning;
Supportive: It includes mechanisms to help faculty and
administrators understand, value and use research;
Readily available: It makes data and information easily
available in user-friendly formats.
College of the Desert
BSI Research Project for Academic Improvement
 Data sets for research agenda



All new students each year
Affective and practical data (SSTK, CCSSE, others)
Academic data
 Baseline data

FA/SP 2005 through 2009
 Longitudinal data


Each year, new cohort
Research, Report
College of the Desert
BSI Research Project for Academic Improvement
 Gathering data



Committee
Contractor
Data Warehouse
 Providing Information




Data based
Usable formats
Standardized
Customized
 Closing the Loop


Training
Research Projects
Is this really our story?
 San Diego Continuing
Education
 Recorded CDCP Progress
 San Diego
Continuing Education
 Actual CDCP Progress
2005-2006
to
2007-2008
2005-2006
to
2007-2008
CDCP Progress and
Achievement Rate
4.2%
CDCP Progress and
Achievement Rate
17.2%
Is this really our story?
San Diego Continuing Education
National Reporting System Performance Report for Level Completion
Rates based on CASAS Testing
ESL Level
SDCE Performance
07’-08’(Against only
pre-post test results)
CA State
Goals 20082009
SDCE Performance
08’-09’(Against only
pre-post test results)
Beg. Lit.
71.81%
42%
70%
Beg. Low
80.14%
35%
78.37%
Beg. High
73.38%
48%
73.70%
Inter. Low
63.05%
44%
67.02%
Inter. High
62.86%
43%
61.90%
Adv.
33.36%
22%
27.80%
Is this really our story?
San Diego Continuing Education
CERTIFICATES AWARDED
ESL (2008-2009)
CTE (2008-2009)
Reported
Estimated
Actual
Reported
Estimated
Actual*
Reported
Estimated
Actual
Beg.
Low=0
90-110
BIT=
150
Baby Sign
Language=0
50-60
Beg.
High=0
90-110
Culinary
Arts=
96
Early Child
Developt=0
60-80
Inter.
Low=0
90-110
CNA=
120
Family
Comm.=0
40-50
Inter.
High=0
90-110
Metals=
80
Effective
Parenting=0
50-65
Adv.=0
90-110
Auto=
60
Family
Relations= 0
40-50
*Reported by Student Manager
Parenting (20082009)
Is this really our story?
San Diego Continuing Education
What Students Are Saying About SDCCE BSI: Accentuate the Qualitative!
52% of the students strongly
agree
they have made progress in
their academic skills ….
48% of students agree
they have made progress in
their academic skills
64% of the students strongly
agree
the instructors understand
their learning needs
36% agree that the instructors
understand their learning
needs
the instructors understand
their learning needs
40% of the students strongly
agree
The counselor(s) are
available for them when they
are needed
52% of students agree
The counselor(s) are
available for them when they
are needed
=100%
Wow!
=100%
Wow!
=92%
Wow!
Is this really our story?
San Diego Continuing Education
SDCE Success Indicators for a New ARCC
Persistence Rates
Pre-Post Test
Scores
*TABE
*CASAS
*Custom
Certificates
Students’ Personal
Goals Achieved
Is this really our story?
2007-08 SCE Award Data from MIS
Award hours
Program Type
SCE Actual Data
Counts
9
Unknown (Top code 99)
192 – to fewer
than 288
Business and Management
288 – to fewer
than 480
Family and Consumer Sciences
288 – to fewer
than 480
Health
960 or more
Interdisciplinary Studies
Total
Data Link
Program Type
Counts
Administrative Assistant
62
Management
10
Early Childhood Education
21
20
Pharmacy Technician
50
214
High School Diploma
322
27
9
279
Total
465
Data Collection Strategies
 SCE’s “You Count!” Campaign
 Collecting more SSN’s
 DREAM team efforts
 Program improvement
 Tracking student progress
 Benefits of Banner
 Assessment scores
 Enrollment trends
 Certificates earned
Data Collection (cont’d)
 Who is your district
ARCC contact?
 Who on your campus is
sending data to CCCCO?
 If it’s an IT person, it’s
simply data
 Establish a relationship
 Find out what’s in the
CCCCO Data Mart
Is this really our story?
2008-09 SCE Award Data from MIS
Award hours
Program Type
192 – to fewer
than 288
Business and
Management
32
288 – to fewer
than 480
Family and Consumer
Sciences
34
288 – to fewer
than 480
Health
40
960 or more
Interdisciplinary Studies
Counts
Total
303
409
SCE Actual Data
Program Type
Counts
42
Administrative Assistant
1
Management
Early Childhood Education
48
Pharmacy Technician
72
High School Diploma
307
Total
470
MiraCosta Noncredit ESL
Data 2008 - 2009
Term I
Persistence %
Promotion %
Morning Classes
80%
(10% Perfect Attendance)
43%
Evening Classes
80%
(7% Perfect Attendance)
47%
Morning Classes
79%
(9% Perfect Attendance)
50%
Evening Classes
79%
(7% Perfect Attendance)
56%
Morning Classes
81%
(8% Perfect Attendance)
30%
Evening Classes
76%
(5% Perfect Attendance)
54%
Morning Classes
78%
(10% Perfect Attendance)
63%
Evening Classes
74%
(8% Perfect Attendance)
46%
Term II
Term III
Term IV
MiraCosta Noncredit ESL
Data 2008 - 2009
 Overall Persistence Rate – 78%
 Overall Promotion Rate – 49%
 Overall Persistence in open entry and
off site – 77% (range from 65% 90%)
Note: We have averaged 79%
persistence and 50% promotion rates
since 2002
MiraCosta Noncredit ESL
Data 2008 - 2009
We also report:
1) FTES and CASAS Benchmarks
https://www.casas.org/home/index.cfm
1) Statewide Performance Goals and our
actual performance
2) Drop out/Stop out reasons
3) Demographics
4) Student Learning Outcome Data
(SLOs)
MiraCosta Noncredit ESL Data
Does it make a difference?
The Superintendent/President of our
college wrote,
“This is an outstanding newsletter
highlighting the outstanding work of
our ESL colleagues. Congratulations
and please extend my thanks to our
folks in ESL.”
We think it does!
Noncredit: “Student Pathways –
 Work
 Credit
 A Better Life
Credit students use Noncredit
Statewide (Since 1992) –
1 out of 6
credit students
have enrolled in
Noncredit
Source: Patrick Perry, Vice Chancellor Technology, Research & Information
Systems, System Office.
Linking Noncredit to Credit
Instruction











Individual Quick Write: Three Guiding Questions
What are the critical skills my noncredit students require to successfully transition
to credit academic and vocational courses?
What has my college done to establish pathways for students to transition from
noncredit to credit academic and vocational courses?
What are the obstacles? What has worked well?
How do I define my role in assisting students in my program transition to credit
academic and vocational courses?
Panel Presentation of current projects that promote the successful transitions
from noncredit to credit academic and vocational courses.
Discussion Groups
Small group discussion, having participants share/discuss the Three Guiding
Questions from Quick Write.
Convene whole group. Share summary of responses from each group
Close with research statements about the importance of the transition process
and a discussion of next steps.
Linking Noncredit to Credit
Instruction
 Matriculation Services: Career Awareness, Career
Assessment, Educational planning, AB 540 implications
Educational Opportunities: Basic Skills, Work
enhancement, Certificates, Degrees, Transfer,
Enrichment
Issues: Foreign Transcript Evaluation, Navigating the
community college process, Support Programs and
Services, English and Math Flow - non-credit to credit
Prerequisite skills and knowledge: Computer skills, Form
completion, Time management
Statewide Efforts
 Noncredit Paper and
Recommendations
 Adjunct
 PCAH
 Title 5
 MQs
 Noncredit Pilot Progress Indicators
Taskforce