Vocational Research and Accountability Committee
Minutes—June 13, 2014
WestEd Offices
Sacramento, CA
DRAFT
Attendees:
Marc Beam, Shasta College/RP Group
Kathy Booth, WestEd
Dianna Chiabotti, Napa Valley College/State Academic Senate
KC Greaney, Santa Rosa Jr. College
Nick Esquivel, CCCCO (new monitor)
Martha Gutierrez, North Orange CCD, School of Continuing Education/ACCE
Craig Hayward, Irvine Valley College
Nick Kremer (Chair)
Jason Rancadore, CCCCO (project monitor)
Nick Real (Cerritos College)
Laura Coleman, “COE
Mollie Smith, Palomar College
Elizabeth Wallner, Committee Staff
Guests:
Sharon Boivin, National Center for Educational Statistics (by phone)
Renah Wolzinger, Golden West College (on assignment with CO) (by phone)
As chair Nick Kremer called the meeting to order at 10:00. He introduced Nick Real who is the Dean of
Technology at Cerritos as a new committee member and then asked the others to introduce themselves.
The minutes were reviewed and approved with minor corrections.
1.
Federal Data Gathering for Alternative Credentials
Nick introduced Sharon Boivin who who is with the National Center for Educational Statistics and chairs a federal interagency task force GEMEnA(Group on Expanded Enrollment and
Attainment) charged with improving federal efforts to collect data on non-degree credentials.
Sharon joined the meeting by phone.
She outlined the make up the task force which includes federal entities such as the Bureau of
Labor Statistics, the Census Bureau, and NSF. The goal is to improve the collection of information on the “prevalence and key characteristics of non-degree credentials.” Additionally it is tasked with “improving federal data on enrollment and participation” in all forms of workforce training.
GEMEnA has categorized alternative credentials into three groups: licenses, certificates, and educational credential. She then summarized some of the results gathered from the Survey of
Income and Program Participation (Census Bureau) . Almost 30% of the population holds some

form of alternative certification. (see powerpoint at http://www.workforceaccountability.org/uploads/3/0/5/8/3058793/sharomn_boivin_ppt_2.pdf
Nick noted that Sharon will be part of a panel he has organized for the National Council for
Workforce Education in October.
2.
Creating a skills builder measure
As an introduction to the topic, Nick K. summarized the work of the skills builder sub-group which had held three phone meetings. (see written summary: http://www.workforceaccountability.org/uploads/3/0/5/8/3058793/skills_builder_summary_no tes_6-5-14.pdf
) . While no conclusions had been reached, the work had helped focus the issues.
The full Committee developed a description of a skills builder students. They:

Have taken some CTE coursework

Are no longer enrolled in the California Community College system

Have not completed a community college degree or certificate

Have not transferred to a four-year college
A distinction was made between skills-builders and lifelong learners, those who continue to take classes for personal growth. Dual enrollment students were also excluded.
The Committee endorsed the idea of preparing at least two alternatives for the scorecard committee. The group debated the validity of use of the intent declaration on the application form, course taking patterns, and number of units as the elements of a cohort. Next the issue of how to measure success with the group was discussed. The two alternatives appear to be course success or wage gain. Craig summarized a study of CSU Northridge graduates highlighting how wage gain was used. While no closure on the issue was reached, the group gained a better understanding of the implications of each possible element in a definition and the challenges of finding a satisfactory way to measure success for this group. The goal will be to finalize the discussion and send recommendations to the October meeting of the Scorecard Committee.
(see rough notes of the discussion attached)
3.
Common measures
Jason Racondore reported on the compilation of the common measure momentum points that
Doing What Matters projects have chosen to use for the 2014-15 year.
The most commonly chosen was MP15 (completing two courses in a pathway). It was selected by 17% of the projects. No other momentum point was selected by more than 7% of the projects. The complete list is available on the committee website.
(http://www.workforceaccountability.org/uploads/3/0/5/8/3058793/doingwhatmatters_-
_data_metrics_presentation.pdf)

4.
Data matching with certification groups
Renah Wolzinger joined the group by phone to report on her work to determine the feasibility of establishing data matching arrangements with California licensure boards and national industry certification groups. She reported it has been a challenging journey. She is participating in a national group that is tackling the issue. One of the critical factors is the challenge of finding a common data element to identify those test takers who have been our students. Many testing agencies do not collect social security numbers. There is an effort to market the value of establishing a data matching relationship to industry certification agencies.
5.
Launch board
Jason and Kathy updated the group on the deployment of the Launchboard . Overall it has gone well. The focus now is to enhance what it offers and fix glitches that have appeared.
6.
Updates
Brief updates were offered on the leaver survey and the Scorecard. KC reported that 35 colleges had followed through and conducted the survey this year. She was heartened that the new
MOU would have a provision for allowing wider use of each college’s data. Nick K. reported that the Scorecard committee had met to recommend measures to the Board of Governors for the goals they have adopted.
7.
Perkins and WIOA
Nick noted it appears that WIA (now WIOA) is finally moving through congress. If it passes we will look at its accountability provisions. There is no movement on a Perkin’s reauthorization
8.
Meeting dates for 2014-15
The Committee agreed on the following meeting dates for the coming academic year:
September 26, 2014, February 6, 2015 and June 5, 2015
Meeting was adjourned at 2:35.
(These draft minutes will be reviewed and approved at the September 26, 2014 meeting of the
Committee)

(rough notes of discussion at VERATAC of developing a skills builder measure not official minutes VERATAC June 13, 2014)
Who
Skills-builders are students who:
 have taken some CTE coursework
 are no longer enrolled in the California community college system
 have not completed a community college degree or certificate (as part of this round of course-taking)
 have not transferred to a four-year college (as part of this round of course-taking)
Skills-builders are not lifelong learning students. They are pursuing work-related skills including upgrading skills for a current job, building skills for a new job, or preparing for the third-party certification or re-certification.
Dual enrollment students will be excluded.
How to Identify

Concerns about the accuracy of both vocational and ed goal data (whether students will be told to pick a transfer goal to get priority enrollment, whether TOP code is correctly assigned)
Anything below 18 units (based on what gets transcripted and reported to MIS)—ideally coursetaking would be linked to intent, to improve our ability to find skills-builders
College Goal
For the affirmative:

Fuller shows that intended goal was a decent measure and captured students whose coursetaking was consistent with the goal, and consistent with ages for skills-builders found in other studies. Greaney shows that intended goal is clearly associated with student reported date (Goals, G, H,
I)

Ed goal likely to get cleaned up because of CCC Apply and the Student Success Act, even if it’s not reliable now

Students are less likely to go all the way down to Goal G if this is what they really mean
For the negative:


All measures in Scorecard are based on behavior, so starting to use Ed Goal will open the door to data tracking that won’t work well for other groups of students

Would leave out some skills-builders

If the group looks small, it won’t serve the purpose of validating skills-builders
Looking backward might be too late, but the Scorecard is currently backward looking
Intent could validate choices but should not define the cohort.
Could do a blended methodology that has both coursetaking and college goal. Then could look to see how predictive college goal is (lets us know how well ed planning is working for skillsbuilders).
Could be that you started out in one path but then moved to another.
Two types: 29 year old who is ready to get the better truck, as well as younger students who end up in a course randomly. Not the same intend, but both may benefit.

Unit threshold: is one class enough? In some disciplines, yes. If do it as a unit, if you have a half unit class, it may be just as valuable in one industry as a degree in another.
Encourage us to think at the level of programs and pathways.
Concerned that single courses might be not-student centered.
Competency based education—such as in apprenticeship— shows the value of recognizing skills gained even at the single course level
Noncredit may also be important. Should we move these courses into noncredit?
How do we handle people who are seeking CEUs or need a single course in a field like technology?
Value of course-taking patterns (like transfer-prepared): it’s different for CTE. You only take English 1A if you want to transfer. But why do students take a woodworking course?
If have two courses, it can be construed as a program.

Stand alone course if they have a goal of G, H, I
Don’t forget that there is value to building skills that isn’t just about money
What if we propose two metrics: one for lifelong learners and one for skills-builders that are career directed
Noncredit may include partial and single courses in CTE, May link single noncredit courses with larger pathways for those who need them. Particularly because many of these students have degrees/credentials already. We need to ensure that whatever system we create, it does not leave out noncredit students and programs.
Suggestion to wait and get research results from Peter Bahr about what constitutes a skills-builder to inform the skillbuilder credit threshold definition used for the Scorecard
MEASURING SUCCESS
Show the average wage gains for skills-builders?

Age limitation an issue? It’s a problem for Salary Surfer but may not be for skills-builders (who are rarely straight from high school)
Whatever gets created for the Scorecard should be based to some degree on Salary Surfer
One way to handle wages:

If no prior wages, if get about the regional living wage, then it’s counted as a success

If there are prior wages, report the wage gain
Could compare to average wage gains for an associate’s degree
(5% wage gain at the nine unit threshold)
Have to take into account inflation, normal progression in an occupation – for example, could use EDD and use the 25% and
75% percentile (need to address the entry level wage