Aligning Workforce Programs with
the Labor Market: Strategies and
Tools for TAACCCT Applications
National Council for Workforce Education (NCWE)
Darlene G. Miller, Ed.D., Executive Director
Heldrich Center for Workforce Development, Rutgers University
Michelle Van Noy, Ph.D., Researcher
Jennifer Cleary, Ph.D., Researcher
Burning Glass Technologies
Kelly R. Bailey, Director of Partnerships and Alliances
What this presentation can do for you
 Provide a framework for thinking about how to build
TAACCCT-funded workforce programs that align with the
labor market, as well as the needs of students and other
stakeholders.
 Offer tools and information to aid in building strong
programs.
 Provide tips on how to build an evaluation-friendly
workforce program.
Overview of SGA and Labor
Market Alignment
USDOL Priorities Challenge Colleges to Create
New Program Models
Key Goals:
 Align programs with the needs of the labor market
and students
 Use sector-based & evidence-based approaches
 Evaluate program implementation and outcomes
Data systems and administration goals
 Open sharing of curricula
 Coordinated data collection across programs
Evidence-Based Design
 Three levels of
evidence:

Strong

Moderate

Evidence of
Promise

Strong Theory
Priority program approaches:
• Sector-based
• Career Pathways & Stackable Credentials
• Employers involved in ALL PARTS OF
PROGRAM
• Use Real-time and other LMI (program
selection, student advising)
• On-line learning
• Accelerated, contextualized remedial education
• Competency-based & Work-based learning
(including entrepreneurship)
• Share outcomes and LMI with students
• Credit granting for non-credit work
Higher Education Labor Market
Program-level Alignment Model
Data Collection/Validation
Outcomes
Credential
Attainment
Incorporate
Employer
Needs
Student
Needs
Employment
outcomes
Program
Selection
Recruitment
&
Enrollment
Curriculum,
Teaching, &
Assessment
Relationship-building
Support
Services
Employer/
Student
Satisfaction
Incorporating data collected into the program is a multifaceted,
complex process.
It is more than program selection!
Support &
Follow-up
Services
Program
selection &
enrollment
Workbased
Learning
Employer &
Student
Needs
Competency
Assessment
/ Self-Paced
Learning
Curriculum
development
Instructional
strategies
Contextual
Factors
•Institutional
Factors
•Program/
Institutional
Structure
•Labor
Market
Dynamics
The SGA requires alignment activities occur at
many organizational levels
System
Policies/systems/facilities to
allow:
- Credit for non-credit work
- Flexible schedule /enrollment
- IT/On-line instruction
- Accelerated & competency-based
learning
- Work-based/hands-on learning
Institution
Dept
Program
- Effective pedagogy for
contextualized and accelerated
learning
- Data systems to share student outcomes
- Policies for open sharing curricula
Class
- Policies to allow for sharing of faculty
for contextualized ed.
Systems and skilled staff for:
-Data collection and analysis
-Incorporating data into program
selection and all program activities
-Measure interim goals
-Build relationships with stakeholders
Strategies for Successful
Program Design
Alignment Activities:
Systemic Change
Grant Leads to System Change within the Institution
(not a boutique program)

Embedded into the culture of the institution


Difficulty of navigating internal barriers
College level commitment
Administrative Engagement
 Cross-functional Leadership Team


For Consortiums
Consortia Leadership Team
 Consortium Director

Alignment Activities:
Relationship Building
Relationships to Improve Outcomes and
Meet Grant Priorities

DOL Priority on Long Term Unemployed


Relationship with Workforce Investment Boards
Strong Employer Engagement and Commitment
Career pathways and stackable credentials
 Employer validation and data validation
 Employer validation of supply/demand
 Role of Industry Associations
 Employer commitment to work-based learning

Alignment Activities:
Relationship Building
Relationships to Improve Outcomes and
Meet Grant Priorities

For Consortium Grants – Peer Learning Networks
Learning best practices from each other
 Sharing curriculum: Don’t reinvent the wheel
 Online and face-to-face
 Platform and compatibility
 Data collection and sharing
 Data systems in synch

Alignment Activities:
Data Collection and Validation
Alignment with Needs of Labor Market and Students

Real Time Jobs Analysis*
Allows understanding of job vacancy and skill needs of local labor
market
 Utilizing multiple “reliable” sources
 LMI Data, Local WIB data, state data, industry association data


Gap analysis of competing programs
What are other colleges in your area doing
 What about local CBOs or apprenticeship programs
 Understand available supply to fill job vacancies

Alignment Activities:
Data Collection and Validation
Alignment with Needs of Labor Market and Students

Convening stakeholder groups to collect information on job
vacancy and skill demand
Multiple relationship building activities (WIBs, employers, etc)
 Ensures multiple sources of evidence for demand and supply
 Validation by the right person (HR manager v CEO)
 Allows for collection of skills information for curriculum
development
 Locally recognized industry-based credentials


Aligning enrollment targets with job vacancies and gaps

Employment data over time
Alignment Activities:
Program Implementation
Curriculum Development

Open sharing of curriculum

Curriculum is no long proprietary or owned by the faculty
 Open Sharing
 On-line using DOL platform
• Faculty support in budget for making the curriculum public and
accessible

Acceleration strategies
Evidence-based design
 Integration and contextualization for low-skilled adult learners

Alignment Activities:
Program Implementation
Curriculum Development

Credit vs Noncredit
Noncredit must be linked to industry-recognized credentials
(NIMS, NCCER, etc)
 Articulation between noncredit and credit


Acceleration and Credit for Prior Learning

BE CLEAR WITH THE FACULTY UPFRONT REGARDING PLA
 College policy
 Accreditation
 Transparency
“Real Time” Jobs Data
Evaluation Approaches and
Considerations
Evaluation Consideration :
Outcomes Measurement and Program Design
 Randomized Controlled Trials
 Plan for recruitment and assignment?
 Regression Discontinuity
 Assessment tests with cut-off scores? Common across colleges?
 Comparison Group with Controls
 Drawn from similar programs at college either from prior or
current years
Evaluation Consideration :
Outcomes Measurement & Data Collection
 Develop a plan to collect key student data

Including Student Name, SSN, DOB
 Labor market outcomes data

Partnerships with state to link with UI wage data
 Other important data also need to be collected

Courses, grades, services received, assessments, scores, # attempts,
activity in on-line courses, attendance
 Plan to store data & ensure consistent data entry
Evaluation Considerations: Program
Implementation Goals
 Document implementation to inform outcomes
analysis

Provide detail on how program was implemented including
strengths and weaknesses
 Uncover opportunities to strengthen implementation
 Evaluation can be an opportunity for on-going learning,
reflection, and feedback
Evaluation Considerations: Program
Implementation Components
 Multiple data collection efforts to provide a full
understanding of implementation may include:



Focus groups
Surveys
Interviews
 Include the perspectives from a variety of
stakeholders including:

College staff, students, employers, other partners
Evaluation Considerations: Program
Implementation Issues
 How did you choose program design?
 Employer involvement? Evidence?
 Does the delivery method fit student and employer needs?
 Are the assessments sufficient and well implemented?
 Abilities, skills and interests
 What ? Who? How implemented? How used?
 Is the program structure sufficient to ensure data collected on
skills, etc. is incorporated in all program components?

Is there a communication or quality control system in place to ensure all parts of
the program are coordinated around stakeholder priorities.
 Are your supportive services directly addressing employer, as well
as, student needs?
 Career guidance for students, support of employer skill priorities
 Are your partners involved in all aspects of the program
 What relationship building activities have you created to recruit, retain, and
engage partners?
Final Thoughts
Final Thoughts

Create a dashboard of metrics right up front along with a
workplan for all of the partners


Don’t bite off more than you can chew


Make sure all of your partners can live with the plan and
understand their role in accountability
Be realistic in your outcomes
 Just because you think a reviewer wants to see this, don’t put
things into the grant that you cannot accomplish
Increased scrutiny by DOL
Congress wants to know that their money was well spent
 Importance of a strong evaluation plan
 Take audits seriously

NCWE Team of Technical
Assistance Providers
Darlene G. Miller, Ed.D., NCWE Executive Director
Nan Poppe, Ed.D., Campus President Emeritus, Portland
Community College
Rebecca Nickoli, Ed.D., former Vice President of Corporate College
(Workforce and Economic Development), Ivy Tech Community
College
Nick Kremer, Chair, CA Vocational Research and Accountability
Committee and former Dean of Career and Technical Education,
Cerritos College
For More Information…
 Darlene Miller, executivedirector@ncwe.org
603-714-1918
 Michelle Van Noy, mvannoy@rutgers.edu
732-932-4100 x6213
 Jennifer Cleary, jcleary@ejb.rutgers.edu
732-932-4100 x6215
 Kelly R. Bailey, kbailey@burning-glass.com
732-800-2484