Transforming the Culture of Evaluation in Career and
Employment Services
Saskatchewan Abilities Council
Saskatchewan Education, Employment and Labour
New Brunswick Post-Secondary Training, Education and
Labour
Canadian Career Development Foundation (CCDF)
Goss-Gilroy Inc.
Canadian Research Working Group on Evidence Based
Practice in Career Development (CRWG)

CCDF: Lynne Bezanson & Céline Renald

Dave Redekopp, Life-Role Development Group

Guylaine Michaud, Université de Sherbrooke

Ken Organ, Goss-Gilroy

Provincial Partners
◦ Managers and Practitioners
◦ Clients: all who agree to participate; no criteria except
not one-off’s
Input  Process  Outcome
Resources available
•
•
•
•
•
•
•
Staff: Number of
staff, level of
training, type of
training
Funding: Budget
Agency mandate
Community
resources
Access to
Employment
Opportunities and
Training
Client Employment
History
Client Life
Circumstances
Activities to promote
positive change
Generic interventions
•
Working alliance,
client engagement
Specific interventions and
progress in:
Job Readiness
Career Decision
Making
Skill Enhancement
Work Search
Work Maintenance
Life Circumstances
The support you provide
and progress in personal
attributes:
self-esteem, self
efficacy, well-being,
self confidence,
self-awareness)
Indicators of client change
1. Learning outcomes
• Knowledge and skills
linked to intervention
2.
Personal attribute
outcomes
• Changes in selfesteem, self efficacy,
well-being, self
confidence, selfawareness)
3.
Impact outcomes
• Impact of #1 & #2 on
client’s life, e.g.,
employment status,
training status,
quality, fit, standard
of livinga


What common indicators are applicable across
different client contexts, different client groups,
different agencies, and different interventions?
What statements can be made about service
effectiveness by tracking common indicators of
inputs, processes and outcomes? If numbers
permit, it will be possible to aggregate data to
begin to address the ultimate question, “What
kinds of interventions in what contexts produce
what kinds of outcomes?”
◦ LABOUR MARKET REALITIES: Catchment Area Profile
 Local labour market opportunities – what is the quality
and range of work opportunities?
 Educational opportunities – easily accessible or not?
 Community supports/resources – adequate?
accessible?
◦ CLIENT REALITIES: Client Employment Potential
(demographic info)





Work experience
Level of education
Language
Health factors
Other
5
◦ PRACTITIONER PROFILE




Age
Gender
Experience
Education/Training
◦ SERVICES OFFERED BY YOUR OFFICE
 Provided by managers
6

Build & Maintain Working Alliance with Client

Conduct Employability Needs Assessment
◦ 5 Employability Dimensions and Additional Life
Circumstances

Develop and Update Action Plan

Interventions (throughout 6 weeks)

Track Client Progress (as appropriate throughout 6
weeks)






PRE-EMPLOYABILITY/JOB READINESS: Has the intention to seek employment and/or enter
training but faces challenges, vulnerabilities and/or skill deficits that need to be addressed
before the client can be successful in achieving employment/training goals
CAREER DECISION MAKING: Not yet clear about work/learning goals; needs to understand
personal skills, interest, values and personality and more about different potential occupations
in order to develop a vision for a preferred future
SKILLS ENHANCEMENT: Has career goal, but needs to explore skills and qualifications needed
for the work they want and how to acquire them
WORK SEARCH: Has career goal and necessary skills/qualifications for desired work; needs
skills, strategies and tools necessary to conduct a successful job search
EMPLOYMENT MAINTENANCE: Has difficulty keeping work or may need to work on the
skills/attitudes required for success at work
OTHER: Many circumstances beyond those covered by the employability dimensions may
impact on the client’s employability outcomes. Some of these life circumstances may
necessitate referral outside of career and employment services. Life circumstance issues may
need to be addressed and resolved before progress toward employment can realistically be
made.

Employability Needs
◦ 5 Employability Dimensions
◦ Additional Life Circumstances

Personal Attributes Needs/Working Alliance
◦ Self-management, self-esteem, well-being, self-efficacy,
self-awareness
◦ Working alliance
◦ Client engagement

Labour Market Indicators
◦ Tailored to client status
Not at
all

Identify and clarify future direction (e.g.,training, education,
employment or change in life circumstances goal)

Identify personal strengths/resources that support future
direction (e.g.,training, education, employment or change in
life circumstances goal)
Resolve specific challenges/vulnerabilities that may impact on
future direction (e.g., mortgage, public transit, day care etc.)


Access community resources/supports that address specific
personal challenges/vulnerabilities and assist in helping move
goal forward (e.g., mental health services; addictions
counselling; public housing etc.)

Develop necessary personal supports needed to move toward
future direction (e.g., family, childcare, transportation)

Acquire life/employment skills and attitudes that support future
direction
Acquire attitudes which support future direction
Develop and follow a plan of action to move future direction
forward
Keep appointments and sustain efforts
Other: (please specify)________________________________




Not
much
A little
Quite a
lot
A lot
Indicators of challenges in client life circumstances
are in evidence:



Improved housing is needed
Improved transportation is needed
Improved capacity to work and/or study is
needed

Increase in sense of responsibility for own
choices and behaviours is needed

Increase in ability to set short and long term
goals is needed

Increase in understanding expectations and
demands of employers is needed

Reduction in destructive behavior is needed

Following medical and medication protocols is
needed


Increased openness to change is needed
Increased access to constructive and positive
support systems is needed

Improved relationships with family and friends
are needed


Increased trust in other people is needed
Other: _________________________________
Not
at all
Not
much
A
little
Quite
a lot
A lot
Client needs assistance in the following areas:
Ability to Self-Manage:

More info
Developing stronger Self-Esteem:

More info
Improving sense of Well-Being: i.e.:

More info
Developing stronger Self-Efficacy: i.e.:

More info
Increasing Self-Awareness:

More info

Other (Please specify):
________________________________________
Not
at all
Not
much
A
little
Quite
a lot
A lot

more info: Self-Esteem:
“a person’s overall emotional
evaluation of his or her own worth,
an attitude to self and a judgement of
oneself. Self-esteem encompasses
beliefs (e.g., I am competent; I am
worthy) and emotions (pride, shame).
Self-esteem is the positive or
negative evaluation of the self”

Demonstrates an optimistic outlook

Sees self as competent and able

Has confidence in ability to interact with others

Uses positive self-talk

Acts with little hesitation

Raises few objections

Asks questions

Articulates skills assertively

Makes eye contact

Is hopeful

Takes reasonable risks

Employability Needs
◦ 5 Employability Dimensions
◦ Additional Life Circumstances

Personal Attributes Needs/Working Alliance
◦ Self-management, self-esteem, well-being, self-efficacy,
self-awareness
◦ Working alliance
◦ Client engagement

Labour Market Indicators
◦ Tailored to client status
 quality and “fit” of employment or education/training, salary,
location

Input & Outcome Connections
◦ More educated clients do better than less educated
clients
◦ Success is positively related to labour market
attachment
◦ “No Barrier” clients do better than “Single Barrier”
clients who do better than “Multiple Barrier” clients
◦ Success is positively related to entry self-efficacy
◦ Success is positively related to entry self-esteem

Outcomes and Processes
◦ Learning Outcomes
 Client skill/knowledge increases with any intervention
◦ Personal Attribute Outcomes
 Client attributes improve with any intervention
 Client self-esteem improves with any intervention
 Client self-efficacy improves with any intervention
◦ Labour Market Outcomes
 Employment/training increases with any intervention
 Quality/fit increases with any intervention
 Working alliance is positively related to employment/
training outcomes

Relationships between Outcomes
◦ Learning outcomes are positively related to:
 personal attribute outcomes
 labour market outcomes
◦ Personal attribute outcomes are positively related to
labour market outcomes

Personal Attribute Outcomes & Processes
◦ Working alliance is positively related to self-efficacy
changes
◦ Working alliance is positively related to self-esteem
changes

Learning Outcomes & Processes
◦ Client skill/knowledge increases are positively
correlated with working alliance

Measurement Relationships
◦ Practitioner and client assessments of working
alliance are positively related
◦ Practitioner assessment of personal attributes is
positively related to client self-esteem scores
◦ Practitioner assessment of personal attributes is
positively related to client self-efficacy scores
Input  Process  Outcome
Resources available
•
•
•
•
•
•
•
Staff: Number of
staff, level of
training, type of
training
Funding: Budget
Agency mandate
Community
resources
Access to
Employment
Opportunities and
Training
Client Employment
History
Client Life
Circumstances
Activities to promote
positive change
Generic interventions
•
Working alliance,
client engagement
Specific interventions and
progress in:
Job Readiness
Career Decision
Making
Skill Enhancement
Work Search
Work Maintenance
Life Circumstances
The support you provide
and progress in personal
attributes:
self-esteem, self
efficacy, well-being,
self confidence,
self-awareness)
Indicators of client change
1. Learning outcomes
• Knowledge and skills
linked to intervention
2.
Personal attribute
outcomes
• Changes in selfesteem, self efficacy,
well-being, self
confidence, selfawareness)
3.
Impact outcomes
• Impact of #1 & #2 on
client’s life, e.g.,
employment status,
training status,
quality, fit, standard
of livinga

More evidence for the field to justify the
power and efficacy of career interventions

More integrity around outcomes

A culture of evaluation built into career and
employment service practice

Over time, the ability to answer:
◦ “What kinds of interventions in what contexts
produce what kinds of outcomes”
20
Research Reports will be completed by May 31,
2013
Follow the CRWG website at:
www.crwg-gdrc.ca
Thank You