Dr. Frank Olmos and Joshua Kahn
Los Angeles County Office of Education
PTC-SC Luncheon Presentation
Monterey Park, CA, May 29, 2013
Internet Recruiting
Broad Outreach
Online Applications
Low-effort applying
Reduced Self-filtering
High unemployment
Large Applicant Pools
Reduced Public Sector Budgets
Workload Problem for many merit systems
2

Limit exposure
▪
▪
▪
▪

Limit posting sites (no Monsters)
Paper bulletins only
Shorten posting period
Do not accept “interest cards”
Random selection
▪
3

Lottery
Possibly skew toward lower
end
None are “merit” based

Increase burden to apply
▪ Paper app, hand pickup and deliver
▪ Charge $ per application ($5 is legal)
▪ Increase requirements for complete app
(proof of diploma, degree, transcripts)


Limit application window
▪ Open for one day or hour
▪ Posting cut-off at x-number of applicants
(first come first serve for testing slots, selfschedule)

Will reduce your pool, but…
 At best, not improve on the
average level of ability


Negative impact on image
as public employer affecting
other recruitments
Increase protests, appeals
How will you defend this?


Large applicant pools increase the potential
utility of a valid selection procedure.
Utility: The expected organizational gain from
your selection procedure
▪ Gain #1: Success rate of those hired
▪ Gain #2: Monetary value of higher performance


4
Validity: the ability of your selection procedure
to identify top talent
Selection Ratio: job openings (n) divided by the
number of job applicants (N). Lower Ratio = Higher Selectivity
Number
The likelihood of recruiting exceptional candidates increases
with large pools.
large
small
Exam Score
5
Large pools enable identifying pass points.
6
Large pools enable high pass points.
Job Performance
Selected mean= 36
Overall mean = 26
Exam Score, Current Employees
7
How to be efficient without sacrificing quality?
1. Analyze where large pools take the most
effort
2. Utilize most efficient job-related screening
tools first
8
9
1.
Self-Selection
2.
Efficient first stage screening
3.
Broad-based testing
Self-selection
10
1.
Focused Recruitment
2.
Clear Job Descriptions and Requirements
3.
Realistic Job Previews

Occupation-related websites
▪ Dice.com, OhSoEZ.com

Social Networking websites
▪ Professional bulletin boards and discussion groups
▪ LinkedIn Groups

Regular community-oriented networking
 Old fashioned paper bulletins and local media distribution
 Career preparatory schools and training organizations
 PTA and booster organizations
 Internal email, intranet, word-of-mouth by employees
11

Clear duty statements and minimum qualifications
▪ “Any combination of education and experience…” ??

Realistic job preview
 Clear statement of rewards and challenges of the job in
bulletin and other job description information.
 Web-based preview/orientation
▪ Paraeducator, Special Education
▪ 3 hour, on-line workshop, must be completed as part of the
application process
▪ Uses Adobe Connect
12
Efficient first-stage testing
13
1.
Automated objective written tests
2.
Un-proctored Internet Testing
3.
Auto-scored supplemental questionnaires



Scantron (mark-sense answer sheets) or NCS
automated scoring and score upload.
Work horse of selection procedures, and still
among most efficient.
Test BEFORE screening for minimum
qualifications
▪ Review minimums for only those who pass the test.
14

Basic skills testing (reading, writing, math, reasoning, etc.)
▪
▪
▪
▪
▪
▪

Offered by all major test publishing and consulting firms
Register or apply on-line
Computer administered and timed
May be computer adaptive testing
Candidate cheating and impersonation remain issues
Those who pass are invited in for verification testing and subsequent exam
parts (performance, orals, etc.)
Completion of online workshop post-testing
▪ Use with on-line preview/orientation with a multiple choice exam.

Online empirically-keyed personality testing, situational
judgment, biodata, and occupational preference/suitability
▪ Less prone to faking and cheating
15
1.
Part of the application process
▪ Could be just a few questions focused on MQs
▪ Could be automated tally of years experience and level of education
▪ Potentially useful but blunt instrument
▪ Prone to candidate inflation due to “interpretation”
2.
Based on job analysis to identify areas to assess
▪ Type and level of duties and related tasks
▪ Skills and facets of the skills
▪ Each area is broken down into individual questionnaire sections and
items
▪ Should require some form of verification
16

To what extent have you performed the following EXCEL/spreadsheet
tasks?
1.
2.
3.
4.
5.
6.
7.

To what extent have you performed the following WORD/word processing
tasks?
1.
2.
3.
4.
5.
6.
7.
17
Set up rows and columns of data
Cut and paste data from one spreadsheet into another
Write formulas in cells that reference more than one worksheet
Create pivot tables
Create graphs
Use built-in statistical functions
Write macros to automate routine functions
Formatted documents with section breaks
Formatted documents in “column” format
Created and formatted tables
Inserted pictures and graphs
Created locked forms with "form fields"
Used "track changes" feature in document editing
Used mail merge to insert Excel source data into templates
0 = I have no background in this
0 = I know what this is but I have not done it
1 = I have assisted others (or received training)
in this, but not done it independently
2 = I have done this independently but not
frequently
4 = I have done this as a regular part of my job
18
19

Describe an accomplishment that illustrates your
skill level in conducting job classification work.
(Please limit your response to 200 words.)

Describe an example of a task you performed that
illustrates your skill level with Excel/spreadsheets.
(Please limit your response to 100 words.)

Or for skills, actual testing to verify (similar to UIT)
Stage 1
Calculate MAR – Sum of Minimally Acceptable Responses for
each item.
2. Set pass point based on self-rated scores. (If based on
applicant flow management, pass 10-20% more than you
actually need.)
1.
Stage 2
For those that exceed the pass point, have SMEs compare the
supporting examples to the self ratings.
2. SMEs apply a weight of 0, .5, or 1.0 to the area of self rating.
Where:
1.
▪ 1.0 = example is consistent with the self ratings
▪ .5 = questionable support for the self ratings
▪ 0.0 = lacking support for the self ratings
3.
20
Apply the corrected scores to the pass point to determine
who actually has passed the hurdle.




21
Core test battery for as many classifications as possible
Score banking and certification to different exam plans
Different weights and cut-off scores for different
classifications based upon relevancy and level needed
College-Board Model (SAT, GRE, etc.)

Many related classifications
 Differ on specific job content and levels of responsibility
 Many obstacles to class consolidation - logistical, political, fiscal

Each class has a unique examination
 Variable content and quality, not consistently updated
 Many of the same candidates
 2-3 month recruitment & examination cycle time

No overall competency structure across the classes
 Lack of core competencies and competency progression
 Pass point paradoxes (lower pass points for higher level jobs based on
selection ratios)
22





23
Improve testing reliability and validity
Improve coherence of clerical examination
plans
Reduce redundant testing
Increase recruitment efficiency
Reduce time to place candidates on
eligibility lists
Method 1:





24
Recruit for broad-based test
Annual calendar of assessment
(e.g., quarterly)
Establishes a pre-tested pool of
potential candidates for the
relevant classes
For most job classifications,
limited or no public posting for
job specific recruitments
Invitations to apply for specific
jobs are sent to pre-tested pool
based on test cut-off score
Recruit
for Job A
General
Recruit To Take
BBC
Population
PRE-TESTED
BBC POOL
Recruit
for Job B
Recruit
for Job C
Method 2:





25
Recruit for a classification or
class job family
Build a pre-tested pool from
each recruitment process
Candidates apply for each
recruitment process
Allow non-tested candidates
to participate in each process
& exams
Pre-tested candidates are
informed of their status based
on test cut-off scores
1.
2.
3.
4.
5.
6.
7.
26
Sequencing and ordering speed and accuracy (timed)
Checking and comparing speed and accuracy (timed)
Computational speed and accuracy (timed)
Following Instructions (MC)
English Usage & Grammar (MC)
Data Entry speed and accuracy (timed, OPAC)
Microsoft Word Skills (Performance, OPAC)
1. Clerk
2. Intermediate Clerk
3. Senior Clerk
4. School Clerk
5. Senior School Clerk
6. Data Control Clerk
7.
Senior Data Control Clerk
8. Typist Clerk
9. Intermediate Typist Clerk
10. Temporary Office Worker
11. Senior Typist Clerk
12. Department Assistant, Dance
13. Department Assistant, Music
27
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
Department Assistant, Theater
Department Assistant, Visual Arts
Receptionist
Reader
Information Resources Specialist
Media Dispatching Clerk
Secretary
Division Secretary
Legal Secretary
School Administrative Secretary
Senior Division Secretary
Executive Legal Secretary
Executive Assistant









28
ADMINISTRATIVE ANALYST
ADMINISTRATIVE ASSISTANT
ADMINISTRATIVE AIDE
ASSISTANT ADMINISTRATIVE
ANALYST
RESOURCE & DEVELOPMENT
ANALYST
PROJECT COORDINATOR
COMPENSATION ANALYST
RESEARCH ANALYST
LEGISLATIVE ANALYST







BUDGET ANALYST
HEAD START PROGRAM
RESULTS SPECIALIST
HUMAN RESOURCES AIDE
HUMAN RESOURCES ANALYST
HS PROGRAM DEVELOPMENT
SPECIALIST
ASSISTANT HUMAN
RESOURCES ANALYST
LABOR RELATIONS SPECIALIST







29
CUSTODIAN
SENIOR CUSTODIAN
MAINTENANCE WORKER
SENIOR MAINTENANCE WORKER
DELIVERY DRIVER
REPROGRAPHICS WORKER
UTILITY WORKER
The Take Back Message
30
31

Self-Selection (Pre-Application Stage)

Focused Recruitment

Clear Job Descriptions and Requirements

Realistic Job Previews

Efficient first stage screening (Pre-Invite Stage)

Automated objective written tests

Un-proctored Internet Testing

Auto-scored supplemental questionnaires

Broad-based testing (Efficient Testing Administration)

One Test for Multiple Classifications

Score Bank Candidate Scores

Improve reliability, validity, and efficiency
What are you doing?
32