C
H
A
P
T
E
R
␣
F
O
U
■
R
Chapter 4
Phase Four: Evaluating Jobs
Key points made in this chapter
The evaluation process:
❍
❍
❍
should be done on a factor-by-factor basis rather than job-by-job
should include a sore-thumbing process which profiles the relative value of each job in
terms of each factor
must avoid gender bias
By using predefined criteria, job evaluation allows jobs that are very different from each other
to be compared. Jobs are evaluated by measuring them one-by-one against each factor and
assigning a rank or level according to relative degrees of difficulty, importance,
disagreeableness, and so on, depending on the factor in question.
Job evaluation, using predefined criteria, does allow jobs that are very different to be
compared. It is possible to compare apples and oranges on criteria such as fibre content,
vitamin content and calories!
Guiding Principles
Respecting the following five principles is important during this phase, as it is at every stage in
the process:
❍
❍
❍
❍
❍
Gender neutrality
Consistency
Fairness and openness
Respect for the organization’s values
Being aware of personal biases and resisting them
Four main ingredients, when used in combination, will help an organization to determine the
appropriate value for jobs.
1. A job information tool that collects accurate, complete, up-to-date and unbiased job
information.
2. Job evaluation factors that are broad and appropriate to the organization’s context, and
which guide the process to the answer.
65
■ ␣ EVALUATING␣ JOBS␣
A N D
␣ A
V O I D I N G
␣ G
E N D E R
␣ B
I A S
3. A job evaluation committee that consists of representative members of the organization
who are trained to be sensitive to gender bias so that they can be as gender neutral as
possible.
4. Sore thumbing the evaluation results using a factor chart that gives a visual presentation of
all the jobs evaluated on each factor so that relationships can be reviewed for consistency
and logic.
All four of these ingredients have the potential to pose their own problems. These include
gender bias or lack of completeness. However, the overall risk of such problems is lower if all
four ingredients are incorporated into the job evaluation process.
Suggestions for Successful Job Evaluations
Chapter 1 discussed pay equity committees in some detail. However, again, the key points
regarding these committees are that:
❍
❍
❍
They play a critical role in minimizing gender bias.
Members need proper training.
They must be representative of the organization in terms of gender balance and range of
positions, and include management and non-management people.
Many of the suggestions listed below involve the work of the job evaluation rating
committee (one of the pay equity committees discussed in Chapter 1 of this Guide).
1. Remember that this exercise is about evaluating jobs not people.
2. Evaluate all jobs (one at a time) against each factor. Evaluating all jobs against each factor,
as opposed to each job against all factors, encourages consistent interpretation and use of
the factors. It also sets the relative values of jobs according to a consistent understanding
of the terms and concepts associated with the factors. Furthermore, if in the future a job
must be rated against all of the factors, a relativity scale will already exist. (See box on
“Factor-by-Factor Evaluation” below.)
66
C
H
A
P
T
E
R
␣
F
O
U
■
R
Factor-by-Factor Evaluation
T
here are a number of reasons for evaluating all jobs against each factor, rather than
each job against all factors:
1.
The job evaluation process is a comparative exercise, and results are relative, not
absolute. If jobs are rated in isolation, the comparative process is compromised.
2.
Factor-by-factor evaluation ensures consistent application of one factor at a time
rather than moving back and forth between factors.
3.
Factor-by-factor evaluation makes it more likely that committee members will be able
to examine each element of the job in a neutral context, without being swayed by their
view of the job as a whole. The committee can eliminate job titles and read job
information without relying on any assumptions about the job.
4.
Existing market values can affect how committee members see a job if they rate the
entire job at once. Factor-by-factor evaluation helps avoid this problem.
5.
If the committee is having difficulty evaluating a job, the factor-by-factor approach
allows the committee to use its evaluation of other jobs to help determine how the
difficult job fits on the factor scale, relative to those already evaluated. This approach
allows a job to be put aside and rated after other jobs have been evaluated.
6.
If an organization has more than one job evaluation rating committee, factor-by-factor
evaluation allows a logical division of responsibilities, with each committee being
responsible for evaluating jobs against a given number of factors.
(Organizations may choose to have more than one committee, or to divide the factors
among members of the committee to speed the process and so that members do not feel as
responsible for their effect on co-workers’ jobs. If an organization chooses more than one
job evaluation rating committee, there should be safeguards to ensure consistency; for
example, at least one person should be assigned to all committees to allow a good flow of
information among them.)
The job evaluation process is a search for the most appropriate rating for a job, not simply an
exercise to come up with a rating. The committee must consider the answer to be correct,
appropriate and defensible. The committee must struggle with the facts until it is comfortable
that it has all the information and is able to make a decision it can stand behind until proven
wrong by someone who may present additional persuasive information.
67
■ ␣ EVALUATING␣ JOBS␣
A N D
␣ A
V O I D I N G
␣ G
E N D E R
␣ B
I A S
3. Eliminate job titles whenever possible. Not having a title forces committee members to
read the job information rather than assume they know the job. Develop some other way
of identifying jobs, such as job codes, to distance members from assumptions about their
ranking and to encourage an open-minded reading of the job information.
4. Rate jobs in random order, not by groups of similar jobs, so that bias is less likely to creep
into the process. (See appendix D for an example of a rating sheet, on which jobs should
be listed in random order by job code or title.)
5. Even if committee members question the information provided about a job, they should
not substitute their own views without checking with a knowledgeable outside source such
as the employee or a supervisor. This is especially true if the job information the employee
has provided has already been verified as part of the evaluation process.
6. Perform the sore thumbing process on each factor. (See the box “Sore Thumbing”, below.)
As the evaluations are done, the hierarchy of jobs in the organization will become clear. As
a final check, before leaving a factor, the committee should use the factor rating sheet to
make sure the hierarchy is consistent and logical. This is the time to discuss any problems
and work out solutions. See appendix E for examples and helpful questions.
Make note of any changes resulting from sore thumbing, such as a job being moved to a
higher or lower level on the factor, and the reasons. A job might be moved because it is more
similar to jobs that were rated at another level, or obviously different from other jobs at its
current level. Or a job might, for example, be moved to a lower level with a note that if the
job changes, the level should be reviewed.
Note: Evaluations examine the relationship between each job and the factor scale: is this job
at Level 1 or Level 2? Sore thumbing profiles jobs at each level of a factor: do all jobs
at Level 1 belong together?
Sore thumbing is not for undoing evaluations because results are different from the status quo;
it looks at the results from an overall perspective and analyzes the logic to ensure ratings make
relative sense. It is an opportunity to question assumptions and note anomalies that will need
rationales to support them when questioned in the future.
The job evaluation results are often full of surprises for organizations. The sore thumbing
process allows the job evaluation committee to make sure that they have taken advantage of
the chance to review results before others do so. If there are results that stick out like a sore
thumb, the committee has the chance to review its logic and make sure that it has the
necessary information and reasons for the rating.
68
C
H
A
P
T
E
R
␣
F
O
U
■
R
Sore Thumbing
S
ore thumbing is the process of looking for things that “stick out like a sore thumb”.
Because job evaluation is about relative values, each job must be evaluated correctly
relative to all other jobs. During sore thumbing, the committee reviews relationships
between jobs and how jobs are grouped to ensure these are appropriate.
If each rating is appropriate, then it is not true that jobs have the same requirement on
Factor Two. On the other hand, it may see that it overlooked something in job A, which
should, in fact, have had the same rating as job B. In this case, the committee would
change the rating for job A to Level 4.
There are four main patterns when jobs are rated against a factor. One is a diamond,
described above — i.e., jobs are clustered in the middle levels, with few or none at the top
and bottom. The second is the ‘top heavy’ factor, where all jobs end up in the top few
levels. The ‘bottom loaded’ pattern occurs when all jobs are in the bottom levels. Finally,
all levels may be used without noticeable pattern. The first three patterns may stimulate
the committee to make sure it agrees with the logic behind the pattern.
After evaluating all jobs on a particular factor, the committee will write up the evaluation
ratings on the factor chart, provided in appendix E. This is a good way to get a visual look
at the results and pick out patterns that may need attention. Sore thumbing should be done
after all jobs have been evaluated against a given factor (or periodically, where there are
many jobs), and again after all jobs have been evaluated against all factors.
7. Do not rate backwards from the desired result. This is also called a “halo” or “pitch fork”
effect, where jobs are given a good rating or a bad one on each factor depending on the
overall image raters have of the job.
8. Take good notes. Since every evaluation is potentially challengeable, notes and rationales
are highly desirable. Any new committee will find them helpful. Indicate in notes instances
in which a job was valued close to the next higher level; in the case of a change in job duty,
extra demands might be all it takes to bump the job up. And if new information or sore
thumbing changes an evaluation, record this in the notes form. (See below and appendix F
for a sample form.)
69
■ ␣ EVALUATING␣ JOBS␣
A N D
␣ A
V O I D I N G
␣ G
E N D E R
␣ B
I A S
Committee Notes and Rationales for Evaluations
Job Title or Code: Purchasing Officer
Factor
Rating
Rationale
Product
Knowledge
4
Must know the material
requirements, substitution
options, suppliers and
prices. Must also know
production cycles and
material requirements for
each cycle.
Responsibility for
Quality
3
Responsible for detecting
obvious problems and
knowing when to return
product.
Disagreeable
Conditions
1
Private office with only
occasional exposure to
disagreeable elements.
Interpersonal Skills
3
Must negotiate with
suppliers when material
is needed quickly.
9. Set up the rules for the job evaluation committee as early as possible, and review them as
needed:
❍
❍
❍
Taking notes — how much should be documented: every decision, or only the difficult
ones or those on the borderline for a factor?
What to do with a stalemate? Usually, putting the job aside until more relationships
among jobs have been established for the particular factor solves most stalemates.
Absence of committee members — how many people must be present to convene the
meeting? Should stand-in committee members be available as needed? Consistency of
committee membership is important for a particular factor, but as each factor is an
independent exercise, new members working on a new factor should not pose any
problems, assuming they have been properly trained.
70
C
❍
❍
H
A
P
T
E
R
␣
F
O
U
R
■
Evaluating committee members’ jobs — this can be a difficult situation, since people have
a tendency to take things more personally than they need. Committee members may be
generous with other members’ jobs rather than face confrontation with team mates. The
committee has an obligation to be as fair with its own as it has been with other employees’
jobs. Of course, it may be wise to leave these jobs until others are done and be guided by
the relative ranks.
Sharing information versus confidentiality — the committee must find a balance between
the need to share information and educate others about the process, and the need to
maintain confidentiality of ratings until the process is complete. Other employees will
often understand little about the process, so if information is to be shared, it must be
accompanied by sufficient explanations.
10. What to do about ambiguous information? The committee might phone or meet with the
employee, or a supervisor, personnel representative, or someone in another department
where the job might have an effect, to shed light on the ambiguous description. Another
alternative is to postpone action until the committee is sure the information is important
to the ranking decision.
11. Make changes when necessary. The committee may discover it needs another level for a
factor, or that terms need to be discussed so that the group has a common understanding
of them. If a committee finds a factor scale too short or too long, it should seek a
solution, even if it requires a decision from a steering committee or help from a
consultant.
The committee should make changes only after carefully reviewing the job information that
was collected and the reasons behind the assigned values. It may happen that although
committee members are not used to thinking of a given job at a particular level, that is indeed
where it belongs. Members must take care that they make changes only to address anomalies,
not to restore their sense of how things should be.
12. Decide on the rules for evaluating prior to doing the evaluations, e.g., “decision by
consensus.” Every decision must be a committee decision. The committee must struggle
with decisions until everyone understands and, if possible, agrees with the result. This will
allow the committee to stand united in the future if there are challenges.
13. As noted earlier in Chapter 1, the most effective committees will have representatives:
❍
❍
❍
of both sexes
at various levels of the hierarchy
from different jobs and departments
71
■ ␣ EVALUATING␣ JOBS␣
❍
A N D
␣ A
V O I D I N G
␣ G
E N D E R
␣ B
I A S
from both union and management (where there is a union) or employee representatives,
for non-unionized jobs
The challenge is often to get sufficient representation while keeping the size reasonable. If a
committee gets too large, it gets unwieldy and often ineffective. A good number to strive for is
no more than six to eight members.
Each committee member participates in each rating. Quite often the best insights or most
probing questions come from people who do not have historical biases or opinions about a
job. Be aware of the baggage or bias, as well as the knowledge, that the committee member
most familiar with a job brings to the table.
Remember, committee members who are quiet often have good insights. Hear what everyone
has to say — by asking.
Try to avoid employees who are hierarchically related, i.e., a boss and subordinates.
14. Remember that the job evaluation process is subjective, as is each of its components.
However it is not arbitrary. If it is performed consistently and neutrally, with proper
attention to the issue of gender bias, it can be fair and can address existing bias.
15. Job evaluation is a living exercise. Jobs will change and, as a consequence, job evaluation
results will need to be reviewed and possibly changed. The process itself, including the
factors and the weights, may even need to be reviewed.
16. Committee members may be concerned about peer perspective. It is important to
remember that this is a significant job, and must be done as fairly and responsibly as
possible. Colleagues may not always agree with each other, but the process is open to
review. It is not uncommon for employees to believe they are undervalued, but because
job evaluation is all about the relative value of jobs within an organization, many jobs
will be found to belong just where they are.
72
C
H
A
P
T
E
R
␣
F
O
U
R
■
Terminology or Assumptions that “Devalue” Jobs
Devaluing (making something appear lower in value than it is) or under-describing has been
very common in describing women’s jobs. Devaluing is most prevalent in the job
information and evaluation stages. Evaluators (and employees themselves) may not include
something because it is “just part of the job.” Thinking about that expression for a
moment, it becomes obvious that those elements are the very ones that need to be captured.
Devaluing terminology and statements have the effect of undermining any neutral
examination of facts. They can lead to gender bias. For example, an expression often used
to describe female-dominated job duties is “basic”, e.g., basic reading, communication,
mathematics, interactions with customers. Often the skill, effort, responsibility, or working
conditions associated with the job extend beyond “basic”. And often degrees of “basic”
may be differentiated. Re-examine any elements assumed to be basic. Other terms that have
devaluing effects are simple, routine or general; if the term only is put before something
else, such as they “only do...,” it tends to make whatever follows sound small.
Things to Bring to the Job Evaluation Table
❍
❍
❍
❍
❍
Job information documents that have been reviewed and synthesized to represent jobs not
individuals
Factors and factor notes for evaluators
Rating forms to record the ratings for all jobs, ratings by factor, and the rationale for each
job’s rating
An open, gender-neutral and fair mind
A willingness to discuss, struggle and represent one’s own point of view, but not with a
mind to a competitive scenario (everyone has the same goal: to determine the right answer
going to the job evaluation committee. Job information must be legible. Only one copy of
each job information package is necessary, as the packages can be circulated.)
Things not to Bring to the Job Evaluation Table
❍
❍
❍
❍
❍
Current salary information or assumptions
Current hierarchical relationship information or assumptions
Hidden agendas
Gender bias or favouritism
An ego or personal sensitivities
73
■ ␣ EVALUATING␣ JOBS␣
A N D
␣ A
V O I D I N G
␣ G
E N D E R
␣ B
I A S
Ten Steps to Evaluating Jobs
Job evaluation committees need sound information in order to do their jobs. Accordingly, it is
essential to ensure that all jobs have been screened and checked to capture appropriate job
information and minimize wrong answers and missing information.
Each committee member should receive a copy of the factor definitions and notes for
evaluators (see Makings of a System) as well as some job information packages that might
include a job description and questionnaires, for example.
The following steps outline the tasks of the committees:
1. Read the factor definition and notes for evaluators
Committee members should familiarize themselves with the factor definition, scale degrees and
the notes for evaluators. It is important to understand the focus of the factor, the variables and
the standard scale. The committee should discuss what each factor means.
2. Discuss work requirements for the top and bottom of the scale
The committee may want to discuss the type of work requirements that would fit at the top
and bottom of the scale. This refers to the work requirements — not necessarily to job titles.
For example, “continuously dealing with extremely loud noises” would rank at the top and
“occasionally dealing with dust” would rank at the bottom of the disagreeable conditions
factor.
3. Each committee member reads the job information for the particular factor
Evaluations should be done on random groups of jobs of a reasonable size. A group of 20 to
30 should be selected and distributed around the table (3 or 4 per person). Each committee
member should review the information provided to him or her, and identify the job
requirements relevant to the factor under consideration.
4. Each committee member ranks each position independently
Committee members must then independently record a rank for each of their assigned jobs
and then pass the bundle on to the next member so that each member can rate all jobs.
74
C
H
A
P
T
E
R
␣
F
O
U
R
■
5. Record independent ranks together.
After each committee member has evaluated all of the jobs in the first round, the group needs
to discuss the ranking and reach a consensus.
One way to do this is to write the titles or codes for the group of positions on a flip chart or
blackboard. All members record their rank for each job. Once this has been done, patterns can
be seen; it also means that all committee members will have documented their results before
any discussion occurs. This will help avoid scenarios where the first or loudest committee
member always sets the tone of the discussion that follows. Committee members should be
encouraged to discuss their rankings even if a member was alone in ranking high or low. This
is an important chance for the committee to probe and to challenge assumptions.
6. If consensus is not possible, put the job aside temporarily
If the committee cannot reach consensus on a particular job, that job should be put aside until
later. This is often means that a relative comparison is needed rather than just the factor scale.
7. Repeat steps 3 to 6
Steps 3 to 6 must be repeated until all of the jobs have been evaluated on the factor.
8. Sore thumb ratings periodically
Sore thumbing can and should be done periodically. Remember that ratings are relative, and
those done at the beginning may need to be changed as the relativity scale gets formalized. For
example, a Level 4 rating may have appeared to be reasonable for a job until it becomes clear
that other jobs subsequently given that rating are indeed more demanding than the original
job.
9. Record rationales
Committee members should record notes or rationales for ratings throughout the process of
determining results. This is particularly important where extended discussions were necessary.
If the committee had any difficulty reaching a decision, others, such as managers or
employees, are also likely to have questions.
75
■ ␣ EVALUATING␣ JOBS␣
A N D
␣ A
V O I D I N G
␣ G
E N D E R
␣ B
I A S
10. Final sore thumbing for the factor
After all the jobs have been evaluated, a final sore thumbing exercise should be performed to
check relationships. Any that are questionable should be investigated. If, for example, a
committee member suggests that a particular job’s rating should be at Level 2 and not Level 3
because similar jobs are at Level 2, the committee should revisit the job information to decide
if the rank (Level 3) still makes sense. Even if the rank is not changed, the committee should
keep notes of the discussion, as the result may again be challenged later.
Some Practical Questions to Minimize Gender Bias
❍
❍
❍
❍
❍
❍
❍
❍
❍
❍
❍
❍
❍
❍
❍
❍
Was the committee balanced and representative in terms of sex, occupational groups and
levels of the organization?
Was the committee given gender bias sensitization training and told to be open to change
and resistant to gender bias?
Was there any pay equity and gender bias expertise sought to ensure fair results?
What did the committee do with titles?
How did the committee deal with wrong or ambiguous information?
How was consistency built into the process?
Did the committee rate factor-by-factor or job-by-job? (job-by-job may lead to inconsistent
application of factors.)
Do the evaluation ranks fairly represent the level of job requirements indicated in the job
information provided?
Was any supplementary information used in the evaluation process? Were employees
involved? Did they review the information for accuracy and completeness?
Did the committee do sore thumbing for patterns or anomalies that should be questioned?
What did factor patterns look like? Were jobs differentiated well or compressed?
Were rationales written for all jobs, for a sample group, or only for problematic ratings?
Do rationales indicate rating logic (relativity to other jobs, factor element)?
If the questionnaire involved scoring by participants, what was the relationship between
the employees’ answers and the ratings? How were discrepancies addressed?
Was there a pattern to any discrepancies between employee response and ranking; was
there a high incidence; was there a gender trend; was there a factor trend?
How did the committee resolve disagreement between employees’ and supervisors’
responses?
76
C
H
A
P
T
E
R
␣
F
O
U
■
R
Avoiding Gender Bias in Job Evaluation
A
s at other stages, gender neutrality is important when applying the job evaluation
system. Even if the job evaluation system is gender neutral, the process for applying
the system may still create gender bias in the results. A gender-neutral system can be
administered in a biased manner, and gender bias at any stage can affect the end result.
Achieving gender neutrality at a given stage may not eliminate the effects of gender bias
elsewhere, but checks and balances can minimize them.
Gender neutrality in evaluation may minimize the effects of gender bias in collecting job
information. For example, a sensitized job evaluation rating committee may interpret
factors liberally to include job information beyond the formal meaning of the factor, if
elements of work would otherwise be overlooked.
Five suggestions to help meet the challenge of gender bias at the evaluation stage:
1.
2.
3.
4.
5.
Ensure the evaluation tool is applied consistently without regard to the genderpredominance of the job.
If a committee is used to evaluate jobs, make sure its members are representative of
employees, managers, and union (if there is one).
If a committee is used, make sure it is sufficiently knowledgeable and well-trained to
allow all members to meet their obligations.
Strive for a decision-making process that is free of gender bias.
Ensure the committee is well-trained to look for frequently overlooked features of
work and open to the changes this process is expected to bring.
77
■ ␣ EVALUATING␣ JOBS␣
78
A N D
␣ A
V O I D I N G
␣ G
E N D E R
␣ B
I A S